Basel Committee on Banking Supervision

Credit loss provisioning is a key accounting issue, reflecting changes in credit risk exposure and affecting reported profits and regulatory capital. It is also a key area of focus for external auditors and users of financial statements, often posing risks of material misstatement (ROMM).

ECL frameworks present significant ongoing challenges for banks and their external auditors, requiring significant judgement, effort, and collaboration among management, audit committees and external auditors. An effective audit of ECL estimates and related disclosures is an important aspect of high-quality financial reporting and disclosure by banks.

The following terms are used throughout this chapter and have the meaning given below:

1. Auditor’s expert: refers to an individual or organisation with specialised knowledge in an area other than accounting or auditing who has the necessary competence, capability and objectivity and whose work in that area is used by the auditor to assist the auditor in obtaining sufficient appropriate audit evidence.
2. Exposure: refers to those financial instruments that are subject to relevant ECL accounting requirements. This may include loans and credit card receivables, lease receivables and loan commitments and financial guarantees (whether on- or off-balance sheet).
3. Recency bias: arises when estimates are overly influenced by recent experience or events.
4. Significant risk: is an identified risk of material misstatement for which the assessment of inherent risk is close to the upper end of the spectrum of inherent risk due to the degree to which inherent risk factors affect the combination of the likelihood of a misstatement occurring and the magnitude of the potential misstatement should that misstatement occur.

This chapter outlines supervisory expectations for the external audit, to help banks' audit committees oversee banks' external audits, including ECL-related work. It also includes questions that audit committees may ask the external auditor, and elaborates on key components of ECL, such as models and macroeconomic scenarios.

The International Auditing and Assurance Standards Board (IAASB) have issued auditing standards relevant to ECL audits, including:

1. International Standard on Auditing (ISA) 200, Overall objectives of the independent auditor and the conduct of an audit in accordance with international standards on auditing;
2. International Standard on Auditing (ISA) 260 (Revised), Communication with Those Charged with Governance;
3. International Standard on Auditing (ISA) 265, Communicating Deficiencies in Internal Control to Those Charged with Governance and Management;
4. International Standard on Auditing (ISA) 300, Planning an Audit of Financial Statements;
5. International Standard on Auditing (ISA) 315 (Revised 2019), Identifying and Assessing the Risks of Material Misstatement Through Understanding of the Entity and Its Environment;
6. International Standard on Auditing (ISA) 330, The Auditor's response to assessed risks;
7. International Standard on Auditing (ISA) 500, Audit Evidence;
8. International Standard on Auditing (ISA) 540, Auditing Accounting Estimates and Related Disclosures;
9. International Standard on Auditing (ISA) 620, Using the Work of an Auditor's Expert;
10. International Standard on Auditing (ISA) 701, Communicating Key Audit Matters in the Independent Auditor's Report; and
11. International Standard on Quality Management 1, Quality Management for Firms that Perform Audits or Reviews of Financial Statements, or Other Assurance or Related Services Engagement.

The Committee does not have the authority to set professional standards for external auditors; as such, this chapter does not replace these or other applicable standards. Nevertheless, as internationally accepted professional standards for external auditors are principles-based, the proper application of the standards to audits of banks includes appropriate tailoring of audit work in response to the risks and issues applicable to banks.

This chapter is not a comprehensive guide to auditing ECL estimates, nor is it a substitute for external auditors to comply with all applicable elements of relevant auditing standards. It highlights particular areas of focus to enhance the quality of those audits. Expectations for the external auditor set out in this chapter, and related questions, are aimed at helping the audit committee in overseeing the audit of ECL.

This chapter applies to banks applying International Financial Reporting Standard (IFRS) 9 and similar ECL frameworks, without replicating specific requirements of these frameworks. Methodologies described here, such as "macroeconomic scenarios and weighting", represent possible approaches, but should not be read as implying a single acceptable way of complying with an accounting standard. Similarly, not all banks develop models for all credit exposures and not all ECL frameworks have requirements in respect of a significant increase in credit risk (SICR).

The ECL estimate requires special consideration from the external auditor. Compared with many other items in the financial statements, the ECL estimate is more susceptible to material misstatement. This is because:

1. the estimation process can be complex and involve multiple data, interrelationships and assumptions. Methods and models are often bespoke to individual banks. In addition, the ECL estimate usually requires input from non-accounting experts; and
2. ECL involves many management judgments, some of which can be more difficult to objectively evaluate (for example because they involve estimates of future economic conditions) or may be subject to management bias.

For audit committees to fulfil their oversight function in relation to the financial statement audit, they require a good understanding of how ECL accounting can impact the financial statements and how external auditors obtain reasonable assurance that financial statements are free from material misstatements. A sound governance structure enabling board oversight of management's ECL estimation processes and outputs is a key area of focus for the audit committee. Governance includes established processes for making and challenging the ultimate decisions about economic inputs, macroeconomic scenarios and weightings, model outputs, use of model adjustments and other judgments involved in the determination of the ECL estimate. Internal controls over ECL are also important to the audit committee (see CGO10).

When auditing the financial statements of a bank, the external auditor considers and responds to the risks of material misstatement relating to ECL estimates in the audit plan, risk assessment, and execution of the audit.

The external auditor's procedures in response to the assessed ROMM include one or more of the following approaches: (i) obtaining audit evidence from events occurring up to the date of the auditor's report; (ii) testing how management made the accounting estimate; and (iii) developing an auditor's point estimate or range. This chapter is generally written based on an assumption that the external auditor tests how management developed the ECL estimate. The relevance of each question will vary based on the method/approach the external auditor uses to audit particular aspects of ECL.

The questions below are phrased in the past tense. However, it may be appropriate for the audit committee to ask questions before, during and/or after audit work has been carried out. In many cases, it may be appropriate to first ask the questions early in the audit so that the audit committee can consider whether the auditor's assessments and planned procedures are adequate. Some questions or elements of questions may not be relevant depending on the bank's particular circumstances, which may change over time. The audit committee may also wish to focus their questions on those areas where the auditor has exercised the most significant judgments. In addition, the audit committee may find it helpful to ask the external auditor to include auditor's experts in their meetings with the audit committee.

Professional scepticism and management bias

For banks, the ECL estimate is likely to be subject to significant estimation uncertainty, complexity and subjectivity, and thus a higher ROMM due to fraud or error, including a greater risk of management bias. If a bank's external auditor were to fail to exercise professional scepticism when auditing the ECL estimate, a material misstatement in the financial statements could go undetected.

Expectation 1: The external auditor should promote an environment conducive to exercising professional scepticism in the audit of ECL by planning, staffing and managing the audit engagement appropriately.

Question 1: Can the external auditor explain how they concluded that appropriate audit planning and resources were in place for professional scepticism to be exercised?

The external auditor should, at a minimum:

1. ensure the engagement team as a whole has sufficient competence and knowledge of the ECL framework and makes appropriate use of the work of auditor's experts where necessary;
2. ensure the engagement team remains independent from the audit client and has sufficient experience to be able to exercise professional scepticism effectively;
3. plan and implement an effective audit approach;
4. allocate sufficient time and appropriate resources to reflect on and perform effective challenge management assumptions, judgements and representations; and
5. ensure the audit firm has adequate internal quality management processes in place to foster, encourage and support the exercise of professional scepticism by the engagement team throughout the course of the audit.

Expectation 2: The external auditor should exercise, and provide evidence that they exercised, a high degree of professional scepticism at all stages of the audit of ECL.

Question 2: Can the external auditor explain:

1. Question 2(a): how they demonstrated professional scepticism and effective challenge of management's methods, significant assumptions and data used in the ECL estimate during the course of the audit? For example:
   1. How was the external auditor satisfied that indicators of management bias were not directionally the same (eg mainly optimistic)?
   2. How was the external auditor satisfied that management's use of forward-looking information (FLI) reflected a proper consideration of expectations of future developments and was not subject to recency bias or other management biases, and that the length of the forecast period is appropriate and the forecast supportable?
   3. How was the external auditor satisfied that management's decisions on the range of scenarios and scenario weightings capture the appropriate extent of expected credit losses required by the relevant ECL framework, including the effect of possible economic conditions (including low probability but high impact conditions), and the effect of non-linearity in how credit losses may respond to changes in economic conditions?
   4. How did the external auditor challenge management's use of model adjustments (in particular, how did the external auditor (i) obtain an understanding of and evaluate management's rationale for the adjustments; and (ii) evaluate whether the adjustments were complete, supported by sufficiently persuasive evidence, and were subject to robust scrutiny and challenge)? Were model adjustments necessary to deal with known data limitations and model weaknesses arising from changes in the economic environment or, more generally, to compensate for potentially underperforming models?
   5. How did the external auditor challenge whether management's SICR threshold facilitates a timely transfer of exposures to stage 2?
   6. How was the external auditor satisfied that management took appropriate steps to develop disclosures about estimation uncertainty that are necessary for fair presentation of the financial statements as a whole?
2. Question 2(b): their assessment of where the bank's ECL estimate, overall and by major component, falls within a reasonable range? Can the external auditor explain how they made that assessment and how the reasonable range was determined (and, where it was produced by management, how did the external auditor assess that management's range was sufficiently relevant and reliable to be used as audit evidence)?
3. Question 2(c): how the "stand back" test was performed and how the external auditor evaluated the reasonableness of the ECL estimate and related disclosures, in light of their original risk assessment, the assessment of controls and the evidence collected? What contradictory evidence was considered and how was it addressed?

For complex accounting estimates, the external auditor's application of professional scepticism is particularly important given the high ROMM associated with ECL. The external auditor should, at a minimum:

1. gather higher quality, more persuasive audit evidence, including evidence from multiple sources, where available, both inside and outside the bank;
2. obtain evidence in a way that is not biased towards obtaining evidence supporting management's positions or excluding evidence that may be contradictory to management's positions;
3. make appropriate use of the knowledge of auditor's experts in assessing and challenging management's assumptions and conclusions;
4. actively search for indicators of management bias (whether incentives or opportunities);
5. evaluate the implications for the audit, and report those indicators to the audit committee (eg estimates consistently at one end of range, whether overly optimistic or overly pessimistic; weak rationale for assumptions or conclusions leading to favourable outputs; responding inappropriately to incentives; or management override of controls);
6. assess whether management's significant assumptions are consistent with other relevant internal information (eg with those of the bank's credit risk management function or with forecasts used in other areas of the bank);
7. review or perform peer comparisons, where there are relevant peer banks or portfolios, to determine if the relative positioning of a bank's ECL estimate (or material components of the estimate) when compared with peers is out of line with the expectations of the external auditor, and assess the implications for the audit - eg a significant difference may identify matters for further investigation);
8. test management's sensitivity analysis or performs their own sensitivity analysis, as applicable, to identify the risk of material misstatement, including the risk from management bias in key variables in FLI selection, scenario design or weighting, or assumptions around the detailed forecast period;
9. verify that the ECL estimate for particular portfolios is sufficiently reactive to changes to key variables and that the effect of potential downside macroeconomic scenarios have been taken into consideration in the ECL estimation process;
10. perform a stand back test before forming an audit opinion to determine whether the assessed ROMMs and the external auditor's response to those remain appropriate. This includes considering whether indicators of bias have been identified, whether management's ECL estimate is in accordance with the relevant accounting standard and whether sufficient audit evidence of appropriate quality has been obtained (eg considering the amount of evidence that corroborates and contradicts management's assertions). The audit of financial statements is a cumulative and iterative process, so the external auditor may need to stand back at various times during the audit (eg obtaining high-quality information that significantly contradicts an initial expectation of the external auditor may be a trigger to stand back and reassess the ROMM; or the results of an audit procedure may indicate that additional work is required or may indicate an inconsistency with other audit work that needs to be followed up);
11. in undertaking the stand back tests: evaluate the evidence gathered against the external auditor's knowledge and expectations; evaluate the combined effect of findings from controls and substantive testing; use senior members on the engagement team who have greater experience and oversight of the overall business and position of the bank; make use of auditor's experts to ensure sufficient technical expertise informs the evaluation; and
12. document their work, including:
    1. indicators of possible management bias related to accounting estimates, if any, and the external auditor's evaluation of the implications for the audit of the ECL estimate and resulting communication to those charged with governance;
    2. the conclusions reached and how those conclusions were supported (ie the rationale);
    3. how professional scepticism was exercised, including how the increased importance of exercising professional scepticism during the audit of ECL was reflected in the work effort; and
    4. the assessment of the adequacy of the auditor's expert's work and the external auditor's conclusions on that work.

For banks, ECL is a complex accounting estimate that has high inherent risk, which means it is highly susceptible to material misstatement.1 ROMM is a key concept in external audit as it determines the extent of audit evidence required and influences the audit approach that will allow the external auditor to express an opinion on a bank's financial statements. It is important that the external auditor correctly identify significant ROMMs given the requirement to test the design and implementation of the corresponding relevant controls.

Expectation 3: When identifying and assessing the ROMM relating to the audit of ECL, the external auditor should give particular focus to the following:

1. understanding banks' lending business models (eg types of credit risk, underwriting criteria and any changes, lending periods and terms of collateral), risk appetite and exposures, credit risk management practices, business segments and geographical locations;
2. understanding banks' oversight and governance, competence of staff, information systems, processes and controls (given that ECL estimates will often involve multiple data sources, modelling, assumptions and judgments); and
3. evaluating the risks in the broader context of the financial statements as a whole to make an overall assessment of the ROMM arising from the ECL financial statement line item, in addition to considering more granular ROMM. The evaluation should include consideration of the regulatory factors, if any, that are relevant to accounting estimates of the bank.

Expectation 4: The external auditor should determine that risk(s) of material misstatement relating to ECL accounting are significant risk(s).2 In many circumstances, the following will be among those components of ECL from which significant risks of material misstatement may arise: forecasts and forward-looking information, macroeconomic scenarios and weighting, models, significant increase in credit risk and disclosure.

Question 4: Can the external auditor explain:

1. Question 4(a): Their assessment of the significant risks of material misstatement relating to ECL, taking into account the potential likelihood, magnitude and the sources of the risk? If the external auditor has concluded that there is no significant risk of material misstatement relating to ECL, can they explain their rationale?
2. Question 4(b): How they assessed the risks of material misstatement in the following key areas: forecasts and forward-looking information, macroeconomic scenarios and weighting, models, significant increase in credit risk and disclosure?
3. Question 4(c): How their assessment of the significant inherent risk factors (ie the sources of risk) influenced the focus of their further audit procedures?

Credit loss provisioning is generally material for a bank's financial statements and to the calculation of capital and key performance metrics. Whether or not the credit loss provision is material in any given reporting period, the way a bank measures and discloses its exposure to credit risk for accounting purposes is important to users of its financial statements.

The external auditor identifies and assesses ROMM by taking account of estimation uncertainty and the degree to which the method, assumptions, data, and determination by management of the ECL estimate and related disclosures are affected by subjectivity, complexity or other inherent risk factors. ECL accounting is generally subject to a high degree of estimation uncertainty, in part because of its forward-looking nature. ECL accounting is also subjective in that it involves management assumptions and judgments, and complex in that it has significant data requirements and often requires modelling. ECL accounting is, therefore, highly susceptible to material misstatement, whether due to fraud or error, and the related estimates are likely to be at the high end of the inherent risk spectrum and, therefore, to significant risks.

The external auditor looks for the reasons why there may be ROMM and will usually assess ROMM for individual portfolios and/or elements of those portfolios, and of the ECL estimate, including key methods, assumptions and data as well as the components discussed in this chapter. This will generally result in one or more significant ROMM related to the ECL estimate and related disclosures being identified.

The external auditor should communicate the significant risks identified to those charged with governance (eg the audit committee). As there are likely to be significant risk(s) related to the ECL estimate communicated to those charged with governance, the external auditor should also determine whether ECL accounting gives rise to key audit matter(s) to communicate in the auditor's report (where required).

The different components of ECL may be subject to different types of inherent risk factors. The external auditor should consider, at a minimum, the key components of ECL to determine the sources and levels of the inherent risks of material misstatement. This consideration is required so that the external auditor can fully assess the ROMM of the ECL estimate and evaluate the design of controls and determine whether they have been implemented where ROMM is significant, and can determine how best to address those risks.

The ECL estimate should be measured in a way that reflects an unbiased and probability-weighted estimate of credit losses. It will reflect inputs that typically include, among others, assumptions about future values of economic variables and potential recovery strategies. There is a high degree of estimation uncertainty and complexity given that the forward-looking inputs into the ECL estimate are interrelated and highly subjective. The nature of FLI is such that it is not precisely knowable before it occurs - only estimable. Nevertheless, the FLI-related estimates and related disclosures should use reasonable and supportable information about past, current and future economic conditions, that is available without undue cost or effort. Forecasts and FLI that are not reasonable nor supportable can lead to a material misstatement in the ECL estimate.

Sources of inherent risk of FLI include:

1. the FLI becomes more uncertain and subjective the longer the period over which a bank needs to incorporate FLI into its ECL estimate.
2. the subjectivity involved in management's selection or development of FLI included in macroeconomic scenarios, where those are developed. Management will need to select FLI that is relevant to the exposures for which the ECL estimation applies. For example, different portfolios may vary in sensitivity to one or more of the following, as well as other factors: unemployment rates; inflation; gross domestic product; house price indices; stock prices; commodity prices; or interest rates;
3. the subjectivity involved in considering what is reasonable and supportable information for the purpose of forecasting future economic conditions over the relevant life of the financial instrument:3
   1. Banks make judgments about what is relevant information for different periods of an instrument's relevant life. As a result, different techniques and levels of detail may be adopted in relation to the different periods. For example, banks may undertake a detailed forecast for a period followed by a transition to a long-term average. In such a case, some subjective management judgments include:
      1. the length of the detailed forecast period where, if it is too short, the ECL estimate may not sufficiently capture all relevant FLI and as a result may potentially be misstated; by contrast, if the detailed forecast period is too long, it may not be reasonable and supportable;
      2. the duration of any short-term macroeconomic trend (upward or downward) that forms part of the detailed forecast period, where there is the risk that not all relevant information was considered in the estimated duration;
      3. the length of the transition period between the detailed forecast period and the long-term average, such that there may be a risk of bias where the transition period is either unreasonably long or short; and
      4. the technique of transitioning to a long-term average, where the risk is that it may not be reasonable and supportable.
   2. Some products have terms that require significant judgments about relevant periods over which ECL is measured - for example where financial instruments may be prepaid, or where there is no fixed contractual term or the ECL framework does not permit the use of contractual maturities - as may be the case for certain revolving facilities;
4. the subjectivity of FLI elements. For example, in calculating ECL, management may need to estimate the effectiveness of future recovery strategies (and their impact on loss estimates) under different circumstances, when there may be a lack of predictive history;
5. the complexity of the relationships between changes in economic indicators and the resulting changes in credit risk for the segments of the bank's portfolio;
6. the subjectivity in incorporating identified non-linear loss patterns into the calculation of the ECL estimate; and
7. the likely use by management of specialised skills and knowledge, indicating that development of FLI and incorporation of FLI into the ECL estimate can be complex.

The ECL estimate reflects an unbiased and probability-weighted amount that is determined by evaluating a range of outcomes and may include using macroeconomic scenarios designed and weighted to incorporate assumptions about future macroeconomic conditions. If management does not properly calibrate the macroeconomic scenario design and weightings, it can lead to a failure to capture the impact of different conditions on credit risk and thereby create the potential for material misstatement of the ECL estimate.

1. Given that the macroeconomic scenarios and weighting are complex, forward-looking and require inclusion of multiple economic factors, there is a high degree of estimation uncertainty, subjectivity and complexity associated with their design.
2. In addition, given that the macroeconomic scenarios and weighting should represent a range of possible outcomes to allow an unbiased ECL calculation, how those scenarios represent possible outcomes, individually and together, and how probabilities have been assigned to them, is highly subjective.

Sources of inherent risk include:

1. macroeconomic scenarios and weighting, which may be complex in an attempt to reflect the real-life economic phenomena they intend to predict. There is also a greater risk that the data inputs into the scenarios are not complete, accurate or internally consistent where there is complexity;
2. the subjectivity involved in the macroeconomic scenario design, for example:
   1. The macroeconomic scenarios and weighting, as complex as they are, are inherently limited in that they cannot capture all the economic variables and their interrelationships and sensitivities to one another as reflected in the real-life economy in which the bank operates.
   2. There is a risk that upon deciding which macroeconomic variables to include in the scenarios, management may have failed to capture some drivers of credit risk and/or credit loss, whether indicators of credit risk or mitigating factors (such as payment relief measures from banks or government support measures) and their interrelationships with other variables. This risk also applies to cases where a bank uses external scenario information; and
3. the subjectivity involved in ensuring the appropriate range of macroeconomic scenarios (ie the number of scenarios, the variance in severity between scenarios), and in determining the probabilities of those scenarios occurring:
   1. Selecting too narrow a range of macroeconomic scenarios leads to the risk that they will fail to capture the full extent of the risk of losses in an economic downturn, in particular because of the potential for credit losses to respond non-linearly to changes in economic circumstances and the potential for the non-linear effect to be amplified as scenarios become more severe.
   2. Assigning inappropriate probabilities to the range of macroeconomic scenarios can result in an unreasonable range and an inappropriately calculated ECL estimate.
   3. Selecting inappropriate periods for the duration of different phases of the scenario design can lead to an inappropriate ECL estimate.
   4. When the short-term forecast represents a temporary economic downturn, consideration may not be given to long-term economic averages.

Management often estimates ECL using models (that involve data and assumptions and the relationships between them). ECL modelling is often complex, as models usually rely on multiple data sources. ECL modelling is also subjective as it involves making numerous interrelated assumptions. Although model adjustments will often be required to compensate for data and or other model limitations, there is subjectivity in determining when model adjustments are appropriate and how they should be determined. A risk of material misstatement of the ECL estimate can arise both from bespoke and purchased models - they could be ineffectively designed and controlled, or not fit for the specific purpose.

Sources of inherent risk from the use of models include:

1. the complexity of modelling:
   1. the overall ECL estimate will typically comprise individual estimates for different exposures, products or portfolios (eg retail mortgages, unsecured personal loans, credit cards). These can have different risk characteristics and can react differently depending upon the economic circumstances. A bank may need different models to reflect the different risks of its exposures.
   2. the ECL estimate may also involve separate models for its different components (eg economic forecasting models, probability of default (PD), and exposure at the point of default (EAD), loss-given-default (LGD) on that exposure). These models will need to be coherent and consistent with regard to common data inputs and assumptions.
   3. a model may involve numerous and varied types of data and assumptions, and the relationships between data and assumptions may be complex.
   4. the computational methods in the models can involve complex (eg statistical) modelling techniques. Banks will often use mathematical and statistical expertise to develop and operate their models.
   5. ECL models may be built upon other models - notably those used to calculate credit risk weights or conduct stress tests for regulatory purposes - but those other models may have some parameters and objectives that are different from those required by an ECL framework. For example, some regulatory models use a through-the-cycle estimate of credit losses rather than a point in time estimate of credit losses. There is a risk that management may fail to make the proper adjustments to ensure that ECL models produce estimates that comply with relevant ECL frameworks; and
2. the subjectivity of ECL modelling:
   1. It is not possible to precisely model the future state of the world. In ECL modelling, management necessarily make assumptions, some of which are simplifications of real-life phenomena. These assumptions make the models inherently subjective.
   2. A modelled output may include in-model adjustments or be subject to post-model adjustments (the latter commonly referred to as "core model overlays"); together, "model adjustments". Management may need to make these model adjustments to compensate for limitations in models, whether due to data or otherwise. The model adjustments are often subjective - management uses experienced credit judgment to determine the necessity for, and amount of, model adjustments. The various models and model adjustments may also each be subject to different levels of governance and control. These factors increase the risk of bias or error. The risks may change over time as management incorporates adjustments into the models to reduce model limitations.
   3. It is possible that data or other model limitations that previously were not apparent become relevant in particular circumstances, such as an unusual economic downturn, necessitating model adjustments or a requirement for proxy data sources. For example, the modelled relationship between economic variables may not hold in the circumstances, or there is a lack of relevant historical data for the new relationships. Identifying when circumstances have sufficiently changed to render a model or a parameter no longer functioning properly, and the appropriate response, may be challenging due to the complexity of relationships between variables and uncertainty about whether the change in relationships is temporary or permanent.
   4. The data supporting a model can increase the complexity and subjectivity of modelling.
   5. Management will have made judgments about the data that best fit the risk characteristics of its current portfolios.
   6. Management may have had to source data from multiple sources, including external sources and from internal sources that are not part of the traditional accounting systems (eg PD data).
   7. Management will have had to determine the historical period over which they should collect the data.
   8. Data may not be available, and consequently assumptions or estimates may be required (eg for portfolios that have experienced few historical losses but for which a bank does not expect continued low losses under all circumstances, a bank may make an assumption that loss data for a different portfolio can be used, as the portfolios have sufficiently similar characteristics). The volume of data may require multiple systems and controls to ensure effective data management and maintain data integrity, such as assessing data quality, relevance, reliability and completeness.
   9. Data integrity may be compromised for certain specific portfolios, historical periods or within certain systems and proxies or alternative sources may have been used.
   10. Due to recent or new implementation, data may be need to be converted, housed within or subject to new applications, interfaces or control activities.

The process of estimating ECL requires a bank to assess whether a significant increase in credit risk has occurred for an exposure or group of exposures. The SICR threshold is important because it determines whether an exposure is in stage 1 (where ECL comprises a 12-month expected credit loss) or stage 2 (where ECL comprises the lifetime expected credit loss). If management's criteria for measuring credit risk are insufficiently comprehensive or forward-looking in nature, or if the threshold that determines when a significant increase in credit risk occurs is inadequately sensitive to changes in risk, there is the risk that the ECL estimate will not accurately reflect the extent to which credit risk has changed in accordance with the accounting framework, resulting in a material misstatement of ECL.

The SICR assessment is highly subjective as ECL frameworks do not prescribe what a SICR is. The SICR assessment should incorporate all reasonable and supportable information (including forward-looking information) that is available without undue cost or effort. It is expected that the SICR assessment will consider both qualitative and quantitative indicators. Further, management should exercise judgment in weighting the information that feeds into its SICR assessment. The SICR threshold may be multifactor and complex.

Sources of inherent risk from SICR include:

1. the subjectivity involved in determining the threshold at which a SICR has occurred. There is a balance between recognising SICR too late (eg setting credit risk indicators that do not capture the relevant drivers of credit risk for exposures and thresholds that are too high) and recognising SICR too early (eg setting credit risk indicators that are too sensitive to short-term effects that do not indicate changes in lifetime credit risk and thresholds that are too low). There is a risk, for example, that overreliance on lagging credit risk indicators means that the SICR threshold may be insufficiently forward looking and transfers to stage 2 will occur too late;
2. the subjectivity and complexity involved in measuring credit risk, particularly:
   1. in cases where a bank believes it is reasonable to use a different PD (eg a 12-month PD) as an approximation of a lifetime PD to assess SICR. There is a greater risk of material misstatement in cases where the most significant cash flows of an exposure with a maturity beyond 12 months arise at or near maturity, such as with bullet exposures;
   2. for banks that use a PD model, estimating PDs at the point of origination as well as at the reporting date of whether there has been a SICR;
   3. when capturing or estimating the remaining relevant life of exposures;
   4. in the judgement around any rebuttable backstops that may be set out in the relevant accounting framework;
   5. in how management select and combine quantitative and qualitative credit risk indicators, and the way in which management incorporate risk factors into PD models:
      1. There is a risk that the measurement of credit risk is incomplete, that increases in credit risk go unidentified and that ECL is understated.
      2. There is a risk of failure to incorporate significant risk factors into PD models over the period covered by the SICR test.
      3. The SICR criteria may be time or circumstance-dependent. For example, missed payments are often considered an indicator of increased credit risk, but they may be due to events unrelated to borrower repayment performance, such as a failure in payment systems. Similarly, loan payment deferrals offered to all customers irrespective of individual credit worthiness may not indicate increased credit risk for each borrower.
3. the subjectivity inherent in incorporating forward-looking information into the SICR criteria;
4. the complexity of incorporating portfolio-level and macroeconomic indicators into the SICR credit risk indicators alongside PDs, based on historical experience, and the complexity of combining the SICR credit risk indicators;
5. the complexity in developing lifetime PDs, which may result in some banks using simplifications. For example, for new products or for lending to customers with no history of credit losses there may be limited internal and external information on which to base PDs. There is a risk that such simplifications do not fully capture changes in credit risk and that losses are recognised late; and
6. the need to ensure that policies for SICR credit risk indicators and thresholds are applied consistently across a global banking group and also adequately reflect local conditions and risk factors for subsidiaries, portfolios and borrowers. There are risks of incompleteness, inconsistency and/or bias in the ECL calculation.

The ECL estimate is subject to significant estimation uncertainty, and involves a number of highly subjective judgments and complex calculations, as explained above. Management's disclosure process involves determining the information it presents to users regarding its ECL estimate, including exposure to and drivers of credit risk, the allocation of exposures between the different ECL stages, how it manages credit risk and the sensitivities to assumptions, judgments and uncertainty associated with the ECL estimate. The risk is that the disclosures associated with ECL are inaccurate, incomplete, non-transparent or otherwise not reasonable, such that there is a material misstatement.

Key areas of inherent risk related to ECL disclosure include:

1. the subjectivity involved in deciding what information and the level of detail to disclose, ie what is material to the users of the financial statements for the purpose of achieving the fair presentation of the financial statements;
2. the subjectivity involved in making the decision as to how best to present the disclosures judged to be material, and therefore decision-useful, to users:
   1. Given many banks' wide-ranging operations and numbers of product lines, there is a risk that ECL disclosures are not sufficiently granular to show users useful and specific information; and
   2. There is a risk that disclosures are so extensive that they obscure material items from users of the financial statements; and
3. the complexity involved in producing ECL disclosures, particularly for internationally active banks, as their ECL provisions often require collecting and assessing data from multiple sources, increasing the risk of incomplete and/or inaccurate disclosure or the risk the management is dis-incentivised from developing reasonable disclosures in a timely way.

ECL is a complex accounting estimate that often involves interrelated processes across multiple areas of a bank and significant data and IT requirements, which means effective internal controls are needed to ensure the completeness, accuracy and reasonableness of the related information, from data collection and throughout complex processing. In response, a bank's external auditor performs appropriate tests on the controls over the ECL estimate to assess whether, and to what extent, the external auditor can rely on them.

Given the complexity, subjectivity, and high degree of estimation uncertainty around the estimate of ECL, and the fact that the ECL estimate can often involve models and multiple data points, substantive procedures alone will usually not provide sufficient appropriate audit evidence that the ECL estimate is not materially misstated. The external auditor should obtain an understanding of and evaluate the design and implementation of the bank's system of internal controls. For controls the external auditor plans rely upon, the external auditor should test the operating effectiveness and determine the extent to which the controls may be relied upon given the result of that testing.

Expectation 5: Substantive procedures alone usually cannot provide sufficient appropriate audit evidence that the ECL estimate is free of material misstatement and the external auditor will want to place some level of reliance on internal controls over the ECL estimate. Therefore, the external auditor should test the operating effectiveness (as well as evaluates the design and implementation) of internal controls on which the external auditor plans to rely.

Question 5(a): In what areas and to what degree did the external auditor rely on internal controls (and how did that impact the level of substantive procedures performed) for the ECL estimate?

Question 5(b): What were the consequences for the audit of ECL where the external auditor identified deficiencies in internal controls? Did any of these deficiencies affect the external auditor's audit plan, the level of substantive testing, opinion or disclosure about those controls in the external auditor's report (regardless of whether an expanded auditor's report is required)?

The external auditor should, at a minimum:

1. evaluate the design of controls, determine that controls have been implemented, and test the operating effectiveness of controls over the ECL estimate, using suitable and available criteria - including controls related to governance and oversight, information systems and the key components of ECL;
2. evaluate the control environments of those locations and processes where internal controls are applied and tested;
3. evaluate the elements of internal control over the ECL estimate and determine whether they are:
   1. present and functioning in design, implementation and operation; and
   2. operating together in an integrated manner to achieve the entity's financial reporting objectives;
4. identify whether deficiencies (in both design and operating effectiveness) exist in internal controls over the ECL estimate and related disclosures. Inadequately designed internal controls will decrease the relevance of testing whether those internal controls are implemented or operating effectively;
5. consider the impact of such deficiencies for the audit approach, including the impact on risk identification, identification and testing of potential compensating controls, and changes to the nature and extent of substantive testing; and
6. communicate all significant deficiencies in internal controls over the ECL estimate and related disclosures to management and those charged with governance (eg the audit committee).

Expectation 6: The external auditor should evaluate internal controls over the ECL estimate sufficiently early in the audit process to ensure that an effective audit approach can be undertaken, including cases where the controls tested are ineffective.

Question 6(a): When during the audit process were internal controls assessed by the external auditor? Did the external auditor perform updated testing, if applicable, to verify that internal controls were effective for the duration of the period of reliance on the internal controls? If evaluated late in the audit process, did the external auditor have sufficient time to adjust the audit plan where necessary?

Question 6(b): Did the external auditor evaluate any previously ineffective, new or enhanced controls implemented by management? If these controls were, or remained, ineffective, what did the external auditor communicate to management and did this result in any change to the audit approach? What audit areas did this involve?

Evaluating the design and testing the operating effectiveness of internal controls early in the audit process allows the external auditor sufficient time to revisit their risk assessment, adjust the planned audit approach to ensure it is appropriately responsive to the risks, and design and perform adjusted audit procedures (eg further substantive procedures and/or testing compensating controls) where feasible. Early identification of deficiencies in internal controls also allows time for management to remediate those deficiencies in order for the external auditor to be able to retest and have the ability to rely on those controls for the period where the controls were operating effectively.

In addition, management's processes and controls evolve. As a result, it follows that where the external auditor wishes to rely on those controls, they will need to perform testing over design, implementation and operation of the controls that have been newly implemented or amended during the year being audited.

Due to the nature and complexity of the ECL estimate, the assessment of the design and operating effectiveness of internal controls over the key components of ECL is an important piece of audit evidence, particularly given that the external auditor will want to rely on those controls as substantive procedures alone will usually not provide sufficient appropriate audit evidence when ECL is audited at internationally active banks. The external auditor should also consider the timing of the testing of the controls they plan to rely on because that may impact the other audit work needed. For the key components, the following are important considerations:

1. well-documented oversight and governance over management's financial reporting process, including disclosure. A strong governance process is needed to ensure the judgmental decisions are well controlled such that the disclosures are complete, accurate, in accordance with the accounting framework, relevant to the bank's portfolios and free from bias;
2. controls over the identification of the need for and use of specialised skills or knowledge over the accounting estimate, including the use of management's experts;
3. controls over the selection and validation of relevant methods and assumptions to determine the ECL estimate. This would include, for example, controls over the selection of forecasts and FLI, the selection and weighting of macroeconomic scenarios, and model adjustments and assumptions. Controls over SICR criteria may consider whether those criteria are suitably responsive and forward-looking, for example using key performance indicators that are regularly monitored, reviewing the process for updating PDs and ensuring that they take account of forward-looking information;
4. the design, validation, implementation and maintenance of models used to estimate ECL. Controls should cover data used in models as well as the accuracy and completeness of the model's application;
5. the entity's information and IT systems. The ECL estimate often involves considerable use of data, including data that are sourced and managed from multiple locations and across multiple systems. Strong IT and data management controls relevant to financial reporting are important to ensure the integrity, accuracy and completeness of data used in the ECL models;
6. manual processes. Some aspects of the ECL estimate, or components of estimates, are made outside models, for example for large corporate exposures. Controls should ensure manual (and automated) processes adhere to the bank's policies and procedures, support reliable outputs and are subject to appropriate independent review and governance;
7. strong governance and management review controls related to model adjustments (in particular in periods where model adjustments are more significant). Model adjustments can be highly subjective (eg due to data limitations). The external auditor will want to assess the adequacy of the bank's governance arrangements to ensure any model adjustments are appropriately justified, are supported by available unbiased evidence and have been subject to robust scrutiny and challenge;
8. governance and controls that consider the final ECL results and/or backtesting and similar reviews. The external auditor will be concerned with the processes a bank has for review and challenge of the ECL estimate and its reliability; and
9. controls over external third party and outsourced services such as external data and model design, to ensure the relevance, integrity and accuracy of any relevant data or models.

The external auditor of a bank should have specialised knowledge and competence in auditing banks. The external auditor will require specialised knowledge and experience in accounting for the ECL estimate, but may also need to use other experts. The identification of the need for auditor's experts requires auditor judgment. Auditor's experts have expertise outside accounting and audit. An auditor's expert can be an external expert or from the audit firm.

Management may also use their own experts for areas outside of accounting to assist the bank in preparing the financial statements. In this chapter, such individuals or organisations are referred to as "management's experts".

The estimation of ECL is an area where a bank is likely to need highly specialised skills, knowledge and experience. This need arises from, individually or collectively: (i) the characteristics of the exposure being accounted for; (ii) the assumptions and modelling techniques that are used to calculate the estimate; and (iii) the IT systems and processes required to calculate and manage the associated data and information flows. The external auditor will also need to possess, or otherwise have access to, the necessary expertise and experience in relation to these areas to be able to audit effectively the estimate of ECL. The necessary expertise will also be essential for the auditor to exercise professional scepticism effectively throughout the audit. That expertise may be provided by an auditor's expert.

Expectation 7: The external auditor should duly require the use of auditor's experts to audit ECL effectively, where needed. Auditor's experts will likely be used for some or all of the following matters in relation to the key components of ECL: credit risk measurement and management, estimating loan recoveries and valuations, macroeconomic forecasts and econometric modelling, credit risk modelling and data and IT systems design.

Question 7: Which auditor's experts did the external auditor use? How did those auditor's experts contribute to the audit? For example, for which components of ECL was their knowledge and experience relied upon, how did the external auditor use the work of auditor's experts to challenge management's assumptions for these components? Did the auditor's experts develop independent expectations to challenge management's assumptions?

The external auditor is required to assess the expert resources it will need to undertake the audit, including in relation to ECL. Auditor's experts should have a deep knowledge of their respective fields, including understanding current and emerging best practices. They therefore have a greater ability to assess the reasonableness of some aspects of management's approach and to provide alternative views and assumptions. They also may have a greater ability and credibility to challenge management and their experts. These attributes also make them suitable to use as part of a stand back test when assessing whether the ROMM conclusions remain appropriate or whether sufficient and appropriate audit evidence has been obtained.

Specialised knowledge can be provided by any member of the engagement team or, specifically, by an auditor's expert.4 Due to the highly specialised nature of the ECL estimate, the use of non-accounting information, the potential use by management of its experts, and the expectation that the ECL is determined by the external auditor to be a significant risk, it is likely that use of auditor's experts will be necessary to obtain sufficient appropriate audit evidence. It is likely that the auditor's work on the ECL estimate will require multiple auditor's experts. The diverse specialisations that are usually required to produce ECL numbers and disclosures mean that, in many cases, it will be unlikely that any one individual, or the engagement team collectively, will have the necessary expertise or depth of knowledge to effectively audit the ECL estimate without using the work of an auditor's expert.

The auditor's experts can be used at each stage of the ECL audit, including to help obtain an understanding of the entity, identify and assess the risks of material misstatement, design effective tests of controls and contribute to and evaluate the sufficiency and appropriateness of audit evidence.

Notwithstanding the use of auditor's experts on the audit, the external auditor retains sole responsibility for the audit opinion expressed.

The external auditor may also need to use the work of additional types of auditor's experts. For example, auditor's legal experts may be needed where contracts or security arrangements are complex, or where litigation may be involved to recover the bank's collateral. The external auditor assesses, as part of its audit planning, the range of expertise that will be required to undertake the audit.

Expectation 8: The external auditor should determine that any auditor's experts used have the competence to carry out the work asked of them and will ensure the work has been properly scoped and evaluated to contribute to the external auditor's assessment of the reasonableness of the ECL estimate and related disclosures.

Question 8: How did the external auditor evaluate and become comfortable with the auditor's experts' competence and objectivity, determine the nature, scope and objectives of the auditor's experts' work and evaluate the work of the experts?

To be able to use the work of an auditor's expert, the external auditor must first ensure that the auditor's expert is appropriate for the external auditor's purposes. The external auditor should evaluate whether the auditor's expert has the right competence, capability and objectivity. And the external auditor should obtain a sufficient understanding of the field of expertise of the external auditor's expert to be able to scope and evaluate the adequacy of their work. For example, some experts are knowledgeable about retail credit risk but not corporate credit risk. In addition, the external auditor should determine and set out the scope and objective of the work that the auditor's expert will undertake and then evaluate whether the work performed is adequate to meet the objective. Such an evaluation includes evaluating the expert's findings, whether any significant assumptions or methods of the expert are relevant and reasonable, and whether any data used by the auditor's expert is sufficiently relevant, complete and accurate. For example, where an auditor's expert has evaluated whether management's forecasts take account of all reasonable and supportable information about credit conditions, the external auditor may wish to document discussions with the auditor's expert on how they made the evaluation and what information the auditor's expert used in making it.

Assessing competence, capability, objectivity and the field of expertise is also important where the engagement team makes use of audit delivery models (ADMs), which can include solutions such as "centres of excellence" or "outsourcing." External auditors are required to apply relevant internationally accepted auditing standards regarding the work of experts used through ADMs.

The following are areas where the external auditor may need to use the work of an auditor's expert.

FLI feeds into macroeconomic scenarios and weighting and can be of a specialised nature. Some examples of FLI include macroeconomic indicators, predictions of credit defaults and estimates of recoveries. Auditor's experts in econometric modelling, credit risk or valuation may support the external auditor in:

1. assessing the reasonableness and appropriateness of management's projected economic variables and detailed forecast period through peer comparison and use of expert judgment;
2. assessing the internal consistency of management's forecasts and elements of FLI that are used for internal and external reporting purposes;
3. assessing the reasonableness of management's key assumptions pertaining to the projection of future recoveries by providing independent views of:
   1. the likelihood of success of proposed recovery action;
   2. the viability of alternative recovery strategies;
   3. the projected recoverable amounts; and
   4. whether management has considered all reasonable available information in making its recovery assumptions;
4. assessing the reasonableness of management's land and property valuation estimates, and the projected valuations of other types of collateral.

The ECL estimate reported in the financial statements may be the sum of probability-weighted outcomes based on macroeconomic scenarios. An external auditor's experts in economics, econometric modelling or credit risk may support the external auditor in some of the following ways:

1. assessing the reasonableness, severity and probability of different macroeconomic scenarios, where a bank is using discrete macroeconomic scenarios. For example, the expert can help the external auditor to assess whether management's range of scenarios considered, and the probability-weights assigned to them, is reasonable considering the expert's and market views of economic uncertainty and is not, for example, overly reliant on the base case macroeconomic scenario. The external auditor's expert may also assist in understanding and assessing the rationale underlying potentially overly pessimistic or optimistic scenarios used to calculate ECL relative to scenarios used for internal stress testing;
2. assessing the appropriateness of the relationships and correlations between variables and of the economic modelling techniques and assumptions; and
3. assessing whether management's macroeconomic scenarios and weighting include key drivers of credit risk for the bank's portfolios.

The external auditor of a bank may use auditor's experts to assist in assessing and responding to the risks of material misstatement arising from the complexity and subjectivity associated with models.

1. A credit modelling expert can help the external auditor to evaluate management's design of models and the appropriateness of methods, assumptions, parameters and techniques used by management in the model. For example, it is not always possible or cost-effective to model every driver of credit risk. The expert may help identify the key parameters to which the ECL estimate is most sensitive and which should be the focus of the external auditor's efforts as to how they have been modelled or otherwise considered in the development of the ECL estimate.
2. The involvement of the external auditor's expert is likely to be beneficial in auditing all elements of a model. This includes reviewing management's design and implementation of a model (including management's approach to model validation and testing) and assessing a model's performance over time. The external auditor's expert may be able to help the external auditor assess whether a model achieves the objectives set out in the relevant ECL framework (including where the model uses or has been built upon regulatory models). The external auditor's expert may also be able to assist in determining, where different sub-models have been used, whether those models fit together to achieve the required objective. Model adjustments (eg to compensate for model weaknesses or data gaps), which often involve application of experienced credit judgment, may be a particular area of focus for an auditor's expert given the judgmental or subjective nature of model adjustments.
3. In terms of market perceptions of best practice, banks may develop and change their models across a number of years (ie model assumptions and techniques are refined to better reflect the risks being modelled), and models need to evolve and adapt to the changing portfolio of the bank's exposures. The external auditor may therefore need to make use of experts on an ongoing basis to assess whether the bank's modelling approaches remain fit for purpose and appropriate in the context of the bank's portfolio of exposures and prevailing economic environment. IT experts may be able to assist the external auditor in testing model change controls.
4. There are various issues around data where an auditor's expert may provide assistance. A key part of an effective modelling approach is the ability to manage data without loss of integrity, in particular where there may be multiple data sources and systems through which the data flows. In addition, data may not have been part of the financial reporting systems prior to the application of ECL frameworks (eg data from risk and regulatory systems). There is also the risk that data does not remain accurate and complete as it flows through the bank's information systems and is subject to various manipulations. As such, the external auditor will likely need to engage IT experts, for example to ensure systems are secure (physically and logically) or to test the ability of systems to accurately and completely process the bank's ECL data.
5. Experts may also be able to assist the external auditor in assessing whether data is complete, accurate and relevant. This is particularly important where an ECL estimate is very data dependent - for example as with most retail portfolios, but also some SME portfolios. It is also important as data, and in particular historic data, may cease to be relevant with changes to circumstances. For example, historic loss data might in some cases only reflect a benign or stable economic environment and will thus be incomplete if used in a model that needs to use downturn FLI. In such circumstances, new data or other actions such as model adjustments will be required.

Auditor's experts in credit risk can help assess the reasonableness of management's SICR criteria and thresholds by:

1. advising on the appropriateness of SICR criteria and thresholds used by management considering measures of credit risk for particular exposures or groups of exposures, including the information and techniques commonly used in the market place. If a PD model is being used, the calibration of the SICR criteria and the qualitative indicators used can also depend on the factors incorporated into the PDs, which auditor's experts can also help assess;
2. assisting with the assessment and interpretation of other credit information available within the bank (eg material produced by a credit risk function) or outside the bank, to identify potential discrepancies or inconsistencies with the SICR threshold, which the external auditor should then investigate;
3. applying their awareness of emerging market conditions (ie portfolio level events) or of troubled exposures that have yet to feed through a bank's automated processes (or PDs) to help the external auditor to determine whether management appropriately selected and calibrated the SICR criteria - to help the external auditor judge whether the SICR criteria contain a comprehensive set of risk factors, either through quantitative or qualitative elements; and
4. assessing whether management has considered all relevant, available and current information (including FLI) in forming its SICR-related judgments.

The nature of the credit risk and how the risk is measured, may be affected by factors such as exposure type (eg mortgages, unsecured loans, credit cards and leasing), the sector (eg lending to retail customers, the oil and gas industry, and SMEs), or geography. As a result, the audit may require more than one credit risk expert depending on the nature, size and complexity of the bank's exposures.

Disclosure

Engagement teams may leverage auditor's experts in credit risk and data to help assess the data underlying ECL disclosure. For ECL disclosure, auditor's experts in credit risk or economics may support the engagement team by:

1. providing credit risk-related knowledge to aid the external auditor in determining whether management has provided the disclosures that are most likely to be material to the users of the financial statements such as those related to FLI and other key judgments; and
2. leveraging knowledge of other areas of the bank's business in assessing the ECL credit risk disclosures for internal consistency within the bank.

Audit evidence

Issue

The external auditor obtains sufficient appropriate audit evidence about whether accounting estimates and the related disclosures are reasonable. Audit evidence is information used by the external auditor in arriving at the conclusion on which the external auditor's opinion is based. Where the risk of material misstatement is higher, the external auditor requires audit evidence that is more persuasive. As it is expected that the external auditor will assess significant risk(s) of material misstatement related to the ECL estimate, the external auditor will need to obtain highly persuasive evidence in addressing the risk. If the external auditor does not obtain sufficiently persuasive evidence, there is a risk that fraud or errors (including those arising from management bias) have gone undetected and the ECL estimate and related disclosures may be materially misstated.

Supervisory expectations for the external auditor

Expectation 9: The external auditor should obtain sufficient appropriate audit evidence about whether the ECL estimate and related disclosures in the financial statements are reasonable in the context of the applicable ECL framework. The external auditor's audit procedures should be responsive to the assessed ROMM in relation to ECL and its components.

Question 9: What were the main sources of evidence gathered to address significant ROMM related to ECL, including ROMM for significant components? How did the external auditor determine that they had sufficient relevant and reliable evidence regarding the reasonableness of the ECL estimate and related disclosures?

Expectation 10: The external auditor should obtain more persuasive evidence to support the audit of the ECL estimate and related disclosures because of the significant ROMM.

Question 10(a): What was the strength of evidence obtained from sources other than management which were responsible for preparing the financial statements, including from other parts of the bank, external sources, the external auditor and the external auditor's experts?

Question 10(b): How did the pieces of evidence contradict or support management's assertions and how did the external auditor evaluate their persuasiveness? What contradictory evidence was gathered, and how was contradictory evidence weighed in relation to supporting evidence?

The auditor reaches their judgment on whether there is a material misstatement on the basis of gathering sufficient appropriate audit evidence. Sufficiency is an assessment of the quantity of information gathered, and appropriateness is an assessment of the quality (relevance and reliability) of the information.

The ECL estimate is likely to be a material estimate for the users of the bank's financial statements. Given this likelihood, along with the expectation that ECL will give rise to significant risk(s) of material misstatement (see Expectation 4), it follows that the external auditor will need to gather highly persuasive evidence to be able to reach a robust judgment that the ECL estimate is not misstated.

Consideration of the following may assist the external auditor in determining whether they have sufficient appropriate evidence:

1. obtaining relevant (and reliable) evidence from sources external to the bank;
2. using multiple evidential sources or procedures;
3. considering evidence that may contradict as well as corroborate management's ECL estimate;
4. performing own (auditor) analysis and developing independent expectations;
5. using the work of auditor's experts (see 'Use of experts' section);
6. critically assessing any evidence provided by management's experts; and
7. using actual observations or outcomes to evaluate or backtest elements of the ECL estimate.

The quality and persuasiveness of audit evidence is influenced by the relevance and reliability of the nature and source of information. For example, evidence from sources external to the bank (such as third party confirmations and industry benchmarking data) is often more reliable than evidence produced by management (such as accounting records, meeting minutes and management representations) because it is produced independently of the audited bank (though the auditor will need to assess it for relevance). Similarly, information that is produced and controlled independently of those making the ECL estimate may be more reliable. Accounting frameworks may require elements of the ECL estimate to be based on a bank's internal data (for example, loss data) and the external auditor will perform audit procedures and analysis on that data to ensure it is relevant, complete and accurate. The external auditor may also want to compare the bank's internal data to other relevant third party sources (for example, external ratings and PDs, if there are concerns with the completeness or integrity of a bank's own internal data). For other elements of the ECL estimate there will be multiple sources of evidence that the auditor may use.

Corroborative evidence from different sources is generally more persuasive than evidence from a single source. For example, testing management's internally generated projection of future unemployment by comparing it with a corresponding projection by a central bank (ie a party external to the audited bank) that was undertaken with the same objective in mind and finding them comparable would likely yield more persuasive audit evidence than simply assessing the rationale underlying management's projection of unemployment and finding it reasonable. Where the external auditor obtains pieces of evidence that are not consistent with each other, the external auditor should consider all the evidence obtained, even if contradictory rather than corroborative, and assess the impact on the quality of evidence, accordingly.

Methods of obtaining evidence from substantive procedures include the results of reperformance tests, recalculations, reconciliations and sensitivity analyses. The external auditor may have more confidence in the reliability of audit evidence when an auditor's expert is involved in the performance of the procedures, depending on the subject matter. For example, the external auditor might test inputs into the ECL process such as PD, EAD or LGD by recalculation. Involving an auditor's credit risk expert with a strong understanding of the data and methods underlying the calculations may support a better ability to determine whether there are errors or inconsistencies.

For ECL-related estimates, evidence of the actual outcome of the estimate, when available, provides more persuasive evidence. For example, if a model were to predict a particular level of future credit losses, but the actual losses turn out to be substantially higher than the model predicted, it may indicate that an assumption within the model is not reasonable and should be updated. However, "a difference between the outcome of an accounting estimate and the amount recognised in the previous period's financial statements does not necessarily represent a material misstatement of the previous period's financial statements" if the difference arises from information that was not available to management at the time that the previous period's financial statements were finalised. Because ECL is a weighted average amount based on a range of scenarios, the outcome will be different from the estimate and cannot be "verified" in retrospect. Nevertheless, elements of the ECL estimate may be able to be backtested. For example, the external auditor may be able to test the reasonableness of a bank's PD model assumptions by backtesting among other tests.

An audit is cumulative and iterative, and therefore the external auditor may need to stand back and reflect on the sufficiency and appropriateness of the evidence being gathered at various points during the audit process. In addition, any stand back test needs to be timely to ensure there is sufficient time to collect any further necessary evidence.

The external auditor's analysis and development of independent expectations (usually of a component of a bank's ECL estimate but sometimes of an estimate as a whole) may be an important audit procedure. The expectations will help establish the standards or benchmarks that the external auditor will use to judge whether the cumulative evidence gathered is sufficient and appropriate. In developing their expectations, the external auditor should consider: (i) the requirements of the accounting framework; (ii) the risks in the bank's portfolios; (iii) market practice and peers; (vi) their experience from prior periods (where applicable); and (v) how the external economic environment may affect the bank's risks and estimates of ECL. Differences between the auditor's independent expectation and management's estimates should be appropriately understood and assessed.

The external auditor may undertake peer comparison for elements of the ECL estimate or for the estimate as a whole. The comparisons do not usually produce persuasive evidence but they can be helpful in identifying areas for investigation. The more the characteristics of the peer and the ECL element or portfolio of the audited bank are aligned, the more beneficial the analysis can be.

Weaknesses in internal controls, prior adverse experience of the bank and its management in making estimates, or errors identified when undertaking audit procedures may also affect the auditor's assessment of sufficient appropriate evidence.

If the external auditor has not obtained sufficient appropriate evidence, they should seek to obtain further audit evidence. If the external auditor is unable to obtain sufficient appropriate evidence they should express a qualified opinion or a disclaimer of opinion on the financial statements.

Considerations for key components of ECL

Where controls testing is undertaken, sufficient appropriate evidence will be obtained from a combination of controls testing and substantive procedures. The nature and extent of the substantive procedures required will, in part, depend on the external auditor's assessment and testing of controls. In performing audit procedures and gathering information for the various components of the ECL estimate, the auditor should reflect on whether the information obtained indicates a need to revise the assessment of the risks of material misstatement and/or the planned audit procedures (for example, because the information gathered contradicts previous findings or the auditor's expectations).

Forecasts and FLI

Some of the sources of more persuasive evidence the external auditor might obtain regarding controls over forecasts and FLI are as follows:

1. evidence of effective review and challenge by management of FLI used in scenarios for reasonableness and mitigation against recency bias (where estimates of FLI are overly influenced by recent conditions and do not fully take account of expectations of future developments); and
2. evidence of the likely effectiveness of management's current estimation process from a review of the outcome of previous estimates (ie a retrospective review).

Evidence from substantive procedures might include:

1. evidence of reviews of peer comparisons of the detailed forecast periods and forecasts used by other banks for portfolios with similar credit characteristics located in similar economic or geographic areas;
2. assessment of internal consistency as to whether the bank has used the same FLI to calculate ECL for management reporting and external reporting, and challenge of differences;
3. comparing previous estimates to actual outcomes and assessing the reasons for differences to understand their impact on current forecasts and FLI estimates;
4. obtaining and assessing data used in FLI that is available from third parties independent of the bank and its management (eg credit ratings, loss data, asset valuations, economic forecasts). The external auditor can obtain such data and compare it to what has been used by the bank and analyse and understand any differences, challenging management where necessary; and
5. examining published economic reports from third parties and comparing these to management's detailed forecast and forecast period used to measure ECL.

A bank's forecasts and FLI should be reasonable and appropriate to its circumstances. Differences between other information and that produced by a bank do not necessarily indicate that the bank is using inappropriate information. The external auditor will want, however, to understand the basis for the bank's information and why it is different from other forecasts and FLI. In particular, the external auditor will want to understand whether there is any indication of bias and how supportable the bank's forecasts or FLI are.

Macroeconomic scenarios and weighting

Some of the sources of more persuasive evidence the external auditor might obtain regarding controls testing of macroeconomic scenarios and weighting are listed below. This evidence is used to evaluate whether management's methods, assumptions (ie the subjective elements of the ECL estimate) and data have been appropriately made.

1. A clear, transparent, approved and implemented policy document that specifies:
   1. the information management requires for generating, selecting and updating the range of scenarios and associated probability-weights used in ECL calculations;
   2. the limitations inherent in management's chosen approach to scenario design;
   3. the criteria that management will use to identify when the chosen approach is insufficient to capture credit risk drivers or economic uncertainties and the actions that will be taken as a result; and
   4. how management ensures that scenarios take account of all reasonable and supportable information about credit conditions at the reporting date, even when forecasts are developed prior to the reporting date (and may, as a result, be lagging).
2. Evidence of independent and effective review and challenge of the range of scenarios, and the associated probability weights and model outputs for reasonableness:
   1. for example, a sensitivity analysis may have been performed to ensure macroeconomic scenarios and weightings captured key ECL drivers and relationships between inputs and outputs; and
   2. additionally, management may have performed backtesting to assess the reasonableness of model inputs and outputs.
3. Evidence of independent and effective review and challenge of the rationale for the number of scenarios, for example:
   1. in cases where management has determined that using only one scenario is appropriate, how management has decided that is consistent with the requirements of the accounting framework that ECL be a probability weighted outcome;
   2. how management has determined that the scenarios selected adequately capture the impact of non-linearity of credit losses; and
   3. where economic uncertainty is elevated, how the number of scenarios used by management takes economic uncertainty fully into account.

In terms of substantive procedures, third party economic forecasts are frequently available and can provide an independent, objective view on the choice of management's scenarios, including the nature of the scenarios (eg expansionary and recessionary), time horizons, their defining parameters (eg unemployment and inflation) and the projected probabilities of occurrence. In particular, a consensus forecast is likely to be available, against which the external auditor may compare management's base case projection.5 Many third party institutions (eg central banks) prepare economic forecasts. The auditor's experts may also produce forecasts. As well as considering the reliability of such forecasts, the external auditor will want to test the forecasts to ensure they are relevant to the bank's circumstances.

The external auditor may compare management's macroeconomic scenarios to those of relevant sources. The bank itself will use economic forecasts across different parts of its business (eg credit risk management, overall business model planning and budgeting). There should be consistency between the forecasts as they will relate to the same underlying business and risks. The external auditor may also compare management's macroeconomic scenarios to those of peers who have similar portfolios and risks.

Models

Some of the sources of more persuasive evidence the external auditor might obtain in undertaking controls testing around models to ensure their reasonableness are as follows:

1. evidence that management's model design, implementation and operating performance includes effective processes for review, validation model change and testing (eg by backtesting or sensitivity analysis, which may help the auditor identify and effectively assess variables that, when altered slightly, can have a significant impact on model outputs);
2. evidence of model validations performed by individuals independent of lending activities and subsequent escalation of findings and remediation. This segregation of duties between the model validator and operator helps mitigate the risk of management bias;
3. evidence that there are effective change management controls that can ensure changes to models are made in a timely, accurate and complete way; and
4. evidence of the existence and operation of an approved policy on model adjustments covering:
   1. a governance framework for reviewing and supporting management's use of model adjustments and the implementation of decisions relating to how they should be calculated, the completeness of model adjustments and when model adjustments should be included or removed. The governance framework should identify those with responsibility to review and the authority to approve model adjustments (such as a multidisciplinary governance committee);
   2. supportable rationales for model adjustments made and for the completeness of model adjustments, including what they are intended to achieve, how they are to be calculated and a log of key model limitations; and
   3. when model adjustments should be included (eg as a result of model or data limitations identified through other procedures, such as model validation or performance reviews) or removed.

Some examples of the more persuasive substantive procedures that the auditor may undertake include reperforming or recalculating elements of the model to independently test that the model operates as intended; and sensitivity analysis to identify whether the model properly takes account of key assumptions and whether non-linearity of relationships between inputs and outputs has been appropriately captured.

The external auditor's experts may also help the external auditor to perform independent testing of the model's design and performance, as well as testing management's controls.

SICR

Some of the sources of more persuasive evidence the external auditor might obtain regarding tests of controls for SICR include:

1. documented and operational SICR thresholds that are aligned to the credit characteristics of the portfolio and where portfolios with similar risks share similar thresholds; and
2. evidence of review and challenge of SICR criteria, including validation criteria and thresholds, to ensure effectiveness of the thresholds.

Some of the more persuasive substantive procedures for SICR are:

1. investigation of management overrides and assessment of the implications of management overrides of SICR criteria (ie instances where management did not adhere to its own SICR thresholds) for the effect on the reasonableness of those thresholds;
2. sensitivity testing of SICR thresholds to ensure reasonable timeliness of transfer from stage 1 to stage 2 (ie from a 12-month ECL to a lifetime ECL);
3. backtesting of SICR thresholds to evaluate the:
   1. timeliness of the transfer to stage 2 (where too high a threshold results in a non-timely transfer to stage 2); and
   2. relationships between changes in certain factors and the corresponding change in credit risk;
4. reviews of individual loan files to help assess whether SICR criteria are complete and accurate and whether the criteria are being applied in a way that is consistent with the bank's policies; and
5. development of the external auditor's own independently formed expectation of SICR criteria based on best practices or analysis of the effect of changes in macroeconomic variables, allowance coverage ratios, percentage of book in stages 1, 2 and 3, and movement between stages. The external auditor may also develop expectations of the composition of stage 2 exposures, for example how much is from changes in probability of default, how much is past due or how much is on a bank's watch list. This analysis may be achieved by using the knowledge and expertise of an auditor's credit risk expert.

Disclosure

Sources of higher quality evidence the external auditor may obtain and evaluate regarding disclosure include:

1. evidence of effective review of ECL-related disclosures by management for accuracy, completeness, reasonableness, and transparency. This may include:
   1. the extent to which the bank's ECL disclosure describes the nature and extent of credit risk arising from the bank's credit exposures, including stage allocations and how the bank manages those risks, and the nature and extent or measurement uncertainty underlying the ECL estimate;
   2. the transparency in which the disclosure communicates key assumptions made by management, including assumptions about the likelihood and effect of future economic conditions; and
   3. evidence that the amounts included in the disclosures have been agreed to or have been reconciled to corresponding data in risk or finance systems and that the data is complete, accurate and relevant.
2. the external auditor performing a stand back test to evaluate the adequacy of the disclosure by considering the relevance to users in understanding the ECL estimate and whether it is fairly presented in the context of the overall financial statements (including whether the disclosures are comprehensive). Disclosures can be excessive, obscuring material information, and both management and the auditor need to consider the balance between disclosures that do not provide a fair and comprehensive presentation and those that may reduce overall understanding. The stand back test includes considering the bank's portfolios and the overall evidence obtained during the audit, including the external auditor's understanding of the bank and the risks to which it is exposed. Although the external auditor is responsible for this assessment, an auditor's expert in credit risk may provide detail as required to support the external auditor's assessment such as:
   1. a comparison of the bank's ECL disclosure against those of peer banks from recent periods; and
   2. consideration of external sources of best practices for high-quality disclosures, such as publications by audit firms and bodies such as the Enhanced Disclosure Task Force of the Financial Stability Board.