Impact of financial regulations: insights from an online repository of studies
The BIS is launching a public, online and interactive repository of studies on the effects of financial regulations, called FRAME. The purpose of this repository is to keep track of, organise, standardise and disseminate the latest findings. FRAME currently covers 83 studies and 139 quantitative impact estimates from 15 countries or groups of countries, offering a new and comprehensive perspective on what the literature has been able to document to date, and where gaps exist. We observe a high degree of heterogeneity across impact estimates, notably in terms of the effects of capital regulation on loan growth: while on average the estimated effect is that more capital leads to more lending, there are large differences across studies. A meta- analysis shows that an important driver of these differences is whether the underlying study incorporates second-round effects.1
JEL classification: E44, G21, G28.
Now that the main elements of the post-crisis banking regulations have been implemented (BIS (2018)), an evaluation of these regulations' impact comes within closer reach. And the G20 Leaders' request to the Financial Stability Board to evaluate the effects of financial reforms (FSB (2017)) has placed such analysis high on the international agenda. In recent years, the number of studies that assess regulatory impacts has grown, allowing judgments to be based on a larger sample of studies. The wide range of findings, however, calls for a systematic approach to understanding their drivers. The work presented here is an element of the response.
This special feature is structured as follows. First, we present the main features of a public repository of studies on the effects of bank regulations, called FRAME (Financial Regulation Assessment: Meta Exercise). This repository, maintained by the BIS, is not intended to be another literature stocktake, but to serve as a novel tool that tracks, organises, standardises and disseminates findings from many studies. It can be updated with new findings on a continuous basis, with the possibility for researchers to report their own studies online. Second, we present selected insights from FRAME on the effects of bank capital and liquidity, and relevant regulations - the area where most analysis has been conducted - on banks' funding costs, the pass-through of funding costs to lending rates, bank lending growth and the probability of a crisis. We show that there is a wide distribution of these estimated effects, especially on bank lending. Third, we try to explain this heterogeneity using meta-analysis techniques. We document that one important driver of the heterogeneity is whether or not the analysis accounts for general equilibrium - that is, second-round - effects.
- FRAME is an online interactive repository of regulatory impact estimates. Its purpose is to keep track of, organise, standardise and disseminate the latest findings. The FRAME repository is also meant as a sharing platform, through which users can report their own studies.
- The repository is structured according to selected bank balance sheet measures and their effects on more than 10 variables. The ratios currently being considered relate to those Basel Committee on Banking Supervision (BCBS) capital and liquidity standards which are balance sheet-based. FRAME currently covers 83 studies and 139 quantitative impact estimates from 15 countries and regions. It classifies each of these along 18 dimensions, ranging from the specific countries analysed and the size of the banks studied, to the specific sample period.
- A comparison of the effects of the various regulatory ratios in normal and crisis times suggests that the Net Stable Funding Ratio has a much stronger countercyclical effect on bank lending than bank capital or liquidity.
- We document significant heterogeneity across quantitative impact estimates, notably regarding the effects of capital (regulation) on loan growth. On average, the effect is found to be positive, but there are large differences, in part due to whether the underlying study incorporates general - or only partial - equilibrium effects.
The FRAME repository in a nutshell
FRAME (https://stats.bis.org/frame) is an online repository of studies on the effects of financial regulations. It tracks, organises, standardises and disseminates the growing number of quantitative impact studies on the effects of financial regulations. It includes studies from academia, policy institutions and the private sector, covering the period 1991 to the present.
FRAME essentially spans and updates existing surveys (eg BCBS (2016, 2019), Dagher et al (2016)). One important novelty of FRAME is that it goes beyond collecting point estimates by adding relevant characteristics of the underlying studies. The repository is structured according to selected bank balance sheet ratios related to key regulatory standards and their effects on selected outcome variables. The regulatory ratios currently included relate to the Basel III capital and liquidity standards. As a convention, the repository refers to the outcome variables as "targets" (Graph 1 and box).
FRAME contains information for 12 different targets, ranging from banks' cost of funding to overall GDP growth, and covers 83 studies and 139 quantitative impact estimates for 15 countries or groups of countries (Graph 2, left-hand panel). It is intended to be representative of the overall state of knowledge. About 60% of the estimates refer to analysis covering banks and other financial intermediaries in the United States, reflecting the greater weight in the academic literature of studies that use US data. The largest number of results are on the effects of bank capital and bank capital regulation on banks' supply of loans, funding costs, lending rates and the probability of a crisis; and on the effect of liquidity ratios and liquidity standards on bank lending (Graph 3). To facilitate comparisons, estimates are all standardised. To allow for verification and full transparency, the repository contains the original estimates, and the methodologies applied to standardise them are described in detail.
FRAME will be updated with new estimates as analyses are completed. Indeed, the repository is meant as a sharing platform through which users can report their own findings.2 To avoid potential selection bias, any study can be referenced in FRAME. The only prerequisite is, of course, that the study be public and accompanied with a disclosure statement as to whether it was sponsored (eg by the financial sector) or conducted independently (eg by academics). Users can filter studies based on this and other characteristics (see box). About half of the studies currently in FRAME were published in academic journals, and one fifth in journals uniformly rated as top ones. The quality of the estimates is expected to further improve over time, eg as researchers get access to more precise data and longer time series.
Further reading
Interpreting the estimates
The estimates reported in FRAME must be interpreted with an eye towards the type of effect they are meant to capture. Two distinctions in particular matter.
One crucial distinction is whether the estimates refer to the effects of an increase (or decrease) in an observed regulatory ratio (eg the Basel III Common Equity Tier 1 (CET1) ratio) or the effects of a change in a minimum regulatory ratio (eg the Basel III 4.5% minimum CET1 ratio). This distinction is important, because changing a minimum regulatory ratio does not necessarily make banks change their actual regulatory ratio - ie their capital ratio measured according to the regulatory standards - to the same degree (or even at all). Vice versa, banks may change their actual regulatory ratios even when the regulatory minimum does not change. As the observed ratios can vary due to many factors, estimates based on them are noisier indicators of what regulatory changes may imply than estimates based on regulatory minima. Unfortunately, few studies exist on the latter.
The gap is very large. Mainly owing to the lack of hindsight and detailed supervisory data, only 15% of the estimates in FRAME (and more generally in the literature) capture the effects of higher minimum regulatory standards stricto sensu (Graph 2, right-hand panel).3 The remaining 85% are based on observed regulatory ratios, if not on proxies thereof. When the regulatory ratio is not observable (eg owing to banks not reporting it), studies typically use proxies, whose quality may vary, not least with the measure considered. Studies of Basel III's Liquidity Coverage Ratio (LCR) or Net Stable Funding Ratio (NSFR), for example, have so far been conducted using relatively coarse approximations, such as the ratio of liquid assets or deposits to total assets: banks have started to disclose their Basel III LCR or NSFR in their financial statements only recently. In contrast, studies of capital requirements can usually rely on a relatively long time series of individual banks' regulatory capital ratios, which banks have been forced to disclose.
The FRAME repository addresses this problem head-on. It gathers all types of estimates, but makes an explicit distinction between observed and minimum regulatory ratios, as well as - whenever applicable - between the various proxies used for the regulatory ratios themselves (see box).
Another important distinction is between transition and permanent effects, ie the impact during the transition towards a tighter standard versus the final effects of that standard.4 The transition effect is about the behaviour of banks (or the economy) as they adjust to a new regulation; it depends on how they adjust, and how fast.5 The permanent (long-term) effect, in contrast, is about how banks (or the economy) behave under the new regulation once they have fully transited to it, compared to how they would have behaved had the regulation not changed. The latter situation not being observable, estimations of the long-term impact most often rely on counterfactual analyses.6 With banks facing adjustment costs to raise their capital, one can expect a regulatory tightening to have a larger impact during the transition than once banks have fully adjusted. In the case of capital regulation, estimates in FRAME include both those referring to the transitional effects of banks increasing their capital ratio by 1 percentage point and the effects of the capital ratio being 1 percentage point higher over the long term, whatever the transition dynamics.
FRAME users can also break down the distributions of impact estimates along 16 other dimensions.
For example, a user can ask whether the estimates hold for crisis or normal times (Graph 2, right-hand panel). About 30% of the estimates in FRAME approximate the benefits if banks enter a crisis with more capital, more liquid assets or more stable funding; the other 70% study such effects in normal times.
Another example is the breakdown between partial and general equilibrium effects (Graph 2, right-hand panel). By affecting many banks simultaneously, a change in the regulatory standards is likely to impact overall quantities (eg lending volume) and prices (eg deposit rate, return on equity, asset prices). These effects can have further repercussions that reinforce, dampen or reverse the initial - first-round - effects of the regulatory change. For example, higher capital requirements may make lending more costly and less easily available - which in turn may lower asset prices (eg house prices), making collateral less valuable and inducing even less lending. About 30% of the estimates in FRAME capture such indirect - yet potentially relevant - effects.
Further breakdowns include classifications by country, type of regulatory ratio (eg risk-weighted or not), methodology, whether the study has been peer-reviewed and other aspects (see box).
Further reading
Selected insights from FRAME
FRAME can be used to spotlight what the literature has, and has not, been able to document to date. Looking across the included studies, there is no indication that raising bank liquidity or stable funding ratios reduces bank lending (Graph 4, left-hand panel, red and yellow bars). Recent studies also find that if a bank has a higher capital ratio this tends to be accompanied by a higher liquidity ratio (beige bar). To the extent that holding more liquid assets is associated with more prudent behaviour, this finding is in line with the notion that banks tend to take less risk when they have more "skin in the game". Studies also confirm that the effects during the transition to tighter regulation may be different from the impact once banks have reached the new standard. For example, a 1 percentage point higher capital ratio is found to have a positive effect on loan growth in the long term, but a negative effect during the transition (compare the red bars in the left- and right-hand panels of Graph 4).7 Some of these findings should be treated as preliminary, however, since there have not always been enough studies to draw robust conclusions.
The FRAME repository: options and lexicon
FRAME (https://stats.bis.org/frame) enables users to plot the distributions of the estimated effects of a given regulation on a number of variables. This box briefly presents the main options available and terms used (Graph A).
1. Regulatory ratio: refers to the bank's balance sheet ratio subject to the regulation (eg capital ratio, LCR, NSFR). While ideally the estimates are based on the changes in the minimum regulatory requirements and individual banks' exposure to those changes, there are, in practice, too few regulatory changes and too little detailed supervisory information to allow for a precise analysis. In FRAME, 85% of the estimates are currently based on changes in the observed ratios, rather than on changes in the minimum regulatory requirement. This proportion is expected to diminish as supervisory data and longer time series become available.
2. Transition and long-term impact: refer to the cost/gain of the transition towards a tighter standard and the final effects of a higher minimum regulatory ratio. The impact estimates are standardised, to capture the transitory effect of banks increasing their regulatory ratio by 1 percentage point, or the long-term effects of the regulatory ratio being 1 percentage point higher. In microeconometric studies, transitory effects are often derived by exploiting the time series dimension of the data (eg by comparing the situation of a given bank at two points in time), whereas the long-term impact is often derived from the cross-sectional dimension of the data (eg by comparing the situations of two or more otherwise similar banks at a given point in time).
Long-term impacts may also be derived from simulating estimated or calibrated macro models.
3. Target: the variable on which the regulation has the impact. Importantly, this term does not refer to any objective of the regulation considered.
4. Level and growth rate: whether the target is a level or a growth rate, ie whether estimates capture the effects on the growth rate of economic aggregates (eg bank lending, investment, GDP) or their level, with the effects accordingly reported as a percentage point variation in growth rate, or as a percentage variation of the level.
5. Breakdown options: the multiple splits of the estimates in FRAME. These include:
- Actual or minimum regulatory ratio: whether the estimated effect is that of a change in the observed ratio, or of a change in the minimum regulatory ratio, with the latter typically derived either from calibrated or estimated macro models, or from micro data on individual banks' exposure to the change in regulation (eg Pillar 2 information, Basel III liquidity ratios).
- Regulatory ratio - detail: definition of the regulatory ratio, or proxy for it, used in the study.
- Regime: whether the estimated effect holds in crisis or normal times.
- Equilibrium type: whether the estimated effect accounts for general equilibrium (second-round) effects. In most cases, microempirical studies are classified as "partial equilibrium". Macroempirical studies and estimates from general equilibrium macro models are classified as "general equilibrium".
- Peer review: whether the study has been peer-reviewed, ie published in an academic journal.
- Mean of the regulatory ratio: refers to the mean of the regulatory ratio over the sample used for the estimation, allowing one to assess how the marginal effect of a change of a regulatory standard varies with its tightness.
- Statistical significance: whether the estimate is statistically significant at the 5% threshold.
Methodology: the methodology used in the study, eg estimation, macro model, theory.
6. Disclosure statement: whether the authors of the study had relevant or material financial interests that relate to their analysis (eg research sponsored or commissioned by lobbying groups).
State-of- the-art microempirical studies identify the long-term impact of regulation using difference-in-difference estimates. Those typically compare the behaviour (eg lending) of two groups of banks with varied exposure to the regulation (eg different regulatory ratios), before and after a given event (eg a crisis). One prerequisite, though, is that the two groups have the same behaviour before the event, ie are comparable in terms of whether, and how far, they had started a transition towards the new requirements. Note that this terminology varies from that used in macroempirical studies, in which the transition and long-term impacts are typically related to the lag structure of the econometric model (the long-term impact being calculated, for example, as the sum of lagged coefficients). The precise standard errors of the estimates, as reported in the studies, are available on request (email frame@bis.org).
Despite recent progress, the literature remains largely focused on the long-term impact of capital regulation on banks' lending activities. As an illustration, we present three insights.
First, higher bank capital raises the overall funding cost - debt plus equity - only marginally. A 1 percentage point higher bank capital ratio is associated with a less than 20 basis point higher average cost of funding (weighted average cost of capital (WACC)), which takes account of an imputed equity funding cost (Graph 5, left-hand panel, blue bars).8 Tighter capital regulation may raise the weight of equity funding in the WACC. But this is almost offset by the lower unit cost of both debt (green bars) and equity funding (red bars). This result should be set against the benchmark in the Modigliani-Miller theorem, which holds that, under ideal conditions, changes in a firm's WACC should not be affected at all by its mix of debt and equity funding.9 The fact that the WACC increases, but only marginally, suggests that the logic underlying the Modigliani-Miller theorem applies to the impact of changes in bank capital, but does not operate to the full extent.
A second insight, provided by a number of papers, is that banks with a higher capital ratio charge a higher loan rate (Graph 5, criss-crossed area). Surprisingly, the loan rate is much higher than what a one-to-one pass-through from the WACC would generate (blue bars). This finding points to a potential inconsistency between the two strands of the literature, which deserves more careful scrutiny.10 A key piece of the puzzle is the impact of higher capital on the cost of non-equity financing, which ought to be unambiguously negative.
A third insight relates to the effects of banking regulation on bank lending growth. One objective (among others) of the post-crisis regulatory reforms was to foster banking system resilience in times of stress (BCBS (2011)), while preventing future crises by deterring excessive credit growth - a strong predictor of banking crises (eg Aldasoro et al (2018)). Accordingly, following regulatory tightening, one would expect banks to be less inclined to boost lending in good times or to cut lending in bad times. To see if banking regulations indeed have this potential to smooth the credit cycle, we report in Graph 6 the effects of higher capital, liquidity and stable (ie core) funding ratios on lending growth, comparing crisis (red bars) and normal (blue bars) times. Only the stable funding ratio (right-hand panel) seems to have a clearly countercyclical impact: banks with more stable funding lend relatively more than other banks during crisis times, ie they are more resilient. They also do not increase loan growth as much in normal times. In contrast, the capital and liquidity ratios appear to have much weaker countercyclical effects.
Heterogeneity of estimates
What explains the heterogeneity of estimates? To understand better why some estimates vary so widely, we use meta-analysis techniques, which are particularly well suited when studies are not directly comparable but evaluate the same - or closely related - questions.11
Averages can mask a fair amount of heterogeneity. On average, banks with a 1 percentage point higher capital ratio (eg 13% versus 12%) have a 0.41 percentage point higher growth rate of loans, while a 1 percentage point increase in a bank's stable funding ratio increases loan growth by 0.15 percentage points (Table 1). But these estimates vary widely across studies (Graph 7). For the effect of bank capitalisation on loan growth, estimates range from -3.89 percentage points to 3.82 percentage points (left-hand panel, red dots). This range is large by the usual statistical standards (the I2 statistic is above 75%; see footnote 11), highlighting the lack of consensus in the literature. In the case of the impact of bank liquidity on loan growth, the studies do not even agree as to the sign of the average effect, which is not statistically significant. However, the literature does agree that a higher bank capital ratio reduces the probability of a crisis. On average, a 1 percentage point higher ratio is associated with a 1 percentage point lower crisis probability, with a very low dispersion in the estimates.12
We next use meta-regressions to test whether specific characteristics of the studies can explain the observed heterogeneity. Given the limited number of observations, we have to restrict our analysis to the effects of bank capital on loan growth, and consider only four potential factors: whether the capital ratio is risk-weighted or not; whether the analysis underlying the estimate controls for loan demand;13 whether the analysis accounts for general equilibrium feedback effects;14 and whether the impact is estimated during a crisis.
The results are shown in Table 2. The "baseline" meta-regression (column 1) yields the average estimate, absent using any control for these four characteristics. By construction, it is the same as the average estimate reported in Table 1 (0.41 percentage points).
The comparison suggests that general equilibrium considerations are an important driver of heterogeneity. While partial equilibrium analyses find a statistically significant and positive average impact of a 1 percentage point higher capital ratio on loan growth (0.29 percentage points), the impact is estimated to be 2.13 percentage points stronger when macroeconomic feedback (second-round) effects are taken into consideration (Table 2, column 4).15 Controlling for those effects reduces the residual heterogeneity across the studies by 1.86 percentage points (compare the I2 statistic in columns 1 and 4). And, while the reduction in heterogeneity is small, it is statistically significant.
None of the other potential drivers seems nearly as relevant. For example, the risk-weighted capital ratio is found to have only a 0.48 percentage point lower impact on loan growth than the leverage ratio (Table 2, column 2), and this difference is not statistically significant. The average estimated impact is also essentially the same, whether or not the study controls for loan demand (column 3) or crisis times (column 5).
Conclusion
FRAME is an online interactive repository of regulatory impact estimates. Its purpose is to track, organise, standardise and disseminate the latest studies on the impact of financial regulation on banks, the financial system and the macroeconomy. Its broad content offers a new and comprehensive perspective on what researchers have been able to document to date, and where gaps exist.
Comparing, inter alia, the effects of regulatory ratios in normal and crisis times, we find that the stable funding ratio has a much stronger countercyclical effect on bank lending than bank capital or liquidity ratios. But the number of studies remains relatively small. We will have much more to learn about these effects as the regulatory reforms are implemented and more analysis is done.
We document significant heterogeneity across quantitative impact estimates, notably regarding the effect of capital regulation on loan growth. The average estimated effect is positive, but varies widely across studies. Studies that incorporate second-round (general equilibrium) effects, in particular, find a much more positive impact than those that do not. The difference is statistically significant and economically sizeable. This suggests that it is very important to account for the macroeconomic impact and other indirect effects of capital regulation.
Aldasoro, I, C Borio and M Drehmann (2018): "Early warning indicators of banking crises: expanding the family", BIS Quarterly Review, March, pp 29-45.
Baker, M and J Wurgler (2015): "Do strict capital requirements raise the cost of capital? Bank regulation, capital structure, and the low-risk anomaly", American Economic Review Papers & Proceedings, vol 105, no 5, pp 315-20.
Bank for International Settlements (2018): "The financial sector: post-crisis adjustment and pressure points", Annual Economic Report 2018, Chapter III, June.
Basel Committee on Banking Supervision (2010): An assessment of the long-term economic impact of stronger capital and liquidity requirements, August.
--- (2011): Basel III: A global regulatory framework for more resilient banks and banking systems, June.
--- (2016): "Literature review on integration of regulatory capital and liquidity instruments", BCBS Working Papers, no 30, March.
--- (2019): "The costs and benefits of bank capital - an update since BCBS (2010)", BCBS Working Papers, no 34, forthcoming.
Bogdanova, B, I Fender and E Takáts (2018): "The ABCs of bank PBRs: What drives bank price-to-book ratios?", BIS Quarterly Review, March, pp 81- 95.
Cerra, V and S Saxena (2017): "Booms, crises, and recoveries: a new paradigm of business cycle and its policy implications", IMF Working Papers, no WP/17/250, November.
Dagher, J, G Dell'Ariccia, L Laeven, L Ratnovski and H Tong (2016): "Benefits and costs of bank capital", IMF Staff Discussion Notes, no 16/04, March.
Financial Stability Board (2017): Framework for post-implementation evaluation of the effects of the G20 financial regulatory reform, July.
Gambacorta, L and A Murcia (2017): "The impact of macroprudential policies and their interaction with monetary policy: an empirical analysis using credit registry data", BIS Working Papers, no 636, May.
Gropp, R, T Mosk, S Ongena and C Wix (2019): "Banks response to higher capital requirements: evidence from a quasi-natural experiment", Review of Financial Studies, vol 32, no 1, pp 266-99.
Harbord, R and J Higgins (2008): "Meta-regression in Stata", The Stata Journal, vol 8, no 4, pp 493-519.
Jordà, O, B Richter, M Schularick and A Taylor (2017): "Bank capital redux: solvency, liquidity, and crisis", NBER Working Papers, no 23287, March.
Khwaja, A and A Mian (2008): "Tracing the impact of bank liquidity shocks: evidence from an emerging market", American Economic Review, vol 98, no 4, pp 1413-442.
Macroeconomic Assessment Group (2010a): "Assessing the macroeconomic impact of the transition to stronger capital and liquidity requirements - Interim Report", August.
--- (2010b): "Assessing the macroeconomic impact of the transition to stronger capital and liquidity requirements - Final Report", December.
Modigliani, F and M Miller (1958): "The cost of capital, corporation finance and the theory of investment", American Economic Review, vol 48, no 3, pp 261-97.
Romer, C and D Romer (2017): "New evidence on the impact of financial crises in advanced countries", NBER Working Papers, no 21021, March.
1 The authors thank Claudio Borio, Ben Cohen, Tara Rice, Hyun Song Shin and Nikola Tarashev for helpful comments. The views expressed in this article are those of the authors and do not necessarily reflect those of the BIS.
2 Users are invited to self-report by filling in a reporting template, downloadable from the website. The data collected are shared upon request; please send requests to frame@bis.org.
3 Those are relatively recent, and most often refer to those capital standards for which implementation is advanced enough to allow for analysis.
4 This latter effect is sometimes referred to as the "long-term economic impact" (BCBS (2010)). In the rest of this article, we use the term "long-term impact" to indicate the final effect of a regulation once banks have fully phased it in.
5 Following a tightening in capital standards, for example, a bank may retain earnings, raise new capital or shed risky assets ("deleverage"). The last option takes less time than the previous two, but is likely to have stronger overall economic effects (Gropp et al (2019), Macroeconomic Assessment Group (2010a,b).
6 There are several ways this can be done. In macroeconometric studies, the long-term impact of a policy change is typically associated with the lag structure of a dynamic model and calculated as the sum of lagged coefficients. This could also be used to measure the impact of regulatory changes, although few studies do so. More common are microeconometric studies, where the effect is often derived from the cross-sectional dimension of the data, eg by comparing, at a given point in time, the behaviour of banks that already comply with the new regulation with that of similar banks not subject to it or otherwise not having started their adjustment. Each approach has pros and cons. Cross-sectional comparisons could be misleading, as they rely on the assumption that the two groups of banks are similar otherwise. At the same time, compared with macroeconometric approaches they can better identify the effects of (changes in) banks' balance sheet conditions on loan supply. They can control for credit demand, for example through firm-time fixed effects. Another approach is using model simulations, which try to address selection, endogeneity and other issues, though these are very sensitive to the modelling assumptions. With its transparency, FRAME allows users to break down the estimates based on the specific approach used and assess robustness to the methodology.
7 As noted earlier, the transition effect probably depends on the way (and speed) banks adjust to the regulation.
8 Investors' required returns on equity are not directly observable and have to be imputed from an equity asset pricing model. The standard capital asset pricing model (CAPM) predicts that a bank's expected return on equity is proportional to its estimated equity beta, obtained by regressing the bank's excess returns on the excess value-weighted market returns (see eg Baker and Wurgler (2015)). Bogdanova et al (2018) provide an application.
9 Modigliani and Miller (1958).
10 One possible explanation could be that the two strands of the literature have been using different methodologies, with some addressing endogeneity issues better than others. Another explanation is that banks' internal markets do not operate as basic models suggest, creating segmentation, with capital adequacy requirements binding more in lending than in other business segments.
11 We assess whether the differences observed across studies reflect "genuine" heterogeneity (eg due to the true impact of regulation varying across countries, periods, methodologies) or "random" heterogeneity (eg due to the studies using different samples of observations for the same country, period, methodology, etc). The meta-analysis allows us to decompose the variance of the estimates into those two sources of heterogeneity. The key statistic is the I2, which is the percentage of the variance of the estimates that is attributable to genuine between-study heterogeneity as opposed to sampling variance. As a rule of thumb, heterogeneity is considered high when I2 > 75% (Harbord and Higgins (2008)). For a more detailed explanation of meta-analysis techniques applied to economics, see Gambacorta and Murcia (2017).
12 This result, which to a large extent is based on studies using a similar approach (BCBS (2010)), has recently been subject to debate. New findings by Jordà et al (2017) have shifted attention from the crisis prevention effects of capital to the cost containment benefits. Using more detailed bank balance sheet and crisis historical data, they conclude that bank capital "-has no value as a crisis predictor but does reduce crisis costs". But as there are still very few studies on the effect of regulation on the cost of crises (eg in terms of GDP loss; see Graph 3), this result has yet to be confirmed. FRAME does not cover the literature on the cost of crises per se (see eg Romer and Romer (2017) and Cerra and Saxena (2017)).
13 One state-of-the-art methodology (Khwaja and Mian (2008)) uses bank-firm loan data (typically from credit registries) and firm fixed effects in the regression to control for loan demand. Under the assumption that firms spread their demand equally across their respective banks, such fixed effects capture variations in loan demand from each firm towards each of its banks. Controlling for those, the effect of regulation or balance sheet ratios on banks' loan supply can then be identified by comparing the lending behaviour of the banks most affected by the regulation with lending by other banks. Accordingly, to account for the quality of estimates, we distinguish between studies using firm fixed effects (ie those that control for loan demand effects) and those that do not.
14 Those are typically estimates derived from macroeconomic analyses.
15 In a meta-regression, each observation is weighted by the inverse of the sum of its variance (here the standard error of the estimate) and the between-study variance (so-called "residual heterogeneity"). Graph 6 (left-hand panel) shows that there are negative point estimates. But because those negative estimates are not precise (their confidence interval extends over zero values; see Graph 7, left-hand panel), they are not given large weights in the meta-regression. This explains why the average partial equilibrium effect is positive and statistically significant in the regression (Table 2, column 4, coefficient 0.29), in spite of some point estimates being negative.
