Claudia Buch: Evaluation and macroprudential policy

Speech by Prof Claudia Buch, Deputy President of the Deutsche Bundesbank, at the Annual meeting of the Verein für Socialpolitik, Münster, 7 September 2015.

Ladies and gentlemen

The theme of this year's annual meeting of the Verein für Socialpolitik is "Economic development - theory and policy". The evaluation of development aid projects has a relatively long tradition - from which many other policy areas may benefit. Much greater use could still be made of research findings on the evaluation of the efficiency and effectiveness of public projects and economic policy measures. I wish to illustrate this using macroprudential policy as an example.

Why are development aid projects frequently evaluated? For one thing, financiers expect accountability on the impact of their financial assistance. For another, researchers can often contribute to the project design. That lets them incorporate the features needed for an ex post impact analysis while the project is still on the drawing board. Ex ante planning allows evaluation data to be collected in a manner conducive to maximising the reliability of econometric identification. Tackling a stimulating research topic without nagging questions about the identification strategy boosts the likelihood of being published. That's an incentive for researchers to get involved in evaluation and the key to putting evidence-based economic policymaking into practice. And that, ladies and gentlemen, is what I think we need to see more of in other policy fields.

But the actual impact of measures is often just as uncertain outside the realm of development aid. In many cases, the projects in question are much larger still and relate to entire countries - tax policy, education policy, health policy - and not forgetting financial market regulation. The question for me is this: how can we make sure that these areas, too, are subject to greater and, above all, scientifically proven evaluation?

There are two hurdles to overcome, in my view, but neither of them are insurmountable. The first is the inadequate data pool, the second a frequent absence of a need for the scientifically-based evaluation of policy measures.

As Deputy President of the Bundesbank, one of my areas of responsibility is financial stability. This includes macroprudential policy. We still don't know all that much about the impact of macroprudential policy measures right now. The available evidence is very sparse; it is often no more than cross-sectional analyses across different countries derived from short time series. We are therefore working to create the conditions for superior policy evaluation.

1. What is financial stability?

Macroprudential policy is a relative newcomer to the policy space. In some respects, the level of knowledge and experience in this area is roughly on a par with that in the field of monetary policy some years ago. The specific policy goals are difficult to quantify, transmission channels are often unknown, and the data sets and analytical methods need to be developed.

Why do we need macroprudential policy? The world of traditional banking regulation offers little scope for curbing the systemic repercussions of financial crises because it focuses primarily on the economic health of individual market participants.

For the most part, microprudential analysis is based on the assumption that the financial system as a whole is stable, that markets are liquid and that market players that run into difficulties will not have a significant detrimental impact on the financial system or the economy as a whole.

But that view is a short-sighted one, because past episodes in which seemingly isolated microprudential risks were either misinterpreted or underestimated have taught us that our work needs to take a broader perspective. Risks to the stability of the financial system can arise if large institutions create negative externalities (too big to fail), if institutions are highly interconnected (too connected to fail), or also if many smaller institutions are exposed to similar types of risk (too many to fail).

This key takeaway from the recent financial crisis prompted the establishment of macroprudential authorities around the world that focus on risks that might threaten the financial system as a whole. Let's begin with the mandate and the objective. The objective of macroprudential policy is to safeguard financial stability. Financial stability is the term used to denote the financial system's ability to perform its key macroeconomic functions - particularly so in the event of unforeseen events, periods of stress or structural upheaval.

For the costs of financial crisis can be heavy - not only with regard to the amount and duration of real economic losses but also concerning the often considerable fiscal costs. It is not about completely curbing risks and fluctuations in the financial markets but instead about reducing negative externalities caused by the distress of individual institutions or market segments.

Here, we need to ask ourselves how "financial stability" can be measured and concrete indicators formulated. After all, even though the situation appears to be stable, the creation of systemic risk through moral hazard can jeopardise financial stability. That's why reducing misguided incentives and increasing the size of risk buffers in the system are key to maximising financial system stability. From this general definition, indicators - such as property prices, risk premiums or credit growth - are derived in the policymaking process and used to assess financial stability.

2. Macroprudential policy in Germany

Main responsibility for financial stability in Europe rests at the national level. In Germany, the Financial Stability Committee (FSC) was created in 2013 as a national macroprudential body. It comprises representatives of the Federal Ministry of Finance, the Federal Financial Supervisory Authority (BaFin), the Bundesbank and the Financial Market Stabilisation Agency (FMSA), which has observer status.

The Bundesbank plays a special role in the FSC, by injecting its experience of macroeconomic analyses and of banking supervision. In a nutshell, the Bundesbank is the FSC's "analytical arm". What is more, besides preparing the Committee's annual report to the German Bundestag on the status of financial stability in Germany, the Bundesbank also drafts warnings and recommendations by the FSC and assesses their implementation.

Of course, merely establishing macroprudential institutions does nothing to boost financial stability - rather, what are needed are a strategy for macroprudential policy and the appropriate instruments.

The internationally agreed Basel III regime that is currently in force already contains a number of instruments designed with macroprudential matters in mind. For instance, systemically important banks have to hold more capital than other institutions, primarily as a way of countering the moral hazard ("too big to fail") caused by their size.

Another instrument is the countercyclical capital buffer, or CCB, which is designed to boost financial system resilience by tightening capital requirements in times of increased lending. The CCB is scheduled to become legally effective in Germany on 1 January 2016, though no decision has been taken hitherto as to its exact level.

However, these instruments are no more than a rudimentary toolkit for combatting systemic risk emanating from real estate markets. And yet the recent financial crisis clearly demonstrated just how important the residential property market is for financial stability. A number of other countries are already using dedicated macroprudential instruments for the real estate market; Germany, by contrast, does not have the requisite legal basis as yet. To close this gap, the FSC submitted a recommendation to the Federal Government on 30 June 2015 proposing the creation of new macroprudential instruments for the residential real estate market. I would expressly point out, however, that this step does not mean that these instruments will be deployed or activated.

The FSC is recommending four instruments: a cap on the loan-to-value (LTV) ratio, a cap on the debt-to-income ratio/debt-service-to-income ratio as well as an amortisation requirement. An LTV ratio, for instance, specifies a minimum amount of equity capital which a debtor must contribute to a residential real estate financing transaction. These instruments have two objectives. One is to reduce the likelihood of overindebtedness and thus of crises. The other is to stem the effects of crises, should they occur anyway, by means of larger risk buffers.

If these instruments are activated in the future, an ex post impact analysis should be carried out. That calls for sound data, which is why the Committee's recommendation asked for the creation of a legal basis for the collection of granular data and a mandatory ex post impact analysis.

3. How do data fit into the equation?

But what are "sound data"? The lessons learned from the US experience tell us that a loosening of credit standards for subprime households can be one of the major root causes of financial crises. A development of that kind can only be identified if granular data are to hand.

The Bundesbank has responded to the need for higher-quality and above all granular data by establishing its Research Data and Service Centre (RDSC).1 A new project has been launched to administer and link microdata and macrodata, and to make them available to both Bundesbank and external researchers. At present, ten members of staff are busy capturing, compiling and maintaining these data, which are being used in 120 projects.

Internationally, too, a host of initiatives have been kick-started in an effort to close data gaps. For instance, Europe's central banks will, in the coming years, be setting up AnaCredit, a comprehensive database which will help them to link loan-level data and banks' balance sheet data. That kind of data pool would open up a wealth of analytical opportunities that we could use in a great many areas.

The Bundesbank's RSDC is specifically looking to establish close ties between research and policy guidance. The idea is for researchers to be given access, free of charge and with a minimum of red tape, to the data - subject, of course, to the appropriate data protection requirements. So by increasing the availability of sound data, we are going a long way towards dismantling the first of the two hurdles we face in this field.

However, data alone are insufficient to tell us how effective an economic policy measure has been. This measure, rather, needs to be embedded in a structured impact study. Applying a set of objective and structured procedures specified upfront can yield information on effectiveness and potentially increase the likelihood that a policy measure will be accepted. They prevent situations in which inconvenient findings go unpublished or access to the necessary data is obstructed.

The guidelines adopted by the Verein für Socialpolitik are an important and a necessary step. At their heart are the general principles of methodological quality, transparency, objectivity and independence. But to make sure data shortages do not scupper evaluations, the data capture process should be planned well ahead of introduction and be enshrined in law. This modus operandi not only cuts the cost of policy evaluation, but also diminishes the risk that the necessary data cannot be provided at all ex post.

And last but not least, making evaluation mandatory would be a step in the right direction. This procedure is widely followed internationally, one example being the Office of Management and Budget in the United States, which is required by law to evaluate legislation with a substantial financial impact, while in Switzerland, policy evaluation is even enshrined in the national constitution. In this connection, let me revisit the FSC recommendation I referred to earlier. It contains a regular impact analysis which can also be verified by third parties.

Ladies and gentlemen, impact analyses are an important tool for finding out more about the efficacy of policy measures, quantifying costs and benefits, and flagging unexpected side-effects. They require guidelines for decision-makers and researchers. However, they can also only work in a forward-looking policy environment which aims to enshrine policy evaluation in law and which provides the necessary financial and data resources.

It is important for researchers to contribute their expertise and ideas, and play a part in this process. More often than not, bright ideas are the "by-products" of stimulating debate with peers. That's the reason why we're all here today, so without further ado, I will bring my speech to a close.

Thank you for your attention.