Assessment of the accuracy of the outlooks for effective GDP and CPI in the euro area – comparison of forecasts
When preparing its outlooks for euro area effective1 GDP and consumer price inflation (CPI), which are important assumptions for the CNB’s macroeconomic forecast, the CNB draws on analysts’ predictions brought together in the Consensus Forecasts (CF), a monthly survey published by the London-based company Consensus Economics, which regularly asks more than 700 economists and analysts for their forecasts. The resulting CF for each country and each indicator is the arithmetic mean of the individual forecasts of the analysts from the institutions surveyed.
As the quality of the CF is one of the key factors of fulfilment of the CNB’s own forecasts, we focused on assessing the CF’s predictive ability compared to the forecasts of international institutions. Specifically, the accuracy of the CF was compared with the forecasts of the European Commission, the IMF, and the OECD.
With regard to the extraordinary situation in 2009 caused by the totally unexpected factors underlying the financial and subsequently economic crisis, Charts 1 and 2 show the gradual evolution of forecasts of the CF and international institutions together with the subsequent actual data for 2009. Point 24 on the x-axis shows the month in which the forecast for 2009 was first published (i.e. January 2008). The forecast horizon gets shorter as one moves along the x-axis until it reaches December 2009 (point 1 on the x-axis). The y-axis shows the forecasted and subsequent actual values of the indicator.
At the forecast horizon, the forecasts for 2009 recorded the biggest ever downward revision in comparison with the forecasts for the previous years 1994–2008. In the case of the annual GDP growth forecast (see Chart 1), the difference between the first published CF forecast (1.9%) and last published CF forecast (-4.3%) for the given year was 6.2 percentage points. The international institutions were slightly more pessimistic in their growth estimates than the CF. In the case of the CPI (see Chart 2) the revision was less significant (1.4 percentage points) and was made later than that for GDP. This was because the CPI forecast for 2009 was initially revised upwards until July 2008 owing to rising oil prices in that period, and only afterwards were the CPI forecasts for 2009 revised sharply downwards.
The statistical accuracy of the forecasts was assessed by comparing the individual forecasts with the subsequent outturns for the period 1994–2009. We used the mean squared error (MSE), which measures the average of the squares of errors of the forecast and hence more strongly “captures” larger deviations from the actual value. The statistical significance of the differences in the accuracy of the forecasts was evaluated using the Diebold-Mariano test (see Table 1).2
Table 1 (Box) Diebold-Mariano test
(test of the statistical significance of differences in the MSE indicator)
| CF vs. | 1994-2009 | |||
|---|---|---|---|---|
| Effective CPI | Effective GDP | |||
| t | t+1 | t | t+1 | |
| EC | -0.01 | -0.01 | 0.00 | -0.22*** |
| IMF | -0.01 | 0.02 | -0.01 | 0.02 |
| OECD | -0.03 | -0.07 | 0.03 | 0.31 |
Note: The values in the table represent the differences of the MSE between the CF and the alternative forecasts. Negative (positive) differences mean that the CF forecasts are more (less) accurate. Asterisks indicate rejection of the null hypothesis that the forecasts under comparison are equally accurate on average with following significance: ***1%, **5%, *10%.; EC a OECD CPI since 2002.
MMF is compared with CF (April, Ocotber). All other forecasts (EC, OECD) are compared with CF (May, November).
t forecast for the current year, t+1 forecast for the next year.
The analysis revealed that none of the international institutions’ forecasts was statistically significantly better than the CF in the entire period of 1994–2009. From this perspective, the CF is – despite occasional significant errors – a comparable prediction source. In addition, the basic difference and practical advantage of the CF consists in the fact that the forecasts are published on a monthly basis, whereas the Commission, IMF and OECD publish their forecasts only twice a year. The CNB will thus continue to use the CF as a source for creating its outlooks for effective GDP and CPI in the euro area.
1 The effective euro area indicator is the indicator for 14 euro area countries weighted by Czech exports to those countries.
2 DIEBOLD, F. X., MARIANO, R. S. (1995) Comparing Predictive Accuracy. Journal of Business & Economic Statistics, Vol. 13, No. 3, pp. 253–263.