Representing uncertainty about response paths: The use of heuristic optimisation methods

doi:10.1016/j.csda.2006.12.023

Computational Statistics & Data Analysis

Volume 52, Issue 1, 15 September 2007, Pages 121-132

https://doi.org/10.1016/j.csda.2006.12.023 Get rights and content

Abstract

One of the standard procedures in impulse response analysis is the construction of confidence intervals for the response at a particular time. The less familiar issue of constructing confidence bands for the path of responses is considered. The feasibility of using heuristic optimisation methods for constructing bootstrap confidence bands is investigated. The coverage properties of bands constructed in this way are evaluated for a stylised empirical vector error correction model.

Introduction

It is routine to carry out impulse response analysis when a vector autoregressive model (VAR) is fitted. There is a considerable literature on how best to construct confidence intervals for the responses, that is for the response at a particular time. By contrast, the construction of confidence bands for the entire response path has received very little attention although it is a subject of considerable interest. Confidence intervals for responses are often plotted against time with the highest points in successive intervals and the lowest points joined so that the reader is almost invited to interpret the area between the two lines as a confidence band for the entire path. This is a rather hazardous proceeding for a coverage of 95%, say, for each of the intervals will not necessarily produce anything like a 95% coverage for the band. The present paper takes off from the belief that there should be some gain in taking a more considered approach than joining the intervals and hoping for the best. The methods proposed combine bootstrapping, which is now widely used for the construction of intervals, with the use of heuristic optimisation methods for finding the narrowest band for the entire path. The approach, which is a natural extension of bootstrapped confidence intervals, appears not to have been tried before although Sims and Zha (1999) have treated band construction from a likelihood/Bayesian viewpoint.

The paper begins with an account of the relationship between confidence intervals and confidence bands, continues with an exposition of different ways of performing the basic bootstrap exercise and then describes alternative methods of constructing bands based on the bootstrap. These alternative methods employ different optimisation techniques. The procedures for producing confidence bands are then evaluated for a simple stationary data generating process (DGP) and for a vector error correction model (VECM) of the kind used in empirical work. It is found that the coverage probabilities obtained for the ad hoc procedures of joining up the confidence intervals are substantially less than the desired value while the proposed methods for constructing confidence bands perform well for short horizons, at least, and always achieve much higher coverage probabilities than the joined up intervals.

Section snippets

Confidence intervals and confidence bands

We begin by noting the inconclusiveness of reasoning from the properties of intervals for particular responses to the properties of bands for the entire response path. The main point is that knowing the confidence levels for the intervals for each period may tell us little about the confidence level of the band produced by joining up the intervals.

The standard result linking joint probabilities to marginal probabilities is Bonferroni's inequality; see Miller (1980). This states that for any

The bootstrap procedure

The objective of impulse response analysis is to discover how the VAR responds to typical economic shocks. This objective has been interpreted in different ways, corresponding to different views of what is economically reasonable and intelligible; see Aldrich and Staszewska (2007) for discussion. Three distinct forms of economic experiment have been proposed leading to traditional, generalised and orthogonalised impulse response analysis; the terms are from Koop et al. (1996). The confidence

Constructing confidence bands

The basic method for constructing a 95% confidence interval for the response for a given period is to order the B bootstrap values for that period and identify the top and bottom 2.5 percentiles. In the case of paths there is no underlying ordering because the paths typically cross. The guiding idea of discarding extreme paths can be retained but the implementation has to be changed; in fact, the idea can be implemented in several different ways.

Formal optimisation methods can be used. There

Preliminaries: a stationary DGP

The main evaluation of the methods is presented in Sections 6 and 7 but it is interesting to consider their performance when applied to a simpler case, a bivariate stationary process of the kind considered by Kilian (1998). Kilian found that his method of interval construction was superior to the standard method. We find that the band methods based on his bias correction procedure also perform quite well.

Following Kilian, the data generating process for the Monte Carlo experiments is: $y_{t} = [\begin{matrix} 0.5 & 0 \\ 0.5 \end{matrix}]$

A nonstationary DGP with cointegration

Our main Monte Carlo investigation is based on a model which is widely used in macroeconometrics, the conditional VEC model: $Δ y_{t} = α (β^{'} [\begin{matrix} y_{t - 1} \\ x_{t - 1} \end{matrix}]) + \sum_{i = 1}^{s} [\begin{matrix} Γ_{yi} & Γ_{xi} \end{matrix}] [\begin{matrix} Δ y_{t - i} \\ Δ x_{t - i} \end{matrix}] + π Δ x_{t} + δ D_{t} + ε_{t},$ where $y_{t}$ is the vector of endogenous variables, $x_{t}$ is the vector of weakly exogenous or conditioning variables and $D_{t}$ is a vector of deterministic terms. The matrix $α$ contains the adjustment coefficients and the matrix $β$ contains the long-run parameters.

The version of this model we use as the data generating process for

Results and evaluation

This section reports the results of the main Monte Carlo experiments investigating coverage probabilities and also the results from some subsidiary studies. Table 5 shows the performance of the various methods for constructing confidence bands for 24 periods ahead; the economic experiment is the same as that behind Figs. 1–3, viz. the traditional impulse response analysis considering the effects on the three endogenous variables of a one standard deviation shock to $w$ .

While the probabilities are

Conclusions

Our main finding is that it is possible to produce confidence bands which are much more successful in achieving a desired confidence level than the technique of joining up confidence intervals. This conclusion is likely to hold in other circumstances although the detailed findings are subject to the important caveat that they relate to a restricted set of experiments using a particular DGP. Of the methods considered the ones using the Kilian bias correction performed best. The performance

Acknowledgements

Support from the EU Commission through MRTN-CT-2006-034270 is gratefully acknowledged. The author would like to thank the editors and the referees for their comments and suggestions.

References (13)

G. Koop et al.
Impulse response analysis in nonlinear multivariate models
J. Econometrics
(1996)
Aldrich, J., Staszewska, A., 2007. The experiment in macroeconometrics. J. Econom. Methodol.,...
A. Benkwitz et al.
Comparison of bootstrap confidence intervals for impulse responses of German monetary systems
Macroecon. Dyn.
(2001)
B.S. Bernanke et al.
Inside the black box: the credit channel of monetary policy transmission
J. Econom. Perspect.
(1995)
Carroll, D.L., 2001. GA-version 1.7a: FORTRAN Genetic Algorithm Driver. CU Aerospace, Urbana, IL....
B. Efron et al.
An Introduction to the Bootstrap
(1993)

There are more references available in the full text version of this article.

Cited by (18)

Trading on short-term path forecasts of intraday electricity prices
2022, Energy Economics
We introduce a profitable trading strategy that can support decision-making in continuous intraday markets for electricity. It utilizes a novel forecasting framework, which generates prediction bands from a pool of path forecasts or approximates them using probabilistic price forecasts. The prediction bands then define a time-dependent price level that, when exceeded, indicates a good trading opportunity. Results for the German intraday market show that, in terms of the energy score, our path forecasts beat two well performing literature benchmarks by over 30%. Moreover, the forecasts provide empirical evidence that the increased computational burden induced by generating realistic price paths is offset by higher trading profits. Still, the proposed approximate and bootstrap-based methods offer a reasonable trade-off — they do not require generating path forecasts and yield only slightly lower profits.
Constructing joint confidence bands for impulse response functions of VAR models – A review
2020, Econometrics and Statistics
Methods for constructing joint confidence bands for impulse response functions which are commonly used in vector autoregressive analysis are reviewed. While considering separate intervals for each horizon individually still seems to be the most common approach, a substantial number of methods have been proposed for making joint inferences about the complete impulse response paths up to a given horizon. A structured presentation of these methods is provided. Furthermore, existing evidence on the small-sample performance of the methods is gathered. The collected information can help practitioners to decide on a suitable confidence band for a structural VAR analysis.
Constructing minimum-width confidence bands
2016, Economics Letters
Citation Excerpt :
The main disadvantage of this approach is its computational complexity for problem sizes typically encountered in real applications. To reduce complexity (Staszewska, 2007) suggests the neighboring pathwise method. She applies the method to construct VAR prediction bands (Staszewska-Bystrova, 2011).
We construct minimum-width confidence bands with a predefined nominal coverage from a finite set of sample paths. In contrast to the several heuristics suggested in the literature, our mixed-integer optimization algorithm calculates global minimum-width pathwise confidence bands. For very large problems, our solution can be used as an approximation with a bounded and known gap to the globally optimal solution.
A prediction interval for a function-valued forecast model: Application to load forecasting
2016, International Journal of Forecasting
Starting from the information contained in the shape of the load curves, we propose a flexible nonparametric function-valued forecast model called KWF (Kernel + Wavelet + Functional) that is well suited to the handling of nonstationary series. The predictor can be seen as a weighted average of the futures of past situations, where the weights increase with the similarity between the past situations and the actual one. In addition, this strategy also provides simultaneous predictions at multiple horizons. These weights induce a probability distribution that can be used to produce bootstrap pseudo predictions. Prediction intervals are then constructed after obtaining the corresponding bootstrap pseudo prediction residuals. We develop two propositions following the KWF strategy directly, and compare it to two alternative methods that arise from proposals by econometricians. The latter involve the construction of simultaneous prediction intervals using multiple comparison corrections through the control of the family-wise error (FWE) or the false discovery rate. Alternatively, such prediction intervals can be constructed by bootstrapping joint probability regions. In this work, we propose to obtain prediction intervals for the KWF model that are valid simultaneously for the $H$ prediction horizons that correspond to the relevant path forecasts, making a connection between functional time series and the econometricians’ framework.
Comparison of methods for constructing joint confidence bands for impulse response functions
2015, International Journal of Forecasting
Citation Excerpt :
Moreover, the bands adapted from the ideas of Wolf and Wunderli (2012) in the context of multiple horizon forecasting have satisfactory coverage levels but may be even wider than the very conservative traditional Bonferroni bands in some situations. Finally, the NP bands proposed by Staszewska (2007) have comparable coverage and width characteristics to the adjusted Bonferroni bands in our simulations. Since impulse responses are conditional forecasts, the adjusted Bonferroni method can also be used for constructing prediction bands for multiple horizon forecasts.
In vector autoregressive analyses, confidence intervals for individual impulse responses are typically reported in order to indicate the sampling uncertainty in the estimation results. Various methods are reviewed, and a new method for the construction of joint confidence bands, given a prespecified coverage level, for the impulse responses at all horizons considered simultaneously, is proposed. The methods are compared in a simulation experiment, and recommendations for empirical work are provided.
Constructing narrowest pathwise bootstrap prediction bands using threshold accepting
2013, International Journal of Forecasting
Citation Excerpt :
The naïve pointwise band (NPB) is created by joining the prediction intervals constructed for individual period forecasts. The pathwise procedures involve methods proposed by Jordà and Marcellino (2010) based on asymptotic arguments, and labeled by them as the Scheffé (ASB) and Bonferroni (ABB) methods, as well as the bootstrap neighbouring paths method (NPM) proposed by Staszewska (2007) in the context of impulse response analysis and used by Staszewska-Bystrova (2011) for creating VAR prediction bands. Instead of relying on simplified approximations such as pointwise or asymptotic theory based methods, we deal with the high computational complexity of constructing narrowest prediction bands by applying a heuristic optimization algorithm.
Typically, prediction bands for path-forecasts are constructed pointwise, while inference relates to the whole forecasted path. In general, no closed form analytical solution is available for pathwise bands in finite samples. We consider a direct construction approach based on bootstrapped prediction bands. The resulting highly complex optimization problem is tackled using the local search heuristic of threshold accepting. A comparison with pointwise and asymptotic bands is provided, demonstrating superior properties of the proposed bands in small samples. Finally, a real application shows the practical implications of using an appropriate tool for generating the prediction bands.

View all citing articles on Scopus

View full text

Representing uncertainty about response paths: The use of heuristic optimisation methods

Abstract

Introduction

Section snippets

Confidence intervals and confidence bands

The bootstrap procedure

Constructing confidence bands

Preliminaries: a stationary DGP

A nonstationary DGP with cointegration

Results and evaluation

Conclusions

Acknowledgements

J. Econometrics

Comparison of bootstrap confidence intervals for impulse responses of German monetary systems

Macroecon. Dyn.

Inside the black box: the credit channel of monetary policy transmission

J. Econom. Perspect.

An Introduction to the Bootstrap