Multi-horizon uniform superior predictive ability revisited: A size-exploiting and consistent test

Monschang, Verena; Trede, Mark; Wilfling, Bernd

Abstract

Quaedvlieg (2021, Journal of Business & Economic Statistics) proposes a uniform Superior Predictive  Ability (uSPA) test for comparing forecasts across multiple horizons. The procedure is based on a ’minimum Diebold-Mariano’ test statistic, and asymptotic critical values are obtained via bootstrapping. We show, theoretically and via simulations, that Quaedvlieg’s test is subject to massive size distortions. In this article, we establish several convergence results for the ’minimum Diebold-Mariano’ statistic, revealing that appropriate asymptotic critical values are given by the quantiles of the standard normal distribution. The uSPA test modified this way (i) always keeps the nominal size, (ii) is size-exploiting along the boundary that separates the parameter subsets of the null and the alternative uSPA hypotheses, and (iii) is consistent. Based on the closed skew normal distribution, we present a procedure for approximating the power function and demonstrate the favorable finite-sample properties of our test. In an empirical replication, we find that Quaedvlieg’s (2021) results on uSPA comparisons between direct and iterative forecasting methods are
statistically inconclusive.

Keywords
Forecast evaluation; Joint-hypothesis testing; Stochastic dominance; Closed skew normal distribution