No single winner: Forecast combination versus neural and tree-based learners on short monthly time series

Abstract

Machine learning (ML) methods are increasingly applied to business and environmental forecasting, yet many practical series are short and dominated by trend and seasonality, conditions under which classical statistical methods have performed strongly in large forecasting competitions. This study compares nine forecasting methods on six public monthly time series of 108–240 observations, covering airline traffic, retail sales, crime, climate and solar activity. The methods comprise naive and seasonal naive benchmarks, exponential smoothing (ETS), the Theta method, a seasonal ARIMA model, an equal-weight combination of the three statistical models (Comb), a multilayer perceptron (MLP), an MLP applied to seasonally adjusted data (MLP-DS) and gradient boosting (GB). Each method produced 12-month forecasts from 25 rolling origins per series, giving 1,350 multi-step forecasts evaluated with the mean absolute scaled error (MASE) and the symmetric mean absolute percentage error. The combination achieved the lowest average MASE (1.076), the best mean rank (3.75 of 9) and the smallest worst-case regret (1.42 times the best method on any series), and it improved on the seasonal naive benchmark at 74.7% of forecast origins. The ML methods were markedly heterogeneous: they were the most accurate family on the airline series (MLP-DS, MASE 0.562) and on the non-seasonal sunspot series (GB, 0.837), but among the least accurate on the car sales and robberies series, where their MASE exceeded that of the combination by 0.46–0.98. Seasonal adjustment improved the MLP on four of the six series. Differences between methods were not significant at the series level (Friedman p = 0.113), reflecting the small number of series. For short monthly series, a simple combination of statistical models is therefore the most reliable default, whereas ML methods should be adopted only after series-specific validation.