  Meta-learning, decision fusion, hybrid models, and representation learning
are topics of investigation with significant traction in time-series
forecasting research. Of these two specific areas have shown state-of-the-art
results in forecasting: hybrid meta-learning models such as Exponential
Smoothing - Recurrent Neural Network (ES-RNN) and Neural Basis Expansion
Analysis (N-BEATS) and feature-based stacking ensembles such as Feature-based
FORecast Model Averaging (FFORMA). However, a unified taxonomy for model fusion
and an empirical comparison of these hybrid and feature-based stacking ensemble
approaches is still missing. This study presents a unified taxonomy
encompassing these topic areas. Furthermore, the study empirically evaluates
several model fusion approaches and a novel combination of hybrid and feature
stacking algorithms called Deep-learning FORecast Model Averaging (DeFORMA).
The taxonomy contextualises the considered methods. Furthermore, the empirical
analysis of the results shows that the proposed model, DeFORMA, can achieve
state-of-the-art results in the M4 data set. DeFORMA, increases the mean
Overall Weighted Average (OWA) in the daily, weekly and yearly subsets with
competitive results in the hourly, monthly and quarterly subsets. The taxonomy
and empirical results lead us to argue that significant progress is still to be
made by continuing to explore the intersection of these research areas.
