  While machine learning has witnessed significant advancements, the emphasis
has largely been on data acquisition and model creation. However, achieving a
comprehensive assessment of machine learning solutions in real-world settings
necessitates standardization throughout the entire pipeline. This need is
particularly acute in time series forecasting, where diverse settings impede
meaningful comparisons between various methods. To bridge this gap, we propose
a unified benchmarking framework that exposes the crucial modelling and machine
learning decisions involved in developing time series forecasting models. This
framework fosters seamless integration of models and datasets, aiding both
practitioners and researchers in their development efforts. We benchmark
recently proposed models within this framework, demonstrating that carefully
implemented deep learning models with minimal effort can rival
gradient-boosting decision trees requiring extensive feature engineering and
expert knowledge.
