  Time series forecasting is a crucial component of many important
applications, ranging from forecasting the stock markets to energy load
prediction. The high-dimensionality, velocity and variety of the data collected
in these applications pose significant and unique challenges that must be
carefully addressed for each of them. In this work, a novel Temporal Logistic
Neural Bag-of-Features approach, that can be used to tackle these challenges,
is proposed. The proposed method can be effectively combined with deep neural
networks, leading to powerful deep learning models for time series analysis.
However, combining existing BoF formulations with deep feature extractors pose
significant challenges: the distribution of the input features is not
stationary, tuning the hyper-parameters of the model can be especially
difficult and the normalizations involved in the BoF model can cause
significant instabilities during the training process. The proposed method is
capable of overcoming these limitations by a employing a novel adaptive scaling
mechanism and replacing the classical Gaussian-based density estimation
involved in the regular BoF model with a logistic kernel. The effectiveness of
the proposed approach is demonstrated using extensive experiments on a
large-scale financial time series dataset that consists of more than 4 million
limit orders.
