  The article describes the use of deep Q-learning models in the problems of
sales time series analytics. In contrast to supervised machine learning which
is a kind of passive learning using historical data, Q-learning is a kind of
active learning with goal to maximize a reward by optimal sequence of actions.
Model free Q-learning approach for optimal pricing strategies and supply-demand
problems was considered in the work. The main idea of the study is to show that
using deep Q-learning approach in time series analytics, the sequence of
actions can be optimized by maximizing the reward function when the environment
for learning agent interaction can be modeled using the parametric model and in
the case of using the model which is based on the historical data. In the
pricing optimizing case study environment was modeled using sales dependence on
extras price and randomly simulated demand. In the pricing optimizing case
study, the environment was modeled using sales dependence on extra price and
randomly simulated demand. In the supply-demand case study, it was proposed to
use historical demand time series for environment modeling, agent states were
represented by promo actions, previous demand values and weekly seasonality
features. Obtained results show that using deep Q-learning, we can optimize the
decision making process for price optimization and supply-demand problems.
Environment modeling using parametric models and historical data can be used
for the cold start of learning agent. On the next steps, after the cold start,
the trained agent can be used in real business environment.
