  This work seeks to answer key research questions regarding the viability of
reinforcement learning over the S&P 500 index. The on-policy techniques of
Value Iteration (VI) and State-action-reward-state-action (SARSA) are
implemented along with the off-policy technique of Q-Learning. The models are
trained and tested on a dataset comprising multiple years of stock market data
from 2000-2023. The analysis presents the results and findings from training
and testing the models using two different time periods: one including the
COVID-19 pandemic years and one excluding them. The results indicate that
including market data from the COVID-19 period in the training dataset leads to
superior performance compared to the baseline strategies. During testing, the
on-policy approaches (VI and SARSA) outperform Q-learning, highlighting the
influence of bias-variance tradeoff and the generalization capabilities of
simpler policies. However, it is noted that the performance of Q-learning may
vary depending on the stability of future market conditions. Future work is
suggested, including experiments with updated Q-learning policies during
testing and trading diverse individual stocks. Additionally, the exploration of
alternative economic indicators for training the models is proposed.
