  Explainable Artificial Intelligence (XAI) research gained prominence in
recent years in response to the demand for greater transparency and trust in AI
from the user communities. This is especially critical because AI is adopted in
sensitive fields such as finance, medicine etc., where implications for
society, ethics, and safety are immense. Following thorough systematic
evaluations, work in XAI has primarily focused on Machine Learning (ML) for
categorization, decision, or action. To the best of our knowledge, no work is
reported that offers an Explainable Reinforcement Learning (XRL) method for
trading financial stocks. In this paper, we proposed to employ SHapley Additive
exPlanation (SHAP) on a popular deep reinforcement learning architecture viz.,
deep Q network (DQN) to explain an action of an agent at a given instance in
financial stock trading. To demonstrate the effectiveness of our method, we
tested it on two popular datasets namely, SENSEX and DJIA, and reported the
results.
