  In this letter, we address the problem of controlling energy storage systems
(ESSs) for arbitrage in real-time electricity markets under price uncertainty.
We first formulate this problem as a Markov decision process, and then develop
a deep reinforcement learning based algorithm to learn a stochastic control
policy that maps a set of available information processed by a recurrent neural
network to ESSs' charging/discharging actions. Finally, we verify the
effectiveness of our algorithm using real-time electricity prices from PJM.
