  With the growing penetration of renewable energy resource, electricity market
prices have exhibited greater volatility. Therefore, it is important for Energy
Storage Systems(ESSs) to leverage the multidimensional nature of energy market
bids to maximize profitability. However, current learning methods cannot fully
utilize the high-dimensional price-quantity bids in the energy markets. To
address this challenge, we modify the common reinforcement learning(RL) process
by proposing a new bid representation method called Neural Network Embedded
Bids (NNEBs). NNEBs refer to market bids that are represented by monotonic
neural networks with discrete outputs. To achieve effective learning of NNEBs,
we first learn a neural network as a strategic mapping from the market price to
ESS power output with RL. Then, we re-train the network with two training
modifications to make the network output monotonic and discrete. Finally, the
neural network is equivalently converted into a high-dimensional bid for
bidding. We conducted experiments over real-world market datasets. Our studies
show that the proposed method achieves 18% higher profit than the baseline and
up to 78% profit of the optimal market bidder.
