  Empirical game-theoretic analysis (EGTA) has recently been applied
successfully to analyze the behavior of large numbers of competing traders in a
continuous double auction market. Multiagent simulation methods like EGTA are
useful for studying complex strategic environments like a stock market, where
it is not feasible to solve analytically for the rational behavior of each
agent. A weakness of simulation-based methods in strategic settings, however,
is that it is typically impossible to prove that the strategy profile assigned
to the simulated agents is stable, as in a Nash equilibrium. I propose using
reinforcement learning to analyze the regret of supposed Nash-equilibrium
strategy profiles found by EGTA. I have developed a new library of
reinforcement learning tools, which I have integrated into an extended version
of the market simulator from our prior work. I provide evidence for the
effectiveness of our library methods, both on a suite of benchmark problems
from the literature, and on non-equilibrium strategy profiles in our market
environment. Finally, I use our new reinforcement learning tools to provide
evidence that the equilibria found by EGTA in our recent continuous double
auction study are likely to have only negligible regret, even with respect to
an extended strategy space.
