  For more than a decade Vytelingum's Adaptive-Aggressive (AA) algorithm has
been recognized as the best-performing automated auction-market trading-agent
strategy currently known in the AI/Agents literature; in this paper, we
demonstrate that it is in fact routinely outperformed by another algorithm when
exhaustively tested across a sufficiently wide range of market scenarios. The
novel step taken here is to use large-scale compute facilities to brute-force
exhaustively evaluate AA in a variety of market environments based on those
used for testing it in the original publications. Our results show that even in
these simple environments AA is consistently out-performed by IBM's GDX
algorithm, first published in 2002. We summarize here results from more than
one million market simulation experiments, orders of magnitude more testing
than was reported in the original publications that first introduced AA. A 2019
ICAART paper by Cliff claimed that AA's failings were revealed by testing it in
more realistic experiments, with conditions closer to those found in real
financial markets, but here we demonstrate that even in the simple experiment
conditions that were used in the original AA papers, exhaustive testing shows
AA to be outperformed by GDX. We close this paper with a discussion of the
methodological implications of our work: any results from previous papers where
any one trading algorithm is claimed to be superior to others on the basis of
only a few thousand trials are probably best treated with some suspicion now.
The rise of cloud computing means that the compute-power necessary to subject
trading algorithms to millions of trials over a wide range of conditions is
readily available at reasonable cost: we should make use of this; exhaustive
testing such as is shown here should be the norm in future evaluations and
comparisons of new trading algorithms.
