  This paper is concerned with evaluating different multiagent learning (MAL)
algorithms in problems where individual agents may be heterogenous, in the
sense of utilizing different learning strategies, without the opportunity for
prior agreements or information regarding coordination. Such a situation arises
in ad hoc team problems, a model of many practical multiagent systems
applications. Prior work in multiagent learning has often been focussed on
homogeneous groups of agents, meaning that all agents were identical and a
priori aware of this fact. Also, those algorithms that are specifically
designed for ad hoc team problems are typically evaluated in teams of agents
with fixed behaviours, as opposed to agents which are adapting their
behaviours. In this work, we empirically evaluate five MAL algorithms,
representing major approaches to multiagent learning but originally developed
with the homogeneous setting in mind, to understand their behaviour in a set of
ad hoc team problems. All teams consist of agents which are continuously
adapting their behaviours. The algorithms are evaluated with respect to a
comprehensive characterisation of repeated matrix games, using performance
criteria that include considerations such as attainment of equilibrium, social
welfare and fairness. Our main conclusion is that there is no clear winner.
However, the comparative evaluation also highlights the relative strengths of
different algorithms with respect to the type of performance criteria, e.g.,
social welfare vs. attainment of equilibrium.
