  There are now several test collections for the formula retrieval task, in
which a system's goal is to identify useful mathematical formulae to show in
response to a query posed as a formula. These test collections differ in query
format, query complexity, number of queries, content source, and relevance
definition. Comparisons among six formula retrieval test collections illustrate
that defining relevance based on query and/or document context can be
consequential, that system results vary markedly with formula complexity, and
that judging relevance after clustering formulas with identical symbol layouts
(i.e., Symbol Layout Trees) can affect system preference ordering.
