  Background: Comparative effectiveness research using network meta-analysis
can present a hierarchy of competing treatments, from the least to most
preferable option. However, the research question associated with the hierarchy
of multiple interventions is never clearly defined in published reviews.
Methods and Results: We introduce the notion of a treatment hierarchy question
that describes the criterion for choosing a specific treatment over one or more
competing alternatives. For example, stakeholders might ask which treatment is
most likely to improve mean survival by at least 2 years or which treatment is
associated with the longest mean survival. The answers to these two questions
are not necessarily the same. We discuss the most commonly used ranking metrics
(quantities that describe or compare the estimated treatment-specific effects),
how the metrics produce a treatment hierarchy and the type of treatment
hierarchy question that each metric can answer. We show that the ranking
metrics encompass the uncertainty in the estimation of the treatment effects in
different ways, which results in different treatment hierarchies. Conclusions:
Network meta-analyses that aim to rank treatments should state in the protocol
the treatment hierarchy question they aim to address and employ the appropriate
ranking metric to answer it.
