  Ranking is at the core of Information Retrieval.
  Classic ranking optimization studies often treat ranking as a sorting problem
with the assumption that the best performance of ranking would be achieved if
we rank items according to their individual utility.
  Accordingly, considerable ranking metrics have been developed and
learning-to-rank algorithms that have been designed to optimize these simple
performance metrics have been widely used in modern IR systems.
  As applications evolve, however, people's need for information retrieval have
shifted from simply retrieving relevant documents to more advanced information
services that satisfy their complex working and entertainment needs.
  Thus, more complicated and user-centric objectives such as user satisfaction
and engagement have been adopted to evaluate modern IR systems today.
  Those objectives, unfortunately, are difficult to be optimized under existing
learning-to-rank frameworks as they are subject to great variance and
complicated structures that cannot be explicitly explained or formulated with
math equations like those simple performance metrics.
  This leads to the following research question -- how to optimize result
ranking for complex ranking metrics without knowing their internal structures?
  To address this question, we conduct formal analysis on the limitation of
existing ranking optimization techniques and describe three research tasks in
\textit{Metric-agnostic Ranking Optimization}.
  Through the discussion of potential solutions to these tasks, we hope to
encourage more people to look into the problem of ranking optimization in
complex search and recommendation scenarios.
