  We introduce EvalAI, an open source platform for evaluating and comparing
machine learning (ML) and artificial intelligence algorithms (AI) at scale.
EvalAI is built to provide a scalable solution to the research community to
fulfill the critical need of evaluating machine learning models and agents
acting in an environment against annotations or with a human-in-the-loop. This
will help researchers, students, and data scientists to create, collaborate,
and participate in AI challenges organized around the globe. By simplifying and
standardizing the process of benchmarking these models, EvalAI seeks to lower
the barrier to entry for participating in the global scientific effort to push
the frontiers of machine learning and artificial intelligence, thereby
increasing the rate of measurable progress in this domain.
