  To understand how well a proposed augmented reality (AR) solution works,
existing papers often conducted tailored and isolated evaluations for specific
AR tasks, e.g., depth or lighting estimation, and compared them to
easy-to-setup baselines, either using datasets or resorting to time-consuming
data capturing. Conceptually simple, it can be extremely difficult to evaluate
an AR system fairly and in scale to understand its real-world performance. The
difficulties arise for three key reasons: lack of control of the physical
environment, the time-consuming data capturing, and the difficulties to
reproduce baseline results.
  This paper presents our design of an AR experimentation platform, ExpAR,
aiming to provide scalable and controllable AR experimentation. ExpAR is
envisioned to operate as a standalone deployment or a federated platform; in
the latter case, AR researchers can contribute physical resources, including
scene setup and capturing devices, and allow others to time share these
resources. Our design centers around the generic
sensing-understanding-rendering pipeline and is driven by the evaluation
limitations observed in recent AR systems papers. We demonstrate the
feasibility of this vision with a preliminary prototype and our preliminary
evaluations suggest the importance of further investigating different device
capabilities to stream in 30 FPS.
  The ExpAR project site can be found at https://cake.wpi.edu/expar.
