  Proper evaluations are crucial for better understanding, troubleshooting,
interpreting model behaviors and further improving model performance. While
using scalar-based error metrics provides a fast way to overview model
performance, they are often too abstract to display certain weak spots and lack
information regarding important model properties, such as robustness. This not
only hinders machine learning models from being more interpretable and gaining
trust, but also can be misleading to both model developers and users.
Additionally, conventional evaluation procedures often leave researchers
unclear about where and how model fails, which complicates model comparisons
and further developments. To address these issues, we propose a novel
evaluation workflow, named Non-Equivariance Revealed on Orbits (NERO)
Evaluation. The goal of NERO evaluation is to turn focus from traditional
scalar-based metrics onto evaluating and visualizing models equivariance,
closely capturing model robustness, as well as to allow researchers quickly
investigating interesting or unexpected model behaviors. NERO evaluation is
consist of a task-agnostic interactive interface and a set of visualizations,
called NERO plots, which reveals the equivariance property of the model. Case
studies on how NERO evaluation can be applied to multiple research areas,
including 2D digit recognition, object detection, particle image velocimetry
(PIV), and 3D point cloud classification, demonstrate that NERO evaluation can
quickly illustrate different model equivariance, and effectively explain model
behaviors through interactive visualizations of the model outputs. In addition,
we propose consensus, an alternative to ground truths, to be used in NERO
evaluation so that model equivariance can still be evaluated with new,
unlabeled datasets.
