  The effectiveness of binary analysis tools and techniques is often measured
with respect to how well they map to a ground truth. We have found that not all
ground truths are created equal. This paper challenges the binary analysis
community to take a long look at the concept of ground truth, to ensure that we
are in agreement with definition(s) of ground truth, so that we can be
confident in the evaluation of tools and techniques. This becomes even more
important as we move to trained machine learning models, which are only as
useful as the validity of the ground truth in the training.
