  The evaluation of object detection models is usually performed by optimizing
a single metric, e.g. mAP, on a fixed set of datasets, e.g. Microsoft COCO and
Pascal VOC. Due to image retrieval and annotation costs, these datasets consist
largely of images found on the web and do not represent many real-life domains
that are being modelled in practice, e.g. satellite, microscopic and gaming,
making it difficult to assert the degree of generalization learned by the
model. We introduce the Roboflow-100 (RF100) consisting of 100 datasets, 7
imagery domains, 224,714 images, and 805 class labels with over 11,170
labelling hours. We derived RF100 from over 90,000 public datasets, 60 million
public images that are actively being assembled and labelled by computer vision
practitioners in the open on the web application Roboflow Universe. By
releasing RF100, we aim to provide a semantically diverse, multi-domain
benchmark of datasets to help researchers test their model's generalizability
with real-life data. RF100 download and benchmark replication are available on
GitHub.
