  Tiny machine learning (TinyML), which enables machine learning applications
on extremely low-power devices, suffers from limited size and quality of
relevant datasets. To address this issue, we introduce Wake Vision, a
large-scale, diverse dataset tailored for person detection, the canonical task
for TinyML visual sensing. Wake Vision comprises over 6 million images,
representing a hundredfold increase compared to the previous standard, and has
undergone thorough quality filtering. We provide two Wake Vision training sets:
Wake Vision (Large) and Wake Vision (Quality), a smaller set with
higher-quality labels. Our results demonstrate that using the Wake Vision
(Quality) training set produces more accurate models than the Wake Vision
(Large) training set, strongly suggesting that label quality is more important
than quantity in our setting. We find use for the large training set for
pre-training and knowledge distillation. To minimize label errors that can
obscure true model performance, we manually label the validation and test sets,
improving the test set error rate from 7.8% in the prior standard to only 2.2%.
In addition to the dataset, we provide a collection of five detailed benchmark
sets to facilitate the evaluation of model quality in challenging real world
scenarios that are often ignored when focusing solely on overall accuracy.
These novel fine-grained benchmarks assess model performance on specific
segments of the test data, such as varying lighting conditions, distances from
the camera, and demographic characteristics of subjects. Our results
demonstrate that using Wake Vision for training results in a 2.49% increase in
accuracy compared to the established dataset. We also show the importance of
dataset quality for low-capacity models and the value of dataset size for
high-capacity models. wakevision.ai
