  We present a crowdsourcing workflow to collect image annotations for visually
similar synthetic categories without requiring experts. In animals, there is a
direct link between taxonomy and visual similarity: e.g. a collie (type of dog)
looks more similar to other collies (e.g. smooth collie) than a greyhound
(another type of dog). However, in synthetic categories such as cars, objects
with similar taxonomy can have very different appearance: e.g. a 2011 Ford
F-150 Supercrew-HD looks the same as a 2011 Ford F-150 Supercrew-LL but very
different from a 2011 Ford F-150 Supercrew-SVT. We introduce a graph based
crowdsourcing algorithm to automatically group visually indistinguishable
objects together. Using our workflow, we label 712,430 images by ~1,000 Amazon
Mechanical Turk workers; resulting in the largest fine-grained visual dataset
reported to date with 2,657 categories of cars annotated at 1/20th the cost of
hiring experts.
