  Learning from the web can ease the extreme dependence of deep learning on
large-scale manually labeled datasets. Especially for fine-grained recognition,
which targets at distinguishing subordinate categories, it will significantly
reduce the labeling costs by leveraging free web data. Despite its significant
practical and research value, the webly supervised fine-grained recognition
problem is not extensively studied in the computer vision community, largely
due to the lack of high-quality datasets. To fill this gap, in this paper we
construct two new benchmark webly supervised fine-grained datasets, termed
WebFG-496 and WebiNat-5089, respectively. In concretely, WebFG-496 consists of
three sub-datasets containing a total of 53,339 web training images with 200
species of birds (Web-bird), 100 types of aircrafts (Web-aircraft), and 196
models of cars (Web-car). For WebiNat-5089, it contains 5089 sub-categories and
more than 1.1 million web training images, which is the largest webly
supervised fine-grained dataset ever. As a minor contribution, we also propose
a novel webly supervised method (termed "{Peer-learning}") for benchmarking
these datasets.~Comprehensive experimental results and analyses on two new
benchmark datasets demonstrate that the proposed method achieves superior
performance over the competing baseline models and states-of-the-art. Our
benchmark datasets and the source codes of Peer-learning have been made
available at
{\url{https://github.com/NUST-Machine-Intelligence-Laboratory/weblyFG-dataset}}.
