  Automatic food recognition is the very first step towards passive dietary
monitoring. In this paper, we address the problem of food recognition by mining
discriminative food regions. Taking inspiration from Adversarial Erasing, a
strategy that progressively discovers discriminative object regions for weakly
supervised semantic segmentation, we propose a novel network architecture in
which a primary network maintains the base accuracy of classifying an input
image, an auxiliary network adversarially mines discriminative food regions,
and a region network classifies the resulting mined regions. The global (the
original input image) and the local (the mined regions) representations are
then integrated for the final prediction. The proposed architecture denoted as
PAR-Net is end-to-end trainable, and highlights discriminative regions in an
online fashion. In addition, we introduce a new fine-grained food dataset named
as Sushi-50, which consists of 50 different sushi categories. Extensive
experiments have been conducted to evaluate the proposed approach. On three
food datasets chosen (Food-101, Vireo-172, and Sushi-50), our approach performs
consistently and achieves state-of-the-art results (top-1 testing accuracy of
$90.4\%$, $90.2\%$, $92.0\%$, respectively) compared with other existing
approaches. Dataset and code are available at
https://github.com/Jianing-Qiu/PARNet
