  Millions of people around the world are sharing COVID-19 related information
on social media platforms. Since not all the information shared on the social
media is useful, a machine learning system to identify informative posts can
help users in finding relevant information. In this paper, we present a BERT
classifier system for W-NUT2020 Shared Task 2: Identification of Informative
COVID-19 English Tweets. Further, we show that BERT exploits some easy signals
to identify informative tweets, and adding simple patterns to uninformative
tweets drastically degrades BERT performance. In particular, simply adding 10
deaths to tweets in dev set, reduces BERT F1- score from 92.63 to 7.28. We also
propose a simple data augmentation technique that helps in improving the
robustness and generalization ability of the BERT classifier.
