  In this work we focus on fine-tuning a pre-trained BERT model and applying it
to patent classification. When applied to large datasets of over two millions
patents, our approach outperforms the state of the art by an approach using CNN
with word embeddings. In addition, we focus on patent claims without other
parts in patent documents. Our contributions include: (1) a new
state-of-the-art method based on pre-trained BERT model and fine-tuning for
patent classification, (2) a large dataset USPTO-3M at the CPC subclass level
with SQL statements that can be used by future researchers, (3) showing that
patent claims alone are sufficient for classification task, in contrast to
conventional wisdom.
