  In recent years, large language models(LLMs) have attracted significant
attention due to their exceptional performance across a multitude of natural
language process tasks, and have been widely applied in various fields.
However, the application of large language models in the Intellectual Property
(IP) domain is challenging due to the strong need for specialized knowledge,
privacy protection, processing of extremely long text in this field. In this
technical report, we present for the first time a low-cost, standardized
procedure for training IP-oriented LLMs, meeting the unique requirements of the
IP domain. Using this standard process, we have trained the PatentGPT series
models based on open-source pretrained models. By evaluating them on the
open-source IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms
GPT-4, indicating the effectiveness of the proposed training procedure and the
expertise of the PatentGPT models in the IP domain. Remarkably, our model
surpassed GPT-4 on the 2019 China Patent Agent Qualification Examination,
scoring 65 and matching human expert levels. Additionally, the PatentGPT model,
which utilizes the SMoE architecture, achieves performance comparable to that
of GPT-4 in the IP domain and demonstrates a better cost-performance ratio on
long-text tasks, potentially serving as an alternative to GPT-4 within the IP
domain.
