  Currently, the construction of large language models in specific domains is
done by fine-tuning on a base model. Some models also incorporate knowledge
bases without the need for pre-training. This is because the base model already
contains domain-specific knowledge during the pre-training process. We build a
large language model for food testing. Unlike the above approach, a significant
amount of data in this domain exists in Scanning format for domain standard
documents. In addition, there is a large amount of untrained structured
knowledge. Therefore, we introduce an incremental pre-training step to inject
this knowledge into a large language model. In this paper, we propose a method
for handling structured knowledge and scanned documents in incremental
pre-training. To overcome the problem of machine hallucination, we constructe a
knowledge graph to serve as an external knowledge base for supporting retrieval
in the large language model. It is worth mentioning that this paper is a
technical report of our pre-release version, and we will report our specific
experimental data in future versions.
