  With the increased popularity of electronic textbooks, there is a growing
interests in developing a new generation of "intelligent textbooks", which have
the ability to guide the readers according to their learning goals and current
knowledge. The intelligent textbooks extend regular textbooks by integrating
machine-manipulatable knowledge such as a knowledge map or a
prerequisite-outcome relationship between sections, among which, the most
popular integrated knowledge is a list of unique knowledge concepts associated
with each section. With the help of this concept, multiple intelligent
operations, such as content linking, content recommendation or student
modeling, can be performed. However, annotating a reliable set of concepts to a
textbook section is a challenge. Automatic unsupervised methods for extracting
key-phrases as the concepts are known to have insufficient accuracy. Manual
annotation by experts is considered as a preferred approach and can be used to
produce both the target outcome and the labeled data for training supervised
models. However, most researchers in education domain still consider the
concept annotation process as an ad-hoc activity rather than an engineering
task, resulting in low-quality annotated data. In this paper, we present a
textbook knowledge engineering method to obtain reliable concept annotations.
The outcomes of our work include a validated knowledge engineering procedure, a
code-book for technical concept annotation, and a set of concept annotations
for the target textbook, which could be used as gold standard in further
research.
