  Video indexing approaches such as visual concept classification and person
recognition are essential to enable fine-grained semantic search in large-scale
video archives such as the historical video collection of former German
Democratic Republic (GDR) maintained by the German Broadcasting Archive (DRA).
Typically, a lexicon of visual concepts has to be defined for semantic search.
However, the definition of visual concepts can be more or less subjective due
to individually differing judgments of annotators, which may have an impact on
annotation quality and subsequently training of supervised machine learning
methods. In this paper, we analyze the inter-coder agreement for historical TV
data of the former GDR for visual concept classification and person
recognition. The inter-coder agreement is evaluated for a group of expert as
well as non-expert annotators in order to determine differences in annotation
homogeneity. Furthermore, correlations between visual recognition performance
and inter-annotator agreement are measured. In this context, information about
image quantity and agreement are used to predict average precision for concept
classification. Finally, the influence of expert vs. non-expert annotations
acquired in the study are used to evaluate person recognition.
