  People are often entities of interest in tasks such as search and information
extraction. In these tasks, the goal is to find as much information as possible
about people specified by their name. However in text, some of the references
to people are by pronouns (she, his) or generic descriptions (the professor,
the German chancellor). It is therefore important that coreference resolution
systems are able to link these different types of mentions to the correct
person name. Here, we evaluate two state of the art coreference resolution
systems on the subtask of Named Person Coreference, in which we are interested
in identifying a person mentioned by name, along with all other mentions of the
person, by pronoun or generic noun phrase. Our analysis reveals that standard
coreference metrics do not reflect adequately the requirements in this task:
they do not penalize systems for not identifying any mentions by name and they
reward systems even if systems find correctly mentions to the same entity but
fail to link these to a proper name (she--the student---no name). We introduce
new metrics for evaluating named person coreference that address these
discrepancies. We present a simple rule-based named entity recognition driven
system, which outperforms the current state-of-the-art systems on these
task-specific metrics and performs on par with them on traditional coreference
evaluations. Finally, we present similar evaluation for coreference resolution
of other named entities and show that the rule-based approach is effective only
for person named coreference, not other named entity types.
