  We examine the effects of different latency penalties in the evaluation of
push notification systems, as operationalized in the TREC 2015 Microblog track
evaluation. The purpose of this study is to inform the design of metrics for
the TREC 2016 Real-Time Summarization track, which is largely modeled after the
TREC 2015 evaluation design.
