  Hashtag recommendation is a crucial task, especially with an increase of
interest in using social media platforms such as Twitter in the last decade.
Hashtag recommendation systems automatically suggest hashtags to a user while
writing a tweet. Most of the research in the area of hashtag recommendation
have used classical metrics such as hit rate, precision, recall, and F1-score
to measure the accuracy of hashtag recommendation systems. These metrics are
based on the exact match of the recommended hashtags with their corresponding
ground truth. However, it is not clear how adequate these metrics to evaluate
hashtag recommendation. The research question that we are interested in seeking
an answer is: are these metrics adequate for evaluating hashtag recommendation
systems when the numbers of ground truth hashtags in tweets are highly
variable? In this paper, we propose a new metric which we call hit ratio for
hashtag recommendation. Extensive evaluation through hypothetical examples and
real-world application across a range of hashtag recommendation models indicate
that the hit ratio is a useful metric. A comparison of hit ratio with the
classical evaluation metrics reveals their limitations.
