  Access to word-sentiment associations is useful for many applications,
including sentiment analysis, stance detection, and linguistic analysis.
However, manually assigning fine-grained sentiment association scores to words
has many challenges with respect to keeping annotations consistent. We apply
the annotation technique of Best-Worst Scaling to obtain real-valued sentiment
association scores for words and phrases in three different domains: general
English, English Twitter, and Arabic Twitter. We show that on all three domains
the ranking of words by sentiment remains remarkably consistent even when the
annotation process is repeated with a different set of annotators. We also, for
the first time, determine the minimum difference in sentiment association that
is perceptible to native speakers of a language.
