  To study emotions at the macroscopic level, affective scientists have made
extensive use of sentiment analysis on social media text. However, this
approach can suffer from a series of methodological issues with respect to
sampling biases and measurement error. To date, it has not been validated if
social media sentiment can measure the day to day temporal dynamics of emotions
aggregated at the macro level of a whole online community. We ran a large-scale
survey at an online newspaper to gather daily self-reports of affective states
from its users and compare these with aggregated results of sentiment analysis
of user discussions on the same online platform. Additionally, we preregistered
a replication of our study using Twitter text as a macroscope of emotions for
the same community. For both platforms, we find strong correlations between
text analysis results and levels of self-reported emotions, as well as between
inter-day changes of both measurements. We further show that a combination of
supervised and unsupervised text analysis methods is the most accurate approach
to measure emotion aggregates. We illustrate the application of such social
media macroscopes when studying the association between the number of new
COVID-19 cases and emotions, showing that the strength of associations is
comparable when using survey data as when using social media data. Our findings
indicate that macro level dynamics of affective states of users of an online
platform can be tracked with social media text, complementing surveys when
self-reported data is not available or difficult to gather.
