  Social media personalization algorithms increasingly influence the flow of
civic information through society, resulting in concerns about "filter
bubbles", "echo chambers", and other ways they might exacerbate ideological
segregation and fan the spread of polarizing content. To address these
concerns, we designed and conducted a sociotechnical audit (STA) to investigate
how Twitter/X's timeline algorithm affects news curation while also tracking
how user perceptions change in response. We deployed a custom-built system
that, over the course of three weeks, passively tracked all tweets loaded in
users' browsers in the first week, then in the second week enacted an
intervention to users' Twitter/X homepage to restrict their view to only the
algorithmic or chronological timeline (randomized). We flipped this condition
for each user in the third week. We ran our audit in late 2023, collecting
user-centered metrics (self-reported survey measures) and platform-centered
metrics (views, clicks, likes) for 243 users, along with over 800,000 tweets.
Using the STA framework, our results are two-fold: (1) Our algorithm audit
finds that Twitter/X's algorithmic timeline resulted in a lower quantity but
higher quality of news -- less ideologically congruent, less extreme, and
slightly more reliable -- compared to the chronological timeline. (2) Our user
audit suggests that although our timeline intervention had significant effects
on users' behaviors, it had little impact on their overall perceptions of the
platform. Our paper discusses these findings and their broader implications in
the context of algorithmic news curation, user-centric audits, and avenues for
independent social science research.
