  Twitter, a popular social network, presents great opportunities for on-line
machine learning research. However, previous research has focused almost
entirely on learning from passively collected data. We study the problem of
learning to acquire followers through normative user behavior, as opposed to
the mass following policies applied by many bots. We formalize the problem as a
contextual bandit problem, in which we consider retweeting content to be the
action chosen and each tweet (content) is accompanied by context. We design
reward signals based on the change in followers. The result of our month long
experiment with 60 agents suggests that (1) aggregating experience across
agents can adversely impact prediction accuracy and (2) the Twitter community's
response to different actions is non-stationary. Our findings suggest that
actively learning on-line can provide deeper insights about how to attract
followers than machine learning over passively collected data alone.
