  This paper addresses the problem of predicting popularity of comments in an
online discussion forum using reinforcement learning, particularly addressing
two challenges that arise from having natural language state and action spaces.
First, the state representation, which characterizes the history of comments
tracked in a discussion at a particular point, is augmented to incorporate the
global context represented by discussions on world events available in an
external knowledge source. Second, a two-stage Q-learning framework is
introduced, making it feasible to search the combinatorial action space while
also accounting for redundancy among sub-actions. We experiment with five
Reddit communities, showing that the two methods improve over previous reported
results on this task.
