  We formulate counting as a sequential decision problem and present a novel
crowd counting model solvable by deep reinforcement learning. In contrast to
existing counting models that directly output count values, we divide one-step
estimation into a sequence of much easier and more tractable sub-decision
problems. Such sequential decision nature corresponds exactly to a physical
process in reality scale weighing. Inspired by scale weighing, we propose a
novel 'counting scale' termed LibraNet where the count value is analogized by
weight. By virtually placing a crowd image on one side of a scale, LibraNet
(agent) sequentially learns to place appropriate weights on the other side to
match the crowd count. At each step, LibraNet chooses one weight (action) from
the weight box (the pre-defined action pool) according to the current crowd
image features and weights placed on the scale pan (state). LibraNet is
required to learn to balance the scale according to the feedback of the needle
(Q values). We show that LibraNet exactly implements scale weighing by
visualizing the decision process how LibraNet chooses actions. Extensive
experiments demonstrate the effectiveness of our design choices and report
state-of-the-art results on a few crowd counting benchmarks. We also
demonstrate good cross-dataset generalization of LibraNet. Code and models are
made available at: https://git.io/libranet
