  To track the target in a video, current visual trackers usually adopt greedy
search for target object localization in each frame, that is, the candidate
region with the maximum response score will be selected as the tracking result
of each frame. However, we found that this may be not an optimal choice,
especially when encountering challenging tracking scenarios such as heavy
occlusion and fast motion. To address this issue, we propose to maintain
multiple tracking trajectories and apply beam search strategy for visual
tracking, so that the trajectory with fewer accumulated errors can be
identified. Accordingly, this paper introduces a novel multi-agent
reinforcement learning based beam search tracking strategy, termed
BeamTracking. It is mainly inspired by the image captioning task, which takes
an image as input and generates diverse descriptions using beam search
algorithm. Accordingly, we formulate the tracking as a sample selection problem
fulfilled by multiple parallel decision-making processes, each of which aims at
picking out one sample as their tracking result in each frame. Each maintained
trajectory is associated with an agent to perform the decision-making and
determine what actions should be taken to update related information. When all
the frames are processed, we select the trajectory with the maximum accumulated
score as the tracking result. Extensive experiments on seven popular tracking
benchmark datasets validated the effectiveness of the proposed algorithm.
