  State of the art methods for target tracking with sensor management (or
controlled sensing) are model-based and are obtained through solutions to
Partially Observable Markov Decision Process (POMDP) formulations. In this
paper a Reinforcement Learning (RL) approach to the problem is explored for the
setting where the motion model for the object/target to be tracked is unknown
to the observer. It is assumed that the target dynamics are stationary in time,
the state space and the observation space are discrete, and there is complete
observability of the location of the target under certain (a priori unknown)
sensor control actions. Then, a novel Markov Decision Process (MDP) rather than
POMDP formulation is proposed for the tracking problem with controlled sensing,
which is termed as Track-MDP. In contrast to the POMDP formulation, the
Track-MDP formulation is amenable to an RL based solution. It is shown that the
optimal policy for the Track-MDP formulation, which is approximated through RL,
is guaranteed to track all significant target paths with certainty. The
Track-MDP method is then compared with the optimal POMDP policy, and it is
shown that the infinite horizon tracking reward of the optimal Track-MDP policy
is the same as that of the optimal POMDP policy. In simulations it is
demonstrated that Track-MDP based RL leads to a policy that can track the
target with high accuracy.
