  We address the problem of simultaneously learning and control in an online
receding horizon control setting. We consider the control of an unknown linear
dynamical system with general cost functions and affine constraints on the
control input. Our goal is to develop an online learning algorithm that
minimizes the dynamic regret, which is defined as the difference between the
cumulative cost incurred by the algorithm and that of the best policy with full
knowledge of the system, cost functions and state and that satisfies the
control input constraints. We propose a novel approach to explore in an online
receding horizon setting. The key challenge is to ensure that the control
generated by the receding horizon controller is persistently exciting. Our
approach is to apply a perturbation to the control input generated by the
receding horizon controller that balances the exploration-exploitation
trade-off. By exploring the conditions under which sub-linear regret is
guaranteed, We show that the proposed controller's performance is upper bounded
by $\tilde{\mathcal{O}}(T^{3/4})$ for both regret and cumulative constraint
violation when the controller has preview of the cost functions for the
interval that doubles in size from one interval to the next.
