  This paper considers online optimal control with affine constraints on the
states and actions under linear dynamics with bounded random disturbances. The
system dynamics and constraints are assumed to be known and time-invariant but
the convex stage cost functions change adversarially. To solve this problem, we
propose Online Gradient Descent with Buffer Zones (OGD-BZ). Theoretically, we
show that OGD-BZ with proper parameters can guarantee the system to satisfy all
the constraints despite any admissible disturbances. Further, we investigate
the policy regret of OGD-BZ, which compares OGD-BZ's performance with the
performance of the optimal linear policy in hindsight. We show that OGD-BZ can
achieve a policy regret upper bound that is the square root of the horizon
length multiplied by some logarithmic terms of the horizon length under proper
algorithm parameters.
