  We consider the problem of controlling an unknown linear time-invariant
dynamical system from a single chain of black-box interactions, with no access
to resets or offline simulation. Under the assumption that the system is
controllable, we give the first efficient algorithm that is capable of
attaining sublinear regret in a single trajectory under the setting of online
nonstochastic control. This resolves an open problem on the stochastic LQR
problem, and in a more challenging setting that allows for adversarial
perturbations and adversarially chosen and changing convex loss functions.
  We give finite-time regret bounds for our algorithm on the order of
$2^{\tilde{O}(\mathcal{L})} + \tilde{O}(\text{poly}(\mathcal{L}) T^{2/3})$ for
general nonstochastic control, and $2^{\tilde{O}(\mathcal{L})} +
\tilde{O}(\text{poly}(\mathcal{L}) \sqrt{T})$ for black-box LQR, where
$\mathcal{L}$ is the system size which is an upper bound on the dimension. The
crucial step is a new system identification method that is robust to
adversarial noise, but incurs exponential cost.
  To complete the picture, we investigate the complexity of the online
black-box control problem, and give a matching lower bound of
$2^{\Omega(\mathcal{L})}$ on the regret, showing that the additional
exponential cost is inevitable. This lower bound holds even in the noiseless
setting, and applies to any, randomized or deterministic, black-box control
method.
