  We use online convex optimization (OCO) for setpoint tracking with uncertain,
flexible loads. We consider full feedback from the loads, bandit feedback, and
two intermediate types of feedback: partial bandit where a subset of the loads
are individually observed and the rest are observed in aggregate, and Bernoulli
feedback where in each round the aggregator receives either full or bandit
feedback according to a known probability. We give sublinear regret bounds in
all cases. We numerically evaluate our algorithms on examples with
thermostatically controlled loads and electric vehicles.
