  In many quantum tasks, there is an unknown quantum object that one wishes to
learn. An online strategy for this task involves adaptively refining a
hypothesis to reproduce such an object or its measurement statistics. A common
evaluation metric for such a strategy is its regret, or roughly the accumulated
errors in hypothesis statistics. We prove a sublinear regret bound for learning
over general subsets of positive semidefinite matrices via the
regularized-follow-the-leader algorithm and apply it to various settings where
one wishes to learn quantum objects. For concrete applications, we present a
sublinear regret bound for learning quantum states, effects, channels,
interactive measurements, strategies, co-strategies, and the collection of
inner products of pure states. Our bound applies to many other quantum objects
with compact, convex representations. In proving our regret bound, we establish
various matrix analysis results useful in quantum information theory. This
includes a generalization of Pinsker's inequality for arbitrary positive
semidefinite operators with possibly different traces, which may be of
independent interest and applicable to more general classes of divergences.
