  Any technology requires precise benchmarking of its components, and the
quantum technologies are no exception. Randomized benchmarking allows for the
relatively resource economical estimation of the average gate fidelity of
quantum gates from the Clifford group, assuming identical noise levels for all
gates, making use of suitable sequences of randomly chosen Clifford gates. In
this work, we report significant progress on randomized benchmarking, by
showing that it can be done for individual quantum gates outside the Clifford
group, even for varying noise levels per quantum gate. This is possible at
little overhead of quantum resources, but at the expense of a significant
classical computational cost. At the heart of our analysis is a
representation-theoretic framework that we develop here which is brought into
contact with classical estimation techniques based on bootstrapping and matrix
pencils. We demonstrate the functioning of the scheme at hand of benchmarking
tensor powers of T-gates. Apart from its practical relevance, we expect this
insight to be relevant as it highlights the role of assumptions made on unknown
noise processes when characterizing quantum gates at high precision.
