  Recent research has introduced a key notion of $H$-consistency bounds for
surrogate losses. These bounds offer finite-sample guarantees, quantifying the
relationship between the zero-one estimation error (or other target loss) and
the surrogate loss estimation error for a specific hypothesis set. However,
previous bounds were derived under the condition that a lower bound of the
surrogate loss conditional regret is given as a convex function of the target
conditional regret, without non-constant factors depending on the predictor or
input instance. Can we derive finer and more favorable $H$-consistency bounds?
In this work, we relax this condition and present a general framework for
establishing enhanced $H$-consistency bounds based on more general inequalities
relating conditional regrets. Our theorems not only subsume existing results as
special cases but also enable the derivation of more favorable bounds in
various scenarios. These include standard multi-class classification, binary
and multi-class classification under Tsybakov noise conditions, and bipartite
ranking.
