  Modern classification problems exhibit heterogeneities across individual
classes: Each class may have unique attributes, such as sample size, label
quality, or predictability (easy vs difficult), and variable importance at
test-time. Without care, these heterogeneities impede the learning process,
most notably, when optimizing fairness objectives. Confirming this, under a
gaussian mixture setting, we show that the optimal SVM classifier for balanced
accuracy needs to be adaptive to the class attributes. This motivates us to
propose CAP: An effective and general method that generates a class-specific
learning strategy (e.g. hyperparameter) based on the attributes of that class.
This way, optimization process better adapts to heterogeneities. CAP leads to
substantial improvements over the naive approach of assigning separate
hyperparameters to each class. We instantiate CAP for loss function design and
post-hoc logit adjustment, with emphasis on label-imbalanced problems. We show
that CAP is competitive with prior art and its flexibility unlocks clear
benefits for fairness objectives beyond balanced accuracy. Finally, we evaluate
CAP on problems with label noise as well as weighted test objectives to
showcase how CAP can jointly adapt to different heterogeneities.
