  Most mathematical distortions used in ML are fundamentally integral in
nature: $f$-divergences, Bregman divergences, (regularized) optimal transport
distances, integral probability metrics, geodesic distances, etc. In this
paper, we unveil a grounded theory and tools which can help improve these
distortions to better cope with ML requirements. We start with a generalization
of Riemann integration that also encapsulates functions that are not strictly
additive but are, more generally, $t$-additive, as in nonextensive statistical
mechanics. Notably, this recovers Volterra's product integral as a special
case. We then generalize the Fundamental Theorem of calculus using an extension
of the (Euclidean) derivative. This, along with a series of more specific
Theorems, serves as a basis for results showing how one can specifically
design, alter, or change fundamental properties of distortion measures in a
simple way, with a special emphasis on geometric- and ML-related properties
that are the metricity, hyperbolicity, and encoding. We show how to apply it to
a problem that has recently gained traction in ML: hyperbolic embeddings with a
"cheap" and accurate encoding along the hyperbolic vs Euclidean scale. We
unveil a new application for which the Poincar\'e disk model has very appealing
features, and our theory comes in handy: \textit{model} embeddings for boosted
combinations of decision trees, trained using the log-loss (trees) and logistic
loss (combinations).
