  We define a general notion of entropy in elementary, algebraic terms. Based
on that, weak forms of a scalar product and a distance measure are derived. We
give basic properties of these quantities, generalize the Cauchy-Schwarz
inequality, and relate our approach to the theory of scoring rules. Many
supporting examples illustrate our approach and give new perspectives on
established notions, such as the likelihood, the Kullback-Leibler divergence,
the uncorrelatedness of random variables, the scalar product itself, the
Tichonov regularization and the mutual information.
