  By calculating the Kullback-Leibler divergence between two probability
measures belonging to different exponential families, we end up with a formula
that generalizes the ordinary Fenchel-Young divergence. Inspired by this
formula, we define the duo Fenchel-Young divergence and reports a majorization
condition on its pair of generators which guarantees that this divergence is
always non-negative. The duo Fenchel-Young divergence is also equivalent to a
duo Bregman divergence. We show the use of these duo divergences by calculating
the Kullback-Leibler divergence between densities of nested exponential
families, and report a formula for the Kullback-Leibler divergence between
truncated normal distributions. Finally, we prove that the skewed Bhattacharyya
distance between nested exponential families amounts to an equivalent skewed
duo Jensen divergence.
