<p><span class="math-container">$\chi^2$</span>-divergence is not a Bregman divergence. </p>

<p>I'll show it for sample size <span class="math-container">$n=1$</span>. We would have
<span class="math-container">$$
(x-y)^2/x=f(x)-f(y)-f'(y)(x-y)$$</span>
If <span class="math-container">$y=0$</span> and <span class="math-container">$x&gt;0$</span> this says
<span class="math-container">$$x=f(x)-f(0)-xf'(0),$$</span>
<span class="math-container">$$1=\frac{f(x)-f(0)}x-f'(0).$$</span>
Taking <span class="math-container">$x\to 0^+$</span> this gives the contradiction <span class="math-container">$1=0$</span>.</p>

CC BY-SA 4.0
2019-10-15T07:43:30.253Z
2019-10-15T07:43:30.253Z
Bjørn Kjos-Hanssen
<p>Is the Chi-squared divergence <span class="math-container">$\sum_{i} \frac{(x(i)-y(i))^2}{x(i)}$</span> a <a href="https://en.m.wikipedia.org/wiki/Bregman_divergence" rel="nofollow noreferrer">Bregman divergence</a>? I.e., can it be written as
<span class="math-container">$\phi(x) - \phi(y) - \langle\phi'(y),x-y\rangle$</span>?</p>

<p>If so, what is the potential function <span class="math-container">$\phi(x)$</span> from which it is generated?
If not, does there exists a Bregman divergence which bounds Chi-square divergence from above?</p>

CC BY-SA 4.0
2019-10-12T15:37:24.233Z
This seems more appropriate for Math, but still, why the downvote?
Sasho Nikolov
CC BY-SA 4.0
2019-10-12T18:41:12.377Z
Not a proof, but since [this paper](https://arxiv.org/abs/1808.06148) introduces a generalization of Bregman divergences and then show this generalization includes the chi-squared divergence, it's fair to assume the "usual" Bregman divergences do not. As for the last question: KL divergence?
Clement C.
CC BY-SA 4.0
2019-10-13T02:04:40.597Z
Thanks!
I it an interesting work.. however, I didn't spot there proof of non-existence (they indeed show their generalization includes the chi-squared, however, they don't show -- as far I understood -- it cannot be realized as a Bregman divergence).
+ I think that the Chi square upper bounds the KL and not vice-versa:
https://cstheory.stackexchange.com/questions/30693/an-upper-bound-for-chi-square-divergence-in-terms-of-kl-divergence-for-general-a
Yonatan
CC BY-SA 4.0
2019-10-13T17:17:26.107Z
@Yonatan Of course, my bad :(
Clement C.
<convex-optimization><st.statistics>
Is the Chi-square divergence a Bregman divergence?
Yonatan
# Is the Chi-square divergence a Bregman divergence?

(Asked by: username_0 on 2019-10-12)

Is the Chi-squared divergence $\sum\_{i} \frac{(x(i)-y(i))^2}{x(i)}$ a [Bregman divergence](https://en.m.wikipedia.org/wiki/Bregman_divergence)? I.e., can it be written as
$\phi(x) - \phi(y) - \langle\phi'(y),x-y\rangle$?

If so, what is the potential function $\phi(x)$ from which it is generated?
If not, does there exists a Bregman divergence which bounds Chi-square divergence from above?

**Tags:** convex-optimization, st.statistics
> **Comments:**
> **username_1**: This seems more appropriate for Math, but still, why the downvote?  
> **username_2**: Not a proof, but since [this paper](https://arxiv.org/abs/1808.06148) introduces a generalization of Bregman divergences and then show this generalization includes the chi-squared divergence, it's fair to assume the "usual" Bregman divergences do not. As for the last question: KL divergence?  
> **username_0**: Thanks! I it an interesting work.. however, I didn't spot there proof of non-existence (they indeed show their generalization includes the chi-squared, however, they don't show -- as far I understood -- it cannot be realized as a Bregman divergence). + I think that the Chi square upper bounds the KL and not vice-versa: https://cstheory.stackexchange.com/questions/30693/an-upper-bound-for-chi-square-divergence-in-terms-of-kl-divergence-for-general-a  
> **username_2**: @Yonatan Of course, my bad :(  

---

## 1 answers

### Answer by username_3 on 2019-10-15. Score: 4 | Accepted answer

$\chi^2$-divergence is not a Bregman divergence.

I'll show it for sample size $n=1$. We would have
$$
(x-y)^2/x=f(x)-f(y)-f'(y)(x-y)$$
If $y=0$ and $x>0$ this says
$$x=f(x)-f(0)-xf'(0),$$
$$1=\frac{f(x)-f(0)}x-f'(0).$$
Taking $x\to 0^+$ this gives the contradiction $1=0$.


---

