<p>A friend suggested a solution, which I will sketch below. It relies on some rank conditions that are hard to interpret, but it is better than nothing.</p>
<p>(1) If either <span class="math-container">$f$</span> or <span class="math-container">$g$</span> is strictly convex, then (as I have already assumed a solution exists) there exists a unique solution <span class="math-container">$x(y)$</span>.</p>
<p>(2) If <span class="math-container">$f$</span> and <span class="math-container">$g$</span> are continuously differentiable and one imposes assumptions so that the constraint binds, then this solution and its associated Lagrange multiplier (which exists by convexity) are <span class="math-container">$N+1$</span> unknowns solving a system of <span class="math-container">$N+1$</span> equations (first order condition plus constraint). If <span class="math-container">$f$</span> and <span class="math-container">$g$</span> are twice continuously differentiable and some rank conditions hold, one can apply the implicit function theorem to argue that <span class="math-container">$x(y)$</span> is continuously differentiable.</p>
<p>(3) Theorem 3 of Milgrom &amp; Segal (2002) is an envelope theorem implying that---under some conditions implied by those assumed in (2)---<span class="math-container">$V(y)$</span> is differentiable and <span class="math-container">$V_y(y) = -f_x(x(y))$</span>. (Here it helps to rewrite the problem as suggested by Ilya's comment on Math Stack Exchange.) Since <span class="math-container">$f(x)$</span> and <span class="math-container">$x(y)$</span> are both continuously differentiable, so is <span class="math-container">$V_y(y)$</span>. So <span class="math-container">$V(y)$</span> is twice continuously differentiable.</p>

CC BY-SA 4.0
2022-03-15T19:36:32.103Z
2022-03-15T19:36:32.103Z
John Sturm
<p><strong>Setup:</strong>
Consider the problem
<span class="math-container">$$
V(y) \quad = \quad \min_{x \in \mathbb R^N} f(x) \quad \text{s.t.} \quad g(x+y) \leq 0
$$</span>
where <span class="math-container">$f$</span> and <span class="math-container">$g$</span> are convex functions and <span class="math-container">$y \in \mathbb R^N$</span> is a parameter taken as given. Assume that this problem admits a (potentially non-unique) solution for all <span class="math-container">$y \in \mathbb R^N$</span>.</p>
<p><strong>Question:</strong>
Under what conditions is <span class="math-container">$V(y)$</span> twice continuously differentiable in <span class="math-container">$y$</span>?</p>
<p>This is related to the envelope theorem (e.g. Milgrom Segal [2002]) but deals with a higher derivative.</p>
<p>Cross posted from <a href="https://math.stackexchange.com/posts/4403538/edit">Math Stack Exchange</a>.</p>

<optimization><real-analysis>
Differentiability of value of convex optimization problem
John Sturm
# Differentiability of value of convex optimization problem

(Asked by: username_0 on 2022-03-15)

**Setup:**
Consider the problem
$$
V(y) \quad = \quad \min\_{x \in \mathbb R^N} f(x) \quad \text{s.t.} \quad g(x+y) \leq 0
$$
where $f$ and $g$ are convex functions and $y \in \mathbb R^N$ is a parameter taken as given. Assume that this problem admits a (potentially non-unique) solution for all $y \in \mathbb R^N$.

**Question:**
Under what conditions is $V(y)$ twice continuously differentiable in $y$?

This is related to the envelope theorem (e.g. Milgrom Segal [2002]) but deals with a higher derivative.

Cross posted from [Math Stack Exchange](https://math.stackexchange.com/posts/4403538/edit).

**Tags:** optimization, real-analysis


---

## 1 answers

### Answer by username_0 on 2022-03-15. Score: 1

A friend suggested a solution, which I will sketch below. It relies on some rank conditions that are hard to interpret, but it is better than nothing.

(1) If either $f$ or $g$ is strictly convex, then (as I have already assumed a solution exists) there exists a unique solution $x(y)$.

(2) If $f$ and $g$ are continuously differentiable and one imposes assumptions so that the constraint binds, then this solution and its associated Lagrange multiplier (which exists by convexity) are $N+1$ unknowns solving a system of $N+1$ equations (first order condition plus constraint). If $f$ and $g$ are twice continuously differentiable and some rank conditions hold, one can apply the implicit function theorem to argue that $x(y)$ is continuously differentiable.

(3) Theorem 3 of Milgrom & Segal (2002) is an envelope theorem implying that---under some conditions implied by those assumed in (2)---$V(y)$ is differentiable and $V\_y(y) = -f\_x(x(y))$. (Here it helps to rewrite the problem as suggested by Ilya's comment on Math Stack Exchange.) Since $f(x)$ and $x(y)$ are both continuously differentiable, so is $V\_y(y)$. So $V(y)$ is twice continuously differentiable.


---

