<p>Upgrading an exchange of comments, a critical point in the question is the expression
\begin{align}
\frac{\partial u(\tilde x)}{\partial \rho}=\frac{1}{(\rho + u'(\tilde x) + 1)^2} 
\end{align}</p>

<p>which is mistaken , because from </p>

<p>\begin{align}
u(\tilde x) = \frac{\rho + u'(\tilde x)}{\rho + u'(\tilde x) + 1}.
\end{align}</p>

<p>we obtain</p>

<p>\begin{align}
\frac{\partial u(\tilde x)}{\partial \rho}=\frac{1+\partial u'(\tilde x)/\partial \rho}{(\rho + u'(\tilde x) + 1)^2} 
\end{align}</p>

<p>The OP notes that the sign of $\partial u'(\tilde x)/\partial \rho$ is indeterinate, and so $\partial u(\tilde x)/\partial \rho$ cannot be said to be monotonically increasing, which was what drove the pinning-down of the steady state.</p>

CC BY-SA 3.0
2015-07-29T23:33:32.053Z
2015-07-29T23:33:32.053Z
Alecos Papadopoulos
<p>The question is related to this <a href="https://economics.stackexchange.com/questions/6428/multiple-equilibria-which-one-to-select">thread</a>. I'd like to derive a unique steady state for an optimal control problem.</p>

<p>Consider the following programm
\begin{align}
&amp;V(x_0) := \max_u \int^\infty_0 e^{-\rho t}F(x(t),u(t))dt\\
s.t.~&amp;\dot x(t)=f(x(t),u(t))\\
&amp;x(0) = x_0
\end{align}
where $\rho &gt; 0$ denotes time preference, $V(\cdot)$ is the value and $F(\cdot)$ an objective function. $x\in X$ is the state variable and $u\in U=[0,1]$ the control. The state is governed by $f(\cdot)$. The Hamilton-Jacobi-Bellman equation reads
\begin{align}
\rho V(x)=\max_u [F(x,u) + V'(x)f(x,u)],\quad \forall t\in[0,\infty)
\end{align}</p>

<p>Now presume that the feedback control is given by
\begin{align}
u(x) = \frac{1}{1+V'(x)} = \arg\max_u [F(x,u) + V'(x)f(x,u)],\quad \forall x\in X.
\end{align}</p>

<p>Suppose there is a fixed point at $x = \tilde x$ and we can derive an alternative representation for the optimal control in the fixed point with
\begin{align}
u(\tilde x) = \frac{\rho + u'(\tilde x)}{\rho + u'(\tilde x) + 1}.
\end{align}</p>

<p>Suppose additionaly the HJB in the fixed point is given by
\begin{align}
\rho V(x)=\ln\left(\frac{1}{1+V'(x)}\right) + 1 - \frac{1}{1+V'(x)}.
\end{align}</p>

<p>If $\rho\to 0$ approches zero we must have $V'(\tilde x) = 0 \Rightarrow u(\tilde x) = 1 \Rightarrow u'(\tilde x) = \infty$. On the other hand if  $\rho\to\infty$ approches infinity we must have $V(\tilde x) = 0$ by the definition of the value function and thus again $V'(\tilde x) = 0 \Rightarrow u(\tilde x) = 1$. Summarizing we have the following properties in equilibrium
\begin{align}
\lim_{\rho\to 0} u(\tilde x)=\lim_{\rho\to\infty} u(\tilde x) = 1.
\end{align}</p>

<p>Well, that's at odds with
 \begin{align}
\frac{\partial u(\tilde x)}{\partial \rho}=\frac{1}{(\rho + u'(\tilde x) + 1)^2} &gt; 0
\end{align}</p>

<p>being a strictly monoton increasing function, contradicting our previous result. Note, however, that we may resolve the issue by observing
\begin{align}
\lim_{u'(\tilde x)\to\infty}\frac{\partial u(\tilde x)}{\partial \rho}= 0
\end{align}</p>

<p>So, can we finally conjecture that we must have in the fixed point $u'(\tilde x) = \infty \Rightarrow u(\tilde x) = 1$ such that 
\begin{align}
\rho V(\tilde x)=\ln(1) + 1 - 1 \Leftrightarrow V(\tilde x) = 0.
\end{align} </p>

CC BY-SA 3.0
2015-07-29T09:34:35.723Z
I am not sure how you arrive at the expression for $\frac{\partial u(\tilde x)}{\partial \rho}$
Alecos Papadopoulos
CC BY-SA 3.0
2015-07-29T10:34:01.317Z
You're right. I think it should read: $\partial u(\tilde x)/\partial \rho = (1+\partial u'(\tilde x)/\partial \rho)/(\rho + u'(\tilde x) + 1)^2$ where the sign is indeterminated and thus, my argumentation is flawed.
clueless
CC BY-SA 3.0
2015-07-29T14:15:37.423Z
How are we going to proceed? Should I post this as a "negative-result" answer so that the post does not remain in the unanswered queue, or will you modify somehow the question?
Alecos Papadopoulos
CC BY-SA 3.0
2015-07-29T22:27:34.447Z
Probably negative result, cause I have no idea how to resolve the issue at hand. I think, I've actually found a different way to show that $u(\tilde x) = 1$ is a stable equlibrium associated with the highest value, since by the one shot deviation principle there is no incentive to deviate from that particular strategy. However, my orginial setting deals with two agents, being different from that one presented here.
clueless
<mathematical-economics><dynamic-programming><steady-state>
Conjecture Steady State from limit properties
clueless
# Conjecture Steady State from limit properties

(Asked by: username_0 on 2015-07-28)

The question is related to this [thread](https://economics.stackexchange.com/questions/6428/multiple-equilibria-which-one-to-select). I'd like to derive a unique steady state for an optimal control problem.

Consider the following programm
\begin{align}
&V(x\_0) := \max\_u \int^\infty\_0 e^{-\rho t}F(x(t),u(t))dt\\
s.t.~&\dot x(t)=f(x(t),u(t))\\
&x(0) = x\_0
\end{align}
where $\rho > 0$ denotes time preference, $V(\cdot)$ is the value and $F(\cdot)$ an objective function. $x\in X$ is the state variable and $u\in U=[0,1]$ the control. The state is governed by $f(\cdot)$. The Hamilton-Jacobi-Bellman equation reads
\begin{align}
\rho V(x)=\max\_u [F(x,u) + V'(x)f(x,u)],\quad \forall t\in[0,\infty)
\end{align}

Now presume that the feedback control is given by
\begin{align}
u(x) = \frac{1}{1+V'(x)} = \arg\max\_u [F(x,u) + V'(x)f(x,u)],\quad \forall x\in X.
\end{align}

Suppose there is a fixed point at $x = \tilde x$ and we can derive an alternative representation for the optimal control in the fixed point with
\begin{align}
u(\tilde x) = \frac{\rho + u'(\tilde x)}{\rho + u'(\tilde x) + 1}.
\end{align}

Suppose additionaly the HJB in the fixed point is given by
\begin{align}
\rho V(x)=\ln\left(\frac{1}{1+V'(x)}\right) + 1 - \frac{1}{1+V'(x)}.
\end{align}

If $\rho\to 0$ approches zero we must have $V'(\tilde x) = 0 \Rightarrow u(\tilde x) = 1 \Rightarrow u'(\tilde x) = \infty$. On the other hand if $\rho\to\infty$ approches infinity we must have $V(\tilde x) = 0$ by the definition of the value function and thus again $V'(\tilde x) = 0 \Rightarrow u(\tilde x) = 1$. Summarizing we have the following properties in equilibrium
\begin{align}
\lim\_{\rho\to 0} u(\tilde x)=\lim\_{\rho\to\infty} u(\tilde x) = 1.
\end{align}

Well, that's at odds with
\begin{align}
\frac{\partial u(\tilde x)}{\partial \rho}=\frac{1}{(\rho + u'(\tilde x) + 1)^2} > 0
\end{align}

being a strictly monoton increasing function, contradicting our previous result. Note, however, that we may resolve the issue by observing
\begin{align}
\lim\_{u'(\tilde x)\to\infty}\frac{\partial u(\tilde x)}{\partial \rho}= 0
\end{align}

So, can we finally conjecture that we must have in the fixed point $u'(\tilde x) = \infty \Rightarrow u(\tilde x) = 1$ such that
\begin{align}
\rho V(\tilde x)=\ln(1) + 1 - 1 \Leftrightarrow V(\tilde x) = 0.
\end{align}

**Tags:** mathematical-economics, dynamic-programming, steady-state

> **Comments:**
> **username_1**: I am not sure how you arrive at the expression for $\frac{\partial u(\tilde x)}{\partial \rho}$  
> **username_0**: You're right. I think it should read: $\partial u(\tilde x)/\partial \rho = (1+\partial u'(\tilde x)/\partial \rho)/(\rho + u'(\tilde x) + 1)^2$ where the sign is indeterminated and thus, my argumentation is flawed.  
> **username_1**: How are we going to proceed? Should I post this as a "negative-result" answer so that the post does not remain in the unanswered queue, or will you modify somehow the question?  
> **username_0**: Probably negative result, cause I have no idea how to resolve the issue at hand. I think, I've actually found a different way to show that $u(\tilde x) = 1$ is a stable equlibrium associated with the highest value, since by the one shot deviation principle there is no incentive to deviate from that particular strategy. However, my orginial setting deals with two agents, being different from that one presented here.  

---

## 1 answers

### Answer by username_1 on 2015-07-29. Score: 1

Upgrading an exchange of comments, a critical point in the question is the expression
\begin{align}
\frac{\partial u(\tilde x)}{\partial \rho}=\frac{1}{(\rho + u'(\tilde x) + 1)^2}
\end{align}

which is mistaken , because from

\begin{align}
u(\tilde x) = \frac{\rho + u'(\tilde x)}{\rho + u'(\tilde x) + 1}.
\end{align}

we obtain

\begin{align}
\frac{\partial u(\tilde x)}{\partial \rho}=\frac{1+\partial u'(\tilde x)/\partial \rho}{(\rho + u'(\tilde x) + 1)^2}
\end{align}

The OP notes that the sign of $\partial u'(\tilde x)/\partial \rho$ is indeterinate, and so $\partial u(\tilde x)/\partial \rho$ cannot be said to be monotonically increasing, which was what drove the pinning-down of the steady state.


---

