<p>What about rewriting the problem in the following way?</p>

<p>$$\underset{\left\{ c_{t}\right\} }{max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right) e^{y_t}\right]e^{-\rho t}dt$$</p>

<p>with the new state variables defined as</p>

<p>$$\begin{align}
\dot{k_{t}}=f\left(k_{t}\right)-c_{t}\\
\dot{y}_{t}=-h\left(k_{t}\right)
\end{align}$$</p>

<p>given the initial conditions $(k(0),y(0))=(k_0,0)$.</p>

CC BY-SA 3.0
2015-09-29T12:00:51.567Z
Thanks ramazan. So, I think in this way the problem reduces to a autonomous problem. Am I right ?
optimal control
CC BY-SA 3.0
2015-09-29T20:45:00.040Z
I am not knowledgeable from this point onwards, I have checked for the definition of an autonomous optimal control problem, but seems that I could not find a clear one.
ramazan
CC BY-SA 3.0
2015-09-29T00:07:36.737Z
2015-09-29T03:53:34.087Z
2015-09-29T03:53:34.087Z
ramazan
<p>Following Caputo, an Optimal Control problem is autonomous when none of the functions appearing in the description of the problem depends explicitly on the time-variable. <strong>But this means that the standard setup with an exogenous utility-discount factor is not autonomous</strong>, <em>but</em> it can be made so by redefining the multiplier(s) and then using the current-value Hamiltonian.  </p>

<p>@ramazan approach starts correctly, by breaking the discount factor and including the naughty part in the instantaneous objective function. Nothing forbids the state variable to appear in the instantaneous objective function, on the contrary, the general theoretical treatment always includes it. Denoting $y_t = \int_{0}^{t}h\left(k_{s}\right)ds$, your problem now becomes</p>

<p>$$\underset{\left\{ c_{t}\right\} }{\max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right)e^{-y_t}\right]e^{-\rho t}dt$$</p>

<p>$$\dot{k_{t}}=f\left(k_{t}\right)-c_{t}$$</p>

<p>with current value Hamiltonian</p>

<p>$$\mathcal{H}^{current}=u\left(c_t\right)e^{-y_t} +\lambda_t\left[f\left(k_t\right)-c_t\right]$$</p>

<p>While taking the derivative with respect to consumption presents no issues, the derivative of the Hamiltonian with respect to capital needs some care (and this is why I kept the $t$-index in the Hamiltonian). If $G$ is the antiderivative of $h$, then $y_t = G(k_t) - G(k_0)$. I guess the rest are easy for you, although of course the first-order conditions will be more complicated.</p>

CC BY-SA 3.0
2015-09-29T11:59:57.240Z
thanks for your time. So, when I look the hamiltonian you write, the system seems to be autonomous. So, you say that this is an autonomous problem ?
Also, I did not understand why you did not put the second state variable $\dot{y}$ as ramazan did in his calculation ?  Because, in a standard optimal control problem, integral could not be present on a hamiltonian,
optimal control
CC BY-SA 3.0
2015-09-29T12:18:57.197Z
Autonomous with the qualification I mentioned in the answer. I don't see any "second state variable", since $y_t$ is just a function of the capital state variable. Have you calculated the usual first-order conditions to see what they give you?
Alecos Papadopoulos
CC BY-SA 3.0
2015-09-29T02:48:36.227Z
2015-09-29T02:48:36.227Z
Alecos Papadopoulos
<p>I think I proved in a rigorous way that the system is autonomous for the model that I have written 3-4 days and I think it is useful for the community, especially for those who are working on macroeconomics of growth.</p>

<p>Let's write the present-value Hamiltonian ;</p>

<p>$$\mathcal{H}=e^{-\triangle}\left[ U\left( c\right)\right]
+\lambda _{2}^{}\left[ f(k)-c\right] +\lambda _{2}\left[ \rho
+h(k)\right] $$</p>

<p>We know that ;</p>

<p>$$\dot{\lambda}_{1}=- \frac{\partial \mathcal{H}}{\partial k}$$</p>

<p>$$\dot{\lambda}_{2}=- \frac{\partial \mathcal{H}}{\partial \triangle}$$</p>

<p>When I differenciate Hamiltonian according to time $t$ ;</p>

<p>$$\frac{d \mathcal{H}}{dt}=\frac{\partial \mathcal{H}}{\partial t} + \mathcal{H}_k \dot{k} + \mathcal{H}_{\lambda_1} \dot{\lambda}_{1} + \mathcal{H}_\triangle \dot{\triangle} + \mathcal{H}_{\lambda_2} \dot{\lambda}_{2} $$</p>

<p>It is straightforward to see that this term reduces to the following one ;</p>

<p>$$\frac{d \mathcal{H}}{dt}=\frac{\partial \mathcal{H}}{\partial t}$$</p>

<p>From page 299, theorem 9.6.1 of Léonard and Dockner <strong><em>(Optimal Control Theory and Static Optimization in Economics)</em></strong> about the transversality condition which shows that </p>

<p>$$\underset{t\rightarrow\infty}{lim}\mathcal{H}\left(t\right)=0$$</p>

<p>when $\rho &gt; 0$ </p>

<p>In this case, it is obvious that along the optimal path, the Hamiltonian takes the value 0, which ensures a autonomous dynamical system.</p>

CC BY-SA 3.0
2015-10-03T19:53:33.373Z
I don't follow the last argument you use. The equality of partial and total derivatives of the Hamiltonian with respect to time is a general result, irrespective of whether the problem is autonomous or not. I have posted a second answer proving directly the autonomous nature of the problem.
Alecos Papadopoulos
CC BY-SA 3.0
2015-10-03T22:21:13.843Z
If the system is in steady state at $t=T$ we must have $H(T) = 0$ and $H(T+\Delta t) = 0$ by the transversality condition. This finding implies $dH/dt=0$ for $t \geq T$. But this is only true in steady state and not necessarily on the saddle path for $t<T$.
clueless
CC BY-SA 3.0
2015-10-02T11:09:35.143Z
2015-10-02T11:09:35.143Z
optimal control
<p>The OP's answer is correct in its conclusion, but he applies a strange argument at the end to arrive there.  </p>

<p>Applying brute-force differentiation, the present value Hamiltonian is</p>

<p>$$\mathcal{H}=e^{-\triangle} U\left( c\right)
+\lambda _{1}^{}\left[ f(k)-c\right] +\lambda _{2}\left[ \rho
+h(k)\right] $$</p>

<p>and so </p>

<p>$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+
e^{-\triangle} U'(c)\dot c + \dot \lambda_1\left[ f(k)-c\right]  + \lambda_1\left[ f'(k)\dot k-\dot c\right] + \dot \lambda _{2}\left[ \rho
+h(k)\right]+\lambda _{2}h'(k) \dot k$$</p>

<p>Re-arranging, and using $\rho +h(k) = \dot \triangle$ we get</p>

<p>$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ [e^{-\triangle}U'(c)-\lambda_1]\dot c   + [\lambda_1f'(k)+\lambda _{2}h'(k)]\dot k + \dot \lambda _{2}\dot \triangle + \dot \lambda_1\left[ f(k)-c\right] $$</p>

<p>Along the optimal path, we have $e^{-\triangle}U'(c)=\lambda_1$ so the second term above vanishes. Also optimally we have $\dot{\lambda}_{1}=- \frac{\partial \mathcal{H}}{\partial k}$ and observe that $\frac{\partial \mathcal{H}}{\partial k} = [\lambda_1f'(k)+\lambda _{2}h'(k)]$. Substituting we get</p>

<p>$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ \frac{\partial \mathcal{H}}{\partial k}\dot k + \dot \lambda _{2}\dot \triangle  - \frac{\partial \mathcal{H}}{\partial k}\left[ f(k)-c\right] $$ </p>

<p>$$=\frac{\partial \mathcal{H}}{\partial k}[\dot k - f(k)+c] + \dot \triangle \cdot\big[\dot \lambda _{2} - e^{-\triangle}U\left( c\right)\big] $$</p>

<p>The first term is zero, from the law of motion of capital. Moreover, another necessary condition for the optimal path, given how the problem has been formulated,  is $\dot{\lambda}_{2}=- \frac{\partial \mathcal{H}}{\partial \triangle}$.  so we arrive at</p>

<p>$$\frac {d\mathcal{H}}{dt} =-\dot \triangle \cdot\left[ \frac{\partial \mathcal{H}}{\partial \triangle} + e^{-\triangle}U\left( c\right)\right]$$</p>

<p>Now
$$\frac{\partial \mathcal{H}}{\partial \triangle} = -e^{-\triangle} U\left( c\right)$$
so we obtain </p>

<p>$$\frac {d\mathcal{H}}{dt} =0$$</p>

<p>This directly proves that the problem is autonomous.</p>

<p>More generally, irrespective of whether the problem is autonomous or not, we have that along the optimal path</p>

<p>$$\frac {d\mathcal{H}}{dt} =\frac {\partial \mathcal{H}}{\partial t}$$</p>

<p>So <em>if</em> it is autonomous, then we get the zero-derivative result.</p>

CC BY-SA 3.0
2015-10-04T11:22:47.720Z
Thanks for your detailed response. What I have used was the direct transversality condition, as clueless mentionned in the comment. But your response is more clear, thanks again.
optimal control
CC BY-SA 3.0
2015-10-03T19:50:42.493Z
2015-10-03T19:50:42.493Z
Alecos Papadopoulos
<p>I have a following system with endogeneous discounting.
$c,k$ and $h(k)$ are consumption, capital and endogeneous discount function based on physical capital. (the properties of this function are not relevant with the question, so I don't write them in order to take less space in this post. The production function is increasing and concave as usual.)</p>

<p>$$\underset{\left\{ c_{t}\right\} }{max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right)\right]e^{-\triangle}dt$$</p>

<p>My state variables are </p>

<p>$$\begin{align}
\dot{k_{t}}=f\left(k_{t}\right)-c_{t}\\
\dot{\triangle}_{t}=\rho + h\left(k_{t}\right)
\end{align}$$</p>

<p>I note $\triangle=\int_{0}^{t} (\rho + h\left(k_{t}\right))dt$</p>

<p>I write the present-value Hamiltonian as </p>

<p>$$\mathcal{H}^{present}=u\left(c\right)e^{-\triangle}+\tilde{\lambda_{1}}\left[f\left(k\right)-c\right]+\tilde{\lambda_{2}}\left[\rho + h\left(k\right)\right]$$</p>

<blockquote>
  <p>In this model, my doubt is that if the system is autonomous or not because when I integrate $\dot{\triangle}$, I have $\triangle=
\rho t+ \int_{0}^{t}h\left(k_{t}\right)dt$, which depends explicitly on time $t$. So, evidently, this differential equation is non-autonomous.</p>
</blockquote>

<p>Is this system really non-autonomous ? </p>

<p>What I try to do, in order to have an autonomous system is to define variables. Let's say $\lambda_{1}=\tilde{\lambda_{1}}e^{\triangle}$ and $\lambda_{2}=\tilde{\lambda_{2}}e^{\triangle}$. In this way, I write the current-value Hamiltonian ;</p>

<p>$$\mathcal{H}^{current}=u\left(c\right) +\lambda_{1}\left[f\left(k\right)-c\right]+\lambda_{2}\left[\rho + h\left(k\right)\right]$$</p>

<p>I am really stuck at this point. For the moment, the system seems to be autonomous as the Hamiltonian does not depend explicitly on time here but I can not really be sure.</p>

<p>Hints or suggestions are all welcome.</p>

CC BY-SA 3.0
2015-09-29T01:25:32.477Z
On the side, It would be better to change the dummy variable of integration from $t$ to something like $s$, to avoid confusion, since $t$ is your actual running index.
Alecos Papadopoulos
CC BY-SA 3.0
2015-09-29T02:16:29.960Z
"Evidently, this differential equation is non-autonomous". That's a wrong statement. The differential equation itself is autonomous (except if $h()$ contains $t$ explicitly), the fact that the time-variable appears after integrating is a different issue.
Alecos Papadopoulos
<macroeconomics><optimal-control>
Autonomous or non-autonomous optimal control system?
optimal control
# Autonomous or non-autonomous optimal control system?

(Asked by: username_0 on 2015-09-28)

I have a following system with endogeneous discounting.
$c,k$ and $h(k)$ are consumption, capital and endogeneous discount function based on physical capital. (the properties of this function are not relevant with the question, so I don't write them in order to take less space in this post. The production function is increasing and concave as usual.)

$$\underset{\left\{ c\_{t}\right\} }{max}\int\_{t=0}^{\infty}\left[u\left(c\_{t}\right)\right]e^{-\triangle}dt$$

My state variables are

$$\begin{align}
\dot{k\_{t}}=f\left(k\_{t}\right)-c\_{t}\\
\dot{\triangle}\_{t}=\rho + h\left(k\_{t}\right)
\end{align}$$

I note $\triangle=\int\_{0}^{t} (\rho + h\left(k\_{t}\right))dt$

I write the present-value Hamiltonian as

$$\mathcal{H}^{present}=u\left(c\right)e^{-\triangle}+\tilde{\lambda\_{1}}\left[f\left(k\right)-c\right]+\tilde{\lambda\_{2}}\left[\rho + h\left(k\right)\right]$$

> In this model, my doubt is that if the system is autonomous or not because when I integrate $\dot{\triangle}$, I have $\triangle=
> \rho t+ \int\_{0}^{t}h\left(k\_{t}\right)dt$, which depends explicitly on time $t$. So, evidently, this differential equation is non-autonomous.

Is this system really non-autonomous ?

What I try to do, in order to have an autonomous system is to define variables. Let's say $\lambda\_{1}=\tilde{\lambda\_{1}}e^{\triangle}$ and $\lambda\_{2}=\tilde{\lambda\_{2}}e^{\triangle}$. In this way, I write the current-value Hamiltonian ;

$$\mathcal{H}^{current}=u\left(c\right) +\lambda\_{1}\left[f\left(k\right)-c\right]+\lambda\_{2}\left[\rho + h\left(k\right)\right]$$

I am really stuck at this point. For the moment, the system seems to be autonomous as the Hamiltonian does not depend explicitly on time here but I can not really be sure.

Hints or suggestions are all welcome.

**Tags:** macroeconomics, optimal-control

> **Comments:**
> **username_1**: On the side, It would be better to change the dummy variable of integration from $t$ to something like $s$, to avoid confusion, since $t$ is your actual running index.  
> **username_1**: "Evidently, this differential equation is non-autonomous". That's a wrong statement. The differential equation itself is autonomous (except if $h()$ contains $t$ explicitly), the fact that the time-variable appears after integrating is a different issue.  

---

## 4 answers

### Answer by username_1 on 2015-10-03. Score: 4 | Accepted answer

The OP's answer is correct in its conclusion, but he applies a strange argument at the end to arrive there.

Applying brute-force differentiation, the present value Hamiltonian is

$$\mathcal{H}=e^{-\triangle} U\left( c\right)
+\lambda \_{1}^{}\left[ f(k)-c\right] +\lambda \_{2}\left[ \rho
+h(k)\right] $$

and so

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+
e^{-\triangle} U'(c)\dot c + \dot \lambda\_1\left[ f(k)-c\right] + \lambda\_1\left[ f'(k)\dot k-\dot c\right] + \dot \lambda \_{2}\left[ \rho
+h(k)\right]+\lambda \_{2}h'(k) \dot k$$

Re-arranging, and using $\rho +h(k) = \dot \triangle$ we get

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ [e^{-\triangle}U'(c)-\lambda\_1]\dot c + [\lambda\_1f'(k)+\lambda \_{2}h'(k)]\dot k + \dot \lambda \_{2}\dot \triangle + \dot \lambda\_1\left[ f(k)-c\right] $$

Along the optimal path, we have $e^{-\triangle}U'(c)=\lambda\_1$ so the second term above vanishes. Also optimally we have $\dot{\lambda}\_{1}=- \frac{\partial \mathcal{H}}{\partial k}$ and observe that $\frac{\partial \mathcal{H}}{\partial k} = [\lambda\_1f'(k)+\lambda \_{2}h'(k)]$. Substituting we get

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ \frac{\partial \mathcal{H}}{\partial k}\dot k + \dot \lambda \_{2}\dot \triangle - \frac{\partial \mathcal{H}}{\partial k}\left[ f(k)-c\right] $$

$$=\frac{\partial \mathcal{H}}{\partial k}[\dot k - f(k)+c] + \dot \triangle \cdot\big[\dot \lambda \_{2} - e^{-\triangle}U\left( c\right)\big] $$

The first term is zero, from the law of motion of capital. Moreover, another necessary condition for the optimal path, given how the problem has been formulated, is $\dot{\lambda}\_{2}=- \frac{\partial \mathcal{H}}{\partial \triangle}$. so we arrive at

$$\frac {d\mathcal{H}}{dt} =-\dot \triangle \cdot\left[ \frac{\partial \mathcal{H}}{\partial \triangle} + e^{-\triangle}U\left( c\right)\right]$$

Now
$$\frac{\partial \mathcal{H}}{\partial \triangle} = -e^{-\triangle} U\left( c\right)$$
so we obtain

$$\frac {d\mathcal{H}}{dt} =0$$

This directly proves that the problem is autonomous.

More generally, irrespective of whether the problem is autonomous or not, we have that along the optimal path

$$\frac {d\mathcal{H}}{dt} =\frac {\partial \mathcal{H}}{\partial t}$$

So *if* it is autonomous, then we get the zero-derivative result.

> **Comments:**
> **username_0**: Thanks for your detailed response. What I have used was the direct transversality condition, as username_3 mentionned in the comment. But your response is more clear, thanks again.  

---

### Answer by username_2 on 2015-09-29. Score: 5

What about rewriting the problem in the following way?

$$\underset{\left\{ c\_{t}\right\} }{max}\int\_{t=0}^{\infty}\left[u\left(c\_{t}\right) e^{y\_t}\right]e^{-\rho t}dt$$

with the new state variables defined as

$$\begin{align}
\dot{k\_{t}}=f\left(k\_{t}\right)-c\_{t}\\
\dot{y}\_{t}=-h\left(k\_{t}\right)
\end{align}$$

given the initial conditions $(k(0),y(0))=(k\_0,0)$.

> **Comments:**
> **username_0**: Thanks username_2. So, I think in this way the problem reduces to a autonomous problem. Am I right ?  
> **username_2**: I am not knowledgeable from this point onwards, I have checked for the definition of an autonomous username_0 problem, but seems that I could not find a clear one.  

---

### Answer by username_1 on 2015-09-29. Score: 5

Following Caputo, an Optimal Control problem is autonomous when none of the functions appearing in the description of the problem depends explicitly on the time-variable. **But this means that the standard setup with an exogenous utility-discount factor is not autonomous**, *but* it can be made so by redefining the multiplier(s) and then using the current-value Hamiltonian.

@username_2 approach starts correctly, by breaking the discount factor and including the naughty part in the instantaneous objective function. Nothing forbids the state variable to appear in the instantaneous objective function, on the contrary, the general theoretical treatment always includes it. Denoting $y\_t = \int\_{0}^{t}h\left(k\_{s}\right)ds$, your problem now becomes

$$\underset{\left\{ c\_{t}\right\} }{\max}\int\_{t=0}^{\infty}\left[u\left(c\_{t}\right)e^{-y\_t}\right]e^{-\rho t}dt$$

$$\dot{k\_{t}}=f\left(k\_{t}\right)-c\_{t}$$

with current value Hamiltonian

$$\mathcal{H}^{current}=u\left(c\_t\right)e^{-y\_t} +\lambda\_t\left[f\left(k\_t\right)-c\_t\right]$$

While taking the derivative with respect to consumption presents no issues, the derivative of the Hamiltonian with respect to capital needs some care (and this is why I kept the $t$-index in the Hamiltonian). If $G$ is the antiderivative of $h$, then $y\_t = G(k\_t) - G(k\_0)$. I guess the rest are easy for you, although of course the first-order conditions will be more complicated.

> **Comments:**
> **username_0**: thanks for your time. So, when I look the hamiltonian you write, the system seems to be autonomous. So, you say that this is an autonomous problem ? Also, I did not understand why you did not put the second state variable $\dot{y}$ as username_2 did in his calculation ?  Because, in a standard username_0 problem, integral could not be present on a hamiltonian,  
> **username_1**: Autonomous with the qualification I mentioned in the answer. I don't see any "second state variable", since $y_t$ is just a function of the capital state variable. Have you calculated the usual first-order conditions to see what they give you?  

---

### Answer by username_0 on 2015-10-02. Score: 3

I think I proved in a rigorous way that the system is autonomous for the model that I have written 3-4 days and I think it is useful for the community, especially for those who are working on macroeconomics of growth.

Let's write the present-value Hamiltonian ;

$$\mathcal{H}=e^{-\triangle}\left[ U\left( c\right)\right]
+\lambda \_{2}^{}\left[ f(k)-c\right] +\lambda \_{2}\left[ \rho
+h(k)\right] $$

We know that ;

$$\dot{\lambda}\_{1}=- \frac{\partial \mathcal{H}}{\partial k}$$

$$\dot{\lambda}\_{2}=- \frac{\partial \mathcal{H}}{\partial \triangle}$$

When I differenciate Hamiltonian according to time $t$ ;

$$\frac{d \mathcal{H}}{dt}=\frac{\partial \mathcal{H}}{\partial t} + \mathcal{H}\_k \dot{k} + \mathcal{H}\_{\lambda\_1} \dot{\lambda}\_{1} + \mathcal{H}\_\triangle \dot{\triangle} + \mathcal{H}\_{\lambda\_2} \dot{\lambda}\_{2} $$

It is straightforward to see that this term reduces to the following one ;

$$\frac{d \mathcal{H}}{dt}=\frac{\partial \mathcal{H}}{\partial t}$$

From page 299, theorem 9.6.1 of Léonard and Dockner ***(Optimal Control Theory and Static Optimization in Economics)*** about the transversality condition which shows that

$$\underset{t\rightarrow\infty}{lim}\mathcal{H}\left(t\right)=0$$

when $\rho > 0$

In this case, it is obvious that along the optimal path, the Hamiltonian takes the value 0, which ensures a autonomous dynamical system.

> **Comments:**
> **username_1**: I don't follow the last argument you use. The equality of partial and total derivatives of the Hamiltonian with respect to time is a general result, irrespective of whether the problem is autonomous or not. I have posted a second answer proving directly the autonomous nature of the problem.  
> **username_3**: If the system is in steady state at $t=T$ we must have $H(T) = 0$ and $H(T+\Delta t) = 0$ by the transversality condition. This finding implies $dH/dt=0$ for $t \geq T$. But this is only true in steady state and not necessarily on the saddle path for $t<T$.  

---

