<p>The idea behind the function <span class="math-container">$h(t,k_t,c_t)$</span> is that it gives the increments of capital as it evolves. By definition of <span class="math-container">$g$</span>, <span class="math-container">$g(t,k_t,c_t)=k_{t+1}$</span>, so <span class="math-container">$h(t,k_t,c_t)=k_{t+1}-k_t$</span>. This <span class="math-container">$h$</span> function allows us to make a connection between the discrete and continuous time settings. This connection is highlighted by the author in equations (13) and (14) later on in the text. Without <span class="math-container">$h$</span>, the connection between discrete and continuous time is unclear since there is no analogous <span class="math-container">$g$</span> function in continuous time as there is no &quot;next&quot; time step from the current one. Expressing the discrete time case in terms <span class="math-container">$h$</span> rather than <span class="math-container">$g$</span> allows us to view the continuous time problem as the &quot;limiting&quot; case of the discrete time problem as time increments become infinitely small.</p>
<p>For point (2), it is just algebra. The equation in (2) is evaluated at the optimum. Given <span class="math-container">$g(t,k_t,c_t)=k_t+h(t,k_t,c_t)$</span>, we have by an envelope argument that <span class="math-container">$g_k(t)=1+h_k(t)$</span> and <span class="math-container">$g_c(t)=h_c(t)$</span>. Therefore,</p>
<p><span class="math-container">$$\begin{align} f_k(t+1)-\frac{f_c(t+1)}{g_c(t+1)}g_k(t+1)&amp;=-\frac{f_c(t)}{g_c(t)} \\
f_k(t+1)-\frac{f_c(t+1)}{h_c(t+1)}[1+h_k(t+1)]&amp;=-\frac{f_c(t)}{h_c(t)} \\ 
f_k(t+1)-\frac{f_c(t+1)}{h_c(t+1)}h_k(t+1) &amp;=\frac{f_c(t+1)}{h_c(t+1)}-\frac{f_c(t)}{h_c(t)} \\
f_k(t+1)-\frac{f_c(t+1)}{h_c(t+1)}h_k(t+1) &amp;= \Delta_t\left(\frac{f_c(t)}{h_c(t)}\right)\end{align}$$</span></p>
<p>For alternative references, I think one of the best presentations of recursive methods is in Stokey-Lucas-Prescott &quot;Recursive methods in economic dynamics&quot; (1989). However, the presentation there is, if anything, too maths heavy. But still, it is a great reference text.</p>

CC BY-SA 4.0
2024-07-06T13:04:18.507Z
Thanks Joseph. I'll check it out carefully and then check it.  I knew (2) was just algebra but I must have been doing something dumb. That's not surprising !!!!!!!!
mark leeds
CC BY-SA 4.0
2024-07-06T13:05:45.113Z
I don't have Stokey-Lucas-Prescott but I'm always hearing about it so maybe that's my hint to purchase it. Thanks again.
mark leeds
CC BY-SA 4.0
2024-07-06T23:36:59.517Z
Joseph: Your answer was very helpful. I still have to figure out what I was doing wrong with Q2 and the answer to Q1 was interesting. I'm looking forward to reading the rest of the paper. All the best.
mark leeds
CC BY-SA 4.0
2024-07-07T05:30:38.157Z
Joseph: I couldn't find my old scribble but I'm pretty sure that I had either $g_k$ or $g_c$ wrong when I did (2).  I just have one more question about this but if I can put it as a seperate question if you want me to ? The very first equation in the paper says max over $c_{t}$. So, when they write that, that seems to imply that there is a different optimal $c_t^{*}$ for each $t$, correct ? So,  the bellman-euler equation in (7) allows one to calculate the optimal $c_t^{*}$ and $k_{t}^{*} ~\forall ~ t$. I hope that's right. Thanks.
mark leeds
CC BY-SA 4.0
2024-07-11T07:10:48.993Z
@markleeds Yes, the $c_t^*$ will vary over time. The idea is that we are given some initial capital level $k_0$. If we also have an initial consumption level $c_0$, we can iterate between the Bellman-Euler equation and the budget constraint to trace out the optimal path (given $(c_0,k_0)$). The initial $c_0$ value is pinned down by the transversality condition which governs the asymptotic behaviour of the optimal path, but $c_0$ usually has no analytic solution.
Joseph Basford
CC BY-SA 4.0
2024-07-12T16:27:55.000Z
Thanks Joseph. That's very clear. My confuson was that when they do $max(c_t)$, $c_t$ is really the vector of $c_t$s. Got it. Thanks a lot.
mark leeds
CC BY-SA 4.0
2024-07-06T07:15:52.157Z
2024-07-06T07:15:52.157Z
Joseph Basford
<p>Hi: I'm reading a proof of &quot;<a href="https://people.hec.edu/rosu/wp-content/uploads/sites/43/2020/03/bellman.pdf" rel="nofollow noreferrer">The Bellman Optimality Principle</a>&quot;</p>
<p>First, if someone knows a clearer proof using any reasonably rigorous methodology, I'm open to reading that one instead. I've only read the first two pages of this one and I already have two questions.  Two pages may not sound like a lot but it's pretty involved so it's much appreciated if anyone can be bothered taking the time required. There are a lot of details.</p>
<p>My two questions are below. Both questions have to do with equation (7) on the second page which the author refers to as the Bellman-Euler equation.</p>
<ol>
<li>What is the justification for using the replacement function</li>
</ol>
<p><span class="math-container">$$ g(t, k_{t}, c_{t}) = k_t + h(t, k_{t}, c_{t}) $$</span></p>
<ol start="2">
<li>Immediately below what is done in question 1.,  the author then defines, <span class="math-container">$\Delta_t \phi$</span> and claims that (8) then reduces to (7). I cannot figure out how (8) reduces to (7). It's just algebra as far as I can tell but my algebra is not leading to (7).</li>
</ol>
<p>I emailed the author and asked him about both questions but he didn't respond.  Again, thanks for any help. Pointing to a different and possibly clearer proof would also be helpful. I'd like to think that I looked pretty hard for a good proof but there could be a better one that I've missed. The author is a finance professor with a Ph.D in mathematics so he could have missed one that lives in the economics world.</p>

<bellman-equations>
two questions regarding a bellman optimality proof
mark leeds
# two questions regarding a bellman optimality proof

(Asked by: username_0 on 2024-07-06)

Hi: I'm reading a proof of "[The Bellman Optimality Principle](https://people.hec.edu/rosu/wp-content/uploads/sites/43/2020/03/bellman.pdf)"

First, if someone knows a clearer proof using any reasonably rigorous methodology, I'm open to reading that one instead. I've only read the first two pages of this one and I already have two questions. Two pages may not sound like a lot but it's pretty involved so it's much appreciated if anyone can be bothered taking the time required. There are a lot of details.

My two questions are below. Both questions have to do with equation (7) on the second page which the author refers to as the Bellman-Euler equation.

1. What is the justification for using the replacement function

$$ g(t, k\_{t}, c\_{t}) = k\_t + h(t, k\_{t}, c\_{t}) $$

2. Immediately below what is done in question 1., the author then defines, $\Delta\_t \phi$ and claims that (8) then reduces to (7). I cannot figure out how (8) reduces to (7). It's just algebra as far as I can tell but my algebra is not leading to (7).

I emailed the author and asked him about both questions but he didn't respond. Again, thanks for any help. Pointing to a different and possibly clearer proof would also be helpful. I'd like to think that I looked pretty hard for a good proof but there could be a better one that I've missed. The author is a finance professor with a Ph.D in mathematics so he could have missed one that lives in the economics world.

**Tags:** bellman-equations


---

## 1 answers

### Answer by username_1 on 2024-07-06. Score: 0 | Accepted answer

The idea behind the function $h(t,k\_t,c\_t)$ is that it gives the increments of capital as it evolves. By definition of $g$, $g(t,k\_t,c\_t)=k\_{t+1}$, so $h(t,k\_t,c\_t)=k\_{t+1}-k\_t$. This $h$ function allows us to make a connection between the discrete and continuous time settings. This connection is highlighted by the author in equations (13) and (14) later on in the text. Without $h$, the connection between discrete and continuous time is unclear since there is no analogous $g$ function in continuous time as there is no "next" time step from the current one. Expressing the discrete time case in terms $h$ rather than $g$ allows us to view the continuous time problem as the "limiting" case of the discrete time problem as time increments become infinitely small.

For point (2), it is just algebra. The equation in (2) is evaluated at the optimum. Given $g(t,k\_t,c\_t)=k\_t+h(t,k\_t,c\_t)$, we have by an envelope argument that $g\_k(t)=1+h\_k(t)$ and $g\_c(t)=h\_c(t)$. Therefore,

$$\begin{align} f\_k(t+1)-\frac{f\_c(t+1)}{g\_c(t+1)}g\_k(t+1)&=-\frac{f\_c(t)}{g\_c(t)} \\
f\_k(t+1)-\frac{f\_c(t+1)}{h\_c(t+1)}[1+h\_k(t+1)]&=-\frac{f\_c(t)}{h\_c(t)} \\
f\_k(t+1)-\frac{f\_c(t+1)}{h\_c(t+1)}h\_k(t+1) &=\frac{f\_c(t+1)}{h\_c(t+1)}-\frac{f\_c(t)}{h\_c(t)} \\
f\_k(t+1)-\frac{f\_c(t+1)}{h\_c(t+1)}h\_k(t+1) &= \Delta\_t\left(\frac{f\_c(t)}{h\_c(t)}\right)\end{align}$$

For alternative references, I think one of the best presentations of recursive methods is in Stokey-Lucas-Prescott "Recursive methods in economic dynamics" (1989). However, the presentation there is, if anything, too maths heavy. But still, it is a great reference text.

> **Comments:**
> **username_0**: Thanks Joseph. I'll check it out carefully and then check it.  I knew (2) was just algebra but I must have been doing something dumb. That's not surprising !!!!!!!!  
> **username_0**: I don't have Stokey-Lucas-Prescott but I'm always hearing about it so maybe that's my hint to purchase it. Thanks again.  
> **username_0**: Joseph: Your answer was very helpful. I still have to figure out what I was doing wrong with Q2 and the answer to Q1 was interesting. I'm looking forward to reading the rest of the paper. All the best.  
> **username_0**: Joseph: I couldn't find my old scribble but I'm pretty sure that I had either $g_k$ or $g_c$ wrong when I did (2).  I just have one more question about this but if I can put it as a seperate question if you want me to ? The very first equation in the paper says max over $c_{t}$. So, when they write that, that seems to imply that there is a different optimal $c_t^{*}$ for each $t$, correct ? So,  the bellman-euler equation in (7) allows one to calculate the optimal $c_t^{*}$ and $k_{t}^{*} ~\forall ~ t$. I hope that's right. Thanks.  
> **username_1**: @markleeds Yes, the $c_t^*$ will vary over time. The idea is that we are given some initial capital level $k_0$. If we also have an initial consumption level $c_0$, we can iterate between the Bellman-Euler equation and the budget constraint to trace out the optimal path (given $(c_0,k_0)$). The initial $c_0$ value is pinned down by the transversality condition which governs the asymptotic behaviour of the optimal path, but $c_0$ usually has no analytic solution.  
> **username_0**: Thanks Joseph. That's very clear. My confuson was that when they do $max(c_t)$, $c_t$ is really the vector of $c_t$s. Got it. Thanks a lot.  

---

