<p>The Dynamic Programming ("Bellman' Equation") formulation <em>incorporates</em> the terminal boundary condition ("transversality conditions") needed in case we use the Lagrangian/Euler equation formulation. The initial conditions are still needed in both approaches. </p>

<p>To see why, consider the problem</p>

<p>$$\text{max}_{\{x_t\}} \sum_{t=0}^{\infty} \beta^t U(c_t)$$</p>

<p>$$s.t.\;\;\; a_{t+1}= (1-r)a_t + w - c_t, \;\;\; a_0 \; \text{given}$$</p>

<p>Solving the budget constraint for $c_t$ and inserting into the objective function, we have compactly</p>

<p>$$\text{max}_{\{a_{t}\}} \sum_{t=0}^{\infty} \beta^t U(a_t, a_{t+1}),\;\;\;\; a_0 \; \text{given}$$</p>

<p>where we treat the stock variable next period as our decision variable.
Now for the optimal sequence, $a^*_{t+1}$ must solve the following problem:</p>

<p>$$\max_y \big[U(a^*_t, y) + \beta U(y, a^*_{t+2})\big],\;\;\; y\;\; \text{feasible given}\;\; a^*_t$$</p>

<p>In the words of Stokey and Lucas <em>"a feasible variation on the sequence $\{a^*_t\}$ cannot lead to an improvement on an optimal policy"</em>.  </p>

<p>Then we get the first order conditions (differentiating with respect to $y$ essentially)</p>

<p>$$U_2(a^*_t, a^*_{t+1}) + \beta U_1(a^*_{t+1}, a^*_{t+2}) =0$$</p>

<p>where the subscripts $1,2$ denote partial derivatives. This is a <em>second-order</em> difference equation, and so it requires <em>two</em> boundary conditions. We have already one of them, the given initial value of wealth $a_0$. We need one more, hence the terminal, "transversality" condition. </p>

<p>In contrast, using the Dynamic Programming approach we arrive at a <em>first-order</em> difference equation, and we do not need an additional boundary condition.</p>

<p>For the full exposition, see <a href="http://rads.stackoverflow.com/amzn/click/0674750969" rel="noreferrer">Stokey &amp; Lucas</a> p. 97-100.</p>

CC BY-SA 3.0
2016-11-19T00:18:38.630Z
2016-11-19T00:18:38.630Z
Alecos Papadopoulos
<p>I'm unsure exactly where your struggle is, in general. To try and address your issues, what kind of boundary conditions might we desire in dynamic problems?</p>

<p>Consider the two period consumption-savings problem where we have</p>

<p>$$c_1 + s_1 = y_1$$
$$c_2 = y_2 + (1+R)s_1$$</p>

<p>$y_1, y_2$ being endowments in each period, $s_1$ as savings, $R$ for interest, and $c_1, c_2$ for consumption. Given some $U(c_1, c_2)$, we want to solve for optimal consumption in each period. We can consolidate the budget as:</p>

<p>$$c_2 = y_2 + (1 + R)(y_1 - c_1)$$
$$\implies c_1 + \frac{c_2}{1 + R} = y_1 + \frac{y_2}{1 + R}$$</p>

<p>and express the Lagrangian problem:</p>

<p>$$\mathcal{L} = U(c_1, c_2) - \lambda(c_1 - \frac{c_2}{1 + R} - y_1 - \frac{y_2}{1 + R})$$</p>

<p>Solving for FOCs gets us:</p>

<p>$$U_{c_1} - \lambda = 0$$
$$U_{c_2} - \frac{\lambda}{1 + R} = 0$$
$$\implies \frac{U_{c_1}}{U_{c_2}} = 1 + R$$</p>

<p>But you can also express this problem as a Bellman, and still keep that original consolidated budget constraint.</p>

<p>$$\max_{c_1, c_2, y_1, y_2} U(c_1, c_2) \quad \text{s.t.} \ c_1 + \frac{c_2}{1 + R} = y_1 + \frac{y_2}{1 + R}$$</p>

<p>where you have a value function for $y_1$ (yes, $y_1$ is still given)</p>

<p>$$V(y_1) = \max_{c_1, c_2, y_2} \ U(c_1, c_2) \quad \text{s.t.} \ y_1 = c_1 + \frac{c_2}{1 + R} - \frac{y_2}{1 + R}$$</p>

<p>To make this problem less trivial for a Bellman, you can have a constraint where $c$ determines $y$ somehow instead. But otherwise you can still incorporate the budget constraint into your problem here, albeit it might look pretty weird.</p>

CC BY-SA 3.0
2016-11-18T08:05:21.073Z
2016-11-18T08:05:21.073Z
Kitsune Cavalry
<p>Consider a simple dynamic consumption-saving problem. A solution can be characterised using a Lagrangian approach that generates a set of first order conditions and some boundary conditions. </p>

<p>An alternative approach is to set use a Bellman equation. What I don't understand is how come a Bellman equation approach does not seem to use information regarding the boundary conditions in the solution methodology? That is, a Bellman equation does have a tradeoff between time period t and t+1 but not some overall budget constraint does not seem to be represented. </p>

CC BY-SA 3.0
2016-11-18T07:37:05.660Z
Are you using a transversality condition? Something that tells you what happens as $\lim\limits_{t \to \infty}$?
Giskard
CC BY-SA 3.0
2016-11-18T08:07:22.437Z
Same question as above. In an infinite period model, transversality conditions are desireable if you want non trivial solutions. If you (the asker) assumes there are those conditions, then I'm not sure what the issue is.
Kitsune Cavalry
<dynamic-optimization>
Dynamic optimisation
econovan
# Dynamic optimisation

(Asked by: username_0 on 2016-11-17)

Consider a simple dynamic consumption-saving problem. A solution can be characterised using a Lagrangian approach that generates a set of first order conditions and some boundary conditions.

An alternative approach is to set use a Bellman equation. What I don't understand is how come a Bellman equation approach does not seem to use information regarding the boundary conditions in the solution methodology? That is, a Bellman equation does have a tradeoff between time period t and t+1 but not some overall budget constraint does not seem to be represented.

**Tags:** dynamic-optimization

> **Comments:**
> **username_1**: Are you using a transversality condition? Something that tells you what happens as $\lim\limits_{t \to \infty}$?  
> **username_2**: Same question as above. In an infinite period model, transversality conditions are desireable if you want non trivial solutions. If you (the asker) assumes there are those conditions, then I'm not sure what the issue is.  

---

## 2 answers

### Answer by username_3 on 2016-11-19. Score: 9

The Dynamic Programming ("Bellman' Equation") formulation *incorporates* the terminal boundary condition ("transversality conditions") needed in case we use the Lagrangian/Euler equation formulation. The initial conditions are still needed in both approaches.

To see why, consider the problem

$$\text{max}\_{\{x\_t\}} \sum\_{t=0}^{\infty} \beta^t U(c\_t)$$

$$s.t.\;\;\; a\_{t+1}= (1-r)a\_t + w - c\_t, \;\;\; a\_0 \; \text{given}$$

Solving the budget constraint for $c\_t$ and inserting into the objective function, we have compactly

$$\text{max}\_{\{a\_{t}\}} \sum\_{t=0}^{\infty} \beta^t U(a\_t, a\_{t+1}),\;\;\;\; a\_0 \; \text{given}$$

where we treat the stock variable next period as our decision variable.
Now for the optimal sequence, $a^\*\_{t+1}$ must solve the following problem:

$$\max\_y \big[U(a^\*\_t, y) + \beta U(y, a^\*\_{t+2})\big],\;\;\; y\;\; \text{feasible given}\;\; a^\*\_t$$

In the words of Stokey and Lucas *"a feasible variation on the sequence $\{a^\*\_t\}$ cannot lead to an improvement on an optimal policy"*.

Then we get the first order conditions (differentiating with respect to $y$ essentially)

$$U\_2(a^\*\_t, a^\*\_{t+1}) + \beta U\_1(a^\*\_{t+1}, a^\*\_{t+2}) =0$$

where the subscripts $1,2$ denote partial derivatives. This is a *second-order* difference equation, and so it requires *two* boundary conditions. We have already one of them, the given initial value of wealth $a\_0$. We need one more, hence the terminal, "transversality" condition.

In contrast, using the Dynamic Programming approach we arrive at a *first-order* difference equation, and we do not need an additional boundary condition.

For the full exposition, see [Stokey & Lucas](http://rads.stackoverflow.com/amzn/click/0674750969) p. 97-100.


---

### Answer by username_2 on 2016-11-18. Score: 4

I'm unsure exactly where your struggle is, in general. To try and address your issues, what kind of boundary conditions might we desire in dynamic problems?

Consider the two period consumption-savings problem where we have

$$c\_1 + s\_1 = y\_1$$
$$c\_2 = y\_2 + (1+R)s\_1$$

$y\_1, y\_2$ being endowments in each period, $s\_1$ as savings, $R$ for interest, and $c\_1, c\_2$ for consumption. Given some $U(c\_1, c\_2)$, we want to solve for optimal consumption in each period. We can consolidate the budget as:

$$c\_2 = y\_2 + (1 + R)(y\_1 - c\_1)$$
$$\implies c\_1 + \frac{c\_2}{1 + R} = y\_1 + \frac{y\_2}{1 + R}$$

and express the Lagrangian problem:

$$\mathcal{L} = U(c\_1, c\_2) - \lambda(c\_1 - \frac{c\_2}{1 + R} - y\_1 - \frac{y\_2}{1 + R})$$

Solving for FOCs gets us:

$$U\_{c\_1} - \lambda = 0$$
$$U\_{c\_2} - \frac{\lambda}{1 + R} = 0$$
$$\implies \frac{U\_{c\_1}}{U\_{c\_2}} = 1 + R$$

But you can also express this problem as a Bellman, and still keep that original consolidated budget constraint.

$$\max\_{c\_1, c\_2, y\_1, y\_2} U(c\_1, c\_2) \quad \text{s.t.} \ c\_1 + \frac{c\_2}{1 + R} = y\_1 + \frac{y\_2}{1 + R}$$

where you have a value function for $y\_1$ (yes, $y\_1$ is still given)

$$V(y\_1) = \max\_{c\_1, c\_2, y\_2} \ U(c\_1, c\_2) \quad \text{s.t.} \ y\_1 = c\_1 + \frac{c\_2}{1 + R} - \frac{y\_2}{1 + R}$$

To make this problem less trivial for a Bellman, you can have a constraint where $c$ determines $y$ somehow instead. But otherwise you can still incorporate the budget constraint into your problem here, albeit it might look pretty weird.


---

