<p>There are two interrelated maximisation problems. The first is the infinite horizon maximisation problem:
<span class="math-container">$$
\begin{align*}
v(k) = &amp;\max_{a_1, a_2, \ldots} \sum_{t = 0}^\infty \delta^t F(k_t, c_t),\\
\text{ subject to } &amp; k_{t+1} = g(k_t, a_t),\\
&amp; a_t = \Gamma(k_t),\\
&amp; k_0 = k
\end{align*}
$$</span>
Here we call <span class="math-container">$a_t$</span> the decision variables, <span class="math-container">$k_t$</span> the state variables. This problem maximises an infinite sum of discounted values <span class="math-container">$F(k_t, c_t)$</span>, subject to a law of motion that determines the next periods state depending on the action and state today. <span class="math-container">$\Gamma(k)$</span> gives a set of feasible actions <span class="math-container">$a$</span> that can be taken and finally <span class="math-container">$k$</span> is set equal to the initial state.</p>
<p>As such, <span class="math-container">$v(k)$</span> is the value of this optimisation problem when the initial state is <span class="math-container">$k$</span>.</p>
<p>The second problem is the Bellman equation:
<span class="math-container">$$
v(k) = \max_{a \in \Gamma(k)}\left\{F(k,a) + \delta v(g(k,a))\right\}.
$$</span>
You should interpret this as an identity involving the function <span class="math-container">$v$</span>, which appears both on the left hand right hand side of the equation, so the function <span class="math-container">$v(.)$</span> is the unknown in this equation (which has to hold for all <span class="math-container">$k$</span>).</p>
<p>Under some conditions it can be shown that for every <span class="math-container">$k$</span>, the function <span class="math-container">$v(k)$</span> of the first problem is equal to the value of the <span class="math-container">$v(k)$</span> that satisfies the second equation.</p>
<p>In order to find this function <span class="math-container">$v(.)$</span> one can define the Bellman operator <span class="math-container">$T$</span>:
<span class="math-container">$$
(Tv)(k) = \max_{a \in \Gamma(k)}\left\{F(k,a) + \delta v(g(k,a))\right\}.
$$</span>
The operator <span class="math-container">$T$</span> takes a function <span class="math-container">$v$</span> (on the right hand side) and produces a new function <span class="math-container">$Tv$</span>.</p>
<p>Again under suitable conditions, one can show that this operator is a contraction mapping. So by iterating this function over and over again we will converge to the fixed point of this operator, which then also gives the solution to the Bellman equation.</p>
<blockquote>
<ol>
<li>How do I interpret the meaning of <span class="math-container">$V$</span>? For example, when we decide a investment level in next time <span class="math-container">$k_{t+1}$</span> given the existing value <span class="math-container">$k_t$</span>, we could calculate a value <span class="math-container">$v(k_t)$</span>. Is this value equal the value given solving maximization problem after <span class="math-container">$t (t+1,t+2⋯)$</span>?</li>
</ol>
</blockquote>
<p>Yes by definition of the first problem, <span class="math-container">$v(k_t)$</span> gives the value of the infinite horizon maximisation problem when <span class="math-container">$k_t$</span> is the initial level of capital.</p>
<blockquote>
<ol start="2">
<li>If the interpretation is correct, why a fixed point procedure yield such a maximized function?</li>
</ol>
</blockquote>
<p>This results from the equivalence between the first optimisation problem and the Bellman equation. The solution <span class="math-container">$v$</span> of the Bellman equation is a fixed point of the Bellman operator.</p>

CC BY-SA 4.0
2021-09-12T08:01:21.783Z
2021-09-12T08:01:21.783Z
tdm
<p>Using an infinite horizon model, a dynamic programming approach uses a fixed point to solve the model: <span class="math-container">$V = \Gamma(V)$</span>.</p>
<ol>
<li><p>How do I interpret the meaning of <span class="math-container">$V$</span>? For example, when we decide a investment level in next time <span class="math-container">$k_{t+1}$</span> given the existing value <span class="math-container">$k_t$</span>, we could calculate a value <span class="math-container">$v(k_t)$</span>. Is this value equal the value given solving maximization problem after <span class="math-container">$t$</span> (<span class="math-container">$t+1, t+2 \cdots$</span>)?</p>
</li>
<li><p>If the interpretation is correct, why a fixed point procedure yield such a maximized function?</p>
</li>
</ol>
<p>Although I checked several lecture notes, I could not get the intuition by words about these points.</p>

CC BY-SA 4.0
2021-09-11T20:36:59.180Z
My (almost forgotten) understanding of functional analysis and dynamic programming is tempted to say that your interpretation of (1) is correct. V is the maximised function of the state. Yet in infinite horizon (2), the max is not guaranteed to exist. But nevertheless, our model requires some type of convergent solution - and this is the fixed point, expressed in terms of something like Banach's fixed point theorem/contraction mapping theorem and Blackwell's sufficiency conditions etc.
EB3112
<dynamic-programming><bellman-equations>
Dynamic programming in infinite horizon model
hrkshr
# Dynamic programming in infinite horizon model

(Asked by: username_0 on 2021-09-11)

Using an infinite horizon model, a dynamic programming approach uses a fixed point to solve the model: $V = \Gamma(V)$.

1. How do I interpret the meaning of $V$? For example, when we decide a investment level in next time $k\_{t+1}$ given the existing value $k\_t$, we could calculate a value $v(k\_t)$. Is this value equal the value given solving maximization problem after $t$ ($t+1, t+2 \cdots$)?
2. If the interpretation is correct, why a fixed point procedure yield such a maximized function?

Although I checked several lecture notes, I could not get the intuition by words about these points.

**Tags:** dynamic-programming, bellman-equations

> **Comments:**
> **username_1**: My (almost forgotten) understanding of functional analysis and dynamic programming is tempted to say that your interpretation of (1) is correct. V is the maximised function of the state. Yet in infinite horizon (2), the max is not guaranteed to exist. But nevertheless, our model requires some type of convergent solution - and this is the fixed point, expressed in terms of something like Banach's fixed point theorem/contraction mapping theorem and Blackwell's sufficiency conditions etc.  

---

## 1 answers

### Answer by username_2 on 2021-09-12. Score: 5 | Accepted answer

There are two interrelated maximisation problems. The first is the infinite horizon maximisation problem:
$$
\begin{align\*}
v(k) = &\max\_{a\_1, a\_2, \ldots} \sum\_{t = 0}^\infty \delta^t F(k\_t, c\_t),\\
\text{ subject to } & k\_{t+1} = g(k\_t, a\_t),\\
& a\_t = \Gamma(k\_t),\\
& k\_0 = k
\end{align\*}
$$
Here we call $a\_t$ the decision variables, $k\_t$ the state variables. This problem maximises an infinite sum of discounted values $F(k\_t, c\_t)$, subject to a law of motion that determines the next periods state depending on the action and state today. $\Gamma(k)$ gives a set of feasible actions $a$ that can be taken and finally $k$ is set equal to the initial state.

As such, $v(k)$ is the value of this optimisation problem when the initial state is $k$.

The second problem is the Bellman equation:
$$
v(k) = \max\_{a \in \Gamma(k)}\left\{F(k,a) + \delta v(g(k,a))\right\}.
$$
You should interpret this as an identity involving the function $v$, which appears both on the left hand right hand side of the equation, so the function $v(.)$ is the unknown in this equation (which has to hold for all $k$).

Under some conditions it can be shown that for every $k$, the function $v(k)$ of the first problem is equal to the value of the $v(k)$ that satisfies the second equation.

In order to find this function $v(.)$ one can define the Bellman operator $T$:
$$
(Tv)(k) = \max\_{a \in \Gamma(k)}\left\{F(k,a) + \delta v(g(k,a))\right\}.
$$
The operator $T$ takes a function $v$ (on the right hand side) and produces a new function $Tv$.

Again under suitable conditions, one can show that this operator is a contraction mapping. So by iterating this function over and over again we will converge to the fixed point of this operator, which then also gives the solution to the Bellman equation.

> 1. How do I interpret the meaning of $V$? For example, when we decide a investment level in next time $k\_{t+1}$ given the existing value $k\_t$, we could calculate a value $v(k\_t)$. Is this value equal the value given solving maximization problem after $t (t+1,t+2⋯)$?

Yes by definition of the first problem, $v(k\_t)$ gives the value of the infinite horizon maximisation problem when $k\_t$ is the initial level of capital.

> 2. If the interpretation is correct, why a fixed point procedure yield such a maximized function?

This results from the equivalence between the first optimisation problem and the Bellman equation. The solution $v$ of the Bellman equation is a fixed point of the Bellman operator.


---

