<p>I would leave this as a comment but I cant. You are on the right track.</p>
<ol>
<li><p>Once you know <span class="math-container">$V_2(k)$</span> then you can plug that into to the first hjb and solve.</p>
</li>
<li><p>To solve for <span class="math-container">$V_2$</span> you need to find the optimal <span class="math-container">$i$</span> as a function of <span class="math-container">$k$</span>. Then plug <span class="math-container">$i(k)$</span> into the 2nd HJB. That will give you a second order ode. Solving that will give you <span class="math-container">$V_2(k)$</span> and you go to 1.</p>
</li>
</ol>

CC BY-SA 4.0
2020-07-16T10:59:02.740Z
Thank you a lot for your comment. Unfortunately, I am not sure how to get $i$ as a function of $k$ (the policy function right?). Also, when I "plug" that $V_2$ in $V_1$, should I distinguish the $k$ from $V_2$ from those from $V_1$?
Mr. Fafa
CC BY-SA 4.0
2020-07-15T15:24:47.107Z
2020-07-15T15:24:47.107Z
user28714
<p>Following the answer of user28714, I tried the following.
First, substituting for the FOC, I rewrite <span class="math-container">$V_2$</span> as
<span class="math-container">\begin{align*}
rV_2 &amp;= f(k_t)-i_t - \tau k_t + i_t-\delta k_t \\
&amp;= f(k_t - \tau k_t - \delta k_t
\end{align*}</span>
Thus, we get
<span class="math-container">$$ V_2 = \frac{1}{r}\left(f(k_t) - k_t(\tau + \delta) \right)$$</span>
Substituting in <span class="math-container">$V_1$</span>, we get
<span class="math-container">$$ rV_1 = \max_{i} \left\{ f(k_t)-i_t + \rho\left(\frac{1}{r}\left(f(k_t) - k_t(\tau + \delta) \right)-V_1\right) + V'_1(i_t-\delta k_t)   \right\}$$</span></p>
<p>The FOC is unchanged:
<span class="math-container">$ V'_1=1$</span>, and the envelope condition becomes
<span class="math-container">\begin{align*}
rV'_1 = f'(k_t)+\rho\left(\frac{1}{r}(f'(k_t)-\tau - \delta)-V'_1\right)+V''_1(i_t-\delta k_t) - \delta V'_1
\end{align*}</span>
Noting that <span class="math-container">$\dot{V'_1} = V''_1 (i_t-\delta k_t)$</span> and substituting using the envelope condition, we find
<span class="math-container">\begin{align*}
\dot{V_1}=V'_1(r+\delta+\rho)-f'(k_t)-\frac{\rho}{r}(f'(k_t)-\tau - \delta)
\end{align*}</span>
Using <span class="math-container">$V'_1=1$</span> and <span class="math-container">$\dot{V'_1}=0$</span>, we get
<span class="math-container">\begin{align*}
f'(k_t)(1+\frac{\rho}{r})&amp;= r+\delta +\rho +\frac{\rho}{r}(\tau + \delta) \\
f'(k_t) &amp;= \frac{r}{r+\rho}\left( r+\delta +\rho +\frac{\rho}{r}(\tau + \delta)  \right) \\
f'(k_t) &amp;= r + \delta + \frac{\rho }{r+\rho}(r+ \frac{\rho}{r}\tau)
\end{align*}</span></p>
<p>Which is not the most elegant result... Could somebody confirm me this result?</p>

CC BY-SA 4.0
2020-07-18T19:26:45.833Z
2020-07-22T09:48:49.250Z
2020-07-22T09:48:49.250Z
Mr. Fafa
<p>I am trying to solve the problem of a firm facing the possibility of a future tax, in continuous time.</p>
<p>The firm maximizes <span class="math-container">$V(k)=\int_{t=0}^{\infty}e^{-rt} \pi_t dt$</span> with <span class="math-container">$\pi_t=f(k_t)-i_t$</span> and <span class="math-container">$\dot{k}=i_t-\delta k_t$</span>. There is a probability <span class="math-container">$\rho$</span> per unit of time to transit to a new state where a tax is imposed, and where the profit becomes <span class="math-container">$f(k_t)-i_t-\tau k_t$</span>. This is a partial equilibrium problem and we assume that <span class="math-container">$r$</span>, <span class="math-container">$\delta$</span> and <span class="math-container">$\rho$</span> are exogenous.</p>
<p>I am trying to solve this problem using Hamiltonian-Jacobi-Bellman (HJB) functions:
<span class="math-container">$$\begin{align*}
rV_1 &amp;= \max_{i} \{f(k_t)-i_t+\rho (V_2-V_1)+\dot{V}_1\} \\
rV_2 &amp;= \max_{i} \{f(k_t)-i-\tau k_{t}+\dot{V}_2\}
\end{align*}$$</span></p>
<p>I know how to solve the second equation, following the method of <a href="https://www.waelde.com/pdf/AIO.pdf" rel="nofollow noreferrer">Walde 2012</a> using dynamic programming. First, rewrite <span class="math-container">$$f(k_t)-i_t-\tau k_t + V'_2(k_t)\dot{k_t}=f(k_t)-i_t-\tau k_t+ V'_2(k_t)(i_t-\delta k_t)$$</span> Then, take the FOC with respect to <span class="math-container">$i_t$</span>, which yields
<span class="math-container">$$V'_2(k_t)=1$$</span>
Then, use the envelope condition to find
<span class="math-container">\begin{align*}
rV'_2=f'(k_t)-\tau+V''_2(k_t)(i_t-\delta k_t)-\delta V'_2(k_t)
\end{align*}</span>
This can be simplified, using the FOC, as<span class="math-container">$$\begin{align*}
f'(k_t)=r+\delta+\tau
\end{align*}$$</span>
which yields the same solution as a simple Hamiltonian would.</p>
<p>However, I am unsure how to proceed next and how to solve for the solution before the tax is imposed and the uncertainty resolved.</p>
<p>Additionally, if you have some references regarding dynamic control in continuous time, I would be very interested, especially if they treat the case of HJB with additional constraints.</p>
<p>Thank you in advance for your help!</p>
<p>EDIT: for clarification, once we reach the new state where the tax is imposed, there is no possibility to go back to the previous state. The only uncertainty is about when the tax will be imposed, i.e, when the uncertainty will be resolved.</p>

CC BY-SA 4.0
2020-07-08T08:03:03.590Z
Hamiltonain is a physics concept and you are not even applying it correctly. Are you forced to do this? You could simply take the current utility function and integrate over time, with a odds parameter. You also need time discounting. How was this ignored?
D J Sims
CC BY-SA 4.0
2020-07-09T05:29:39.730Z
Hi. My previous comment was deleted when your post was changed into a comment, but:
1. This is not the same Hamiltonian as in physics, this is the Pontryagin's maximum principle, which is not what I try to use here. I try to use the Hamiltonian-Jacobi-Bellman equation instead to solve this problem as it makes stochastic formulations easier.
2. There is indeed time discounting: the firm exponentially discount future profit at the exogenous interest rate $r$, which, in the HJB, is apparent through the $rV$. You can check Walde 2012 for the derivation of the HJB.
Mr. Fafa
<mathematical-economics><optimization><dynamic-programming><dynamic-optimization><continuous-time>
Solving a HJB with a probability to transit to a new state
Mr. Fafa
# Solving a HJB with a probability to transit to a new state

(Asked by: username_0 on 2020-07-07)

I am trying to solve the problem of a firm facing the possibility of a future tax, in continuous time.

The firm maximizes $V(k)=\int\_{t=0}^{\infty}e^{-rt} \pi\_t dt$ with $\pi\_t=f(k\_t)-i\_t$ and $\dot{k}=i\_t-\delta k\_t$. There is a probability $\rho$ per unit of time to transit to a new state where a tax is imposed, and where the profit becomes $f(k\_t)-i\_t-\tau k\_t$. This is a partial equilibrium problem and we assume that $r$, $\delta$ and $\rho$ are exogenous.

I am trying to solve this problem using Hamiltonian-Jacobi-Bellman (HJB) functions:
$$\begin{align\*}
rV\_1 &= \max\_{i} \{f(k\_t)-i\_t+\rho (V\_2-V\_1)+\dot{V}\_1\} \\
rV\_2 &= \max\_{i} \{f(k\_t)-i-\tau k\_{t}+\dot{V}\_2\}
\end{align\*}$$

I know how to solve the second equation, following the method of [Walde 2012](https://www.waelde.com/pdf/AIO.pdf) using dynamic programming. First, rewrite $$f(k\_t)-i\_t-\tau k\_t + V'\_2(k\_t)\dot{k\_t}=f(k\_t)-i\_t-\tau k\_t+ V'\_2(k\_t)(i\_t-\delta k\_t)$$ Then, take the FOC with respect to $i\_t$, which yields
$$V'\_2(k\_t)=1$$
Then, use the envelope condition to find
\begin{align\*}
rV'\_2=f'(k\_t)-\tau+V''\_2(k\_t)(i\_t-\delta k\_t)-\delta V'\_2(k\_t)
\end{align\*}
This can be simplified, using the FOC, as$$\begin{align\*}
f'(k\_t)=r+\delta+\tau
\end{align\*}$$
which yields the same solution as a simple Hamiltonian would.

However, I am unsure how to proceed next and how to solve for the solution before the tax is imposed and the uncertainty resolved.

Additionally, if you have some references regarding dynamic control in continuous time, I would be very interested, especially if they treat the case of HJB with additional constraints.

Thank you in advance for your help!

EDIT: for clarification, once we reach the new state where the tax is imposed, there is no possibility to go back to the previous state. The only uncertainty is about when the tax will be imposed, i.e, when the uncertainty will be resolved.

**Tags:** mathematical-economics, optimization, dynamic-programming, dynamic-optimization, continuous-time

> **Comments:**
> **username_1**: Hamiltonain is a physics concept and you are not even applying it correctly. Are you forced to do this? You could simply take the current utility function and integrate over time, with a odds parameter. You also need time discounting. How was this ignored?  
> **username_0**: Hi. My previous comment was deleted when your post was changed into a comment, but: 1. This is not the same Hamiltonian as in physics, this is the Pontryagin's maximum principle, which is not what I try to use here. I try to use the Hamiltonian-Jacobi-Bellman equation instead to solve this problem as it makes stochastic formulations easier. 2. There is indeed time discounting: the firm exponentially discount future profit at the exogenous interest rate $r$, which, in the HJB, is apparent through the $rV$. You can check Walde 2012 for the derivation of the HJB.  

---

## 2 answers

### Answer by username_2 on 2020-07-15. Score: 2

I would leave this as a comment but I cant. You are on the right track.

1. Once you know $V\_2(k)$ then you can plug that into to the first hjb and solve.
2. To solve for $V\_2$ you need to find the optimal $i$ as a function of $k$. Then plug $i(k)$ into the 2nd HJB. That will give you a second order ode. Solving that will give you $V\_2(k)$ and you go to 1.

> **Comments:**
> **username_0**: Thank you a lot for your comment. Unfortunately, I am not sure how to get $i$ as a function of $k$ (the policy function right?). Also, when I "plug" that $V_2$ in $V_1$, should I distinguish the $k$ from $V_2$ from those from $V_1$?  

---

### Answer by username_0 on 2020-07-18. Score: 0

Following the answer of username_2, I tried the following.
First, substituting for the FOC, I rewrite $V\_2$ as
\begin{align\*}
rV\_2 &= f(k\_t)-i\_t - \tau k\_t + i\_t-\delta k\_t \\
&= f(k\_t - \tau k\_t - \delta k\_t
\end{align\*}
Thus, we get
$$ V\_2 = \frac{1}{r}\left(f(k\_t) - k\_t(\tau + \delta) \right)$$
Substituting in $V\_1$, we get
$$ rV\_1 = \max\_{i} \left\{ f(k\_t)-i\_t + \rho\left(\frac{1}{r}\left(f(k\_t) - k\_t(\tau + \delta) \right)-V\_1\right) + V'\_1(i\_t-\delta k\_t) \right\}$$

The FOC is unchanged:
$ V'\_1=1$, and the envelope condition becomes
\begin{align\*}
rV'\_1 = f'(k\_t)+\rho\left(\frac{1}{r}(f'(k\_t)-\tau - \delta)-V'\_1\right)+V''\_1(i\_t-\delta k\_t) - \delta V'\_1
\end{align\*}
Noting that $\dot{V'\_1} = V''\_1 (i\_t-\delta k\_t)$ and substituting using the envelope condition, we find
\begin{align\*}
\dot{V\_1}=V'\_1(r+\delta+\rho)-f'(k\_t)-\frac{\rho}{r}(f'(k\_t)-\tau - \delta)
\end{align\*}
Using $V'\_1=1$ and $\dot{V'\_1}=0$, we get
\begin{align\*}
f'(k\_t)(1+\frac{\rho}{r})&= r+\delta +\rho +\frac{\rho}{r}(\tau + \delta) \\
f'(k\_t) &= \frac{r}{r+\rho}\left( r+\delta +\rho +\frac{\rho}{r}(\tau + \delta) \right) \\
f'(k\_t) &= r + \delta + \frac{\rho }{r+\rho}(r+ \frac{\rho}{r}\tau)
\end{align\*}

Which is not the most elegant result... Could somebody confirm me this result?


---

