Dynamic Programming in the One-Sector Growth Model

\[ \definecolor{name}{RGB}{200,200,210} \newcommand{\condition}[1]{\color{name} \text{(#1)}} \newcommand{\mytag}[1]{\tag*{\(\condition{#1}\)}} \]

Dynamic Programming in the One-Sector Growth Model

Setup:

Consider the following Planner's Problem: \[\max \sum \beta^t u(c_t,l_t)\]
subject to
\begin{gather*} c_t + x_{t} \leq f(k_t,n_t) \\ k_{t+1} \leq (1-\delta ) k_t + x_{t} \\ l_t + n_t \leq 1 \\ k_0 \text{ given.} \end{gather*}

Write this problem as a dynamic programming problem in sequence form, that is, as maximizing an infinite discounted sum of a time-stationary return function, with a time-stationary constraint set.

Define the following set of feasible continuation allocations: \[\Gamma(k) \equiv \left\{ (c,n,l,k^\prime,x) \in \mathbb{R}_+^4 \times \mathbb{R} \middle| \begin{gather} c + x \leq f(k,n) \\ k^\prime \leq (1-\delta) k + x \\ l + n \leq 1 \\ \end{gather} \right\} \]

The sequential programming problem is \[ V(k_0) = \max_{\left\{c_t,n_t, l_t, k_{t+1}, x_t \right\}_t} \sum_{t=0}^\infty \beta^t u(c_t, l_t) \]

such that for all \(t\):

\[ (c_t,n_t, l_t, k_{t+1}, x_t) \in \Gamma(k_t)\]

Write the functional equation (Bellman's equation) for the problem above.

The Bellman's equation is: \[V(k) = \max_{(c,n,l,k^\prime,x)\in \Gamma(k)} \left[u(c,l) + \beta V(k^\prime) \right]\]

Provide assumptions on \(u, f, \beta\), and \(delta\) that are sufficient to guarantee that there is a unique bounded and continuous function satisfying the functional equation. Explain briefly why each assumption is needed.

Assume that \(\beta\in(0,1)\). Assume that \(\sigma\in (0,1]\).

Assume that \(u\) is strictly increasing, and continuous.

Assume that f is strictly increasing, continuous, \(f(0)=0\), and there is some maximimum sustainable capital, \(\hat{k}\)

The assumptions on \(f\) and \(\delta\) allow us to conclude that capital is bounded (in \([0,\max{\hat{k},k_0]\)). This, along with the budget constraint, allows us to conclude that consumption is bounded. Then the assumptions on \(u\) allow us to conclude that per-period utility is bounded as well. Finally, with \(\beta\in(0,1)\), we can then conclude that \(v(k_0)\) is bounded.

The boundedness of \(v(k_0)\) allows us to safely apply the sup norm and so use Blackwell's sufficient conditions (the problem clearly satisfies monotonicity and discounting) to conclude that there is a unique fixed point \(v\)

The Theorem of the maximum establishes the continuousness.

Assume the labor supply is inelastic. Write the functional equation and characterize the optimal policy function through the first-order and envelope condition.

With inelastic labor supply, \(n_t = \bar{n}\). For simplicity, let \(u(c)\equiv u(c,0\) and let \(f(k)\equiv f(k,\bar{n}\). Additionally, if we assume that the utility and production functions are strictly increasing in consumption and capital, respectively, then the resource and investment constraints will bind. Combining them yields: \[c_t = f(k_t) +(1-\delta)k_t - k_{t+1}\]

Redefine the set of feasible continuation allocations: \[\dot{\Gamma}(k) \equiv [0, f(k_t) +(1-\delta)k_t]\]

Now the functional equation is \[ V(k) = \max_{k^\prime \in \dot{\Gamma}(k)} \left[ u(f(k) +(1-\delta)k - k^\prime) + \beta V(k^\prime) \right] \]

Taking the First Order condition of this, we get that at the optimum: \[0 = {\partial \over \partial k^\prime} \left[ u(f(k) +(1-\delta)k - k^\prime) + \beta V(k^\prime) \right] = -u^\prime(f(k) +(1-\delta)k - k^\prime) + \beta V^\prime(k^\prime) \]

Let \(k^\prime(k)\) be the policy function for \(k^\prime\).Then the envelope condition is: \[ V^\prime(k) = {\partial \over \partial k} \left[ u(f(k) +(1-\delta)k - k^\prime(k)) + \beta V(k^\prime(k)) \right] = u'(f(k) +(1-\delta)k - k^\prime(k)) (f^\prime(k) + 1 - \delta ) + {\partial \left[ u(f(k) +(1-\delta)k - k^\prime(k)) + \beta V(k^\prime(k)) \right]\over \partial k^\prime}{\partial k^\prime \over \partial k} \] And in the optimum, the FOC tells us that the last term is equal to zero, so the Envelope condition is: \[ V^\prime(k) = u'(f(k) +(1-\delta)k - k^\prime(k)) (f^\prime(k) + 1 - \delta ) \]

TODO: characterize by combinnigining

Define \(✉(k,k^\prime) \equiv \left[ u(f(k) +(1-\delta)k - k^\prime) + \beta V(k^\prime) \right] \) Now the functional equation is \[ V(k) = \max_{k^\prime \in \dot{\Gamma}(k)} ✉(k,k^\prime) \] FOC: \[0 = {\partial \over \partial k^\prime}✉(k,k^\prime)\] Envelope: \[{\partial \over \partial k} V(k) = {\partial \over \partial k} ✉(k,k^\prime(k)) = {\partial \over \partial k} ✉(k,k^\prime) + {\partial \over \partial k^\prime} ✉(k,k^\prime) {\partial k^\prime \over \partial k} ✉(k,k^\prime) = {\partial \over \partial k} ✉(k,k^\prime) \]

(a) Show that the value function for this problem is homogeneous of degree \((1-\sigma)\) in the initial capital stocks, \((h_0, k_0)\).

(b) Characterize the homogeneity properties of the optimal decision rules in initial capital stocks, \((h_0, k_0)\). Specifically if the initial conditions on the capital stocks are \((\eta h_0, \eta k_0)\) instead of \((h_0, k_0)\) where \(\eta > 0\), how are the optimal time paths for consumption, labor supply, investment, etc., affected? Prove your claims.

The easy but tedious way:

Assume that \(\alpha, \beta \in (0,1)\), \(\delta_k, \delta_h \in (0,1]\), and \(\sigma,A,h_0,k_0 > 0\).

Because leisure does not enter into the utility function, we can safely assume that \(n_t=1 \; \forall t\), and because the utility and production function are strictly increasing, the other constraints will bind. This suggests some obvious simplifications to to make in the constraints via the following substitutions: \begin{align} z_t & = h_t \\ x_{kt} & = k_{t+1} - (1-\delta_k ) k_t \\ x_{ht} & = h_{t+1} - (1-\delta_h ) h_t \\ \end{align}

Then the value function for the planner's problem above is equivalent to the following: \[V(k_0, h_0) = \max_{\{c_t,k_{t+1},h_{t+1}\}_t} \sum_{t=0}^\infty \beta^t {c_t^{1-\sigma} \over 1-\sigma}\]

such that, for all \(t\),

\begin{gather*} c_t + k_{t+1} - (1-\delta_k ) k_t + h_{t+1} - (1-\delta_h ) h_t = Ak_t^\alpha h_t^{1-\alpha} \\ h_0, k_0 \text{ given.} \end{gather*}

The first order conditions for this simplified problem are: \begin{align} 0 & = \beta^t c_t^{-\sigma} - \lambda_t \\ 0 & = - \lambda_t + (1-\delta_k )\lambda_{t+1} + Ak_{t+1}^{\alpha-1}h_{t+1}^{1-\alpha} \lambda_{t+1} \\ 0 & = - \lambda_t + (1-\delta_h )\lambda_{t+1} + Ak_{t+1}^{\alpha}h_{t+1}^{-\alpha} \lambda_{t+1} \\ \end{align}

The FOCs, together with the budget constraint, characterize the optimal sequence of allocations. Suppose that the sequence \(\{\hat{c}_t,\hat{k}_{t+1},\hat{h}_{t+1}\}_t\) is a solution to the above maximization problem for initial stocks \((\bar{k}_0,\bar{h}_0)\). Then there exist some lagrange multipliers \(\{\hat{\lambda}_t\}_t\) such that: \begin{align} 0 & = \beta^t \hat{c}_t^{-\sigma} - \hat{\lambda}_t \\ 0 & = - \hat{\lambda}_t + (1-\delta_k )\hat{\lambda}_{t+1} + A\hat{k}_{t+1}^{\alpha-1}\hat{h}_{t+1}^{1-\alpha} \hat{\lambda}_{t+1} \\ 0 & = - \hat{\lambda}_t + (1-\delta_h )\hat{\lambda}_{t+1} + A\hat{k}_{t+1}^{\alpha}\hat{h}_{t+1}^{-\alpha} \hat{\lambda}_{t+1} \\ \hat{c}_t + \hat{k}_{t+1} + \hat{h}_{t+1} & = A\hat{k}_t^\alpha \hat{h}_t^{1-\alpha} + (1-\delta_k ) \hat{k}_t + (1-\delta_h ) \hat{h}_t \\ \hat{c}_0 + \hat{k}_{1} + \hat{h}_{1} & = A\bar{k}_0^\alpha \bar{h}_0^{1-\alpha} + (1-\delta_k ) \bar{k}_0 + (1-\delta_h ) \bar{h}_0 \\ \end{align}

add term including initial condition, show multiopliers still work, badabing

The agent has preferences given by \[ \sum_{t}\beta^t c_t \]

If the agent is unemployed at the start of the period (state \( S^U \)), then a random wage offer \(w\) is drawn from the fixed distribution \(F(w)\), which has support \([0,B]\). The agent can choose to accept or reject the offer.
- If they accept, they earn income \(w\), and then enter the next period employed at wage \(w\).
- If they reject, they earn fixed income \(b\), and then enter the next period unemployed. Assume \(b \in[0,B)\).
If the agent is employed at the start of the period with wage \(w\) (state \(S^E(w)\)), they earn income \(w\), and then enter the next period employed with wage \(w\). Jobs can never be lost.
Income is in terms of units of consumption, and cannot be saved. There is also no borrowing nor lending, so the agent will consume their income each period.

This system can be summarized by the following diagram. (key)

Flowchart for a simple job search with no firing.

The loop of being employed is on the right, and the loop of searching for a job is on the left. \(V(w)\) is the present value of having offer \(w\) in hand. \(V^U\) is the present value of starting a period unemployed.

Characterizing the Decision Rule

Note that per-period utility flow is bounded, and so with discounting, all present values are bounded as well. At this point, we don't know what these present values are numerically, nor even if they are computable. But by the time-invariance of the process, we know that they exist, and can establish some relationships between them: \[V^R = b +\beta V^U\] \[V^U = E[V(w)] \] \[V(w) = \max \left\{V^A (w),V^R\right\} \] \[ V^A (w) = V^E (w) = w + \beta V^E (w) = {w \over 1-\beta} \] It is optimal to accept an offer iff \({w \over 1-\beta}\geq V^R\). The expression \({w \over 1-\beta}\) is strictly increasing in \(w\), and \( V^R\) is constant, though we don't know it yet. By the intermediate value theorem, there must be a reservation wage \(\bar{w}\) such that \({\bar{w} \over 1-\beta}= V^R\) and it is optimal to accept an offer iff \(w\geq\bar{w}\). Therefore: \begin{split} V^U = E\left[V(w)\right] & = \Pr(w<\bar{w})V^R & + \Pr(w\geq\bar{w})E\left[{w \over 1-\beta}\mid w\geq\bar{w}\right] \\ V^U = E\left[V(w)\right] & = F(\bar{w})V^R & + \int_{\bar{w}}^{B} {w \over 1-\beta} f(w) dw \end{split} \[ {\bar{w} \over 1-\beta} = V^R = b +\beta V^U = b + \beta F(\bar{w}){\bar{w} \over 1-\beta} +\beta \int_{\bar{w}}^{B} {w \over 1-\beta} f(w) dw \] The above formula characterizes the reservation wage for given \(b\) and \(F\). Rearranging: \[ 0 = \bar{w}\left({1-\beta F(\bar{w}) \over 1-\beta}\right) -b - \beta \int_{\bar{w}}^{B} {w \over 1-\beta} f(w) dw \] You can verify that this has only exactly solution by evaluating the right-hand side as a continuous function of \(\bar{w}\). The RHS is negative at \(\bar{w}=0\), positive at \(\bar{w}=B\), and the derivative is strictly positive over \((0,B)\).

Comparative Statics: Increasing Unemployment Benefits \(b\).

Consider two economies with different unemployment benefits \(b_1\) and \(b_2\), but that are otherwise identical.

Let \(b_2 > b_1\). Define: \[ \hat{b}(x) \equiv x\left({1-\beta F(x) \over 1-\beta}\right) - \beta \int_{x}^{B} {w \over 1-\beta} f(w) dw \] From above, we know that \(b_i = \hat{b}(\bar{w_i})\). Taking the derivative, \[ \hat{b}'(x) = \left({1-\beta F(x) \over 1-\beta}\right) - x{\beta\over 1-\beta}f(x) - {\beta\over 1-\beta} (-xf(x)) = \left({1-\beta F(x) \over 1-\beta}\right) > 0\] So because \(\hat{b}(x)\) is strictly increasing, \(b_2>b_1 \implies \hat{b}(\bar{w_2})>\hat{b}(\bar{w_1}) \implies \bar{w_2} > \bar{w_1} \)

More intuitively, but a bit less rigourously, let \(V^U_i\) be the present value of starting a period unemployed in economy \(i\). We know that \({\bar{w_i} \over 1-\beta} = V^E(\bar{w_i})=b_i+\beta V^U_i\) and so \(\bar{w_i}=(1-\beta)(b_i+\beta V^U_i)\)

Note that \(V_2^U \geq V_1^U\). If the agents in each economy follow the same strategy, and witness the same sequence of random events, then the agent in economy 2 will always have per-period income at least as high as the agent in economy 1. So it can't possibly be that \(V_2^U < V_1^U\). And so: \[\bar{w_2}=(1-\beta)(b_2+\beta V^U_2) > (1-\beta)(b_1+\beta V^U_1) = \bar{w_1}\] This makes sense. Higher unemployment benefits make being unemployed a more attractive prospect, increasing the opportunity cost of accepting any given job offer, which means that fewer job offers will be accepted.

Comparative Statics: Riskier Job Market.

Consider two economies with different distributions for job offers \(F_1\) and \(F_2\), but that are otherwise identical.

Let \(F_2\) be a mean-preserving spread of \(F_1\), and let both distributions have support within \([0,B]\). We will prove that the reservation wage is at least as high in economy 2: \(\bar{w_2}\geq\bar{w_1}\)

Definition: Mean Preserving Spread
Let \(F_1\) and \(F_2\) be the distribution functions of random variables \(X_1\) and \(X_2\).
\(X_2\) is a mean preserving spread of (and is 'riskier than') \(X_1\) iff:

\(X_1\) and \(X_2\) have the same mean: \[ E[X_1] = \int_{-\infty}^{\infty}wf_1(w)dw = \int_{-\infty}^{\infty}wf_2(w)dw = E[X_2] \]
And \(X_2\) gives more weight to the tails of the distribution: \[\forall x, \int_{-\infty}^{x} F_2(w) dw \geq \int_{-\infty}^{x} F_1(w) dw\]

Now to apply this definition to the characterizing formula for reservation wage \(\bar{w_i}\), we need to rearrange the integral in a few ways. First, notice that in this environment, \[ E_i[w] = \int_{0}^{B} {w} f_i(w) dw = \int_{0}^{\bar{w_i}} {w} f_i(w) dw + \int_{\bar{w_i}}^{B} {w} f_i(w) dw \] \[\therefore \int_{\bar{w_i}}^{B} {w} f_i(w) dw = E_i[w] - \int_{0}^{\bar{w_i}} {w} f_i(w) dw \] Next, note that \(f(w)dw=dF(w)\), so applying integration by parts: \[ \int_{0}^{\bar{w_i}} {w} f_i(w) dw = \int_{0}^{\bar{w_i}} {w} dF_i(w) = \left[F_i(w)w\right]_0^{\bar{w_i}} - \int_{0}^{\bar{w_i}} F_i(w) dw = F_i(\bar{w_i})\bar{w_i} - \int_{0}^{\bar{w_i}} F_i(w) dw \] Putting this together, we have that \[\int_{\bar{w_i}}^{B} {w} f_i(w) dw = E_i[w] - F_i(\bar{w_i})\bar{w_i} + \int_{0}^{\bar{w_i}} F_i(w) dw\] And substituting this into the characterizing formula for reservation wage \(\bar{w_i}\): \begin{split} {\bar{w_i} \over 1-\beta} & = V_i^R = b +\beta V_i^U = b + \beta F_i(\bar{w_i}){\bar{w_i} \over 1-\beta} +\beta \int_{\bar{w_i}}^{B} {w \over 1-\beta} f_i(w) dw \\ & = b + {\beta \over 1-\beta}F_i(\bar{w_i}){\bar{w_i}} + {\beta \over 1-\beta} \left[ E_i[w] - \bar{w_i}F_i(\bar{w_i}) + \int_{0}^{\bar{w_i}} F_i(w) dw \right]\\ & = b + {\beta \over 1-\beta} \left[ E_i[w] + \int_{0}^{\bar{w_i}} F_i(w) dw \right] \end{split}

If you are comparing distributions of the form \(F(w,r)\) where \(r\) is a parameter such that riskiness increases as \(r\) increases, then you can take the derivative of both sides with respect to \(r\) and find that \(\bar{w_i}'(r) > 0\). Otherwise:

Recall that the properties of a mean preserving spread imply that \(E_1[w]=E_2[w]\) and that for any value of \(\bar{w}\), \(\int_{0}^{\bar{w}} F_2(w) dw \geq \int_{0}^{\bar{w}} F_1(w) dw\). Then: \begin{split} \bar{w_2} - \bar{w_1} & = (1-\beta)b + \beta E_2[w] + \beta \int_{0}^{\bar{w_2}} F_2(w) dw - (1-\beta)b - \beta E_1[w] - \beta \int_{0}^{\bar{w_1}} F_1(w) dw \\ & = \beta \int_{0}^{\bar{w_2}} F_2(w) dw - \beta \int_{0}^{\bar{w_1}} F_1(w) dw \\ & = \beta \int_{\bar{w_1}}^{\bar{w_2}} F_2(w) dw + \beta \int_{0}^{\bar{w_1}} \left[ F_2(w)-F_1(w)\right] dw \geq \beta \int_{\bar{w_1}}^{\bar{w_2}} F_2(w) dw \\ \end{split} Note that if \(\bar{w_2} \geq \bar{w_1}\), then \(\int_{\bar{w_1}}^{\bar{w_2}} F_2(w) dw \geq 0 \) and \(F_2(\bar{w_1}) \leq F(w)\) for all \(w \in [\bar{w_1}, \bar{w_2}]\).
Likewise, if \(\bar{w_2} \leq \bar{w_1}\), then \(\int_{\bar{w_1}}^{\bar{w_2}} F_2(w) dw \leq 0 \) and \(F_2(\bar{w_1}) \geq F(w)\) for all \(w \in [\bar{w_2}, \bar{w_1}]\). In either case, \[\bar{w_2} - \bar{w_1} = \beta \int_{\bar{w_1}}^{\bar{w_2}} F_2(w) dw \geq \beta \int_{\bar{w_1}}^{\bar{w_2}} F_2(\bar{w_1}) dw = (\bar{w_2} - \bar{w_1}) \beta F_2(\bar{w_1})\] \[ \implies (\bar{w_2} - \bar{w_1}) (1 - \beta F_2(\bar{w_1})) \geq 0 \implies \bar{w_2} \geq \bar{w_1} \]

It is also worth noting that a higher optimal reservation wage implies \(V^R_2 > V^R_1\), and so the agent's present value will be at least as high in economy 2 for any given wage offer. Intuitively, when the distribution of wages is spread out, the expected value of accepting a job offer in the next period remains the same; it's just that the chances of very good offers and very bad offers increase. But the agent just ignores the very bad offers.