betaFeedback

WorksMathematics

Comparison Principle and Semi-Lagrangian Methods for a Hamilton–Jacobi–Bellman System with Recursive Utility

Abstract

We consider a system of coupled first-order Hamilton–Jacobi–Bellman equations arising in continuous-time heterogeneous agent models with recursive utility, a class of models that has attracted increasing attention in macroeconomics. The first part of the paper establishes a comparison principle for the system when preferences exhibit early resolution of uncertainty. A key ingredient in the proof is an ordinally equivalent transformation of the recursive utility representation. In the second part, we develop semi-Lagrangian schemes and iterative algorithms for solving the associated nonlinear discrete systems. For preferences exhibiting late resolution of uncertainty, we establish a first-order error estimate for the semi-Lagrangian approximation. Numerical experiments demonstrate that the proposed methods yield results consistent with those obtained using the previously developed finite difference schemes.

Introduction

In this paper, we consider a system of coupled first order Hamilton–Jacobi–Bellman equations with recursive utility, recently analyzed in [5, 4]. Such a system of equations arises from the Aiyagari–Bewley–Huggett type heterogeneous agent (HA for brevity) models in macroeconomic theory [22], extended to continuous time in [2]. HA models provide a standard framework for studying consumption–saving decisions, wealth distributions, and equilibrium asset prices in incomplete markets. By incorporating idiosyncratic income risk and borrowing constraints, these models capture the interaction between individual saving behavior and the distribution of wealth in general equilibrium. Their numerical solution involves Hamilton–Jacobi–Bellman (HJB) equations coupled with equations describing the invariant distribution of agents and the equilibrium conditions for asset prices.

The HJB equation for characterizing the agent’s consumption–saving decision involves the Epstein–Zin recursive utility, which is specified through the aggregator (cf. [26])

f(c,v)=ρ1−ψ−1c1−ψ−1−((1−γ)v)θ((1−γ)v)θ−1,θ=1−ψ−11−γ.(1)f(c,v)=\frac{\rho}{1-\psi^{-1}}\frac{c^{1-\psi^{-1}}-((1-\gamma)v)^\theta}{((1-\gamma)v)^{\theta-1}},\qquad\theta=\frac{1-\psi^{-1}}{1-\gamma}. \tag*{(1)}

In (1), cc denotes consumption, vv is the value function, ρ\rho is the subjective discount rate, ψ\psi is the elasticity of intertemporal substitution (EIS), and γ\gamma is the coefficient of relative risk aversion. We focus on the parameter region 0<ψ<10 < \psi< 1 and γ>1\gamma> 1. The time-additive constant relative risk aversion (CRRA) specification is recovered when γ=ψ−1\gamma= \psi^{-1}. The parameter θ\theta determines the preference for the timing of the resolution of uncertainty: early resolution is preferred when θ<1\theta< 1, while late resolution is preferred when θ>1\theta> 1 [19].

The system of HJB equations with recursive utility and a given interest rate rr is

0=max⁡c≥0{f(c,vj)+(rx+yj−c)Dvj(x)}+λj(vjˉ(x)−vj(x)),j∈{1,2},jˉ=3−j.(2)0 = \max_{c \ge0} \left\{ f(c,v_j) + (rx+y_j-c)D v_j(x) \right\} + \lambda_j\left(v_{\bar{j}}(x)-v_j(x)\right), \qquad j \in\{1,2\},\quad\bar{j}=3-j. \tag*{(2)}

The equations are coupled through the switching terms λj(vjˉ−vj)\lambda_j(v_{\bar{j}}-v_j), encoding the transitions between income states with Poisson rates λj\lambda_j. The state constraint x≥x‾x \ge\underline{x} (a borrowing limit) is imposed to prevent negative savings at x‾\underline{x}: rx+yj−cj(x)≥0rx+y_j-c_j(x) \ge0.

As a first theoretical contribution, we prove a comparison principle for solutions of (2) in the case of preference for early resolution of uncertainty, i.e. 0<θ<10 < \theta< 1. The uniqueness of solutions to this type of problem in the discrete-time setting has been analyzed in [23, 24, 25]. In the continuous-time setting, the strong comparison principle (between semicontinuous sub- and supersolutions) in the case θ=1\theta= 1 has been established in [13] and then extended to θ>1\theta> 1 in [5]. The main difficulty in the case 0<θ<10 < \theta< 1 is that the nonlinear part of the aggregator f(c,v)f(c,v) in (2) becomes an increasing function for the vv-argument. The existence of extremal solutions has been addressed in [4] but the uniqueness of solution was not established. In this paper we exploit an ordinal transformation of the recursive utility problem to obtain a more tractable equivalent representation. Because the transformation is invertible, comparison principle in the transformed representation translates into that of the original problem. This yields a self-contained proof of uniqueness to the HJB equation in the parameter regime 0<θ<10 < \theta< 1. Similar changes of variables have been used in [6, 18] for second order HJB equation involving recursive utility.

We next address the numerical solution of such HJB systems and related HAHA models. The finite difference methods for solving (2) have been considered recently in [4]. Building on the semi-Lagrangian framework for state-constrained heterogeneous agent models with time-additive CRRA utility developed in [13], we extend the HJB discretization to Epstein–Zin recursive utility, whose nonlinear dependence on the value function requires new analytical arguments and iterative algorithms. SL methods discretize the dynamic programming principle by following the controlled dynamics over a discrete time interval and evaluating the value function at the resulting off-grid points through interpolation. We refer to [8, 20] for the theoretical foundations, to [10] for approximation of coupled HJB systems, to [15] for convergence rate analysis and to [16, 7] for SL schemes for Mean Field Games. Concerning iterative algorithms, [14] studied a learning value iteration algorithm and its acceleration by policy iteration for a semi-Lagrangian discretization of Mean Field Games. In the abstract dynamic programming framework, [25, 23] developed monotonicity-based and order-theoretic approaches to establish convergence of iterative algorithms, for solving problems with recursive preferences.

We consider the underlying stochastic control problem associated with the Epstein–Zin utility, cf. [17, 18, 26]:

v(x,y)=sup⁡cτE[∫t∞f(cτ,vτ) dτ | xt=x, yt=y],s.t.{dxτ=(rxτ+yτ−cτ) dτ,τ>t,xτ≥x‾.(3)v(x,y)=\sup_{c_\tau}\mathbb{E}\left[\int_t^\infty f(c_\tau,v_\tau)\,\mathrm{d}\tau\,\middle|\, x_t=x,\ y_t=y\right],\qquad\text{s.t.}\quad\left\{\begin{aligned} \mathrm{d}x_\tau&= (rx_\tau+y_\tau-c_\tau)\,\mathrm{d}\tau,\quad\tau>t,\\ x_\tau&\ge\underline{x}. \end{aligned}\right. \tag*{(3)}

Here, xτx_\tau and yτy_\tau respectively stand for the agent’s wealth and labor income at time τ\tau. The control variable is the consumption cτc_\tau and the flow ff depends also on the continuation value vτ=v(xτ,yτ)v_\tau= v(x_\tau, y_\tau). It is assumed that yτy_\tau is a two-state continuous-time Markov chain with Poisson switching rates. We observe that the same ordinal equivalent transformation for the HJB equation can also apply to the SL scheme, which allows us to establish a discrete comparison principle in the case 0<θ<10 < \theta< 1. Building on the Howard policy iteration in [13], we develop a Howard–Newton algorithm in which the nonlinear policy evaluation arising from recursive utility is performed using Newton’s method. We prove convergence of the outer policy iteration in both the late- and early-resolution preference regimes. As an alternative, in the early-resolution case 0<θ<10 < \theta< 1, we formulate the discrete problem as a fixed point of a monotone and invariant map and develop a corresponding value iteration algorithm.

This paper is organized as follows. In § 2, we introduce the relevant background and previous results for the HJB equation, building on [5, 4]. In § 3, we prove the comparison principle and uniqueness of solution to the HJB equation in the case 0<θ<10 < \theta< 1 (a result which is not available in [5, 4]), based on an ordinal equivalent transformation. In § 4, we propose an SL scheme. We develop a Howard–Newton algorithm and prove a discrete comparison principle for the solution to the scheme. In the early-resolution case, we propose additionally a monotone value iteration algorithm. For the late-resolution case, the SL scheme has a first-order convergence rate to the unique solution of the HJB equation. In § 5, we give the solution methods for the Fokker–Planck–Kolmogorov (FPK) equations and a general equilibrium Mean Field Game model. Finally, numerical experiments in § 6 demonstrate the effectiveness of the proposed methods and confirm their agreement with the results obtained using the finite difference methods from [4].

Preliminaries

The assumptions that follow will be made in the whole paper:

γ>1,0<ψ<1,ρ>r>0,y2>y1>0,x≤0,ρx+y1>0.(4)\gamma> 1,\quad0 < \psi< 1,\quad\rho> r > 0,\quad y_2 > y_1 > 0,\quad x \le0,\quad\rho x + y_1 > 0. \tag*{(4)}

It has been proved in [5] that the equilibrium interest rate r∗r^* is smaller than ρ\rho, if there exists a solution to (5.3) below. This justifies the bounds on rr in (2.1). With the constant bb defined in Table 1 below, (2.1) yields r<b<ρr < b < \rho, rx+y2>rx+y1>0rx + y_2 > rx + y_1 > 0 and b(x+y1/r)>0b(x + y_1/r) > 0. Therefore, the functions in (2.10) below are well defined.

We observe that the Epstein–Zin utility can be decomposed as, using ρ/θ=ζ\rho/\theta= \zeta,

f(c,vj)=F(c,vj)−ζvj,F(c,vj)=ρ1−ψ−1((1−γ)vj)θ−1c1−ψ−1.(5)f(c,v_j) = \mathcal{F}(c,v_j) - \zeta v_j,\qquad\mathcal{F}(c,v_j) = \frac{\rho}{1-\psi^{-1}((1-\gamma)v_j)^{\theta-1}}c^{1-\psi^{-1}}. \tag*{(5)}

We use (2.2) to rewrite the HJB equation (1.2) as

ρθvj(x)=H(x,yj,vj,Dvj)+λj(vjˉ(x)−vj(x)),(6)\frac{\rho}{\theta}v_j(x) = H(x,y_j,v_j,Dv_j) + \lambda_j\left(v_{\bar{j}}(x)-v_j(x)\right), \tag*{(6)}

where we use the notation from (2.2) and

H(x,yj,vj,p):=max⁡c≥0{F(c,vj)+(rx+yj−c)p}.(7)H(x,y_j,v_j,p) := \max_{c\ge0}\left\{\mathcal{F}(c,v_j) + (rx+y_j-c)p\right\}. \tag*{(7)}

From the first order necessary optimality condition, the Hamiltonian in (7) is

H(x,y,v,p)={(rx+y)p+ρψψ−1p1−ψ((1−γ)v)1−γψ1−γ,if p≥0,+∞,if p<0.(8)H(x,y,v,p)=\begin{cases} (rx+y)p+\dfrac{\rho^\psi}{\psi-1}p^{1-\psi}\bigl((1-\gamma)v\bigr)^{\frac{1-\gamma\psi}{1-\gamma}}, & \text{if } p\ge0,\\ +\infty, & \text{if } p<0. \end{cases} \tag*{(8)}

We observe that H(x,y,v,p)H(x,y,v,p) defined in (8) is strictly convex in pp for fixed (x,y,v)(x,y,v) with p>0p>0. The optimal consumption (away from the borrowing limit) is given by the first order necessary optimality condition, whenever Dvj>0Dv_j>0,

cj=arg⁡max⁡c≥0{F(c,vj)−cDvj}=ρψ(Dvj)−ψ((1−γ)vj)1−γψ1−γ.(9)c_j=\underset{c\ge0}{\arg\max}\left\{\mathcal{F}(c,v_j)-cDv_j\right\}=\rho^\psi(Dv_j)^{-\psi}\bigl((1-\gamma)v_j\bigr)^{\frac{1-\gamma\psi}{1-\gamma}}. \tag*{(9)}

We summarize the notation in Table 1.

SubjectSymbols
Subjective discount rateρ>0\rho>0
Risk aversionγ>1\gamma>1
Elasticity of intertemporal substitution (EIS)0<ψ<10<\psi<1
A parameter arising in Epstein-Zin utilityθ:=1−ψ−11−γ>0\theta:=\dfrac{1-\psi^{-1}}{1-\gamma}>0
A convenient parameterζ:=ρ/θ\zeta:=\rho/\theta
A convenient parameterb:=ρ[r+ψ(ρ−r)ρ]11−ψb:=\rho\left[\dfrac{r+\psi(\rho-r)}{\rho}\right]^{\frac{1}{1-\psi}}

Table 1. Symbols

For any c>0c>0, if θ>1\theta>1 (resp. 0<θ<10<\theta<1) then F(c,v)\mathcal{F}(c,v) is decreasing (resp. increasing) in vv, i.e.

References

  1. [1]Y. Achdou, J. Brumm, and L. Frank, Mastering stochastic OLG models in continuous time. arXiv preprint arXiv:2608.11134, 2026, https://arxiv.org/abs/2608.11134.

Paper details

vlnc:W_n2h9z8cc
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Claude (Anthropic), ChatGPT (OpenAI)

View statement
Looking for related papers…
Contents