Introduction
Plant-herbivore interactions are shaped by the interplay between resource growth, grazing pressure, and defence mechanisms developed by plants to reduce herbivore exploitation. Such interactions may generate a wide variety of ecological outcomes, including species persistence, extinction scenarios, bistability, and sustained population oscillations [1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 ] .
Among the various defence strategies adopted by plants, the production of toxic secondary metabolites plays a particularly important role, as it may directly affect herbivore feeding activity and alter the balance between resource availability and consumption pressure. Therefore, the ecological consequences of toxin-mediated interactions have been investigated through a series of mathematical models incorporating defence-dependent functional responses. Indeed, previous studies have shown that plant toxicity may substantially modify the existence and stability of coexistence states, generate complex bifurcation structures, and promote oscillatory population dynamics [12 , 13 , 14 , 15 , 16 , 17 ] . More recently, attention has also extended to spatially distributed formulations, where diffusion and cross-diffusion mechanisms have been shown to induce pattern formation and spatiotemporal self-organization [17 , 18 ] .
Despite these advances, most studies have focused on the existence and stability of equilibria, local bifurcations, and numerical exploration of the resulting dynamics. Although several important global results are available [14 , 15 , 16 ] , the geometric mechanisms organizing the phase portrait remain less explicitly characterized. In particular, the role played by invariant manifolds and by heteroclinic and homoclinic structures in shaping the long-term dynamics of toxin-mediated plant-herbivore systems has not yet been systematically investigated. A geometric description of these invariant structures is essential for understanding how local bifurcation phenomena are embedded into the global organization of the phase plane and for clarifying the dynamical scenarios observed in numerical studies.
Motivated by these considerations, in this work we revisit a toxin-mediated plant-herbivore model characterized by a non-monotone functional response accounting for inhibitory effects on grazing activity. The novelty of the present work is twofold. First, we provide a complete geometric classification of the global phase portraits arising in the model. Second, we perform a detailed analysis of the parameter plane, showing how changes in the model parameters reorganize the global geometry of the system. In particular, we characterize the bifurcation scenarios connecting different dynamical regimes, including the onset of periodic solutions through Hopf bifurcation and the appearance of heteroclinic and homoclinic structures [19 , 20 , 21 ] . This unified geometric framework reveals how the interaction between plant toxicity and herbivore exploitation shapes the global dynamics, leading to distinct ecological outcomes ranging from extinction to sustained oscillatory behavior.
The manuscript is organized as follows. In Section 2 we introduce the model and discuss its biological assumptions. Section 3 is devoted to the analysis of equilibria, local stability properties, and Hopf bifurcation. The global phase-plane organization, including heteroclinic and homoclinic structures, is investigated in Section 4. Finally, conclusions and perspectives for future research are presented in Section 5.
Model formulation and geometric preliminaries
We consider the toxin-mediated plant–herbivore model proposed in [12 , 13 , 14 , 18 ] . The dimensionless equations are
B ˙ = B ( 1 − B α ) − m C ( B ) H , H ˙ = ( m C ( B ) − θ ) H (1) \dot{B}=B\left(1-\frac{B}{\alpha}\right)-mC(B)H,
\qquad
\dot{H}=(mC(B)-\theta)H \tag*{(1)} B ˙ = B ( 1 − α B ) − m C ( B ) H , H ˙ = ( m C ( B ) − θ ) H (1)
where B ( t ) B(t) B ( t ) and H ( t ) H(t) H ( t ) denote the densities of vegetation biomass and herbivores at time t t t , respectively, and
C ( B ) = B 1 + B ( 1 − μ B 1 + B ) (2) C(B)=\frac{B}{1+B}\left(1-\frac{\mu B}{1+B}\right) \tag*{(2)} C ( B ) = 1 + B B ( 1 − 1 + B μ B ) (2)
characterizes the defence-dependent functional response. In system (1), vegetation grows logistically with carrying capacity α > 0 \alpha> 0 α > 0 , while herbivores undergo a constant per-capita mortality rate θ > 0 \theta> 0 θ > 0 and the parameter m > 0 m > 0 m > 0 measures the conversion efficiency from consumed vegetation into herbivore biomass. The ecological interaction between the two populations is described by the nonlinear consumption function C ( B ) C(B) C ( B ) . Indeed, the first factor B / ( 1 + B ) B/(1+B) B / ( 1 + B ) represents a classical saturating functional response accounting for feeding limitations associated with searching, handling and resource processing, whereas the second factor introduces toxin-mediated inhibition through the parameter μ ∈ [ 1 / 4 , 1 ] \mu\in[1/4,1] μ ∈ [ 1/4 , 1 ] [12 , 13 , 14 , 18 ] : for low vegetation densities, resource availability dominates the interaction and consumption increases with biomass; for sufficiently large vegetation densities, toxin accumulation reduces grazing efficiency and may eventually overcome the beneficial effects of resource availability.
Lemma 2.1 (Geometric behaviour of C ( B ) C(B) C ( B ) )
The qualitative behaviour of the toxin-dependent consumption function C ( B ) C(B) C ( B ) depends on the toxicity parameter μ \mu μ . In particular,
Let μ ∈ [ 1 / 4 , 1 / 2 ] \mu\in[1/4,1/2] μ ∈ [ 1/4 , 1/2 ] , then C ( B ) C(B) C ( B ) is strictly increasing;
Let μ ∈ ] 1 / 2 , 1 ] \mu\in]1/2,1] μ ∈ ] 1/2 , 1 ] , then C ( B ) C(B) C ( B ) possesses a unique positive critical point B M = 1 / ( 2 μ − 1 ) B_M = 1/(2\mu- 1) B M = 1/ ( 2 μ − 1 ) , which corresponds to a global maximum, before tending to an asymptote at C ( B ) = 1 − μ C(B) = 1 - \mu C ( B ) = 1 − μ .
Remark 2.1 The threshold μ = 1 / 2 \mu= 1/2 μ = 1/2 separates two qualitatively different ecological regimes. For μ ≤ 1 / 2 \mu\le1/2 μ ≤ 1/2 , defence effects are not sufficiently strong to overcome the increase in resource availability, and the consumption rate grows monotonically with vegetation biomass. Conversely, when μ > 1 / 2 \mu> 1/2 μ > 1/2 , defence-induced inhibition becomes dominant at high biomass levels, generating a unimodal functional response. This transition constitutes the key mechanism responsible for the appearance of multiple coexistence states and the bifurcation scenarios investigated in the following sections.
Equilibrium structure and local bifurcations
The geometric properties of the functional response established in § 2 provide the foundation for the equilibrium structure of system (1). In particular, the transition from a monotone to a non-monotone consumption function introduces the possibility of multiple coexistence states and distinct dynamical regimes. However, the local dynamics of defence-mediated plant-herbivore systems have been extensively investigated in previous studies [12 , 13 , 14 , 15 , 16 ] . Therefore, our objective is not to revisit the classical qualitative analysis of the model, but rather to identify the global geometric structures that organize the dynamics. To this aim, in this section we briefly recall the main results concerning the equilibrium structure and local bifurcations that are relevant to the subsequent analysis and we refer to [14 , 15 , 18 ] for more details.
Equilibria and local stability
Equilibria represent stationary configurations in which vegetation growth, herbivore consumption, and herbivore mortality are exactly balanced. System (1) always admits the trivial equilibrium (TE) and the herbivore-free equilibrium (HFE) [ T E ] : ( B ∗ , H ∗ ) = ( 0 , 0 ) [TE] : (B^*, H^*) = (0,0) [ T E ] : ( B ∗ , H ∗ ) = ( 0 , 0 ) and [ H F E ] : ( B ∗ , H ∗ ) = ( α , 0 ) [HFE] : (B^*, H^*) = (\alpha,0) [ H F E ] : ( B ∗ , H ∗ ) = ( α , 0 ) together with two possible coexistence states. These latter are determined by the intersections between the toxin-dependent consumption function C ( B ) C(B) C ( B ) and the threshold value θ / m \theta/m θ / m , representing the minimum consumption level required to compensate herbivore mortality, combined with the biological admissibility requirement 0 < B < α 0 < B < \alpha 0 < B < α .
Lemma 3.1 (Existence of C E − CE_- C E − and C E + CE_+ C E + ) Besides the T E TE T E and H F E HFE H F E equilibria, system (1) may admit two positive coexistence equilibria
C E − : ( B ∗ , H ∗ ) = ( B − , H − ) = ( m − 2 θ − m ( m − 4 μ θ ) 2 ( θ − m + μ m ) , B − ( α − B − ) α θ ) , CE_- : (B^*,H^*) = (B_-,H_-) = \left(\frac{m-2\theta-\sqrt{m(m-4\mu\theta)}}{2(\theta-m+\mu m)},\frac{B_-(\alpha-B_-)}{\alpha\theta}\right), C E − : ( B ∗ , H ∗ ) = ( B − , H − ) = ( 2 ( θ − m + μ m ) m − 2 θ − m ( m − 4 μ θ ) , α θ B − ( α − B − ) ) , C E + : ( B ∗ , H ∗ ) = ( B + , H + ) = ( m − 2 θ + m ( m − 4 μ θ ) 2 ( θ − m + μ m ) , B + ( α − B + ) α θ ) . CE_+ : (B^*,H^*) = (B_+,H_+) = \left(\frac{m-2\theta+\sqrt{m(m-4\mu\theta)}}{2(\theta-m+\mu m)},\frac{B_+(\alpha-B_+)}{\alpha\theta}\right). C E + : ( B ∗ , H ∗ ) = ( B + , H + ) = ( 2 ( θ − m + μ m ) m − 2 θ + m ( m − 4 μ θ ) , α θ B + ( α − B + ) ) . More precisely, the equilibrium C E − CE_- C E − exists and is biologically meaningful whenever μ < min { μ H F E , μ M } \mu< \min\{\mu_{HFE},\mu_M\} μ < min { μ H F E , μ M } or μ M < μ < μ S N \mu_M < \mu< \mu_{SN} μ M < μ < μ S N , whereas the equilibrium C E + CE_+ C E + for max { μ H F E , μ M } < μ < μ S N \max\{\mu_{HFE},\mu_M\} < \mu< \mu_{SN} max { μ H F E , μ M } < μ < μ S N , being
μ H F E = ( 1 + α ) [ m α − θ ( 1 + α ) ] m α 2 , μ M = α + 1 2 α and μ S N = m 4 θ . \mu_{HFE} = \frac{(1+\alpha)[m\alpha-\theta(1+\alpha)]}{m\alpha^2}, \qquad\mu_M = \frac{\alpha+1}{2\alpha} \qquad\text{and} \qquad\mu_{SN} = \frac{m}{4\theta}. μ H F E = m α 2 ( 1 + α ) [ m α − θ ( 1 + α )] , μ M = 2 α α + 1 and μ S N = 4 θ m . Note that, the equation C ( B ) = θ / m C(B)=\theta/m C ( B ) = θ / m admits at most one positive solution when μ ≤ 1 / 2 \mu\le1/2 μ ≤ 1/2 , whereas two positive solutions may occur when μ > 1 / 2 \mu> 1/2 μ > 1/2 , provided that μ < μ S N \mu< \mu_{SN} μ < μ S N (see Figure 1 ).
Figure 1. Geometrical characterization of the toxin-dependent consumption function C ( B ) C(B) C ( B ) and the nullclines of system (1). Panel (a) corresponds to μ ∈ [ 1 / 4 , 1 / 2 ] \mu\in[1/4,1/2] μ ∈ [ 1/4 , 1/2 ] , whereas panel (b) to μ ∈ ( 1 / 2 , 1 ] \mu\in(1/2,1] μ ∈ ( 1/2 , 1 ] .
Remark 3.1
Lemma 3.1 reveals a distinction between the existence of algebraic solutions of the equation m C ( B ) = θ mC(B)=\theta m C ( B ) = θ and their biological admissibility. Indeed, when
θ > θ M : = m α 2 ( 1 + α ) , \theta> \theta_M := \frac{m\alpha}{2(1+\alpha)}, θ > θ M := 2 ( 1 + α ) m α ,
one has μ H F E < μ S N < μ M \mu_{HFE} < \mu_{SN} < \mu_M μ H F E < μ S N < μ M . Therefore, for parameter values satisfying μ H F E < μ < μ S N \mu_{HFE} < \mu< \mu_{SN} μ H F E < μ < μ S N , the equation m C ( B ) = θ mC(B)=\theta m C ( B ) = θ admits two distinct positive solutions B − B_- B − and B + B_+ B + , but both satisfy B ± > α B_\pm> \alpha B ± > α .
Geometrically, this situation occurs because the horizontal level θ / m \theta/m θ / m intersects the graph of C ( B ) C(B) C ( B ) twice, but both intersections lie to the right of the carrying capacity threshold B = α B=\alpha B = α . As a result, the region μ H F E < μ < μ S N \mu_{\mathrm{HFE}}<\mu<\mu_{\mathrm{SN}} μ HFE < μ < μ SN contains no biologically admissible interior equilibrium despite the presence of two algebraic solutions of the coexistence equation.
Ecologically, the equilibrium C E − CE_{-} C E − corresponds to a low-biomass coexistence regime, whereas C E + CE_{+} C E + is associated with a high-biomass state in which herbivores persist despite the inhibitory effects induced by plant toxicity.
Lemma 3.2 (Equilibria stability) Assume that the corresponding equilibria are biologically admissible, then
(i) T E TE T E is always a hyperbolic saddle,
(ii) H F E HFE H F E is a stable node for μ > μ H F E \mu>\mu_{\mathrm{HFE}} μ > μ HFE , and a hyperbolic saddle for μ < μ H F E \mu<\mu_{\mathrm{HFE}} μ < μ HFE ,
(iii) C E − CE_{-} C E − is locally asymptotically stable for θ > θ H \theta>\theta_{\mathrm{H}} θ > θ H , and unstable for θ < θ H \theta<\theta_{\mathrm{H}} θ < θ H ,
(iv) C E + CE_{+} C E + is always a hyperbolic saddle,
where θ H \theta_{\mathrm{H}} θ H denotes the Hopf threshold
θ H = m [ 1 + α ( 1 − μ ) ] [ 1 + α ( 1 + 5 μ ) − 4 μ ] + [ α ( 3 μ − 1 ) − 1 ] [ 1 + α ( 1 − μ ) ] [ α ( 1 − μ ) + 1 + 8 μ ] 8 μ ( 1 + α ) 2 . (3) \theta_{\mathrm{H}}=m\frac{[1+\alpha(1-\mu)][1+\alpha(1+5\mu)-4\mu]+[\alpha(3\mu-1)-1]\sqrt{[1+\alpha(1-\mu)][\alpha(1-\mu)+1+8\mu]}}{8\mu(1+\alpha)^2}. \tag*{(3)} θ H = m 8 μ ( 1 + α ) 2 [ 1 + α ( 1 − μ )] [ 1 + α ( 1 + 5 μ ) − 4 μ ] + [ α ( 3 μ − 1 ) − 1 ] [ 1 + α ( 1 − μ )] [ α ( 1 − μ ) + 1 + 8 μ ] . (3)
As it has been already observed, the emergence of coexistence equilibria C E ± CE_{\pm} C E ± is associated with the intersection between the graph of C ( B ) C(B) C ( B ) and the threshold θ / m \theta/m θ / m , so suggesting the presence of saddle-node bifurcation mechanisms in the parameter space.
Proposition 3.1 (Saddle-node bifurcation of coexistence equilibria) Assume μ > 1 / 2 \mu>1/2 μ > 1/2 and consider θ \theta θ as bifurcation parameter. At the critical value θ S N = m / ( 4 μ ) \theta_{\mathrm{SN}}=m/(4\mu) θ SN = m / ( 4 μ ) , the coexistence equilibria C E − CE_{-} C E − and C E + CE_{+} C E + match at
( B , H ) = ( B M , B M ( α − B M ) α θ S N ) . (4) (B,H)=\left(B_M,\frac{B_M(\alpha-B_M)}{\alpha\theta_{\mathrm{SN}}}\right). \tag*{(4)} ( B , H ) = ( B M , α θ SN B M ( α − B M ) ) . (4) Moreover, provided B M < α B_M<\alpha B M < α , system (1) undergoes a codimension-one saddle-node bifurcation.
Proof. For μ > 1 / 2 \mu>1/2 μ > 1/2 , Lemma 2.1 implies that the consumption function C ( B ) C(B) C ( B ) is unimodal and attains its unique maximum at B M B_M B M (see Figure 1 (b)). Indeed, when θ = θ S N \theta=\theta_{\mathrm{SN}} θ = θ SN , the line θ / m \theta/m θ / m is tangent to C ( B ) C(B) C ( B ) at B = B M B=B_M B = B M and the two coexistence equilibria C E − CE_{-} C E − and C E + CE_{+} C E + match at (4) . For θ < θ S N \theta<\theta_{\mathrm{SN}} θ < θ SN the level θ / m \theta/m θ / m lies below the maximum of C ( B ) C(B) C ( B ) and intersects the graph twice, yielding two coexistence equilibria; for θ > θ S N \theta>\theta_{\mathrm{SN}} θ > θ SN no intersection exists. Hence, provided B M < α B_M<\alpha B M < α , the collision of C E − CE_{-} C E − and C E + CE_{+} C E + at θ = θ S N \theta=\theta_{\mathrm{SN}} θ = θ SN generates a codimension-one saddle-node bifurcation.
Hopf bifurcation and limit cycles
The loss of stability of the coexistence equilibrium C E − CE_{-} C E − is associated with the emergence of oscillatory dynamics. Let us now characterize the critical configuration at which this transition occurs.
Theorem 3.1 (Hopf bifurcation at C E − CE_{-} C E − ) Assume that C E − CE_{-} C E − exists in a neighbourhood of θ = θ H \theta=\theta_H θ = θ H . At the critical threshold θ = θ H \theta=\theta_H θ = θ H , C E − CE_{-} C E − has a pair of purely imaginary eigenvalues λ 1 , 2 = ± i det J ( C E − ) \lambda_{1,2}=\pm i\sqrt{\det J(CE_{-})} λ 1 , 2 = ± i det J ( C E − ) . Moreover, the transversality condition
d d θ Re λ ( θ ) ∣ θ = θ H ≠ 0 \left.\frac{\mathrm{d}}{\mathrm{d}\theta}\operatorname{Re}\lambda(\theta)\right|_{\theta=\theta_H}\ne0 d θ d Re λ ( θ ) θ = θ H = 0 is satisfied, and the first Lyapunov coefficient l 1 ( θ H ) l_1(\theta_H) l 1 ( θ H ) is nonzero. Therefore, system (1) undergoes a nondegenerate Hopf bifurcation at C E − CE_{-} C E − for θ = θ H \theta=\theta_H θ = θ H with a Hopf frequency
ω 0 = θ H m C ′ ( B − ( θ H ) ) H − ( θ H ) . \omega_0=\sqrt{\theta_H m C'(B_{-}(\theta_H))H_{-}(\theta_H)}. ω 0 = θ H m C ′ ( B − ( θ H )) H − ( θ H ) .
Proof. Since the equilibrium C E − CE_{-} C E − lies on the increasing branch of the toxin-dependent functional response, det J ( C E − ) > 0 \det J(CE_{-})>0 det J ( C E − ) > 0 so that its stability is entirely determined by the sign of the Jacobian trace tr J ( C E − ) = θ H ′ ( B − ) \operatorname{tr}J(CE_{-})=\theta H'(B_{-}) tr J ( C E − ) = θ H ′ ( B − ) . At the critical configuration θ = θ H \theta=\theta_H θ = θ H , in which H ′ ( B − ) = 0 H'(B_{-})=0 H ′ ( B − ) = 0 , it follows tr J ( C E − ) = 0 \operatorname{tr}J(CE_{-})=0 tr J ( C E − ) = 0 . Then, the Jacobian admits a pair of purely imaginary eigenvalues λ 1 , 2 = ± i det J ( C E − ) \lambda_{1,2}=\pm i\sqrt{\det J(CE_{-})} λ 1 , 2 = ± i det J ( C E − ) and the transversality condition is satisfied [18 ] , being
d Re { λ ( θ ) } d θ ∣ θ = θ H = 1 2 [ θ H ′ ′ ( B − ) ∂ B − ∂ θ ] ∣ θ = θ H < 0. \left.\frac{\mathrm{d}\operatorname{Re}\{\lambda(\theta)\}}{\mathrm{d}\theta}\right|_{\theta=\theta_H}
=\left.\frac{1}{2}\left[\theta H''(B_{-})\frac{\partial B_{-}}{\partial\theta}\right]\right|_{\theta=\theta_H}<0. d θ d Re { λ ( θ )} θ = θ H = 2 1 [ θ H ′′ ( B − ) ∂ θ ∂ B − ] θ = θ H < 0.
Since system (1) is two-dimensional, the 2D centre manifold at the Hopf bifurcation coincides locally with the phase plane so that the normal-form computation can be performed directly in the original coordinates. To reduce the dynamics in such a manifold, let us introduce x = B − B H x=B-B_H x = B − B H and v = H − H H v=H-H_H v = H − H H , being B H : = B − ( θ H ) B_H:=B_{-}(\theta_H) B H := B − ( θ H ) and H H : = H − ( θ H ) H_H:=H_{-}(\theta_H) H H := H − ( θ H ) , and, for convenience, set
c k : = C ( k ) ( B H ) , k = 0 , 1 , 2 , 3 , c_k:=C^{(k)}(B_H),\qquad k=0,1,2,3, c k := C ( k ) ( B H ) , k = 0 , 1 , 2 , 3 ,
with c 0 = C ( B H ) c_0=C(B_H) c 0 = C ( B H ) . At θ = θ H \theta=\theta_H θ = θ H , the system reduces to
( x ˙ v ˙ ) = ( 0 − θ H m c 1 H H 0 ) ( x v ) + O ( ∥ ( x , v ) ∥ 2 ) . \begin{pmatrix}\dot{x}\\\dot{v}\end{pmatrix}
=
\begin{pmatrix}0&-\theta_H\\mc_1H_H&0\end{pmatrix}
\begin{pmatrix}x\\v\end{pmatrix}
+\mathcal{O}(\lVert(x,v)\rVert^2). ( x ˙ v ˙ ) = ( 0 m c 1 H H − θ H 0 ) ( x v ) + O (∥( x , v ) ∥ 2 ) .
and the corresponding Hopf frequency is given by
ω 0 = det J ( C E − ( θ H ) ) = θ H m c 1 H H > 0. \omega_0=\sqrt{\det J(CE_{-}(\theta_H))}=\sqrt{\theta_Hmc_1H_H}>0. ω 0 = det J ( C E − ( θ H )) = θ H m c 1 H H > 0.
Introducing the normalized real coordinate y = θ H v / ω 0 y=\theta_Hv/\omega_0 y = θ H v / ω 0 , the linearized system becomes
x ˙ = − ω 0 y , y ˙ = ω 0 x . \dot{x}=-\omega_0y,\qquad\dot{y}=\omega_0x. x ˙ = − ω 0 y , y ˙ = ω 0 x .
Thus, the complex center coordinate is defined by
z = x + i y = B − B H + i θ H ω 0 ( H − H H ) z=x+iy=B-B_H+i\frac{\theta_H}{\omega_0}(H-H_H) z = x + i y = B − B H + i ω 0 θ H ( H − H H )
and the vector field takes the form
z ˙ = i ω 0 z + F 2 ( z , z ˉ ) + F 3 ( z , z ˉ ) + O ( ∣ z ∣ 4 ) , \dot{z}=i\omega_{0}z+F_{2}(z,\bar{z})+F_{3}(z,\bar{z})+O\left(\lvert z\rvert^{4}\right), z ˙ = i ω 0 z + F 2 ( z , z ˉ ) + F 3 ( z , z ˉ ) + O ( ∣ z ∣ 4 ) ,
where F j F_j F j denotes the homogeneous component of degree j j j . By the Poincaré-Dulac normal-form theorem [22 ] , there exists a near-identity transformation
z = w + h 2 ( w , w ˉ ) + h 3 ( w , w ˉ ) , z=w+h_{2}(w,\bar{w})+h_{3}(w,\bar{w}), z = w + h 2 ( w , w ˉ ) + h 3 ( w , w ˉ ) ,
where h j h_j h j is homogeneous of degree j j j , which, for θ \theta θ sufficiently close to θ H \theta_H θ H , eliminates all nonresonant quadratic terms. Therefore, the vector field to the centre manifold can be written as
w ˙ = ( β ( θ ) + i Ω ( θ ) ) w + ( l 1 ( θ ) + i ν 1 ( θ ) ) w ∣ w ∣ 2 + O ( ∣ w ∣ 4 ) . \dot{w}=(\beta(\theta)+i\Omega(\theta))w+(l_{1}(\theta)+i\nu_{1}(\theta))w\lvert w\rvert^{2}+O\left(\lvert w\rvert^{4}\right). w ˙ = ( β ( θ ) + i Ω ( θ )) w + ( l 1 ( θ ) + i ν 1 ( θ )) w ∣ w ∣ 2 + O ( ∣ w ∣ 4 ) .
with
β ′ ( θ H ) = θ H 2 H ′ ′ ( B H ) B H ′ ( θ H ) ≠ 0 , β ( θ H ) = 0 , Ω ( θ H ) = ω 0 , (5) \beta'(\theta_H)=\frac{\theta_H}{2}H''(B_H)B'_H(\theta_H)\ne0,\qquad\beta(\theta_H)=0,\qquad\Omega(\theta_H)=\omega_0, \tag*{(5)} β ′ ( θ H ) = 2 θ H H ′′ ( B H ) B H ′ ( θ H ) = 0 , β ( θ H ) = 0 , Ω ( θ H ) = ω 0 , (5)
and, at θ = θ H \theta=\theta_H θ = θ H , the first Lyapunov coefficient l 1 l_1 l 1 is given by
l 1 ( θ H ) = 2 ( c 1 2 + c 0 c 2 ) + α m H H ( c 0 c 2 2 + c 1 2 c 2 − c 0 c 1 c 3 ) 4 α m H H c 1 3 / 2 c 2 3 / 2 . (6) l_1(\theta_H)=\frac{2(c_1^2+c_0c_2)+\alpha mH_H\left(c_0c_2^2+c_1^2c_2-c_0c_1c_3\right)}{4\alpha m\sqrt{H_H}\,c_1^{3/2}c_2^{3/2}}. \tag*{(6)} l 1 ( θ H ) = 4 α m H H c 1 3/2 c 2 3/2 2 ( c 1 2 + c 0 c 2 ) + α m H H ( c 0 c 2 2 + c 1 2 c 2 − c 0 c 1 c 3 ) . (6)
Note that, l 1 ( θ H ) l_1(\theta_H) l 1 ( θ H ) may vanish only when the model parameters satisfy the degeneracy condition
2 ( c 1 2 + c 0 c 2 ) + α m H H ( c 0 c 2 2 + c 1 2 c 2 − c 0 c 1 c 3 ) = 0. 2(c_1^2+c_0c_2)+\alpha mH_H\left(c_0c_2^2+c_1^2c_2-c_0c_1c_3\right)=0. 2 ( c 1 2 + c 0 c 2 ) + α m H H ( c 0 c 2 2 + c 1 2 c 2 − c 0 c 1 c 3 ) = 0.
which, whenever nonempty, defines a codimension-one degeneracy locus in the parameter space. Therefore, away from this exceptional set, the equilibrium C E − CE_- C E − loses stability through a nondegenerate Hopf bifurcation at θ = θ H \theta=\theta_H θ = θ H . □ \square □
Corollary 3.1 (Limit cycles and stability)
Let the hypotheses of Theorem 3.1 be satisfied. Then, l 1 ( θ H ) < 0 l_1(\theta_H)<0 l 1 ( θ H ) < 0 and every Hopf bifurcation occurring along the coexistence branch C E − CE_- C E − is supercritical. Moreover, since β ′ ( θ H ) < 0 \beta'(\theta_H)<0 β ′ ( θ H ) < 0 , a family of asymptotically stable periodic orbits bifurcates from C E − CE_- C E − for θ < θ H \theta<\theta_H θ < θ H .
Proof. At the θ = θ H \theta=\theta_H θ = θ H , the denominator of the Lyapunov coefficient (6) is strictly positive so that
sgn l 1 ( θ H ) = sgn [ 2 ( c 1 2 + c 0 c 2 ) + α m H H ( c 0 c 2 2 + c 1 2 c 2 − c 0 c 1 c 3 ) ] . \operatorname{sgn}l_1(\theta_H)=\operatorname{sgn}\left[2(c_1^2+c_0c_2)+\alpha mH_H\left(c_0c_2^2+c_1^2c_2-c_0c_1c_3\right)\right]. sgn l 1 ( θ H ) = sgn [ 2 ( c 1 2 + c 0 c 2 ) + α m H H ( c 0 c 2 2 + c 1 2 c 2 − c 0 c 1 c 3 ) ] .
After some algebraic manipulations, it is possible to recast Δ H \Delta_H Δ H into
Δ H = 4 B H P ( B H , μ ) ( 1 + B H ) 2 [ ( 1 − μ ) B H + μ + 1 ] , \Delta_H=\frac{4B_HP(B_H,\mu)}{(1+B_H)^2\left[(1-\mu)B_H+\mu+1\right]}, Δ H = ( 1 + B H ) 2 [ ( 1 − μ ) B H + μ + 1 ] 4 B H P ( B H , μ ) ,
where
P ( B , μ ) = ( 1 − μ ) 2 ( 2 μ − 1 ) B 4 − ( μ − 1 ) ( μ 2 + 6 μ − 4 ) B 3 + 3 ( μ 2 + 2 μ − 2 ) B 2 + ( 3 μ 2 − 2 μ − 4 ) B − ( 2 μ + 1 ) . P(B,\mu)=(1-\mu)^2(2\mu-1)B^4-(\mu-1)(\mu^2+6\mu-4)B^3+3(\mu^2+2\mu-2)B^2+(3\mu^2-2\mu-4)B-(2\mu+1). P ( B , μ ) = ( 1 − μ ) 2 ( 2 μ − 1 ) B 4 − ( μ − 1 ) ( μ 2 + 6 μ − 4 ) B 3 + 3 ( μ 2 + 2 μ − 2 ) B 2 + ( 3 μ 2 − 2 μ − 4 ) B − ( 2 μ + 1 ) .
For μ ∈ [ 1 / 4 , 1 / 2 ] \mu\in[1/4,1/2] μ ∈ [ 1/4 , 1/2 ] , all the coefficients of P P P are nonpositive and its constant term is strictly negative so that P ( B , μ ) < 0 P(B,\mu)<0 P ( B , μ ) < 0 . For μ ∈ ( 1 / 2 , 1 ] \mu\in(1/2,1] μ ∈ ( 1/2 , 1 ] , the condition C ′ ′ ( B H ) > 0 C''(B_H)>0 C ′′ ( B H ) > 0 gives 0 < ( 2 μ − 1 ) B H < 1 0<(2\mu-1)B_H<1 0 < ( 2 μ − 1 ) B H < 1 so that P ( B H , μ ) < 0 P(B_H,\mu)<0 P ( B H , μ ) < 0 (see the Appendix A for more details). Therefore, Δ H < 0 \Delta_H<0 Δ H < 0 implies l 1 ( θ H ) < 0 l_1(\theta_H)<0 l 1 ( θ H ) < 0 and the Hopf bifurcation is supercritical. Finally, since β ′ ( θ H ) < 0 \beta'(\theta_H)<0 β ′ ( θ H ) < 0 (see (5)), the stable periodic orbits occur for θ ≲ θ H \theta\lesssim\theta_H θ ≲ θ H .
□ \square □
From an ecological viewpoint, the Hopf bifurcation highlights the transition from stable coexistence to persistent vegetation-herbivore oscillations driven by the feedback between plant recovery, herbivore grazing, and toxin-mediated inhibition.
Before proceeding to the global phase-plane analysis, let us consider the parameter regime in which both coexistence equilibria C E − CE_- C E − and C E + CE_+ C E + are biologically admissible. In this case, the local bifurcation structure can be summarized by the qualitative diagram shown in Figure 2 . This diagram does not represent the entire parameter space, but rather provides a schematic overview of dynamical transitions occurring as the herbivore mortality parameter θ \theta θ varies. In particular, the coexistence equilibrium C E − CE_- C E − loses stability at the Hopf threshold θ H \theta_H θ H (yellow square), where a family of stable periodic orbits emerges (light blue solid lines), while the two coexistence branches subsequently merge at the saddle-node threshold θ S N \theta_{SN} θ S N (black circle). The stability transition of the herbivore-free equilibrium H F E HFE H F E at θ H F E \theta_{HFE} θ H F E is also shown (magenta line). In detail, the diagram highlights four qualitatively distinct dynamical regimes:
Figure 2. Qualitative bifurcation diagram with respect to the herbivore mortality parameter θ \theta θ . Solid and dashed curves denote stable and unstable branches, respectively. Yellow square represents the Hopf bifurcation, whereas the black circle the saddle-node one.
i) for θ < θ H \theta<\theta_H θ < θ H , the equilibrium C E − CE_- C E − is unstable and the long-term dynamics are governed by a stable limit cycle;
ii) for θ H < θ < θ H F E \theta_H<\theta<\theta_{HFE} θ H < θ < θ H F E , the equilibrium C E − CE_- C E − is asymptotically stable, providing the unique attractor of the system;
iii) for θ H F E < θ < θ S N \theta_{HFE}<\theta<\theta_{SN} θ H F E < θ < θ S N , both C E − CE_- C E − and H F E HFE H F E are locally asymptotically stable, leading to a bistable regime in which the asymptotic outcome depends on the initial conditions;
iv) for θ > θ S N \theta>\theta_{SN} θ > θ S N , the coexistence equilibria disappear through the saddle-node bifurcation and H F E HFE H F E becomes the only stable equilibrium.
Although Figure 2 illustrates only one representative bifurcation scenario, the local results obtained above completely determine the qualitative equilibrium configurations arising throughout the parameter space. Different combinations of biologically admissible equilibria and their stability properties give rise to different phase-plane organizations. We therefore proceed to a global classification in the parameter plane.
Global phase-plane organization
The local analysis developed in Section 3 provides a complete characterization of the equilibria and of the bifurcation mechanisms supported by system (1). However, local information alone is insufficient to determine the global dynamics. The long-term behaviour of the solutions is also shaped by the arrangement of invariant manifolds and by the possible occurrence of heteroclinic and homoclinic connections [23 , 24 ] . The aim of this section is to identify the invariant structures that organize the phase portrait and to determine how they depend on the model parameters.
Qualitative phase-plane classification
The existence and stability results established in Section 3 provide the basic ingredients required to characterize the global dynamics of system (1). Let us investigate a qualitative classification of the phase-plane configurations, which provides the framework for the analysis of the heteroclinic and homoclinic structures discussed below. For later convenience, let us introduce the Hopf threshold in the parameter plane by considering the inverse relation μ = μ H ( θ ) \mu= \mu_H(\theta) μ = μ H ( θ ) , defined implicitly by θ = θ H ( μ ) \theta= \theta_H(\mu) θ = θ H ( μ ) .
Proposition 4.1 (Parameter-plane classification) Let system (1) be defined in the biologically admissible region Ω = R + 2 \Omega= \mathbb{R}^{2}_{+} Ω = R + 2 . Then exactly one of the following qualitative dynamical regimes occurs:
(I) If ( μ > μ S N ∧ θ < θ M ) ∨ ( μ > μ H F E ∧ θ > θ M ) (\mu> \mu_{SN} \wedge\theta< \theta_M) \vee(\mu> \mu_{HFE} \wedge\theta> \theta_M) ( μ > μ S N ∧ θ < θ M ) ∨ ( μ > μ H F E ∧ θ > θ M ) , the only biologically admissible equilibria are T E TE T E and H F E HFE H F E . Moreover, T E TE T E is a saddle point and H F E HFE H F E is a stable node;
(II) If max { μ H , μ H F E } < μ < μ S N ∧ θ < θ M \max\{\mu_H,\mu_{HFE}\} < \mu< \mu_{SN} \wedge\theta< \theta_M max { μ H , μ H F E } < μ < μ S N ∧ θ < θ M , the equilibria are T E TE T E , H F E HFE H F E , C E + CE_{+} C E + and C E − CE_{-} C E − , where T E TE T E and C E + CE_{+} C E + are saddles whereas H F E HFE H F E is a stable node and C E − CE_{-} C E − a stable node or focus;
(III) If μ H F E < μ < μ H \mu_{HFE} < \mu< \mu_H μ H F E < μ < μ H , the equilibria are T E TE T E , H F E HFE H F E , C E + CE_{+} C E + and C E − CE_{-} C E − , where T E TE T E and C E + CE_{+} C E + are saddles whereas C E − CE_{-} C E − is an unstable node or focus and H F E HFE H F E is a stable node;
(IV) If μ < min { μ H F E , μ H } \mu< \min\{\mu_{HFE},\mu_H\} μ < min { μ H F E , μ H } , the equilibria are T E TE T E , H F E HFE H F E , C E − CE_{-} C E − . Moreover, T E TE T E and H F E HFE H F E are saddles, whereas C E − CE_{-} C E − is an unstable node or focus;
(V) If μ H < μ < μ H F E \mu_H < \mu< \mu_{HFE} μ H < μ < μ H F E , the equilibria are T E TE T E , H F E HFE H F E , C E − CE_{-} C E − , where T E TE T E and H F E HFE H F E are saddles and C E − CE_{-} C E − is a stable node or focus.
Proof. The result follows directly by combining Lemmas 3.1-3.2. Indeed, Lemma 3.1 determines the regions of the parameter space in which the coexistence equilibria C E − CE_{-} C E − and C E + CE_{+} C E + are biologically admissible, whereas Lemma 3.2 completely characterizes the stability of all equilibrium states. Consequently, the thresholds μ H F E \mu_{HFE} μ H F E , μ S N \mu_{SN} μ S N , μ H \mu_H μ H , and θ M \theta_M θ M partition the parameter space into five qualitatively distinct configurations, corresponding to different arrangements of stable and unstable manifolds.
The resulting decomposition is summarized in Figure 3 , where each region is characterized by a specific combination of extinction and coexistence equilibria and therefore by a different set of admissible heteroclinic or homoclinic connections.
Figure 3. Partition of the parameter plane ( θ , μ ) (\theta,\mu) ( θ , μ ) induced by the thresholds μ S N \mu_{SN} μ S N , μ H F E \mu_{HFE} μ H F E , and μ H \mu_H μ H . The different regions correspond to qualitatively distinct equilibria configurations in the phase plane of system (1).
Heteroclinic connections
Theorem 4.1 (Existence of heteroclinic connections) Let us consider the system (1) in the closed positive quadrant Ω = R + 2 \Omega= \mathbb{R}^{2}_{+} Ω = R + 2 and assume that all the equilibria are hyperbolic. Then the following statements hold:
(i) The coordinate axes ∂ B Ω : = { ( 0 , H ) : H ≥ 0 } \partial_B\Omega:= \{(0,H) : H \ge0\} ∂ B Ω := {( 0 , H ) : H ≥ 0 } and ∂ H Ω : = { ( B , 0 ) : B ≥ 0 } \partial_H\Omega:= \{(B,0) : B \ge0\} ∂ H Ω := {( B , 0 ) : B ≥ 0 } are invariant under the flow of system (1) . Moreover, the open boundary segment Γ : = { ( B , 0 ) : 0 < B < α } \Gamma:= \{(B,0) : 0 < B < \alpha\} Γ := {( B , 0 ) : 0 < B < α } is a heteroclinic orbit connecting the trivial equilibrium T E TE T E to the herbivore-free equilibrium H F E HFE H F E . More precisely, Γ ⊂ W u ( T E ) ∩ W s ( H F E ) \Gamma\subset W^u(TE) \cap W^s(HFE) Γ ⊂ W u ( T E ) ∩ W s ( H F E ) , and
lim t → − ∞ γ ( t ) = T E , lim t → + ∞ γ ( t ) = H F E \lim_{t\to-\infty}\gamma(t)=TE,\qquad\lim_{t\to+\infty}\gamma(t)=HFE t → − ∞ lim γ ( t ) = T E , t → + ∞ lim γ ( t ) = H F E for every parametrization γ \gamma γ of Γ \Gamma Γ compatible with the flow. Hence, T E → H F E TE \to HFE T E → H F E is present in every parameter region;
(ii) No orbit contained in int Ω \operatorname{int}\Omega int Ω can converge to T E TE T E as t → + ∞ t \to+\infty t → + ∞ so that no heteroclinic connection contained in int Ω \operatorname{int}\Omega int Ω can have T E TE T E as its final equilibrium. Whenever H F E HFE H F E is a saddle, no orbit contained in int Ω \operatorname{int}\Omega int Ω can converge to H F E HFE H F E as t → + ∞ t \to+\infty t → + ∞ so that the only heteroclinic connection terminating at H F E HFE H F E is the boundary orbit Γ \Gamma Γ ;
(iii) In Region I, the boundary connection T E → H F E TE \to HFE T E → H F E is the only heteroclinic orbit in Ω \Omega Ω ;
(iv) In Region II, besides the boundary heteroclinic connection T E → H F E TE \to HFE T E → H F E , there exists a heteroclinic orbit connecting the C E + CE_{+} C E + to H F E HFE H F E . More precisely, the branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) leaving C E + CE_{+} C E + in the southeastern direction is contained in the positively invariant region R + : = [ B + , α ] × [ 0 , H + ] R_{+}:=[B_{+},\alpha]\times[0,H_{+}] R + := [ B + , α ] × [ 0 , H + ] and converges to H F E HFE H F E as t → + ∞ t\to+\infty t → + ∞ . Moreover, the only other heteroclinic connection between distinct equilibria that is compatible with the equilibrium configuration is C E + → C E − CE_{+}\to CE_{-} C E + → C E − which, in the absence of homoclinic connections generated by C E + CE_{+} C E + , is characterized by the branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) leaving C E + CE_{+} C E + through the northwestern sector;
(v) In Region III, besides the boundary heteroclinic connection T E → H F E TE\to HFE T E → H F E , there exists a heteroclinic orbit C E + → H F E CE_{+}\to HFE C E + → H F E as obtained in Region II. Moreover, no other heteroclinic connection between distinct equilibria contained in Ω \Omega Ω may exist;
(vi) In Region IV, the only heteroclinic connection is T E → H F E TE\to HFE T E → H F E ;
(vii) In Region V, the heteroclinic connections T E → H F E TE\to HFE T E → H F E and H F E → C E − HFE\to CE_{-} H F E → C E − exist and are the only heteroclinic orbits between distinct equilibria contained in Ω \Omega Ω .
Proof. Let
F ( B , H ) = ( B ( 1 − B α ) − m C ( B ) H H ( m C ( B ) − θ ) ) (7) F(B,H)=\begin{pmatrix}
B\left(1-\frac{B}{\alpha}\right)-mC(B)H\\
H(mC(B)-\theta)
\end{pmatrix} \tag*{(7)} F ( B , H ) = ( B ( 1 − α B ) − m C ( B ) H H ( m C ( B ) − θ ) ) (7)
denote the vector field associated with system (1). Since F ∈ C ∞ ( Ω ) F\in C^{\infty}(\Omega) F ∈ C ∞ ( Ω ) , the corresponding initial-value problem admits a unique local solution through every point of Ω \Omega Ω [25 ] . Therefore, let us divide the proof into three steps.
Step 1: invariance of the coordinate axes and the boundary connection. The projection of system (1) to the vertical axis B = 0 B=0 B = 0 is
B ˙ = 0 , H ˙ = − θ H . \dot{B}=0,\qquad\dot{H}=-\theta H. B ˙ = 0 , H ˙ = − θ H .
Therefore, the set ∂ B Ω = { ( 0 , H ) : H ≥ 0 } \partial_{B}\Omega=\{(0,H):H\geq0\} ∂ B Ω = {( 0 , H ) : H ≥ 0 } is invariant. Similarly, along the horizontal axis H = 0 H=0 H = 0 we have
B ˙ = B ( 1 − B α ) , H ˙ = 0. \dot{B}=B\left(1-\frac{B}{\alpha}\right),\qquad\dot{H}=0. B ˙ = B ( 1 − α B ) , H ˙ = 0.
so that ∂ H Ω = { ( B , 0 ) : B ≥ 0 } \partial_{H}\Omega=\{(B,0):B\geq0\} ∂ H Ω = {( B , 0 ) : B ≥ 0 } is invariant too. Moreover, for 0 < B < α 0<B<\alpha 0 < B < α , the scalar equation on ∂ H Ω \partial_{H}\Omega ∂ H Ω satisfies B ˙ > 0 \dot{B}>0 B ˙ > 0 so that its solution with initial condition B ( 0 ) = B 0 ∈ ( 0 , α ) B(0)=B_{0}\in(0,\alpha) B ( 0 ) = B 0 ∈ ( 0 , α ) is given by
B ( t ) = α B 0 e t α + B 0 ( e t − 1 ) . B(t)=\frac{\alpha B_{0}e^{t}}{\alpha+B_{0}(e^{t}-1)}. B ( t ) = α + B 0 ( e t − 1 ) α B 0 e t .
Consequently,
lim t → − ∞ B ( t ) = 0 , lim t → + ∞ B ( t ) = α . \lim_{t\to-\infty}B(t)=0,\qquad\lim_{t\to+\infty}B(t)=\alpha. t → − ∞ lim B ( t ) = 0 , t → + ∞ lim B ( t ) = α .
and, if γ ( t ) : = ( B ( t ) , 0 ) \gamma(t) := (B(t),0) γ ( t ) := ( B ( t ) , 0 ) , then
lim t → − ∞ γ ( t ) = T E , lim t → + ∞ γ ( t ) = H F E . \lim_{t\to-\infty}\gamma(t)=TE,\qquad\lim_{t\to+\infty}\gamma(t)=HFE. t → − ∞ lim γ ( t ) = T E , t → + ∞ lim γ ( t ) = H F E .
Therefore, it follows that the boundary segment Γ \Gamma Γ is a heteroclinic orbit satisfying Γ ⊂ W u ( T E ) ∩ W s ( H F E ) \Gamma\subset W^{u}(TE) \cap W^{s}(HFE) Γ ⊂ W u ( T E ) ∩ W s ( H F E ) so that T E → H F E TE \to HFE T E → H F E exists for every admissible choice of the parameters.
Step 2: boundary localization of the stable and unstable manifolds. At the trivial equilibrium T E TE T E , the eigenvalues and corresponding eigenspaces are
λ u = 1 , E u ( T E ) = span { ( 1 , 0 ) } , \lambda_{u}=1,\qquad E^{u}(TE)=\operatorname{span}\{(1,0)\}, λ u = 1 , E u ( T E ) = span {( 1 , 0 )} ,
λ s = − θ , E s ( T E ) = span { ( 0 , 1 ) } . \lambda_{s}=-\theta,\qquad E^{s}(TE)=\operatorname{span}\{(0,1)\}. λ s = − θ , E s ( T E ) = span {( 0 , 1 )} .
The invariant horizontal and vertical axes are tangent at T E TE T E to E u ( T E ) E^{u}(TE) E u ( T E ) and E s ( T E ) E^{s}(TE) E s ( T E ) , respectively. By the uniqueness of the local stable and unstable manifolds of a hyperbolic equilibrium [25 ] ,
W l o c u ( T E ) ∩ Ω ⊆ { H = 0 } , W l o c s ( T E ) ∩ Ω ⊆ { B = 0 } . W^{u}_{\mathrm{loc}}(TE)\cap\Omega\subseteq\{H=0\},\qquad W^{s}_{\mathrm{loc}}(TE)\cap\Omega\subseteq\{B=0\}. W loc u ( T E ) ∩ Ω ⊆ { H = 0 } , W loc s ( T E ) ∩ Ω ⊆ { B = 0 } .
We claim that no orbit contained in int Ω \operatorname{int}\Omega int Ω can converge to T E TE T E as t → + ∞ t\to+\infty t → + ∞ . Indeed, suppose that a solution x ( t ) = ( B ( t ) , H ( t ) ) ∈ int Ω x(t)=(B(t),H(t))\in\operatorname{int}\Omega x ( t ) = ( B ( t ) , H ( t )) ∈ int Ω satisfies
lim t → + ∞ x ( t ) = T E . \lim_{t\to+\infty}x(t)=TE. t → + ∞ lim x ( t ) = T E .
For sufficiently large t t t , the orbit must belong to W l o c s ( T E ) W^{s}_{\mathrm{loc}}(TE) W loc s ( T E ) . Therefore, it must lie on the invariant axis B = 0 B=0 B = 0 , contradicting x ( t ) ∈ int Ω x(t)\in\operatorname{int}\Omega x ( t ) ∈ int Ω . Hence W s ( T E ) ∩ int Ω = ∅ W^{s}(TE)\cap\operatorname{int}\Omega=\varnothing W s ( T E ) ∩ int Ω = ∅ so that no heteroclinic orbit contained in int Ω \operatorname{int}\Omega int Ω can terminate at T E TE T E . Note that the same argument also shows that no heteroclinic orbit from T E TE T E can enter int Ω \operatorname{int}\Omega int Ω . Indeed, any orbit converging to T E TE T E as t → − ∞ t\to-\infty t → − ∞ must eventually belong to W l o c u ( T E ) W^{u}_{\mathrm{loc}}(TE) W loc u ( T E ) , which is contained in the invariant axis H = 0 H=0 H = 0 . By uniqueness of solutions, such an orbit remains on H = 0 H=0 H = 0 for all times for which it is defined. Therefore, the only heteroclinic orbit issuing from T E TE T E in Ω \Omega Ω is the boundary connection T E ⟶ H F E TE\longrightarrow HFE T E ⟶ H F E .
Let us now consider H F E HFE H F E . Its negative eigenvalue is λ s = − 1 \lambda_{s}=-1 λ s = − 1 and the corresponding eigenspace is E s ( H F E ) = span { ( 1 , 0 ) } E^{s}(HFE)=\operatorname{span}\{(1,0)\} E s ( H F E ) = span {( 1 , 0 )} . Since the horizontal axis H = 0 H=0 H = 0 is invariant and tangent to E s ( H F E ) E^{s}(HFE) E s ( H F E ) at H F E HFE H F E , uniqueness of the local stable manifold gives W l o c s ( H F E ) ∩ Ω ⊆ { H = 0 } W^{s}_{\mathrm{loc}}(HFE)\cap\Omega\subseteq\{H=0\} W loc s ( H F E ) ∩ Ω ⊆ { H = 0 } . As done for T E TE T E , if an orbit contained in int Ω \operatorname{int}\Omega int Ω converged to H F E HFE H F E as t → + ∞ t\to+\infty t → + ∞ , then it would eventually belong to W l o c s ( H F E ) W^{s}_{\mathrm{loc}}(HFE) W loc s ( H F E ) and hence to H = 0 H=0 H = 0 , which is impossible. Thus, whenever H F E HFE H F E is a saddle, it follows W s ( H F E ) ∩ int Ω = ∅ W^{s}(HFE)\cap\operatorname{int}\Omega=\varnothing W s ( H F E ) ∩ int Ω = ∅ so that the only heteroclinic connection ending at H F E HFE H F E is Γ \Gamma Γ .
Step 3: classification in the five parameter regions. Let us now combine the previous invariant-manifold properties with the equilibrium classification established in Proposition 4.1.
Region I.
The only equilibria in Ω \Omega Ω are T E TE T E and H F E HFE H F E , with T E TE T E a saddle and H F E HFE H F E asymptotically stable. Since the unstable manifold of T E TE T E in Ω \Omega Ω is contained in the invariant axis H = 0 H=0 H = 0 , its admissible branch generates the boundary connection Γ \Gamma Γ . There is no other saddle equilibrium from which a distinct heteroclinic orbit could originate so that T E ⟶ H F E TE\longrightarrow HFE T E ⟶ H F E is the only connection contained in Ω \Omega Ω .
Region II
In this region, T E TE T E and C E + CE_{+} C E + are saddles, whereas H F E HFE H F E and C E − CE_{-} C E − are asymptotically stable. As proved above, no heteroclinic orbit can terminate at T E TE T E and no interior ones can exit from it (statement (i)). Therefore, any heteroclinic connection involving T E TE T E must coincide with T E → H F E TE \to HFE T E → H F E and the only remaining saddle from which a heteroclinic orbit may issue is C E + CE_{+} C E + . If a branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) converges to an equilibrium, the only remaining possibilities are H F E HFE H F E and C E − CE_{-} C E − .
Let us now show that at least one branch of the unstable manifold of C E + CE_{+} C E + converges to H F E HFE H F E . To this aim, let us consider the positively invariant closed rectangle R + : = [ B + , α ] × [ 0 , H + ] \mathcal{R}_{+}:=[B_{+},\alpha]\times[0,H_{+}] R + := [ B + , α ] × [ 0 , H + ] . Indeed, on the left boundary (B = B + B=B_{+} B = B + and 0 ≤ H ≤ H + 0\leq H\leq H_{+} 0 ≤ H ≤ H + ) one has
B ˙ = B + ( 1 − B + α ) − m C ( B + ) H = θ ( H + − H ) ≥ 0 , \dot{B}=B_{+}\left(1-\frac{B_{+}}{\alpha}\right)-mC(B_{+})H=\theta(H_{+}-H)\geq0, B ˙ = B + ( 1 − α B + ) − m C ( B + ) H = θ ( H + − H ) ≥ 0 ,
on the right one (B = α B=\alpha B = α ) it follows B ˙ = − m C ( α ) H ≤ 0 \dot{B}=-mC(\alpha)H\leq0 B ˙ = − m C ( α ) H ≤ 0 , on the upper one (H = H + H=H_{+} H = H + and B + < B ≤ α B_{+}<B\leq\alpha B + < B ≤ α ) H ˙ = H + ( m C ( B ) − θ ) < 0 \dot{H}=H_{+}(mC(B)-\theta)<0 H ˙ = H + ( m C ( B ) − θ ) < 0 and the lower one (H = 0 H=0 H = 0 ) is invariant (see Figure 4 ).
Figure 4. Geometric organization of the stable and unstable manifolds of the saddle equilibrium C E + CE_{+} C E + in Region II. The southeastern branch γ S E \gamma_{\mathrm{SE}} γ SE (lower green curve) enters the positively invariant rectangle R + = [ B + , α ] × [ 0 , H + ] R_{+}=[B_{+},\alpha]\times[0,H_{+}] R + = [ B + , α ] × [ 0 , H + ] via W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) , whereas the northwestern one γ N W \gamma_{\mathrm{NW}} γ NW (upper green curve) lies on the opposite side of the stable manifold W s ( C E + ) W^{s}(CE_{+}) W s ( C E + ) (magenta curve), which separates the basins of attraction of C E − CE_{-} C E − and H F E HFE H F E . Blue (red) arrows indicate the direction of the eigenvectors associated with the unstable (stable) eigenspace of the linearized system at C E + CE_{+} C E + . Black crosses denote saddle equilibria and red circles the asymptotically stable ones.
Let λ u > 0 \lambda_{u}>0 λ u > 0 be the unstable eigenvalue and v u = ( λ u , m C ′ ( B + ) H + ) T v_{u}=(\lambda_{u},mC'(B_{+})H_{+})^{T} v u = ( λ u , m C ′ ( B + ) H + ) T the associated eigenvector. Being C E + CE_{+} C E + on the decreasing branch of C C C , C ′ ( B + ) < 0 C'(B_{+})<0 C ′ ( B + ) < 0 so that one branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) enters the interior of R + \mathcal{R}_{+} R + through the southeast sector B > B + B>B_{+} B > B + and H < H + H<H_{+} H < H + . Let γ S E ( t ) \gamma_{\mathrm{SE}}(t) γ SE ( t ) denote the corresponding orbit. By the positive invariance and compactness of R + \mathcal{R}_{+} R + , its positive semiorbit is bounded and its ω \omega ω -limit set is nonempty, compact, connected, and invariant. Furthermore, H ˙ < 0 \dot{H}<0 H ˙ < 0 along every portion of γ S E \gamma_{\mathrm{SE}} γ SE contained in int R + \operatorname{int}\mathcal{R}_{+} int R + so that H H H is strictly decreasing along γ S E \gamma_{\mathrm{SE}} γ SE , which excludes both periodic orbits and a homoclinic return to C E + CE_{+} C E + . Since the only equilibria contained in R + \mathcal{R}_{+} R + are C E + CE_{+} C E + and H F E HFE H F E , the Poincaré-Bendixson theorem [25 ] yields ω ( γ S E ) = { H F E } \omega(\gamma_{\mathrm{SE}})=\{HFE\} ω ( γ SE ) = { H F E } . Hence γ S E ⊂ W u ( C E + ) ∩ W s ( H F E ) \gamma_{\mathrm{SE}}\subset W^{u}(CE_{+})\cap W^{s}(HFE) γ SE ⊂ W u ( C E + ) ∩ W s ( H F E ) , and the heteroclinic connection C E + → H F E CE_{+}\to HFE C E + → H F E exists.
Let γ N W ( t ) \gamma_{\mathrm{NW}}(t) γ NW ( t ) denote the opposite branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) , which locally leaves C E + CE_{+} C E + through the northwest sector B < B + B<B_{+} B < B + and H > H + H>H_{+} H > H + . By the boundedness of the positive solutions of system (1) [15 ] , its positive semiorbit is bounded. Hence ω ( γ N W ) \omega(\gamma_{\mathrm{NW}}) ω ( γ NW ) is nonempty, compact, connected, and invariant. We first show that the northwestern branch cannot converge to H F E HFE H F E . Let B ( C E − ) \mathcal{B}(CE_{-}) B ( C E − ) and B ( H F E ) \mathcal{B}(HFE) B ( H F E ) denote the basins of attraction of the two asymptotically stable equilibria in Region II. These sets are nonempty, open, disjoint, and positively invariant. Moreover, since C E − CE_{-} C E − is locally asymptotically stable, Lemma 3.2 gives H ′ ( B − ) < 0 H'(B_{-})<0 H ′ ( B − ) < 0 [18 ] . Therefore, by Proposition 3.2 in [15 ] , system (1) does not admit nonconstant periodic orbit in int Ω \operatorname{int}\Omega int Ω . Consequently, the Poincaré-Bendixson theorem implies that every bounded positive orbit that does not belong to the stable manifold of C E + CE_{+} C E + converges to one of the two sinks C E − CE_{-} C E − and H F E HFE H F E . We claim that the common boundary of the two basins in int Ω \operatorname{int}\Omega int Ω is the stable manifold of C E + CE_{+} C E + , namely W s ( C E + ) W^{s}(CE_{+}) W s ( C E + ) , provided no homoclinic loop based at C E + CE_{+} C E + exists. No point on this boundary can converge to either C E − CE_{-} C E − or H F E HFE H F E , since their basins are open. Its ω \omega ω -limit set cannot contain T E TE T E (statement (i)) and it cannot be a periodic orbit [15 ] . Since C E + CE_{+} C E + is the only saddle equilibrium in int Ω \operatorname{int}\Omega int Ω , the Poincaré-Bendixson theorem gives
∂ B ( C E − ) ∩ int Ω ⊆ W s ( C E + ) , ∂ B ( H F E ) ∩ int Ω ⊆ W s ( C E + ) . \partial\mathcal{B}(CE_{-})\cap\operatorname{int}\Omega\subseteq W^{s}(CE_{+}),\qquad
\partial\mathcal{B}(HFE)\cap\operatorname{int}\Omega\subseteq W^{s}(CE_{+}). ∂ B ( C E − ) ∩ int Ω ⊆ W s ( C E + ) , ∂ B ( H F E ) ∩ int Ω ⊆ W s ( C E + ) .
Therefore, the local stable manifold of C E + CE_{+} C E + separates a sufficiently small neighbourhood of the saddle into two connected components. Since the southeastern branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) belongs to B ( H F E ) \mathcal{B}(HFE) B ( H F E ) , whereas the northwestern branch belongs to B ( C E − ) \mathcal{B}(CE_{-}) B ( C E − ) , every point of W s ( C E + ) W^{s}(CE_{+}) W s ( C E + ) is an accumulation point for both basins. Therefore W s ( C E + ) ⊆ ∂ B ( C E − ) ∩ ∂ B ( H F E ) W^{s}(CE_{+})\subseteq\partial\mathcal{B}(CE_{-})\cap\partial\mathcal{B}(HFE) W s ( C E + ) ⊆ ∂ B ( C E − ) ∩ ∂ B ( H F E ) so that
∂ B C E − ∩ int Ω = ∂ B H F E ∩ int Ω = W s ( C E + ) . \partial B_{CE_{-}}\cap\operatorname{int}\Omega=\partial B_{HFE}\cap\operatorname{int}\Omega=W^{s}(CE_{+}). ∂ B C E − ∩ int Ω = ∂ B H F E ∩ int Ω = W s ( C E + ) .
Since the southeastern branch converges to H F E HFE H F E , γ S E ⊂ B ( H F E ) \gamma_{\mathrm{SE}}\subset\mathcal{B}(HFE) γ SE ⊂ B ( H F E ) . Therefore the northwestern branch lies on the B ( C E − ) \mathcal{B}(CE_{-}) B ( C E − ) side of the stable branch and cannot converge to H F E HFE H F E so that ω ( γ N W ) ⊆ { C E − , C E + } \omega(\gamma_{\mathrm{NW}})\subseteq\{CE_{-},CE_{+}\} ω ( γ NW ) ⊆ { C E − , C E + } . If γ N W \gamma_{\mathrm{NW}} γ NW returns to C E + CE_{+} C E + , it generates a homoclinic loop based at C E + CE_{+} C E + , otherwise its ω \omega ω -limit set cannot contain C E + CE_{+} C E + and ω ( γ N W ) = { C E − } \omega(\gamma_{\mathrm{NW}})=\{CE_{-}\} ω ( γ NW ) = { C E − } . Consequently, if the northwestern branch does not form a homoclinic loop, it must converge to C E − CE_{-} C E − yielding the heteroclinic connection C E + → C E − CE_{+}\to CE_{-} C E + → C E − . However, since the existence of a homoclinic loop does not preclude the existence of such a heteroclinic connection, the contrary is not necessarily true. Note that a qualitative sketch of the above dynamics is provided in Figure 4 .
Region III
The argument concerning T E TE T E and C E + CE_{+} C E + is the same as in Region II. In particular, the branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) entering the positively invariant region R + R_{+} R + converges to H F E HFE H F E , thereby generating the heteroclinic connection C E + → H F E CE_{+}\to HFE C E + → H F E . Since C E − CE_{-} C E − is an unstable node or focus, its stable manifold is trivial, namely W s ( C E − ) = { C E − } W^{s}(CE_{-})=\{CE_{-}\} W s ( C E − ) = { C E − } so that no nonconstant orbit can converge to C E − CE_{-} C E − as t → + ∞ t\to+\infty t → + ∞ . Moreover, the periodic orbit Γ H \Gamma_{H} Γ H surrounding C E − CE_{-} C E − separates C E − CE_{-} C E − from H F E HFE H F E (see Theorem 3.1). Since trajectories of the planar system cannot cross the invariant curve Γ H \Gamma_{H} Γ H no orbit issuing from C E − CE_{-} C E − can converge to H F E HFE H F E . Thus, the heteroclinic connection C E − → H F E CE_{-}\to HFE C E − → H F E is also excluded. Thus the only heteroclinic connections contained in Ω \Omega Ω are T E → H F E TE\to HFE T E → H F E and C E + → H F E CE_{+}\to HFE C E + → H F E .
Region IV
In this region, T E TE T E and H F E HFE H F E are saddle points, whereas C E − CE_{-} C E − is a hyperbolic unstable equilibrium. The arguments of Step 2 exclude any interior heteroclinic orbit terminating at T E TE T E or at H F E HFE H F E . Moreover, no interior heteroclinic orbit can issue from T E TE T E , since its unstable manifold in Ω \Omega Ω is contained in H = 0 H=0 H = 0 . It follows that the boundary connection T E → H F E TE\to HFE T E → H F E is the only heteroclinic connection involving T E TE T E . Moreover, since W s ( C E − ) = { C E − } W^{s}(CE_{-})=\{CE_{-}\} W s ( C E − ) = { C E − } , no nonconstant orbit can converge to C E − CE_{-} C E − as t → + ∞ t \to+\infty t → + ∞ so that no heteroclinic connection having C E − CE_{-} C E − as terminal equilibrium can exist.
Region V.
The equilibria T E TE T E and H F E HFE H F E are saddles, whereas C E − CE_{-} C E − is locally asymptotically stable. As before, the only heteroclinic orbit issuing from T E TE T E is the boundary connection T E → H F E TE \to HFE T E → H F E .
Furthermore, since H F E HFE H F E is a saddle whose unstable eigendirection points transversally to the invariant boundary, a biologically admissible branch of W u ( H F E ) W^{u}(HFE) W u ( H F E ) may enter into Ω \Omega Ω . Let γ H F E \gamma_{HFE} γ H F E denote such a branch. Since all positive solutions of system (1) are bounded [15 ] , the positive semiorbit of γ H F E \gamma_{HFE} γ H F E is bounded as well. Therefore, its ω \omega ω -limit set is nonempty, compact, connected, and invariant. By Step 2, no orbit contained in int Ω \operatorname{int}\Omega int Ω can converge to either T E TE T E or H F E HFE H F E so that ω ( γ H F E ) \omega(\gamma_{HFE}) ω ( γ H F E ) cannot reduce to either boundary equilibrium. Since Region V admits no periodic orbits and C E − CE_{-} C E − is the only equilibrium contained in int Ω \operatorname{int}\Omega int Ω [15 ] , the Poincaré-Bendixson theorem yields ω ( γ H F E ) = { C E − } \omega(\gamma_{HFE}) = \{CE_{-}\} ω ( γ H F E ) = { C E − } so that γ H F E ⊂ W u ( H F E ) ∩ W s ( C E − ) \gamma_{HFE} \subset W^{u}(HFE) \cap W^{s}(CE_{-}) γ H F E ⊂ W u ( H F E ) ∩ W s ( C E − ) and the heteroclinic connection H F E → C E − HFE \to CE_{-} H F E → C E − exists.
Homoclinic connections
The heteroclinic classification obtained above completely determines the possible equilibrium-to-equilibrium connections supported by the system. A natural question is whether trajectories can also connect an equilibrium to itself, generating homoclinic structures. We now show that the geometric restrictions imposed by the invariant boundary and the equilibrium configuration severely limit such possibility.
Theorem 4.2 (Non-existence of homoclinic orbits)
Consider system (1) in the closed positive quadrant Ω = R + 2 \Omega= \mathbb{R}_{+}^{2} Ω = R + 2 and assume that all its equilibria are hyperbolic. Then the following statements hold:
(i) Neither T E TE T E nor H F E HFE H F E can be the base point of a nonconstant homoclinic orbit contained in Ω \Omega Ω ;
(ii) If C E − CE_{-} C E − exists, then no nonconstant homoclinic orbit to C E − CE_{-} C E − is admitted;
(iii) System (1) does not admit any homoclinic orbit in Regions I, IV, and V of Proposition 4.1;
(iv) In Regions II and III, any homoclinic orbit contained in Ω \Omega Ω , if it exists, must be based at C E + CE_{+} C E + .
Moreover, the southeastern branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) cannot belong to the homoclinic orbit.
Proof. Since a nonconstant homoclinic orbit must converge to the same equilibrium as t → ± ∞ t \to\pm\infty t → ± ∞ , its base point must be a saddle. At T E TE T E , the stable and unstable eigenspaces are E s ( T E ) = span { ( 0 , 1 ) } E^{s}(TE) = \operatorname{span}\{(0,1)\} E s ( T E ) = span {( 0 , 1 )} , E u ( T E ) = span { ( 1 , 0 ) } E^{u}(TE) = \operatorname{span}\{(1,0)\} E u ( T E ) = span {( 1 , 0 )} and the coordinate axes are tangent to these eigenspaces. By uniqueness of the local stable and unstable manifolds, W l o c s ( T E ) ∩ Ω ⊆ { B = 0 } W^{s}_{\mathrm{loc}}(TE) \cap\Omega\subseteq\{B = 0\} W loc s ( T E ) ∩ Ω ⊆ { B = 0 } and W l o c u ( T E ) ∩ Ω ⊆ { H = 0 } W^{u}_{\mathrm{loc}}(TE) \cap\Omega\subseteq\{H = 0\} W loc u ( T E ) ∩ Ω ⊆ { H = 0 } so that a nonconstant orbit leaving T E TE T E in Ω \Omega Ω lies on the horizontal axis, whereas an orbit converging to T E TE T E lies on the vertical axis. Since the two invariant axes intersect only at T E TE T E , they cannot form a nonconstant homoclinic loop. Thus no homoclinic orbit to T E TE T E exists.
Then, once H F E HFE H F E is asymptotically stable, it cannot be the base point of a nonconstant homoclinic orbit. On the other hand, if it is a saddle, its local stable manifold in Ω \Omega Ω is contained in the invariant axis H = 0 H = 0 H = 0 and the corresponding unstable branch entering into Ω \Omega Ω cannot return to H F E HFE H F E , because no orbit contained in int Ω \operatorname{int}\Omega int Ω can converge to H F E HFE H F E as t → + ∞ t \to+\infty t → + ∞ (see statement (ii) of Theorem 4.1). Therefore, no homoclinic orbit to H F E HFE H F E exists. Whenever C E − CE_{-} C E − is asymptotically stable (unstable), its unstable (stable) manifold is trivial so that W s ( C E − ) ∩ W u ( C E − ) = { C E − } W^{s}(CE_{-}) \cap W^{u}(CE_{-}) = \{CE_{-}\} W s ( C E − ) ∩ W u ( C E − ) = { C E − } , and hence no nonconstant homoclinic orbit to C E − CE_{-} C E − can occur. In Regions I, IV, and V, all possible base points of a homoclinic orbit have already been excluded. Therefore, no homoclinic orbit is admitted in such regions.
Finally, in Regions II and III the only interior saddle is C E + CE_{+} C E + so that any homoclinic orbit contained in Ω \Omega Ω , if present, must be based at such an equilibrium. By Theorem 4.1, the branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) entering the sector B > B + B > B_{+} B > B + and H < H + H < H_{+} H < H + remains in R + \mathcal{R}_{+} R + and converges to H F E HFE H F E . Consequently, this branch cannot return to C E + CE_{+} C E + and cannot be part of a homoclinic orbit. Thus, any possible homoclinic connection to C E + CE_{+} C E + would necessarily involve the opposite northwest branch of W u ( C E + ) W^{u}(CE_{+}) W u ( C E + ) .
Theorem 4.2 drastically restricts the possible occurrence of homoclinic connections. Having identified the unique geometric configuration compatible with such orbits, the next question is whether the corresponding stable and unstable manifolds can actually coincide for suitable parameter values. To this end, we employ a distance function between invariant manifolds [22 , 25 , 26 , 27 ] , which provides a criterion for the existence or nonexistence of homoclinic loops associated with C E + CE_{+} C E + .
Lemma 4.1 (Transverse section near a reference homoclinic connection). Assume that, for η = η ∗ \eta= \eta^{*} η = η ∗ , system (1) admits a homoclinic orbit Γ h o m \Gamma_{\mathrm{hom}} Γ hom based at the hyperbolic saddle C E + ( η ∗ ) CE_{+}(\eta^{*}) C E + ( η ∗ ) . Suppose that Γ h o m \Gamma_{\mathrm{hom}} Γ hom leaves C E + ( η ∗ ) CE_{+}(\eta^{*}) C E + ( η ∗ ) along the unstable branch γ N W u ( η ∗ ) \gamma_{\mathrm{NW}}^{u}(\eta^{*}) γ NW u ( η ∗ ) and returns to it along the selected stable branch γ s ( η ∗ ) \gamma^{s}(\eta^{*}) γ s ( η ∗ ) . Let q ∗ ∈ Γ h o m ∖ { C E + ( η ∗ ) } q^{*} \in\Gamma_{\mathrm{hom}} \setminus\{CE_{+}(\eta^{*})\} q ∗ ∈ Γ hom ∖ { C E + ( η ∗ )} . Then there exist a neighbourhood U U U of q ∗ q^{*} q ∗ , a local transverse section Σ ⊂ U \Sigma\subset U Σ ⊂ U through q ∗ q^{*} q ∗ , and an open interval J 0 ∋ η ∗ J_{0} \ni\eta^{*} J 0 ∋ η ∗ such that ∀ η ∈ J 0 \forall\eta\in J_{0} ∀ η ∈ J 0 :
(i) the continuation of the unstable orbit segment γ N W u ( η ∗ ) \gamma_{\mathrm{NW}}^{u}(\eta^{*}) γ NW u ( η ∗ ) reaching q ∗ q^{*} q ∗ intersects Σ \Sigma Σ transversely at a unique point q u ( η ) ∈ Σ ∩ U q_{u}(\eta) \in\Sigma\cap U q u ( η ) ∈ Σ ∩ U ;
(ii) the continuation of the stable orbit segment γ s ( η ∗ ) \gamma^{s}(\eta^{*}) γ s ( η ∗ ) reaching q ∗ q^{*} q ∗ in backward time intersects Σ \Sigma Σ transversely at a unique point q s ( η ) ∈ Σ ∩ U q_{s}(\eta) \in\Sigma\cap U q s ( η ) ∈ Σ ∩ U ;
(iii) the maps η ↦ q u ( η ) \eta\mapsto q_{u}(\eta) η ↦ q u ( η ) and η ↦ q s ( η ) \eta\mapsto q_{s}(\eta) η ↦ q s ( η ) are continuous in J J J . If the vector field depends C r C^{r} C r -smoothly on η \eta η , then these maps are C r C^{r} C r -smooth;
(iv) at the reference parameter, q u ( η ∗ ) = q s ( η ∗ ) = q ∗ q_{u}(\eta^{*}) = q_{s}(\eta^{*}) = q^{*} q u ( η ∗ ) = q s ( η ∗ ) = q ∗ .
Proof. Let us write system (1) as
x ˙ = F ( x , η ) , x = ( B , H ) ∈ R 2 , \dot{x}=F(x,\eta), \qquad x=(B,H)\in\mathbb{R}^{2}, x ˙ = F ( x , η ) , x = ( B , H ) ∈ R 2 ,
and denote by Φ η t \Phi_{\eta}^{t} Φ η t its local flow. Then, assume that F F F is at least C 1 C^{1} C 1 with respect to ( x , η ) (x,\eta) ( x , η ) and fix q ∗ ∈ Γ h o m ∖ { C E + ( η ∗ ) } q^{*}\in\Gamma_{\mathrm{hom}}\setminus\{CE_{+}(\eta^{*})\} q ∗ ∈ Γ hom ∖ { C E + ( η ∗ )} so that, since the point q ∗ q^{*} q ∗ is not an equilibrium, F ( q ∗ , η ∗ ) ≠ 0 F(q^{*},\eta^{*})\ne0 F ( q ∗ , η ∗ ) = 0 . Then, let us choose a vector n ∗ ∈ R 2 n^{*}\in\mathbb{R}^{2} n ∗ ∈ R 2 satisfying ⟨ n ∗ , F ( q ∗ , η ∗ ) ⟩ ≠ 0 \langle n^{*},F(q^{*},\eta^{*})\rangle\ne0 ⟨ n ∗ , F ( q ∗ , η ∗ )⟩ = 0 , and define
Σ : = { x ∈ U : ⟨ n ∗ , x − q ∗ ⟩ = 0 } , \Sigma:= \{x \in U : \langle n^*,x-q^*\rangle= 0\}, Σ := { x ∈ U : ⟨ n ∗ , x − q ∗ ⟩ = 0 } ,
where U U U is a sufficiently small neighbourhood of q ∗ q^* q ∗ . Thus, Σ \Sigma Σ is a local section through q ∗ q^* q ∗ transverse to the vector field at η = η ∗ \eta=\eta^* η = η ∗ (see Figure 5 ). By continuity of F F F , after shrinking U U U and restricting the parameter to a sufficiently small open interval J 0 ∋ η ∗ J_0 \ni\eta^* J 0 ∋ η ∗ , we may assume that
Figure 5. Local transverse section Σ \Sigma Σ used to define the splitting function Δ ( η ) \Delta(\eta) Δ ( η ) in Lemma 4.1.
⟨ n ∗ , F ( x , η ) ⟩ ≠ 0 ∀ ( x , η ) ∈ U × J . \langle n^*,F(x,\eta)\rangle\ne0 \quad\forall(x,\eta) \in U \times J. ⟨ n ∗ , F ( x , η )⟩ = 0 ∀ ( x , η ) ∈ U × J .
Consequently, Σ \Sigma Σ is transverse to the vector field for every η ∈ J 0 \eta\in J_0 η ∈ J 0 . Moreover, U U U can be chosen so that the reference homoclinic orbit intersects U U U in a single orbit arc containing q ∗ q^* q ∗ . In particular, this arc intersects Σ \Sigma Σ only at q ∗ q^* q ∗ .
We first consider the unstable orbit segment. Since Γ h o m \Gamma_{\mathrm{hom}} Γ hom leaves C E + ( η ∗ ) CE_+(\eta^*) C E + ( η ∗ ) along γ N W u ( η ∗ ) \gamma^u_{NW}(\eta^*) γ N W u ( η ∗ ) , we can choose a point
a u ∗ ∈ W l o c u ( C E + ( η ∗ ) ) ∩ γ N W u ( η ∗ ) a_u^* \in W^u_{\mathrm{loc}}(CE_+(\eta^*)) \cap\gamma^u_{NW}(\eta^*) a u ∗ ∈ W loc u ( C E + ( η ∗ )) ∩ γ N W u ( η ∗ )
and a finite time T u ∗ > 0 T_u^* > 0 T u ∗ > 0 such that
Φ η ∗ T u ∗ ( a u ∗ ) = q ∗ . \Phi_{\eta^*}^{T_u^*}(a_u^*) = q^*. Φ η ∗ T u ∗ ( a u ∗ ) = q ∗ .
Since C E + ( η ∗ ) CE_+(\eta^*) C E + ( η ∗ ) is hyperbolic, hyperbolicity persists for η \eta η sufficiently close to η ∗ \eta^* η ∗ . The parameter-dependent stable manifold theorem [25 ] then provides a continuous continuation
a u ( η ) ∈ W l o c u ( C E + ( η ) ) ∩ γ N W u ( η ) , η ∈ J 0 , a_u(\eta) \in W^u_{\mathrm{loc}}(CE_+(\eta)) \cap\gamma^u_{NW}(\eta), \qquad\eta\in J_0, a u ( η ) ∈ W loc u ( C E + ( η )) ∩ γ N W u ( η ) , η ∈ J 0 ,
such that a u ( η ∗ ) = a u ∗ a_u(\eta^*)=a_u^* a u ( η ∗ ) = a u ∗ . Define the unstable hitting function
G u ( t , η ) : = ⟨ n ∗ , Φ η t ( a u ( η ) ) − q ∗ ⟩ , G_u(t,\eta) := \langle n^*,\Phi_\eta^t(a_u(\eta))-q^*\rangle, G u ( t , η ) := ⟨ n ∗ , Φ η t ( a u ( η )) − q ∗ ⟩ ,
which, at the reference parameter, is characterized by G u ( T u ∗ , η ∗ ) = 0 G_u(T_u^*,\eta^*)=0 G u ( T u ∗ , η ∗ ) = 0 . Furthermore,
∂ G u ∂ t ( T u ∗ , η ∗ ) = ⟨ n ∗ , F ( Φ η ∗ T u ∗ ( a u ∗ ) , η ∗ ) ⟩ = ⟨ n ∗ , F ( q ∗ , η ∗ ) ⟩ ≠ 0. \frac{\partial G_u}{\partial t}(T_u^*,\eta^*) = \langle n^*,F(\Phi_{\eta^*}^{T_u^*}(a_u^*),\eta^*)\rangle= \langle n^*,F(q^*,\eta^*)\rangle\ne0. ∂ t ∂ G u ( T u ∗ , η ∗ ) = ⟨ n ∗ , F ( Φ η ∗ T u ∗ ( a u ∗ ) , η ∗ )⟩ = ⟨ n ∗ , F ( q ∗ , η ∗ )⟩ = 0.
Therefore, by the implicit function theorem, after possibly restricting J J J , there exists a unique continuous function
T u : J 0 → R , T u ( η ∗ ) = T u ∗ , T_u:J_0 \to\mathbb{R}, \qquad T_u(\eta^*)=T_u^*, T u : J 0 → R , T u ( η ∗ ) = T u ∗ ,
such that G u ( T u ( η ) , η ) = 0 G_u(T_u(\eta),\eta)=0 G u ( T u ( η ) , η ) = 0 ∀ η ∈ J \forall\eta\in J ∀ η ∈ J . Let us set
q u ( η ) : = Φ η T u ( η ) ( a u ( η ) ) . q_u(\eta) := \Phi_\eta^{T_u(\eta)}(a_u(\eta)). q u ( η ) := Φ η T u ( η ) ( a u ( η )) .
It follows that q u ( η ) ∈ Σ ∩ U q_u(\eta) \in\Sigma\cap U q u ( η ) ∈ Σ ∩ U and q u ( η ∗ ) = q ∗ q_u(\eta^*)=q^* q u ( η ∗ ) = q ∗ . The local uniqueness in the implicit function theorem, together with the choice of U U U , shows that q u ( η ) q_u(\eta) q u ( η ) is the unique transverse intersection of the continued unstable orbit segment with Σ ∩ U \Sigma\cap U Σ ∩ U .
We now consider the stable orbit segment. Since Γ h o m \Gamma_{\mathrm{hom}} Γ hom returns to C E + ( η ∗ ) CE_+(\eta^*) C E + ( η ∗ ) along γ s ( η ∗ ) \gamma^s(\eta^*) γ s ( η ∗ ) , we can choose
a s ∗ ∈ W l o c s ( C E + ( η ∗ ) ) ∩ γ s ( η ∗ ) a_s^* \in W^s_{\mathrm{loc}}(CE_+(\eta^*)) \cap\gamma^s(\eta^*) a s ∗ ∈ W loc s ( C E + ( η ∗ )) ∩ γ s ( η ∗ )
and a finite time T s ∗ > 0 T_s^* > 0 T s ∗ > 0 such that
Φ η ∗ − T s ∗ ( a s ∗ ) = q ∗ . \Phi_{\eta^*}^{-T_s^*}(a_s^*) = q^*. Φ η ∗ − T s ∗ ( a s ∗ ) = q ∗ .
The parameter-dependent stable manifold theorem provides a continuous continuation
a s ( η ) ∈ W l o c s ( C E + ( η ) ) ∩ γ s ( η ) , η ∈ J 0 , a_s(\eta) \in W_{\mathrm{loc}}^s(CE_+(\eta)) \cap\gamma^s(\eta), \qquad\eta\in J_0, a s ( η ) ∈ W loc s ( C E + ( η )) ∩ γ s ( η ) , η ∈ J 0 ,
satisfying a s ( η ∗ ) = a s ∗ a_s(\eta^*) = a_s^* a s ( η ∗ ) = a s ∗ . Define the stable hitting function
G s ( t , η ) : = ⟨ n ∗ , Φ η − t ( a s ( η ) ) − q ∗ ⟩ G_s(t,\eta) := \langle n^*, \Phi_\eta^{-t}(a_s(\eta)) - q^* \rangle G s ( t , η ) := ⟨ n ∗ , Φ η − t ( a s ( η )) − q ∗ ⟩
so that G s ( T s ∗ , η ∗ ) = 0 G_s(T_s^*,\eta^*) = 0 G s ( T s ∗ , η ∗ ) = 0 , and
∂ G s ∂ t ( T s ∗ , η ∗ ) = − ⟨ n ∗ , F ( q ∗ , η ∗ ) ⟩ ≠ 0. \frac{\partial G_s}{\partial t}(T_s^*,\eta^*) = -\langle n^*,F(q^*,\eta^*)\rangle\ne0. ∂ t ∂ G s ( T s ∗ , η ∗ ) = − ⟨ n ∗ , F ( q ∗ , η ∗ )⟩ = 0.
A second application of the implicit function theorem yields a unique continuous function
T s : J 0 ⟶ R , T s ( η ∗ ) = T s ∗ , T_s:J_0 \longrightarrow\mathbb{R}, \qquad T_s(\eta^*)=T_s^*, T s : J 0 ⟶ R , T s ( η ∗ ) = T s ∗ ,
such that G s ( T s ( η ) , η ) = 0 G_s(T_s(\eta),\eta)=0 G s ( T s ( η ) , η ) = 0 . Hence, defining
q s ( η ) : = Φ η − T s ( η ) ( a s ( η ) ) , q_s(\eta):=\Phi_\eta^{-T_s(\eta)}(a_s(\eta)), q s ( η ) := Φ η − T s ( η ) ( a s ( η )) ,
we obtain q s ( η ) ∈ Σ ∩ U q_s(\eta)\in\Sigma\cap U q s ( η ) ∈ Σ ∩ U and q s ( η ∗ ) = q ∗ q_s(\eta^*)=q^* q s ( η ∗ ) = q ∗ . As in the unstable case, the transversality and the choice of the neighbourhood imply that q s ( η ) q_s(\eta) q s ( η ) is the unique intersection of the continued stable orbit segment with Σ ∩ U \Sigma\cap U Σ ∩ U . Note that a qualitative sketch of the above dynamics is provided in Figure 5 for the case where the northwestern branch of W u ( C E + ) W^u(CE_+) W u ( C E + ) converges to C E − CE_- C E − through the stable branch γ S W s \gamma^s_{SW} γ S W s . Analogous phase-plane configuration arises when the convergence occurs via the stable branch γ N E s \gamma^s_{NE} γ N E s .
Therefore, we have proved that, for every η ∈ J 0 \eta\in J_0 η ∈ J 0 , the selected unstable and stable orbit segments intersect Σ ∩ U \Sigma\cap U Σ ∩ U transversely at the unique points q u ( η ) q_u(\eta) q u ( η ) and q s ( η ) q_s(\eta) q s ( η ) , respectively. Moreover,
q u ( η ∗ ) = q s ( η ∗ ) = q ∗ . q_u(\eta^*)=q_s(\eta^*)=q^*. q u ( η ∗ ) = q s ( η ∗ ) = q ∗ .
Then, the continuity of the maps
η ⟼ q u ( η ) , η ⟼ q s ( η ) \eta\longmapsto q_u(\eta), \qquad\eta\longmapsto q_s(\eta) η ⟼ q u ( η ) , η ⟼ q s ( η )
follows from the continuous dependence of the local invariant manifolds, the flow, and the corresponding hitting times on the parameter. Finally, if F F F and the parameter-dependent local invariant manifolds are C r C^r C r in η \eta η , the implicit function theorem in its C r C^r C r version implies that T u T_u T u , T s T_s T s , q u q_u q u , and q s q_s q s are C r C^r C r as well.
Theorem 4.3 (Homoclinic bifurcation criterion)
Assume that, for η = η ∗ \eta=\eta^{*} η = η ∗ , system (1) admits a homoclinic orbit Γ h o m \Gamma_{\mathrm{hom}} Γ hom based at the hyperbolic saddle C E + ( η ∗ ) CE_{+}(\eta^{*}) C E + ( η ∗ ) . Suppose that Γ h o m \Gamma_{\mathrm{hom}} Γ hom leaves C E + ( η ∗ ) CE_{+}(\eta^{*}) C E + ( η ∗ ) along the northwestern unstable branch γ N W u ( η ∗ ) \gamma^{u}_{\mathrm{NW}}(\eta^{*}) γ NW u ( η ∗ ) and returns to it along the selected stable branch γ s ( η ∗ ) \gamma^{s}(\eta^{*}) γ s ( η ∗ ) . Let Σ \Sigma Σ be the transverse section and let q u ( η ) , q s ( η ) ∈ Σ q_{u}(\eta),q_{s}(\eta)\in\Sigma q u ( η ) , q s ( η ) ∈ Σ be the corresponding intersection points provided by Lemma 4.1. Let us choose a local coordinate s : Σ → R s:\Sigma\to\mathbb{R} s : Σ → R and define the distance function
Δ ( η ) : = s ( q u ( η ) ) − s ( q s ( η ) ) . \Delta(\eta):=s(q_{u}(\eta))-s(q_{s}(\eta)). Δ ( η ) := s ( q u ( η )) − s ( q s ( η )) . Then Δ \Delta Δ is continuous in a neighbourhood of η ∗ \eta^{*} η ∗ and
Δ ( η ) = 0 ⟺ system (1) admits a homoclinic orbit of the selected type \Delta(\eta)=0\ \Longleftrightarrow\ \text{system (1) admits a homoclinic orbit of the selected type} Δ ( η ) = 0 ⟺ system (1) admits a homoclinic orbit of the selected type More precisely:
(i) If Δ ( η ) ≠ 0 \Delta(\eta)\ne0 Δ ( η ) = 0 , then no homoclinic orbit of the selected type exists;
(ii) If there exist η 1 , η 2 \eta_{1},\eta_{2} η 1 , η 2 in the parameter interval under consideration such that Δ ( η 1 ) Δ ( η 2 ) < 0 \Delta(\eta_{1})\Delta(\eta_{2})<0 Δ ( η 1 ) Δ ( η 2 ) < 0 , then there exists η ˉ ∈ ( η 1 , η 2 ) \bar{\eta}\in(\eta_{1},\eta_{2}) η ˉ ∈ ( η 1 , η 2 ) such that Δ ( η ˉ ) = 0 \Delta(\bar{\eta})=0 Δ ( η ˉ ) = 0 and system (1) admits a homoclinic orbit;
(iii) If Δ \Delta Δ is differentiable at η ∗ \eta^{*} η ∗ , Δ ( η ∗ ) = 0 \Delta(\eta^{*})=0 Δ ( η ∗ ) = 0 and Δ ′ ( η ∗ ) ≠ 0 \Delta'(\eta^{*})\ne0 Δ ′ ( η ∗ ) = 0 , then the corresponding homoclinic connection is nondegenerate and defines a codimension-one homoclinic bifurcation.
Proof. By Lemma 4.1, there exists an open interval J 0 ∋ η ∗ J_{0}\ni\eta^{*} J 0 ∋ η ∗ , a local transverse section Σ \Sigma Σ , and two uniquely defined transverse intersection points, namely q u ( η ) q_{u}(\eta) q u ( η ) and q s ( η ) q_{s}(\eta) q s ( η ) , associated with the continuations of the selected unstable and stable orbit segments, respectively. Moreover, q u ( η ∗ ) = q s ( η ∗ ) = q ∗ q_{u}(\eta^{*})=q_{s}(\eta^{*})=q^{*} q u ( η ∗ ) = q s ( η ∗ ) = q ∗ , and the maps η ↦ q u ( η ) \eta\mapsto q_{u}(\eta) η ↦ q u ( η ) and η ↦ q s ( η ) \eta\mapsto q_{s}(\eta) η ↦ q s ( η ) are continuous on J 0 J_{0} J 0 . Since the local coordinate s : Σ → R s:\Sigma\to\mathbb{R} s : Σ → R is continuous, the splitting function Δ ( η ) = s ( q u ( η ) ) − s ( q s ( η ) ) \Delta(\eta)=s(q_{u}(\eta))-s(q_{s}(\eta)) Δ ( η ) = s ( q u ( η )) − s ( q s ( η )) is continuous on J 0 J_{0} J 0 .
Suppose first that Δ ( η ) = 0 \Delta(\eta)=0 Δ ( η ) = 0 , then s ( q u ( η ) ) = s ( q s ( η ) ) s(q_{u}(\eta))=s(q_{s}(\eta)) s ( q u ( η )) = s ( q s ( η )) . Since s s s is a local coordinate on Σ \Sigma Σ , it is locally injective, and therefore q u ( η ) = q s ( η ) = : p q_{u}(\eta)=q_{s}(\eta)=:p q u ( η ) = q s ( η ) =: p . The point p p p belongs simultaneously to the selected unstable and stable orbit segments so that p ∈ W u ( C E + ( η ) ) ∩ W s ( C E + ( η ) ) p\in W^{u}(CE_{+}(\eta))\cap W^{s}(CE_{+}(\eta)) p ∈ W u ( C E + ( η )) ∩ W s ( C E + ( η )) . By uniqueness of solutions, the two orbit segments through p p p coincide. Consequently, the resulting orbit satisfies
lim t → − ∞ γ ( t ) = C E + ( η ) , lim t → + ∞ γ ( t ) = C E + ( η ) , \lim_{t\to-\infty}\gamma(t)=CE_{+}(\eta),\qquad\lim_{t\to+\infty}\gamma(t)=CE_{+}(\eta), t → − ∞ lim γ ( t ) = C E + ( η ) , t → + ∞ lim γ ( t ) = C E + ( η ) ,
and therefore defines a homoclinic orbit based at C E + ( η ) CE_{+}(\eta) C E + ( η ) .
Conversely, assume that, for the parameter value η = η ∗ \eta= \eta^{*} η = η ∗ , system (1) admits a homoclinic orbit of the selected type; namely, Γ h o m \Gamma_{\mathrm{hom}} Γ hom leaves C E + ( η ∗ ) CE_{+}(\eta^{*}) C E + ( η ∗ ) along the northwestern unstable branch γ N W u ( η ∗ ) \gamma^{u}_{NW}(\eta^{*}) γ N W u ( η ∗ ) and returns to it along the selected stable branch γ s ( η ∗ ) \gamma^{s}(\eta^{*}) γ s ( η ∗ ) . Since Γ h o m \Gamma_{\mathrm{hom}} Γ hom is simultaneously contained in W u ( C E + ( η ∗ ) ) W^{u}(CE_{+}(\eta^{*})) W u ( C E + ( η ∗ )) and W s ( C E + ( η ∗ ) ) W^{s}(CE_{+}(\eta^{*})) W s ( C E + ( η ∗ )) , there exists a point q ∗ ∈ Γ h o m ∖ { C E + ( η ∗ ) } q^{*} \in\Gamma_{\mathrm{hom}} \setminus\{CE_{+}(\eta^{*})\} q ∗ ∈ Γ hom ∖ { C E + ( η ∗ )} that belongs to both the selected unstable and stable orbit segments, namely
q ∗ ∈ ( Γ h o m ( η ∗ ) ∖ { C E + ( η ∗ ) } ) ∩ ( W u ( C E + ( η ∗ ) ) ∩ W s ( C E + ( η ∗ ) ) ) . q^{*} \in\left(\Gamma_{\mathrm{hom}}(\eta^{*}) \setminus\{CE_{+}(\eta^{*})\}\right) \cap\left(W^{u}(CE_{+}(\eta^{*})) \cap W^{s}(CE_{+}(\eta^{*}))\right). q ∗ ∈ ( Γ hom ( η ∗ ) ∖ { C E + ( η ∗ )} ) ∩ ( W u ( C E + ( η ∗ )) ∩ W s ( C E + ( η ∗ )) ) .
Applying Lemma 4.1 to the pair ( q ∗ , η ∗ ) (q^{*},\eta^{*}) ( q ∗ , η ∗ ) , we obtain a local transverse section Σ \Sigma Σ together with the corresponding intersection points q u ( η ∗ ) q_{u}(\eta^{*}) q u ( η ∗ ) and q s ( η ∗ ) q_{s}(\eta^{*}) q s ( η ∗ ) . Since both orbit segments belong to the same homoclinic trajectory, they intersect Σ \Sigma Σ at the same point. Therefore q u ( η ∗ ) = q s ( η ∗ ) q_{u}(\eta^{*}) = q_{s}(\eta^{*}) q u ( η ∗ ) = q s ( η ∗ ) so that
Δ ( η ∗ ) = s ( q u ( η ∗ ) ) − s ( q s ( η ∗ ) ) = 0. \Delta(\eta^{*}) = s(q_{u}(\eta^{*})) - s(q_{s}(\eta^{*})) = 0. Δ ( η ∗ ) = s ( q u ( η ∗ )) − s ( q s ( η ∗ )) = 0.
This proves that the existence of a homoclinic orbit of the selected type implies the vanishing of the splitting function.
Having established the equivalence
Δ ( η ) = 0 ⟺ system (1) admits a homoclinic orbit of the selected type, \Delta(\eta) = 0 \quad\Longleftrightarrow\quad\text{system (1) admits a homoclinic orbit of the selected type,} Δ ( η ) = 0 ⟺ system (1) admits a homoclinic orbit of the selected type,
the first statement follows immediately. Indeed, if Δ ( η ) ≠ 0 \Delta(\eta) \ne0 Δ ( η ) = 0 , then q u ( η ) ≠ q s ( η ) q_{u}(\eta) \ne q_{s}(\eta) q u ( η ) = q s ( η ) , and therefore the selected unstable and stable orbit segments do not coincide so that no homoclinic orbit can exist.
Now let η 1 , η 2 ∈ J 0 \eta_{1},\eta_{2} \in J_{0} η 1 , η 2 ∈ J 0 satisfy Δ ( η 1 ) Δ ( η 2 ) < 0 \Delta(\eta_{1})\Delta(\eta_{2}) < 0 Δ ( η 1 ) Δ ( η 2 ) < 0 . Since Δ \Delta Δ is continuous on J 0 J_{0} J 0 , the Intermediate Value Theorem yields a point η ˉ ∈ ( η 1 , η 2 ) \bar{\eta} \in(\eta_{1},\eta_{2}) η ˉ ∈ ( η 1 , η 2 ) such that Δ ( η ˉ ) = 0 \Delta(\bar{\eta}) = 0 Δ ( η ˉ ) = 0 . Applying again the established equivalence, it follows that system (1) admits a homoclinic orbit based at C E + ( η ˉ ) CE_{+}(\bar{\eta}) C E + ( η ˉ ) .
Finally, assume that Δ ( η ∗ ) = 0 \Delta(\eta^{*}) = 0 Δ ( η ∗ ) = 0 and Δ ′ ( η ∗ ) ≠ 0 \Delta'(\eta^{*}) \ne0 Δ ′ ( η ∗ ) = 0 . Then η ∗ \eta^{*} η ∗ is a simple zero of the splitting function so that the intersection points q u ( η ) q_{u}(\eta) q u ( η ) and q s ( η ) q_{s}(\eta) q s ( η ) move with nonzero relative speed along the transverse section as the parameter varies. Consequently, the selected branches of W u ( C E + ( η ) ) W^{u}(CE_{+}(\eta)) W u ( C E + ( η )) and W s ( C E + ( η ) ) W^{s}(CE_{+}(\eta)) W s ( C E + ( η )) coincide only at η = η ∗ \eta= \eta^{*} η = η ∗ and split for nearby parameter values. Therefore the corresponding homoclinic connection is nondegenerate and defines a codimension-one homoclinic bifurcation. ▫
The previous criterion characterizes the occurrence of homoclinic connections through the splitting function Δ \Delta Δ . We now establish some geometric properties that every simple homoclinic loop must satisfy.
Proposition 4.2 (Geometry of simple homoclinic loops)
Assume that system (1) admits a simple homoclinic loop Γ h o m \Gamma_{\mathrm{hom}} Γ hom based at the saddle equilibrium C E + CE_{+} C E + . Then the following statements hold:
(i) Γ h o m \Gamma_{\mathrm{hom}} Γ hom encloses C E − CE_{-} C E − , or equivalently C E − ∈ D h o m CE_{-} \in D_{\mathrm{hom}} C E − ∈ D hom , being D h o m D_{\mathrm{hom}} D hom the bounded connected component of R 2 ∖ Γ h o m \mathbb{R}^{2} \setminus\Gamma_{\mathrm{hom}} R 2 ∖ Γ hom ;
(ii) Let B c ∈ ( 0 , α ) B_{c} \in(0,\alpha) B c ∈ ( 0 , α ) be the unique critical point of the positive B B B -nullcline. Then Γ h o m \Gamma_{\mathrm{hom}} Γ hom intersects both regions U − = { ( B , H ) ∈ int Ω : B < B c } \mathcal{U}_{-} = \{(B,H) \in\operatorname{int}\Omega: B < B_{c}\} U − = {( B , H ) ∈ int Ω : B < B c } and U + = { ( B , H ) ∈ int Ω : B > B c } \mathcal{U}_{+} = \{(B,H) \in\operatorname{int}\Omega: B > B_{c}\} U + = {( B , H ) ∈ int Ω : B > B c } .
Proof. Let Γ h o m \Gamma_{\mathrm{hom}} Γ hom be a simple homoclinic loop based at the hyperbolic saddle C E + CE_{+} C E + , and let D h o m D_{\mathrm{hom}} D hom denote the bounded region enclosed by Γ h o m \Gamma_{\mathrm{hom}} Γ hom . We first consider the equilibria enclosed by the loop. A standard index result for planar vector fields states that the Poincaré index of a simple homoclinic loop based at a hyperbolic saddle, computed by a sufficiently small regularization of the loop near the saddle, is equal to one [25 ] . Consequently,
∑ P ∈ D h o m ind ( P ) = 1 , \sum_{P\in D_{\mathrm{hom}}} \operatorname{ind}(P)=1, P ∈ D hom ∑ ind ( P ) = 1 ,
where the sum is taken over the equilibria P P P lying strictly inside D h o m D_{\mathrm{hom}} D hom . Since Γ h o m ⊂ int Ω \Gamma_{\mathrm{hom}} \subset\operatorname{int}\Omega Γ hom ⊂ int Ω , the bounded region D h o m D_{\mathrm{hom}} D hom is contained in int Ω \operatorname{int}\Omega int Ω so that neither T E TE T E nor H F E HFE H F E belongs to D h o m D_{\mathrm{hom}} D hom . Moreover, C E + CE_{+} C E + lies on the loop and is therefore not counted among the equilibria contained strictly inside D h o m D_{\mathrm{hom}} D hom . Since C E − CE_{-} C E − is a hyperbolic non-saddle equilibrium, i.e. ind ( C E − ) = 1 \operatorname{ind}(CE_{-})=1 ind ( C E − ) = 1 , and it is the only remaining equilibrium that may lie inside D h o m D_{\mathrm{hom}} D hom , the index identity then implies C E − ∈ D h o m CE_{-}\in D_{\mathrm{hom}} C E − ∈ D hom so that the simple homoclinic loop Γ h o m \Gamma_{\mathrm{hom}} Γ hom encloses C E − CE_{-} C E − .
Let us now prove the second statement. Recall that the positive B B B -nullcline can be written as
H = N ( B ) , N ( B ) : = ( 1 − B / α ) ( 1 + B ) 2 m [ 1 + ( 1 − μ ) B ] . H=N(B),\qquad N(B):=\frac{(1-B/\alpha)(1+B)^2}{m[1+(1-\mu)B]}. H = N ( B ) , N ( B ) := m [ 1 + ( 1 − μ ) B ] ( 1 − B / α ) ( 1 + B ) 2 .
Consider the Dulac function [25 ]
D ( B , H ) : = 1 C ( B ) H , ( B , H ) ∈ int Ω . D(B,H):=\frac{1}{C(B)H},\qquad(B,H)\in\operatorname{int}\Omega. D ( B , H ) := C ( B ) H 1 , ( B , H ) ∈ int Ω.
Since C ( B ) > 0 C(B)>0 C ( B ) > 0 for B > 0 B>0 B > 0 , the function D D D is continuously differentiable on int Ω \operatorname{int}\Omega int Ω . Consequently,
∇ ⋅ ( D ( B , H ) F ( B , H ) ) = ∂ ∂ B ( m N ( B ) H − m ) + ∂ ∂ H ( m − θ C ( B ) ) = m N ′ ( B ) H , \nabla\cdot\bigl(D(B,H)F(B,H)\bigr)
=\frac{\partial}{\partial B}\left(\frac{mN(B)}{H}-m\right)
+\frac{\partial}{\partial H}\left(m-\frac{\theta}{C(B)}\right)
=\frac{mN'(B)}{H}, ∇ ⋅ ( D ( B , H ) F ( B , H ) ) = ∂ B ∂ ( H m N ( B ) − m ) + ∂ H ∂ ( m − C ( B ) θ ) = H m N ′ ( B ) ,
being F ( B , H ) F(B,H) F ( B , H ) the vector field (7) . A direct differentiation gives
N ′ ( B ) = 1 + B m α [ 1 + ( 1 − μ ) B ] 2 Q ( B ) , (8) N'(B)=\frac{1+B}{m\alpha[1+(1-\mu)B]^2}Q(B), \tag*{(8)} N ′ ( B ) = m α [ 1 + ( 1 − μ ) B ] 2 1 + B Q ( B ) , (8)
where
Q ( B ) = α ( 1 + μ ) + [ α ( 1 − μ ) − 2 ( 1 + μ ) ] B − 3 ( 1 − μ ) B 2 . Q(B)=\alpha(1+\mu)+[\alpha(1-\mu)-2(1+\mu)]B-3(1-\mu)B^2. Q ( B ) = α ( 1 + μ ) + [ α ( 1 − μ ) − 2 ( 1 + μ )] B − 3 ( 1 − μ ) B 2 .
Since the prefactor in (8) is strictly positive, sgn N ′ ( B ) = sgn Q ( B ) \operatorname{sgn}N'(B)=\operatorname{sgn}Q(B) sgn N ′ ( B ) = sgn Q ( B ) . For μ < 1 \mu<1 μ < 1 , the constant term of Q Q Q is positive and the product of its roots is negative so that it possesses exactly one positive root. Since Q ( 0 ) = α ( 1 + μ ) > 0 Q(0)=\alpha(1+\mu)>0 Q ( 0 ) = α ( 1 + μ ) > 0 and Q ( α ) = − α ( 1 + α ) [ 1 + α ( 1 − μ ) ] < 0 Q(\alpha)=-\alpha(1+\alpha)[1+\alpha(1-\mu)]<0 Q ( α ) = − α ( 1 + α ) [ 1 + α ( 1 − μ )] < 0 , the unique positive root
B c = α ( 1 − μ ) − 2 ( 1 + μ ) + [ α ( 1 − μ ) − 2 ( 1 + μ ) ] 2 + 12 α ( 1 − μ ) ( 1 + μ ) 6 ( 1 − μ ) B_c=\frac{\alpha(1-\mu)-2(1+\mu)+\sqrt{[\alpha(1-\mu)-2(1+\mu)]^2+12\alpha(1-\mu)(1+\mu)}}{6(1-\mu)} B c = 6 ( 1 − μ ) α ( 1 − μ ) − 2 ( 1 + μ ) + [ α ( 1 − μ ) − 2 ( 1 + μ ) ] 2 + 12 α ( 1 − μ ) ( 1 + μ )
belongs to ( 0 , α ) (0,\alpha) ( 0 , α ) . Moreover, Q Q Q is a strictly concave quadratic polynomial, i.e.
N ′ ( B ) > 0 for 0 < B < B c , N'(B)>0\qquad\text{for}\qquad0<B<B_c, N ′ ( B ) > 0 for 0 < B < B c ,
N ′ ( B ) < 0 for B c < B < α . N'(B)<0\qquad\text{for}\qquad B_c<B<\alpha. N ′ ( B ) < 0 for B c < B < α .
Suppose, by contradiction, that Γ h o m \Gamma_{\mathrm{hom}} Γ hom were entirely contained in the region U − \mathcal{U}_{-} U − , region in which ∇ ⋅ ( D F ) > 0 \nabla\cdot(DF)>0 ∇ ⋅ ( D F ) > 0 . Since U − \mathcal{U}_{-} U − is simply connected, the Bendixson-Dulac criterion excludes a closed invariant curve formed by regular orbit segments, including a homoclinic loop, entirely contained in U − \mathcal{U}_{-} U − [25 ] . Similarly, if Γ h o m \Gamma_{\mathrm{hom}} Γ hom were entirely contained in U + \mathcal{U}_{+} U + , then ∇ ⋅ ( D F ) < 0 \nabla\cdot(DF)<0 ∇ ⋅ ( D F ) < 0 throughout U + \mathcal{U}_{+} U + . A second application of the Bendixson–Dulac criterion gives a contradiction. Therefore, Γ h o m \Gamma_{\mathrm{hom}} Γ hom cannot be contained entirely on either side of the line B = B c B=B_c B = B c so that it intersects both regions U ± \mathcal{U}_{\pm} U ± .
The geometric restrictions established above provide only a partial description of the homoclinic configuration. To understand the dynamical role played by a simple homoclinic loop, it is necessary to investigate the invariant regions generated by the loop and their relation with the other invariant sets of the system. The next results show how these geometric configurations determine the basin structure and the long-term dynamics in Regions II and III.
Lemma 4.2 (Relative position of B + B_{+} B + and B c B_c B c )
Assume that the coexistence equilibrium C E + CE_{+} C E + exists and let B c ∈ ( 0 , α ) B_c \in(0,\alpha) B c ∈ ( 0 , α ) denote the unique critical point of the positive B B B -nullcline. Then the following statements hold.
(i) If the parameters belong to Region II, then B + > B c B_{+}>B_c B + > B c .
(ii) If the parameters belong to Region III, then
B + > B c ⟺ α < α c : = B + [ 2 ( 1 + μ ) + 3 ( 1 − μ ) B + ] 1 + μ + ( 1 − μ ) B + . B_{+}>B_c \quad\Longleftrightarrow\quad\alpha<\alpha_c:=\frac{B_{+}\left[2(1+\mu)+3(1-\mu)B_{+}\right]}{1+\mu+(1-\mu)B_{+}}. B + > B c ⟺ α < α c := 1 + μ + ( 1 − μ ) B + B + [ 2 ( 1 + μ ) + 3 ( 1 − μ ) B + ] .
Proof. Let us recall from Proposition 4.2 that sgn N ′ ( B ) = sgn Q ( B ) \operatorname{sgn} N'(B)=\operatorname{sgn} Q(B) sgn N ′ ( B ) = sgn Q ( B ) . Moreover, Q Q Q admits a unique positive root B c B_c B c so that Q ( B ) > 0 Q(B)>0 Q ( B ) > 0 for B < B c B<B_c B < B c and Q ( B ) < 0 Q(B)<0 Q ( B ) < 0 for B > B c B>B_c B > B c .
Let us first consider Region II. By Lemma 3.2, the equilibrium C E − CE_{-} C E − is asymptotically stable. Since tr J ( C E − ) = θ N ′ ( B − ) \operatorname{tr} J(CE_{-})=\theta N'(B_{-}) tr J ( C E − ) = θ N ′ ( B − ) , stability implies N ′ ( B − ) < 0 N'(B_{-})<0 N ′ ( B − ) < 0 . Hence B − > B c B_{-}>B_c B − > B c and, since B − < B + B_{-}<B_{+} B − < B + , it follows that B c < B − < B + B_c<B_{-}<B_{+} B c < B − < B + .
Let now the parameters belong to Region III. Since B c B_c B c is the unique positive root of Q Q Q , one has B + > B c B_{+}>B_c B + > B c iff Q ( B + ) < 0 Q(B_{+})<0 Q ( B + ) < 0 . Evaluating Q Q Q at B + B_{+} B + yields
Q ( B + ) = α [ 1 + μ + ( 1 − μ ) B + ] − B + [ 2 ( 1 + μ ) + 3 ( 1 − μ ) B + ] , Q(B_{+})=\alpha\left[1+\mu+(1-\mu)B_{+}\right]-B_{+}\left[2(1+\mu)+3(1-\mu)B_{+}\right], Q ( B + ) = α [ 1 + μ + ( 1 − μ ) B + ] − B + [ 2 ( 1 + μ ) + 3 ( 1 − μ ) B + ] ,
so that
Q ( B + ) < 0 ⟺ α < B + [ 2 ( 1 + μ ) + 3 ( 1 − μ ) B + ] 1 + μ + ( 1 − μ ) B + . Q(B_{+})<0\quad\Longleftrightarrow\quad\alpha<\frac{B_{+}\left[2(1+\mu)+3(1-\mu)B_{+}\right]}{1+\mu+(1-\mu)B_{+}}. Q ( B + ) < 0 ⟺ α < 1 + μ + ( 1 − μ ) B + B + [ 2 ( 1 + μ ) + 3 ( 1 − μ ) B + ] .
Combining the previous equivalences, we conclude that
B + > B c ⟺ α < B + [ 2 ( 1 + μ ) + 3 ( 1 − μ ) B + ] 1 + μ + ( 1 − μ ) B + . B_{+}>B_c\quad\Longleftrightarrow\quad\alpha<\frac{B_{+}\left[2(1+\mu)+3(1-\mu)B_{+}\right]}{1+\mu+(1-\mu)B_{+}}. B + > B c ⟺ α < 1 + μ + ( 1 − μ ) B + B + [ 2 ( 1 + μ ) + 3 ( 1 − μ ) B + ] .
Region II
Theorem 4.4 (Nonexistence of homoclinic loops in Region II). Assume that the parameters belong to Region II. Then system (1) does not admit a simple homoclinic loop based at the saddle equilibrium C E + CE_{+} C E + .
Proof. Assume, by contradiction, that system (1) admits a simple homoclinic loop Γ h o m \Gamma_{\mathrm{hom}} Γ hom based at C E + CE_{+} C E + , and let D h o m D_{\mathrm{hom}} D hom denote the bounded connected component of R 2 ∖ Γ h o m \mathbb{R}^{2} \setminus\Gamma_{\mathrm{hom}} R 2 ∖ Γ hom . By Proposition 4.2, the loop encloses the coexistence stable equilibrium C E − CE_{-} C E − , so that C E − ∈ D h o m CE_{-} \in D_{\mathrm{hom}} C E − ∈ D hom . Let λ s < 0 < λ u \lambda_{s}<0<\lambda_{u} λ s < 0 < λ u denote the eigenvalues of the Jacobian matrix at C E + CE_{+} C E + . The saddle quantity [25 ] associated with C E + CE_{+} C E + is
σ + : = λ s + λ u = tr J ( C E + ) = θ N ′ ( B + ) . (9) \sigma_{+} := \lambda_{s}+\lambda_{u}=\operatorname{tr} J(CE_{+})=\theta N'(B_{+}). \tag*{(9)} σ + := λ s + λ u = tr J ( C E + ) = θ N ′ ( B + ) . (9)
By Lemma 4.2, one has B + > B c B_{+}>B_{c} B + > B c throughout Region II. Since B c B_{c} B c is the unique critical point of N N N and N ′ ( B ) < 0 N'(B)<0 N ′ ( B ) < 0 for B > B c B>B_{c} B > B c , from (9) it follows that σ + < 0 \sigma_{+}<0 σ + < 0 . The classical local theory of planar homoclinic loops then implies that Γ h o m \Gamma_{\mathrm{hom}} Γ hom is attracting from the side on which the local Poincaré return map is defined [25 , 26 ] . Since Γ h o m \Gamma_{\mathrm{hom}} Γ hom is a simple loop enclosing D h o m D_{\mathrm{hom}} D hom , this return map is defined on a one-sided transverse section contained in D h o m D_{\mathrm{hom}} D hom . Consequently, there exists a neighbourhood U h o m U_{\mathrm{hom}} U hom of Γ h o m \Gamma_{\mathrm{hom}} Γ hom in D h o m D_{\mathrm{hom}} D hom such that
dist ( Φ t ( x ) , Γ h o m ) ⟶ 0 as t ⟶ + ∞ \operatorname{dist}(\Phi_{t}(x),\Gamma_{\mathrm{hom}})\longrightarrow0 \qquad\text{as } t\longrightarrow+\infty dist ( Φ t ( x ) , Γ hom ) ⟶ 0 as t ⟶ + ∞
for every x ∈ U h o m ∩ D h o m x\in U_{\mathrm{hom}}\cap D_{\mathrm{hom}} x ∈ U hom ∩ D hom . On the other hand, since C E − CE_{-} C E − is locally asymptotically stable, there exists a positively invariant neighbourhood U − U_{-} U − of C E − CE_{-} C E − such that
dist ( Φ t ( x ) , C E − ) ⟶ 0 as t ⟶ + ∞ \operatorname{dist}(\Phi_{t}(x),CE_{-})\longrightarrow0 \qquad\text{as } t\longrightarrow+\infty dist ( Φ t ( x ) , C E − ) ⟶ 0 as t ⟶ + ∞
for every x ∈ U − x\in U_{-} x ∈ U − . Let us choose x − ∈ U − x_{-}\in U_{-} x − ∈ U − and x h o m ∈ U h o m ∩ D h o m x_{\mathrm{hom}}\in U_{\mathrm{hom}}\cap D_{\mathrm{hom}} x hom ∈ U hom ∩ D hom . Since D h o m D_{\mathrm{hom}} D hom is an open connected subset of R 2 \mathbb{R}^{2} R 2 , it is path connected. Hence there exists a continuous path γ : [ 0 , 1 ] ⟶ D h o m \gamma:[0,1]\longrightarrow D_{\mathrm{hom}} γ : [ 0 , 1 ] ⟶ D hom such that γ ( 0 ) = x − \gamma(0)=x_{-} γ ( 0 ) = x − and γ ( 1 ) = x h o m \gamma(1)=x_{\mathrm{hom}} γ ( 1 ) = x hom (see Figure 6 ). Let
Figure 6. Geometric configuration associated with a hypothetical simple homoclinic loop Γ h o m \Gamma_{\mathrm{hom}} Γ hom , supposed in Theorem 4.4 , based at the saddle equilibrium C E + CE_{+} C E + in Region II.
B ( C E − ) : = { x ∈ D h o m : Φ t ( x ) ⟶ C E − as t ⟶ + ∞ } . \mathcal{B}(CE_{-}) := \{x\in D_{\mathrm{hom}}:\Phi_{t}(x)\longrightarrow CE_{-}\text{ as }t\longrightarrow+\infty\}. B ( C E − ) := { x ∈ D hom : Φ t ( x ) ⟶ C E − as t ⟶ + ∞ } .
Since C E − CE_{-} C E − is asymptotically stable, B ( C E − ) \mathcal{B}(CE_{-}) B ( C E − ) is open such that B ( C E − ) ⊆ D h o m \mathcal{B}(CE_{-})\subseteq D_{\mathrm{hom}} B ( C E − ) ⊆ D hom . Its boundary ∂ B ( C E − ) \partial\mathcal{B}(CE_{-}) ∂ B ( C E − ) is nonempty, because γ ( 0 ) ∈ B ( C E − ) \gamma(0)\in\mathcal{B}(CE_{-}) γ ( 0 ) ∈ B ( C E − ) whereas γ ( 1 ) ∉ B ( C E − ) \gamma(1)\notin\mathcal{B}(CE_{-}) γ ( 1 ) ∈ / B ( C E − ) . Let us choose y ∈ γ ( [ 0 , 1 ] ) ∩ ∂ B ( C E − ) y\in\gamma([0,1])\cap\partial\mathcal{B}(CE_{-}) y ∈ γ ([ 0 , 1 ]) ∩ ∂ B ( C E − ) . The basin boundary is positively invariant. Since D h o m D_{\mathrm{hom}} D hom is bounded and positively invariant, the positive semiorbit of y y y is bounded, and therefore ω ( y ) \omega(y) ω ( y ) is nonempty, compact, connected, and invariant. The set ω ( y ) \omega(y) ω ( y ) cannot contain C E − CE_{-} C E − . Indeed, if C E − ∈ ω ( y ) CE_{-}\in\omega(y) C E − ∈ ω ( y ) , then the local asymptotic stability of C E − CE_{-} C E − would imply Φ t ( y ) ⟶ C E − \Phi_{t}(y)\longrightarrow CE_{-} Φ t ( y ) ⟶ C E − , and hence y ∈ B ( C E − ) y\in\mathcal{B}(CE_{-}) y ∈ B ( C E − ) , contradicting y ∈ ∂ B ( C E − ) y\in\partial\mathcal{B}(CE_{-}) y ∈ ∂ B ( C E − ) . Moreover, ω ( y ) \omega(y) ω ( y ) cannot coincide with Γ h o m \Gamma_{\mathrm{hom}} Γ hom . Indeed, the basin of the attracting homoclinic loop is open on the D h o m D_{\mathrm{hom}} D hom side, whereas y y y belongs to the invariant boundary separating that basin from B ( C E − ) \mathcal{B}(CE_{-}) B ( C E − ) . Thus ω ( y ) \omega(y) ω ( y ) contains no equilibrium and, by the Poincaré-Bendixson theorem, ω ( y ) \omega(y) ω ( y ) is a nonconstant periodic orbit contained in D h o m D_{\mathrm{hom}} D hom . This contradicts the nonexistence of nonconstant periodic orbits in Region II established in Proposition 3.2 of [15 ] . Therefore, system (1) cannot admit a simple homoclinic loop based at C E + CE_{+} C E + in Region II. A qualitative sketch of the above structures is provided in Figure 6 .
Region III
Lemma 4.3 (Relative position of Γ H \Gamma_H Γ H and Γ h o m \Gamma_{\mathrm{hom}} Γ hom ) Assume that system (1) admits a simple homoclinic loop Γ h o m \Gamma_{\mathrm{hom}} Γ hom based at C E + CE_{+} C E + in Region III. Let D h o m D_{\mathrm{hom}} D hom and D H D_H D H denote the bounded domains enclosed by Γ h o m \Gamma_{\mathrm{hom}} Γ hom and Γ H \Gamma_H Γ H , respectively. Then exactly one of the following alternatives holds:
(i) If C E + ∉ D H CE_{+} \notin D_H C E + ∈ / D H , then D H ‾ ⊂ D h o m \overline{D_H} \subset D_{\mathrm{hom}} D H ⊂ D hom , that is, the stable periodic orbit Γ H \Gamma_H Γ H is enclosed by the homoclinic loop;
(ii) If C E + ∈ D H CE_{+} \in D_H C E + ∈ D H , then D h o m ‾ ⊂ D H \overline{D_{\mathrm{hom}}} \subset D_H D hom ⊂ D H , that is, the homoclinic loop is enclosed by the stable periodic orbit Γ H \Gamma_H Γ H .
Proof. By Proposition 4.2, C E − ∈ D h o m CE_{-} \in D_{\mathrm{hom}} C E − ∈ D hom . Moreover, since Γ H \Gamma_H Γ H surrounds C E − CE_{-} C E − , one also has C E − ∈ D H CE_{-} \in D_H C E − ∈ D H . Since both Γ H \Gamma_H Γ H and Γ h o m ∖ { C E + } \Gamma_{\mathrm{hom}} \setminus\{CE_{+}\} Γ hom ∖ { C E + } are trajectories of system (1), uniqueness of solutions implies that they cannot intersect, namely Γ H ∩ Γ h o m = ∅ \Gamma_H \cap\Gamma_{\mathrm{hom}} = \varnothing Γ H ∩ Γ hom = ∅ . Consequently, the corresponding bounded domains must be nested. If C E + ∉ D H CE_{+} \notin D_H C E + ∈ / D H , then the homoclinic loop cannot be contained in D H D_H D H because C E + ∈ Γ h o m CE_{+} \in\Gamma_{\mathrm{hom}} C E + ∈ Γ hom so that D H ‾ ⊂ D h o m \overline{D_H} \subset D_{\mathrm{hom}} D H ⊂ D hom . Conversely, if C E + ∈ D H CE_{+} \in D_H C E + ∈ D H , then the homoclinic loop is entirely contained in D H D_H D H , yielding D h o m ‾ ⊂ D H \overline{D_{\mathrm{hom}}} \subset D_H D hom ⊂ D H . The two alternatives are mutually exclusive.
Theorem 4.5 (Dynamical role of simple homoclinic loops in Region III) Assume that the parameters belong to Region III and that system (1) admits a simple homoclinic loop Γ h o m \Gamma_{\mathrm{hom}} Γ hom based at the saddle equilibrium C E + CE_{+} C E + . Let Γ H \Gamma_H Γ H denote the asymptotically stable periodic orbit generated through the supercritical Hopf bifurcation of C E − CE_{-} C E − . Let D h o m D_{\mathrm{hom}} D hom and D H D_H D H denote the bounded domains enclosed by Γ h o m \Gamma_{\mathrm{hom}} Γ hom and Γ H \Gamma_H Γ H , respectively. Then the following statements hold:
(i) Assume that D H ⊂ D h o m D_H \subset D_{\mathrm{hom}} D H ⊂ D hom . If α < α c \alpha< \alpha_c α < α c , then Γ h o m \Gamma_{\mathrm{hom}} Γ hom is asymptotically stable from the D h o m D_{\mathrm{hom}} D hom side. Moreover, Γ H \Gamma_H Γ H and Γ h o m \Gamma_{\mathrm{hom}} Γ hom are distinct attractors contained in the same positively invariant domain D h o m D_{\mathrm{hom}} D hom so that there exists a nonempty compact invariant set K ⊂ D h o m K \subset D_{\mathrm{hom}} K ⊂ D hom such that
K ∩ Γ H = ∅ , K ∩ Γ h o m = ∅ , K \cap\Gamma_H = \varnothing,\qquad K \cap\Gamma_{\mathrm{hom}} = \varnothing, K ∩ Γ H = ∅ , K ∩ Γ hom = ∅ , and K ⊂ ∂ B ( Γ H ) ∩ ∂ B ( Γ h o m ) K \subset\partial B(\Gamma_H) \cap\partial B(\Gamma_{\mathrm{hom}}) K ⊂ ∂ B ( Γ H ) ∩ ∂ B ( Γ hom ) . In particular, system (1) possesses an additional invariant structure inside D h o m D_{\mathrm{hom}} D hom ;
(ii) Assume that D H ⊂ D h o m D_H \subset D_{\mathrm{hom}} D H ⊂ D hom . If α > α c \alpha> \alpha_c α > α c , then Γ h o m \Gamma_{\mathrm{hom}} Γ hom is repelling from the D h o m D_{\mathrm{hom}} D hom side. Furthermore, if Γ H \Gamma_H Γ H is the only periodic orbit contained in D h o m D_{\mathrm{hom}} D hom , then D h o m ∖ { C E − } ⊂ B ( Γ H ) D_{\mathrm{hom}} \setminus\{CE_-\} \subset B(\Gamma_H) D hom ∖ { C E − } ⊂ B ( Γ H ) , that is,
lim t → + ∞ dist ( Φ t ( x ) , Γ H ) = 0 ∀ x ∈ D h o m ∖ { C E − } ; \lim_{t\to+\infty} \operatorname{dist}(\Phi_t(x),\Gamma_H)=0 \qquad\forall x\in D_{\mathrm{hom}}\setminus\{CE_-\}; t → + ∞ lim dist ( Φ t ( x ) , Γ H ) = 0 ∀ x ∈ D hom ∖ { C E − } ;
(iii) Assume that D h o m ⊂ D H D_{\mathrm{hom}} \subset D_H D hom ⊂ D H , α < α c \alpha< \alpha_c α < α c and that Γ h o m \Gamma_{\mathrm{hom}} Γ hom is the only periodic invariant set contained in D h o m D_{\mathrm{hom}} D hom . Then, Γ h o m \Gamma_{\mathrm{hom}} Γ hom is asymptotically stable from the D h o m D_{\mathrm{hom}} D hom side and D h o m ∖ { C E − } ⊂ B ( Γ h o m ) D_{\mathrm{hom}} \setminus\{CE_-\} \subset B(\Gamma_{\mathrm{hom}}) D hom ∖ { C E − } ⊂ B ( Γ hom ) , that is,
lim t → + ∞ dist ( Φ t ( x ) , Γ h o m ) = 0 ∀ x ∈ D h o m ∖ { C E − } . \lim_{t\to+\infty} \operatorname{dist}(\Phi_t(x),\Gamma_{\mathrm{hom}})=0 \qquad\forall x\in D_{\mathrm{hom}}\setminus\{CE_-\}. t → + ∞ lim dist ( Φ t ( x ) , Γ hom ) = 0 ∀ x ∈ D hom ∖ { C E − } .
Moreover, if Γ H \Gamma_H Γ H is the only periodic orbit contained in D H ∖ D h o m ‾ D_H \setminus\overline{D_{\mathrm{hom}}} D H ∖ D hom , then D H ∖ D h o m ‾ ⊂ B ( Γ H ) ∪ W s ( C E + ) D_H \setminus\overline{D_{\mathrm{hom}}} \subset B(\Gamma_H) \cup W^s(CE_+) D H ∖ D hom ⊂ B ( Γ H ) ∪ W s ( C E + ) , that is,
lim t → + ∞ dist ( Φ t ( x ) , Γ H ) = 0 ∀ x ∈ D H ∖ { D h o m ‾ ∪ W s ( C E + ) } . \lim_{t\to+\infty} \operatorname{dist}(\Phi_t(x),\Gamma_H)=0 \qquad\forall x\in D_H\setminus\{\overline{D_{\mathrm{hom}}}\cup W^s(CE_+)\}. t → + ∞ lim dist ( Φ t ( x ) , Γ H ) = 0 ∀ x ∈ D H ∖ { D hom ∪ W s ( C E + )} .
Finally, the coexistence of the two attractors Γ H \Gamma_H Γ H and H F E HFE H F E implies the existence of a nonempty compact invariant set K ⊂ R + 2 ∖ D H K \subset\mathbb{R}_+^2 \setminus D_H K ⊂ R + 2 ∖ D H contained in the common boundary of the corresponding attraction basins, K ⊂ ∂ B ( Γ H ) ∩ ∂ B ( H F E ) K \subset\partial B(\Gamma_H) \cap\partial B(HFE) K ⊂ ∂ B ( Γ H ) ∩ ∂ B ( H F E ) . In particular, system (1) possesses an additional invariant structure separating the asymptotic regimes associated with Γ H \Gamma_H Γ H and H F E HFE H F E ;
(iv) Assume that D h o m ⊂ D H D_{\mathrm{hom}} \subset D_H D hom ⊂ D H . If α > α c \alpha> \alpha_c α > α c , then Γ h o m \Gamma_{\mathrm{hom}} Γ hom is repelling from the D h o m D_{\mathrm{hom}} D hom side. Moreover, system (1) possesses at least one compact invariant set K ⊂ D h o m ∖ { C E − } K \subset D_{\mathrm{hom}} \setminus\{CE_-\} K ⊂ D hom ∖ { C E − } . In particular, D h o m D_{\mathrm{hom}} D hom contains at least one periodic orbit K K K distinct from Γ H \Gamma_H Γ H .
The degenerate case α = α C \alpha= \alpha_C α = α C cannot be determined from the saddle quantity alone and requires a higher-order analysis of the associated Poincaré return map.
Proof. By Lemma 4.3, the periodic orbit Γ H \Gamma_H Γ H and the homoclinic loop Γ h o m \Gamma_{\mathrm{hom}} Γ hom are disjoint, and exactly one of the following two alternatives holds: D H ⊂ D h o m D_H \subset D_{\mathrm{hom}} D H ⊂ D hom or D h o m ⊂ D H D_{\mathrm{hom}} \subset D_H D hom ⊂ D H . Moreover, by Proposition (4.2), C E − ∈ D h o m CE_- \in D_{\mathrm{hom}} C E − ∈ D hom . Since Γ h o m \Gamma_{\mathrm{hom}} Γ hom is a simple closed invariant curve, uniqueness of solutions implies that no trajectory can cross it so that D h o m D_{\mathrm{hom}} D hom is positively invariant. Let λ s < 0 < λ u \lambda_s < 0 < \lambda_u λ s < 0 < λ u be the eigenvalues of the Jacobian matrix at C E + CE_+ C E + and let σ + : = λ s + λ u \sigma_+ := \lambda_s + \lambda_u σ + := λ s + λ u denote the associated saddle quantity. By Lemma 4.2, it follows that
σ + ⋛ 0 ⟺ α ⋛ α c . \sigma_+ \gtreqless0 \qquad\Longleftrightarrow\qquad\alpha\gtreqless\alpha_c. σ + ⋛ 0 ⟺ α ⋛ α c .
Consequently, by the classical local theory of planar homoclinic loops [25 ] , α < α c \alpha< \alpha_c α < α c implies that Γ h o m \Gamma_{\mathrm{hom}} Γ hom is asymptotically stable from the D h o m D_{\mathrm{hom}} D hom side. Conversely, when α > α c \alpha> \alpha_c α > α c , Γ h o m \Gamma_{\mathrm{hom}} Γ hom is repelling from the D h o m D_{\mathrm{hom}} D hom side. Let us now consider separately the four configurations appearing in the statement.
Proof of statement (i). Assume that D H ⊂ D h o m D_H \subset D_{\mathrm{hom}} D H ⊂ D hom and α < α c \alpha< \alpha_c α < α c so that Γ h o m \Gamma_{\mathrm{hom}} Γ hom is asymptotically stable from the D h o m D_{\mathrm{hom}} D hom side and there exists a nonempty open set U H ⊂ D h o m U_H \subset D_{\mathrm{hom}} U H ⊂ D hom such that
lim t → + ∞ dist ( Φ t ( x ) , Γ H ) = 0 for every x ∈ U H . \lim_{t\to+\infty} \operatorname{dist}(\Phi_t(x),\Gamma_H)=0 \qquad\text{for every }x\in U_H. t → + ∞ lim dist ( Φ t ( x ) , Γ H ) = 0 for every x ∈ U H .
Similarly, the one-sided asymptotic stability of Γ h o m \Gamma_{\mathrm{hom}} Γ hom provides a nonempty open set U h o m ⊂ D h o m U_{\mathrm{hom}} \subset D_{\mathrm{hom}} U hom ⊂ D hom such that
lim t → + ∞ dist ( Φ t ( x ) , Γ h o m ) = 0 for every x ∈ U h o m . \lim_{t \to+\infty} \operatorname{dist}(\Phi_t(x),\Gamma_{\mathrm{hom}})=0 \qquad\text{for every } x \in U_{\mathrm{hom}}. t → + ∞ lim dist ( Φ t ( x ) , Γ hom ) = 0 for every x ∈ U hom .
Thus the relative basins B ( Γ H ) \mathcal{B}(\Gamma_H) B ( Γ H ) and B D h o m ( Γ h o m ) : = B ( Γ h o m ) ∩ D h o m \mathcal{B}_{D_{\mathrm{hom}}}(\Gamma_{\mathrm{hom}}):=\mathcal{B}(\Gamma_{\mathrm{hom}})\cap D_{\mathrm{hom}} B D hom ( Γ hom ) := B ( Γ hom ) ∩ D hom are nonempty, open, and disjoint subsets of the connected domain D h o m D_{\mathrm{hom}} D hom . Consequently, their relative boundaries in D h o m D_{\mathrm{hom}} D hom are nonempty so that let us define S : = ∂ B D h o m ( Γ H ) ∩ ∂ B ( Γ h o m ) S:=\partial\mathcal{B}_{D_{\mathrm{hom}}}(\Gamma_H)\cap\partial\mathcal{B}(\Gamma_{\mathrm{hom}}) S := ∂ B D hom ( Γ H ) ∩ ∂ B ( Γ hom ) . Then S S S is nonempty, closed relative to D h o m D_{\mathrm{hom}} D hom , and invariant under the flow. Indeed, let us choose y ∈ S y\in S y ∈ S . Since D ‾ h o m \overline{D}_{\mathrm{hom}} D hom is compact and positively invariant, the positive semiorbit of y y y is bounded and K : = ω ( y ) K:=\omega(y) K := ω ( y ) is a nonempty compact invariant set contained in D ‾ h o m \overline{D}_{\mathrm{hom}} D hom and it cannot intersect Γ H \Gamma_H Γ H . Indeed, if ω ( y ) ∩ Γ H ≠ ∅ \omega(y)\cap\Gamma_H\ne\varnothing ω ( y ) ∩ Γ H = ∅ , the asymptotic stability of Γ H \Gamma_H Γ H would imply that ω ( y ) = Γ H \omega(y)=\Gamma_H ω ( y ) = Γ H and hence that y y y belongs to the basin of Γ H \Gamma_H Γ H , contradicting the fact that y y y belongs to its boundary. The same argument applies to Γ h o m \Gamma_{\mathrm{hom}} Γ hom , using its one-sided asymptotic stability. Therefore,
K ∩ Γ H = ∅ , K ∩ Γ h o m = ∅ , with K ⊂ ∂ B ( Γ H ) ∩ ∂ B ( Γ h o m ) . K\cap\Gamma_H=\varnothing,\qquad K\cap\Gamma_{\mathrm{hom}}=\varnothing,\qquad\text{with}\qquad K\subset\partial\mathcal{B}(\Gamma_H)\cap\partial\mathcal{B}(\Gamma_{\mathrm{hom}}). K ∩ Γ H = ∅ , K ∩ Γ hom = ∅ , with K ⊂ ∂ B ( Γ H ) ∩ ∂ B ( Γ hom ) .
Thus the coexistence of the two attractors forces the existence of an additional compact invariant structure in D h o m D_{\mathrm{hom}} D hom .
Proof of statement (ii).
Assume that D H ⊂ D h o m D_H\subset D_{\mathrm{hom}} D H ⊂ D hom and α > α c \alpha>\alpha_c α > α c so that Γ h o m \Gamma_{\mathrm{hom}} Γ hom is repelling from the D h o m D_{\mathrm{hom}} D hom side. Let x ∈ D h o m ∖ { C E − } x\in D_{\mathrm{hom}}\setminus\{CE_-\} x ∈ D hom ∖ { C E − } . Since D h o m D_{\mathrm{hom}} D hom is bounded and positively invariant, the positive semiorbit of x x x is bounded. It follows that ω ( x ) \omega(x) ω ( x ) is nonempty, compact, connected, and invariant. In Region III, the equilibrium C E − CE_- C E − is a hyperbolic repeller so that W s ( C E − ) = { C E − } W^s(CE_-)=\{CE_-\} W s ( C E − ) = { C E − } , and hence ω ( x ) ≠ { C E − } \omega(x)\ne\{CE_-\} ω ( x ) = { C E − } . Furthermore, the local stable branches of the saddle C E + CE_+ C E + forming the homoclinic configuration belong to the boundary of D h o m D_{\mathrm{hom}} D hom so that W s ( C E + ) ∩ D h o m = ∅ W^s(CE_+)\cap D_{\mathrm{hom}}=\varnothing W s ( C E + ) ∩ D hom = ∅ , and no point of D h o m D_{\mathrm{hom}} D hom can converge to C E + CE_+ C E + . Finally, since Γ h o m \Gamma_{\mathrm{hom}} Γ hom is repelling from the D h o m D_{\mathrm{hom}} D hom side, it cannot be the ω \omega ω -limit set of a point in D h o m D_{\mathrm{hom}} D hom . Therefore ω ( x ) \omega(x) ω ( x ) contains no equilibrium and cannot coincide with Γ h o m \Gamma_{\mathrm{hom}} Γ hom . By the Poincaré–Bendixson theorem, ω ( x ) \omega(x) ω ( x ) is a periodic orbit contained in D h o m D_{\mathrm{hom}} D hom . If Γ H \Gamma_H Γ H is the only periodic orbit contained in D h o m D_{\mathrm{hom}} D hom , then necessarily ω ( x ) = Γ H \omega(x)=\Gamma_H ω ( x ) = Γ H . Hence
lim t → + ∞ dist ( Φ t ( x ) , Γ H ) = 0 for every x ∈ D h o m ∖ { C E − } . \lim_{t \to+\infty}\operatorname{dist}(\Phi_t(x),\Gamma_H)=0 \qquad\text{for every } x\in D_{\mathrm{hom}}\setminus\{CE_-\}. t → + ∞ lim dist ( Φ t ( x ) , Γ H ) = 0 for every x ∈ D hom ∖ { C E − } .
Proof of statement (iii).
Assume that D h o m ⊂ D H D_{\mathrm{hom}}\subset D_H D hom ⊂ D H and α < α c \alpha<\alpha_c α < α c so that Γ h o m \Gamma_{\mathrm{hom}} Γ hom is asymptotically stable from the D h o m D_{\mathrm{hom}} D hom side. Assume that Γ h o m \Gamma_{\mathrm{hom}} Γ hom is the only periodic invariant set contained in D h o m D_{\mathrm{hom}} D hom and let x ∈ D h o m ∖ { C E − } x\in D_{\mathrm{hom}}\setminus\{CE_-\} x ∈ D hom ∖ { C E − } . As before, D h o m D_{\mathrm{hom}} D hom is bounded and positively invariant, so that ω ( x ) \omega(x) ω ( x ) is nonempty, compact, connected, and invariant. Since C E − CE_- C E − is a hyperbolic repeller, ω ( x ) ≠ { C E − } \omega(x)\ne\{CE_-\} ω ( x ) = { C E − } , and, since W s ( C E + ) ∩ D h o m = ∅ W^s(CE_+)\cap D_{\mathrm{hom}}=\varnothing W s ( C E + ) ∩ D hom = ∅ , the set ω ( x ) \omega(x) ω ( x ) cannot contain C E + CE_+ C E + . If ω ( x ) \omega(x) ω ( x ) contains no equilibrium, the Poincaré–Bendixson theorem implies that it is either a periodic orbit or a finite union of equilibria and connecting trajectories. The latter alternative is excluded because the only saddle on ∂ D h o m \partial D_{\mathrm{hom}} ∂ D hom is C E + CE_+ C E + and the only corresponding invariant graphic is the homoclinic loop itself. By the uniqueness assumption on periodic invariant sets, one obtains ω ( x ) = Γ h o m \omega(x)=\Gamma_{\mathrm{hom}} ω ( x ) = Γ hom so that
lim t → + ∞ dist ( Φ t ( x ) , Γ h o m ) = 0 for every x ∈ D h o m ∖ { C E − } . \lim_{t\to+\infty}\operatorname{dist}\left(\Phi_t(x),\Gamma_{\mathrm{hom}}\right)=0 \qquad\text{for every }x\in D_{\mathrm{hom}}\setminus\{CE_{-}\}. t → + ∞ lim dist ( Φ t ( x ) , Γ hom ) = 0 for every x ∈ D hom ∖ { C E − } .
Consider now the open annular region A : = D H ∖ D h o m ‾ A:=D_H\setminus\overline{D_{\mathrm{hom}}} A := D H ∖ D hom . Both boundary components of A A A are invariant, and therefore A A A is positively invariant. Let x ∈ A ∖ W s ( C E + ) x\in A\setminus W^s(CE_{+}) x ∈ A ∖ W s ( C E + ) . Its positive semiorbit is contained in the bounded set A A A , and hence ω ( x ) \omega(x) ω ( x ) is nonempty, compact, connected, and invariant. The set ω ( x ) \omega(x) ω ( x ) cannot contain C E − CE_{-} C E − , because C E − ∈ D h o m CE_{-}\in D_{\mathrm{hom}} C E − ∈ D hom , and it cannot contain C E + CE_{+} C E + because x ∉ W s ( C E + ) x\notin W^s(CE_{+}) x ∈ / W s ( C E + ) . Moreover, by hypothesis there is no periodic orbit in A A A other than Γ H \Gamma_H Γ H . Since Γ h o m \Gamma_{\mathrm{hom}} Γ hom does not attract trajectories from A A A , the Poincaré–Bendixson theorem gives ω ( x ) = Γ H \omega(x)=\Gamma_H ω ( x ) = Γ H so that
lim t → + ∞ dist ( Φ t ( x ) , Γ H ) = 0 ∀ x ∈ D H ∖ { D h o m ‾ ∪ W s ( C E + ) } . \lim_{t\to+\infty}\operatorname{dist}\left(\Phi_t(x),\Gamma_H\right)=0 \qquad\forall x\in D_H\setminus\left\{\overline{D_{\mathrm{hom}}}\cup W^s(CE_{+})\right\}. t → + ∞ lim dist ( Φ t ( x ) , Γ H ) = 0 ∀ x ∈ D H ∖ { D hom ∪ W s ( C E + ) } .
Finally, assume that both B ( Γ H ) ‾ \overline{B(\Gamma_H)} B ( Γ H ) and B ( H F E ) ‾ \overline{B(HFE)} B ( H F E ) have nonempty intersections with R + 2 ∖ D H ‾ \mathbb{R}_{+}^{2}\setminus\overline{D_H} R + 2 ∖ D H . These intersections are open, disjoint, and positively invariant. If they coexist in the connected component of R + 2 ∖ D H ‾ \mathbb{R}_{+}^{2}\setminus\overline{D_H} R + 2 ∖ D H adjacent to Γ H \Gamma_H Γ H , their common relative boundary is nonempty. Choose a point z ∈ ∂ B R + 2 ∖ D H ‾ ( Γ H ) ∩ ∂ B ( H F E ) z\in\partial B_{\mathbb{R}_{+}^{2}\setminus\overline{D_H}}(\Gamma_H)\cap\partial B(HFE) z ∈ ∂ B R + 2 ∖ D H ( Γ H ) ∩ ∂ B ( H F E ) . By boundedness of the positive solutions, ω ( z ) \omega(z) ω ( z ) is a nonempty compact invariant set. Setting K : = ω ( z ) K:=\omega(z) K := ω ( z ) , one obtains K ⊂ R + 2 ∖ D H K\subset\mathbb{R}_{+}^{2}\setminus D_H K ⊂ R + 2 ∖ D H and K ⊂ ∂ B ( Γ H ) ∩ ∂ B ( H F E ) K\subset\partial B(\Gamma_H)\cap\partial B(HFE) K ⊂ ∂ B ( Γ H ) ∩ ∂ B ( H F E ) . Moreover, K K K is distinct from both Γ H \Gamma_H Γ H and H F E HFE H F E , since an asymptotically stable invariant set belongs to the interior of its basin. Thus an additional compact invariant structure separates the asymptotic regimes governed by Γ H \Gamma_H Γ H and H F E HFE H F E .
Proof of statement (iv). Assume that D h o m ⊂ D H D_{\mathrm{hom}}\subset D_H D hom ⊂ D H and α > α c \alpha>\alpha_c α > α c so that Γ h o m \Gamma_{\mathrm{hom}} Γ hom is repelling from the D h o m D_{\mathrm{hom}} D hom side and choose x ∈ D h o m ∖ { C E − } x\in D_{\mathrm{hom}}\setminus\{CE_{-}\} x ∈ D hom ∖ { C E − } . The positive semiorbit of x x x is bounded because D h o m D_{\mathrm{hom}} D hom is bounded and positively invariant. Therefore ω ( x ) \omega(x) ω ( x ) is nonempty, compact, connected, and invariant. Since C E − CE_{-} C E − is a hyperbolic repeller, ω ( x ) ≠ { C E − } \omega(x)\ne\{CE_{-}\} ω ( x ) = { C E − } . Moreover, W s ( C E + ) ∩ D h o m = ∅ W^s(CE_{+})\cap D_{\mathrm{hom}}=\varnothing W s ( C E + ) ∩ D hom = ∅ , so ω ( x ) \omega(x) ω ( x ) cannot contain C E + CE_{+} C E + . Since the homoclinic loop is repelling from the D h o m D_{\mathrm{hom}} D hom side, ω ( x ) ≠ Γ h o m \omega(x)\ne\Gamma_{\mathrm{hom}} ω ( x ) = Γ hom . It follows that ω ( x ) \omega(x) ω ( x ) contains no equilibrium. By the Poincaré–Bendixson theorem, ω ( x ) \omega(x) ω ( x ) is a periodic orbit contained in D h o m D_{\mathrm{hom}} D hom . Define K : = ω ( x ) K:=\omega(x) K := ω ( x ) a compact invariant set satisfying K ⊂ D h o m ∖ { C E − } K\subset D_{\mathrm{hom}}\setminus\{CE_{-}\} K ⊂ D hom ∖ { C E − } . Since D h o m ⊂ D H D_{\mathrm{hom}}\subset D_H D hom ⊂ D H and Γ H = ∂ D H \Gamma_H=\partial D_H Γ H = ∂ D H , one has Γ H ∩ D h o m = ∅ \Gamma_H\cap D_{\mathrm{hom}}=\varnothing Γ H ∩ D hom = ∅ . Consequently, K ≠ Γ H K\ne\Gamma_H K = Γ H . Thus D h o m D_{\mathrm{hom}} D hom contains at least one periodic orbit distinct from the Hopf cycle Γ H \Gamma_H Γ H .
A qualitative sketch of the above cases is provided in Figure 7 . Finally, if α = α c \alpha=\alpha_c α = α c , then σ + = 0 \sigma_{+}=0 σ + = 0 . In this case the leading-order term determining the stability of the homoclinic return map vanishes. Therefore the stability of Γ h o m \Gamma_{\mathrm{hom}} Γ hom cannot be inferred from the saddle quantity alone and requires a higher-order analysis of the Poincaré return map.
Figure 7. Schematic phase-plane sketches illustrating the qualitative organization of the invariant structures considered in Theorem 4.5 . Panels (a)-(d) correspond to cases (i)-(iv), respectively.
Section summary
The classification established in Proposition 4.1, together with the heteroclinic and homoclinic results of Theorems 4.1-4.5 , provides a complete qualitative description of the global phase-plane organization of system (1). The corresponding dynamical configurations are summarized in Figure 8 , where each panel represents the phase portrait associated with one of the five regions of the parameter plane. In particular:
Figure 8. Qualitative phase-plane configurations associated with the five regions of the parameter plane. Red dots denote stable equilibria, blue circles unstable equilibria, black crosses saddle points, dash-dotted blue curves stable limit cycles, dashed purple curves homoclinic loops, and solid curves heteroclinic connections.
i) Region I corresponds to the simplest dynamical regime. The only biologically admissible equilibria are the trivial equilibrium T E TE T E and the herbivore-free equilibrium H F E HFE H F E . The invariant boundary segment Γ \Gamma Γ provides the heteroclinic connection T E → H F E TE \to HFE T E → H F E , while H F E HFE H F E is globally attracting within the biologically relevant region int Ω \operatorname{int}\Omega int Ω . Consequently, all trajectories eventually converge to herbivore extinction.
ii) Region II represents a rich equilibrium configuration in which both coexistence equilibria C E − CE_{-} C E − and C E + CE_{+} C E + are present, with C E − CE_{-} C E − asymptotically stable and C E + CE_{+} C E + a saddle. Besides the boundary connection T E → H F E TE \to HFE T E → H F E , the unstable manifold of C E + CE_{+} C E + generates heteroclinic trajectories toward both H F E HFE H F E and C E − CE_{-} C E − .
iii) Region III is characterized by the loss of stability of C E − CE_{-} C E − through the supercritical Hopf bifurcation described in Theorem 3.1. As a consequence, a stable limit cycle Γ H \Gamma_{H} Γ H surrounds the unstable coexistence equilibrium C E − CE_{-} C E − . The saddle equilibrium C E + CE_{+} C E + persists and still generates
the heteroclinic connection toward H F E HFE H F E . The global dynamics may, however, exhibit several additional configurations depending on the possible occurrence and geometric arrangement of homoclinic structures associated with C E + CE_{+} C E + . These scenarios, together with their dynamical implications, are analyzed in detail in Theorem 4.5 (or, in a qualitative sketch, in Figure 7 ). For the sake of clarity, Figure 8 displays only a representative qualitative phase portrait of Region III rather than all admissible configurations. Consequently, the long-term dynamics are generally governed by stable periodic oscillations and/or alternative attractors separated by the invariant manifolds associated with C E + CE_{+} C E + , while additional invariant structures may arise in the presence of homoclinic configurations (see Theorem 4.5 ).
iv) Region IV is characterized by the disappearance of the saddle equilibrium C E + CE_{+} C E + , while the coexistence equilibrium C E − CE_{-} C E − remains unstable. As a consequence, the stable periodic orbit generated through the Hopf bifurcation persists and becomes the unique attractor in the biologically relevant region. The only heteroclinic connection supported by the system is the invariant boundary orbit T E → H F E TE \to HFE T E → H F E .
v) Region V corresponds to a stable coexistence regime. The coexistence equilibrium C E − CE_{-} C E − is asymptotically stable, whereas both T E TE T E and H F E HFE H F E are saddles. In addition to the boundary connection T E → H F E TE \to HFE T E → H F E , the unstable manifold of H F E HFE H F E enters the interior of the phase plane and converges to C E − CE_{-} C E − , generating the heteroclinic connection H F E → C E − HFE \to CE_{-} H F E → C E − . Hence every biologically relevant trajectory belonging to int Ω \operatorname{int}\Omega int Ω converges to the coexistence equilibrium.
The five configurations illustrate how the interaction between local bifurcations and global invariant manifolds reorganizes the phase portrait. Depending on the parameter values, the system may exhibit herbivore extinction, stable coexistence, bistability, or persistent oscillatory dynamics, all of which are organized by a small number of heteroclinic, homoclinic, and periodic structures.
While Figure 8 emphasizes the geometric organization of the phase portraits, Table 1 provides a synthetic overview of the invariant structures supported by the system in each parameter region. In particular, it highlights the attractors, the admissible heteroclinic connections, the possible homoclinic configurations, and the occurrence of stable limit cycles.
Table 1. Summary of the invariant structures and asymptotic dynamics in the five parameter regions.
Numerical validation
In this section, we provide a numerical validation of the analytical results established in § 3 and § 4 . Our objectives are twofold. First, we verify the local bifurcation scenario described by Proposition 3.1, Theorem 3.1, and Corollary 3.1 through numerical continuation of equilibria and periodic solutions. Second, we reconstruct representative phase portraits associated with the five parameter regions identified in Proposition 4.1 and Theorems 4.1-4.5 , and compare the resulting invariant structures with the theoretical configurations summarized in Figure 8 .
Unless otherwise stated, the model parameters are selected so as to belong to the interior of the corresponding region in the parameter plane displayed in Figure 3 . Direct numerical integrations of system (1) were performed in MATLAB using the adaptive Runge-Kutta solver ode45 [28 ] . Relative and absolute tolerances were fixed to RelTol = 10 − 8 10^{-8} 1 0 − 8 and AbsTol = 10 − 10 10^{-10} 1 0 − 10 , and all computations were repeated with stricter tolerances to verify numerical convergence. All simulations were carried out in the biologically admissible region Ω = R + 2 \Omega= \mathbb{R}^{2}_{+} Ω = R + 2 , and were integrated over sufficiently long time intervals to eliminate transient dynamics and identify the corresponding ω \omega ω -limit sets. Numerical bifurcation analysis was performed using the continuation package AUTO-07P [29 ] . Throughout the continuation procedure, the herbivore mortality parameter θ \theta θ was used as the principal continuation parameter.
Numerical continuation of equilibria and periodic orbits
Let us begin by investigating the local bifurcation structure described in § 3 through numerical continuation performed with AUTO-07P. The purpose of this analysis is to verify the existence and stability properties of the equilibrium branches predicted by Lemmas 3.1-3.2 , as well as the Hopf and saddle-node bifurcations established in Proposition 3.1 and Theorem 3.1. On the other hand, we also corroborate the qualitative bifurcation scenario summarized in Figure 2 , showing that the sequence of stability transitions and bifurcation events predicted analytically is accurately reproduced by numerical continuation.
In this context, Figure 9 shows the bifurcation diagram obtained by using the herbivore mortality θ \theta θ as continuation parameter. The diagram depicted in panel (a) reproduces all equilibrium branches predicted by the theoretical analysis. In particular, the herbivore-free equilibrium H F E HFE H F E and the trivial state T E TE T E persist for all parameter values, while the two coexistence equilibria C E − CE_{-} C E − and C E + CE_{+} C E + appear for intermediate values of θ \theta θ , namely for 0 < θ ≤ θ S N 0 < \theta\leq\theta_{SN} 0 < θ ≤ θ S N and θ H F E < θ ≤ θ S N \theta_{HFE} < \theta\leq\theta_{SN} θ H F E < θ ≤ θ S N , respectively. As expected from Lemma 3.2 , the branch C E + CE_{+} C E + is always unstable, whereas the stability of C E − CE_{-} C E − changes along the branch according to the position of the Hopf threshold. Indeed, the numerical continuation detects a Hopf bifurcation point (H) on the coexistence branch C E − CE_{-} C E − , where the equilibrium loses stability and a family of periodic solutions emerges. In agreement with Corollary 3.1, the bifurcating limit cycles are asymptotically stable for θ < θ H \theta< \theta_{H} θ < θ H sufficiently close to the Hopf threshold. As θ \theta θ is further decreased, the periodic branch remains stable over a wide parameter interval. However, the cycles are numerically detected as unstable along the terminal portion of the computed branch. Therefore, the supercritical nature of the Hopf bifurcation determines the local stability of the periodic solutions near θ H \theta_{H} θ H , but does not imply stability of the entire global family. The continuation terminates at θ = θ E \theta= \theta_{E} θ = θ E , where AUTO-07P reports an MX point. This label does not identify a bifurcation of the dynamical system, but indicates that the continuation could not be advanced within the prescribed numerical bounds. In particular, the endpoint may result from the increasing difficulty of resolving the periodic orbit as its period grows and its temporal profile becomes strongly nonuniform. Accordingly, θ E \theta_E θ E should be interpreted as the terminal parameter value reached by the numerical continuation rather than as a bifurcation threshold.
Figure 9. Numerical bifurcation diagram of system (1) with respect to the herbivore mortality parameter θ \theta θ , computed using AUTO–07P. (a) Equilibrium and periodic-solution branches. Solid and dashed curves denote stable and unstable branches, respectively. The coexistence equilibrium C E − CE_- C E − loses stability at the Hopf point (H), generating a family of periodic orbits. The coexistence equilibria C E − CE_- C E − and C E + CE_+ C E + collide at the saddle-node bifurcation (SN), while the herbivore-free equilibrium HFE changes stability at θ H F E \theta_{HFE} θ H F E . (b) Period T T T of the periodic solutions as a function of θ \theta θ . Parameter values: α = 2 \alpha= 2 α = 2 , m = 10 m = 10 m = 10 and μ = 0.85 \mu= 0.85 μ = 0.85 .
Panel (b) shows that the period T T T of the oscillations increases rapidly as the branch approaches its terminal point. Such a growth is typically regarded as a classical signature of periodic solutions approaching a global bifurcation and suggests that the family may be converging towards one of the homoclinic configurations analysed in § 4.4.1 . This interpretation is supported by the geometric results obtained there. Indeed, Theorem 4.2 identifies the saddle equilibrium C E + CE_+ C E + as the only equilibrium that can support a homoclinic loop, while Theorem 4.5 characterizes the dynamical role of the resulting homoclinic structures in Region III. Therefore, the observed growth of the period, together with the numerical termination of the branch at the MX point, is consistent with the approach of the periodic orbit to a homoclinic configuration involving C E + CE_+ C E + . Since neither the MX label nor the period growth alone provide a rigorous detection criterion, this interpretation should be viewed as numerical evidence rather than a proof of the existence of a homoclinic connection.
The diagram also confirms the saddle-node bifurcation predicted by Proposition 3.1. Indeed, the coexistence equilibria C E − CE_- C E − and C E + CE_+ C E + approach each other and match at a turning point corresponding to the threshold θ S N \theta_{SN} θ S N . For larger values of θ \theta θ , no positive coexistence equilibrium exists and HFE becomes the only biologically admissible equilibrium. A further feature visible in Figure 9 is the stability change of HFE at the threshold θ H F E \theta_{HFE} θ H F E . For θ < θ H F E \theta< \theta_{HFE} θ < θ H F E , HFE is unstable, whereas for θ > θ H F E \theta> \theta_{HFE} θ > θ H F E it becomes locally asymptotically stable. Consequently, the interval θ H F E < θ < θ S N \theta_{HFE} < \theta< \theta_{SN} θ H F E < θ < θ S N corresponds to a bistable regime in which HFE and C E − CE_- C E − coexist as alternative attractors and the asymptotic outcome depends on the initial conditions.
Overall, the numerical continuation reproduces all local bifurcation mechanisms derived analytically in § 3 and provides quantitative confirmation of the theoretical bifurcation scenario underlying the global phase-plane organization investigated in the following subsections.
Representative phase portraits in Regions I - V
To complement the bifurcation analysis presented above, we now reconstruct representative phase portraits corresponding to the five parameter regions identified in Proposition 4.1. The aim is to verify numerically the qualitative phase-plane organization predicted by the theoretical analysis and sketched in Figure 8 , and to illustrate the invariant structures characterizing each dynamical regime. Figure 10 summarizes these results by showing the partition of the parameter plane together with representative trajectories computed for parameter values selected within each region.
Figure 10. Numerical phase portraits associated with the five dynamical regions identified in Proposition 4.1. Panel (a) shows the partition of the parameter plane ( θ , μ ) (\theta,\mu) ( θ , μ ) into Regions I - V. The markers indicate the parameter pairs ( θ , μ ) (\theta,\mu) ( θ , μ ) selected for the corresponding phase portraits displayed in panels (b)-(f), respectively. Specifically, panel (b) represents Region I, panel (c) Region II, panel (d) Region III, panel (e) Region IV, and panel (f) Region V. Other parameter values fixed as in Figure 9 .
In particular, Figure 10 (a) displays the partition of the parameter plane ( θ , μ ) (\theta,\mu) ( θ , μ ) introduced in Proposition 4.1. The coloured markers identify the specific parameter pairs selected for the numerical simulations reported in panels (b)-(f). In particular, the black square, yellow triangle, blue diamond, magenta star, and green circle correspond to representative parameter values taken in Regions I - V, respectively. The inset provides a zoomed-in view of the parameter plane, where Region II becomes narrowly separated and difficult to distinguish at the scale of the full diagram.
Figure 10 (b)-(Figure 10 f) display representative phase portraits corresponding to the five parameter regions identified in Proposition 4.1. Throughout these panels, black crosses denote saddles, red circles asymptotically stable equilibria, and blue circles unstable one. In particular, panel (b) corresponds to Region I in which, in agreement with Proposition 4.1, the only biologically admissible equilibria are the trivial equilibrium TE and the herbivore-free equilibrium HFE. The former is a saddle, whereas the latter is a stable node and represents the unique attractor of the system. The red trajectory reproduces the heteroclinic connection T E → H F E TE \to HFE T E → H F E predicted by Theorem 4.1(iii), whereas the blue one, started from a generic point in the biologically relevant region, converges asymptotically to HFE, confirming that the basin of attraction of HFE coincides with the whole phase plane. Consequently, every biologically relevant solution approaches the herbivore-free equilibrium as t → + ∞ t \to+\infty t → + ∞ , validating the extinction scenario predicted by the theoretical analysis.
A markedly richer dynamical configuration is observed in panel (c), corresponding to Region II. In agreement with Proposition 4.1, the phase plane contains two saddle equilibria, namely TE and C E + CE_{+} C E + , together with the two asymptotically stable equilibria HFE and C E − CE_{-} C E − . The trajectories shown in the figure highlight both the heteroclinic connections predicted by Theorem 4.1(iv) and representative solution orbits converging towards the attractors of the system. In particular, the boundary orbit T E → H F E TE \to HFE T E → H F E is recovered along the invariant axis H = 0 H=0 H = 0 , while the unstable manifold of C E + CE_{+} C E + generates the heteroclinic trajectories C E + → H F E CE_{+} \to HFE C E + → H F E and C E + → C E − CE_{+} \to CE_{-} C E + → C E − . The latter connection is consistent with the theoretical prediction in the absence of homoclinic loops associated with C E + CE_{+} C E + . The blue and red trajectories illustrate the bistable nature of this regime: depending on the initial condition, solutions converge either to HFE or to C E − CE_{-} C E − , confirming that both equilibria act as attractors. The stable manifold of the saddle equilibrium C E + CE_{+} C E + , displayed as a dashed black curve, clearly acts as the separatrix between the attraction basins of HFE and C E − CE_{-} C E − . Note that the stable invariant manifolds of the saddle equilibrium were reconstructed numerically from the eigenspaces of the linearized system starting from C E + ± ε v s CE_{+} \pm\varepsilon v_s C E + ± ε v s and C E + ± ε v u CE_{+} \pm\varepsilon v_u C E + ± ε v u , with 0 < ε ≪ 1 0 < \varepsilon\ll1 0 < ε ≪ 1 , being v s v_s v s and v u v_u v u the normalized eigenvectors associated with the stable and unstable eigenvalues, respectively. Initial conditions located on opposite sides of this manifold converge to different attractors, in complete agreement with the analytical prediction of Theorem 4.1.
A further qualitative transition is observed in panel (d), corresponding to Region III. In agreement with Proposition 4.1, the equilibria TE and C E + CE_{+} C E + remain saddle points, HFE is still asymptotically stable, whereas the coexistence equilibrium C E − CE_{-} C E − has lost stability through the Hopf bifurcation described in Theorem 3.1. As a consequence, a stable limit cycle emerges around C E − CE_{-} C E − , in accordance with Theorem 3.1 and Corollary 3.1. The phase portrait clearly reproduces the heteroclinic connections predicted by Theorem 4.1. In particular, the boundary orbit T E → H F E TE \to HFE T E → H F E persists, while one branch of the unstable manifold of C E + CE_{+} C E + generates the heteroclinic connection C E + → H F E CE_{+} \to HFE C E + → H F E . At the same time, trajectories initialized in a neighbourhood of the unstable equilibrium C E − CE_{-} C E − are repelled from it and converge towards the stable periodic orbit, confirming its attracting character. The numerical simulations therefore validate the theoretical prediction that Region III is dominated by persistent oscillatory dynamics. Consequently, the long-term dynamics are governed either by the stable limit cycle or by the herbivore-free equilibrium H F E HFE H F E , while the stable manifold of C E + CE_{+} C E + acts as the separatrix between their attraction basins.
Panel (e) illustrates the subsequent dynamical regime arising after the disappearance of the saddle equilibrium C E + CE_{+} C E + . In agreement with Proposition 4.1, the phase plane now contains only the boundary equilibria T E TE T E and H F E HFE H F E , both of saddle type, together with the unstable coexistence equilibrium C E − CE_{-} C E − . The stable periodic orbit generated through the Hopf bifurcation in Region III persists after the disappearance of C E + CE_{+} C E + and surrounds C E − CE_{-} C E − . In particular, it remains the unique attractor of the biologically relevant region. The numerical trajectories confirm this prediction. Solutions initiated from different points of the phase plane are first repelled from C E − CE_{-} C E − and subsequently approach the stable limit cycle. Moreover, the only heteroclinic connection supported by the system is the boundary orbit T E → H F E TE \to HFE T E → H F E , consistently with Theorem 4.1. Consequently, the long-term dynamics are entirely dominated by persistent oscillations, validating the theoretical phase-plane organization predicted for Region IV.
Finally, panel (f) corresponds to Region V, where stable coexistence is recovered. In agreement with Proposition 4.1, the phase plane contains the two saddle equilibria T E TE T E and H F E HFE H F E together with the asymptotically stable coexistence equilibrium C E − CE_{-} C E − . The trajectories displayed in the phase portrait reproduce the heteroclinic structure predicted by Theorem 4.1. In particular, the boundary connection T E → H F E TE \to HFE T E → H F E persists, while the unstable manifold of H F E HFE H F E enters the interior of the phase plane and generates the heteroclinic connection H F E → C E − HFE \to CE_{-} H F E → C E − . Furthermore, representative trajectories initialized in the biologically relevant region converge asymptotically to C E − CE_{-} C E − , confirming that the coexistence equilibrium constitutes the unique attractor of the system. Consequently, every biologically meaningful solution approaches a stable coexistence state between vegetation and herbivores, validating the theoretical phase-plane organization predicted for Region V.
Numerical detection of homoclinic connections
The analytical results of § 4.4.1 identify the saddle equilibrium C E + CE_{+} C E + as the only equilibrium that may support a homoclinic connection and characterize the geometric configurations compatible with such a connection. The purpose of this section is to provide a numerical validation of these predictions. To this aim, we reconstruct the stable and unstable manifolds of C E + CE_{+} C E + and introduce a numerical splitting function measuring their relative position on a suitable transverse section. This approach allows us to illustrate the exclusion mechanism established in Theorem 4.4 within Region II and to detect a homoclinic connection in Region III through a sign change of the splitting function, consistently with the criterion provided by Theorem 4.3.
To this aim, we introduce a transverse section
Σ = { ( B , H ) ∈ Ω : B = B Σ } , B Σ < B + , \Sigma= \left\{(B,H) \in\Omega: B = B_{\Sigma}\right\}, \qquad B_{\Sigma} < B_{+}, Σ = { ( B , H ) ∈ Ω : B = B Σ } , B Σ < B + ,
and follow the relative position of W N W u ( C E + ) W^{u}_{NW}(\mathrm{CE}_{+}) W N W u ( CE + ) and W s ( C E + ) W^{s}(\mathrm{CE}_{+}) W s ( CE + ) that, in this case, reduces to W S W s ( C E + ) W^{s}_{SW}(\mathrm{CE}_{+}) W S W s ( CE + ) . Denoting by
q u ( θ ) = ( B Σ , H Σ u ( θ ) ) and q s ( θ ) = ( B Σ , H Σ s ( θ ) ) q^{u}(\theta) = (B_{\Sigma}, H^{u}_{\Sigma}(\theta)) \qquad\text{and} \qquad q^{s}(\theta) = (B_{\Sigma}, H^{s}_{\Sigma}(\theta)) q u ( θ ) = ( B Σ , H Σ u ( θ )) and q s ( θ ) = ( B Σ , H Σ s ( θ ))
their corresponding intersections with Σ \Sigma Σ , we define the numerical splitting function
Δ n u m ( θ ) = H Σ u ( θ ) − H Σ s ( θ ) . \Delta_{\mathrm{num}}(\theta) = H^{u}_{\Sigma}(\theta) - H^{s}_{\Sigma}(\theta). Δ num ( θ ) = H Σ u ( θ ) − H Σ s ( θ ) .
According to Theorem 4.3, a zero of Δ n u m \Delta_{\mathrm{num}} Δ num corresponds to an intersection between the selected stable and unstable manifold branches on the transverse section and therefore provides evidence of a homoclinic connection associated with C E + \mathrm{CE}_{+} CE + . Figure 11 summarizes the numerical results. The upper row corresponds to a continuation path entirely contained in Region II (panels (a)-(c)), whereas the lower row corresponds to a continuation path in Region III (panels (d)-(f)). Indeed, the first row provides a numerical illustration of the exclusion mechanism established analytically in Theorem 4.4 . In particular, the reconstructed stable and unstable manifolds remain clearly separated throughout the biologically admissible continuation interval (see panel (a)) and the same behaviour is observed on the transverse section, where the intersection coordinates H Σ u H^{u}_{\Sigma} H Σ u and H Σ s H^{s}_{\Sigma} H Σ s remain distinct for all considered parameter values (see panel (b)). Consequently, the numerical splitting function Δ n u m \Delta_{\mathrm{num}} Δ num does not approach zero and preserves a constant sign along the entire continuation path (panel (c)). Therefore, the selected manifold branches never exchange their relative position and no homoclinic configuration is detected. Although this calculation does not constitute a proof of nonexistence, it is fully consistent with the global analytical exclusion result provided by Theorem 4.4 .
Figure 11. Numerical validation of the homoclinic framework of § 4.4.1 . The top row corresponds to a continuation path in Region II, whereas the bottom one corresponds to Region III. The left panels show the reconstructed stable manifolds W s ( C E + ) W^{s}(\mathrm{CE}_{+}) W s ( CE + ) (red) and unstable manifolds W u ( C E + ) W^{u}(\mathrm{CE}_{+}) W u ( CE + ) (black), together with the transverse section Σ \Sigma Σ (B Σ = 1.5 B_{\Sigma}=1.5 B Σ = 1.5 in panels (a)–(c) and B Σ = 1 B_{\Sigma}=1 B Σ = 1 in panels (d)–(f)). In Region II (III) the manifolds are displayed for θ = 2.79 \theta=2.79 θ = 2.79 (θ = 2.46 \theta=2.46 θ = 2.46 ) in dashed lines, θ = 2.80 \theta=2.80 θ = 2.80 (θ = 2.4732 \theta=2.4732 θ = 2.4732 ) in solid lines, and θ = 2.81 \theta=2.81 θ = 2.81 (θ = 2.49 \theta=2.49 θ = 2.49 ) in dotted one. The middle panels report the corresponding intersection coordinates H Σ u H_{\Sigma}^{u} H Σ u and H Σ s H_{\Sigma}^{s} H Σ s on the transverse section as functions of θ \theta θ . The right panels show the associated numerical splitting function Δ n u m ( θ ) \Delta_{\mathrm{num}}(\theta) Δ num ( θ ) .
On the other hand, the second row illustrates a qualitatively different situation arising in Region III. As the parameter θ \theta θ varies, the selected manifold branches approach each other and eventually exchange their relative position (see panel (d)). This transition is reflected in the section coordinates H Σ u H_{\Sigma}^{u} H Σ u and H Σ s H_{\Sigma}^{s} H Σ s , which intersect at a parameter value close to θ h o m ≈ 2.4732 \theta_{\mathrm{hom}}\approx2.4732 θ hom ≈ 2.4732 (see panel (e)). Correspondingly, the splitting function Δ n u m \Delta_{\mathrm{num}} Δ num changes sign and vanishes at an intermediate parameter value. The observed sign reversal indicates that the unstable and stable manifold branches pass through each other on the transverse section (see blue diamond in panels (e) and (f)), exactly as predicted by the geometric criterion of Theorem 4.3. The manifold reconstruction at θ ≈ θ h o m \theta\approx\theta_{\mathrm{hom}} θ ≈ θ hom simultaneously reveals the formation of a homoclinic loop based at C E + \mathrm{CE}_{+} CE + . These observations provide numerical evidence of a codimension-one homoclinic connection in Region III.
The numerical computations of Figure 11 therefore reproduce the two fundamental geometric scenarios predicted by the theoretical analysis: (i) in Region II the manifold branches remain separated and the splitting function stays away from zero, consistently with Theorem 4.4 , so that no homoclinic solution emerges; (ii) in Region III the splitting function undergoes a sign change and a homoclinic loop is observed, in agreement with the criterion established in Theorem 4.3.
Conclusion
In this work, we investigated the global dynamics of a toxin-mediated plant-herbivore model with a non-monotone defence-dependent functional response.
Building upon the local bifurcation analysis previously developed for this class of models, we focused on the geometric structures that organize the phase portrait and determine the long-term behaviour of the ecological system. First, we provided a complete characterization of the biologically admissible equilibria and their stability properties. The analysis revealed how the transition from a monotone to a non-monotone consumption function generates qualitatively distinct equilibrium configurations and partitions the parameter space into different dynamical regimes. In particular, the coexistence equilibrium C E − CE_{-} C E − may lose stability through a supercritical Hopf bifurcation, leading to the emergence of stable periodic oscillations. The main contribution of the paper is the complete geometric classification of the global phase-plane organization of the model, including the characterization of the heteroclinic framework, the admissible homoclinic configurations, and the associated parameter-space partition. In particular, we obtain a complete classification of the heteroclinic connections supported by the model and derive rigorous constraints on the occurrence of homoclinic configurations. These results complement the existing local bifurcation theory by revealing how equilibrium states, periodic orbits, and global connections are embedded within a unified phase-plane structure. From an ecological perspective, the analysis clarifies how toxin-mediated inhibition can produce markedly different outcomes, including herbivore extinction, stable coexistence, and persistent population oscillations. More importantly, it shows that these ecological regimes are organized by a relatively small set of invariant structures whose configuration changes as the model parameters vary.
Several questions remain open. The most immediate one concerns the multiscale nature of the oscillatory regime. Numerical evidence suggests that some periodic solutions may display slow-fast behaviour, and a rigorous analysis within the framework of Geometric Singular Perturbation Theory could provide further insight into the mechanisms underlying these oscillations [30 , 31 , 32 , 33 ] . Then, the interplay between spatial effects and multiscale dynamics may generate complex spatiotemporal behaviours, including relaxation oscillations, localized structures, and pattern transitions occurring on multiple temporal and spatial scales. Developing a unified geometric framework for these phenomena represents a promising direction for future research [34 , 35 , 36 , 37 ] .
Acknowledgments
This research was funded by INdAM-GNFM.
Data availability
No data was used for the research described in the article.
Sign of the first Lyapunov coefficient
The purpose of this appendix is to prove that the first Lyapunov coefficient associated with the Hopf bifurcation occurring along the coexistence branch C E − CE_{-} C E − is always negative. Recall that, after imposing the Hopf condition, the quantity determining the sign of the first Lyapunov coefficient can be written as
Δ H = 4 B H ( 1 + B H ) 7 [ ( 1 − μ ) B H + μ + 1 ] P ( B H , μ ) , \Delta_H = \frac{4B_H}{(1+B_H)^7\left[(1-\mu)B_H+\mu+1\right]}P(B_H,\mu), Δ H = ( 1 + B H ) 7 [ ( 1 − μ ) B H + μ + 1 ] 4 B H P ( B H , μ ) ,
where
P ( B H , μ ) = ( 1 − μ ) 2 ( 2 μ − 1 ) B H 4 + ( 1 − μ ) ( μ 2 + 6 μ − 4 ) B H 3 + 3 ( μ 2 + 2 μ − 2 ) B H 2 + ( 3 μ 2 − 2 μ − 4 ) B H − ( 2 μ + 1 ) . P(B_H,\mu)=(1-\mu)^2(2\mu-1)B_H^4+(1-\mu)(\mu^2+6\mu-4)B_H^3+3(\mu^2+2\mu-2)B_H^2+(3\mu^2-2\mu-4)B_H-(2\mu+1). P ( B H , μ ) = ( 1 − μ ) 2 ( 2 μ − 1 ) B H 4 + ( 1 − μ ) ( μ 2 + 6 μ − 4 ) B H 3 + 3 ( μ 2 + 2 μ − 2 ) B H 2 + ( 3 μ 2 − 2 μ − 4 ) B H − ( 2 μ + 1 ) .
Since B H > 0 B_H>0 B H > 0 and 1 / 4 < μ < 1 1/4<\mu<1 1/4 < μ < 1 , the sign of Δ H \Delta_H Δ H coincides with the sign of P P P .
Let us first assume that 1 / 4 < μ ≤ 1 / 2 1/4<\mu\leq1/2 1/4 < μ ≤ 1/2 . Since all the coefficients here involved are non positive, it is easy to ascertain that P ( B H , μ ) < 0 P(B_H,\mu)<0 P ( B H , μ ) < 0 for any μ ∈ [ 1 / 4 , 1 / 2 ] \mu\in[1/4,1/2] μ ∈ [ 1/4 , 1/2 ] .
Then, to prove the assert, let us now assume that 1 / 2 < μ ≤ 1 1/2<\mu\leq1 1/2 < μ ≤ 1 . Since the equilibrium C E − \mathrm{CE}_{-} CE − belongs to the increasing branch of C ( B ) C(B) C ( B ) , one has C ′ ( B H ) > 0 C'(B_H)>0 C ′ ( B H ) > 0 so that 0 < ( 2 μ − 1 ) B H < 1 0<(2\mu-1)B_H<1 0 < ( 2 μ − 1 ) B H < 1 . By introducing the variables s : = 2 μ − 1 s:=2\mu-1 s := 2 μ − 1 and t : = s B H t:=sB_H t := s B H , it follows that 0 < s ≤ 1 0<s\leq1 0 < s ≤ 1 , 0 < t < 1 0<t<1 0 < t < 1 , μ = ( 1 + s ) / 2 \mu=(1+s)/2 μ = ( 1 + s ) /2 , and B H = t / s B_H=t/s B H = t / s . Then, defining
F ( s , t ) : = 16 s 4 P ( t s , 1 + s 2 ) , F(s,t):=16s^4P\left(\frac{t}{s},\frac{1+s}{2}\right), F ( s , t ) := 16 s 4 P ( s t , 2 1 + s ) ,
it can be proved that, since s > 0 s>0 s > 0 , the sign of P ( B H , μ ) P(B_H,\mu) P ( B H , μ ) is entirely determined by the sign of F ( s , t ) F(s,t) F ( s , t ) . Therefore, direct computations yield
F ( s , t ) = 4 s ( 1 − s ) 2 t 4 − 2 s ( s 3 + 13 s 2 − 17 s + 3 ) t 3 + 12 s 2 ( s 2 + 6 s − 3 ) t 2 + 4 s 3 ( 3 s 2 + 2 s − 17 ) t − 16 s 4 ( s + 2 ) . F(s,t)=4s(1-s)^2t^4-2s(s^3+13s^2-17s+3)t^3+12s^2(s^2+6s-3)t^2+4s^3(3s^2+2s-17)t-16s^4(s+2). F ( s , t ) = 4 s ( 1 − s ) 2 t 4 − 2 s ( s 3 + 13 s 2 − 17 s + 3 ) t 3 + 12 s 2 ( s 2 + 6 s − 3 ) t 2 + 4 s 3 ( 3 s 2 + 2 s − 17 ) t − 16 s 4 ( s + 2 ) .
Since F F F has degree five in s s s and degree four in t t t , it admits the tensor-product Bernstein representation [38 ]
F ( s , t ) = ∑ i = 0 5 ∑ j = 0 4 b i j T i 5 ( s ) T j 4 ( t ) , F(s,t)=\sum_{i=0}^{5}\sum_{j=0}^{4}b_{ij}T_i^5(s)T_j^4(t), F ( s , t ) = i = 0 ∑ 5 j = 0 ∑ 4 b ij T i 5 ( s ) T j 4 ( t ) ,
where
T i 5 ( s ) = ( 5 i ) s i ( 1 − s ) 5 − i , T j 4 ( t ) = ( 4 j ) t j ( 1 − t ) 4 − j . T_i^5(s)=\binom{5}{i}s^i(1-s)^{5-i},\qquad T_j^4(t)=\binom{4}{j}t^j(1-t)^{4-j}. T i 5 ( s ) = ( i 5 ) s i ( 1 − s ) 5 − i , T j 4 ( t ) = ( j 4 ) t j ( 1 − t ) 4 − j .
Therefore, the Bernstein coefficients are denoted by
[ b i j ] = [ 0 0 0 0 0 0 0 0 − 3 10 − 2 5 0 0 − 3 5 − 31 20 − 9 5 0 − 17 10 − 4 − 59 10 − 6 − 32 5 − 64 5 − 88 5 − 98 5 − 88 5 − 48 − 60 − 64 − 60 − 48 ] . [b_{ij}]=
\begin{bmatrix}
0 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & -\frac{3}{10} & -\frac{2}{5}\\
0 & 0 & -\frac{3}{5} & -\frac{31}{20} & -\frac{9}{5}\\
0 & -\frac{17}{10} & -4 & -\frac{59}{10} & -6\\
-\frac{32}{5} & -\frac{64}{5} & -\frac{88}{5} & -\frac{98}{5} & -\frac{88}{5}\\
-48 & -60 & -64 & -60 & -48
\end{bmatrix}. [ b ij ] = 0 0 0 0 − 5 32 − 48 0 0 0 − 10 17 − 5 64 − 60 0 0 − 5 3 − 4 − 5 88 − 64 0 − 10 3 − 20 31 − 10 59 − 5 98 − 60 0 − 5 2 − 5 9 − 6 − 5 88 − 48 .
Since the Bernstein bases T i 5 ( s ) > 0 T_i^5(s)>0 T i 5 ( s ) > 0 and T j 4 ( t ) > 0 T_j^4(t)>0 T j 4 ( t ) > 0 for every ( s , t ) ∈ ] 0 , 1 [ × ] 0 , 1 [ (s,t)\in]0,1[\times]0,1[ ( s , t ) ∈ ] 0 , 1 [ × ] 0 , 1 [ , the sign of F ( s , t ) F(s,t) F ( s , t ) is entirely determined by the Bernstein coefficients. In our case, they satisfy b i j ≤ 0 b_{ij}\leq0 b ij ≤ 0 where some of them are strictly negative, e.g. b 40 = − 32 / 5 < 0 b_{40}=-32/5<0 b 40 = − 32/5 < 0 , so that it follows that F ( s , t ) < 0 F(s,t)<0 F ( s , t ) < 0 on ] 0 , 1 ] × ] 0 , 1 [ ]0,1]\times]0,1[ ] 0 , 1 ] × ] 0 , 1 [ and, consequently, P ( B H , μ ) < 0 P(B_H,\mu)<0 P ( B H , μ ) < 0 , leading to l 1 ( θ H ) < 0 l_1(\theta_H)<0 l 1 ( θ H ) < 0 .