betaFeedback

WorksPhysics and Astronomy

Emergent stigmergy and pheromone-guided decisions in a foraging collective

Abstract

Ants in a colony forage and recruit other nest-mates for cooperative transport of food through stigmergy. Traditional approach to capture the foraging dynamics relies on fine-tuning phenomenological rules for pheromone deposition, chemotaxis, evaporation, and behavioral switching after reaching the food source. Here, we show that stigmergy emerges from an infinite-horizon stochastic optimal control problem involving outbound foragers traveling from the nest to the food source and returning agents traveling from the food source to the nest. Pheromone dynamics along a trail emerge from the forward propagating Hamilton–Jacobi–Bellman (HJB) equation, with the Bellman discount rate setting the rate of pheromone evaporation. Using this mapping, we identify the conditions under which trail feedback suppresses exploration or leads to the exploitation of established trails. For ants exploring multiple trails, we also find flux-potential relations, analogous to Ohm's law, with ant flux and inverse of squared value field playing the roles of current and resistance on the network. Our model captures collective behavior observed in the classic double-bridge experiments: trail selection on identical routes through a symmetry-breaking instability, selection of the shorter bridge, and bistability in the trail-length ratio that can leave ants transiently committed to the longer route. Our formulation places collective foraging in the broader framework of stochastic optimal control, while simultaneously providing an unifying picture on their emergent behavior consistent with experimental observations.

Introduction—Ant colonies discover and exploit foraging trails even though individuals hold only local information. They achieve this via stigmergy [1, 2, 3], the process by which successful foragers lay pheromone to recruit nestmates through chemo-sensing. The positive feedback set by returning ants depositing a volatile pheromone biases the motion of the outbound ants up the gradient, while the evaporation discards stale information [4, 5, 6, 7]. The collective leverages the type of pheromone and the deposition rate to select a worthwhile trail to successfully transport food to the nest. Classical controlled experiments [8, 9, 10, 11] on bridges of different lengths between food and nest (which force the collective to choose a trail) to understand the decision making process point to: (a) no preference in the trail but commitment to a single trail even when the trails are of the same length, (b) preference towards the shortest trail, given an option between shorter and longer one, (c) trapping in the long trail when a shorter route is made available after commitment.

Although phenomenological approaches remain the dominant paradigm for modeling the dynamics of such search and exploitation problems, navigation strategies have also been derived as the solution of a search game [12, 13, 14, 15], and recently reinforcement-learning formulations of collective chemotaxis have reproduced Keller–Segel dynamics [16, 17, 18]. The technical device in such treatments involves the Cole–Hopf transformation, which linearizes the Hamilton–Jacobi–Bellman (HJB) equation of a quadratically regularized control problem [19, 20, 21]. In this work, we pose the collective foraging task as an infinite-horizon optimal control problem, where the foragers rely on information carried by returners with a reward for reaching the food source and an attractive homing potential that guides them towards the nest [22, 23]. Our model on single trail shows: (i) emergence of stigmergy with the chemotaxis of foragers (Cole–Hopf transformed value field, sometimes referred as the desirability, acting as the pheromone) and pheromone laying behavior by returners, (ii) transition from camping phase (with ants confined to the nest) to foraging phase (successfully reaching food source) with the phase boundary determined by the quality of food source and the colony parameters, (iii) an effective Ohm’s law for the foragers and returners with the current and resistance determined by the ant density and the value field along the trail. In the double-bridge setting with trails marked by both foragers and returners, our model captures: (i) a symmetry breaking instability in identical trails in which ants commit to a particular trail – driven by negative differential resistance, (ii) selection of the shorter trail over longer one – owing to lower resistance, (iii) bistability in the commitment when the trail length ratio is not unity – consistent with experimental observations [8, 9, 11].

Model—Ants move in Ω⊆R2\Omega\subseteq\mathbb{R}^{2}, which contains a nest region ΩN\Omega_N and a food region ΩF\Omega_F (see Figure 1(a) for a schematic). Each ant stays in a behavioral state St∈{+,−}S_t \in\{+,-\}, where ++ denotes an outbound foraging ant traveling from nest towards food and −- a returning one, so that the full state is the Markov process Yt=(Xt,St)Y_t=(X_t,S_t) with

Figure 1. Single trail foraging and emergence of Ohm’s law. (a) Foraging ants X+(t)X_{+}(t) walk from the nest to the food, where they switch to returners X−(t)X_{-}(t) at rate Γ+\Gamma_{+}; returners switch back at the nest at rate Γ−\Gamma_{-} and lay the trail that onward ants follow. (b) Phase diagram of foraging on a single trail: Contour plot showing the smallest outbound drift min⁡x~w′\min_{\tilde{x}} w' in the plane of colony size QQ and scaled diffusion ε\varepsilon. The smallest drift vanishes in the foraging phase, where ww rises monotonically to the food, and is negative once ww develops an interior maximum (camping). Solid line: camping onset w′(1)=0w'(1)=0; dashed line: equality in the necessary condition Eq. (12), Θ=ln⁡Z(xF)\Theta=\ln Z(x_F); markers: the two steady-states shown in (c). See SM Video 1 for the dynamics in foraging and camping phase. (c) Value fields w=ln⁡Zw=\ln Z and v=ln⁡Ψv=\ln\Psi (top) and densities per ant ϱ±/Q\varrho_{\pm}/Q (bottom) at ε=0.05\varepsilon=0.05, in the foraging (Q=5Q=5, solid) and camping (Q=45Q=45, dashed) phases. The maximum density of returners is at the boundary in the foraging phase and in the bulk in the camping phase. (d) Circulation per ant C/QC/Q against QQ for three values of ε\varepsilon; dots mark the camping onset. (e) Ohm’s law: nest potential φ+(0)=CR+\varphi_{+}(0)=C R^{+} and food potential φ−(1)=CR−\varphi_{-}(1)=C R^{-}, Eqs. (14)–(15), against the circulation CC as QQ is swept at ε=0.05\varepsilon=0.05. Dotted lines: Ohmic response with the zero-current resistances; dots: camping onset. Both potentials pass through a maximum, beyond which the trail carries more current at a lower potential. (f) Potential profiles φ+/C=∫x~1e−2w\varphi_{+}/C=\int_{\tilde{x}}^{1} e^{-2w} (solid) and φ−/C=∫0x~e−2v\varphi_{-}/C=\int_{0}^{\tilde{x}} e^{-2v} (dashed) at ε=0.05\varepsilon=0.05 for Q=0.3,1,3,10,30Q=0.3, 1, 3, 10, 30; arrows point towards increasing QQ. All the results are at gF=6g_F=6, gN=2g_N=2 and κ~=4\tilde{\kappa}=4 (see SM Secs. SVIII, SX for details).

dXt=u±(Xt,t) dt+2D dWt.(1)\mathrm{d}X_t=u_{\pm}(X_t,t)\,\mathrm{d}t+\sqrt{2D}\,\mathrm{d}W_t. \tag*{(1)}

Ants can switch between states through positiondependent jump process, P(St+dt=∓∣St=±,Xt=x)=Γ±(x) dt\mathbb{P}(S_{t+dt}=\mp\mid S_t=\pm,X_t=x)=\Gamma_{\pm}(x)\,dt, with Γ+\Gamma_{+} defined near ΩF\Omega_F and Γ−\Gamma_{-} near ΩN\Omega_N (shown schematically in Figure 1(a)). We simplify the model by assuming the diffusivity DD and the mobility β\beta to be the same in both states. Here β\beta sets the cost of control through the running cost ∣u∣2/2β|u|^2/2\beta, so large β\beta means cheap control.

Each agent minimises a discounted cost up to its next switching time, taking the opposite value function as the terminal payoff. The value field V±(x,t)V^{\pm}(x,t) is the minimum expected cost for an ant in state ±\pm at position xx and time $t, and it obeys the coupled Bellman equations,

V+=min⁡u+E[∫tτ+ds e−λ(s−t)(∣u+∣22β−αϱ−)+e−λ(τ+−t)(V−−rF)](2)V^{+}=\min_{u_{+}}\mathbb{E}\left[\int_{t}^{\tau_{+}} ds\,e^{-\lambda(s-t)}\left(\frac{|u_{+}|^{2}}{2\beta}-\alpha\varrho_{-}\right)+e^{-\lambda(\tau_{+}-t)}(V^{-}-r_F)\right] \tag*{(2)}
V−=min⁡u−E[∫tτ−ds e−λ(s−t)(∣u−∣22β+U)+e−λ(τ−−t)(V+−rN)].(3)V^{-}=\min_{u_{-}}\mathbb{E}\left[\int_{t}^{\tau_{-}} ds\,e^{-\lambda(s-t)}\left(\frac{|u_{-}|^{2}}{2\beta}+U\right)+e^{-\lambda(\tau_{-}-t)}(V^{+}-r_N)\right]. \tag*{(3)}

Here ϱ−\varrho_{-} is the density of ants returning to the nest from the food source, and the value field for the foragers consists of a positive running reward at rate αϱ−\alpha\varrho_{-} for being where the returners are and also collecting rFr_F when reaching food. We assume a simple attractive homing potential U(x)=κ∣x∣2/2U(\mathbf{x})=\kappa|\mathbf{x}|^2/2, with κ\kappa being the stiffness of the potential, that guide the returners towards the nest and they also collect reward rNr_N at the nest. It is important to note that no pheromone appears in the specification of the problem and the reward structure is one-way, in that returners are followed but outbound ants are not.

Coupled HJB equations and the sensing law—Applying Bellman’s principle across the switching event (see SM Sec. SIII) gives the coupled system

∂tV±+D∇2V±−β2∣∇V±∣2−λV±+f±+Γ±(V∓−V±−δ±)=0,(4)\partial_t V^{\pm}+D\nabla^2V^{\pm}-\frac{\beta}{2}\lvert\nabla V^{\pm}\rvert^2-\lambda V^{\pm}+f_{\pm}+\Gamma_{\pm}\left(V^{\mp}-V^{\pm}-\delta_{\pm}\right)=0, \tag*{(4)}

with running costs f+=−αϱ−f_+=-\alpha\varrho_- and f−=Uf_-=U, conversion rewards δ+=rF\delta_+=r_F and δ−=rN\delta_-=r_N, and optimal feedback u±∗=−β∇V±\mathbf{u}^{*}_{\pm}=-\beta\nabla V^{\pm}. The Cole–Hopf map

V+=−2Dβln⁡Z,V−=−2Dβln⁡Ψ,(5)V^+=-\frac{2D}{\beta}\ln Z,\qquad V^-=-\frac{2D}{\beta}\ln\Psi, \tag*{(5)}

cancels the quadratic gradient term identically and turns the feedback into

u+∗=2D∇ln⁡Z,u−∗=2D∇ln⁡Ψ.(6)\mathbf{u}^{*}_{+}=2D\nabla\ln Z,\qquad\mathbf{u}^{*}_{-}=2D\nabla\ln\Psi. \tag*{(6)}

The optimal action an agent takes is to climb the logarithm of the field with drift velocity 2D∇Z/Z2D\nabla Z/Z (which is nothing but Weber–Fechner chemosensing [24, 25]). Substituting Eq. (5) into Eq. (4) (derived in SM Sec. SIV) gives the exact transformed dynamics for the value fields that guide the agents towards the food and the nest as

∂tZ=−(D∇2Z+αβ2Dϱ−Z−λZln⁡Z)−Γ+Z(−ln⁡Z+ln⁡Ψ+βrF2D),(7)\partial_t Z=-\left(D\nabla^2 Z+\frac{\alpha\beta}{2D}\varrho_-Z-\lambda Z\ln Z\right)-\Gamma_+Z\left(-\ln Z+\ln\Psi+\frac{\beta r_F}{2D}\right), \tag*{(7)}
∂tΨ=−(D∇2Ψ−β2DUΨ−λΨln⁡Ψ)−Γ−Ψ(−ln⁡Ψ+ln⁡Z+βrN2D).(8)\partial_t\Psi=-\left(D\nabla^2\Psi-\frac{\beta}{2D}U\Psi-\lambda\Psi\ln\Psi\right)-\Gamma_-\Psi\left(-\ln\Psi+\ln Z+\frac{\beta r_N}{2D}\right). \tag*{(8)}

Collective dynamics—The stochastic dynamics of foragers and returners in Eq. (1) can be converted to collective dynamics in the form of a density evolution equation (derived in SM Sec. SV)

∂tϱ±+2D∇⋅(ϱ±∇ln⁡χ±)=D∇2ϱ±∓Γ+ϱ+±Γ−ϱ−,(9)\partial_t\varrho_{\pm}+2D\nabla\cdot\left(\varrho_{\pm}\nabla\ln\chi_{\pm}\right)=D\nabla^2\varrho_{\pm}\mp\Gamma_+\varrho_+\pm\Gamma_-\varrho_-, \tag*{(9)}

with χ+=Z\chi_+=Z, χ−=Ψ\chi_-=\Psi. Eqs. (7)–(9) form a closed forward–backward system: ϱ−\varrho_- sources ZZ, and ∇ln⁡Z\nabla\ln Z transports ϱ+\varrho_+.

We see that wherever the outbound current J+=2Dϱ+∇ln⁡Z−D∇ϱ+J_+=2D\varrho_+\nabla\ln Z-D\nabla\varrho_+ vanishes, integration gives

ϱ+∝Z2=e−βV+/D,(10)\varrho_+\propto Z^2=e^{-\beta V^+/D}, \tag*{(10)}

References

  1. [1]P.-P. Grassé, Insectes Sociaux 6, 41 (1959).
  2. [2]G. Theraulaz and E. Bonabeau, Artificial Life 5, 97 (1999).
  3. [3]F. Heylighen, Cognitive Systems Research 38, 4 (2016).
  4. [4]Dorigo, M., & Stützle, T. (2004). Ant Colony Optimization.
  5. [5]E. Boissard, P. Degond, and S. Motsch, Journal of Mathematical Biology 66, 1267 (2013).
  6. [6]P. Amorim, Journal of Theoretical Biology 385, 160 (2015).
  7. [7]A. Perna, B. Granovskiy, S. Garnier, S. C. Nicolis, M. Labédan, G. Theraulaz, V. Fourcassié, and D. J. T. Sumpter, PLoS Computational Biology 8, e1002592 (2012).
  8. [8]W. Hangartner, Insectes Sociaux 16, 55 (1969).
  9. [9]S. Goss, S. Aron, J. L. Deneubourg, and J. M. Pasteels, Naturwissenschaften 76, 579 (1989).
  10. [10]D. J. T. Sumpter and M. Beekman, Animal Behaviour 66, 273 (2003).
  11. [11]J. L. Deneubourg, S. Aron, S. Goss, and J. M. Pasteels, Journal of Insect Behavior 3, 159 (1990).
  12. [12]V. Krishnan and L. Mahadevan, arXiv preprint arXiv:2601.04111 (2026).arXiv
  13. [13]M. Vergassola, E. Villermaux, and B. I. Shraiman, Nature 445, 406 (2007).
  14. [14]R. A. Heinonen, L. Biferale, A. Celani, and M. Vergassola, Physical Review E 107, 055105 (2023).
  15. [15]A. Pezzotta, M. Adorisio, and A. Celani, Physical Review E 98, 042401 (2018).
  16. [16]M. Kato and T. J. Kobayashi, PRX Life 3, 043005 (2025).
  17. [17]E. F. Keller and L. A. Segel, Journal of Theoretical Biology 26, 399 (1970).
  18. [18]S. G. Prasath, S. Mandal, F. Giardina, J. Kennedy, V. N. Murphy, and L. Mahadevan, Elife 11, e79638 (2022).
  19. [19]H. J. Kappen, Physical Review Letters 95, 202001 (2005).
  20. [20]E. Todorov, Proceedings of the National Academy of Sciences USA 106, 11478 (2009).
  21. [21]W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, 2nd ed., Stochastic Modelling and Applied Probability, Vol. 25 (Springer, New York, 2006).DOI
  22. [22]R. Wehner, Journal of Comparative Physiology A 189, 579 (2003).
  23. [23]R. Wehner, Desert Navigator: The Journey of an Ant (Belknap Press of Harvard University Press, Cambridge, MA, 2020).
  24. [24]Y. V. Kalinin, L. Jiang, Y. Tu, and M. Wu, Biophysical Journal 96, 2439 (2009).
  25. [25]M. D. Lazova, T. Ahmed, D. Bellomo, R. Stocker, and T. S. Shimizu, Proceedings of the National Academy of Sciences USA 108, 13870 (2011).
  26. [26]Joedel, E. J., & Oldeman, B. E. (2012). AUTO-07P: Continuation and Bifurcation Software for Ordinary Differential Equations.
  27. [27]R. Beckers, J. L. Deneubourg, and S. Goss, Journal of Theoretical Biology 159, 397 (1992).

Paper details

vlnc:W_kgcrhhuv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Claude Opus (multiple versions)

View statement
Looking for related papers…
Contents