betaFeedback

WorksPhysics and Astronomy

Optimal Control of Chemical Reaction Networks with Thermodynamic Cost

Abstract

Thermodynamic control seeks to realize desired functions at minimal thermodynamic cost, yet existing frameworks remain largely confined to slow driving regimes or state-to-state transport. Here we establish a thermodynamic optimal-control theory for Markov jump processes (MJPs) and chemical reaction networks (CRNs) that accommodates objectives over both states and reaction fluxes beyond slow-driving limit. By deriving the thermodynamic Hamilton–Jacobi–Bellman equation, we demonstrate that the optimal dynamics take the form of a perturbed generalized-gradient flow driven by the value function. A convex representation of the entropy-production rate via generalized means identifies its Legendre–Fenchel dual as the dissipation function. The resulting dynamics shares the thermodynamic structure of CRNs and admits an information-geometric characterization. We further derive a thermodynamic Pontryagin equation by recasting the control problem in terms of thermodynamic forces, providing a complementary description of optimal trajectories and clarifying connections to thermodynamic optimal transport. Our results establish a thermodynamic and geometric foundation for controlling MJPs and CRNs beyond mere transport, opening avenues for optimizing biological processes and finite-time thermodynamic operations in far-from-equilibrium regimes.

Introduction

Challenge in Thermodynamics

A central challenge in thermodynamics is designing protocols that drive systems to perform desired functions at minimal thermodynamic cost. This challenge spans diverse applications, from engineering efficient heat engines to elucidating the design principles of biochemical reaction networks and the fundamental physical limits of computation [1, 2, 3, 4]. Recent advances in stochastic thermodynamics [5, 6, 7, 8] have overcome the limitations of classical finite-time thermodynamics [9, 10] by providing a framework for quantifying work, heat, and entropy production under externally driven operations at both the distribution and trajectory levels.1

Building on this theoretical foundation, extensive efforts have been dedicated to characterizing thermodynamic bounds and identifying optimal protocols for canonical tasks, such as trajectory tracking and state-to-state transitions, under various constraints and assumptions [11, 12, 13].

Near-equilibrium thermodynamic control and thermodynamic geometry

In the near-equilibrium regime where linear response theory holds, slowly driven finite-time protocols that minimize dissipation correspond to geodesics on a Riemannian manifold endowed with a generalized friction metric [14]. This geometric formulation casts the search for optimal, minimum-dissipation protocols into a problem governed by the thermodynamic metric [15, 16].

Since its inception[17, 18, 19] via the integration of finite-time thermodynamics[20, 21, 22, 23, 24] and Weinhold–Ruppeiner equilibrium geometry[25, 26], the geometric approach has been refined theoretically [27, 28, 29] and applied to a diverse range of model systems [12, 30]. Moreover, leveraging the Hatano–Sasa excess heat, the scope of the framework has been expanded from a near-equilibrium tracking problem to transitions between non-equilibrium steady states [31], evolving into non-equilibrium quasi-static control [32, 33, 34, 35].

Additionally, shortcuts to adiabaticity or to isothermality have been developed, which realize a target sequence of equilibrium or steady states in finite time by introducing auxiliary controls [36, 37, 38, 39].

Far from equilibrium transport and its extensions

While powerful, these frameworks are restricted to near-equilibrium or steady-state regimes. Two pioneering studies explored possible ways to resolve this difficulty[40, 41]; Notably, the work by Aurell et al., which showed that finding a transport protocol with minimizing excess work and heat can be mapped to an optimal transport problem [40, 42], established a theoretical foundation for far-from-equilibrium transport. After a gap of about a decade since its proposal, theoretical and experimental efforts have rapidly accumulated in recent years, particularly for overdamped Langevin systems [43, 44, 45, 46, 47, 48, 49, 50, 51]. These insights have subsequently been extended to discrete Markov Jump Processes (MJPs) and chemical reaction networks (CRNs)[52, 53, 54, 55, 56]. Moreover, optimal transport offers a Wasserstein-geometric perspective that allows the derivation of various bounds in general thermodynamic settings.

Nevertheless, existing approaches remain limited to simple transport problems (Figure 1 (a)). In practical thermodynamic control, however, one often faces demands that extend far beyond mere transport. Specifically, maximizing target transitions and fluxes (Figure 1 (b))[57, 58, 59] is essential for optimizing work extraction[60], reaction fluxes[61, 62], and free energy transduction[63, 64]. Furthermore, confining a system to a prescribed region is often crucial to the stability of thermodynamic systems (Figure 1 (c))[64, 65]. Even though such problems, especially flux optimization with minimal costs, have been addressed in the context of biochemical machines and networks operating at steady state, their integration with transport phenomena is yet to be achieved.

Figure 1. Outline of thermodynamic control theory for MJPs and CRNs. (a) The primary task of thermodynamic control is to drive the state of the system from xinit\mathbf{x}_{init} at t=0t = 0 to xend\mathbf{x}_{end} at t=Tt = T. Here, the state represents the probability distribution over discrete states for MJPs and the concentration of molecular species for CRNs. (b) The induction of specific transitions (MJPs) or reaction fluxes (CRNs) is indispensable for converting energy into functions. (c) In practical situations, additional constraints on desirable or undesirable states must be accounted for when designing control. (d) Optimal control theory accommodates all these tasks and requirements, along with the thermodynamic cost of control (entropy production or heat dissipation), into a single objective function and minimizes it to obtain the optimal control. (e) The thermodynamic Hamilton-Jacobi-Bellman (HJB) equation characterizes the value of control via the value function, the gradient of which induces the perturbed generalized gradient flow dynamics as the optimally controlled dynamics. (f) The thermodynamic Pontryagin’s equation characterizes the optimally controlled Hamiltonian dynamics, which is equivalent to the dynamics obtained by the thermodynamic HJB equation. Here, the equation for F=const.F = \mathrm{const.} is shown for notational simplicity. D(f):=−diag⁡[2−ψm(f)ψm′′(f)(ψm′(f))2]−1D(f) := -\operatorname{diag}\left[2 - \frac{\psi_m(f)\psi''_m(f)}{(\psi'_m(f))^2}\right]^{-1}.

Optimal Control and Thermodynamics

Optimal control theory, which generalizes variational calculus, offers a natural mathematical setting for treating such diverse requirements without assuming slow driving[13, 66, 67, 68].

Optimal control theory incorporates a variety of requirements and constraints in the form of utilities and costs into a single objective functional and provides two principled ways to solve the resulting functional optimization problems. On the one hand, the Hamilton–Jacobi–Bellman (HJB) equation, grounded in Bellman’s principle of optimality, provides the state-time global landscape of desirability that the optimal trajectory should follow. On the other hand, the Hamiltonian dynamics derived from Pontryagin’s maximum principle can directly generate the state-time local individual optimal trajectory and dynamic driving control without the knowledge of the global desirability landscape[12]. These two together provide a coherent view of the global and local aspects of the optimal control problem.

Yet, despite their solid and profound mathematical basis, in the context of thermodynamic control, variational and optimal-control methods have largely been used as mere technical tools to derive problem-specific optimal protocols for individual setups[41, 69, 70, 71, 72, 73, 74, 75], rather than as a theoretical foundation for thermodynamic control comparable to thermodynamic geometry or optimal transport[76].

In addition, its application has been heavily skewed toward continuous diffusion processes described by the Fokker-Planck equation, since existing optimal control theory is predominantly built on such processes. Extensions to discrete-state MJPs and CRNs, which possess mathematical structures distinct from their continuous counterpart, remain limited. Indeed, recent studies on thermodynamic control of CRNs are restricted to the slow-driving limit [77, 78]. A notable exception is the work of Muratore-Ginanneschi et al., which treated thermodynamic transport in MJPs within an optimal-control formulation [79, 80]. However, their approach relies on the coordinate transformation from the probability variable pp to a wave-function-like variable ϕ=p\phi= \sqrt{p} to employ the optimal control techniques for quantum-mechanical systems. This construction differs substantially from the standard optimal-control formulation, rendering it non-trivial to extend it to other contexts, such as CRNs.

Outline of this work

In this work, we fill this gap by establishing a thermodynamic optimal-control theory that is consistent with the thermodynamic and geometric structures of MJPs and CRNs. We consider a class of control problems in which reaction rate constants serve as control variables [81]. In addition to an entropy-production-based control cost, we introduce state and flux utility functions into the objective functional (Figure 1 (d)). This formulation allows one to treat not only finite-time state transport but also a broader class of control tasks where the occupation of desired states and induction of specified reaction fluxes are balanced against the thermodynamic control cost.

By deriving the thermodynamic version of the Hamilton–Jacobi-Bellman (HJB) equation (Figure 1 (e)), we show that the optimal dynamics is a perturbed generalized-gradient flow, with the value function as the driving potential. This result is obtained through a convex representation of the entropy-production rate (EPR) function based on generalized means, and the generalized gradient flow is induced by the convex conjugate of this EPR function as the dissipation function (a nonlinear generalization of the friction metric) [82]. The derived dynamics has the same thermodynamic structure as the equilibrium and nonequilibrium chemical reaction dynamics [83, 84, 85], which admits its information-geometric characterization [82].

We further derive a thermodynamic Pontryagin equation by reframing the Pontryagin’s Hamiltonian equation in terms of thermodynamic forces as control variables (Figure 1 (f)).

This equation manifests the structure specific to thermodynamic control and clarifies its relation to results from optimal transport and related approaches. We obtained analytic optimal control solutions for constant activity and a constrained activity budget, which unify the thermodynamic optimal transport solution and steady flux maximization. Our results establish a theoretical basis for thermodynamic control, with possible applications to biological reaction systems and finite-time thermodynamic operations.

Structure of this paper

The remainder of this paper is organized as follows. Section II reviews the thermodynamic structure of MJPs and CRNs and formulates the optimal control problem in a general setting. In § 0.3, we derive the thermodynamic HJB and Pontryagin equations by employing a convex representation of the EPR function; this formulation uncovers a perturbed generalized gradient flow structure along with the intrinsic information-geometric features of thermodynamic optimal control problems. Building on these results, Sec. IV presents analytical solutions that unify thermodynamic transport and steady-flux maximization. Finally, § 0.5 concludes the paper with a discussion of future directions.

Preliminary: Reversible deterministic chemical reaction network and its optimal control

Reversible chemical reaction networks

Let X:={X1,⋯ ,X∣X∣}\mathcal{X} := \{X_1, \cdots, X_{|\mathcal{X}|}\} be the molecular species involved in the system. To investigate thermodynamic control, we consider reversible CRN, each pair of forward and backward reactions are defined as

∑i=1∣X∣γi,r+Xi⇌∑i=1∣X∣γi,r−Xi(1)\sum_{i=1}^{|\mathcal{X}|} \gamma_{i,r}^{+} X_i \rightleftharpoons\sum_{i=1}^{|\mathcal{X}|} \gamma_{i,r}^{-} X_i \tag*{(1)}

where r∈[1,∣R∣]r \in[1, |\mathcal{R}|] is the index of reaction. Here, γi,r+\gamma_{i,r}^{+} and γi,r−\gamma_{i,r}^{-} denote the number of XiX_i molecules consumed and produced in the forward reaction (left to right), respectively. In the reverse reaction (right to left), these roles are inverted, with γi,r−\gamma_{i,r}^{-} and γi,r+\gamma_{i,r}^{+} representing the reactants and products. We define the forward and backward stoichiometric matrices

S±∈Z≥0∣X∣×∣R∣S^{\pm} \in\mathbb{Z}_{\geq0}^{|\mathcal{X}|\times|\mathcal{R}|} as Si,r±:=γi,r±S_{i,r}^{\pm} := \gamma_{i,r}^{\pm}. Then, the standard stoichiometric matrix is defined as S:=S−−S+∈Z∣X∣×∣R∣S := S^{-} - S^{+} \in\mathbb{Z}^{|\mathcal{X}|\times|\mathcal{R}|}.

The deterministic CRN dynamics for the concentration vector x∈R≥0∣X∣\boldsymbol{x} \in\mathbb{R}_{\geq0}^{|\mathcal{X}|} of molecular species are described by the chemical rate equation (CRE):

dxdt=S[j+(t,x)−j−(t,x)]=Sj(t,x)(2)\frac{\mathrm{d}\boldsymbol{x}}{\mathrm{d}t} = S\left[\boldsymbol{j}^{+}(t,\boldsymbol{x})-\boldsymbol{j}^{-}(t,\boldsymbol{x})\right] = S\boldsymbol{j}(t,\boldsymbol{x}) \tag*{(2)}

where j±(t,x)∈R≥0∣R∣\boldsymbol{j}^{\pm}(t,\boldsymbol{x}) \in\mathbb{R}_{\geq0}^{|\mathcal{R}|} are forward and reverse reaction fluxes and their difference j(t,x):=j+(t,x)−j−(t,x)\boldsymbol{j}(t,\boldsymbol{x}) := \boldsymbol{j}^{+}(t,\boldsymbol{x})-\boldsymbol{j}^{-}(t,\boldsymbol{x}) is the (total) thermodynamic flux. In this work, we consider the case that the trajectory is confined within x∈R>0∣X∣\boldsymbol{x} \in\mathbb{R}_{>0}^{|\mathcal{X}|}. Given the local detailed balance assumption [86, 87], thermodynamic force f\boldsymbol{f} is defined by

f(t,x):=ln⁡j+(t,x)−ln⁡j−(t,x).(3)\boldsymbol{f}(t,\boldsymbol{x}) := \ln\boldsymbol{j}^{+}(t,\boldsymbol{x})-\ln\boldsymbol{j}^{-}(t,\boldsymbol{x}). \tag*{(3)}

The total entropy production rate (EPR) at time tt and state x\boldsymbol{x} is obtained by

Σ˙(t,x)RΩ=⟨j(t,x),f(t,x)⟩=∑r=1∣R∣(jr+(t,x)−jr−(t,x))ln⁡jr+(t,x)jr−(t,x)(4)\frac{\dot{\Sigma}(t,\boldsymbol{x})}{R\Omega} = \langle\boldsymbol{j}(t,\boldsymbol{x}),\boldsymbol{f}(t,\boldsymbol{x})\rangle= \sum_{r=1}^{|\mathcal{R}|}\left(j_{r}^{+}(t,\boldsymbol{x})-j_{r}^{-}(t,\boldsymbol{x})\right)\ln\frac{j_{r}^{+}(t,\boldsymbol{x})}{j_{r}^{-}(t,\boldsymbol{x})} \tag*{(4)}
=DKL[j+(t,x)∥j−(t,x)]+DKL[j−(t,x)∥j+(t,x)]≥0(5)= D_{KL}[\boldsymbol{j}^{+}(t,\boldsymbol{x})\Vert\boldsymbol{j}^{-}(t,\boldsymbol{x})] + D_{KL}[\boldsymbol{j}^{-}(t,\boldsymbol{x})\Vert\boldsymbol{j}^{+}(t,\boldsymbol{x})] \geq0 \tag*{(5)}

where RR is the gas constant, Ω\Omega is the volume, ⟨f,j⟩:=∑r=1∣R∣frjr\langle\boldsymbol{f},\boldsymbol{j}\rangle:= \sum_{r=1}^{|\mathcal{R}|} f_rj_r is the bilinear pairing, and DKL[j∥j′]:=∑r=1∣R∣[jrln⁡jrjr′−(jr−jr′)]D_{KL}[\boldsymbol{j}\Vert\boldsymbol{j}'] := \sum_{r=1}^{|\mathcal{R}|}\left[j_r\ln\frac{j_r}{j'_r}-(j_r-j'_r)\right] is the Kullbuck-Leibler divergence for non-negative densities. In the following, we set RΩ=1R\Omega=1 for simplicity.

Mass Action Kinetics

While the definitions of thermodynamic force and EPR are independent of the functional forms of the fluxes with respect to x\boldsymbol{x}, specific kinetics for the forward and backward fluxes are required to instantiate the general theory that we derive. Mass action kinetics (MA kinetics) is the representative one. For the reversible CRN(Eq. (1)), their forward and backward flux functions under MA kinetics are

jr±(x):=kr±x(γr±)T=kr±∏i=1∣X∣xiγi,r±(6)j_r^{\pm}(\boldsymbol{x}) := k_r^{\pm}\boldsymbol{x}^{(\boldsymbol{\gamma}_r^{\pm})^{\mathsf{T}}} = k_r^{\pm}\prod_{i=1}^{|\mathcal{X}|}x_i^{\gamma_{i,r}^{\pm}} \tag*{(6)}

where k±:=(k1±,⋯ ,k∣R∣±)⊤∈R≥0∣R∣\mathbf{k}^{\pm}:=(k_1^{\pm},\cdots,k_{|\mathcal{R}|}^{\pm})^{\top}\in\mathbb{R}_{\geq0}^{|\mathcal{R}|} are the kinetic rate constants and x(γr)⊤\mathbf{x}^{(\gamma_r)^{\top}} is a short-hand notation of ∏i=1∣X∣xiγi,r\prod_{i=1}^{|\mathcal{X}|}x_i^{\gamma_{i,r}} (see also the Appendix for the detailed setup.). In addition, if all reactions are monomolecular and thus their fluxes are linear, the CRE (Eq. (2)) is reduced to the master equation for MJP on a graph. The flux functions are reduced to

jr±(x):=kr±x(br±)⊤=kr±∏i=1∣X∣xibi,r±=(∗)kr±[∑i=1∣X∣bi,r±xi]=kr±(br±)⊤x.(7)j_r^{\pm}(\mathbf{x}):=k_r^{\pm}\mathbf{x}^{(\mathbf{b}_r^{\pm})^{\top}}=k_r^{\pm}\prod_{i=1}^{|\mathcal{X}|}x_i^{b_{i,r}^{\pm}}\overset{(*)}{=}k_r^{\pm}\left[\sum_{i=1}^{|\mathcal{X}|}b_{i,r}^{\pm}x_i\right]=k_r^{\pm}(\mathbf{b}_r^{\pm})^{\top}\mathbf{x}. \tag*{(7)}

where bi,r±∈{0,1}b_{i,r}^{\pm}\in\{0,1\} is 1 if the ith molecule is the reactant (product) of the rth reaction, respectively, and 0 otherwise. For B±:={bi,r}\mathbb{B}^{\pm}:=\{b_{i,r}\}, B=B+−B−∈{−1,0,+1}∣X∣×∣R∣\mathbb{B}=\mathbb{B}^{+}-\mathbb{B}^{-}\in\{-1,0,+1\}^{|\mathcal{X}|\times|\mathcal{R}|} is the graph incidence matrix for the MJP.3 RΩR\Omega in Eq. (5) is replaced with kBk\mathbb{B}.

Optimal control of deterministic CRNs

To introduce external control to a CRN, we decompose the forward and backward fluxes as

j±(t,x)=k±(t)∘h±(x),(8)\mathbf{j}^{\pm}(t,\mathbf{x})=\mathbf{k}^{\pm}(t)\circ\mathbf{h}^{\pm}(\mathbf{x}), \tag*{(8)}

where k±(t)≥0\mathbf{k}^{\pm}(t)\geq0 are the reaction rate constants and h±(x)≥0\mathbf{h}^{\pm}(\mathbf{x})\geq0 are the x\mathbf{x} dependence of the flux functions that satisfies h±(x)>0\mathbf{h}^{\pm}(\mathbf{x})>0 if x∈R>0∣X∣\mathbf{x}\in\mathbb{R}_{>0}^{|\mathcal{X}|}. We suppose that we can control the reaction rate constants k±(t)\mathbf{k}^{\pm}(t) independently. Under this setup, we consider the EPR Σ˙(x,{k±})\dot{\Sigma}(\mathbf{x},\{\mathbf{k}^{\pm}\}) as the cost for controlling the kinetic rate of a CRN to {k±}:=(k+,k−)\{\mathbf{k}^{\pm}\}:=(\mathbf{k}^{+},\mathbf{k}^{-}) at state x\mathbf{x}:

C(x,{k±}):=Σ˙(x,{k±}).(9)C(\mathbf{x},\{\mathbf{k}^{\pm}\}):=\dot{\Sigma}(\mathbf{x},\{\mathbf{k}^{\pm}\}). \tag*{(9)}

Then, we introduce the control cost functional over the trajectory x0:T:={xt∣t∈[0,T]}\mathbf{x}_{0:T}:=\{\mathbf{x}_t\mid t\in[0,T]\} and k0:T±:={kt±∣t∈[0,T]}\mathbf{k}_{0:T}^{\pm}:=\{\mathbf{k}_t^{\pm}\mid t\in[0,T]\} as

C±[x0:T,{k0:T±}]:=∫0TC(xt,{kt±})dt=∫0TΣ˙(xt,{kt±})dt.(10)\mathcal{C}^{\pm}[\mathbf{x}_{0:T},\{\mathbf{k}_{0:T}^{\pm}\}]:=\int_0^T C(\mathbf{x}_t,\{\mathbf{k}_t^{\pm}\})\mathrm{d}t=\int_0^T\dot{\Sigma}(\mathbf{x}_t,\{\mathbf{k}_t^{\pm}\})\mathrm{d}t. \tag*{(10)}

Conventional thermodynamic transport problems only account for this control cost. However, in various practical situations, we may have additional constraints and requirements for the operation of the thermodynamic system[57, 58, 59, 60, 61, 62, 63, 64, 65]. To integrate these requirements, we introduce the instantaneous utility function U(t,x)∈RU(t,\boldsymbol{x}) \in\mathbb{R}, the terminal utility function U^T(x)∈R\hat{U}_{T}(\boldsymbol{x}) \in\mathbb{R} and the flux utility function F±(t,x)=(F1±(t,x),…,F∣R∣±(t,x))T∈R∣R∣\mathbf{F}^{\pm}(t,\boldsymbol{x}) = (F_{1}^{\pm}(t,\boldsymbol{x}), \ldots, F_{|\mathcal{R}|}^{\pm}(t,\boldsymbol{x}))^{\mathsf{T}} \in\mathbb{R}^{|\mathcal{R}|}. The instantaneous utility function U(t,x)U(t,\boldsymbol{x}) specifies the preferred state of x\boldsymbol{x} at time tt (Figure 1 (c)), whereas the terminal utility determines the state where the system is encouraged to reach at the end of control. If the terminal state xend\boldsymbol{x}_{end} is specified, then U^T(x)=0\hat{U}_{T}(\boldsymbol{x}) = 0 if x=xend\boldsymbol{x} = \boldsymbol{x}_{end} and U^T(x)=−∞\hat{U}_{T}(\boldsymbol{x}) = -\infty, otherwise. The rrth component of the flux utilities, Fr+(t,x)F_{r}^{+}(t,\boldsymbol{x}) and Fr−(t,x)F_{r}^{-}(t,\boldsymbol{x}), define the desirability of the occurrence of the rrth forward and backward reactions, respectively (Figure 1 (b)). In this work, we assume that the flux utilities for each reversible reaction are anti-symmetric (reversible with respect to flux), i.e.: Fr+(t,x)=−Fr−(t,x)=:Fr(t,x)F_{r}^{+}(t,\boldsymbol{x}) = -F_{r}^{-}(t,\boldsymbol{x}) =: F_{r}(t,\boldsymbol{x}). For these utilities, we introduce their integral over the trajectory:

U[x0:T]:=∫0TU(t,x(t))dt+U^T(x(T))(11)\mathcal{U}[\boldsymbol{x}_{0:T}] := \int_{0}^{T} U(t,\boldsymbol{x}(t))\mathrm{d}t + \hat{U}_{T}(\boldsymbol{x}(T)) \tag*{(11)}
F[x0:T,{k0:T±}]:=∫0T∑s∈{−,+}∑rFrs(t,x(t))j˙rs(t)dt=∫0T⟨F(t,x(t)),j˙(t)⟩dt(12)\mathcal{F}[\boldsymbol{x}_{0:T},\{k_{0:T}^{\pm}\}] := \int_{0}^{T} \sum_{s\in\{-,+\}} \sum_{r} F_{r}^{s}(t,\boldsymbol{x}(t))\dot{j}_{r}^{s}(t)\mathrm{d}t = \int_{0}^{T} \langle\mathbf{F}(t,\boldsymbol{x}(t)),\dot{\boldsymbol{j}}(t)\rangle\mathrm{d}t \tag*{(12)}

where we used the anti-symmetry of flux utilities and abbreviate the dependence of the flux on {k±(t)}\{k^{\pm}(t)\} as j˙(t)=j˙(x(t);{k±(t)})\dot{\boldsymbol{j}}(t) = \dot{\boldsymbol{j}}(\boldsymbol{x}(t);\{k^{\pm}(t)\}).

Then, the optimal control problem for a finite horizon TT starting from xinit\boldsymbol{x}_{init} is defined as

V0(x0;β):=max⁡{k0:T±}[β1U[x0:T]+β2F[x0:T,{k0:T±}]−C[x0:T,{k0:T±}]∣x(0)=xinit]V_{0}(\boldsymbol{x}_{0};\boldsymbol{\beta}) := \max_{\{k_{0:T}^{\pm}\}} \left[\beta_{1}\mathcal{U}[\boldsymbol{x}_{0:T}] + \beta_{2}\mathcal{F}[\boldsymbol{x}_{0:T},\{k_{0:T}^{\pm}\}] - C[\boldsymbol{x}_{0:T},\{k_{0:T}^{\pm}\}] \mid\boldsymbol{x}(0) = \boldsymbol{x}_{init}\right]
s.t.dxdt=Sj˙(x(t);{k±(t)}),(13)\mathrm{s.t.}\qquad\frac{\mathrm{d}\boldsymbol{x}}{\mathrm{d}t} = \mathbf{S}\dot{\boldsymbol{j}}(\boldsymbol{x}(t);\{k^{\pm}(t)\}), \tag*{(13)}

where β=(β1,β2)\boldsymbol{\beta} = (\beta_{1},\beta_{2}) is the weighting parameters (Figure 1 (d)).

Bellman’s Principle of Optimality and HJB equation

A standard approach for solving this control problem is to follow Bellman Principle of Optimality, which decomposes the optimization problem over t′∈[t,T]t' \in[t,T] into one at tt and the sub-problem over t′∈[t+dt,T]t' \in[t+\mathrm{d}t,T]. To conduct this decomposition operationally, we introduce the value function:

Vt(x;β):=max⁡{kt:T±}[β1U[xt:T]+β2F[xt:T,{kt:T±}]−C[xt:T,{kt:T±}]∣x(t)=x](14)V_{t}(\boldsymbol{x};\boldsymbol{\beta}) := \max_{\{k_{t:T}^{\pm}\}} \left[\beta_{1}\mathcal{U}[\boldsymbol{x}_{t:T}] + \beta_{2}\mathcal{F}[\boldsymbol{x}_{t:T},\{k_{t:T}^{\pm}\}] - C[\boldsymbol{x}_{t:T},\{k_{t:T}^{\pm}\}] \mid\boldsymbol{x}(t) = \boldsymbol{x}\right] \tag*{(14)}

which satisfies the Hamilton Jacobi Bellman (HJB) equation:

−∂Vt(x;β)∂t=β1U(t,x)+max⁡{k±}∈R≥02∣R∣[⟨∂Vt(x;β)∂x,dxdt⟩+β2⟨F(t,x),j(x;{k±})⟩−C(x,{k±})],(15)-\frac{\partial V_t(x;\beta)}{\partial t}=\beta_1 U(t,x)+\max_{\{k^\pm\}\in\mathbb{R}_{\geq0}^{2|\mathcal{R}|}}\left[\left\langle\frac{\partial V_t(x;\beta)}{\partial x},\frac{\mathrm{d}x}{\mathrm{d}t}\right\rangle+\beta_2\langle\mathbf{F}(t,x),\mathbf{j}(x;\{k^\pm\})\rangle-C(x,\{k^\pm\})\right], \tag*{(15)}

where the terminal boundary condition is VT(x;β)=U^T(x)V_T(x;\beta)=\widehat{U}_T(x). In the HJB equation, the optimization over the trajectory is replaced by the optimization at tt given Vt(x;β)V_t(x;\beta). Thus, the optimal kt±k_t^\pm becomes

kt±(x;β):=arg⁡max⁡{k±}∈R≥02∣R∣[⟨∂Vt(x;β)∂x,dxdt⟩+β2⟨F(t,x),j(x;{k±})⟩−C(x,{k±})](16)k_t^\pm(x;\beta):=\arg\max_{\{k^\pm\}\in\mathbb{R}_{\geq0}^{2|\mathcal{R}|}}\left[\left\langle\frac{\partial V_t(x;\beta)}{\partial x},\frac{\mathrm{d}x}{\mathrm{d}t}\right\rangle+\beta_2\langle\mathbf{F}(t,x),\mathbf{j}(x;\{k^\pm\})\rangle-C(x,\{k^\pm\})\right] \tag*{(16)}
=arg⁡max⁡{k±}∈R≥02∣R∣[⟨(ST∂Vt(x;β)∂x+β2F(t,x)),j(x;{k±})⟩−C(x,{k±})](17)=\arg\max_{\{k^\pm\}\in\mathbb{R}_{\geq0}^{2|\mathcal{R}|}}\left[\left\langle\left(\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x;\beta)}{\partial x}+\beta_2\mathbf{F}(t,x)\right),\mathbf{j}(x;\{k^\pm\})\right\rangle-C(x,\{k^\pm\})\right] \tag*{(17)}

where we used dxdt=Sj(x;{k±})\frac{\mathrm{d}x}{\mathrm{d}t}=\mathbf{S}\mathbf{j}(x;\{k^\pm\}). If an explicit form of kt±k_t^\pm is provided, the value function can be obtained by solving

−∂Vt(x;β)∂t=β1U(t,x)+⟨(ST∂Vt(x;β)∂x+β2F(t,x)),j(x;{kt±(x)})⟩−C(x,{kt±(x)}),(18)-\frac{\partial V_t(x;\beta)}{\partial t}=\beta_1U(t,x)+\left\langle\left(\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x;\beta)}{\partial x}+\beta_2\mathbf{F}(t,x)\right),\mathbf{j}(x;\{k_t^\pm(x)\})\right\rangle-C(x,\{k_t^\pm(x)\}), \tag*{(18)}

in a time-backward manner from the terminal boundary condition VT(x;β)=U^T(x)V_T(x;\beta)=\widehat{U}_T(x).

Thermodynamic representations of optimal control for CRNs

Solving Eq. (17) explicitly is intractable for general control problems with nonlinearities in the cost and utility functions. If the cost function is strictly convex with respect to the control variable, the problem becomes manageable at least formally by using the Legendre transformation. However, the thermodynamic cost C(x,{kt±(x)})=Σ˙C(x,\{k_t^\pm(x)\})=\dot{\Sigma} is convex but not strict with respect to k±k^\pm due to a degeneracy4. We resolve this degeneracy by applying transformations based on generalized means[54], which decouples the degeneracy and the strict convexity.

Generalized mean and thermodynamic representation of flux

For two non-negative scalars a,b≥0a,b \ge0, let us consider a class of their generalized mean m[a,b]≥0\mathbf{m}[a,b] \ge0, which satisfies min⁡[a,b]≤m[a,b]≤max⁡[a,b]\min[a,b] \le\mathbf{m}[a,b] \le\max[a,b], equality m[a,a]=a\mathbf{m}[a,a] = a, symmetry m[a,b]=m[b,a]\mathbf{m}[a,b] = \mathbf{m}[b,a], and homogeneity m(λa,λb)=λm[a,b]\mathbf{m}(\lambda a,\lambda b) = \lambda\mathbf{m}[a,b] for any λ>0\lambda> 0. By defining m(r):=m[r,1]\mathbf{m}(r) := \mathbf{m}[r,1], we can represent m[a,b]=bm(a/b)\mathbf{m}[a,b] = b\mathbf{m}(a/b).5 With m(r)\mathbf{m}(r), the generalized average flux is introduced[54] as

ω:=m[j+,j−]∈R≥0∣R∣(19)\boldsymbol{\omega} := \mathbf{m}[\boldsymbol{j}^{+},\boldsymbol{j}^{-}] \in\mathbb{R}_{\ge0}^{|\mathcal{R}|} \tag*{(19)}

where ωr=m[jr+,jr−]\omega_r = \mathbf{m}[j_r^{+},j_r^{-}]. This quantity (up to constant scaling) is called “traffic” for the arithmetic mean[88], “Onsager coefficients” or “state mobility” for logarithmic mean[54, 88], and “frenetic activity” for geometric mean[85, 89]. Here we call it ”general activity” or simply ”activity” as in [54]. Then, the total flux can be represented as

j:=j+−j−=ω∘ψm(f)(20)\boldsymbol{j} := \boldsymbol{j}^{+} - \boldsymbol{j}^{-} = \boldsymbol{\omega} \circ\boldsymbol{\psi}_{\mathbf{m}}(\boldsymbol{f}) \tag*{(20)}

where ψm(f)=(ψm(f1),⋯ ,ψm(f∣R∣))T\boldsymbol{\psi}_{\mathbf{m}}(\boldsymbol{f}) = (\psi_{\mathbf{m}}(f_1),\cdots,\psi_{\mathbf{m}}(f_{|\mathcal{R}|}))^{\mathsf{T}} with

ψm(f):=ef−1m(ef)∈R.(21)\psi_{\mathbf{m}}(f) := \frac{e^f-1}{\mathbf{m}(e^f)} \in\mathbb{R}. \tag*{(21)}

The m\mathbf{m}-induced function ψm(f)\psi_{\mathbf{m}}(f) satisfies oddity ψm(f)=−ψm(−f)\psi_{\mathbf{m}}(f) = -\psi_{\mathbf{m}}(-f), which leads to ψm(f)=0\psi_{\mathbf{m}}(f) = 0 if f=0f = 0. Given ω\boldsymbol{\omega} and f≠0f \ne0, the corresponding j±\boldsymbol{j}^{\pm} are recovered as j+=ω∘ψm(f)1−exp⁡(−f)\boldsymbol{j}^{+} = \boldsymbol{\omega} \circ\frac{\psi_{\mathbf{m}}(f)}{1-\exp(-f)}, j−=ω∘ψm(f)exp⁡(f)−1\boldsymbol{j}^{-} = \boldsymbol{\omega} \circ\frac{\psi_{\mathbf{m}}(f)}{\exp(f)-1} for f≠0f \ne0, and j+=j−=ω\boldsymbol{j}^{+} = \boldsymbol{j}^{-} = \boldsymbol{\omega} for f=0f = 0.6 In this work, we consider only the differentiable generalized means that satisfy the two physical conditions

m′(r)+m′(1/r)>0andm′′(r)>−r+1r(r−1)2[m′(r)+m′(1/r)](22)\mathbf{m}'(r) + \mathbf{m}'(1/r) > 0 \qquad\text{and}\qquad \mathbf{m}''(r) > -\frac{r+1}{r(r-1)^2}\left[\mathbf{m}'(r)+\mathbf{m}'(1/r)\right] \tag*{(22)}

for r>1r > 1 as in [54]. Examples of m\mathbf{m} and ψm\psi_{\mathbf{m}} are listed in Table I, all of which satisfy these physical conditions.

Type of meanm[j+,j−]\mathrm{m}[j^{+},j^{-}]m(r)\mathrm{m}(r)ψm(f)\psi_{\mathrm{m}}(f)ψm−1(u)\psi_{\mathrm{m}}^{-1}(u)Domain DuD^{u}
Harmonic2j+j−j++j−\frac{2j^{+}j^{-}}{j^{+}+j^{-}}2rr+1\frac{2r}{r+1}sinh⁡(f)\sinh(f)sinh⁡−1(u)\sinh^{-1}(u)u∈(−∞,+∞)u \in(-\infty,+\infty)
Geometricj+j−\sqrt{j^{+}j^{-}}r\sqrt{r}2sinh⁡(f/2)2\sinh(f/2)2sinh⁡−1(u/2)2\sinh^{-1}(u/2)u∈(−∞,+∞)u \in(-\infty,+\infty)
Logarithmicj+−j−ln⁡j+−ln⁡j−\frac{j^{+}-j^{-}}{\ln j^{+}-\ln j^{-}}r−1ln⁡(r)\frac{r-1}{\ln(r)}ffuuu∈(−∞,+∞)u \in(-\infty,+\infty)
Arithmeticj++j−2\frac{j^{+}+j^{-}}{2}r+12\frac{r+1}{2}2tanh⁡(f/2)2\tanh(f/2)2tanh⁡−1(u/2)2\tanh^{-1}(u/2)u∈(−2,+2)u \in(-2,+2)
Contraharmonic(j+)2+(j−)2j++j−\frac{(j^{+})^{2}+(j^{-})^{2}}{j^{+}+j^{-}}r2+1r+1\frac{r^{2}+1}{r+1}tanh⁡(f)\tanh(f)tanh⁡−1(u)\tanh^{-1}(u)u∈(−1,+1)u \in(-1,+1)

Table I. A list of generalized means and the corresponding functional forms of ψm(f)\psi_{\mathrm{m}}(f) and ψm−1(u)\psi_{\mathrm{m}}^{-1}(u) with their domains for uu. The domain for ff is Df=(−∞,+∞)D^{f}=(-\infty,+\infty) for all means.

When the first physical condition in Eq. (22) is satisfied, u=ψm(f)u = \psi_{\mathbf{m}}(f) becomes strictly monotonous and thus invertible f=ψm−1(u)f = \psi_{\mathbf{m}}^{-1}(u). Then, give ω\boldsymbol{\omega} and j\boldsymbol{j}, the corresponding j±\boldsymbol{j}^{\pm} are recovered as j±=±j1−exp⁡(∓ψm−1(j/ω))\boldsymbol{j}^{\pm} = \pm\frac{\boldsymbol{j}}{1-\exp(\mp\psi_{\mathbf{m}}^{-1}(\boldsymbol{j}/\boldsymbol{\omega}))} if u=j/ωu = \boldsymbol{j}/\boldsymbol{\omega} is in the corresponding domain. Furthermore, by defining dissipation functions, Ψω(j)\Psi_{\omega}(j), Ψω∗(f)∈R≥0\Psi_{\omega}^{*}(f) \in\mathbb{R}_{\ge0} as

Ψω(j)=∑r=1∣R∣ωr∫0∣jr∣/ωrψm−1(u′) du′Ψω∗(f)=∑r=1∣R∣ωr∫0∣fr∣ψm(f′) df′,(23)\Psi_{\omega}(j)=\sum_{r=1}^{|\mathcal{R}|}\omega_{r}\int_{0}^{|j_{r}|/\omega_{r}}\psi_{\mathrm{m}}^{-1}(u')\,\mathrm{d}u' \qquad \Psi_{\omega}^{*}(f)=\sum_{r=1}^{|\mathcal{R}|}\omega_{r}\int_{0}^{|f_{r}|}\psi_{\mathrm{m}}(f')\,\mathrm{d}f', \tag*{(23)}

they are strictly convex, symmetric, and convex conjugate to each other. The thermodynamic force and flux are associated as the Legendre dual of these functions:

j=∂fΨω∗[f],f=∂jΨω[j],(24)j=\partial_{f}\Psi_{\omega}^{*}[f],\qquad f=\partial_{j}\Psi_{\omega}[j], \tag*{(24)}

where ∂f:=∂∂f\partial_{f}:=\frac{\partial}{\partial f} and ∂j:=∂∂j\partial_{j}:=\frac{\partial}{\partial j}. Owing to the Fenchel–Young inequality, EPR (Eq. (5)) is represented as

Σ˙=⟨j,f⟩=Ψω(j)+Ψω∗(f)(25)\dot{\Sigma}=\langle j,f\rangle=\Psi_{\omega}(j)+\Psi_{\omega}^{*}(f) \tag*{(25)}

Moreover, Eq. (2) with the MA kinetics is represented as the perturbed generalized gradient flow (Figure 2 (a))7:

Figure 2. The thermodynamic structure of autonomous CRNs (a) and the optimally controlled CRNs (b). Both dynamics are the perturbed generalized gradient flow. For the autonomous CRNs (a), the dynamics is induced by the thermodynamic force generated by the gradient of the thermodynamic potential function D[x∥xeq]\mathcal{D}[x\Vert x_{eq}] perturbed by the external nonequilibrium force fnef_{ne}. The flux jj is the Legendre transformation of the force ff via the dissipation potential Ψω(x)∗(f)\Psi_{\omega(x)}^{*}(f). For the optimally controlled CRNs (b), the dynamics is induced by the pseudo-force η\eta generated by the gradient of the value function Vt(x)V_t(x) perturbed by the flux utility F=F(t,x)F=F(t,x). The flux jj is the Legendre transformation of the pseudo-force via the Legendre dual of the EPR Σ˙ω(x)∗(η)\dot{\Sigma}_{\omega(x)}^{*}(\eta).

dxdt=Sj(x)=S∂fΨω(x)∗[f(x)]=S∂fΨω(x)∗[−ST∂DKL(x∥xeq)∂x+fne].(26)\frac{\mathrm{d}x}{\mathrm{d}t}=Sj(x)=S\partial_{f}\Psi_{\omega(x)}^{*}[f(x)]=S\partial_{f}\Psi_{\omega(x)}^{*}\left[-S^{\mathsf{T}}\frac{\partial D_{\mathrm{KL}}(x\|x_{\mathrm{eq}})}{\partial x}+f_{\mathrm{ne}}\right]. \tag*{(26)}

Convex representation of entropy production rate by generalized mean

While the representation Eq. (25) of EPR by the pair of convex dissipation functions is fundamental in the thermodynamics of CRNs, it may not be employed for the control problem because it is a sum of two strictly convex functions with different arguments, i.e., jj and ff. The EPR is more directly represented as functions of ff, jj, or uu by using Eq. (20)

and its inverse f=ψm−1(jω)f=\psi_m^{-1}\left(\frac{j}{\omega}\right) (Figure 3 (a) and (b)):

Figure 3. The functional form of EPR as a function of ff (a) and u=j/ωu=j/\omega (b) for each generalized mean where ∣R∣=1|\mathcal{R}|=1. (c) Comparison of the pseudo-force η\eta (the solid curve) and the thermodynamic force ff (the dashed curve) as function of u=j/ωu=j/\omega for each generalized mean.

Σ˙ω(f)=⟨ω,f∘ψm(f)⟩(27)\dot{\Sigma}_{\omega}(f)=\left\langle\omega,f\circ\psi_m(f)\right\rangle \tag*{(27)}
Σ˙ω(j)=⟨ω,jω∘ψm−1(jω)⟩(28)\dot{\Sigma}_{\omega}(j)=\left\langle\omega,\frac{j}{\omega}\circ\psi_m^{-1}\left(\frac{j}{\omega}\right)\right\rangle \tag*{(28)}
Σ˙ω(u)=⟨ω,u∘ψm−1(u)⟩(29)\dot{\Sigma}_{\omega}(u)=\left\langle\omega,u\circ\psi_m^{-1}(u)\right\rangle \tag*{(29)}

If the second condition in Eq. (22) is met additionally to the first one8, Σ˙ω(u)\dot{\Sigma}_{\omega}(u) becomes strictly convex. Furthermore, Σ˙ω(u)\dot{\Sigma}_{\omega}(u) is symmetric and greater than or equal to 0, the equality of which holds if and only if u=0u=0. The same true for Σ˙ω(j)\dot{\Sigma}_{\omega}(j).9 Then, the convex conjugate of Σ˙ω(j)\dot{\Sigma}_{\omega}(j) is defined as

Σ˙ω∗(η):=sup⁡j[⟨η,j⟩−Σ˙ω(j)](30)\dot{\Sigma}_{\omega}^{*}(\eta):=\sup_{j}\left[\langle\eta,j\rangle-\dot{\Sigma}_{\omega}(j)\right] \tag*{(30)}

which is also strictly convex, symmetric, and greater than or equal to 0 from the properties of the convex conjugate. Thus, the pair of Σ˙ω(j)\dot{\Sigma}_{\omega}(j) and Σ˙ω∗(η)\dot{\Sigma}_{\omega}^{*}(\eta) induce the Legendre duality between jj and η\eta as

j=∂ηΣ˙ω∗(η),η=∂jΣ˙ω(j).(31)j=\partial_{\eta}\dot{\Sigma}_{\omega}^{*}(\eta),\qquad\eta=\partial_j\dot{\Sigma}_{\omega}(j). \tag*{(31)}

We refer to η\eta, the Legendre dual of jj via EPR, as the pseudo-force because η\eta differs from the thermodynamic force ff (Figure 3 (c)), which is the Legendre dual of jj induced by the dissipation function (Eq. (24)). The explicit representation of η\eta as a function of jj is η=ψm−1(j/ω)+jω∘(ψm−1)′(j/ω)\eta=\psi_m^{-1}(j/\omega)+\frac{j}{\omega}\circ(\psi_m^{-1})'(j/\omega) where (ψm−1)′(f):=dψm−1(f)/df(\psi_m^{-1})'(f):=\mathrm{d}\psi_m^{-1}(f)/\mathrm{d}f. In contrast, jj and also Σ˙ω∗(η)\dot{\Sigma}_{\omega}^{*}(\eta) are not explicitly represented by η\eta, except for the logarithmic mean. Nevertheless, as we see in § 0.3 E, Σ˙ω∗(η(j)):=Σ˙ω∗(η)∣η=∂Σ˙ω(j)\dot{\Sigma}_{\omega}^{*}(\eta(j)):=\left.\dot{\Sigma}_{\omega}^{*}(\eta)\right|_{\eta=\partial\dot{\Sigma}_{\omega}(j)}, Σ˙ω∗(η)\dot{\Sigma}_{\omega}^{*}(\eta) as a function of jj, has an explicit form Σ˙ω∗(η(j))=⟨j,∂2Ψω(j)∂j2j⟩\dot{\Sigma}_{\omega}^{*}(\eta(j))=\left\langle j,\frac{\partial^2\Psi_{\omega}(j)}{\partial j^2}j\right\rangle, which manifests that it is identical to the generalized pseudo-EPR Πω(j)\Pi_{\omega}(j).

Explicit form of value function and HJB equation

By employing the representation Σ˙ω(j):=⟨j,ψm−1(j/ω)⟩\dot{\Sigma}_{\omega}(j):=\left\langle j,\psi_m^{-1}(j/\omega)\right\rangle in Eq. (28), Eq. (17) can be explicitly transformed as

j˙t±(x;β)=arg⁡max⁡{j±}∈R≥02∣R∣[⟨(ST∂Vt(x;β)∂x+β2F(t,x)),j⟩−Σ˙ω(j)](32)\dot{j}^{\pm}_t(x;\beta)=\underset{\{j^{\pm}\}\in\mathbb{R}_{\geq0}^{2|\mathcal{R}|}}{\arg\max}\left[\left\langle\left(\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x;\beta)}{\partial x}+\beta_2F(t,x)\right),j\right\rangle-\dot{\Sigma}_{\omega}(j)\right] \tag*{(32)}

where we replace the control variable from {k±}\{k^\pm\} to {j±}\{j^\pm\} because we can arbitrary modulate j±j^\pm by changing {k±}\{k^\pm\} for x∈R>0∣X∣x\in\mathbb{R}_{>0}^{|\mathcal{X}|}. By further transforming the control variable from {j±}\{j^\pm\} to ω\omega and jj, we formally obtain

(ωt†(x;β),jt†(x;β))=arg⁡max⁡ω∈R≥0∣R∣,j∈R∣R∣[⟨(ST∂Vt(x;β)∂x+β2F(t,x)),j⟩−Σ˙ω(j)].(33)(\omega_t^\dagger(x;\beta),j_t^\dagger(x;\beta))=\underset{\omega\in\mathbb{R}_{\geq0}^{|\mathcal{R}|},j\in\mathbb{R}^{|\mathcal{R}|}}{\arg\max}\left[\left\langle\left(\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x;\beta)}{\partial x}+\beta_2F(t,x)\right),j\right\rangle-\dot{\Sigma}_{\omega}(j)\right]. \tag*{(33)}

Partial optimization of Eq. (33) with respect to ω\omega for a fixed jj is equivalent to arg⁡min⁡ωΣ˙ω(j)=⟨j,ψm−1(j/ω)⟩\arg\min_{\omega}\dot{\Sigma}_{\omega}(j)=\langle j,\psi_m^{-1}(j/\omega)\rangle. Since ψm−1(u)\psi_m^{-1}(u) is odd and monotonous, Σ˙ω(j)→0\dot{\Sigma}_{\omega}(j)\to0 as ω→∞\omega\to\infty. Thus, without additional constraints over possible values of ω\omega, the solution becomes singular[52]. A representative constraint is directly fixing ω\omega to a certain value or function ω(x)\omega(x)[79, 80, 88]. In the main body, we mainly focus on the case where ωt†\omega_t^\dagger is fixed for each xx and tt as ωt(x)\omega_t(x). Another constraint being investigated before is the total activity budget constraint; ⟨1,ω⟩≤Ω\langle1,\omega\rangle\leq\Omega for each time point or 1T∫0T⟨1,ωt⟩≤Ω\frac{1}{T}\int_0^T\langle1,\omega_t\rangle\leq\Omega over the whole trajectory [52, 53, 54], which is treated in Appendix.

Type of meanω[j+,j−]\omega[j^{+},j^{-}]Σ˙ω(j)\dot{\Sigma}_{\omega}(j)η=∂jΣ˙ω(j)\eta=\partial_{j}\dot{\Sigma}_{\omega}(j)Σ˙ω∗(η)\dot{\Sigma}_{\omega}^{*}(\eta)j=∂Σ˙ω∗(η)j=\partial\dot{\Sigma}_{\omega}^{*}(\eta)
Harmonic2j+∘j−j++j−\frac{2j^{+}\circ j^{-}}{j^{+}+j^{-}}j∘sinh⁡−1(jω)j\circ\sinh^{-1}\left(\frac{j}{\omega}\right)j2ω2+j2+sinh⁡−1(jω)\frac{j^{2}}{\sqrt{\omega^{2}+j^{2}}}+\sinh^{-1}\left(\frac{j}{\omega}\right)NANA
Geometricj+∘j−\sqrt{j^{+}\circ j^{-}}2j∘sinh⁡−1(j2ω)2j\circ\sinh^{-1}\left(\frac{j}{2\omega}\right)j2ω2+j24+2sinh⁡−1(j2ω)\frac{j^{2}}{\sqrt{\omega^{2}+\frac{j^{2}}{4}}}+2\sinh^{-1}\left(\frac{j}{2\omega}\right)NANA
Logarithmicj+−j−ln⁡j+−ln⁡j−\frac{j^{+}-j^{-}}{\ln j^{+}-\ln j^{-}}j2ω\frac{j^{2}}{\omega}2jω\frac{2j}{\omega}ω∘(η2)2\omega\circ\left(\frac{\eta}{2}\right)^{2}ω2∘η\frac{\omega}{2}\circ\eta
Arithmeticj++j−2\frac{j^{+}+j^{-}}{2}2j∘tanh⁡−1(j2ω)2j\circ\tanh^{-1}\left(\frac{j}{2\omega}\right)ωjω2−j24+2tanh⁡−1(j2ω)\frac{\omega j}{\omega^{2}-\frac{j^{2}}{4}}+2\tanh^{-1}\left(\frac{j}{2\omega}\right)NANA
Contraharmonic(j+)2+(j−)2j++j−\frac{(j^{+})^{2}+(j^{-})^{2}}{j^{+}+j^{-}}j∘tanh⁡−1(jω)j\circ\tanh^{-1}\left(\frac{j}{\omega}\right)ωjω2−j2+tanh⁡−1(jω)\frac{\omega j}{\omega^{2}-j^{2}}+\tanh^{-1}\left(\frac{j}{\omega}\right)NANA

Table 2. The functional forms of the ERP and its conjugate and induced Legendre transformation for representative generalized means. NA stands for no analytic representation by elementary functions available.

Given a fixed ωt(x)\omega_t(x), we then solve the optimization with respect to jj, which has the (linear)-(convex) Legendre form. Thus, the optimal flux jt†j_t^\dagger is obtained as

jt†(x)=arg⁡max⁡j[⟨(ST∂Vt(x)∂x+β2F(t,x)),j⟩−Σ˙ω(x)(j)](34)j_t^\dagger(x)=\underset{j}{\arg\max}\left[\left\langle\left(\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x)}{\partial x}+\beta_2F(t,x)\right),j\right\rangle-\dot{\Sigma}_{\omega(x)}(j)\right] \tag*{(34)}
=∂ηΣ˙ωt(x)∗[ST∂Vt(x)∂x+β2F(t,x)]=∂ηΣ˙ωt(x)∗[ηt†(x)],(35)=\partial_{\eta}\dot{\Sigma}^{*}_{\omega_t(x)}\left[\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x)}{\partial x}+\beta_2F(t,x)\right]=\partial_{\eta}\dot{\Sigma}^{*}_{\omega_t(x)}\left[\eta_t^\dagger(x)\right], \tag*{(35)}

where the optimal pseudo-force is defined as

ηt†(x):=ST∂Vt(x)∂x+β2F(t,x).(36)\eta_t^\dagger(x):=\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x)}{\partial x}+\beta_2F(t,x). \tag*{(36)}

From Eq. (35), we can see that the optimally controlled dynamics have the form (Figure 2(b)):

dxdt=Sjt†(x)=S∂ηΣ˙ωt(x)∗[ST∂Vt(x)∂x+β2F(t,x)](37)\frac{dx}{dt}=\mathbf{S}j_t^\dagger(x)=\mathbf{S}\partial_{\eta}\dot{\Sigma}^{*}_{\omega_t(x)}\left[\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x)}{\partial x}+\beta_2F(t,x)\right] \tag*{(37)}

Note that the optimal thermodynamic force ft†f_t^\dagger is

ft†(x)=ψm−1(jt†(x)ωt(x))=ψm−1(1ωt(x)∘∂ηΣ˙ωt(x)∗[ST∂Vt(x)∂x+β2F(t,x)])(38)f_t^\dagger(x)=\psi_m^{-1}\left(\frac{j_t^\dagger(x)}{\omega_t(x)}\right)=\psi_m^{-1}\left(\frac{1}{\omega_t(x)}\circ\partial_{\eta}\dot{\Sigma}^{*}_{\omega_t(x)}\left[\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(x)}{\partial x}+\beta_2F(t,x)\right]\right) \tag*{(38)}

and the optimal kinetic parameters are obtained as

ht+(x)=ωt∘ψm(ft†(x))1−e−ft†(x)∘1h+(x),ht−(x)=ωt∘ψm(ft†(x))eft†(x)−1∘1h−(x).(39)h_t^{+}(x)=\omega_t\circ\frac{\psi_m(f_t^\dagger(x))}{1-e^{-f_t^\dagger(x)}}\circ\frac{1}{h^{+}(x)},\qquad h_t^{-}(x)=\omega_t\circ\frac{\psi_m(f_t^\dagger(x))}{e^{f_t^\dagger(x)}-1}\circ\frac{1}{h^{-}(x)}. \tag*{(39)}

To gain an explicit form of the HJB equation (Eq. (18)), we introduce a Hamiltonian

Hω[t,x,η,j]:=β1U(t,x)+⟨j,η⟩+β2⟨F(t,x),j⟩−Σ˙ω(j),(40)\mathcal{H}_{\omega}[t,\boldsymbol{x},\boldsymbol{\eta},\boldsymbol{j}] := \beta_1 U(t,\boldsymbol{x}) + \langle\boldsymbol{j},\boldsymbol{\eta}\rangle+ \beta_2\langle F(t,\boldsymbol{x}),\boldsymbol{j}\rangle- \dot{\Sigma}_{\omega}(\boldsymbol{j}), \tag*{(40)}

Then, the HJB equation after transformation is

−∂Vt(x)∂t=max⁡j∈RRHωt(x)[t,x,S⊤∂Vt(x)∂x,j]=Hωt(x)[t,x,S⊤∂Vt(x)∂x,j˙†(x)]=β1U(t,x)+Σ˙ωt(x)∗[S⊤∂Vt(x)∂x+β2F(t,x)]=Hωt(x)†[t,x,S⊤∂Vt(x)∂x],(41)\begin{aligned} -\frac{\partial V_t(\boldsymbol{x})}{\partial t} = \max_{\boldsymbol{j}\in\mathbb{R}^{R}} \mathcal{H}_{\omega_t(\boldsymbol{x})}\left[t,\boldsymbol{x},\mathsf{S}^{\top}\frac{\partial V_t(\boldsymbol{x})}{\partial\boldsymbol{x}},\boldsymbol{j}\right] = \mathcal{H}_{\omega_t(\boldsymbol{x})}\left[t,\boldsymbol{x},\mathsf{S}^{\top}\frac{\partial V_t(\boldsymbol{x})}{\partial\boldsymbol{x}},\dot{\boldsymbol{j}}^{\dagger}(\boldsymbol{x})\right] \\ &= \beta_1 U(t,\boldsymbol{x})+\dot{\Sigma}^{*}_{\omega_t(\boldsymbol{x})}\left[\mathsf{S}^{\top}\frac{\partial V_t(\boldsymbol{x})}{\partial\boldsymbol{x}}+\beta_2F(t,\boldsymbol{x})\right]=\mathcal{H}^{\dagger}_{\omega_t(\boldsymbol{x})}\left[t,\boldsymbol{x},\mathsf{S}^{\top}\frac{\partial V_t(\boldsymbol{x})}{\partial\boldsymbol{x}}\right], \tag*{(41)} \end{aligned}

where the optimal Hamiltonian Hω†\mathcal{H}^{\dagger}_{\omega} is defined as

Hω†[t,x,η]:=β1U(t,x)+Σ˙ω∗[η+β2F(t,x)].(42)\mathcal{H}^{\dagger}_{\omega}[t,\boldsymbol{x},\boldsymbol{\eta}] := \beta_1 U(t,\boldsymbol{x})+\dot{\Sigma}^{*}_{\omega}\left[\boldsymbol{\eta}+\beta_2F(t,\boldsymbol{x})\right]. \tag*{(42)}

The corresponding optimal Lagrangian is

Lω†[t,x,j]:=max⁡η[⟨j,η⟩−Hω†[t,x,η]]=−β1U(t,x)−β2⟨F(t,x),j⟩+Σ˙ω[j](43)\mathcal{L}^{\dagger}_{\omega}[t,\boldsymbol{x},\boldsymbol{j}] := \max_{\boldsymbol{\eta}}\left[\langle\boldsymbol{j},\boldsymbol{\eta}\rangle-\mathcal{H}^{\dagger}_{\omega}[t,\boldsymbol{x},\boldsymbol{\eta}]\right] =-\beta_1U(t,\boldsymbol{x})-\beta_2\langle F(t,\boldsymbol{x}),\boldsymbol{j}\rangle+\dot{\Sigma}_{\omega}[\boldsymbol{j}] \tag*{(43)}

The value function along the optimal dynamics (Eq. (37)) satisfies

dVt(t,xt)dt=∂Vt(t,xt)∂t+⟨∂Vt(t,xt)∂x,dxdt⟩(44)\frac{\mathrm{d}V_t(t,\boldsymbol{x}_t)}{\mathrm{d}t}=\frac{\partial V_t(t,\boldsymbol{x}_t)}{\partial t}+\left\langle\frac{\partial V_t(t,\boldsymbol{x}_t)}{\partial\boldsymbol{x}},\frac{\mathrm{d}\boldsymbol{x}}{\mathrm{d}t}\right\rangle \tag*{(44)}
=−β1U(t,x)−Σ˙ωt(x)∗[ηt†(x)]+⟨∂Vt(t,xt)∂x,Sj˙t†(x)⟩(45)=-\beta_1U(t,\boldsymbol{x})-\dot{\Sigma}^{*}_{\omega_t(\boldsymbol{x})}[\boldsymbol{\eta}^{\dagger}_t(\boldsymbol{x})]+\left\langle\frac{\partial V_t(t,\boldsymbol{x}_t)}{\partial\boldsymbol{x}},\mathsf{S}\dot{\boldsymbol{j}}^{\dagger}_t(\boldsymbol{x})\right\rangle \tag*{(45)}
=−β1U(t,x)−Σ˙ωt(x)∗[ηt†(x)]+⟨ηt†(x)−β2F(t,x),j˙t†(x)⟩(46)=-\beta_1U(t,\boldsymbol{x})-\dot{\Sigma}^{*}_{\omega_t(\boldsymbol{x})}[\boldsymbol{\eta}^{\dagger}_t(\boldsymbol{x})]+\left\langle\boldsymbol{\eta}^{\dagger}_t(\boldsymbol{x})-\beta_2F(t,\boldsymbol{x}),\dot{\boldsymbol{j}}^{\dagger}_t(\boldsymbol{x})\right\rangle \tag*{(46)}
=−β1U(t,x)−β2⟨j˙t†(x),F(t,x)⟩+Σ˙ωt(x)[j˙t†(x)](47)=-\beta_1U(t,\boldsymbol{x})-\beta_2\left\langle\dot{\boldsymbol{j}}^{\dagger}_t(\boldsymbol{x}),F(t,\boldsymbol{x})\right\rangle+\dot{\Sigma}_{\omega_t(\boldsymbol{x})}[\dot{\boldsymbol{j}}^{\dagger}_t(\boldsymbol{x})] \tag*{(47)}
=Lωt(x)†[t,x,j˙t†(x)].(48)=\mathcal{L}^{\dagger}_{\omega_t(\boldsymbol{x})}[t,\boldsymbol{x},\dot{\boldsymbol{j}}^{\dagger}_t(\boldsymbol{x})]. \tag*{(48)}

Thus,

Vt(xt)=β1U^T(xT)−∫tTLωt′(xt′)†[t′,xt′,j˙†(xt′)]dt′(49)V_t(\boldsymbol{x}_t)=\beta_1\widehat{U}_T(\boldsymbol{x}_T)-\int_t^T\mathcal{L}^{\dagger}_{\omega_{t'}(\boldsymbol{x}_{t'})}[t',\boldsymbol{x}_{t'},\dot{\boldsymbol{j}}^{\dagger}(\boldsymbol{x}_{t'})]\mathrm{d}t' \tag*{(49)}

This is consistent with the original definition of the value function by Eq. (14).10

Pontryagin's maximum principle and Hamiltonian dynamics

Although Bellman’s principle provides the global perspective on the optimal control problem and the induced dynamics, solving the HJB equation, an infinite-dimensional equation in general, is often computationally prohibitive both analytically and numerically even for low-dimensional CRNs. Pontryagin’s maximum principle offers an alternative way to solve the same control problem via Hamiltonian dynamics, yet locally for each trajectory. Specifically, the optimal control solution (Eq. (37)) can be casted to the solution of Hamiltonian dynamics.

By assuming that the optimal value function is smooth, we introduce p\mathbf{p} as

pt:=∂Vt(x)∂x∣x=xt†(50)\mathbf{p}_t := \left.\frac{\partial V_t(\mathbf{x})}{\partial\mathbf{x}}\right|_{\mathbf{x}=\mathbf{x}_t^\dagger} \tag*{(50)}

where xt†\mathbf{x}_t^\dagger is the solution of Eq. (37). Then, the dynamics of (xt,pt)(\mathbf{x}_t,\mathbf{p}_t) is determined by the Pontryagin’s Hamiltonian dynamics:

dxdt=∂Hωt(x)†[t,x,STp]∂p=S∂ηΣ˙ωt(x)∗[STp+β2F(t,x)](51)\frac{\mathrm{d}\mathbf{x}}{\mathrm{d}t} = \frac{\partial\mathcal{H}_{\omega_t(\mathbf{x})}^{\dagger}[t,\mathbf{x},\mathbf{S}^{\mathsf{T}}\mathbf{p}]}{\partial\mathbf{p}} = \mathbf{S}\partial_{\eta}\dot{\Sigma}_{\omega_t(\mathbf{x})}^{*}\left[\mathbf{S}^{\mathsf{T}}\mathbf{p}+\beta_2\mathbf{F}(t,\mathbf{x})\right] \tag*{(51)}
dpdt=−∂Hωt(x)†[t,x,STp]∂xt=−β1∂U(t,x)∂x−∂∂x{Σ˙ωt(x)∗[STp+β2F(t,x)]}(52)\frac{\mathrm{d}\mathbf{p}}{\mathrm{d}t} = -\frac{\partial\mathcal{H}_{\omega_t(\mathbf{x})}^{\dagger}[t,\mathbf{x},\mathbf{S}^{\mathsf{T}}\mathbf{p}]}{\partial\mathbf{x}_t} = -\beta_1\frac{\partial U(t,\mathbf{x})}{\partial\mathbf{x}}-\frac{\partial}{\partial\mathbf{x}}\left\{\dot{\Sigma}_{\omega_t(\mathbf{x})}^{*}\left[\mathbf{S}^{\mathsf{T}}\mathbf{p}+\beta_2\mathbf{F}(t,\mathbf{x})\right]\right\} \tag*{(52)}

This finite-dimensional equation solved with appropriate boundary conditions provides candidates of the optimally controlled trajectory11.

Thermodynamic representation of Pontryagin’s equation

Although Hamiltonian dynamics via Pontryagin’s principle, Eq. (51), and Eq. (52), is the standard form for solving general optimal control, most of the previous work on thermodynamic transport and control adopted the thermodynamic force f\mathbf{f} rather than η\boldsymbol{\eta} as the control variable. In addition, the convex conjugate EPR Σ˙ω∗(η)\dot{\Sigma}_{\omega}^{*}(\boldsymbol{\eta}) embedded in the equations does not have an analytic representation as a function of η\boldsymbol{\eta} for most of the generalized means except the logarithmic one12. We resolve these problems by showing that Σ˙ω∗\dot{\Sigma}_{\omega}^{*} can be represented analytically as a function of f\mathbf{f}, which establishes a thermodynamic representation of Pontryagin’s equation. From Eq. (28), we obtain

η=∂jΣ˙ω(j)=ψm−1(jω)+jω∘1ψm′[ψm−1(jω)]=f+ψm(f)ψm′(f)(53)\boldsymbol{\eta}=\partial_j\dot{\Sigma}_{\omega}(j)=\psi_m^{-1}\left(\frac{j}{\omega}\right)+\frac{j}{\omega}\circ\frac{1}{\psi_m'\left[\psi_m^{-1}\left(\frac{j}{\omega}\right)\right]}=\mathbf{f}+\frac{\psi_m(\mathbf{f})}{\psi_m'(\mathbf{f})} \tag*{(53)}

where ψm′(f):=dψm(f)/df\psi'_m(f) := \mathrm{d}\psi_m(f)/\mathrm{d}f (Figure 2 (c)) 13. Thus, η−f=ψm(f)ψm′(f)\boldsymbol{\eta}-\boldsymbol{f} = \frac{\psi_m(f)}{\psi'_m(f)} holds. This relation is also rephrased by using the dissipation function as

η−f=∂Σ˙(j)∂j−f=∂(Ψω(j)+Ψω∗(f(j)))∂j−f(54)\boldsymbol{\eta}-\boldsymbol{f} = \frac{\partial\dot{\Sigma}(\boldsymbol{j})}{\partial\boldsymbol{j}}-\boldsymbol{f} = \frac{\partial\left(\Psi_\omega(\boldsymbol{j})+\Psi_\omega^*(\boldsymbol{f}(\boldsymbol{j}))\right)}{\partial\boldsymbol{j}}-\boldsymbol{f} \tag*{(54)}
=∂Ψω∗(f(j))∂j=∂f(j)∂j∂Ψω∗(f)∂f=∂2Ψω(j)∂j2j,(55)= \frac{\partial\Psi_\omega^*(\boldsymbol{f}(\boldsymbol{j}))}{\partial\boldsymbol{j}} = \frac{\partial\boldsymbol{f}(\boldsymbol{j})}{\partial\boldsymbol{j}}\frac{\partial\Psi_\omega^*(\boldsymbol{f})}{\partial\boldsymbol{f}} = \frac{\partial^2\Psi_\omega(\boldsymbol{j})}{\partial\boldsymbol{j}^2}\boldsymbol{j}, \tag*{(55)}

where ∂2Ψω(j)∂j2\frac{\partial^2\Psi_\omega(\boldsymbol{j})}{\partial\boldsymbol{j}^2} is the Hessian of Ψω(j)\Psi_\omega(\boldsymbol{j}). Then, we have

Σ˙ω∗(f):=Σ˙ω∗(η)=⟨η,j⟩−Σ˙ω(j)=⟨η,j⟩−⟨f,j⟩=⟨η−f,j⟩(56)\dot{\Sigma}_\omega^*(\boldsymbol{f}) := \dot{\Sigma}_\omega^*(\boldsymbol{\eta}) = \langle\boldsymbol{\eta},\boldsymbol{j}\rangle-\dot{\Sigma}_\omega(\boldsymbol{j}) = \langle\boldsymbol{\eta},\boldsymbol{j}\rangle-\langle\boldsymbol{f},\boldsymbol{j}\rangle= \langle\boldsymbol{\eta}-\boldsymbol{f},\boldsymbol{j}\rangle \tag*{(56)}
=⟨ψm(f)ψm′(f),j⟩=⟨ψm(f)ψm′(f),ω∘ψm(f)⟩=⟨ω,ψm(f)ψm′(f)⟩,(57)= \left\langle\frac{\psi_m(\boldsymbol{f})}{\psi'_m(\boldsymbol{f})},\boldsymbol{j}\right\rangle= \left\langle\frac{\psi_m(\boldsymbol{f})}{\psi'_m(\boldsymbol{f})},\boldsymbol{\omega}\circ\psi_m(\boldsymbol{f})\right\rangle= \left\langle\boldsymbol{\omega},\frac{\psi_m(\boldsymbol{f})}{\psi'_m(\boldsymbol{f})}\right\rangle, \tag*{(57)}

where we used Eq. (53) from the first to the second lines. Note that, as a function of j\boldsymbol{j}, Σ˙ω∗(η)\dot{\Sigma}_\omega^*(\boldsymbol{\eta}) becomes

Σ˙ω∗(j):=Σ˙ω∗(η)=⟨η−f,j⟩=⟨j,∂2Ψω(j)∂j2j⟩=Πω(j),(58)\dot{\Sigma}_\omega^*(\boldsymbol{j}) := \dot{\Sigma}_\omega^*(\boldsymbol{\eta}) = \langle\boldsymbol{\eta}-\boldsymbol{f},\boldsymbol{j}\rangle= \left\langle\boldsymbol{j},\frac{\partial^2\Psi_\omega(\boldsymbol{j})}{\partial\boldsymbol{j}^2}\boldsymbol{j}\right\rangle= \Pi_\omega(\boldsymbol{j}), \tag*{(58)}

where Πω(j):=⟨j,∂2Ψω(j)∂j2j⟩\Pi_\omega(\boldsymbol{j}) := \left\langle\boldsymbol{j},\frac{\partial^2\Psi_\omega(\boldsymbol{j})}{\partial\boldsymbol{j}^2}\boldsymbol{j}\right\rangle is a generalization of the pseudo-entropy production rate (pEPR) via the generalized means[85, 90, 91]. Although pEPR has been recognized as a proxy for EPR when extending the thermodynamic uncertainty relation to MJPs[52, 91, 92], our result clarifies that it has a fundamental meaning as the dual of EPR14.

Using Eq. (36) and Eq. (53), the value function and the optimal force are associated by the following equation:

ST∂Vt(x)∂x+β2F(t,x)=ηt†(x)=ft†(x)+ψm(ft†(x))ψm′(ft†(x)).(59)\mathbf{S}^{\mathsf{T}}\frac{\partial V_t(\boldsymbol{x})}{\partial\boldsymbol{x}}+\beta_2\boldsymbol{F}(t,\boldsymbol{x})=\boldsymbol{\eta}_t^\dagger(\boldsymbol{x})=\boldsymbol{f}_t^\dagger(\boldsymbol{x})+\frac{\psi_m(\boldsymbol{f}_t^\dagger(\boldsymbol{x}))}{\psi'_m(\boldsymbol{f}_t^\dagger(\boldsymbol{x}))}. \tag*{(59)}

The HJB equation (Eq. (41)) is transformed into

−∂Vt(x)∂t=β1U(t,x)+Σ˙ωt∗[ηt†(x)]=β1U(t,x)+⟨ωt(x),ψm(ft†(x))2ψm′(ft†(x))⟩(60)-\frac{\partial V_t(\boldsymbol{x})}{\partial t}=\beta_1U(t,\boldsymbol{x})+\dot{\Sigma}_{\omega_t}^*\left[\boldsymbol{\eta}_t^\dagger(\boldsymbol{x})\right]=\beta_1U(t,\boldsymbol{x})+\left\langle\boldsymbol{\omega}_t(\boldsymbol{x}),\frac{\psi_m(\boldsymbol{f}_t^\dagger(\boldsymbol{x}))^2}{\psi'_m(\boldsymbol{f}_t^\dagger(\boldsymbol{x}))}\right\rangle \tag*{(60)}

where we used Eq. (57). For Pontryagin’s equation, p\boldsymbol{p} and f\boldsymbol{f} are associated as

STp+β2F(t,x)=η=f+ψm(f)ψm′(f),(61)\mathbf{S}^{\mathsf{T}}\boldsymbol{p}+\beta_2\boldsymbol{F}(t,\boldsymbol{x})=\boldsymbol{\eta}=\boldsymbol{f}+\frac{\psi_m(\boldsymbol{f})}{\psi'_m(\boldsymbol{f})}, \tag*{(61)}

By using Eq. (20) and Eq. (57), Eq. (51) and Eq. (52) become

dxdt=S[ωt(x)∘ψm(f)],(62)\frac{d\mathbf{x}}{dt}=\mathbf{S}[\boldsymbol{\omega}_t(\mathbf{x})\circ\boldsymbol{\psi}_m(\mathbf{f})], \tag*{(62)}
dpdt=−β1∂U(t,x)∂x−β2(∂F(t,x)∂x)Tωt(x)∘ψm(f)−(∂ωt(x)∂x)Tψm(f)2ψm′(f)(63)\frac{d\mathbf{p}}{dt}=-\beta_1\frac{\partial U(t,\mathbf{x})}{\partial\mathbf{x}}-\beta_2\left(\frac{\partial F(t,\mathbf{x})}{\partial\mathbf{x}}\right)^{\mathsf{T}}\boldsymbol{\omega}_t(\mathbf{x})\circ\boldsymbol{\psi}_m(\mathbf{f})-\left(\frac{\partial\boldsymbol{\omega}_t(\mathbf{x})}{\partial\mathbf{x}}\right)^{\mathsf{T}}\frac{\boldsymbol{\psi}_m(\mathbf{f})^2}{\psi'_m(\mathbf{f})} \tag*{(63)}

By differentiating Eq. (61) along xt†\mathbf{x}_t^\dagger with respect to tt, we have

diag⁡[2−ψm(f)∘ψm′′(f)(ψm′(f))2]dfdt=STdpdt+β2[∂F(t,x)∂t+(∂F(t,x)∂x)dxdt](64)\operatorname{diag}\left[2-\frac{\psi_m(\mathbf{f})\circ\psi''_m(\mathbf{f})}{(\psi'_m(\mathbf{f}))^2}\right]\frac{d\mathbf{f}}{dt}=\mathbf{S}^{\mathsf{T}}\frac{d\mathbf{p}}{dt}+\beta_2\left[\frac{\partial F(t,\mathbf{x})}{\partial t}+\left(\frac{\partial F(t,\mathbf{x})}{\partial\mathbf{x}}\right)\frac{d\mathbf{x}}{dt}\right] \tag*{(64)}

By inserting Eq. (62) and Eq. (63), we have

dfdt=diag⁡[2−ψm(f)∘ψm′′(f)(ψm′(f))2]−1{−β1ST∂U(t,x)∂x−ST(∂ωt(x)∂x)Tψm(f)2ψm′(f)+β2[∂F(t,x)∂t+{(∂F(t,x)∂x)S−(∂F(t,x)∂xS)T}ωt(x)∘ψm(f)]}(65)\frac{d\mathbf{f}}{dt}=\operatorname{diag}\left[2-\frac{\psi_m(\mathbf{f})\circ\psi''_m(\mathbf{f})}{(\psi'_m(\mathbf{f}))^2}\right]^{-1}\left\{-\beta_1\mathbf{S}^{\mathsf{T}}\frac{\partial U(t,\mathbf{x})}{\partial\mathbf{x}}-\mathbf{S}^{\mathsf{T}}\left(\frac{\partial\boldsymbol{\omega}_t(\mathbf{x})}{\partial\mathbf{x}}\right)^{\mathsf{T}}\frac{\boldsymbol{\psi}_m(\mathbf{f})^2}{\psi'_m(\mathbf{f})}+\beta_2\left[\frac{\partial F(t,\mathbf{x})}{\partial t}+\left\{\left(\frac{\partial F(t,\mathbf{x})}{\partial\mathbf{x}}\right)\mathbf{S}-\left(\frac{\partial F(t,\mathbf{x})}{\partial\mathbf{x}}\mathbf{S}\right)^{\mathsf{T}}\right\}\boldsymbol{\omega}_t(\mathbf{x})\circ\boldsymbol{\psi}_m(\mathbf{f})\right]\right\} \tag*{(65)}

We also have an algebraic constraint

f+ψm(f)ψm′(f)−β2F(t,x)=STp∈Im⁡ST,(66)\mathbf{f}+\frac{\boldsymbol{\psi}_m(\mathbf{f})}{\psi'_m(\mathbf{f})}-\beta_2F(t,\mathbf{x})=\mathbf{S}^{\mathsf{T}}\mathbf{p}\in\operatorname{Im}\mathbf{S}^{\mathsf{T}}, \tag*{(66)}

which originates from Eq. (53) and accounts for the difference of dimensions of p\mathbf{p} and f\mathbf{f}. The set of equations, Eq. (62), Eq. (65) and Eq. (66) establish the thermodynamic version of Pontryagin’s Hamiltonian dynamics.

Analytic solutions and its relationship with thermodynamic transport problems

In this section, we demonstrate that the transport problem with the constant flux utility is a theoretically sound extension that unifies the usual thermodynamic transport problem and the flux optimization problem. Specifically, we consider the setup such that U(t,x)=0U(t,\mathbf{x})=0, and β2F(t,x)=F\beta_2F(t,\mathbf{x})=F, and the boundary conditions are x0=xinit\mathbf{x}_0=\mathbf{x}_{init} and xT=xend\mathbf{x}_T=\mathbf{x}_{end}, satisfying xinit−xend∈Im⁡S\mathbf{x}_{init}-\mathbf{x}_{end}\in\operatorname{Im}\mathbf{S}. Then, the thermodynamic Pontryagin equation for f\mathbf{f} is reduced to

dfdt=−diag⁡[2−ψm(f)∘ψm′′(f)(ψm′(f))2]−1{ST(∂ωt(x)∂x)Tψm(f)2ψm′(f)}.(67)\frac{d\mathbf{f}}{dt}=-\operatorname{diag}\left[2-\frac{\psi_m(\mathbf{f})\circ\psi''_m(\mathbf{f})}{(\psi'_m(\mathbf{f}))^2}\right]^{-1}\left\{\mathbf{S}^{\mathsf{T}}\left(\frac{\partial\boldsymbol{\omega}_t(\mathbf{x})}{\partial\mathbf{x}}\right)^{\mathsf{T}}\frac{\boldsymbol{\psi}_m(\mathbf{f})^2}{\psi'_m(\mathbf{f})}\right\}. \tag*{(67)}

While solving this equation together with Eq. (62) and Eq. (66) analytically is still impossible in general, it becomes tractable if the right-hand side of Eq. (67) becomes 0 or effectively 0. Most previous works for thermodynamic transport exploit such situations[52, 53, 54, 88].

Optimal thermodynamic transports with flux utility under constant activity and with total activity constraints

If the activity ωt(x)\omega_t(x) is constant ω\omega, the thermodynamic Pontryagin’s equation substantially simplifies to

dxdt=Sω∘ψm(f),dfdt=0,f+ψm(f)ψm′(f)−F∈Im⁡ST(68)\frac{\mathrm{d}x}{\mathrm{d}t} = S\omega\circ\psi_m(f), \qquad\frac{\mathrm{d}f}{\mathrm{d}t} = 0, \qquad f + \frac{\psi_m(f)}{\psi_m'(f)} - F \in\operatorname{Im} S^{\mathsf{T}} \tag*{(68)}

Then, the optimal trajectories are formally f†(t)=f†=const.f^\dagger(t) = f^\dagger= \mathrm{const.} and x†(t)=xinit+tv†x^\dagger(t) = x_{\mathrm{init}} + tv^\dagger where v†:=(xend−xinit)/Tv^\dagger:= (x_{\mathrm{end}} - x_{\mathrm{init}})/T. If v†∈{Sj∣jr/ωr∈Du for all r∈R}v^\dagger\in\{Sj \mid j_r/\omega_r \in D^u \text{ for all } r \in\mathcal{R}\} holds, the optimal constant force f†f^\dagger is obtained by solving Sω∘ψm(f†)=Sj†=v†S\omega\circ\psi_m(f^\dagger) = Sj^\dagger= v^\dagger and η†−F=f†+ψm(f†)ψm′(f†)−F∈Im⁡ST\eta^\dagger- F = f^\dagger+ \frac{\psi_m(f^\dagger)}{\psi_m'(f^\dagger)} - F \in\operatorname{Im} S^{\mathsf{T}}. These equations are equivalent to j†∈Ker⁡S+j′j^\dagger\in\operatorname{Ker} S + j' and η†=∂Σ˙ω(j†)∈Im⁡ST+F\eta^\dagger= \partial\dot{\Sigma}_{\omega}(j^\dagger) \in\operatorname{Im} S^{\mathsf{T}} + F where j′j' is a particular solution of Sj=v†Sj = v^\dagger. Since Im⁡ST\operatorname{Im} S^{\mathsf{T}} and Ker⁡S\operatorname{Ker} S are orthogonal complement, the optimal j†j^\dagger is uniquely determined as j†=(Ker⁡S+j′)∩∂Σ˙ω∗[Im⁡ST+F]j^\dagger= (\operatorname{Ker} S + j') \cap\partial\dot{\Sigma}_{\omega}^{*}[\operatorname{Im} S^{\mathsf{T}} + F] by the information-geometric orthogonality[82]. Then f†f^\dagger and η†\eta^\dagger are uniquely determined from j†j^\dagger. For this setup, the value function is obtained analytically as

Vt(x)=−(T−t)L^ω†(xend−xT−t)(69)V_t(x) = -(T-t)\widehat{L}^{\dagger}_{\omega}\left(\frac{x_{\mathrm{end}}-x}{T-t}\right) \tag*{(69)}

where L^ω†(v)\widehat{L}^{\dagger}_{\omega}(v) is the contracted (optimal) Lagrangian defined as

L^ω†(v):=min⁡v=SjLω†[j]={min⁡v=Sj[Σ˙ω(j)−⟨F,j⟩],if v∈Im⁡S+∞,otherwise(70)\widehat{L}^{\dagger}_{\omega}(v) := \min_{v=Sj} L^{\dagger}_{\omega}[j] = \begin{cases} \min_{v=Sj}\left[\dot{\Sigma}_{\omega}(j)-\langle F,j\rangle\right], & \text{if } v \in\operatorname{Im} S \\ +\infty, & \text{otherwise} \end{cases} \tag*{(70)}

(see Appendix for derivation).

Yoshimura et al. considered this situation for F=0F=0 by using the logarithmic mean and directly constraining ω\omega to be constant[88]. In such a case, the optimal force becomes conservative because η†=f†+ψm(f†)ψm′(f†)=2f†∈Im⁡ST\eta^\dagger= f^\dagger+ \frac{\psi_m(f^\dagger)}{\psi_m'(f^\dagger)} = 2f^\dagger\in\operatorname{Im} S^{\mathsf{T}} for the logarithmic mean. Also, the optimal flux j†=ω∘f†j^\dagger= \omega\circ f^\dagger is the dual of f†f^\dagger w.r.t. the Riemannian metric diag⁡ω\operatorname{diag}\omega.

In addition, our formulation includes the flux maximization at the steady state[61, 62, 63, 64] as a special case. Specifically, if xinit=xendx_{\mathrm{init}} = x_{\mathrm{end}}, the value function reduces to V0(xinit)=−TL^ω†(0)=Tmax⁡Sj=0[⟨F,j⟩−Σ˙ω(j)]V_0(x_{\mathrm{init}}) = -T\widehat{L}^{\dagger}_{\omega}(0) = T\max_{Sj=0}\left[\langle F,j\rangle- \dot{\Sigma}_{\omega}(j)\right], which is equivalent to the maximization of the steady flux with the EPR cost.

A similar result is also obtained by minimizing the entropy production under the constraint that the time-average total activity budget is bounded:

1T∫0Tdt′⟨1,ωt′(xt′)⟩≤Ω(71)\frac{1}{T}\int_0^T \mathrm{d}t'\langle1,\omega_{t'}(x_{t'})\rangle\le\Omega \tag*{(71)}

The same argument applies as before; the value function becomes

Vt(x,m)=−mL^†(xend−xm).(72)V_t(x,m)=-m\hat{\mathcal{L}}^\dagger\left(\frac{x_{end}-x}{m}\right). \tag*{(72)}

where, for h(u)=uψm−1(u)h(u)=u\psi_m^{-1}(u),

L^†(v):={min⁡v=Sξ[h(∥ξ∥1)−⟨F,ξ⟩].if v∈Im⁡S+∞otherwise(73)\hat{\mathcal{L}}^\dagger(v):= \begin{cases} \min_{v=S\xi}\left[h\left(\lVert\xi\rVert_1\right)-\langle F,\xi\rangle\right]. & \text{if }v\in\operatorname{Im}S\\ +\infty& \text{otherwise} \end{cases} \tag*{(73)}

and m≥0m\ge0 is the remaining activity budget at tt whose initial condition at t=0t=0 is m0=TΩm_0=T\Omega. If we reduce the result to the conventional transport problem without flux utility, i.e., F=0F=0, then the value function is further simplified as

Vt(x,m)=−mh(VS(xend−x)m)(74)V_t(x,m)=-mh\left(\frac{V_S(x_{end}-x)}{m}\right) \tag*{(74)}

where

VS(v):={min⁡ξ:Sξ=v∥ξ∥1if v∈Im⁡S+∞otherwise.(75)V_S(v):= \begin{cases} \min_{\xi:S\xi=v}\lVert\xi\rVert_1 & \text{if }v\in\operatorname{Im}S\\ +\infty& \text{otherwise.} \end{cases} \tag*{(75)}

is a norm on Im⁡S\operatorname{Im}S. This situation has been investigated for the arithmetic mean [52], the logarithmic mean [53], and the general means [54]. If this result is applied to MJP where SS is replaced with the graph incidence matrix B\mathbb{B}, W(x,x′):=VS(x′−x)W(x,x'):=V_S(x'-x) becomes the graph L1L^1-Wasserstein distance.

Since optimizing ω\omega under this constraint complicates the derivation of the HJB and Pontryagin’s equations, we defer the details to Appendix.

Optimal thermodynamic transports with xx–dependent activity

If ω\omega is xx-dependent, the dynamics of force (Eq. (67)) is no longer constant. Muratore-Ginanneschi et al. in their pioneering work, investigated the optimal thermodynamic transport over MJP with xx-dependent ω(x)\omega(x) where ω(x)\omega(x) is specified by the geometric mean of MA linear kinetics [79]. Remlein and Seifert employed this result to show that non-conservative force is optimal under this choice of activity [80]. This choice can be interpreted as defining an activity field based on uncontrolled MA kinetics as a reference. While an analytic expression of the value function and the optimal trajectory cannot be obtained for this xx-dependent activity in general, we can immediately obtain the main results of [79] for the generalized mean via our formulation. Specifically, the optimal force condition (Eq. (52) in [79]), the dynamics of auxiliary field (Eq. (54) in [79]), and the dynamics of optimal force (Eq. (56) in [79]) are equivalent to Eq. (59), Eq. (63), and Eq. (65), respectively (See Appendix for the detailed comparison).

In addition, the non-conservativeness of the optimal thermodynamic force for generalized means other than the logarithmic one[80] is a consequence of Eq. (59), which means that the optimal force ft†(x)f_t^{\dagger}(x) is a nonlinear function of the gradient of the value function ft†(x)+ψm(ft†(x))ψm′(ft†(x))=S⊤∂Vt(x)∂xf_t^{\dagger}(x) + \frac{\psi_m(f_t^{\dagger}(x))}{\psi_m'(f_t^{\dagger}(x))} = \mathrm{S}^{\top}\frac{\partial V_t(x)}{\partial x} and thus non-conservative.15 Dechant & Meer proposed to replace the EPR, Σ˙ω(x)(j)\dot{\Sigma}_{\omega(x)}(j), with twice the dissipation function 2Ψω(x)(j)2\Psi_{\omega(x)}(j) as the control cost to recover the conservativeness of the optimal force[93], the underlying idea of which is close to the variational representation of thermodynamic flux[82]. In contrast, our result clarified that the optimal pseudo-force is always the gradient of the value function for F=0\mathbf{F} = \mathbf{0} as ηt†(x)=S⊤∂Vt(x)∂x\eta_t^{\dagger}(x) = \mathrm{S}^{\top}\frac{\partial V_t(x)}{\partial x}, which shares the same dynamical property as the equilibrium dynamics(Eq. (26)). Since the transient dynamics has non-negative EPR even if driven by a conservative force, the generalized gradient flow property is more fundamental for characterizing the optimal thermodynamic transport dynamics. (See Appendix for more detailed comparison and discussion).

In contrast to the geometric mean, the logarithmic mean is not compatible with xx-dependent activity because ω(x)\omega(x) becomes a complicated function even if the simplest linear kinetics is employed.

Conclusion and Discussion

In this work, we have established a thermodynamic optimal control theory for CRNs that aligns with their underlying thermodynamic and geometric structures. By treating reaction rate constants as control variables and incorporating both state and flux utilities alongside the EPR cost, this framework encompasses conventional thermodynamic transport far from equilibrium as well as a broader class of nonequilibrium control tasks.

Our central result clarifies that the optimal dynamics derived from the thermodynamic Hamilton–Jacobi–Bellman (HJB) equation takes the form of the perturbed generalized gradient flow driven by the value function. This characterization stems from the convex representation of the EPR function via generalized means, where its convex conjugate naturally acts as a nonlinear generalization of the friction metric. The resulting dynamics shares the thermodynamic structure of equilibrium and nonequilibrium CRNs and admits an information-geometric characterization. Furthermore, we derive a thermodynamic Pontryagin equation by recasting the Pontryagin Hamiltonian system using thermodynamic forces as the control variables. This formulation highlights the intrinsic thermodynamic and geometric structure of the control problem and clarifies its direct connections to optimal transport and related approaches.

These results lay a theoretical foundation for the thermodynamic control of CRNs, with potential applications to biological systems and finite-time thermodynamics. In addition, they pave the way for addressing the following challenges.

From transport to time-cyclic control

The first important problem for future work is the design of optimal thermodynamic cycles. Although thermodynamic transport between two prescribed state distributions at minimal thermodynamic cost is a basic building block of thermodynamic control, constructing optimal thermodynamic engines requires identifying time-periodic control trajectories that balance dissipation minimization with utility maximization[94, 95]. Within near-equilibrium thermodynamic geometry, this task can be formulated as a geometric problem to a finding optimal closed curve on a manifold through an isoperimetric formulation[96, 97, 98]. Extensions to far-from-equilibrium regimes have also been explored[99, 100, 101, 102].

Building upon the framework developed here, a natural next step is to investigate this problem from a geometric perspective, establish conditions for the existence of optimal periodic trajectories, and characterize their thermodynamic properties in more general settings. In parallel, practical numerical schemes for computing optimal cycles should be developed, leveraging the rich toolkit of modern control theory.

From deterministic to stochastic control

A second challenge lies in extending our framework to stochastic optimal control. The conventional thermodynamic transport and control based on thermodynamic or Wasserstein geometry fall within the class of deterministic optimal control, as they treat control problems for probability distributions or densities governed by deterministic evolution equations, i.e., the Fokker–Planck and master equations. Our results for CRNs, which generalize these approaches, belong likewise to this category. Owing to the deterministic nature, an experimental implementation of such control strategies does not require feedback control via observing the systems[51].

However, recent advances in single-molecule techniques, such as imaging and manipulation, enable us to access individual stochastic trajectories and directly perturb their dynamics[103, 104, 105]. Bridging this gap between theory and experiment calls for extending our framework from deterministic to stochastic optimal control. Adapting the established theory of stochastic optimal control[81, 106] to thermodynamic settings therefore represents a crucial frontier for future research[107].

From nonautonomous to autonomous control

Third, although external control is the standard setup for studying thermodynamic systems, real heat engines and living systems operate autonomously[108, 109]. Designing autonomous heat engines and understanding the thermodynamics of living systems therefore require identifying optimal control laws implemented within the systems themselves[110, 111].

Automatic differentiation and learning theory have been employed to obtain such autonomous systems[95, 112, 113]. Among others, reinforcement learning (RL), which builds on optimal control theory, provides methods for formulating and optimizing time-invariant feedback policies[114]. The integration of RL with our theory would provide a systematic framework for autonomous thermodynamic control[115].

Applications to physics of intelligence

Finally, the growing interest in the physics of intelligence, spurred in part by advances in artificial intelligence, has stimulated efforts to employ stochastic thermodynamics and related methods to information science[116, 117, 118, 119, 120, 121]. For example, recent studies have drawn on stochastic thermodynamics to assess the performance of generative diffusion models[122]. These models exploit time reversal to learn generative dynamics, while optimal control theory offers a more general formulation by seeking control laws that generate reverse-time trajectories. This control-theoretic perspective is applied to state-of-the-art diffusion models and their fine-tuning. Extending the theoretical framework developed here beyond thermodynamics to diverse problems in information science could thus open new avenues for physics of intelligence.

Acknowledgement

We used OpenAI’s ChatGPT for literature review, scientific discussions, and English editing and proofreading. All AI-assisted content was independently verified by the authors, who take full responsibility for the manuscript.

We thank Praful Gagrani, Dimitri Loutchko, and Keisuke Sugie for fruitful discussion and comments. This work was supported by JST CREST (Grant Numbers JPMJCR25Q2), JST PRESTO (Grant No. JPMJPR25KB), and JSPSKAKENHI (Grant Number 25H01365).

References

  1. [1]J. McGrath, J. Clarke, C. Johnson, and J. Alvarado, Control across scales: Signals, informa- tion, and adaptive biological mechanical function, Biophysics Rev. 7, 021309 (2026).arxiv.org/abs/2509.03418
  2. [2]A. Datta, P. Pietzonka, and A. C. Barato, Second Law for Active Heat Engines, Phys. Rev. X 12, 031034 (2022).DOI
  3. [3]A. Sahin, D. R. Weilandt, and V. Hatzimmanikatis, Optimal enzyme utilization suggests that concentrations and thermodynamics determine binding mechanisms and enzyme saturations, Nat Commun 14, 2618 (2023).DOI
  4. [4]D. H. Wolpert, J. Korbel, C. W. Lynn, F. Tasnim, J. A. Grochow, G. Kardes, J. B. Aimone, V. Balasubramanian, E. De Giuli, D. Doty, N. Freitas, M. Marsili, T. E. Ouldridge, A. W. Richa, P. Riechers, É. Roldán, B. Rubenstein, Z. Toroczkai, and J. Paradiso, Is stochastic thermodynamics the key to understanding the energy costs of computation?, Proceedings of the National Academy of Sciences 121, e2321112121 (2024).DOI
  5. [5]K. Sekimoto, Langevin Equation and Thermodynamics, Prog Theor Phys 130, 17 (1998).DOI
  6. [6]L. Peliti and S. Pigolotti, Stochastic Thermodynamics: An Introduction (Princeton Univer- sity Press, 2021).
  7. [7]U. Seifert, Stochastic Thermodynamics (Cambridge University Press, Cambridge, 2025).
  8. [8]F. Avanzini, M. Bilancioni, V. Cavina, S. Dal Cengio, M. Esposito, G. Falasco, D. Forastiere, J. N. Freitas, A. Garilli, P. E. Harunari, V. Lecomte, A. Lazarescu, S. G. Marehallis Srinivas, C. Moslonka, I. Neri, E. Penocchio, W. D. Piñeros, M. Polettini, A. Raghu, P. Raux, K. Sekimoto, and A. Soret, Methods and conversations in (post)modern thermodynamics, SciPost Physics Lecture Notes , 080 (2024).DOI
  9. [9]F. L. Curzon and B. Ahlborn, Efficiency of a Carnot engine at maximum power output, Am. J. Phys. 43, 22 (1975).DOI
  10. [10]B. Andresen and P. Salamon, The First Fifty Years of Finite-Time Thermodynamics, Entropy 28, 49 (2026).DOI
  11. [11]S. Deffner and M. V. S. Bonança, Thermodynamic control —An old paradigm with new applications, EPL 131, 20001 (2020).
  12. [12]S. Blaber and D. A. Sivak, Optimal control in stochastic thermodynamics, J. Phys. Commun. 7, 033001 (2023).DOI
  13. [13]J. Alvarado, E. G. Teich, D. A. Sivak, and J. Bechhoefer, Optimal Control in Soft and Active Matter, Annual Review of Condensed Matter Physics 17, 327 (2026).DOI
  14. [14]G. E. Crooks, Measuring Thermodynamic Length, Phys. Rev. Lett. 99, 100602 (2007).DOI
  15. [15]D. A. Sivak and G. E. Crooks, Thermodynamic Metrics and Optimal Paths, Phys. Rev. Lett. 108, 190602 (2012).DOI
  16. [16]P. R. Zulkowski, D. A. Sivak, G. E. Crooks, and M. R. DeWeese, Geometry of thermodynamic control, Phys. Rev. E 86, 041148 (2012).
  17. [17]P. Salamon, B. Andresen, P. D. Gait, and R. S. Berry, The significance of Weinhold’s length, J. Chem. Phys. 73, 1001 (1980).DOI
  18. [18]P. Salamon and R. S. Berry, Thermodynamic Length and Dissipated Availability, Phys. Rev. Lett. 51, 1127 (1983).DOI
  19. [19]B. Andresen, Finite-time thermodynamics and thermodynamic length, Revue Générale de Thermique 35, 647 (1996).DOI
  20. [20]Y.-H. Ma and X.-H. Zhao, Finite-time thermodynamics: A journey beginning with optimizing heat engines (2024).DOI
  21. [21]B. Andresen, Current Trends in Finite-Time Thermodynamics, Angewandte Chemie International Edition 50, 2690 (2011).DOI
  22. [22]B. Andresen, P. Salamon, and R. S. Berry, Thermodynamics in finite time: Extremals for imperfect heat engines, J. Chem. Phys. 66, 1571 (1977).DOI
  23. [23]B. Andresen, R. S. Berry, A. Nitzan, and P. Salamon, Thermodynamics in finite time. I. The step-Carnot cycle, Phys. Rev. A 15, 2086 (1977).DOI
  24. [24]P. Salamon, B. Andresen, and R. S. Berry, Thermodynamics in finite time. II. Potentials for finite-time processes, Phys. Rev. A 15, 2094 (1977).DOI
  25. [25]F. Weinhold, Metric geometry of equilibrium thermodynamics, The Journal of Chemical Physics 63, 2479 (1975).DOI
  26. [26]G. Ruppeiner, Thermodynamics: A Riemannian geometric model, Phys. Rev. A 20, 1608 (1979).DOI
  27. [27]Z. Li and Y. Izumida, Decomposition of metric tensor in thermodynamic geometry in terms of relaxation timescales, Phys. Rev. E 111, 034113 (2025).DOI
  28. [28]A. Plastino, A thermodynamic-geometric view of irreversibility, Mod. Phys. Lett. A 41, 2650123 (2026).DOI
  29. [29]G. Watanabe, G.-H. Xu, and Y. Minami, Unified geometric formalism for dissipation and its fluctuations in finite-time microscopic heat engines (2026), arXiv:2604.04620 [cond-mat.stat-mech].DOI
  30. [30]W. C. Wareham and D. A. Sivak, Multiparameter optimal control of {\mathrm{F}}_{-1}-ATPase, Phys. Rev. E 111, 064410 (2025).DOI
  31. [31]P. R. Zulkowski, D. A. Sivak, and M. R. DeWeese, Optimal Control of Transitions between Nonequilibrium Steady States, PLOS ONE 8, e82754 (2013).DOI
  32. [32]D. Kamp and K. Proesmans, Thermodynamic control of nonequilibrium systems, Phys. Rev. E 113, 014140 (2026).DOI
  33. [33]J. R. Sawchuk and D. A. Sivak, Global thermodynamic manifold for conservative control of stochastic systems, Phys. Rev. Res. 8, 013004 (2026).DOI
  34. [34]T. Matsuo and A. Sonoda, Analysis of entropy production in finitely slow processes between nonequilibrium steady states, Phys. Rev. E 106, 064119 (2022).DOI
  35. [35]Z. Wang, Thermodynamic Geometry of Nonequilibrium Fluctuations in Cyclically Driven Transport, Phys. Rev. Lett. 132, 10.1103/PhysRevLett.132.207101 (2024).DOI
  36. [36]D. Guéry-Odelin, A. Ruschhaupt, A. Kiely, E. Torrontegui, S. Martínez-Garaot, and J. G. Muga, Shortcuts to adiabaticity: Concepts, methods, and applications, Rev. Mod. Phys. 91, 045001 (2019).
  37. [37]A. Patra and C. Jarzynski, Shortcuts to adiabaticity using flow fields, New J. Phys. 19, 125009 (2017).
  38. [38]D. Guéry-Odelin, C. Jarzynski, C. A. Plata, A. Prados, and E. Trizac, Driving rapidly while remaining in control: Classical shortcuts from Hamiltonian to stochastic dynamics, Rep. Prog. Phys. 86, 035902 (2023).
  39. [39]E. İlker, Ö. Güngör, B. Kuznets-Speck, J. Chiel, S. Deffner, and M. Hinczewski, Shortcuts in Stochastic Systems and Control of Biophysical Processes, Phys. Rev. X 12, 021048 (2022).
  40. [40]E. Aurell, C. Mejía-Monasterio, and P. Muratore-Ginanneschi, Optimal Protocols and Optimal Transport in Stochastic Thermodynamics, Phys. Rev. Lett. 106, 250601 (2011).pubmed.ncbi.nlm.nih.gov/21770620
  41. [41]P. Muratore-Ginanneschi, On the use of stochastic differential geometry for non-equilibrium thermodynamic modeling and control, J. Phys. A: Math. Theor. 46, 275002 (2013).
  42. [42]E. Aurell, K. Gawędzki, C. Mejía-Monasterio, R. Mohayaee, and P. Muratore-Ginanneschi, Refined Second Law of Thermodynamics for Fast Random Processes, J Stat Phys 147, 487 (2012).DOI
  43. [43]Y. Chen, T. T. Georgiou, and A. Tannenbaum, Stochastic Control and Nonequilibrium Thermodynamics: Fundamental Limits, IEEE Transactions on Automatic Control 65, 2979 (2020).DOI
  44. [44]M. Nakazato and S. Ito, Geometrical aspects of entropy production in stochastic thermodynamics based on Wasserstein distance, Phys. Rev. Res. 3, 043093 (2021).DOI
  45. [45]O. Movilla Miangolarra, A. Taghvaei, Y. Chen, and T. T. Georgiou, Geometry of Finite-Time Thermodynamic Cycles With Anisotropic Thermal Fluctuations, IEEE Control Systems Letters 6, 3409 (2022).arxiv.org/abs/2203.12483
  46. [46]S. Chennakesavalu and G. M. Rotskoff, Unified, Geometric Framework for Nonequilibrium Protocol Optimization, Phys. Rev. Lett. 130, 107101 (2023).DOI
  47. [47]S. Ito, Geometric thermodynamics for the Fokker–Planck equation: Stochastic thermodynamic links between information geometry and optimal transport, Info. Geo. 7, 441 (2024).DOI
  48. [48]T. Kamijima, A. Takatsu, K. Funo, and T. Sagawa, Optimal finite-time Maxwell’s demons in Langevin systems, Phys. Rev. Res. 7, 023159 (2025).arxiv.org/abs/2410.11603
  49. [49]J. Klinger and G. M. Rotskoff, Universal energy-speed-accuracy trade-offs in driven nonequilibrium systems, Phys. Rev. E 111, 014114 (2025).DOI
  50. [50]K. Tojo, R. Nagase, K. Funo, and T. Sagawa, Optimizing optimal transport: Role of final distributions in finite-time thermodynamics (2025), arXiv:2509.11314 [cond-mat.stat-mech].arxiv.org/abs/2509.11314
  51. [51]S. Oikawa, Y. Nakayama, S. Ito, T. Sagawa, and S. Toyabe, Experimentally achieving minimal dissipation via thermodynamically optimal transport, Nat Commun 16, 10424 (2025).DOI
  52. [52]A. Dechant, Minimum entropy production, detailed balance and Wasserstein distance for continuous-time Markov processes, J. Phys. A: Math. Theor. 55, 094001 (2022).DOI
  53. [53]T. Van Vu and K. Saito, Thermodynamic Unification of Optimal Transport: Thermodynamic Uncertainty Relation, Minimum Dissipation, and Thermodynamic Speed Limits, Phys. Rev. X 13, 011013 (2023).DOI
  54. [54]R. Nagayama, K. Yoshimura, and S. Ito, Infinite variety of thermodynamic speed limits with general activities, Phys. Rev. Res. 7, 013307 (2025).DOI
  55. [55]R. Nagayama, Geometric thermodynamics of reaction-diffusion systems: Thermodynamic trade-off relations and optimal transport for pattern formation, Phys. Rev. Res. 7, 10.1103/PhysRevResearch.7.033011 (2025).DOI
  56. [56]J. van der Meer and A. Dechant, Near-optimality of conservative driving in discrete systems (2026), arXiv:2602.18321 [cond-mat].arxiv.org/abs/2602.18321
  57. [57]A. I. Brown and D. A. Sivak, Allocating dissipation across a molecular machine cycle to maximize flux, Proceedings of the National Academy of Sciences 114, 11057 (2017).DOI
  58. [58]D. Andrieux, Minimal entropy production under thermodynamic constraints: An application of cycle decomposition for Markov chain design (2023), arXiv:2303.05930 [cond-mat.stat-mech].arxiv.org/abs/2303.05930
  59. [59]Y.-J. Yang and K. A. Dill, A principled basis for nonequilibrium network flows, Nat Commun 10.1038/s41467-026-71239-9 (2026).DOI
  60. [60]T. Schmiedl and U. Seifert, Efficiency at maximum power: An analytically solvable model for stochastic heat engines, EPL 81, 20003 (2007).DOI
  61. [61]R. M. T. Fleming, C. M. Maes, M. A. Saunders, Y. Ye, and B. Ø. Palsson, A variational principle for computing nonequilibrium fluxes and potentials in genome-scale biochemical networks, Journal of Theoretical Biology 292, 71 (2012).arxiv.org/abs/1105.1513
  62. [62]P. Gagrani, N. Lauber, E. Smith, and C. Flamm, Thermodynamic ranking of pathways in reaction networks, J. Stat. Mech. 2025, 123208 (2026).
  63. [63]J. Piñero, R. Solé, and A. Kolchinsky, Optimization of nonequilibrium free energy harvesting illustrated on bacteriorhodopsin, Phys. Rev. Res. 6, 013275 (2024).
  64. [64]M. Bilancioni and M. Esposito, Gears in chemical reaction networks for optimizing energy transduction efficiency, Nat Commun 16, 5765 (2025).DOI
  65. [65]J. M. Horowitz, K. Zhou, and J. L. England, Minimum energetic cost to maintain a target nonequilibrium state, Phys. Rev. E 95, 042102 (2017).DOI
  66. [66]A. E. Bryson, Applied Optimal Control: Optimization, Estimation and Control (CRC Press, 1975).
  67. [67]D. Liberzon, Calculus of Variations and Optimal Control Theory: A Concise Introduction (Princeton University Press, 2011).
  68. [68]D. P. Bertsekas, Dynamic Programming and Optimal Control: Notes, Sources, and Exercises, p.71 (Athena Scientific, 2012).
  69. [69]T. Schmiedl and U. Seifert, Optimal Finite-Time Processes In Stochastic Thermodynamics, Phys. Rev. Lett. 98, 108301 (2007).DOI
  70. [70]S. Bo, E. Aurell, R. Eichhorn, and A. Celani, Optimal stochastic transport in inhomogeneous thermal environments, EPL 103, 10010 (2013).
  71. [71]A. Zhong and M. R. DeWeese, Limited-control optimal protocols arbitrarily far from equilibrium, Phys. Rev. E 106, 044135 (2022).DOI
  72. [72]S. Monter, S. A. M. Loos, and C. Bechinger, Optimal transitions between nonequilibrium steady states, Proceedings of the National Academy of Sciences 122, e2510654122 (2025).DOI
  73. [73]Y. Kurokawa and Y. Hasegawa, Optimal Control to Minimize Dissipation and Fluctuations in Open Quantum Systems Beyond Slow and Rapid Regimes (2026), arXiv:2602.01688 [quant-ph].arxiv.org/abs/2602.01688
  74. [74]J. Meibohm, S. Monter, S. A. M. Loos, and C. Bechinger, Finite-time transitions in optimal control and non-equilibrium relaxation (2026), arXiv:2604.25798 [cond-mat.stat-mech].arxiv.org/abs/2604.25798
  75. [75]E. Panizon, Optimal Control of a Mesoscopic Information Engine (2026), arXiv:2603.29804 [cond-mat.stat-mech].arxiv.org/abs/2603.29804
  76. [76]M. Pavon, Stochastic control and nonequilibrium thermodynamical systems, Appl Math Optim 19, 187 (1989).DOI
  77. [77]Y. Zhang, Q. Ouyang, and Y. Tu, Geometry of Optimal Control in Chemical Reaction Networks in the Adiabatic Limit, Phys. Rev. Lett. 134, 018001 (2025).DOI
  78. [78]A. T. Mohite and H. Rieger, Generalized Finite-time Optimal Control Framework in Stochas- tic Thermodynamics (2026), arXiv:2511.00974 [cond-mat.stat-mech].DOI
  79. [79]P. Muratore-Ginanneschi, C. Mejía-Monasterio, and L. Peliti, Heat Release by Controlled Continuous-Time Markov Jump Processes, J Stat Phys 150, 181 (2013).
  80. [80]B. Remlein and U. Seifert, Optimality of nonconservative driving for finite-time processes with discrete states, Phys. Rev. E 103, L050105 (2021).DOI
  81. [81]S. A. Horiguchi and T. J. Kobayashi, Optimal Control of Stochastic Reaction Networks with Entropic Control Cost and Emergence of Mode-Switching Strategies, PRX Life 3, 033027 (2025).
  82. [82]T. J. Kobayashi, D. Loutchko, A. Kamimura, S. A. Horiguchi, and Y. Sugiyama, Information geometry of dynamics on graphs and hypergraphs, Info. Geo. 7, 97 (2024).arxiv.org/abs/2211.14455
  83. [83]Y. Sugiyama, D. Loutchko, A. Kamimura, and T. J. Kobayashi, Hessian geometric structure of chemical thermodynamic systems with stoichiometric constraints, Phys. Rev. Research 4, 033065 (2022).
  84. [84]T. J. Kobayashi, D. Loutchko, A. Kamimura, and Y. Sugiyama, Kinetic derivation of the Hessian geometric structure in chemical reaction networks, Phys. Rev. Research 4, 033066 (2022).DOI
  85. [85]T. J. Kobayashi, D. Loutchko, A. Kamimura, and Y. Sugiyama, Hessian geometry of nonequilibrium chemical reaction networks and entropy production decompositions, Phys. Rev. Research 4, 033208 (2022).DOI
  86. [86]R. Rao and M. Esposito, Nonequilibrium Thermodynamics of Chemical Reaction Networks: Wisdom from Stochastic Thermodynamics, Phys. Rev. X 6, 041064 (2016).DOI
  87. [87]C. Maes, Local detailed balance, SciPost Physics Lecture Notes , 032 (2021).DOI
  88. [88]K. Yoshimura, Housekeeping and excess entropy production for general nonlinear dynamics, Phys. Rev. Res. 5, 10.1103/PhysRevResearch.5.013017 (2023).DOI
  89. [89]C. Maes, Non-Dissipative Effects in Nonequilibrium Systems, SpringerBriefs in Complexity (Springer International Publishing, Cham, 2018).
  90. [90]K. Yoshimura, Thermodynamic Uncertainty Relation and Thermodynamic Speed Limit in Deterministic Chemical Reaction Networks, Phys. Rev. Lett. 127, 10.1103/PhysRevLett.127.160601 (2021).DOI
  91. [91]N. Shiraishi, Optimal Thermodynamic Uncertainty Relation in Markov Jump Processes, J Stat Phys 185, 19 (2021).DOI
  92. [92]N. Shiraishi, Wasserstein distance in speed limit inequalities for Markov jump processes, J. Stat. Mech. 2024, 074003 (2024).DOI
  93. [93]A. Dechant and J. van der Meer, Renormalized entropy production for optimal transport in jump processes: Make conservative forces optimal again (2026), arXiv:2605.00656 [cond-mat.stat-mech].arxiv.org/abs/2605.00656
  94. [94]K. Brandner, K. Saito, and U. Seifert, Thermodynamics of Micro- and Nano-Systems Driven by Periodic Temperature Variations, Phys. Rev. X 5, 031019 (2015).
  95. [95]W. C. Wareham and D. A. Sivak, Optimal Control of Periodic Nonequilibrium Mechanochemical Systems via Automatic Differentiation (2026), arXiv:2609.03217 [cond-mat.stat-mech].arxiv.org/abs/2609.03217
  96. [96]K. Brandner and K. Saito, Thermodynamic Geometry of Microscopic Heat Engines, Phys. Rev. Lett. 124, 040602 (2020).DOI
  97. [97]P. Abiuso and M. Perarnau-Llobet, Optimal Cycles for Low-Dissipation Heat Engines, Phys. Rev. Lett. 124, 110606 (2020).DOI
  98. [98]A. G. Frim and M. R. DeWeese, Geometric Bound on the Efficiency of Irreversible Thermodynamic Cycles, Phys. Rev. Lett. 128, 230601 (2022).DOI
  99. [99]O. Movilla Miangolarra, A. Taghvaei, R. Fu, Y. Chen, and T. T. Georgiou, Energy harvesting from anisotropic fluctuations, Phys. Rev. E 104, 044101 (2021).DOI
  100. [100]O. Movilla Miangolarra, A. Taghvaei, Y. Chen, and T. T. Georgiou, Geometry of Finite-Time Thermodynamic Cycles With Anisotropic Thermal Fluctuations, IEEE Control Systems Letters 6, 3409 (2022).arxiv.org/abs/2203.12483
  101. [101]J. Ventura Siches, O. Movilla Miangolarra, and T. T. Georgiou, Refined bounds on energy harvesting from anisotropic fluctuations, Phys. Rev. E 109, 064155 (2024).arxiv.org/abs/2402.01973
  102. [102]A. Zhong, A. G. Frim, and M. R. DeWeese, Optimal active engines obey the thermodynamic Lorentz force law, Phys. Rev. E 114, L042106 (2026).DOI
  103. [103]S. Toyabe, T. Sagawa, M. Ueda, E. Muneyuki, and M. Sano, Experimental demonstration of information-to-energy conversion and validation of the generalized Jarzynski equality, Nature Phys 6, 988 (2010).DOI
  104. [104]M. Ribezzi-Crivellari and F. Ritort, Large work extraction and the Landauer limit in a continuous Maxwell demon, Nat. Phys. 15, 660 (2019).
  105. [105]T. K. Saha, J. N. E. Lucero, J. Ehrich, D. A. Sivak, and J. Bechhoefer, Bayesian Information Engine that Optimally Exploits Noisy Measurements, Phys. Rev. Lett. 129, 130601 (2022).DOI
  106. [106]W. Fleming and R. Rishel, Deterministic and Stochastic Optimal Control (Springer, New York, NY, 1975).
  107. [107]L. F. Reinalter, E. Panizon, L. Muruga, and C. Bechinger, Work Extraction via Backward Motion in Optimal Closed-Loop Stochastic Control (2026), arXiv:2606.17624 [cond-mat.stat-mech].arxiv.org/abs/2606.17624
  108. [108]M. Serra-Garcia, A. Foehr, M. Molerón, J. Lydon, C. Chong, and C. Daraio, Mechanical Autonomous Stochastic Heat Engine, Phys. Rev. Lett. 117, 010602 (2016).
  109. [109]C. Jarzynski, S. Deffner, and S. Rahav, Fluctuation theorems for autonomous work, Proceedings of the National Academy of Sciences 122, e2524775122 (2025).
  110. [110]X. Yang, M. Heinemann, J. Howard, G. Huber, S. Iyer-Biswas, G. Le Treut, M. Lynch, K. L. Montooth, D. J. Needleman, S. Pigolotti, J. Rodenfels, P. Ronceray, S. Shankar, I. Tavassoly, S. Thutupalli, D. V. Titov, J. Wang, and P. J. Foster, Physical bioenergetics: Energy fluxes, budgets, and constraints in cells, Proceedings of the National Academy of Sciences 118, e2026786118 (2021).DOI
  111. [111]S. Thutupalli, The cost of cellular patterns, Nat. Phys. 20, 1704 (2024).DOI
  112. [112]M. C. Engel, J. A. Smith, and M. P. Brenner, Optimal Control of Nonequilibrium Systems through Automatic Differentiation, Phys. Rev. X 13, 041032 (2023).arxiv.org/abs/2201.00098
  113. [113]J. Yan, H. Touchette, and G. M. Rotskoff, Learning nonequilibrium control forces to characterize dynamical phase transitions, Phys. Rev. E 105, 024115 (2022).DOI
  114. [114]R. S. Sutton and A. G. Barto, Reinforcement Learning, Second Edition: An Introduction (MIT Press, 2018).
  115. [115]S. Whitelam, Demon in the Machine: Learning to Extract Work and Absorb Entropy from Fluctuating Nanosystems, Phys. Rev. X 13, 021005 (2023).DOI
  116. [116]N. X. Kodama and M. Hinczewski, Thermodynamic Performance Limits for Score-Based Diffusion Models (2025), arXiv:2510.06174 [cs.LG].arxiv.org/abs/2510.06174
  117. [117]S. H. D’Ambrosia, S. M. Daniels, M. R. DeWeese, and A. Sahai, The Thermodynamic Costs of Simple Linear Regression, IEEE Journal on Selected Areas in Information Theory 7, 458 (2026).arxiv.org/abs/2605.19195
  118. [118]T. Sagawa, Stochastic Thermodynamics for Autoregressive Generative Models: A Non-Markovian Perspective (2026), arXiv:2604.07867 [cond-mat.stat-mech].DOI
  119. [119]X. Ding, H. T. Quan, and Y. Tu, Stochastic Thermodynamics of Score Matching in Diffusion Models (2026), arXiv:2606.17252 [cond-mat.dis-nn].arxiv.org/abs/2606.17252
  120. [120]T. Aslyamov and M. Esposito, Physical limits to stochastic classifiers (2026), arXiv:2610.20481 [cond-mat.stat-mech].arxiv.org/abs/2610.20481
  121. [121]Y. Hasegawa, Thermodynamics of classifiers (2026), arXiv:2605.24365 [cond-mat.stat-mech].arxiv.org/abs/2605.24365
  122. [122]K. Ikeda, T. Uda, D. Okanohara, and S. Ito, Speed-Accuracy Relations for Diffusion Models: Wisdom from Nonequilibrium Thermodynamics and Optimal Transport, Phys. Rev. X 15, 031031 (2025).DOI

Paper details

vlnc:W_6ugw5vym
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

OpenAI’s ChatGPT

View statement
Looking for related papers…
Contents