WorksPhysics and Astronomy
Optimal Control of Chemical Reaction Networks with Thermodynamic Cost
Abstract
Thermodynamic control seeks to realize desired functions at minimal thermodynamic cost, yet existing frameworks remain largely confined to slow driving regimes or state-to-state transport. Here we establish a thermodynamic optimal-control theory for Markov jump processes (MJPs) and chemical reaction networks (CRNs) that accommodates objectives over both states and reaction fluxes beyond slow-driving limit. By deriving the thermodynamic Hamilton–Jacobi–Bellman equation, we demonstrate that the optimal dynamics take the form of a perturbed generalized-gradient flow driven by the value function. A convex representation of the entropy-production rate via generalized means identifies its Legendre–Fenchel dual as the dissipation function. The resulting dynamics shares the thermodynamic structure of CRNs and admits an information-geometric characterization. We further derive a thermodynamic Pontryagin equation by recasting the control problem in terms of thermodynamic forces, providing a complementary description of optimal trajectories and clarifying connections to thermodynamic optimal transport. Our results establish a thermodynamic and geometric foundation for controlling MJPs and CRNs beyond mere transport, opening avenues for optimizing biological processes and finite-time thermodynamic operations in far-from-equilibrium regimes.
Introduction
Challenge in Thermodynamics
A central challenge in thermodynamics is designing protocols that drive systems to perform desired functions at minimal thermodynamic cost. This challenge spans diverse applications, from engineering efficient heat engines to elucidating the design principles of biochemical reaction networks and the fundamental physical limits of computation [1, 2, 3, 4]. Recent advances in stochastic thermodynamics [5, 6, 7, 8] have overcome the limitations of classical finite-time thermodynamics [9, 10] by providing a framework for quantifying work, heat, and entropy production under externally driven operations at both the distribution and trajectory levels.1
Building on this theoretical foundation, extensive efforts have been dedicated to characterizing thermodynamic bounds and identifying optimal protocols for canonical tasks, such as trajectory tracking and state-to-state transitions, under various constraints and assumptions [11, 12, 13].
Near-equilibrium thermodynamic control and thermodynamic geometry
In the near-equilibrium regime where linear response theory holds, slowly driven finite-time protocols that minimize dissipation correspond to geodesics on a Riemannian manifold endowed with a generalized friction metric [14]. This geometric formulation casts the search for optimal, minimum-dissipation protocols into a problem governed by the thermodynamic metric [15, 16].
Since its inception[17, 18, 19] via the integration of finite-time thermodynamics[20, 21, 22, 23, 24] and Weinhold–Ruppeiner equilibrium geometry[25, 26], the geometric approach has been refined theoretically [27, 28, 29] and applied to a diverse range of model systems [12, 30]. Moreover, leveraging the Hatano–Sasa excess heat, the scope of the framework has been expanded from a near-equilibrium tracking problem to transitions between non-equilibrium steady states [31], evolving into non-equilibrium quasi-static control [32, 33, 34, 35].
Additionally, shortcuts to adiabaticity or to isothermality have been developed, which realize a target sequence of equilibrium or steady states in finite time by introducing auxiliary controls [36, 37, 38, 39].
Far from equilibrium transport and its extensions
While powerful, these frameworks are restricted to near-equilibrium or steady-state regimes. Two pioneering studies explored possible ways to resolve this difficulty[40, 41]; Notably, the work by Aurell et al., which showed that finding a transport protocol with minimizing excess work and heat can be mapped to an optimal transport problem [40, 42], established a theoretical foundation for far-from-equilibrium transport. After a gap of about a decade since its proposal, theoretical and experimental efforts have rapidly accumulated in recent years, particularly for overdamped Langevin systems [43, 44, 45, 46, 47, 48, 49, 50, 51]. These insights have subsequently been extended to discrete Markov Jump Processes (MJPs) and chemical reaction networks (CRNs)[52, 53, 54, 55, 56]. Moreover, optimal transport offers a Wasserstein-geometric perspective that allows the derivation of various bounds in general thermodynamic settings.
Nevertheless, existing approaches remain limited to simple transport problems (Figure 1 (a)). In practical thermodynamic control, however, one often faces demands that extend far beyond mere transport. Specifically, maximizing target transitions and fluxes (Figure 1 (b))[57, 58, 59] is essential for optimizing work extraction[60], reaction fluxes[61, 62], and free energy transduction[63, 64]. Furthermore, confining a system to a prescribed region is often crucial to the stability of thermodynamic systems (Figure 1 (c))[64, 65]. Even though such problems, especially flux optimization with minimal costs, have been addressed in the context of biochemical machines and networks operating at steady state, their integration with transport phenomena is yet to be achieved.
Figure 1. Outline of thermodynamic control theory for MJPs and CRNs. (a) The primary task of thermodynamic control is to drive the state of the system from at to at . Here, the state represents the probability distribution over discrete states for MJPs and the concentration of molecular species for CRNs. (b) The induction of specific transitions (MJPs) or reaction fluxes (CRNs) is indispensable for converting energy into functions. (c) In practical situations, additional constraints on desirable or undesirable states must be accounted for when designing control. (d) Optimal control theory accommodates all these tasks and requirements, along with the thermodynamic cost of control (entropy production or heat dissipation), into a single objective function and minimizes it to obtain the optimal control. (e) The thermodynamic Hamilton-Jacobi-Bellman (HJB) equation characterizes the value of control via the value function, the gradient of which induces the perturbed generalized gradient flow dynamics as the optimally controlled dynamics. (f) The thermodynamic Pontryagin’s equation characterizes the optimally controlled Hamiltonian dynamics, which is equivalent to the dynamics obtained by the thermodynamic HJB equation. Here, the equation for is shown for notational simplicity. .
Optimal Control and Thermodynamics
Optimal control theory, which generalizes variational calculus, offers a natural mathematical setting for treating such diverse requirements without assuming slow driving[13, 66, 67, 68].
Optimal control theory incorporates a variety of requirements and constraints in the form of utilities and costs into a single objective functional and provides two principled ways to solve the resulting functional optimization problems. On the one hand, the Hamilton–Jacobi–Bellman (HJB) equation, grounded in Bellman’s principle of optimality, provides the state-time global landscape of desirability that the optimal trajectory should follow. On the other hand, the Hamiltonian dynamics derived from Pontryagin’s maximum principle can directly generate the state-time local individual optimal trajectory and dynamic driving control without the knowledge of the global desirability landscape[12]. These two together provide a coherent view of the global and local aspects of the optimal control problem.
Yet, despite their solid and profound mathematical basis, in the context of thermodynamic control, variational and optimal-control methods have largely been used as mere technical tools to derive problem-specific optimal protocols for individual setups[41, 69, 70, 71, 72, 73, 74, 75], rather than as a theoretical foundation for thermodynamic control comparable to thermodynamic geometry or optimal transport[76].
In addition, its application has been heavily skewed toward continuous diffusion processes described by the Fokker-Planck equation, since existing optimal control theory is predominantly built on such processes. Extensions to discrete-state MJPs and CRNs, which possess mathematical structures distinct from their continuous counterpart, remain limited. Indeed, recent studies on thermodynamic control of CRNs are restricted to the slow-driving limit [77, 78]. A notable exception is the work of Muratore-Ginanneschi et al., which treated thermodynamic transport in MJPs within an optimal-control formulation [79, 80]. However, their approach relies on the coordinate transformation from the probability variable to a wave-function-like variable to employ the optimal control techniques for quantum-mechanical systems. This construction differs substantially from the standard optimal-control formulation, rendering it non-trivial to extend it to other contexts, such as CRNs.
Outline of this work
In this work, we fill this gap by establishing a thermodynamic optimal-control theory that is consistent with the thermodynamic and geometric structures of MJPs and CRNs. We consider a class of control problems in which reaction rate constants serve as control variables [81]. In addition to an entropy-production-based control cost, we introduce state and flux utility functions into the objective functional (Figure 1 (d)). This formulation allows one to treat not only finite-time state transport but also a broader class of control tasks where the occupation of desired states and induction of specified reaction fluxes are balanced against the thermodynamic control cost.
By deriving the thermodynamic version of the Hamilton–Jacobi-Bellman (HJB) equation (Figure 1 (e)), we show that the optimal dynamics is a perturbed generalized-gradient flow, with the value function as the driving potential. This result is obtained through a convex representation of the entropy-production rate (EPR) function based on generalized means, and the generalized gradient flow is induced by the convex conjugate of this EPR function as the dissipation function (a nonlinear generalization of the friction metric) [82]. The derived dynamics has the same thermodynamic structure as the equilibrium and nonequilibrium chemical reaction dynamics [83, 84, 85], which admits its information-geometric characterization [82].
We further derive a thermodynamic Pontryagin equation by reframing the Pontryagin’s Hamiltonian equation in terms of thermodynamic forces as control variables (Figure 1 (f)).
This equation manifests the structure specific to thermodynamic control and clarifies its relation to results from optimal transport and related approaches. We obtained analytic optimal control solutions for constant activity and a constrained activity budget, which unify the thermodynamic optimal transport solution and steady flux maximization. Our results establish a theoretical basis for thermodynamic control, with possible applications to biological reaction systems and finite-time thermodynamic operations.
Structure of this paper
The remainder of this paper is organized as follows. Section II reviews the thermodynamic structure of MJPs and CRNs and formulates the optimal control problem in a general setting. In § 0.3, we derive the thermodynamic HJB and Pontryagin equations by employing a convex representation of the EPR function; this formulation uncovers a perturbed generalized gradient flow structure along with the intrinsic information-geometric features of thermodynamic optimal control problems. Building on these results, Sec. IV presents analytical solutions that unify thermodynamic transport and steady-flux maximization. Finally, § 0.5 concludes the paper with a discussion of future directions.
Preliminary: Reversible deterministic chemical reaction network and its optimal control
Reversible chemical reaction networks
Let be the molecular species involved in the system. To investigate thermodynamic control, we consider reversible CRN, each pair of forward and backward reactions are defined as
where is the index of reaction. Here, and denote the number of molecules consumed and produced in the forward reaction (left to right), respectively. In the reverse reaction (right to left), these roles are inverted, with and representing the reactants and products. We define the forward and backward stoichiometric matrices
as . Then, the standard stoichiometric matrix is defined as .
The deterministic CRN dynamics for the concentration vector of molecular species are described by the chemical rate equation (CRE):
where are forward and reverse reaction fluxes and their difference is the (total) thermodynamic flux. In this work, we consider the case that the trajectory is confined within . Given the local detailed balance assumption [86, 87], thermodynamic force is defined by
The total entropy production rate (EPR) at time and state is obtained by
where is the gas constant, is the volume, is the bilinear pairing, and is the Kullbuck-Leibler divergence for non-negative densities. In the following, we set for simplicity.
Mass Action Kinetics
While the definitions of thermodynamic force and EPR are independent of the functional forms of the fluxes with respect to , specific kinetics for the forward and backward fluxes are required to instantiate the general theory that we derive. Mass action kinetics (MA kinetics) is the representative one. For the reversible CRN(Eq. (1)), their forward and backward flux functions under MA kinetics are
where are the kinetic rate constants and is a short-hand notation of (see also the Appendix for the detailed setup.). In addition, if all reactions are monomolecular and thus their fluxes are linear, the CRE (Eq. (2)) is reduced to the master equation for MJP on a graph. The flux functions are reduced to
where is 1 if the ith molecule is the reactant (product) of the rth reaction, respectively, and 0 otherwise. For , is the graph incidence matrix for the MJP.3 in Eq. (5) is replaced with .
Optimal control of deterministic CRNs
To introduce external control to a CRN, we decompose the forward and backward fluxes as
where are the reaction rate constants and are the dependence of the flux functions that satisfies if . We suppose that we can control the reaction rate constants independently. Under this setup, we consider the EPR as the cost for controlling the kinetic rate of a CRN to at state :
Then, we introduce the control cost functional over the trajectory and as
Conventional thermodynamic transport problems only account for this control cost. However, in various practical situations, we may have additional constraints and requirements for the operation of the thermodynamic system[57, 58, 59, 60, 61, 62, 63, 64, 65]. To integrate these requirements, we introduce the instantaneous utility function , the terminal utility function and the flux utility function . The instantaneous utility function specifies the preferred state of at time (Figure 1 (c)), whereas the terminal utility determines the state where the system is encouraged to reach at the end of control. If the terminal state is specified, then if and , otherwise. The th component of the flux utilities, and , define the desirability of the occurrence of the th forward and backward reactions, respectively (Figure 1 (b)). In this work, we assume that the flux utilities for each reversible reaction are anti-symmetric (reversible with respect to flux), i.e.: . For these utilities, we introduce their integral over the trajectory:
where we used the anti-symmetry of flux utilities and abbreviate the dependence of the flux on as .
Then, the optimal control problem for a finite horizon starting from is defined as
where is the weighting parameters (Figure 1 (d)).
Bellman’s Principle of Optimality and HJB equation
A standard approach for solving this control problem is to follow Bellman Principle of Optimality, which decomposes the optimization problem over into one at and the sub-problem over . To conduct this decomposition operationally, we introduce the value function:
which satisfies the Hamilton Jacobi Bellman (HJB) equation:
where the terminal boundary condition is . In the HJB equation, the optimization over the trajectory is replaced by the optimization at given . Thus, the optimal becomes
where we used . If an explicit form of is provided, the value function can be obtained by solving
in a time-backward manner from the terminal boundary condition .
Thermodynamic representations of optimal control for CRNs
Solving Eq. (17) explicitly is intractable for general control problems with nonlinearities in the cost and utility functions. If the cost function is strictly convex with respect to the control variable, the problem becomes manageable at least formally by using the Legendre transformation. However, the thermodynamic cost is convex but not strict with respect to due to a degeneracy4. We resolve this degeneracy by applying transformations based on generalized means[54], which decouples the degeneracy and the strict convexity.
Generalized mean and thermodynamic representation of flux
For two non-negative scalars , let us consider a class of their generalized mean , which satisfies , equality , symmetry , and homogeneity for any . By defining , we can represent .5 With , the generalized average flux is introduced[54] as
where . This quantity (up to constant scaling) is called “traffic” for the arithmetic mean[88], “Onsager coefficients” or “state mobility” for logarithmic mean[54, 88], and “frenetic activity” for geometric mean[85, 89]. Here we call it ”general activity” or simply ”activity” as in [54]. Then, the total flux can be represented as
where with
The -induced function satisfies oddity , which leads to if . Given and , the corresponding are recovered as , for , and for .6 In this work, we consider only the differentiable generalized means that satisfy the two physical conditions
for as in [54]. Examples of and are listed in Table I, all of which satisfy these physical conditions.
| Type of mean | Domain | ||||
| Harmonic | |||||
| Geometric | |||||
| Logarithmic | |||||
| Arithmetic | |||||
| Contraharmonic |
Table I. A list of generalized means and the corresponding functional forms of and with their domains for . The domain for is for all means.
When the first physical condition in Eq. (22) is satisfied, becomes strictly monotonous and thus invertible . Then, give and , the corresponding are recovered as if is in the corresponding domain. Furthermore, by defining dissipation functions, , as
they are strictly convex, symmetric, and convex conjugate to each other. The thermodynamic force and flux are associated as the Legendre dual of these functions:
where and . Owing to the Fenchel–Young inequality, EPR (Eq. (5)) is represented as
Moreover, Eq. (2) with the MA kinetics is represented as the perturbed generalized gradient flow (Figure 2 (a))7:
Figure 2. The thermodynamic structure of autonomous CRNs (a) and the optimally controlled CRNs (b). Both dynamics are the perturbed generalized gradient flow. For the autonomous CRNs (a), the dynamics is induced by the thermodynamic force generated by the gradient of the thermodynamic potential function perturbed by the external nonequilibrium force . The flux is the Legendre transformation of the force via the dissipation potential . For the optimally controlled CRNs (b), the dynamics is induced by the pseudo-force generated by the gradient of the value function perturbed by the flux utility . The flux is the Legendre transformation of the pseudo-force via the Legendre dual of the EPR .
Convex representation of entropy production rate by generalized mean
While the representation Eq. (25) of EPR by the pair of convex dissipation functions is fundamental in the thermodynamics of CRNs, it may not be employed for the control problem because it is a sum of two strictly convex functions with different arguments, i.e., and . The EPR is more directly represented as functions of , , or by using Eq. (20)
and its inverse (Figure 3 (a) and (b)):
Figure 3. The functional form of EPR as a function of (a) and (b) for each generalized mean where . (c) Comparison of the pseudo-force (the solid curve) and the thermodynamic force (the dashed curve) as function of for each generalized mean.
If the second condition in Eq. (22) is met additionally to the first one8, becomes strictly convex. Furthermore, is symmetric and greater than or equal to 0, the equality of which holds if and only if . The same true for .9 Then, the convex conjugate of is defined as
which is also strictly convex, symmetric, and greater than or equal to 0 from the properties of the convex conjugate. Thus, the pair of and induce the Legendre duality between and as
We refer to , the Legendre dual of via EPR, as the pseudo-force because differs from the thermodynamic force (Figure 3 (c)), which is the Legendre dual of induced by the dissipation function (Eq. (24)). The explicit representation of as a function of is where . In contrast, and also are not explicitly represented by , except for the logarithmic mean. Nevertheless, as we see in § 0.3 E, , as a function of , has an explicit form , which manifests that it is identical to the generalized pseudo-EPR .
Explicit form of value function and HJB equation
By employing the representation in Eq. (28), Eq. (17) can be explicitly transformed as
where we replace the control variable from to because we can arbitrary modulate by changing for . By further transforming the control variable from to and , we formally obtain
Partial optimization of Eq. (33) with respect to for a fixed is equivalent to . Since is odd and monotonous, as . Thus, without additional constraints over possible values of , the solution becomes singular[52]. A representative constraint is directly fixing to a certain value or function [79, 80, 88]. In the main body, we mainly focus on the case where is fixed for each and as . Another constraint being investigated before is the total activity budget constraint; for each time point or over the whole trajectory [52, 53, 54], which is treated in Appendix.
| Type of mean | |||||
| Harmonic | NA | NA | |||
| Geometric | NA | NA | |||
| Logarithmic | |||||
| Arithmetic | NA | NA | |||
| Contraharmonic | NA | NA |
Table 2. The functional forms of the ERP and its conjugate and induced Legendre transformation for representative generalized means. NA stands for no analytic representation by elementary functions available.
Given a fixed , we then solve the optimization with respect to , which has the (linear)-(convex) Legendre form. Thus, the optimal flux is obtained as
where the optimal pseudo-force is defined as
From Eq. (35), we can see that the optimally controlled dynamics have the form (Figure 2(b)):
Note that the optimal thermodynamic force is
and the optimal kinetic parameters are obtained as
To gain an explicit form of the HJB equation (Eq. (18)), we introduce a Hamiltonian
Then, the HJB equation after transformation is
where the optimal Hamiltonian is defined as
The corresponding optimal Lagrangian is
The value function along the optimal dynamics (Eq. (37)) satisfies
Thus,
This is consistent with the original definition of the value function by Eq. (14).10
Pontryagin's maximum principle and Hamiltonian dynamics
Although Bellman’s principle provides the global perspective on the optimal control problem and the induced dynamics, solving the HJB equation, an infinite-dimensional equation in general, is often computationally prohibitive both analytically and numerically even for low-dimensional CRNs. Pontryagin’s maximum principle offers an alternative way to solve the same control problem via Hamiltonian dynamics, yet locally for each trajectory. Specifically, the optimal control solution (Eq. (37)) can be casted to the solution of Hamiltonian dynamics.
By assuming that the optimal value function is smooth, we introduce as
where is the solution of Eq. (37). Then, the dynamics of is determined by the Pontryagin’s Hamiltonian dynamics:
This finite-dimensional equation solved with appropriate boundary conditions provides candidates of the optimally controlled trajectory11.
Thermodynamic representation of Pontryagin’s equation
Although Hamiltonian dynamics via Pontryagin’s principle, Eq. (51), and Eq. (52), is the standard form for solving general optimal control, most of the previous work on thermodynamic transport and control adopted the thermodynamic force rather than as the control variable. In addition, the convex conjugate EPR embedded in the equations does not have an analytic representation as a function of for most of the generalized means except the logarithmic one12. We resolve these problems by showing that can be represented analytically as a function of , which establishes a thermodynamic representation of Pontryagin’s equation. From Eq. (28), we obtain
where (Figure 2 (c)) 13. Thus, holds. This relation is also rephrased by using the dissipation function as
where is the Hessian of . Then, we have
where we used Eq. (53) from the first to the second lines. Note that, as a function of , becomes
where is a generalization of the pseudo-entropy production rate (pEPR) via the generalized means[85, 90, 91]. Although pEPR has been recognized as a proxy for EPR when extending the thermodynamic uncertainty relation to MJPs[52, 91, 92], our result clarifies that it has a fundamental meaning as the dual of EPR14.
Using Eq. (36) and Eq. (53), the value function and the optimal force are associated by the following equation:
The HJB equation (Eq. (41)) is transformed into
where we used Eq. (57). For Pontryagin’s equation, and are associated as
By using Eq. (20) and Eq. (57), Eq. (51) and Eq. (52) become
By differentiating Eq. (61) along with respect to , we have
By inserting Eq. (62) and Eq. (63), we have
We also have an algebraic constraint
which originates from Eq. (53) and accounts for the difference of dimensions of and . The set of equations, Eq. (62), Eq. (65) and Eq. (66) establish the thermodynamic version of Pontryagin’s Hamiltonian dynamics.
Analytic solutions and its relationship with thermodynamic transport problems
In this section, we demonstrate that the transport problem with the constant flux utility is a theoretically sound extension that unifies the usual thermodynamic transport problem and the flux optimization problem. Specifically, we consider the setup such that , and , and the boundary conditions are and , satisfying . Then, the thermodynamic Pontryagin equation for is reduced to
While solving this equation together with Eq. (62) and Eq. (66) analytically is still impossible in general, it becomes tractable if the right-hand side of Eq. (67) becomes 0 or effectively 0. Most previous works for thermodynamic transport exploit such situations[52, 53, 54, 88].
Optimal thermodynamic transports with flux utility under constant activity and with total activity constraints
If the activity is constant , the thermodynamic Pontryagin’s equation substantially simplifies to
Then, the optimal trajectories are formally and where . If holds, the optimal constant force is obtained by solving and . These equations are equivalent to and where is a particular solution of . Since and are orthogonal complement, the optimal is uniquely determined as by the information-geometric orthogonality[82]. Then and are uniquely determined from . For this setup, the value function is obtained analytically as
where is the contracted (optimal) Lagrangian defined as
(see Appendix for derivation).
Yoshimura et al. considered this situation for by using the logarithmic mean and directly constraining to be constant[88]. In such a case, the optimal force becomes conservative because for the logarithmic mean. Also, the optimal flux is the dual of w.r.t. the Riemannian metric .
In addition, our formulation includes the flux maximization at the steady state[61, 62, 63, 64] as a special case. Specifically, if , the value function reduces to , which is equivalent to the maximization of the steady flux with the EPR cost.
A similar result is also obtained by minimizing the entropy production under the constraint that the time-average total activity budget is bounded:
The same argument applies as before; the value function becomes
where, for ,
and is the remaining activity budget at whose initial condition at is . If we reduce the result to the conventional transport problem without flux utility, i.e., , then the value function is further simplified as
where
is a norm on . This situation has been investigated for the arithmetic mean [52], the logarithmic mean [53], and the general means [54]. If this result is applied to MJP where is replaced with the graph incidence matrix , becomes the graph -Wasserstein distance.
Since optimizing under this constraint complicates the derivation of the HJB and Pontryagin’s equations, we defer the details to Appendix.
Optimal thermodynamic transports with –dependent activity
If is -dependent, the dynamics of force (Eq. (67)) is no longer constant. Muratore-Ginanneschi et al. in their pioneering work, investigated the optimal thermodynamic transport over MJP with -dependent where is specified by the geometric mean of MA linear kinetics [79]. Remlein and Seifert employed this result to show that non-conservative force is optimal under this choice of activity [80]. This choice can be interpreted as defining an activity field based on uncontrolled MA kinetics as a reference. While an analytic expression of the value function and the optimal trajectory cannot be obtained for this -dependent activity in general, we can immediately obtain the main results of [79] for the generalized mean via our formulation. Specifically, the optimal force condition (Eq. (52) in [79]), the dynamics of auxiliary field (Eq. (54) in [79]), and the dynamics of optimal force (Eq. (56) in [79]) are equivalent to Eq. (59), Eq. (63), and Eq. (65), respectively (See Appendix for the detailed comparison).
In addition, the non-conservativeness of the optimal thermodynamic force for generalized means other than the logarithmic one[80] is a consequence of Eq. (59), which means that the optimal force is a nonlinear function of the gradient of the value function and thus non-conservative.15 Dechant & Meer proposed to replace the EPR, , with twice the dissipation function as the control cost to recover the conservativeness of the optimal force[93], the underlying idea of which is close to the variational representation of thermodynamic flux[82]. In contrast, our result clarified that the optimal pseudo-force is always the gradient of the value function for as , which shares the same dynamical property as the equilibrium dynamics(Eq. (26)). Since the transient dynamics has non-negative EPR even if driven by a conservative force, the generalized gradient flow property is more fundamental for characterizing the optimal thermodynamic transport dynamics. (See Appendix for more detailed comparison and discussion).
In contrast to the geometric mean, the logarithmic mean is not compatible with -dependent activity because becomes a complicated function even if the simplest linear kinetics is employed.
Conclusion and Discussion
In this work, we have established a thermodynamic optimal control theory for CRNs that aligns with their underlying thermodynamic and geometric structures. By treating reaction rate constants as control variables and incorporating both state and flux utilities alongside the EPR cost, this framework encompasses conventional thermodynamic transport far from equilibrium as well as a broader class of nonequilibrium control tasks.
Our central result clarifies that the optimal dynamics derived from the thermodynamic Hamilton–Jacobi–Bellman (HJB) equation takes the form of the perturbed generalized gradient flow driven by the value function. This characterization stems from the convex representation of the EPR function via generalized means, where its convex conjugate naturally acts as a nonlinear generalization of the friction metric. The resulting dynamics shares the thermodynamic structure of equilibrium and nonequilibrium CRNs and admits an information-geometric characterization. Furthermore, we derive a thermodynamic Pontryagin equation by recasting the Pontryagin Hamiltonian system using thermodynamic forces as the control variables. This formulation highlights the intrinsic thermodynamic and geometric structure of the control problem and clarifies its direct connections to optimal transport and related approaches.
These results lay a theoretical foundation for the thermodynamic control of CRNs, with potential applications to biological systems and finite-time thermodynamics. In addition, they pave the way for addressing the following challenges.
From transport to time-cyclic control
The first important problem for future work is the design of optimal thermodynamic cycles. Although thermodynamic transport between two prescribed state distributions at minimal thermodynamic cost is a basic building block of thermodynamic control, constructing optimal thermodynamic engines requires identifying time-periodic control trajectories that balance dissipation minimization with utility maximization[94, 95]. Within near-equilibrium thermodynamic geometry, this task can be formulated as a geometric problem to a finding optimal closed curve on a manifold through an isoperimetric formulation[96, 97, 98]. Extensions to far-from-equilibrium regimes have also been explored[99, 100, 101, 102].
Building upon the framework developed here, a natural next step is to investigate this problem from a geometric perspective, establish conditions for the existence of optimal periodic trajectories, and characterize their thermodynamic properties in more general settings. In parallel, practical numerical schemes for computing optimal cycles should be developed, leveraging the rich toolkit of modern control theory.
From deterministic to stochastic control
A second challenge lies in extending our framework to stochastic optimal control. The conventional thermodynamic transport and control based on thermodynamic or Wasserstein geometry fall within the class of deterministic optimal control, as they treat control problems for probability distributions or densities governed by deterministic evolution equations, i.e., the Fokker–Planck and master equations. Our results for CRNs, which generalize these approaches, belong likewise to this category. Owing to the deterministic nature, an experimental implementation of such control strategies does not require feedback control via observing the systems[51].
However, recent advances in single-molecule techniques, such as imaging and manipulation, enable us to access individual stochastic trajectories and directly perturb their dynamics[103, 104, 105]. Bridging this gap between theory and experiment calls for extending our framework from deterministic to stochastic optimal control. Adapting the established theory of stochastic optimal control[81, 106] to thermodynamic settings therefore represents a crucial frontier for future research[107].
From nonautonomous to autonomous control
Third, although external control is the standard setup for studying thermodynamic systems, real heat engines and living systems operate autonomously[108, 109]. Designing autonomous heat engines and understanding the thermodynamics of living systems therefore require identifying optimal control laws implemented within the systems themselves[110, 111].
Automatic differentiation and learning theory have been employed to obtain such autonomous systems[95, 112, 113]. Among others, reinforcement learning (RL), which builds on optimal control theory, provides methods for formulating and optimizing time-invariant feedback policies[114]. The integration of RL with our theory would provide a systematic framework for autonomous thermodynamic control[115].
Applications to physics of intelligence
Finally, the growing interest in the physics of intelligence, spurred in part by advances in artificial intelligence, has stimulated efforts to employ stochastic thermodynamics and related methods to information science[116, 117, 118, 119, 120, 121]. For example, recent studies have drawn on stochastic thermodynamics to assess the performance of generative diffusion models[122]. These models exploit time reversal to learn generative dynamics, while optimal control theory offers a more general formulation by seeking control laws that generate reverse-time trajectories. This control-theoretic perspective is applied to state-of-the-art diffusion models and their fine-tuning. Extending the theoretical framework developed here beyond thermodynamics to diverse problems in information science could thus open new avenues for physics of intelligence.
Acknowledgement
We used OpenAI’s ChatGPT for literature review, scientific discussions, and English editing and proofreading. All AI-assisted content was independently verified by the authors, who take full responsibility for the manuscript.
We thank Praful Gagrani, Dimitri Loutchko, and Keisuke Sugie for fruitful discussion and comments. This work was supported by JST CREST (Grant Numbers JPMJCR25Q2), JST PRESTO (Grant No. JPMJPR25KB), and JSPSKAKENHI (Grant Number 25H01365).
References
- [1]J. McGrath, J. Clarke, C. Johnson, and J. Alvarado, Control across scales: Signals, informa- tion, and adaptive biological mechanical function, Biophysics Rev. 7, 021309 (2026).arxiv.org/abs/2509.03418
- [2]A. Datta, P. Pietzonka, and A. C. Barato, Second Law for Active Heat Engines, Phys. Rev. X 12, 031034 (2022).DOI
- [3]A. Sahin, D. R. Weilandt, and V. Hatzimmanikatis, Optimal enzyme utilization suggests that concentrations and thermodynamics determine binding mechanisms and enzyme saturations, Nat Commun 14, 2618 (2023).DOI
- [4]D. H. Wolpert, J. Korbel, C. W. Lynn, F. Tasnim, J. A. Grochow, G. Kardes, J. B. Aimone, V. Balasubramanian, E. De Giuli, D. Doty, N. Freitas, M. Marsili, T. E. Ouldridge, A. W. Richa, P. Riechers, É. Roldán, B. Rubenstein, Z. Toroczkai, and J. Paradiso, Is stochastic thermodynamics the key to understanding the energy costs of computation?, Proceedings of the National Academy of Sciences 121, e2321112121 (2024).DOI
- [5]K. Sekimoto, Langevin Equation and Thermodynamics, Prog Theor Phys 130, 17 (1998).DOI
- [6]L. Peliti and S. Pigolotti, Stochastic Thermodynamics: An Introduction (Princeton Univer- sity Press, 2021).
- [7]U. Seifert, Stochastic Thermodynamics (Cambridge University Press, Cambridge, 2025).
- [8]F. Avanzini, M. Bilancioni, V. Cavina, S. Dal Cengio, M. Esposito, G. Falasco, D. Forastiere, J. N. Freitas, A. Garilli, P. E. Harunari, V. Lecomte, A. Lazarescu, S. G. Marehallis Srinivas, C. Moslonka, I. Neri, E. Penocchio, W. D. Piñeros, M. Polettini, A. Raghu, P. Raux, K. Sekimoto, and A. Soret, Methods and conversations in (post)modern thermodynamics, SciPost Physics Lecture Notes , 080 (2024).DOI
- [9]F. L. Curzon and B. Ahlborn, Efficiency of a Carnot engine at maximum power output, Am. J. Phys. 43, 22 (1975).DOI
- [10]B. Andresen and P. Salamon, The First Fifty Years of Finite-Time Thermodynamics, Entropy 28, 49 (2026).DOI
- [11]S. Deffner and M. V. S. Bonança, Thermodynamic control —An old paradigm with new applications, EPL 131, 20001 (2020).
- [12]S. Blaber and D. A. Sivak, Optimal control in stochastic thermodynamics, J. Phys. Commun. 7, 033001 (2023).DOI
- [13]J. Alvarado, E. G. Teich, D. A. Sivak, and J. Bechhoefer, Optimal Control in Soft and Active Matter, Annual Review of Condensed Matter Physics 17, 327 (2026).DOI
- [14]G. E. Crooks, Measuring Thermodynamic Length, Phys. Rev. Lett. 99, 100602 (2007).DOI
- [15]D. A. Sivak and G. E. Crooks, Thermodynamic Metrics and Optimal Paths, Phys. Rev. Lett. 108, 190602 (2012).DOI
- [16]P. R. Zulkowski, D. A. Sivak, G. E. Crooks, and M. R. DeWeese, Geometry of thermodynamic control, Phys. Rev. E 86, 041148 (2012).
- [17]P. Salamon, B. Andresen, P. D. Gait, and R. S. Berry, The significance of Weinhold’s length, J. Chem. Phys. 73, 1001 (1980).DOI
- [18]P. Salamon and R. S. Berry, Thermodynamic Length and Dissipated Availability, Phys. Rev. Lett. 51, 1127 (1983).DOI
- [19]B. Andresen, Finite-time thermodynamics and thermodynamic length, Revue Générale de Thermique 35, 647 (1996).DOI
- [20]Y.-H. Ma and X.-H. Zhao, Finite-time thermodynamics: A journey beginning with optimizing heat engines (2024).DOI
- [21]B. Andresen, Current Trends in Finite-Time Thermodynamics, Angewandte Chemie International Edition 50, 2690 (2011).DOI
- [22]B. Andresen, P. Salamon, and R. S. Berry, Thermodynamics in finite time: Extremals for imperfect heat engines, J. Chem. Phys. 66, 1571 (1977).DOI
- [23]B. Andresen, R. S. Berry, A. Nitzan, and P. Salamon, Thermodynamics in finite time. I. The step-Carnot cycle, Phys. Rev. A 15, 2086 (1977).DOI
- [24]P. Salamon, B. Andresen, and R. S. Berry, Thermodynamics in finite time. II. Potentials for finite-time processes, Phys. Rev. A 15, 2094 (1977).DOI
- [25]F. Weinhold, Metric geometry of equilibrium thermodynamics, The Journal of Chemical Physics 63, 2479 (1975).DOI
- [26]G. Ruppeiner, Thermodynamics: A Riemannian geometric model, Phys. Rev. A 20, 1608 (1979).DOI
- [27]Z. Li and Y. Izumida, Decomposition of metric tensor in thermodynamic geometry in terms of relaxation timescales, Phys. Rev. E 111, 034113 (2025).DOI
- [28]A. Plastino, A thermodynamic-geometric view of irreversibility, Mod. Phys. Lett. A 41, 2650123 (2026).DOI
- [29]G. Watanabe, G.-H. Xu, and Y. Minami, Unified geometric formalism for dissipation and its fluctuations in finite-time microscopic heat engines (2026), arXiv:2604.04620 [cond-mat.stat-mech].DOI
- [30]W. C. Wareham and D. A. Sivak, Multiparameter optimal control of {\mathrm{F}}_{-1}-ATPase, Phys. Rev. E 111, 064410 (2025).DOI
- [31]P. R. Zulkowski, D. A. Sivak, and M. R. DeWeese, Optimal Control of Transitions between Nonequilibrium Steady States, PLOS ONE 8, e82754 (2013).DOI
- [32]D. Kamp and K. Proesmans, Thermodynamic control of nonequilibrium systems, Phys. Rev. E 113, 014140 (2026).DOI
- [33]J. R. Sawchuk and D. A. Sivak, Global thermodynamic manifold for conservative control of stochastic systems, Phys. Rev. Res. 8, 013004 (2026).DOI
- [34]T. Matsuo and A. Sonoda, Analysis of entropy production in finitely slow processes between nonequilibrium steady states, Phys. Rev. E 106, 064119 (2022).DOI
- [35]Z. Wang, Thermodynamic Geometry of Nonequilibrium Fluctuations in Cyclically Driven Transport, Phys. Rev. Lett. 132, 10.1103/PhysRevLett.132.207101 (2024).DOI
- [36]D. Guéry-Odelin, A. Ruschhaupt, A. Kiely, E. Torrontegui, S. Martínez-Garaot, and J. G. Muga, Shortcuts to adiabaticity: Concepts, methods, and applications, Rev. Mod. Phys. 91, 045001 (2019).
- [37]A. Patra and C. Jarzynski, Shortcuts to adiabaticity using flow fields, New J. Phys. 19, 125009 (2017).
- [38]D. Guéry-Odelin, C. Jarzynski, C. A. Plata, A. Prados, and E. Trizac, Driving rapidly while remaining in control: Classical shortcuts from Hamiltonian to stochastic dynamics, Rep. Prog. Phys. 86, 035902 (2023).
- [39]E. İlker, Ö. Güngör, B. Kuznets-Speck, J. Chiel, S. Deffner, and M. Hinczewski, Shortcuts in Stochastic Systems and Control of Biophysical Processes, Phys. Rev. X 12, 021048 (2022).
- [40]E. Aurell, C. Mejía-Monasterio, and P. Muratore-Ginanneschi, Optimal Protocols and Optimal Transport in Stochastic Thermodynamics, Phys. Rev. Lett. 106, 250601 (2011).pubmed.ncbi.nlm.nih.gov/21770620
- [41]P. Muratore-Ginanneschi, On the use of stochastic differential geometry for non-equilibrium thermodynamic modeling and control, J. Phys. A: Math. Theor. 46, 275002 (2013).
- [42]E. Aurell, K. Gawędzki, C. Mejía-Monasterio, R. Mohayaee, and P. Muratore-Ginanneschi, Refined Second Law of Thermodynamics for Fast Random Processes, J Stat Phys 147, 487 (2012).DOI
- [43]Y. Chen, T. T. Georgiou, and A. Tannenbaum, Stochastic Control and Nonequilibrium Thermodynamics: Fundamental Limits, IEEE Transactions on Automatic Control 65, 2979 (2020).DOI
- [44]M. Nakazato and S. Ito, Geometrical aspects of entropy production in stochastic thermodynamics based on Wasserstein distance, Phys. Rev. Res. 3, 043093 (2021).DOI
- [45]O. Movilla Miangolarra, A. Taghvaei, Y. Chen, and T. T. Georgiou, Geometry of Finite-Time Thermodynamic Cycles With Anisotropic Thermal Fluctuations, IEEE Control Systems Letters 6, 3409 (2022).arxiv.org/abs/2203.12483
- [46]S. Chennakesavalu and G. M. Rotskoff, Unified, Geometric Framework for Nonequilibrium Protocol Optimization, Phys. Rev. Lett. 130, 107101 (2023).DOI
- [47]S. Ito, Geometric thermodynamics for the Fokker–Planck equation: Stochastic thermodynamic links between information geometry and optimal transport, Info. Geo. 7, 441 (2024).DOI
- [48]T. Kamijima, A. Takatsu, K. Funo, and T. Sagawa, Optimal finite-time Maxwell’s demons in Langevin systems, Phys. Rev. Res. 7, 023159 (2025).arxiv.org/abs/2410.11603
- [49]J. Klinger and G. M. Rotskoff, Universal energy-speed-accuracy trade-offs in driven nonequilibrium systems, Phys. Rev. E 111, 014114 (2025).DOI
- [50]K. Tojo, R. Nagase, K. Funo, and T. Sagawa, Optimizing optimal transport: Role of final distributions in finite-time thermodynamics (2025), arXiv:2509.11314 [cond-mat.stat-mech].arxiv.org/abs/2509.11314
- [51]S. Oikawa, Y. Nakayama, S. Ito, T. Sagawa, and S. Toyabe, Experimentally achieving minimal dissipation via thermodynamically optimal transport, Nat Commun 16, 10424 (2025).DOI
- [52]A. Dechant, Minimum entropy production, detailed balance and Wasserstein distance for continuous-time Markov processes, J. Phys. A: Math. Theor. 55, 094001 (2022).DOI
- [53]T. Van Vu and K. Saito, Thermodynamic Unification of Optimal Transport: Thermodynamic Uncertainty Relation, Minimum Dissipation, and Thermodynamic Speed Limits, Phys. Rev. X 13, 011013 (2023).DOI
- [54]R. Nagayama, K. Yoshimura, and S. Ito, Infinite variety of thermodynamic speed limits with general activities, Phys. Rev. Res. 7, 013307 (2025).DOI
- [55]R. Nagayama, Geometric thermodynamics of reaction-diffusion systems: Thermodynamic trade-off relations and optimal transport for pattern formation, Phys. Rev. Res. 7, 10.1103/PhysRevResearch.7.033011 (2025).DOI
- [56]J. van der Meer and A. Dechant, Near-optimality of conservative driving in discrete systems (2026), arXiv:2602.18321 [cond-mat].arxiv.org/abs/2602.18321
- [57]A. I. Brown and D. A. Sivak, Allocating dissipation across a molecular machine cycle to maximize flux, Proceedings of the National Academy of Sciences 114, 11057 (2017).DOI
- [58]D. Andrieux, Minimal entropy production under thermodynamic constraints: An application of cycle decomposition for Markov chain design (2023), arXiv:2303.05930 [cond-mat.stat-mech].arxiv.org/abs/2303.05930
- [59]Y.-J. Yang and K. A. Dill, A principled basis for nonequilibrium network flows, Nat Commun 10.1038/s41467-026-71239-9 (2026).DOI
- [60]T. Schmiedl and U. Seifert, Efficiency at maximum power: An analytically solvable model for stochastic heat engines, EPL 81, 20003 (2007).DOI
- [61]R. M. T. Fleming, C. M. Maes, M. A. Saunders, Y. Ye, and B. Ø. Palsson, A variational principle for computing nonequilibrium fluxes and potentials in genome-scale biochemical networks, Journal of Theoretical Biology 292, 71 (2012).arxiv.org/abs/1105.1513
- [62]P. Gagrani, N. Lauber, E. Smith, and C. Flamm, Thermodynamic ranking of pathways in reaction networks, J. Stat. Mech. 2025, 123208 (2026).
- [63]J. Piñero, R. Solé, and A. Kolchinsky, Optimization of nonequilibrium free energy harvesting illustrated on bacteriorhodopsin, Phys. Rev. Res. 6, 013275 (2024).
- [64]M. Bilancioni and M. Esposito, Gears in chemical reaction networks for optimizing energy transduction efficiency, Nat Commun 16, 5765 (2025).DOI
- [65]J. M. Horowitz, K. Zhou, and J. L. England, Minimum energetic cost to maintain a target nonequilibrium state, Phys. Rev. E 95, 042102 (2017).DOI
- [66]A. E. Bryson, Applied Optimal Control: Optimization, Estimation and Control (CRC Press, 1975).
- [67]D. Liberzon, Calculus of Variations and Optimal Control Theory: A Concise Introduction (Princeton University Press, 2011).
- [68]D. P. Bertsekas, Dynamic Programming and Optimal Control: Notes, Sources, and Exercises, p.71 (Athena Scientific, 2012).
- [69]T. Schmiedl and U. Seifert, Optimal Finite-Time Processes In Stochastic Thermodynamics, Phys. Rev. Lett. 98, 108301 (2007).DOI
- [70]S. Bo, E. Aurell, R. Eichhorn, and A. Celani, Optimal stochastic transport in inhomogeneous thermal environments, EPL 103, 10010 (2013).
- [71]A. Zhong and M. R. DeWeese, Limited-control optimal protocols arbitrarily far from equilibrium, Phys. Rev. E 106, 044135 (2022).DOI
- [72]S. Monter, S. A. M. Loos, and C. Bechinger, Optimal transitions between nonequilibrium steady states, Proceedings of the National Academy of Sciences 122, e2510654122 (2025).DOI
- [73]Y. Kurokawa and Y. Hasegawa, Optimal Control to Minimize Dissipation and Fluctuations in Open Quantum Systems Beyond Slow and Rapid Regimes (2026), arXiv:2602.01688 [quant-ph].arxiv.org/abs/2602.01688
- [74]J. Meibohm, S. Monter, S. A. M. Loos, and C. Bechinger, Finite-time transitions in optimal control and non-equilibrium relaxation (2026), arXiv:2604.25798 [cond-mat.stat-mech].arxiv.org/abs/2604.25798
- [75]E. Panizon, Optimal Control of a Mesoscopic Information Engine (2026), arXiv:2603.29804 [cond-mat.stat-mech].arxiv.org/abs/2603.29804
- [76]M. Pavon, Stochastic control and nonequilibrium thermodynamical systems, Appl Math Optim 19, 187 (1989).DOI
- [77]Y. Zhang, Q. Ouyang, and Y. Tu, Geometry of Optimal Control in Chemical Reaction Networks in the Adiabatic Limit, Phys. Rev. Lett. 134, 018001 (2025).DOI
- [78]A. T. Mohite and H. Rieger, Generalized Finite-time Optimal Control Framework in Stochas- tic Thermodynamics (2026), arXiv:2511.00974 [cond-mat.stat-mech].DOI
- [79]P. Muratore-Ginanneschi, C. Mejía-Monasterio, and L. Peliti, Heat Release by Controlled Continuous-Time Markov Jump Processes, J Stat Phys 150, 181 (2013).
- [80]B. Remlein and U. Seifert, Optimality of nonconservative driving for finite-time processes with discrete states, Phys. Rev. E 103, L050105 (2021).DOI
- [81]S. A. Horiguchi and T. J. Kobayashi, Optimal Control of Stochastic Reaction Networks with Entropic Control Cost and Emergence of Mode-Switching Strategies, PRX Life 3, 033027 (2025).
- [82]T. J. Kobayashi, D. Loutchko, A. Kamimura, S. A. Horiguchi, and Y. Sugiyama, Information geometry of dynamics on graphs and hypergraphs, Info. Geo. 7, 97 (2024).arxiv.org/abs/2211.14455
- [83]Y. Sugiyama, D. Loutchko, A. Kamimura, and T. J. Kobayashi, Hessian geometric structure of chemical thermodynamic systems with stoichiometric constraints, Phys. Rev. Research 4, 033065 (2022).
- [84]T. J. Kobayashi, D. Loutchko, A. Kamimura, and Y. Sugiyama, Kinetic derivation of the Hessian geometric structure in chemical reaction networks, Phys. Rev. Research 4, 033066 (2022).DOI
- [85]T. J. Kobayashi, D. Loutchko, A. Kamimura, and Y. Sugiyama, Hessian geometry of nonequilibrium chemical reaction networks and entropy production decompositions, Phys. Rev. Research 4, 033208 (2022).DOI
- [86]R. Rao and M. Esposito, Nonequilibrium Thermodynamics of Chemical Reaction Networks: Wisdom from Stochastic Thermodynamics, Phys. Rev. X 6, 041064 (2016).DOI
- [87]C. Maes, Local detailed balance, SciPost Physics Lecture Notes , 032 (2021).DOI
- [88]K. Yoshimura, Housekeeping and excess entropy production for general nonlinear dynamics, Phys. Rev. Res. 5, 10.1103/PhysRevResearch.5.013017 (2023).DOI
- [89]C. Maes, Non-Dissipative Effects in Nonequilibrium Systems, SpringerBriefs in Complexity (Springer International Publishing, Cham, 2018).
- [90]K. Yoshimura, Thermodynamic Uncertainty Relation and Thermodynamic Speed Limit in Deterministic Chemical Reaction Networks, Phys. Rev. Lett. 127, 10.1103/PhysRevLett.127.160601 (2021).DOI
- [91]N. Shiraishi, Optimal Thermodynamic Uncertainty Relation in Markov Jump Processes, J Stat Phys 185, 19 (2021).DOI
- [92]N. Shiraishi, Wasserstein distance in speed limit inequalities for Markov jump processes, J. Stat. Mech. 2024, 074003 (2024).DOI
- [93]A. Dechant and J. van der Meer, Renormalized entropy production for optimal transport in jump processes: Make conservative forces optimal again (2026), arXiv:2605.00656 [cond-mat.stat-mech].arxiv.org/abs/2605.00656
- [94]K. Brandner, K. Saito, and U. Seifert, Thermodynamics of Micro- and Nano-Systems Driven by Periodic Temperature Variations, Phys. Rev. X 5, 031019 (2015).
- [95]W. C. Wareham and D. A. Sivak, Optimal Control of Periodic Nonequilibrium Mechanochemical Systems via Automatic Differentiation (2026), arXiv:2609.03217 [cond-mat.stat-mech].arxiv.org/abs/2609.03217
- [96]K. Brandner and K. Saito, Thermodynamic Geometry of Microscopic Heat Engines, Phys. Rev. Lett. 124, 040602 (2020).DOI
- [97]P. Abiuso and M. Perarnau-Llobet, Optimal Cycles for Low-Dissipation Heat Engines, Phys. Rev. Lett. 124, 110606 (2020).DOI
- [98]A. G. Frim and M. R. DeWeese, Geometric Bound on the Efficiency of Irreversible Thermodynamic Cycles, Phys. Rev. Lett. 128, 230601 (2022).DOI
- [99]O. Movilla Miangolarra, A. Taghvaei, R. Fu, Y. Chen, and T. T. Georgiou, Energy harvesting from anisotropic fluctuations, Phys. Rev. E 104, 044101 (2021).DOI
- [100]O. Movilla Miangolarra, A. Taghvaei, Y. Chen, and T. T. Georgiou, Geometry of Finite-Time Thermodynamic Cycles With Anisotropic Thermal Fluctuations, IEEE Control Systems Letters 6, 3409 (2022).arxiv.org/abs/2203.12483
- [101]J. Ventura Siches, O. Movilla Miangolarra, and T. T. Georgiou, Refined bounds on energy harvesting from anisotropic fluctuations, Phys. Rev. E 109, 064155 (2024).arxiv.org/abs/2402.01973
- [102]A. Zhong, A. G. Frim, and M. R. DeWeese, Optimal active engines obey the thermodynamic Lorentz force law, Phys. Rev. E 114, L042106 (2026).DOI
- [103]S. Toyabe, T. Sagawa, M. Ueda, E. Muneyuki, and M. Sano, Experimental demonstration of information-to-energy conversion and validation of the generalized Jarzynski equality, Nature Phys 6, 988 (2010).DOI
- [104]M. Ribezzi-Crivellari and F. Ritort, Large work extraction and the Landauer limit in a continuous Maxwell demon, Nat. Phys. 15, 660 (2019).
- [105]T. K. Saha, J. N. E. Lucero, J. Ehrich, D. A. Sivak, and J. Bechhoefer, Bayesian Information Engine that Optimally Exploits Noisy Measurements, Phys. Rev. Lett. 129, 130601 (2022).DOI
- [106]W. Fleming and R. Rishel, Deterministic and Stochastic Optimal Control (Springer, New York, NY, 1975).
- [107]L. F. Reinalter, E. Panizon, L. Muruga, and C. Bechinger, Work Extraction via Backward Motion in Optimal Closed-Loop Stochastic Control (2026), arXiv:2606.17624 [cond-mat.stat-mech].arxiv.org/abs/2606.17624
- [108]M. Serra-Garcia, A. Foehr, M. Molerón, J. Lydon, C. Chong, and C. Daraio, Mechanical Autonomous Stochastic Heat Engine, Phys. Rev. Lett. 117, 010602 (2016).
- [109]C. Jarzynski, S. Deffner, and S. Rahav, Fluctuation theorems for autonomous work, Proceedings of the National Academy of Sciences 122, e2524775122 (2025).
- [110]X. Yang, M. Heinemann, J. Howard, G. Huber, S. Iyer-Biswas, G. Le Treut, M. Lynch, K. L. Montooth, D. J. Needleman, S. Pigolotti, J. Rodenfels, P. Ronceray, S. Shankar, I. Tavassoly, S. Thutupalli, D. V. Titov, J. Wang, and P. J. Foster, Physical bioenergetics: Energy fluxes, budgets, and constraints in cells, Proceedings of the National Academy of Sciences 118, e2026786118 (2021).DOI
- [111]S. Thutupalli, The cost of cellular patterns, Nat. Phys. 20, 1704 (2024).DOI
- [112]M. C. Engel, J. A. Smith, and M. P. Brenner, Optimal Control of Nonequilibrium Systems through Automatic Differentiation, Phys. Rev. X 13, 041032 (2023).arxiv.org/abs/2201.00098
- [113]J. Yan, H. Touchette, and G. M. Rotskoff, Learning nonequilibrium control forces to characterize dynamical phase transitions, Phys. Rev. E 105, 024115 (2022).DOI
- [114]R. S. Sutton and A. G. Barto, Reinforcement Learning, Second Edition: An Introduction (MIT Press, 2018).
- [115]S. Whitelam, Demon in the Machine: Learning to Extract Work and Absorb Entropy from Fluctuating Nanosystems, Phys. Rev. X 13, 021005 (2023).DOI
- [116]N. X. Kodama and M. Hinczewski, Thermodynamic Performance Limits for Score-Based Diffusion Models (2025), arXiv:2510.06174 [cs.LG].arxiv.org/abs/2510.06174
- [117]S. H. D’Ambrosia, S. M. Daniels, M. R. DeWeese, and A. Sahai, The Thermodynamic Costs of Simple Linear Regression, IEEE Journal on Selected Areas in Information Theory 7, 458 (2026).arxiv.org/abs/2605.19195
- [118]T. Sagawa, Stochastic Thermodynamics for Autoregressive Generative Models: A Non-Markovian Perspective (2026), arXiv:2604.07867 [cond-mat.stat-mech].DOI
- [119]X. Ding, H. T. Quan, and Y. Tu, Stochastic Thermodynamics of Score Matching in Diffusion Models (2026), arXiv:2606.17252 [cond-mat.dis-nn].arxiv.org/abs/2606.17252
- [120]T. Aslyamov and M. Esposito, Physical limits to stochastic classifiers (2026), arXiv:2610.20481 [cond-mat.stat-mech].arxiv.org/abs/2610.20481
- [121]Y. Hasegawa, Thermodynamics of classifiers (2026), arXiv:2605.24365 [cond-mat.stat-mech].arxiv.org/abs/2605.24365
- [122]K. Ikeda, T. Uda, D. Okanohara, and S. Ito, Speed-Accuracy Relations for Diffusion Models: Wisdom from Nonequilibrium Thermodynamics and Optimal Transport, Phys. Rev. X 15, 031031 (2025).DOI


