WorksPhysics and Astronomy
Variational Newton-Krylov descent for periodic orbits of two-dimensional Kolmogorov flow
Abstract
A matrix-free Newton-Krylov method for computing long unstable periodic orbits of turbulent flows is presented. The method arises from the space-time variational formulation of Lan & Cvitanović (Phys. Rev. E, vol. 69, 2004, 016217), and runs on graphics processing units with a preconditioner from time-spectral methods that retains advection. In two-dimensional Kolmogorov flow at , near recurrences remain about as frequent at long periods as at moderate ones, and searches from them converge to 183 distinct solutions with periods up to 191, about forty escape times, all but one verified by multiple shooting. The primary question is then whether such a long orbit can be taken, on its own, as a proxy for the attractor, and thus capture the statistics of the turbulent flow. The mean dissipation and energy of the long orbits lie close to the medians of the flow, and the distribution of the dissipation along a long orbit is comparable with that of a turbulent segment of the same duration, except in its tail, which most orbits miss. Their leading Floquet exponents appear to lie slightly below the Lyapunov spectrum of the flow, as may be expected of quieter orbits. The typical distance of the turbulent flow from a long orbit decreases as the period grows, but with an unfavourable scaling set by the correlation dimension of the attractor, which suggests that ever longer orbits may be needed as the Reynolds number grows.
Banner appropriate to article type will appear here in typeset article
Introduction
Over the past two decades a view of turbulence has emerged in which a turbulent trajectory shadows a skeleton of unstable invariant solutions of the Navier–Stokes equations, repelled from one towards others along their unstable manifolds [34, 29]. Unstable periodic orbits, which capture the cycles of growth and breakdown of coherent structures, play the central role in this skeleton, and since the first was found [33] many of relatively short period have been computed in canonical flows [72, 11, 8]. The main motivation for computing them is the prospect of a deterministic account of turbulent statistics. Periodic orbit theory expresses long-time averages as weighted sums over such orbits (Artuso et al. 1990a,b; Cvitanović et al. 2025), but its cycle expansions converge only once all the short orbits up to a given period are known, a requirement out of reach in turbulent flows of many degrees of freedom, although a complete set has recently been computed in a minimal Taylor–Couette flow ([73]). Libraries of a few tens to hundreds of short orbits therefore approximate the density of an observable by a weighted combination of those of the individual orbits, with equal weights or weights from periodic orbit theory ([11, 44]) or, more recently, with data-driven weights obtained a posteriori from how often the turbulent flow visits each orbit ([52]) or from a fit, by minimising the Kullback–Leibler divergence between the reconstructed and the turbulent densities ([58, 14]). The recent libraries of two-dimensional Kolmogorov flow, of a hundred or more mostly short orbits, reproduce the densities of the dissipation and of the energy closely, with weights that decrease with the instability of the orbits ([58]). Since each short orbit covers only a narrow range of the dissipation, these libraries need bursting orbits to reach the tail, and searches have targeted them explicitly ([50 b]).
Such reconstructions are a valuable test of the theory, but our focus is on a different question, one that relates to many problems in flow control, optimisation and system identification. A turbulence simulation does not tell us directly how the means change with the parameters of the flow. This sensitivity is often obtained with adjoint-based approaches when many parameters are involved, but on an open chaotic trajectory the adjoint solution grows exponentially ([41]). Libraries do not give this sensitivity directly either. Their weights are obtained at one set of parameters, and the reconstructions deteriorate when the orbits are continued in the Reynolds number ([14]). For this purpose we proposed to solve the adjoint problem on periodic orbits ([39]), where the periodicity condition removes the exponential growth, despite the instability, and gives a well-conditioned sensitivity of the mean over the orbit (see also Kazantsev [35]).
Noting that the sensitivity varies widely among short orbits, we then turned to one or a few orbits of long period ([40]). The hypothesis is that a long enough orbit captures all the relevant dynamical events, such as the interactions between scales and the transfer of energy among them ([71]), at a resolution sufficient for its probability distribution to model that of the chaotic state. The results in low-dimensional systems were encouraging. In the Lorenz equations and in a minimal Kuramoto–Sivashinsky system, orbits about two orders of magnitude longer than the shortest cycles, converged from near recurrences, have period averages and Floquet exponents that appear to approach, as the period grows, the long-time averages and the Lyapunov exponents of chaotic trajectories. The sensitivities of these orbits also converge as the period grows, but their limit is not necessarily the response of the chaotic state to finite changes of the parameters. Orbits may also meet bifurcations as the parameters change, and at a fold their sensitivities diverge, producing power-law tails in the probability density of the sensitivities. High-dimensional systems may behave better, since their statistics may respond smoothly to the parameters, as in the thermodynamic limit of Ruelle [60]. For turbulent flows the main question remains open, namely whether the statistics of a single long orbit approach those of the flow as the period grows. The yardstick is a turbulent segment of the same duration, and the test is demanding in a flow with rare bursts, since, unlike a library, a single long orbit must visit both the bulk and the tail on its own, in the right proportion. Matching the statistics is also a necessary condition for sensitivities, since an orbit that misses the means gives no reason to trust the derivative of its means. A related question is whether the Floquet exponents of long orbits approach the Lyapunov exponents of the flow, or whether the orbits that a search finds are special ones, perhaps quieter than the flow. More fundamentally, if a long orbit is to stand for the turbulent flow, one would like to know how well it represents turbulent trajectories in their entirety, that is, how close the flow stays to it, and how this changes as the period grows. Answering these questions requires long orbits, but computing them with the standard methods is hard (see the review of Ashtari et al. 2027).
Shooting has become the dominant method to locate these solutions, largely because it can be built on existing codes for direct numerical simulation (DNS). Matrix-free Newton–Krylov methods, developed for steady states (Edwards et al. 1994), with the generalised minimal residual method (GMRES) [62] for the Newton systems and a trust region for the steps, made it practical [72]. Time integration itself acts as a preconditioner for these systems (Tuckerman et al. 2019). Errors in the initial state, however, grow exponentially over the period, so that the radius of convergence of Newton’s method shrinks as the period and the instability of the orbit grow (Chandler & Kerswell 2013; Budanur et al. 2017). Multiple shooting limits this growth by splitting the period into segments joined by continuity conditions, at the cost of more unknowns [63], and has been used to find longer orbits of pipe flow (Budanur et al. 2017).
Variational methods [38] avoid the exponential growth of errors by treating the whole orbit as a single unknown, a closed loop in state space or, equivalently, a field over space and time, which is deformed until it satisfies the governing equations everywhere along it (Figure 1). They are closely related to multiple shooting, since multiple shooting with one time step per segment is itself a discretisation of the whole loop. Which of the two is more robust in practical terms remains a point of discussion in the community. Lan & Cvitanović [38] deformed the loop by a Newton descent, which converges quadratically near an orbit and has converged very long orbits of low-dimensional systems [40], where the whole linear system can be factorised. For a three-dimensional flow, however, the loop is a four-dimensional field, whose Jacobian is far too large to be stored and factorised, and Newton’s method on the loop has been judged too expensive for fluid flows (Farazmand 2016; Azimi et al. 2022; Zheng et al. 2025). Fazendeiro et al. (2010) instead minimised the residual of the loop by gradient descent, which needs no linear solve, and found periodic orbits of weak three-dimensional turbulence, on tens of thousands of processor cores. Adjoint-based methods descend the residual in the same spirit, with a gradient that only requires the application of a differential operator local in time (Farazmand 2016; Azimi et al. 2022; Parker & Schneider 2022). They converge from rough guesses, but only linearly [3].
Figure 1. The space-time residual for the normal form of a Hopf bifurcation, , , whose limit cycle is the unit circle. (a) A loop that is not an orbit, an ellipse with frequency , along which its tangent (blue) and the vector field (orange) differ by the residual. (b) The orbit, to which the vector field is everywhere tangent. Grey lines are trajectories of the system.
The alternative is to keep Newton’s method and solve the linear systems iteratively, but the obstacle is then preconditioning [36]. This problem has long been faced in computational fluid dynamics by the harmonic-balance and time-spectral methods, which solve for a whole period of a periodic flow at once (Hall et al. 2002; McMullen et al. 2006; Thomas et al. 2013; Leffell et al. 2017). Their linear systems couple the instants of the period through the spectral derivative in time, which is diagonal on the harmonics, and the points of the flow at each instant through the spatial operator. The approximate factorisation of Thomas et al. (2013) replaces the sum of the two operators by their product, so that the two factors can be inverted separately, the temporal one at each point and the spatial one at each instant, and Mundis & Mavriplis [47] and Ramezanian et al. (2020) use this product as the preconditioner of GMRES. In these applications a few harmonics usually suffice (Hall et al. 2002; McMullen et al. 2006). Harmonic balance has more recently reached the fluid mechanics community, to compute limit cycles (Douglas et al. 2021; Sierra-Ausín et al. 2022), their sensitivity (Sierra et al. 2021) and the response of transitional flows to forcing periodic in time and in the spanwise direction (Rigas et al. 2021; Poulain et al. 2024), and, in linear form, to analyse the amplification of perturbations about periodic flows (Padovan et al. 2020). Its preconditioners there treat each harmonic separately, linearised about the mean flow, and work best when the harmonics interact weakly [59, 19]. Our problem, however, is more complex. The unstable periodic orbits of turbulent flows are multiscale in space and time, their harmonics interact strongly, and the number of harmonics they need grows in proportion to the period, so that a long orbit may need hundreds or thousands of them, not just a handful. Making Newton’s method on the loop practical for such orbits is the first purpose of this paper.
To this end, we build on this approximate factorisation, which we use as the preconditioner of flexible GMRES within a trust-region Newton method on the loop, and implement the method on graphics processing units (GPUs). The second purpose is to use the method to explore the questions above in two-dimensional Kolmogorov flow at a Reynolds number of 40, a standard testbed for the invariant solutions of turbulence [11, 21, 54, 52]. As in our earlier work, we do not seek a complete hierarchy of orbits, but the properties of the long orbits that searches from near recurrences can find. Sensitivities, the goal that motivates this work, are not computed here. We examine how these searches depend on their initial loops, take a first step towards a comparison with multiple shooting from the same seeds, compare the statistics and the instability of the long orbits with those of the flow, and measure how close the flow stays to them, at this one Reynolds number.
Section 2 formulates the search for orbits as a space-time root-finding problem, and § 3 presents the method. Sections 4 and 5 analyse the initial loops and the convergence of the searches. Section 6 reports the orbits found, § 7 verifies them by multiple shooting and takes a first step towards comparing the two methods, and § 8 relates the long orbits to the turbulent flow. Section 9 concludes.
Formulation
We introduce Kolmogorov flow, its vorticity formulation and its symmetries (§ 2.1), and pose the computation of its relative periodic orbits as a root-finding problem on a space of closed loops (§ 2.2).
Kolmogorov flow
We consider an incompressible fluid in a doubly periodic square of side , driven along by a body force of amplitude per unit mass that varies sinusoidally in , with wavelengths across the domain. With lengths scaled by and velocities by , as in Chandler & Kerswell [11], the domain becomes the torus , with position , and the Reynolds number is , with the kinematic viscosity. We work directly with the vorticity of the velocity , for which we write rather than the customary , reserved for the frequency of an orbit of period . It obeys
where is the velocity of zero mean induced by , with the inverse Laplacian on the fields of zero mean, and the skew gradient. In this form incompressibility holds by construction, which spares us from dealing with the pressure [3], a problem we leave to future work.
We take and , a widely studied configuration (§ 1), so that our orbits can be compared with those of other methods. The laminar flow is then unstable and the flow is weakly chaotic, spending most of its time in quiet states in the bulk of the attractor, which it leaves in intermittent bursts of dissipation [55, 11, 23, 50]. Denoting by the average of a function over the domain, the kinetic energy, the energy input and the dissipation,
satisfy , and we report them relative to their laminar values, and .
The equations are equivariant under the translations along the force, which form a continuous group, and under a discrete shift-reflect, whose square is a translation by across the force, and a rotation by [see 11].
Relative periodic orbits as a space-time problem
Because of the continuous symmetry, the recurrent solutions of (1) are relative periodic orbits, which repeat after a period up to a translation by a shift ,
The discrete symmetries permit further orbits, which repeat up to a translation across the force or a shift-reflect, but as in Parker & Schneider [54] and Page et al. [52] we consider only orbits of the form (3).
The unknown period and shift , or equivalently the frequency and the drift speed , are built into the equations by a change of variables. In the frame that drifts along at the speed the orbit is periodic, and in the rescaled time its period is , whatever . By the chain rule, and since commutes with translations, (1) becomes
in which and are unknowns together with the field.
In these variables the orbit is periodic in both directions of space and in the rescaled time, and thus a real field on the three-dimensional torus , which traces a closed loop in state space. We seek it among the loops of zero spatial mean at every instant, since the vorticity of a periodic velocity field has zero mean,
where the fields are square integrable together with the derivatives that appear in (4). We equip with the inner product
and the norm . At one instant we use, with the star the complex conjugate, the inner product of two fields and its norm , so that the average of over the period is . Measuring the vorticity rather than the velocity, both products weight the small scales more heavily than the energy norm would, which suits a residual dominated by temporal and spatial derivatives.
The frequency enters the problem through its logarithm , which keeps it positive during the iterations and makes its changes relative to its value, and the unknowns are collected in
The orbits are then the zeros of the space-time residual
The residual (8) maps into , since the derivatives and the equivariant operator preserve the zero mean and the periodicity, and measures how far a candidate loop is from satisfying (4). Geometrically, it is the difference between the tangent of the loop in state space and the vector field of the drifting frame evaluated on it (figure 1). Similar formulations, in a rescaled time and, for relative solutions, a frame that follows their symmetry, have been used for the periodic orbits of the Kuramoto–Sivashinsky equation [38, 76], the relative periodic orbits of the cubic–quintic Ginzburg–Landau equation, whose phase rotates over the period [76], and the travelling waves and relative periodic orbits of Kolmogorov flow [21, 54].
Newton–Krylov method for the space-time problem
We formulate Newton’s method for the root-finding problem in the space , independently of the discretisation. We then discretise the loops, precondition the linear systems, solve them by flexible GMRES with the steps restricted to a trust region, and describe the implementation on GPUs and the resolution in time.
The Newton system
Newton’s method corrects an iterate by , which cancels the residual to first order. The correction changes the whole loop at once, through the space-time field . Expanding (8) about , with , gives
where
is the vorticity equation linearised about the current loop, and
is the linearisation of at the instant , which acts on each instant separately. Setting (9) to zero leaves the correction undetermined, since the resulting system has two more unknowns than equations, and , and is singular at an orbit. The translational symmetries in and in imply that a shifted solution is also a solution, so that and span, generically, the null space of at an orbit. As in shooting methods, two phase conditions,
exclude these shifts from the correction. They add two rows to , which fix the components of the correction along the null space. The derivatives of the residual with respect to and add two columns, through which a change of the period and of the drift speed removes the part of the residual outside the range of . The bordered Newton system
is thus invertible at a non-degenerate orbit, although is singular there [28]. It has the same bordered structure as the Newton descent of Lan & Cvitanović [38], with one more row and column for the drift. For the remainder of this section the subscript is dropped from the Newton system, and kept only on the current iterate, , and .
The Newton system (13) is far too large to be formed, and we solve it iteratively by GMRES [62], right-preconditioned by an operator that approximates . GMRES minimises the residual of the system over a Krylov space and needs only products of with vectors (§ 3.4).
Discretisation in space and time
The preconditioner is built from the structure of on the discretised loop, which we therefore describe first. We replace by the space spanned by the Fourier modes with wavenumbers , , and temporal harmonics , in which a loop reads
where the coefficients of vanish. We take , and store the loop by the spatial Fourier coefficients of its vorticity at the equispaced instants , . Derivatives are applied spectrally, the one in by fast Fourier transforms in time. The nonlinear term is formed at each instant on a physical grid of points, with the aliasing removed by the rule, a resolution comparable with that of previous work at this Reynolds number [44]. In time the products are not dealiased, since dealiasing them leaves the residual practically unchanged, as we verified on the initial loops of § 4.
Preconditioning the Newton systems
GMRES converges quickly when the eigenvalues of the operator are clustered away from the origin, and slowly when they fill a wide region of the complex plane [68 Lecture 35], as those of do (§ 5.1). The preconditioner should thus be close enough to that the preconditioned operator is close to the identity, yet far easier to invert than . It acts on the field and on the two parameters of the correction separately. On the parameters it scales and by the norms of their columns in (3.5), and , so that these columns of the preconditioned operator have unit norm. On the field, its block follows from the structure of , the sum of three terms,
the time derivative, the drift and the viscous term, and the convective term. The time derivative couples all instants but acts on each spatial Fourier mode separately. The drift and the viscous term are diagonal in the spatial Fourier modes and act on each instant separately. The convective term acts on each instant separately but couples all spatial scales and, since its coefficients vary along the period, all temporal harmonics. Which of these couplings is simple depends on the representation of the loop. In the time domain, where the loop is represented by its fields at the instants, as it is stored (§ 3.2), the field block reads
with the entries of the differentiation matrix of trigonometric interpolation, as in time-spectral methods [47]. In the frequency domain, where the loop is represented by its space-time Fourier modes (3.6), it reads instead
where the block in row and column of the convective term is , the harmonic of the periodic operator . No representation makes the whole field block simple, but each part is simple in one of the two.
In the space-time Fourier modes, the diagonal part suggests the simplest preconditioner, , which multiplies the mode by
These numbers fill a region of the complex plane that extends to along the real axis and to about along the imaginary axis, on which GMRES converges slowly. This is the space-time form of Stokes preconditioning, in which the inverse of the viscous operator preconditions the Newton systems of steady states and travelling waves, and which weakens as advection strengthens [69, 70].
A preconditioner of this kind, which keeps only the highest derivatives in space and time, suffices for the Kuramoto–Sivashinsky and cubic–quintic Ginzburg–Landau equations in one space dimension [76], as we also found for the Kuramoto–Sivashinsky equation, but in Kolmogorov flow the convective term dominates the spectrum of the Jacobian (§ 5.1) and couples the temporal harmonics strongly. In the harmonic-balance computations of weakly forced shear flows, by contrast, the harmonics interact weakly, and a block-diagonal preconditioner, each block linearised about the mean flow, is efficient [59]. Here such a preconditioner is not enough, whether about the mean shear or about the time-averaged flow (not shown), since the advection by the loop changes in time.
A preconditioner for Kolmogorov flow must therefore retain the convective term, as Tuckerman et al. [70] suggest for Stokes preconditioning, and retain it about the loop itself. Writing , with as the spatial part, we follow the approximate factorisation of the time-spectral solvers [67], used as a preconditioner of GMRES by Mundis & Mavriplis [47] and Ramezanian et al. [57], and replace this sum by a product of a temporal factor, diagonal on the harmonics, and a spatial factor, block diagonal on the instants. Both factors are shifted by a positive number , the inverse of a pseudo-time step in the time-spectral solvers [67]. The shift makes the temporal factor invertible, by moving the zero of the time derivative on the mean harmonic to , while in practice we found the spatial factor to remain invertible for the values of we used. Divided by , their product
contains , with the error of the factorisation. The inverse of inverts each factor in its own representation,
with and the discrete Fourier transform in time. Read from right to left, applies the Fourier transform in time, divides each harmonic by , which inverts the temporal factor, transforms back to the instants, and then solves the spatial block of each instant independently, which inverts the spatial factor, convection included. The shift decides which modes are well conditioned. We tested fixed shifts between 0.5 and 4 along whole searches, and found that suits the short loops and a smaller shift the longer ones (not shown).
It remains to invert the spatial blocks, one per instant,
which we do by ten steps of GMRES, right-preconditioned by . This operator multiplies each spatial Fourier mode by and is thus inverted exactly. With and represented by their spatial Fourier modes, GMRES sees the operator
with the discrete Fourier transform in space. Read from right to left, its second term divides each spatial Fourier mode by , forms the gradient and the induced velocity of the result, transforms them to the physical grid, where they multiply the velocity and the vorticity gradient of the loop at the instant , and transforms the products back. Since convection involves at most first derivatives while grows as , the perturbation of the identity decays at high wavenumbers, and GMRES reduces the residual of every preconditioned block by about a decade in ten steps, at every instant and at the spatial resolutions we tested, which has proved sufficient. Including also the advection by the mean shear in this preconditioner slowed the inner solves (not shown). The block systems are independent but solved together. Every inner step applies all the blocks at once, as one application of the linearised operator to the whole loop, so that the inner solve takes ten such applications.
The borders of the Newton system are not approximated. The whole preconditioner, , combines the field preconditioner with the scaling of the parameters, and the preconditioned Newton operator reads
the preconditioned field block, extended by the identity on the parameters, plus a matrix that is nonzero only in its last two columns and rows and has rank at most four. In practice GMRES converges on the Newton system about as it would on the field block alone, with a few more steps (not shown), and the preconditioner needs to approximate only .
The preconditioner is necessary for convergence (§ 5.1), but it is expensive. With ten inner steps, an application of takes as long as 13 to 21 products with on an NVIDIA H200 GPU, as detailed in appendix A, so that the inner steps take most of the time spent on the operators in an Arnoldi step of GMRES (§ 3.4). This time is the price of retaining convection.
Krylov solution and the hookstep
The preconditioner enters on the right of , as in (3.15), and the correction is written as
and GMRES solves
Since , right preconditioning leaves unchanged the residual of the Newton system that GMRES minimises, and changes only the space in which the correction is sought. Each Arnoldi step adds to the basis of a loop, a space-time field together with two scalars, at the cost of one application of and one product with , and orthogonalises it against the previous ones by two passes of the Gram–Schmidt process. The orthonormal basis satisfies the Arnoldi relation , with upper Hessenberg of size , so that writing turns (3.17) into a small least-squares problem with for the coordinates , with .
The inner GMRES steps of §3.3, however, do not apply as a fixed linear operator, so that the preconditioner changes slightly from one Arnoldi step to the next. Standard GMRES tolerates this change only if the inner solutions are accurate, and the searches then often fail with fewer than twenty inner steps. The flexible GMRES of Saad [61] admits a changing preconditioner, and the searches converge with ten inner steps, in about half the wall time of standard GMRES with twenty (not shown). It stores the preconditioned vectors of the Arnoldi vectors in a matrix , with , so that the correction follows from the same coordinates .
Newton’s method converges only from loops close to an orbit. Following Viswanath [72], we therefore restrict the correction to a trust region of radius . The change of variables (3.16), which leaves the residual of the Newton system unchanged, sets instead the metric in which the length of the correction is measured, , similar in spirit to the scaling of the adjoint descent of Farazmand [21]. With , for instance, the trust region is an ellipsoid with semi-axes , shorter along the high frequencies and wavenumbers that amplifies. In the coordinates , where for a fixed preconditioner, the step solves the comparatively small hookstep problem
computed in negligible time through the singular value decomposition of .
We measure the progress of a search by the global residual , with the space-time residual (2.8), which vanishes only at the orbits and which near is approximated by the linear model . The ratio of the actual reduction of to the reduction predicted by the model, which follows from (3.18) without products with the Jacobian, measures how well the model holds. The radius adapts to by the classical rules of Dennis & Schnabel [17], and a step is accepted when . A rejected step is solved again at a smaller radius in the same Krylov space.
The Arnoldi iteration stops when the GMRES relative residual falls below , since far from the orbit the Newton system need not be solved exactly. In the campaigns of §6, with ten to twenty inner steps, a search stops when falls below in double precision or in single precision, after 50 Newton iterations, or when the residual has not halved over the last ten to fifteen iterations.
Implementation
The method is implemented in the Julia language. Its solver needs only the residual, the product of the Jacobian with a vector, the application of a preconditioner and an inner product, and so applies to other systems as well. For Kolmogorov flow the product with applies the linearised equation (3.3) exactly, while for systems whose linearisation is not available it could be approximated by a finite difference of residuals, as in Jacobian-free Newton–Krylov methods ([36]). Every operator of Kolmogorov flow is a short sequence of fast Fourier transforms and element-wise operations on one three-dimensional array, which holds the spatial Fourier modes of the vorticity at all the instants. The same code runs on central processing units (CPUs) and on GPUs, but this structure suits GPUs, which reach their throughput only when they have enough independent work to hide the latency of their memory. Time stepping offers such work within each instant and, in multiple shooting, across the segments, but the instants of a segment are computed one after the other. The space-time formulation adds time as a further direction of parallelism, as parallel-in-time methods do. On a GPU the operators run tens to hundreds of times faster than on one CPU core (Appendix § 10). An Arnoldi step of the searches takes about 0.2 s on an NVIDIA H200 GPU for a loop of a thousand instants.
Without the mean mode, each instant carries real unknowns, so that the size of a loop grows in proportion to its period. The longest orbit we found, of period 190.79, was converged on 1575 instants, about eight per unit time, with unknowns that occupy 28 MB in single precision. The memory that a search needs is set instead by its Krylov space, since flexible GMRES stores two loops for every Krylov vector, and with the present preconditioner loops of period 100 use about a thousand vectors (§ 5.3). The preconditioner reduces the number of Arnoldi steps, and with it the size of the Krylov space and the time of its orthogonalisation, which grows as the square of the number of steps. Most searches ran in single precision, which halves the memory, and on four to eight instants per unit time (§ 6.1).
The number of instants per unit time sets the temporal resolution, which we judge by the temporal spectrum of the loop, the energies
of its temporal harmonics, which add up to . The temporal spectra of the orbits we converged fall steeply, even for those with the most active dynamics, and on four instants per unit time they span at least six decades. Figure 2 and Table 1 illustrate this on P1, a short periodic orbit, converged again on one, two and four instants per unit time, and on RPO114, a relative periodic orbit that visits a burst, converged again on two, four and six instants per unit time. Here and below we name the solutions of Chandler & Kerswell [11] as they did, and the others by their labels in table S1 of the supplementary material. RPO114 has nearly the temporal spectrum of the turbulent flow, while that of P1 falls much faster. Yet on four instants per unit time, that is 121 temporal harmonics, RPO114 agrees with its version on about ten instants per unit time to about five digits (Table 1), although its dissipation nearly triples during the burst and falls back within two time units (Figure 2e). On six instants per unit time the agreement improves to five to eight digits, and on two it falls to one to three.
Figure 2. The effect of the resolution in time, on the periodic orbit P1 of Chandler & Kerswell [11], of period 5.38 (a, b), and on RPO114, of period 60.80, which visits a burst (c–e). P1 is converged on one, two and four instants per unit time (blue, red and green) and RPO114 on two, four and six (red, green and purple), both also on about ten (grey). (a, c) The dissipation along the period, and (e) the burst of RPO114 in the window marked in (c). (b, d) The temporal spectrum (3.19), relative to , over that of the turbulent flow (black), with the level and the highest frequency resolved on four instants per unit time (dashed).
| P1 | 5 | 2 | 0.9 | 5.36885181 | 0 | 0.09270589 | 0.1010344 |
| 11 | 5 | 2.0 | 5.38044894 | 0 | 0.09280488 | 0.1011521 | |
| 21 | 10 | 3.9 | 5.38039990 | 0 | 0.09280455 | 0.1011520 | |
| 63 | 31 | 11.7 | 5.38039990 | 0 | 0.09280455 | 0.1011520 | |
| RPO114 | 121 | 60 | 2.0 | 60.86203797 | 2.89346129 | 0.09407992 | 0.2715097 |
| 243 | 121 | 4.0 | 60.80369687 | 2.90349719 | 0.09417033 | 0.2730566 | |
| 375 | 187 | 6.2 | 60.80397746 | 2.90350094 | 0.09417014 | 0.2730501 | |
| 625 | 312 | 10.3 | 60.80397757 | 2.90350061 | 0.09417014 | 0.2730574 |
Table 1. The orbits of figure 2 on instants, temporal harmonics and instants per unit time, with their period, shift, and mean and largest dissipation. Underlined digits agree with the orbits of the first campaign (§6.1), on 63 and 625 instants.
Initial loops from near recurrences
Near recurrences of long period
As in previous studies ([33, 72, 11, 44]), we take initial loops from near recurrences of long simulations of (2.1), the local minima of the relative distance
which Figure 3(a) shows along one simulation for and up to 2000, and Figure 3(c) in a small window. The distance is homogeneous over this map, in and, beyond about a hundred time units, in . Two states a long time apart are thus far apart as independent states of the attractor, and the distribution of over no longer changes (Figure 3b). This suggests that recurrences as good as those of moderate period can be found at much longer ones. We keep the minima of below 0.25 (Figure 3c), a distance that independent states rarely reach. Using this criterion up to a period of 200, we obtained a database of near recurrences, of which the searches of § 6 used 3 530 as seeds (§ 6.1).
Figure 3. Near recurrences of the turbulent flow. (a) The distance (28) for and up to 2 000 along one simulation, with its local minima below the threshold 0.25 (red), and (c) the box marked in (a). (b) The median, the quartiles and the 1% and 0.1% quantiles of over , in bins of , over sixty-four simulations. (d) The correlation integral , the fraction of pairs of states closer than , in three bands of , with a power law fitted to the pairs with for (dashed). (e) The rate of near recurrences per unit time, unit period and unit distance. Red dashed lines mark the threshold.
Near recurrences concentrate at short periods (Figure 3e). These close recurrences cluster around the period 5.38 of the periodic orbit P1 of Chandler & Kerswell [11] and its multiples, an orbit that Page et al. [52] also recover frequently. There are also clusters around the period 12.21 of the relative periodic orbit R19 and, at the shortest periods, just below the period 2.83 of the periodic orbit P2. These represent short orbits that the flow visits often. Above these clusters fade, and beyond about a hundred time units the rate of near recurrences per unit time, per unit period and per unit distance becomes nearly
invariant with , since the probability that two states come within the threshold, which sets this rate, no longer depends on (Figure 3b, Figure 3d).
The fraction of pairs closer than in Figure 3(d), the probability that two independent states lie within of each other, is the correlation integral of Grassberger & Procaccia [30]. Its slope, about seven (dashed line) and rising slowly as falls, estimates the correlation dimension of the attractor without the translation along , below its Kaplan–Yorke dimension of about 10.5, also without the translation, given by the Lyapunov exponents of § 8.1. The threshold 0.25 that we chose thus corresponds to a probability of about for independent states.
That this probability does not depend on may seem somewhat counterintuitive, since in a uniformly hyperbolic flow a segment that nearly closes follows a periodic orbit over its whole length. While the probability that a trajectory stays near a given orbit decays exponentially with the period, as , with the real part of the sum of its unstable Floquet exponents (§ 8.1), the number of orbits also grows exponentially with the period, as , with the topological entropy (Cvitanović et al. 2025, §21.3), and the two factors balance. At , for instance, taking for both rates the sum of the positive Lyapunov exponents, about 0.27 (§ 8.1), a trajectory stays near a given orbit with a probability of order , but there should be of the order of orbits. Kolmogorov flow is not uniformly hyperbolic, however, and this argument is only a guide. What becomes rare with the period is not a near recurrence but a second near recurrence of the same orbit.
Smoothing the jump of the loop
A question we asked from the outset is how much convergence depends on the initial loop, and on its residual in particular. The loop given by a near recurrence has a jump of norm where its end meets its start. Spectral differentiation in time turns the jump into a peak of the residual at the junction, which decays only as with the time from the junction (Figure 4c). The jump makes the loop differ from a smooth one by a sawtooth, whose temporal harmonics have amplitudes . Differentiation in time multiplies harmonic by , and the jump thus pollutes the whole temporal spectrum (3.19) of the residual, with about the same energy in every harmonic (Figure 4d).
Figure 4. Smoothing the jump of an initial loop of period 108.8 that passes through a burst, on eight instants per unit time, by low-pass filters at three cut-offs and by the weighted average (4.2), against the time from the junction, with the window shaded. (a) The real part of the vorticity coefficient that jumps most at the junction, , of wavenumber . (b) The relative change of the loop at each instant. (c) The norm of the residual at each instant and (d) its temporal spectrum (3.19) against the frequency , both with the weighted average also on six and ten instants per unit time and with the jump alone (dashed).
A common approach is to low-pass filter the loop in time [38, 5]. This smooths the jump but also moves the loop off the trajectory at every instant, the more so the lower the cut-off frequency (Figure 4a, Figure 4b). Thus creates a residual away from the junction, where that of the loop as built is small (Figure 4c). Expecting a smaller initial residual to help convergence, we initially adopted instead, and ran most of our searches with, a smoothing localised in time, which replaces the loop near the junction by a weighted average of the trajectories that enter and leave it. It assumes that, far enough from the junction, the loop lies close to an orbit and needs no change. Within a window about the junction the smoothed loop is
The weight ensures that the loop passes smoothly from the end of the segment to its start, with a continuous derivative in time (Figure 4a). The weighted average confines the residual to the window, where it leaves a plateau with steep edges (Figure 4c), and replaces the flat spectrum of the jump by one that falls by several decades (Figure 4d). Outside the window only the error of the spectral derivative of the sampled trajectory remains. The instants cannot resolve the fastest variations of the trajectory, which end up in the highest harmonics, where the derivative amplifies them most, so that the spectrum rises again near the last harmonics. As the number of instants per unit time grows, this error decreases and
| seeds converged | initial | Newton iterations | hooksteps | total Arnoldi steps | |
| weighted average | 112 (6) | 0.042 | 16 | 14 | 6517 |
| low-pass filter | 107 (2) | 0.106 | 15 | 13 | 6383 |
| as built | 105 (1) | 0.235 | 16 | 14 | 6371 |
the rise moves to higher frequencies and lower energies (Figure 4c, Figure 4d). On the initial loops we tested, the weighted average lowers the norm (6) of the residual several-fold with respect to the loop as built for half-widths between 1 and 4, while on wider windows the two trajectories separate and the residual grows steeply (not shown). We therefore applied the weighted average with to all the initial loops of the campaigns (§ 6.1).
Whether this smoothing matters for convergence is a question we returned to a posteriori. We searched again the seeds of the 112 distinct orbits that the third campaign had found, and as many random seeds of the same periods whose first search had failed, from loops low-pass filtered at one cycle per unit time and from the loops without any smoothing, with otherwise the same settings. The weighted average has only a marginal effect on convergence, compared with the low-pass filter and even with the loops as built (Table 2). With it the searches converge just a few more seeds than with either, and where they converge they mostly reach the same orbit. Despite initial residuals several times larger, the low-pass filtered loops and the loops as built reach the same orbits in about as many Newton iterations and Arnoldi steps. The residual of the initial loop is therefore not, by itself, a criterion for whether a search converges.
| seeds converged | initial | Newton iterations | hooksteps | total Arnoldi steps | |
| weighted average | 112 (6) | 0.042 | 16 | 14 | 6517 |
| low-pass filter | 107 (2) | 0.106 | 15 | 13 | 6383 |
| as built | 105 (1) | 0.235 | 16 | 14 | 6371 |
Table 2. The effect of smoothing on the searches from 224 seeds of the third campaign, half of them the seeds of the 112 distinct orbits it had found by then and half seeds whose first search failed, from loops smoothed by the weighted average (4.2), low-pass filtered at one cycle per unit time, or left as built. In brackets, the seeds that converge with that smoothing only. The medians are over the 87 seeds that converge to the same orbit with all three, and the hooksteps are the steps on the boundary of the trust region.
Convergence of the searches
We illustrate the behaviour of the method on single searches. The Newton system of one initial loop shows the action of the preconditioner (§ 5.1), two searches show how hooksteps and inexact Newton steps share the work and how a search stalls, and a third how the Newton correction differs from the gradient (§ 5.2), and the searches of the campaigns give the wall time and the memory (§ 5.3).
The Newton systems
The preconditioner is meant to cluster the eigenvalues away from the origin (§ 3.3). Figure 5 shows its effect on the Newton system of the initial loop of period 5.4 that converges to P1. The grey dots are the eigenvalues (18) of the diagonal part of the Jacobian, one ladder for each spatial wavenumber with rungs spaced by , which together fill a rectangle of width , set by the viscous term, and half-height , set by the time derivative (Figure 5a). Advection by the loop stretches the spectrum of the full Jacobian much further, to imaginary parts far larger than , a sign that advection is fundamental to this problem. The outer part of the spectrum, which the Ritz values approximate, no longer changes between 400 and 2 000 Arnoldi steps, for the Jacobian as for the two preconditioned operators (not shown). The preconditioner , which inverts the time derivative, the drift and the viscous term but not convection, spreads the spectrum around the origin (Figure 5b), and GMRES does worse with it than without (Figure 5d). With the factorised preconditioner the Ritz values gather about the circle of diameter one centred at one half (Figure 5c), which encloses the eigenvalues of preconditioned in the same way, and GMRES converges within a few hundred Arnoldi steps, a number that does not change when the loop is refined twofold in space or in time (Figure 5d). The Ritz values of longer loops have the same structure, only more numerous (not shown), and on these loops GMRES first stays on a plateau that lengthens with the period (§5.3). On the initial loop of RPO178, the longest orbit that we found, the GMRES relative residual does not change either when the loop is refined by about 50% in space or in time (not shown).

Figure 5. The effect of the preconditioner on the Newton system of the initial loop of period 5.4 that converges to P1. (a) The Ritz values of after 2 000 Arnoldi steps, over the eigenvalues (3.10) of (grey), which fill the rectangle of width and half-height . (b) The same for and (c) for with , with the circle of diameter one centred at one half (dashed). (d) The GMRES relative residual against the Arnoldi step, also with twice the Fourier modes in space (circles) and in time (triangles).
The course of a search
Figure 6 shows two searches from loops of similar period, one that converges to RPO168 (Figure 6a–Figure 6d) and one that stalls (Figure 6e–Figure 6h). Their course follows from the form of the hookstep. Without preconditioner, a step on the boundary of the trust region is
Figure 6. The progress of two searches, one per row, from a loop of period 100.0 that converges to RPO168 (a–d) and from one of period 86.3 that stalls (e–h). (a, e) The residual at each Newton iteration, open for hooksteps on the boundary of the trust region and filled for Newton steps. (b, f) The radius of the trust region and the length of the step, with a cross at each rejected step. (c, g) The ratio of §3.4, with the thresholds 1/4, below which (shaded) the radius shrinks, and 3/4. (d, h) The GMRES relative residual against the Arnoldi step, one line per Newton iteration, light to dark.
with the adjoint of in the inner product (2.7) and such that . It tends to the Newton step on a large trust region and to the direction of steepest descent on a small one [17], and with the preconditioner the same holds within the Krylov space, in the metric of the preconditioner. The first search thus passes from a slow descent, with steps on the boundary of the trust region, to an inexact Newton iteration whose residual falls by about two decades per iteration, the relative tolerance of GMRES.
Far from an orbit the hookstep thus descends , as the adjoint-based variational methods do [21, 54], but on a quadratic model of , so that its steps also move in part along the Newton step, the more so as the model improves. With the robustness of descent thus built in, we did not find a separate stage of descent necessary. Like any descent, the hookstep may converge to minima of that are not orbits. [17], as the second search seems to do, never leaving the first phase while its residual stops falling and the trust region shrinks. Such minima may be ghosts of orbits that exist at nearby values of the parameters [77], or come from near recurrences whose ends are close in norm but far apart dynamically, perhaps on different sheets of a strongly folded attractor.
The adjoint-based methods compute the gradient of directly. Integrating by parts in , and in the inner product (2.6), as in Farazmand [21] and Parker & Schneider [54], gives
where, for Kolmogorov flow,
is the adjoint of the linearised operator (3.3). Figure 7 shows how the gradient and the Newton correction differ, on a search from a loop of period 42.9 that converges to RPO78. The residual of the initial loop is largest in the window of the weighted average (figure 7a). The first Newton correction spreads over the whole loop (figure 7b), since the Newton system couples all instants, and lowers the residual at the junction while raising it elsewhere to about , after which the residual falls at every instant at a similar rate (figure 7d). The gradient, local in time, is confined to the window (figure 7c), so that a correction by descent reaches the rest of the loop only over many steps, which helps to explain why methods of descent converge slowly. A gradient, however, costs about one product with , far less than a hookstep, which builds a whole Krylov space. A first stage of descent, as is done with shooting ([52, 49]), may thus still save time, and is left to future work.
Figure 7. The first Newton correction and the gradient in a search from a loop of period 42.9 that converges to RPO78. (a) The residual of the initial loop, (b) the correction of the first Newton iteration and (c) the gradient (31) of on the initial loop, at , against and . (d) The norm of the residual at each instant, at the start (orange) and every third Newton iteration (light to dark), with the window of the weighted average shaded, its end dashed in (a–c).
Wall time and memory
The Newton iterations of the searches that converge grow only slowly with the period of the initial loop (Figure 8a), but the Arnoldi steps per Newton iteration grow linearly with it beyond the shortest loops (Figure 8b). The extra steps form the plateau of the GMRES histories (Figure 6d, Figure 6h), on which GMRES resolves the eigenvalues of near the origin one by one ([68 Lecture 35]). They belong to the slow scales of the loop, on which is small and the preconditioner is about , so that it leaves them near the origin. Their number grows with the period, as the plateau does, and the corresponding Ritz vectors have most of their energy at temporal frequencies below (not shown).
Figure 8. The Newton iterations, Arnoldi steps and wall time of the searches against the period of the initial loop. (a) The Newton iterations of the searches that converge. (b) The median number of Arnoldi steps per Newton iteration, for the searches that converge (blue) and those that do not (grey crosses). (c) The wall time of the searches that converge. The dashed lines are proportional to in (b) and to in (c).
Since the time of an Arnoldi step also grows in proportion to the period (appendix § 10), the wall time of the searches that converge grows unfavourably, about as the square of the period (Figure 8c), and so does the memory of the Krylov space, whose vectors grow in number and in size. This suggests that further algorithmic steps are needed to bring this scaling to a more favourable order in the period. The same growth set the longest periods of our campaigns, about 200, and may contribute to the stalling of some of the longest searches, which fill their Krylov space before GMRES reaches its tolerance (Figure 8b), although a search may still converge with a truncated Krylov space, if more slowly.
Campaigns of searches
We report the three campaigns (§ 6.1), the seeds that converge (§ 6.2) and the orbits found, focusing on the long ones (§ 6.3).
The campaigns
The searches ran in three campaigns as the method matured (Table 3), on 3 530 distinct seeds, some searched in more than one campaign. The seeds are near recurrences of § 4, of which we keep only the closest among those of the same simulation within 50 time units and 10% in period of each other, which are taken to follow the same orbit. Several GPUs took the seeds in turn from classes of period ten time units wide, so that all periods were searched from the start, for 1 515 GPU hours in all, about 23 minutes per search. Late in the third campaign we also tried the hookstep at several radii in the same Krylov space, keeping the step of smallest residual, and restricted the seeds, first to periods above 20 and then mostly to periods above 50 whose dissipation stays below 0.16 of the laminar value, preferring the seeds of larger dissipation. The seeds searched are therefore not a uniform sample in period or in dissipation. A converged orbit counts as new unless one of the same kind was found before with the same mean dissipation, to a relative difference of , and a period in an integer ratio to its own, a loose tolerance because the searches in single precision fix the mean dissipation only to a few parts in .
| campaign | precision | Krylov | searches | converged | solutions | ||
| first | 10 | double | 2.5 | 600–1 000 | 494 | 107 | 16 |
| second | 4 | mixed, single | 1, 2.5 | 1 500 | 537 | 188 | 52 |
| third | 8, 6–7 | single | 1 | 1 500–3 000 | 2 931 | 736 | 140 |
| all | 3 962 | 1 031 | 183 |
Table 3. The three campaigns, with the instants per unit time, the precision, the shift of the preconditioner, the largest Krylov space and the numbers of searches, converged searches and distinct solutions, which overlap between campaigns. Mixed precision is single precision down to and double precision after. The searches ran on NVIDIA L4, L40S, A100 and H200 GPUs, with 24, 48, 80 and 141 GB of memory.
Which seeds converge
Of the 3 962 searches, 1 031 converged, to 183 distinct solutions, 178 of them relative periodic orbits. The longest, RPO178, of period 190.79, is more than twice as long as the longest of Beck & Schneider [6], of period 74.07. The fraction of searches that converge falls steeply with the period of the seed (Figure 9a). This fall is commonly ascribed to the Floquet multipliers of long orbits, which grow exponentially with the period, but this growth need not make good seeds rarer (§ 4). Part of the difficulty may lie in the large Krylov spaces of the longest loops (§ 5.3), although multiple shooting also fails on almost all the seeds of period above 100 that we tried (§ 7). Another part may come from the bursts, which a longer seed is more likely to approach, since the fraction also falls with the largest dissipation along the seed, at similar periods (Figure 9b). Over all periods, the quietest seeds mostly converge, to orbits in the bulk of the attractor, while only a few per cent of those that approach the bursts do so, mostly to long bursting orbits.

Figure 9. The outcome of the 3 962 searches. (a) The fraction of searches that converge, in bins of the period of the seed, and the distinct orbits reached per search (triangles). (b) The same fraction in bins of the largest dissipation along the seed, for all the seeds (blue) and for (grey), without bins of fewer than five searches. (c) The number of searches that converge to each periodic (diamonds) and relative periodic orbit (discs) against its period, without the travelling wave and the equilibrium, found 308 and 73 times.
Most converged searches are duplicates. Two thirds of them, and almost all of those from seeds of period below 20, return to four short solutions of [11], the equilibrium E1, the travelling wave T1, the periodic orbit P1 and the relative periodic orbit R19 (Figure 9c), since E1 and T1 solve the space-time problem at any period and P1 and R19 at multiples of their own. The turbulent flow visits these solutions often [11], so that near recurrences cluster around the periods of P1 and R19 (§ 4) and multiple shooting returns to them as well (not shown). Most long orbits, by contrast, were found by a single search, since near recurrences of long period rarely return to the same orbit (§ 4), as for the long orbits of low-dimensional systems [40]. Above a period of about 40, almost every search that converges thus finds a new orbit (Figure 9a).
The orbits found
Our solutions, listed in table S1 of the supplementary material, barely overlap with the short orbits of the two libraries of Page et al. (Page et al. 2024a,b; Cleary & Page 2025b), with only seven orbits in both, and occupy different ranges of the energy input and of the dissipation (Figure 10a). These libraries are collected at invariant-sets.org, together with several thousand equilibria and travelling waves at this Reynolds number [27]. Ours are mostly long and quiet, almost all with a mean dissipation below that of the turbulent flow. Nearly half of the orbits of [11] and few of those of [52] are among ours, while only a handful of the solutions of [58] are in either set. Our searches thus appear to miss the short bursting orbits that other methods find, as Figure 9(b) suggests, although the form (2.3), which excludes orbits that close with a discrete symmetry, and the distance (4.1) may also play a part.

Figure 10. The orbits at in the plane of the energy input and the dissipation, over the joint probability density of the turbulent flow (grey, on a logarithmic scale relative to its peak), with the diagonal (dotted). (a) The orbits found only by our searches (blue), only in the libraries of Page et al. (orange) and by both (black). (b) RPO178, the longest orbit (red), and RPO150, our long orbit of highest mean dissipation (purple).
Some long orbits pass through both the quiet states of the flow and the region of its bursts. RPO178 covers the bulk of the turbulent density in the plane of the energy input and the dissipation and rises once towards the bursts (Figure 10b). Its flow is a pair of vortices of opposite sign, which this rise distorts but does not destroy (Figure 11a). In RPO150, our long orbit of highest mean dissipation, the vortices instead dissolve into jets along the force and form again (not shown), as it climbs far into the region of the bursts (Figure 10b). Long orbits thus need not stay quiet throughout their period, even when their mean dissipation is low. In space and time, the form in which the method represents and corrects an orbit,
Figure 11. RPO178, the longest orbit of our searches. (a) Its dissipation (solid) and energy input (dashed) along the period, and its vorticity at four instants A–D a quarter of the period apart (dots), the last at the largest dissipation, on a scale from −10 (blue) through 0 (black) to 10 (red), with the contours (white). (b) The isosurfaces (red) and (blue) over one period, in the frame that drifts with the orbit, lighter where the faces of the box cut a vortex, with the vorticity on the scale of (a) at the instants A–D on semi-transparent slices (dashed) and at on the end face. The orbit is translated by along .
RPO178 appears as two vortex tubes that persist over the whole period and wander slowly across the domain, with small scales only near the peak of the dissipation (Figure 11b).
Verification by multiple shooting and a first comparison
We used multiple shooting to verify that the space-time solutions, most of which converged in single precision on a few instants per unit time, are orbits of the time stepper of the simulations. We also used this verification to take a first step towards a question often discussed in the literature, whether variational methods are more robust than shooting [38, 21, 54]. On this question, which is not the main aim of this paper, we only sought a first indication.
Multiple shooting splits the period into segments of about ten time units. Its unknowns are the states at the start of the segments, the period and the shift, and its equations require each segment, advanced by the time stepper, to end at the start of the next, and the last, translated by the shift, at the start of the first, with phase conditions in time and along [4]. The time stepper is that of the simulations, the third-order implicit-explicit scheme CB3R2R3e [10] with steps of 0.01, which also advances the linearised equations, so that the Jacobian is that of the discrete flow map. We solve these equations in double precision by the Newton–Krylov hookstep method of Viswanath [72], with GMRES unpreconditioned, to a relative residual of in at most 600 steps, and the trust region of §3.4. A search stops when the residual of the shooting equations falls below , after 50 accepted Newton steps, or when this residual has not halved over the last 15 iterations.
Started from each space-time solution, with one of its instants at the start of each segment, multiple shooting converged all but one, most of them to within in two Newton iterations. The space-time solutions thus lie well inside the basin of quadratic convergence of an orbit of the time stepper, at a distance consistent with their residual.
A first step towards a comparison of the two methods
Robustness can be measured in several ways, all somewhat arbitrary. We take the probability that a search from a seed converges within the budget of its method, without matching the budgets, since the space-time searches have larger, preconditioned Krylov spaces. Multiple shooting searched a sample of seeds of the second and third campaigns, stratified in period and chosen by the outcome of their space-time searches, with about as many converged as failed below a period of 50 and mostly failed ones above.
The two methods fail on largely the same seeds (Table 4), which points to the seeds rather than to the method as the main cause of failure. Multiple shooting converges a few more seeds below a period of 20, where it is single shooting or nearly so, the space-time method a few more above, and above a period of 100 both converge almost none, but these differences rest on a few tens of seeds among two to three hundred in each band. With segments over which perturbations grow less than tenfold, our multiple shooting shares much of the robustness that variational methods owe to treating the whole orbit at once (Ashtari {italic}et al.{/italic} 2027), as expected from the close relation between the two (§1). The seeds, finally, are near recurrences, and the two methods may differ more from rougher guesses (Azimi {italic}et al.{/italic} 2022; Parker & Schneider 2022), while whether one has a larger basin for long and strongly unstable orbits needs a larger study, on a sample chosen independently of the outcomes of the searches.
| both | space-time only | shooting only | neither | |
| 133 | 7 | 17 | 112 | |
| 122 | 23 | 9 | 136 | |
| 49 | 17 | 7 | 152 | |
| 0 | 2 | 1 | 197 |
Table 4. The seeds of a sample of the second and third campaigns, by band of period, that converge with both the space-time method and multiple shooting, with one only and with neither.
Figure 12 compares the work of the two methods on the seeds of the third campaign that both converge. They take similar numbers of Newton iterations and, interestingly, of evaluations of the vector field or of its linearisation at one instant (counted as in the caption). The space-time method takes about eight times as many Arnoldi steps (not shown), but each on far fewer instants, about eight per unit time against the 100 steps per unit time of the time stepper, since its grid need only resolve the orbit (§3.5), while the time steps are limited by the Courant–Friedrichs–Lewy condition of the explicit terms. Per unit time, an Arnoldi step thus evaluates the linearised operator of order a hundred times, once for the product with and once per inner step at each instant, against about six hundred evaluations for a Krylov step of multiple shooting, two at each of the three stages of every time step, so that the larger number of Arnoldi steps is roughly balanced. In our view the two methods are about equally robust from these seeds, within the uncertainty of the sample, and we discuss what may set them apart in §9.
Figure 12. The work of the two methods on the 263 seeds of the third campaign that both converge, for the space-time method (discs) and multiple shooting (triangles), against the period of the seed. (a) The Newton iterations. (b) The evaluations of the vector field or of its linearisation at one instant, over the instants of the products with and of the inner steps of the preconditioner for the space-time method, and over the stages of the time steps for multiple shooting.
Long orbits and the statistics of the turbulent flow
We now turn to the main point of the paper, the properties of long orbits. One remark is in order. We examine individual orbits, which are not a sample of all the orbits of a given period but, as in our earlier study [40], those that the searches could converge from near recurrences, a selection strongly biased here towards quiet orbits (Figure 9 and Figure 10). These are, however, the orbits that searches find in practice, and the results below describe them, not all the admissible ones.
The instability of the orbits
A long orbit that stands for the flow should be about as unstable as the flow, and we compare the Floquet exponents of the long orbits with the Lyapunov exponents of the turbulent flow.
We computed the leading Floquet exponents by the matrix-free subspace iteration of Lust [45], also used in appendix A of Lasagna [40], on the flow map of the time stepper on which the orbits were converged by multiple shooting. It iterates a subspace of chosen dimension towards that of the leading multipliers, and keeps the projection of the monodromy matrix onto it as a product of small factors, corresponding to the action of the Jacobian of the flow map over segments of chosen length. The multipliers follow from the periodic Schur decomposition of these factors, without forming the product, which resolves them over many decades for long orbits, whereas an Arnoldi iteration on the monodromy matrix loses those whose ratio to the leading one falls below the machine precision. The Lyapunov exponents of table 5, denoted by , were computed by the method of Benettin et al. [7]. For P2, P1 and R19 of Chandler & Kerswell [11], and for their P3, which our searches did not find but the libraries of Page et al. hold, the number of unstable exponents and the leading exponent agree with those of Page et al. [52] to the digits given there.
Figure 13 shows that the long orbits are much less unstable than the short orbits of the libraries of Page et al., which have large leading exponents and often more than ten unstable ones, while the orbits of Redfern et al. [58], of periods up to about 50, link the two sets. Between periods of 50 and 100 the sum of the unstable exponents of the long orbits is about a third below that of the positive Lyapunov exponents, and they mostly have three unstable exponents against four positive Lyapunov exponents (Figure 13c and table 5), a number that varies from orbit to orbit, a sign that the flow is not uniformly hyperbolic, since on a uniformly hyperbolic attractor all orbits have the same number of unstable directions. Their leading Floquet exponents capture the Lyapunov spectrum quite well, perhaps slightly below it (Figure 13d). The long orbits thus appear a little less unstable than the flow, which may reflect the under-representation of bursting seeds, since the flow is most unstable as it enters a burst [22]. Indeed, the long orbits that burst, whose largest dissipation exceeds 0.14 of the laminar value, are typically about as unstable as the flow in , and , while the quiet ones lie below it (Figure 13e–Figure 13g).
Figure 13. The instability of the orbits against their period. (a) The leading Floquet exponent , (b) the sum of the unstable exponents and (c) their number , without the two zero exponents, also, where available, for the orbits of the libraries of Page et al., of table S1 in the supplementary material of Page et al. [52] and of Redfern et al. [58], with the same quantities for the turbulent flow (dashed), averaged over the trajectories of Table 5. (d) The leading Floquet exponents of our orbits against their index , coloured by period, with the mean Lyapunov exponents, both without the two zero exponents. (e–g) The quantities of (a–c) for our orbits longer than 50 against the largest dissipation along them, relative to the laminar dissipation, coloured and sized by period, with the value 0.14 above which an orbit bursts (dotted).
These results can be explained by noting that both sets of exponents are averages of a local rate of growth. Along a trajectory , an orbit or the turbulent flow, a tangent perturbation obeys , with the linearisation ((11)) of about .
Along a Floquet vector of an orbit, or a covariant Lyapunov vector of the flow, denoted in the same way, a tangent vector grows or decays at the instantaneous rate
the local exponent [18], with the adjoint of , and
the first average over the period of the orbit and the second along a turbulent trajectory. A short orbit samples these rates only in the small region of the attractor that it visits, which would explain why the short orbits of the libraries, most of which spend their whole period in the tail of the dissipation, are so unstable. A long orbit that stays close to a turbulent segment (§8.2) visits the quiet phases and the bursts in proportions closer to those of the flow, so that its exponents, like its averages, may approach those of the flow as its period grows, as Figure 13 suggests.
How far Newton's method moves the loop
We now ask how close the orbits stay to the turbulent segments from which they were found, that is, how far the Newton iterations move the initial loops. Let , , be the initial loop of the search that found the orbit , , both in the frames that drift with them. The distance
is normalised as the recurrence distance (4.1). If the loop follows the orbit, stays small, the minimiser advances at the rate , close to one, and the minimiser stays nearly constant, conditions used in studies of shadowing [8, 66, 37]. We measure time in escape times , the mean time over which trajectories leave a small neighbourhood of the orbit.
Figure 14 shows four long orbits against the initial loops of their searches. In the first three (Figure 14a–Figure 14c) the distance stays small over most of the period, for tens of escape times, with near and a nearly constant (not shown). It falls by more than a decade away from the junction, so that Newton’s method does very little to these guesses. This closeness is largely built in, since each orbit is computed from the segment with which we compare it. That such segments exist may nevertheless seem surprising, since the probability of following a given orbit over its period falls as , and Page & Kerswell [51] noted that recurrent flow analysis needs the flow to shadow an orbit over at least a whole period, which may become ever rarer as the Reynolds number grows. As discussed in §4.1, however, any of the exponentially many orbits of similar period would do, and what matters is the probability of a return (Figure 3d).
Figure 14. Four long orbits against the initial loops of their searches, (a) RPO178, (b) RPO175, (c) RPO174 and (d) RPO169. In each block, from the top, the dissipation of the initial loop at and of the orbit, their difference, the distance (35) with the recurrence distance of the seed, the threshold 0.075 and the fit (36), and . On the right, the leading Floquet exponents of the orbit, up to twenty, from the subspace iteration of Lust [45] (dots, red if unstable, grey if neutral, blue if stable) and from the Arnoldi iteration on the monodromy matrix (circles), with the rates and of the decaying and the growing part of the fit.
In these three orbits the distance falls from the start of the period and rises again towards its end, which we describe by the least-squares fit
with rates , here commensurate with the leading Floquet exponents (Figure 14, right), although not in general, as we discuss below. The fit works only where the orbit stays close to its loop. In the fourth orbit, whose distance falls below 0.075 for only a fifth of the period, there is no clear approach, and the fitted rates are close to zero (Figure 14d).
The other orbits show similar trends (Figure 15). The time average of falls about as beyond the shortest orbits (Figure 15a), as expected from the boundary layers of approach and departure at the two ends of the period, which weigh less on the average as the period grows. Likewise, the fraction of the period spent within 0.075 of the loop (Figure 15b) grows as , the boundary layers taking about time units whatever the period, and covers nearly the whole period in a few orbits, tens of escape times for the longest, although it depends on this threshold. The recurrence distance of the seeds, by contrast, does not fall with the period (crosses in Figure 15a), so that the Newton iterations move the loops of longer orbits less. This is almost paradoxical, since the searches of longer orbits start from loops closer to the orbit but need no fewer Newton iterations than those of shorter ones (Figure 8a), as in our earlier searches in the Lorenz and Kuramoto–Sivashinsky systems [40]. We speculate that the slow scales of the loop, whose number grows with the period (§ 5.3), must still be corrected, and that their nonlinearity limits the progress of each iteration.
Figure 15. How close the orbits of seed period stay to their initial loops, against their period. (a) The time average of the distance (35), with the recurrence distance of the seed (crosses) and a line proportional to . (b) The fraction of the period with below the threshold 0.075, with the whole period, 80% and half of it (grey) and the line fitted by least squares. Quiet orbits are open circles and those that approach the region of the bursts filled triangles, and RPO178, RPO175, RPO174 and RPO169 of Figure 14 are ringed by a circle, a square, a diamond and a hexagon, in (a) also at the recurrence distance of their seeds.
Most distances fall and rise as (36) describes, but across the orbits found the fitted rates do not always lie close to a specific Floquet exponent (not shown), and in many orbits the distance decays more slowly than any stable exponent, or barely at all, so that the mechanism is probably more complex. The distance nevertheless falls and rises as if the segment approached the orbit along its stable manifold and left it along its unstable one, as in the picture of Ding et al. [18] for the Kuramoto–Sivashinsky equation, where this happens within the span of the leading stable and unstable Floquet vectors of the orbit, its entangled physical modes Yang et al. [75], which span the tangent space of the inertial manifold, if one exists. Among the stable directions the least stable one dominates only after a time set by the gaps between the stable exponents and by how the initial difference projects onto their directions. Estimated from the Lyapunov exponents of the flow, close to those of the long orbits (§ 8.1), the least stable direction gains a factor of ten over the next, from equal amplitudes, in a time of about 110 time units (Table 5), while in most of our orbits the approach is too brief for a single direction to dominate. A projection of the difference between the turbulent segment and the orbit onto the Floquet vectors, as Ding et al. [18] did, together with the angles between these vectors, through which the non-normality of the linearised dynamics can make the distance grow or decay transiently at other rates, would test this picture directly, and we leave it for future work.
| 0.126 | 0.081 | 0.045 | 0.017 |
Table 5. The leading Lyapunov exponents of the turbulent flow, averaged over four trajectories of 20 000 time units, without the neutral exponents and , below in magnitude.
Shadowing of long orbits by the turbulent flow
Previous studies of shadowing have considered short orbits, asking whether turbulent trajectories come close to them and follow them [37, 15], and how much of the time they spend near them [66]. If a long orbit is to stand for the turbulent flow, and capture all its dynamically relevant states and events, the question is rather how much of a turbulent trajectory lies near the orbit, and how this grows with the period, since a longer orbit holds more states, spread over more of the attractor. We measure the distance of a turbulent trajectory from an orbit by
where is now a state of the flow, , , a state of the orbit, and runs over the 16 symmetric copies given by the shift-reflect and the rotation (§ 2.1). We followed 384 trajectories of 2 500 time units each (1 152 for the distribution of RPO178 and 768 for that of RPO86 in figure 16c), recorded after a transient of 500 time units from initial conditions other than those of our campaigns, and took as a yardstick the same distance from 32 turbulent segments of the period of each orbit, cut from other trajectories.
Figure 16(a) shows the distance of the trajectory that comes closest to RPO178, in escape times of the orbit, together with the range of its distance from the 32 turbulent segments of the same duration, and the two behave alike. Most of the time the distance stays near 0.25, the threshold of the near recurrences of §4.1, and it rises close to one during the bursts. Just before one of these bursts the flow comes within a few per cent of the orbit and follows it for about seven escape times (figure 16b), while the instant of the orbit advances with (not shown), a shadowing event in the sense of Krygier et al. [37]. Over all the trajectories, the cumulative distribution of the distance, the fraction of the time that the flow lies within of the orbit, is about the same as that from turbulent segments of the same duration (figure 16c, red and grey), so that, seen from the turbulent flow, RPO178 behaves as a turbulent segment of its length. Its median, about 0.29 (red star), is low compared with the median distance between two independent states of the flow, about 0.66 (black star), and lower than that of the shorter RPO86, of period 48 (green star).
A simple argument explains this behaviour, and predicts what to expect from orbits of longer period. Let be the correlation integral of §4.1, the probability that two independent states of the flow lie within of each other, here over the symmetric copies. When a periodic orbit visits the attractor like the flow, a state sampled on it lies within of the turbulent state with probability . If the orbit held independent states, all of them would lie farther than with probability . But the states of such a trajectory are not independent, since the flow needs some time to move to a new region, and we count an orbit of period as independent states, with a decorrelation time, so that the flow lies farther than from it at a given instant with probability
where the last form holds when . The probability that the turbulent flow does not come within of the orbit thus decays exponentially with the period, as the probability that a trajectory has not yet entered a small ball decays with time in many chaotic systems [26]. Two predictions follow. The typical distance , the median of , at which this probability is one half, satisfies
and since increases with , the median distance falls as the period grows. At small distances, where , (38) gives , so that the distribution of the distance has there the shape of the correlation integral, and rises in proportion to the period. Both predictions rest on counting the states of the orbit as independent, which holds only if the orbit keeps visiting new regions of the attractor, as a mixing trajectory does. An orbit that returns near states it has already visited, for instance one that follows a shorter orbit several times, holds fewer independent states than , and the flow then comes close to it less often than (38) predicts.
The decorrelation time depends on how it is measured. The autocorrelations of the dissipation, of the energy and of the coefficient of the forcing mode fall to 1/e in about 11 time units, that of the vorticity field in about 30, and that of the velocity field more slowly still (not shown). These follow the slow large scales and the drift along , and say little about where on the attractor the flow lies. We therefore fit to the median distance of turbulent segments of durations above 10, computing (39) by inverting the measured correlation integral (Figure 16d). The fit gives time units, of the same order as the 1.7 over which the distance (37) between a state of the flow and its own future reaches a fraction of that between two independent states, and far from the decorrelation times of the global quantities.
Figure 16. The distance (37) of turbulent trajectories from long orbits. (a) To RPO178, along the trajectory that comes closest to it, over 1 000 time units about its closest approach at , in escape times of the orbit, with the range of the distance to 32 turbulent segments of the same duration (grey). (b) The closest approach. (c) The cumulative distribution of the distance, the fraction of samples below , to RPO178, over 1 152 trajectories of 2 500 time units (red, with a star at the median ), to RPO86 over 768 trajectories (green, with a star at its median), to the 32 turbulent segments of the same duration, over the 384 trajectories (grey, median and range), and between two independent states of the flow, the correlation integral (black, with a star at its median), with a line of slope . (d) The typical distance against the duration , for turbulent segments (grey), from (8.7) with the measured and fitted for (dashed), and for 25 orbits drawn across the periods, ours (blue) and from the libraries of Page et al. (orange), with RPO178 and RPO86 ringed. Here the states of the flow are those of the segments, one every time unit.
Figure 16(c, Figure 16d) bears out both predictions. With this single fitted constant, (39) follows the median distance of the turbulent segments to within a few per cent over durations from 10 to 190 (Figure 16d), and the 25 orbits drawn across the periods, RPO178 among them, follow the median of the segments of the same duration, except the short orbits of the libraries that burst, which sit far above it, in the region of the bursts, where the flow seldom goes. That these medians keep falling up to the longest periods suggests that the orbits, like the segments, keep visiting new regions of the attractor, as the count of independent states assumes. With the copies, the correlation integral has a slope between and , an estimate of the correlation dimension close to the seven of §4.1, and the distributions of rise with about the same slope at small , their tails roughly in proportion to the period, within a factor of about two, from RPO86 to RPO178 (Figure 16c).
At a given instant, the turbulent flow thus comes within a given distance of a long orbit with a probability that grows with the period, since a longer orbit spans more of the attractor, as a turbulent segment does with its duration. This is favourable to the use of long orbits as proxies for the flow, but the growth is very slow, since its exponent is set by the dimension of the attractor. Where , (39) gives for long orbits, so that having the typical distance require a period about times longer, a factor that depends somewhat on the norm in which is estimated and that grows exponentially with the dimension of the attractor, which itself grows with the Reynolds number [13]. Even so, at this Reynolds number the flow lies half of the time within about of RPO178, whereas two independent states of the flow lie this close in less than one case in a hundred.
Closeness at one instant is not shadowing, in which the flow also follows the orbit for a while, and whether long orbits are shadowed more often than short ones, at some point of their period, is a separate question, which would need a longer analysis. Yet, since the tails of Figure 16(c) rise in proportion to the period, as (38) predicts, it is as if close approaches, and with them the chances of shadowing, became more frequent with the period.
The statistics of the orbits
The longest orbits found from near recurrences are close copies of turbulent segments over many escape times, and the longest is about as close to the turbulent flow as such a segment (§ 8.2, § 8.3). Their statistics should then follow those of turbulent segments of the same duration, unless the closure of a segment selects orbits whose statistics differ systematically, as [12] showed can happen for the trajectories of one-dimensional maps that shadow those of slightly perturbed maps.
We compare the orbits with the turbulent flow through the distributions of the dissipation and of the kinetic energy (Figure 17), each relative to its laminar value. Along an orbit of period either quantity is a smooth periodic function , and its probability density is
Figure 17. The statistics of single orbits against those of the turbulent flow. (a, b) The Kullback–Leibler divergence (42) of each orbit from the flow against its period, for the dissipation (a) and for the kinetic energy (b), coloured as in (c), with the median (grey line) and the quartiles (grey band) of turbulent segments of duration , and a line through the median at (dashed). Orbits beyond the largest value of the flow have an infinite divergence (triangles at the top). (c) The mean dissipation of each orbit against its period, with vertical lines spanning the dissipation along it, and the mean (dashed) and the median (dotted) of the flow. On the right, the probability density of the dissipation of the flow (black) and along RPO150 and RPO177. (d) The same for the kinetic energy, without the median, which nearly equals the mean, and without four short orbits of larger energy. RPO178, RPO150 and RPO177 are ringed by a circle, a triangle and a pentagon.
the fraction of the period that it spends near each value ,
where the sum runs over the instants of the period at which [16 eqs. 19.7 and 19.16]. It is singular as at every extremum of , where vanishes. We evaluate its average over bins of width ,
the total time that spends in the bin divided by , from the instants at which , interpolated linearly between the steps of the time stepper on the orbit converged by multiple shooting (§ 7), crosses the edges of the bin, so that the peaks at the extrema need no special treatment. The densities of the long orbits RPO150 and RPO177 spread over the bulk of the turbulent densities, with peaks at these extrema, and reach into the tail of the dissipation during their bursts (right of Figure 17c, Figure 17d).
We measure how far an orbit is from the turbulent flow by the Kullback–Leibler divergence of its density from the density of the turbulent flow,
the measure that Cleary & Page [14] minimise to fit the weights of their library, and Redfern et al. [58] a loss of the same kind with the two densities exchanged. It vanishes when the two densities coincide and grows as the orbit spends its time where the turbulent flow seldom goes. Only the values of that the orbit visits contribute, so that it is finite whenever the turbulent flow also visits them, whereas the reverse divergence is infinite unless the orbit reaches the whole range of the turbulent flow. We evaluate (42) on the same bins, with from the samples of the turbulent flow, and compare each orbit with turbulent segments of the same duration, cut from about time units of turbulent trajectories, whose divergence should decay as for long durations, as for the empirical distribution of independent samples [53].
The long orbits of our searches follow the decrease of the divergence of these segments with the duration (Figure 17a), which reaches the law only beyond about a thousand time units, once the segments visit the rare bins of the bursts many times, and those of period between 50 and 200 lie close to the median of the segments of the same duration. Of the two bursting orbits whose densities are drawn, RPO150 lies at the lower quartile and RPO177 below it. For the energy the long orbits scatter more widely (Figure 17b), and RPO177 lies above the band, so that the energy is the more demanding test, as in the reconstructions of Cleary & Page [14] with weights fitted on the dissipation. The densities of the long orbits thus differ from that of the flow about as much as those of turbulent segments of the same duration. This must be qualified, since the divergence does not penalise an orbit for missing the tail of the turbulent density. A quiet orbit that stays near the peak of the density, as most of ours do (Figure 17c), can therefore score as well as a turbulent segment that visits a burst, although it never reproduces the tail (right of Figure 17c).
The mean dissipation and the mean energy of the long orbits lie near the median of the flow, while those of the short orbits scatter far more widely (Figure 17c, Figure 17d), and only some bursting orbits reach the mean dissipation, which the rare bursts raise above the median. If the long orbits keep representing turbulent segments of their duration as the period grows, their divergence should keep falling, eventually as , and their means should approach those of the flow as , as those of turbulent segments do, two trends that remain to be confirmed. Testing this requires orbits about ten times longer, which would need an advance of the method or larger computational resources.
Discussion and conclusions
This work is part of a programme to compute the statistics of a turbulent flow, and their sensitivity to its parameters, on one or a few long unstable periodic orbits embedded in the attractor [39, 40]. On a periodic orbit the periodicity condition removes the exponential growth that the adjoint solution suffers on an open chaotic trajectory, and long orbits that capture the relevant dynamical events of the flow may stand as a proxy for how its attractor changes with the parameters, a step beyond the computation of averages from periodic orbits. Testing this hypothesis requires long orbits of a turbulent flow, which are hard to compute with the standard methods. We have therefore taken a step towards methods that can compute them, and examined, in two-dimensional Kolmogorov flow at , whether the long orbits that can be found model the statistics of the flow.
The method is a preconditioned, matrix-free Newton–Krylov iteration on the whole loop, as advocated by Lan & Cvitanović [38], implemented on GPUs. With the approximate factorisation of time-spectral methods as preconditioner, which retains advection, Newton’s method on the loop appears to become practical for flows of this size, with a number of Arnoldi steps per Newton iteration that appears not to depend on the spatial resolution, at least at this Reynolds number. The classical trust-region approach [17], known as the hookstep in the literature on shooting [72], links the method to recent adjoint descent approaches [5], since far from an orbit the hookstep descends the residual as these methods do, while near the orbit the iteration becomes an inexact Newton method that converges fast. A hookstep costs far more than a step of descent, but it also does more, since from the very first iterations it corrects the whole loop, while a gradient is local in time and may need many steps to propagate the correction through the period. Our searches did not need a separate stage of descent, but whether a hybrid method that first descends at a lower cost and then switches to hooksteps, as is done with shooting [52, 49], would be faster remains to be verified.
A secondary question of this work was whether a method on the whole loop is more robust than multiple shooting. From near recurrences the two methods appear similar, since they fail on largely the same seeds and take similar numbers of Newton iterations and of evaluations of the vector field, although they may differ from rougher guesses [5, 54]. Comparing their computational costs would need a more careful analysis, on the same hardware and at the same precision, and with adjoint-based methods as well, which is beyond the scope of this work. The two methods also differ in what they require. A space-time method need only resolve the orbit in time, free of the stability limit of time stepping, and needs no time stepper, only the right-hand side, the linearised operator and a preconditioner, so that it may be simpler to implement once a preconditioner is available, although shooting can leverage existing time-stepping codes. Expanded in Fourier modes in time, the loop could also be projected onto resolvent or spectral proper orthogonal decomposition modes, as in searches for invariant solutions on a resolvent basis [9] and in the space-time reduced-order models of Frame & Towne [25] and Li & Lasagna (2026), which could link the study of invariant solutions with model reduction. Both methods were started from near recurrences of the turbulent flow, whose rate of occurrence we examined against the period. Recurrent flow analysis requires the flow to follow an orbit over a whole period (Page & Kerswell [51]), and the instability of long orbits might be expected to make this ever less likely as their period grows. The proliferation of long orbits, however, acts the other way, and the two effects may balance, as near recurrences of long period are about as frequent as those of moderate period (Figure 3e). From them the searches, mostly in single precision, converged 183 distinct solutions with periods up to 191, all but one verified in double precision by multiple shooting. The success of a search falls steeply with the largest dissipation along the seed and with its period, possibly because a longer recurrence is more likely to approach a burst and because the Krylov spaces of long loops grow large, while it depends little on how close the recurrence is, in the norm of the vorticity. Since multiple shooting fails on largely the same seeds, we speculate that, once the instability is tamed, the success of a search may depend on whether the two ends of its seed are dynamically connected, for instance on the same sheet of the folded attractor, which a small distance does not guarantee ([8, 50]). Building initial guesses whose ends are dynamically connected, for instance by gluing shorter orbits ([8]), remains an open problem, and dynamic mode decomposition, which needs no near recurrence (Page & Kerswell [51]), would be worth testing on long orbits. The orbits that converge are mostly quiet, with a few bursting ones that capture part of the tail of the dissipation, and since they lie where the turbulent flow spends much of its time they may complement the short bursting orbits of existing libraries.
The orbits that we found are by necessity a selected sample of the unstable periodic orbits that exist, and they give partial answers to the questions raised in the introduction. A single long orbit appears to be a better proxy for the statistical properties of the turbulent flow than a single short one. Its mean dissipation and mean energy lie close to the medians of the flow, a necessary condition for its use, and the distribution of the dissipation along it is comparable with that of a turbulent segment of the same duration, that of the energy less so. This may be expected, since Newton’s method moves these loops little, so that the longest orbits stay close to the segments from which they were found over most of their period, and their statistics appear to be those of a turbulent segment of their own length. The agreement does not reach the tail, since most orbits miss the rare bursts that raise the mean dissipation of the flow above its median. The missing bursts may also show in the instability of the orbits. The leading Floquet exponents of the long orbits follow the Lyapunov spectrum of the flow, perhaps slightly below it, as may be expected of orbits quieter than the flow, which is most unstable as it enters a burst. Seen from the turbulent flow, a long orbit also behaves as a turbulent segment of its length. A simple argument, with one fitted correlation time, predicts how its typical distance from the flow falls with the period, about as where the correlation integral follows a power law, with the correlation dimension of the attractor. In this scaling, akin to the curse of dimensionality, the period needed to bring the flow within a given distance of the orbit grows as that distance to the power , that is, exponentially with the dimension of the attractor, which grows with the Reynolds number (Cleary & Page [13]), so that a single orbit of given period may represent more strongly turbulent flows less closely. At this Reynolds number, however, the flow already lies half of the time within a small distance of our longest orbit, one within which two independent states of the flow lie in less than one case in a hundred. The tails of the distribution of this distance also rise in proportion to the period, as if close approaches, and with them the chances of shadowing, became more frequent along longer orbits, although shadowing itself remains to be measured.
The work has limitations, and several steps should follow. Above all, our orbits, of periods up to about forty escape times, are not yet long enough to show how their statistics evolve as the period grows, but they are a step towards such orbits. The work also considers a single weakly chaotic flow, at in a small doubly periodic domain. The method must still be extended to flows bounded by walls, where treating the pressure is an issue [3], either as a variable of its own or implicitly through a Poisson equation, in the residual as in the preconditioner. Higher Reynolds numbers, larger domains and three-dimensional flows may also need the loop and its Krylov space to be distributed over several GPUs. The inner steps of the preconditioner take most of the time spent on the operators. The preconditioner also leaves the eigenvalues of the slow scales of the loop near the origin, so that the Arnoldi steps grow linearly with the period, and the wall time and the memory grow about as its square. Preconditioners that also capture these slow scales, at a cost that grows only linearly with the period, could make longer orbits accessible, for instance by projecting the variational equation onto the unstable and nearly neutral directions, which keeps the Newton systems small [74]. Besides this practical interest, it may be important to understand what these slow scales are and what role these nearly neutral modes play in the dynamics. Long orbits also give access to the structure of the tangent space of the flow, and the Floquet vectors of many orbits could estimate the dimension of the inertial manifold, if one exists, as for the Kuramoto–Sivashinsky equation [18].
Finally, the sensitivity of turbulent statistics to parameters remains the goal of this work. With long orbits at hand, the adjoint method for periodic orbits [39] could now be tried on a turbulent flow, and compared with the change of the long-time averages between simulations at nearby parameter values. A caveat is that the sensitivity of an orbit diverges at a bifurcation point. In low-dimensional systems this produces power-law tails in the density of the sensitivities, which decays as their inverse cube, set by the normal form of the bifurcation, so that the sensitivity of a single orbit may occasionally be far from the typical value [40]. This may also matter in Kolmogorov flow, whose branches of orbits fold repeatedly as the Reynolds number grows [14]. Most of our long orbits have no Floquet multiplier close to one, apart from the two of the symmetries, but a few have one within about 10% of it (not shown) and may lie near such a bifurcation, so that how long orbits depend on the parameters, and whether they survive bifurcations, matters for this goal. At higher Reynolds numbers, where by the argument of § 8.3 an orbit of given period may represent the flow less closely, one would also need to understand how the sensitivity of such an orbit, the proxy, compares with that of the attractor it represents, an object of ever higher dimension. Whether a single long orbit can capture how the statistics of a turbulent flow change with the parameters is the question we would most like to answer.
Acknowledgements. This work was developed during a sabbatical in the autumn of 2026 at the Emergent Complexity in Physical Systems Laboratory of the École Polytechnique Fédérale de Lausanne (EPFL). D.L. is grateful to Tobias Schneider for the financial support of the visit and for a calm and productive environment, without which this work would not have been possible, and to the members of the laboratory for their welcome and for many useful conversations. The authors acknowledge the use of the IRIDIS High Performance Computing Facility, and associated support services at the University of Southampton, in the completion of this work.
Funding. This work was funded by a Leverhulme Trust Research Project Grant (RPG-2024-368).
Declaration of interests. The authors report no conflict of interest.
Declaration of generative AI use. Generative AI was used throughout this work, through Claude (Anthropic, Claude Opus 5.5, through Claude Code). The interaction was a productive collaboration, in which the tool was essential to implement and test numerical ideas rapidly, many of which were then abandoned. The codes on which this paper rests were developed in this way, through a careful analysis of the generated code and the crafting of physically relevant unit and integration tests, as is standard practice for scientific software. Our role was to set the direction of the work, and to guide, filter and select what may be useful to the community, an experience that echoes the view of computers as ‘Monte Carlo’ theory machines of Jiménez [32], which pose many questions of which only some are later found useful. In our experience the creative part of research lies increasingly in providing this direction. An initial draft of the text was also produced with the same tool, and was then edited into its final form by the authors through a long process of human supervision and revision, and the plotting scripts of all the figures were also written with it. The authors take full responsibility for the content of the paper.
Data availability statement. The codes used in this work are available at https://github.com/Davide-Lasagna-s-Lab. The catalogue of our solutions (§ 6.3) is available at invariant-sets.org.
Author ORCIDs.
D. Lasagna, https://orcid.org/0000-0002-6501-6041;
Thomas Burton https://orcid.org/0000-0001-7998-2278;
Wall time of the operators on CPUs and GPUs
Figure 18 compares the wall time of the operators of Kolmogorov flow at on one CPU core and on GPUs. On the NVIDIA H200 GPU a product with takes from a fraction of a millisecond to a few tens of milliseconds (Figure 18a). The speed-up over the CPU core grows with the number of instants, from a few tens to a few hundred, until the device is fully occupied (Figure 18b). The residual and the product with , which consist of transforms and element-wise operations alone, gain the most. The application of the preconditioner gains somewhat less, because at every inner step it exchanges numbers with the host. With ten inner steps it thus takes as long as 13 to 21 products with on the H200, against about a dozen on the CPU core. Across the four GPUs (Figure 18c), the speed-up of the product with in double precision saturates at a level roughly proportional to their memory bandwidth, which limits such operations, and is an order of magnitude higher on the H200 than on the L4. In single precision the L4 and the L40S reach two to three times their speed-up in double precision, and the H200 only slightly more.
Figure 18. The performance of the operators on GPUs, for Kolmogorov flow at , against the number of instants , for the residual, the product with and an application of with the settings of the searches, in double (solid) and single precision (dashed). (a) The wall time on an NVIDIA H200 GPU. (b) Its speed-up over one core of an AMD EPYC 7502 processor. (c) The speed-up of the product with on NVIDIA L4, L40S, A100 and H200 GPUs. The dotted line marks the largest loop of the searches.
References
- [3]Ashtari, O. & Schneider, T. M. 2023 Identifying invariant solutions of wall-bounded three-dimensional shear flows using robust adjoint-based variational techniques. J. Fluid Mech. 977, A7.arxiv.org/abs/2306.00165
- [4]Ashtari, O., Tuckerman, L. S. & Schneider, T. M. 2027 Computing simple invariant solutions in chaotic fluid flows. Annu. Rev. Fluid Mech. (in press).DOI
- [5]Azimi, S., Ashtari, O. & Schneider, T. M. 2022 Constructing periodic orbits of high-dimensional chaotic systems by an adjoint-based variational method. Phys. Rev. E 105, 014217.DOI
- [6]Beck, P. & Schneider, T. M. 2026 Identifying recurrent flows in high-dimensional dissipative chaos from low-dimensional embeddings. arXiv:2601.01590.arxiv.org/abs/2601.01590
- [7]Benettin, G., Galgani, L., Giorgilli, A. & Strelcyn, J.-M. 1980 Lyapunov characteristic exponents for smooth dynamical systems and for Hamiltonian systems; a method for computing all of them. Part 1: theory. Meccanica 15, 9–20.DOI
- [8]Budanur, N. B., Short, K. Y., Farazmand, M., Willis, A. P. & Cvitanović, P. 2017 Relative periodic orbits form the backbone of turbulent pipe flow. J. Fluid Mech. 833, 274–301.DOI
- [9]Burton, T., Symon, S. & Lasagna, D. 2026 Searching for invariant solutions to wall-bounded flows using resolvent-based optimisation. J. Fluid Mech. 1038, A12.DOI
- [10]Cavaglieri, D. & Bewley, T. 2015 Low-storage implicit/explicit Runge–Kutta schemes for the simulation of stiff high-dimensional ODE systems. J. Comput. Phys. 286, 172–193.DOI
- [11]Chandler, G. J. & Kerswell, R. R. 2013 Invariant recurrent solutions embedded in a turbulent two-dimensional Kolmogorov flow. J. Fluid Mech. 722, 554–595.DOI
- [12]Chandramoorthy, N. & Wang, Q. 2021 On the probability of finding nonphysical solutions through shadowing. J. Comput. Phys. 440, 110389.DOI
- [13]Cleary, A. & Page, J. 2025a Characterizing the Reynolds number dependence of the chaotic attractor in two-dimensional turbulence with dimension-minimizing autoencoders. Phys. Rev. E 112, 055105.arxiv.org/abs/2505.21361
- [14]Cleary, A. & Page, J. 2025b Dynamical relevance of periodic orbits under increasing Reynolds number and connections to inviscid dynamics. J. Fluid Mech. 1020, A52.DOI
- [15]Crowley, C. J., Pughe-Sanford, J. L., Toler, W., Krygier, M. C., Grigoriev, R. O. & Schatz, M. F. 2022 Turbulence tracks recurrent solutions. Proc. Natl. Acad. Sci. USA 119, e2120655119.DOI
- [16]Cvitanović, P., Artuso, R., Mainieri, R., Tanner, G. & Vattay, G. 2025 Chaos: Classical and Quantum. ChaosBook.org, edition 17.6.8.
- [17]Dennis, J. E. & Schnabel, R. B. 1983 Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Englewood Cliffs: Prentice-Hall.
- [18]Ding, X., Chaté, H., Cvitanović, P., Siminos, E. & Takeuchi, K. A. 2016 Estimating the dimension of an inertial manifold from unstable periodic orbits. Phys. Rev. Lett. 117, 024101.DOI
- [19]Douglas, C. M., Emerson, B. L. & Lieuwen, T. C. 2021 Nonlinear dynamics of fully developed swirling jets. J. Fluid Mech. 924, A14.DOI
- [21]Farazmand, M. 2016 An adjoint-based approach for finding invariant solutions of Navier–Stokes equations. J. Fluid Mech. 795, 278–312.DOI
- [22]Farazmand, M. & Sapsis, T. P. 2016 Dynamical indicators for the prediction of bursting phenomena in high-dimensional systems. Phys. Rev. E 94, 032212.DOI
- [23]Farazmand, M. & Sapsis, T. P. 2017 A variational approach to probing extreme events in turbulent dynamical systems. Sci. Adv. 3, e1701533.DOI
- [25]Frame, P. & Towne, A. 2027 Nonlinear space-time model reduction in the frequency domain. J. Comput. Phys. 568, 115403.arxiv.org/abs/2411.13531
- [26]Freitas, A. C. M., Freitas, J. M. & Todd, M. 2010 Hitting time statistics and extreme value theory. Probab. Theory Relat. Fields 147, 675–710.arxiv.org/abs/0804.2887
- [27]Golden, M. 2026 A catalog of exact steady states in Kolmogorov flow. arXiv:2609.27062.arxiv.org/abs/2609.27062
- [28]Govaerts, W. J. F. 2000 Numerical Methods for Bifurcations of Dynamical Equilibria. Philadelphia: SIAM.DOI
- [29]Graham, M. D. & Floryan, D. 2021 Exact coherent states and the nonlinear dynamics of wall-bounded turbulent flows. Annu. Rev. Fluid Mech. 53, 227–253.DOI
- [30]Grassberger, P. & Procaccia, I. 1983 Characterization of strange attractors. Phys. Rev. Lett. 50, 346–349.DOI
- [32]Jiménez, J. 2020 Computers and turbulence. Eur. J. Mech. B/Fluids 79, 1–11.
- [33]Kawahara, G. & Kida, S. 2001 Periodic motion embedded in plane Couette turbulence: regeneration cycle and burst. J. Fluid Mech. 449, 291–300.DOI
- [34]Kawahara, G., Uhlmann, M. & van Veen, L. 2012 The significance of simple invariant solutions in turbulent flows. Annu. Rev. Fluid Mech. 44, 203–225.DOI
- [35]Kazantsev, E. 2001 Sensitivity of the attractor of the barotropic ocean model to external influences: approach by unstable periodic orbits. Nonlinear Process. Geophys. 8, 281–300.
- [36]Knoll, D. A. & Keyes, D. E. 2004 Jacobian-free Newton–Krylov methods: a survey of approaches and applications. J. Comput. Phys. 193, 357–397.DOI
- [37]Krygier, M. C., Pughe-Sanford, J. L. & Grigoriev, R. O. 2021 Exact coherent structures and shadowing in turbulent Taylor–Couette flow. J. Fluid Mech. 923, A7.
- [38]Lan, Y. & Cvitanović, P. 2004 Variational method for finding periodic orbits in a general flow. Phys. Rev. E 69, 016217.
- [39]Lasagna, D. 2018 Sensitivity analysis of chaotic systems using unstable periodic orbits. SIAM J. Appl. Dyn. Syst. 17, 547–580.DOI
- [40]Lasagna, D. 2020 Sensitivity of long periodic orbits of chaotic systems. Phys. Rev. E 102, 052220.DOI
- [41]Lea, D. J., Allen, M. R. & Haine, T. W. N. 2000 Sensitivity analysis of the climate of a chaotic system. Tellus A 52, 523–532.
- [44]Lucas, D. & Kerswell, R. R. 2015 Recurrent flow analysis in spatiotemporally chaotic 2-dimensional Kolmogorov flow. Phys. Fluids 27, 045106.
- [45]Lust, K. 1997 Numerical bifurcation analysis of periodic solutions of partial differential equations. PhD thesis, Katholieke Universiteit Leuven.
- [47]Mundis, N. L. & Mavriplis, D. J. 2017 Toward an optimal solver for time-spectral fluid-dynamic and aeroelastic solutions on unstructured meshes. J. Comput. Phys. 345, 132–161.
- [49]Page, J. 2025 Computation of simple invariant solutions in fluid turbulence with the aid of deep learning. Nonlinear Dyn. 113, 31895–31915.
- [50]Page, J., Holey, J., Brenner, M. P. & Kerswell, R. R. 2024a Exact coherent structures in two-dimensional turbulence identified with convolutional autoencoders. J. Fluid Mech. 991, A10.
- [51]Page, J. & Kerswell, R. R. 2020 Searching turbulence for periodic orbits with dynamic mode decomposition. J. Fluid Mech. 886, A28.
- [52]Page, J., Norgaard, P., Brenner, M. P. & Kerswell, R. R. 2024b Recurrent flow patterns as a basis for two-dimensional turbulence: predicting statistics from structures. Proc. Natl. Acad. Sci. USA 121, e2320007121.
- [53]Paninski, L. 2003 Estimation of entropy and mutual information. Neural Comput. 15, 1191–1253.
- [54]Parker, J. P. & Schneider, T. M. 2022 Variational methods for finding periodic orbits in the incompressible Navier–Stokes equations. J. Fluid Mech. 941, A17.
- [55]Platt, N., Sirovich, L. & Fitzmaurice, N. 1991 An investigation of chaotic Kolmogorov flows. Phys. Fluids A 3, 681–696.
- [57]Ramezanian, D., Mavriplis, D. J. & Ahrabi, B. R. 2020 An order N log N parallel solver for time-spectral problems. J. Comput. Phys. 411, 109319.
- [58]Redfern, E. M., Lazer, A. L. & Lucas, D. 2024 Dynamically relevant recurrent flows obtained via a nonlinear recurrence function from two-dimensional turbulence. Phys. Rev. Fluids 9, 124401.
- [59]Rigas, G., Sipp, D. & Colonius, T. 2021 Nonlinear input/output analysis: application to boundary layer transition. J. Fluid Mech. 911, A15.DOI
- [60]Ruelle, D. 1999 Smooth dynamics and new theoretical ideas in nonequilibrium statistical mechanics. J. Stat. Phys. 95, 393–468.
- [61]Saad, Y. 1993 A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci. Comput. 14, 461–469.DOI
- [62]Saad, Y. & Schultz, M. H. 1986 GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856–869.DOI
- [63]Sánchez, J. & Net, M. 2010 On the multiple shooting continuation of periodic orbits by Newton–Krylov methods. Int. J. Bifurcation Chaos 20, 43–61.DOI
- [66]Suri, B., Kageorge, L., Grigoriev, R. O. & Schatz, M. F. 2020 Capturing turbulent dynamics and statistics in experiments with unstable periodic orbits. Phys. Rev. Lett. 125, 064501.DOI
- [67]Thomas, J. P., Custer, C. H., Dowell, E. H., Hall, K. C. & Corre, C. 2013 Compact implementation strategy for a harmonic balance method within implicit flow solvers. AIAA J. 51, 1374–1381.DOI
- [68]Trefethen, L. N. & Bau, D. 1997 Numerical Linear Algebra. Philadelphia: SIAM.DOI
- [69]Tuckerman, L. S. 1989 Steady-state solving via Stokes preconditioning; recursion relations for elliptic operators. In 11th International Conference on Numerical Methods in Fluid Dynamics (ed. D. L. Dwyer, M. Y. Hussaini & R. G. Voigt), Lecture Notes in Physics, vol. 323, pp. 573–577. Berlin: Springer.
- [70]Tuckerman, L. S., Langham, J. & Willis, A. P. 2019 Order-of-magnitude speedup for steady states and traveling waves via Stokes preconditioning in Channelflow and Openpipeflow. In Computational Modelling of Bifurcations and Instabilities in Fluid Dynamics (ed. A. Gelfgat), Computational Methods in Applied Sciences, vol. 50, pp. 3–31. Cham: Springer.
- [71]van Veen, L., Vela-Martín, A. & Kawahara, G. 2019 Time-periodic inertial range dynamics. Phys. Rev. Lett. 123, 134502.DOI
- [72]Viswanath, D. 2007 Recurrent motions within plane Couette turbulence. J. Fluid Mech. 580, 339–358.DOI
- [73]Wang, B., Ayats, R., Deguchi, K., Meseguer, A. & Mellibovsky, F. 2025 Mathematically established chaos and forecast of statistics with recurrent patterns in Taylor–Couette flow. J. Fluid Mech. 1011, R2.arxiv.org/abs/2409.09234
- [74]Wang, D. & Lan, Y. 2023 A reduced variational approach for searching cycles in high-dimensional systems. Nonlinear Dyn. 111, 5579–5592.
- [75]Yang, H.-L., Takeuchi, K. A., Ginelli, F., Chaté, H. & Radons, G. 2009 Hyperbolicity and the effective dimension of spatially extended dissipative systems. Phys. Rev. Lett. 102, 074102.pubmed.ncbi.nlm.nih.gov/19257674
- [76]Yang, J. 2015 A numerical method for computing time-periodic solutions in dissipative wave systems. Stud. Appl. Math. 134, 420–455.arxiv.org/abs/1408.6490
- [77]Zheng, Z., Beck, P., Yang, T., Ashtari, O., Parker, J. P. & Schneider, T. M. 2025 Ghost states underlying spatial and temporal patterns: how nonexistent invariant solutions control nonlinear dynamics. Phys. Rev. E 112, 024212.DOI














