betaFeedback

WorksPhysics and Astronomy

Model-Free Forecasting of Partially Observable Spatiotemporally Chaotic Systems

Abstract

Reservoir computing is a powerful tool for forecasting turbulence because its simple architecture has the computational efficiency to handle large systems. Its implementation, however, often requires full state-vector measurements and knowledge of the system nonlinearities. We use nonlinear projector functions to expand the system measurements to a high dimensional space and then feed them to a reservoir to obtain forecasts. We demonstrate the application of such reservoir computing networks on spatiotemporally chaotic systems, which model several features of turbulence. We show that using radial basis functions as nonlinear projectors enables complex system nonlinearities to be captured robustly even with only partial observations and without knowing the governing equations. Finally, we show that when measurements are sparse or incomplete and noisy, such that even the governing equations become inaccurate, our networks can still produce reasonably accurate forecasts, thus paving the way towards model-free forecasting of practical turbulent systems.

Introduction

The availability of large datasets and advances in computational hardware have led to revolutionary applications of machine learning (ML), such as in speech recognition [1] and medical diagnosis [2]. This has led to renewed interest in the application of ML to predict the behaviour of turbulent flow systems, such as for weather and climate forecasting [3, 4, 5, 6, 7, 8, 9]. Turbulent systems are defined by processes spanning a wide range of spatial and temporal scales. These processes are coupled nonlinearly such that even small changes at the microscopic scale can lead to large changes at the macroscopic scale [10], a phenomenon poetically termed the “butterfly effect” by Lorenz [11]. Modelling such nonlinearly coupled dynamical systems with recurrent neural networks (RNNs) would require the network dimension to be exceedingly large, limiting their applicability to either coarse predictions [3, 4] or necessitating dimensionality reduction of the input states [12, 13, 14].

Reservoir-computing-based RNNs (RC-RNNs) have been shown to be more computationally efficient at learning large chaotic systems as compared to backpropagation RNNs [15, 16]. Their implementation, however, often requires knowledge of the type of nonlinearities in the governing equations [17, 15]. This hinders the application of RC-RNNs to natural and engineering systems where full state-vector measurements are often not available and/or the governing equations are not known. In this paper, we introduce a modification to RC-RNNs whereby nonlinear projectors are used to first expand the observables to a high dimensional space before they are fed to the reservoir. We demonstrate the application of such networks for forecasting spatiotemporally chaotic systems, which are nonlinear dynamical systems with a large number of degrees of freedom that can model several important features of turbulence [18, 19]. In particular, we explore the use of radial basis functions (RBFs) as nonlinear projectors to enable the networks to capture complex system nonlinearities without knowing the governing equations. We then explore scenarios in which only sparse or noisy and incomplete measurements are available, and compare the predictions from RC-RNNs with those from conventional time-marching.

Network details

We choose the echo-state network (ESN) of Jaeger [20] for its ability to forecast chaotic systems. However, we emphasize that our work is applicable to other RNNs as well. In ESNs the input state u(t)∈Rdu\mathbf{u}(t) \in\mathbb{R}^{d_u}, which consists of dud_u concurrent measurements in space, is connected to a high-dimensional dynamical system, say R\mathcal{R}, known as the reservoir. The reservoir has the activation state r(t)∈Rdr\mathbf{r}(t) \in\mathbb{R}^{d_r}, which is effectively a nonlinear function of the input history u(t)\mathbf{u}(t), u(t−Δt)\mathbf{u}(t-\Delta t), …\ldots, where Δt\Delta t is a discrete time step (Eq. 1a). The output v(t)∈Rdv\mathbf{v}(t) \in\mathbb{R}^{d_v} is then obtained as a linear readout from r(t)\mathbf{r}(t) using Eq. 1b.

r(t)=N(Ar(t−Δt)+Winu(t)),(1)\mathbf{r}(t) = \mathcal{N}\left(A\mathbf{r}(t-\Delta t) + W_{\mathrm{in}}\mathbf{u}(t)\right), \tag*{(1)}
v(t)=Woutr(t).(2)\mathbf{v}(t) = W_{\mathrm{out}}\mathbf{r}(t). \tag*{(2)}

Here, Win∈Rdr×duW_{\mathrm{in}} \in\mathbb{R}^{d_r \times d_u} connects the input state u(t)\mathbf{u}(t) to the reservoir nodes, A∈Rdr×drA \in\mathbb{R}^{d_r \times d_r} is an adjacency matrix that interconnects the reservoir nodes, N\mathcal{N} is a known nonlinear function (tanh here) that acts on each element separately (i.e. N([a1,a2,…])=[N(a1),N(a2),…]\mathcal{N}([a_1,a_2,\ldots]) = [\mathcal{N}(a_1),\mathcal{N}(a_2),\ldots]) and Wout∈Rdv×drW_{\mathrm{out}} \in\mathbb{R}^{d_v \times d_r} connects the reservoir nodes to the output (see the schematic in Figure 1 (a)).

Schematic of standard, parallelized, and modified echo state networks

Figure 1. (a) Schematic of a standard ESN. The input nodes are connected to a high-dimensional reservoir, which is the only hidden layer in the network. A large enough reservoir can capture the system nonlinearities from which the output (future predictions here) is obtained via linear readout. (b) The network in the prediction phase is closed-loop, i.e. the output of the network becomes its input at the next step. (c) Schematic of the ESN from [17]; it has two improvements: (i) it is parallelized and (ii) the reservoir state is expanded to a higher-dimension (such as from rr to [r,r2][r,r^2]). (d) Schematic of the proposed modified ESN; the observed vector u(t)\mathbf{u}(t) is expanded to unl(t)\mathbf{u}_{\mathrm{nl}}(t) via nonlinear projectors A1\mathcal{A}_1, A2\mathcal{A}_2, etc (i.e. unl(t)=[u(t),A1(u(t)),A2(u(t)),…]\mathbf{u}_{\mathrm{nl}}(t)=[\mathbf{u}(t),\mathcal{A}_1(\mathbf{u}(t)),\mathcal{A}_2(\mathbf{u}(t)),\ldots]. This means that the nonlinearity in the modified network is introduced before the reservoir instead of after it as in the network in (c).

The elements of WinW_{\mathrm{in}} and AA are fixed by drawing them from random distributions, such as from a uniform distribution in [−1,1][-1,1]. The elements of AA are scaled to keep its spectral radius (ρ\rho) usually below one so that the network satisfies the fading memory property, i.e. the future state depends only on a finite sequence of past inputs. Finally, the elements of WoutW_{\mathrm{out}} are optimized via a one-step linear regression that minimizes the mean-square error between v(t)\mathbf{v}(t) and the target signal v^(t)\hat{\mathbf{v}}(t) during the training time −T≤t≤0-T \le t \le0. In forecasting problems, which are of interest here, the target signal v^(t)=q(u(t+1))\hat{\mathbf{v}}(t) = q(\mathbf{u}(t+1)), where qq is a bijective function. Consequently, the network becomes closed-loop in the prediction phase, i.e. the model output becomes its input at the next step as shown in Figure 1 (b). See Appendix A for further details and [21] for practical guidance on designing ESNs.

If the complex nonlinearities of turbulent systems are to be captured, drd_r may need to be several orders of magnitude larger than dud_u. This can be computationally costly for turbulent systems for which dud_u is already large. Recognizing this, Pathak et al. (2018) [17] proposed two improvements (see Figure 1 (c)). First, they parallelized the ESNs based on the spatial locality of turbulent interactions, which facilitates parallel computing. The parallelized network has two additional hyperparameters: the number of parallel reservoirs NgN_g and the interaction length NiN_i. Second, they explicitly expanded the reservoir activation state to a higher-dimensional space such that the output is obtained as v(t)=Wout[r,r2]\mathbf{v}(t) = W_{\mathrm{out}}[\mathbf{r},\mathbf{r}^{2}]. The main drawback of this approach is that in order to capture the system nonlinearities, the nonlinearity added explicitly should closely match that present in the system equations themselves [15].

We will show that the network from Pathak et al. (2018) [17] can be improved if the expansion to the high-dimensional space happens before the reservoir. In other words, we expand the observables (i.e. u∈Rdu\mathbf{u}\in\mathbb{R}^{d_u}) to a high-dimensional space using nonlinear projector functions Al\mathcal{A}_l (l=1,2,…,Ll=1,2,\ldots,L). The expanded vector unl∈R(L+1)du\mathbf{u}_{\mathrm{nl}}\in\mathbb{R}^{(L+1)d_u} is then the input to the reservoir from which the output is trained via linear readout (see Figure 1 (d)).

Predictions from existing and modified networks

We consider the Kuramoto–Sivashinsky (KS) equation as our first example:

yt=−yyx−yxx−νyxxxx,x∈[0,L].(3)y_t=-yy_x-y_{xx}-\nu y_{xxxx},\qquad x\in[0,L]. \tag*{(3)}

The scalar field y(x,t)y(x,t) is periodic in space. The subscripts xx and tt denote the partial derivatives in space and time, respectively. The parameter ν\nu acts like the viscosity in the Navier–Stokes equations [22] and LL is the system size with which the attractor dimension increases linearly [23]. We numerically solve Eq. (3) for (ν,L)=(1,22)(\nu,L)=(1,22) with N=128N=128 gridpoints and timestep dt=0.25dt=0.25 [24]. In this study Δt\Delta t of networks is set as dtdt.

Fig. 2 (a) shows the exact signal from the simulations and its prediction from different ESNs. All the networks feature a single reservoir of size dr=1000d_r=1000, which is only an order of magnitude larger than du=128d_u=128. The networks are trained for −10000≤t≤0-10000\leq t\leq0, training with much smaller time history is possible but the predictive accuracy will then be lower as shown in [17]. We note that the KSr\mathrm{KS}_r network, which is an instance of the standard network (Figure 1 (a)), is unable to learn the system dynamics, with the corresponding forecast soon becoming uninterpretable. Nevertheless, the results drastically improve when the networks with added nonlinearities are used. Here KSr² and KSy² are instances of the network from [17] (Figure 1 (c)) and the proposed network (Figure 1 (d)), respectively. Both networks predict the exact signal in the short-term after which the patterns in the predictions appear to be spatiotemporally shifted. This shift is an inherent property of chaotic systems: their trajectories diverge exponentially in time even when they start from infinitesimally close initial conditions. The rate of separation is given by exp⁡(Λmax⁡t)\exp(\Lambda_{\max}t), where Λmax⁡\Lambda_{\max} is the maximal Lyapunov exponent and is positive for chaotic systems [25].

Exact signal and predictions of the Kuramoto-Sivashinsky system from different ESNs, mean prediction error, and valid prediction time

Figure 2. (a) The exact signal for the KS system and its predictions from different RC-RNNs: KSr (an instance of a standard network shown in Figure 1 (a)), KSr² (an instance of the network from [17] shown in Figure 1 (c)) and KSy² (an instance of the proposed network shown in Figure 1 (d)). All networks here are of size dr=1000d_r = 1000 and Ng=1N_g = 1 (i.e. the reservoir size is only an order of magnitude larger than dud_u). The KSr network, which has no explicitly added nonlinearity, cannot capture the system nonlinearities (unless maybe for much larger drd_r). The networks with added square nonlinearities (KSr² and KSy²) are able to capture the system nonlinearities well. The mean prediction errors (b) and the valid prediction time (c) from the different networks are also shown. The modified network KSy² has relatively slower growth of error as compared to the network of [17] (KSr²). The improvement is significant considering that the KSr² network would require four times more computational power to achieve similar performance to the KSy² network. The KSGrbf network is also an instance of the proposed network but with input to reservoir as [y,exp⁡(−0.03y2)][y,\exp(-0.03y^2)], this network is discussed with Figure 4 (d-Figure 4f).

We quantify the prediction error as E(t)=⟨(v(t)−v^(t))2⟩/σyE(t)=\sqrt{\langle(v(t)-\hat{v}(t))^2\rangle}/\sigma_y, where ⟨⋅⟩\langle\cdot\rangle denotes spatial averaging and σy\sigma_y is the standard deviation of y(x,t)y(x,t). To account for variability when comparing the performance of different ESNs, we take the mean of the error (E‾(t)\overline{E}(t)) for a series of predictions (see Appendix § 10 for details). Fig. refKSpreds (b) shows E‾(t)\overline{E}(t) for predictions from various ESNs. The mean error increases exponentially from any infinitesimal differences in the predictions at t=0t=0. A slower increase in E‾(t)\overline{E}(t) would reduce the effect of both measurement and model uncertainties on future predictions. However, we note that the mean-square-error contains only limited information [26]. For the predictions to be physically interpretable, their probability distribution should be close to that of the exact signal [27]. (This is the reason for the popularity of generative-adversarial networks [28], which can give better interpretable results often at the cost of increased mean-square-error [29]). If the predictions are physically interpretable in the long-term, E‾(t)\overline{E}(t) would saturate to a value close to 2\sqrt{2}.

As expected from the predictions in Fig. 2 (a), E‾(t)\overline{E}(t) from the standard network KSr not only increases rapidly but it also does not saturate to 2\sqrt{2}, i.e. the predictions are not interpretable. For all other networks used here, E‾(t)\overline{E}(t) saturates to 2\sqrt{2} thus indicating statistically consistent predictions. We note that the modified network KSy² has smaller growth-rate of error as compared to that for the network from [17] (KSr²). In fact, the KSr² network requires almost four times

@page 4/15 starts_mid=1 ends_mid=1

Exact signal and predictions of the Kuramoto-Sivashinsky system from different ESNs, mean prediction error, and valid prediction time

Figure 2 (PDF p. 4). (a) The exact signal for the KS system and its predictions from different RC-RNNs: KSr (an instance of a standard network shown in Figure 1 (a)), KSr² (an instance of the network from [17] shown in Figure 1 (c)) and KSy² (an instance of the proposed network shown in Figure 1 (d)). All networks here are of size dr=1000d_r = 1000 and Ng=1N_g = 1 (i.e. the reservoir size is only an order of magnitude larger than dud_u). The KSr network, which has no explicitly added nonlinearity, cannot capture the system nonlinearities (unless maybe for much larger drd_r). The networks with added square nonlinearities (KSr² and KSy²) are able to capture the system nonlinearities well. The mean prediction errors (b) and the valid prediction time (c) from the different networks are also shown. The modified network KSy² has relatively slower growth of error as compared to the network of [17] (KSr²). The improvement is significant considering that the KSr² network would require four times more computational power to achieve similar performance to the KSy² network. The KSGrbf network is also an instance of the proposed network but with input to reservoir as [y,exp⁡(−0.03y2)][y,\exp(-0.03y^2)], this network is discussed with Figure 4 (d-Figure 4f).

the results drastically improve when the networks with added nonlinearities are used. Here KSr² and KSy² are instances of the network from [17] (Figure 1 (c)) and the proposed network (Figure 1 (d)), respectively. Both networks predict the exact signal in the short-term after which the patterns in the predictions appear to be spatiotemporally shifted. This shift is an inherent property of chaotic systems: their trajectories diverge exponentially in time even when they start from infinitesimally close initial conditions. The rate of separation is given by exp⁡(Λmax⁡t)\exp(\Lambda_{\max}t), where Λmax⁡\Lambda_{\max} is the maximal Lyapunov exponent and is positive for chaotic systems [25].

We quantify the prediction error as E(t)=⟨(v(t)−v^(t))2⟩/σyE(t)=\sqrt{\langle(v(t)-\hat{v}(t))^2\rangle}/\sigma_y, where ⟨⋅⟩\langle\cdot\rangle denotes spatial averaging and σy\sigma_y is the standard deviation of y(x,t)y(x,t). To account for variability when comparing the performance of different ESNs, we take the mean of the error (E‾(t)\overline{E}(t)) for a series of predictions (see Appendix § 10 for details). Fig. refKSpreds (b) shows E‾(t)\overline{E}(t) for predictions from various ESNs. The mean error increases exponentially from any infinitesimal differences in the predictions at t=0t=0. A slower increase in E‾(t)\overline{E}(t) would reduce the effect of both measurement and model uncertainties on future predictions. However, we note that the mean-square-error contains only limited information [26]. For the predictions to be physically interpretable, their probability distribution should be close to that of the exact signal [27]. (This is the reason for the popularity of generative-adversarial networks [28], which can give better interpretable results often at the cost of increased mean-square-error [29]). If the predictions are physically interpretable in the long-term, E‾(t)\overline{E}(t) would saturate to a value close to 2\sqrt{2}.

As expected from the predictions in Fig. 2 (a), E‾(t)\overline{E}(t) from the standard network KSr not only increases rapidly but it also does not saturate to 2\sqrt{2}, i.e. the predictions are not interpretable. For all other networks used here, E‾(t)\overline{E}(t) saturates to 2\sqrt{2} thus indicating statistically consistent predictions. We note that the modified network KSy² has smaller growth-rate of error as compared to that for the network from [17] (KSr²). In fact, the KSr² network requires almost four times more computational power (i.e. Ng=4N_g = 4) to achieve a similar level of accuracy as the KSu2u^2 network. Although both networks have explicitly added square nonlinearities, the KSy2y^2 network outperforms the KSr2r^2 network. This is because in the KSy2y^2 network the reservoir receives more information about the system. In other words, the reservoir in the KSy2y^2 network is preconditioned with square nonlinearity and can thus better approximate the system nonlinearity.

Another metric recently introduced for evaluating the predictive performance of models is the valid prediction time (VPT) [16]. It refers to the time at which E(t)\mathcal{E}(t) crosses a threshold; the threshold is set as 0.5 following [16]. The advantage of using VPT is that it is a scalar measure that quantifies the predictive accuracy of different methods. Networks with slower growth-rate of the mean-square error also have larger VPT as shown in Fig. 2 (c) for the KS system; this trend is independent of the threshold value.

Partially observable systems: selection of nonlinear projectors

For the KS system, we showed that the networks with explicitly added square nonlinearities (KSr2r^2 and KSy2y^2) work well. As noted in [15], this is because the added nonlinearity in the networks matches the nonlinearity in the system equations (i.e. square nonlinearity in the KS system here). The form of system nonlinearity, however, is not always known; for example the governing equations are not known or full state-vector measurements are not available. To illustrate, we consider the Gray–Scott (GS) system, which models the reaction-diffusion of two species, as our second example,

y1t=(F−y1y22)−Fy1+D1y1xx,(4)y_{1t} = (F-y_1y_2^2)-Fy_1+D_1y_{1xx}, \tag*{(4)}
y2t=y1y22−(F+k)y2+D2y2xx.(5)y_{2t} = y_1y_2^2-(F+k)y_2+D_2y_{2xx}. \tag*{(5)}

where y1(x,t)y_1(x,t) and y2(x,t)y_2(x,t) are the concentration of two chemical species. We choose D1=2.0e−5D_1 = 2.0\mathrm{e}^{-5}, D2=1.0e−5D_2 = 1.0\mathrm{e}^{-5}, F=0.015F = 0.015, k=0.0415k = 0.0415 and x∈[0,1]x \in[0,1] with periodic boundary conditions for which the system exhibits annihilation type spatiotemporal chaos [30]. We numerically solve the GS system with 200 chebyshev points and dt=0.5dt = 0.5 and save the results on an equispaced grid containing 256 nodes [31]. The time evolution of the concentration of species one (y1y_1) is shown in Figure 3 (a).

Observed signal of the GS system and its predictions from different ESNs

Figure 3. (a) The observed signal, y1y_1, of the GS system and its predictions from the GSP3^3, GSP5^5, GSGrbf\mathrm{G}_{rbf} and GS2Grbf2\mathrm{G}_{rbf} networks of size dr=1500d_r = 1500 and Ng=4N_g = 4 (i.e. the total reservoir size is 6000, which is only an order of magnitude larger than dud_u). Although all four networks can predict the short-term dynamics, the networks with monomial projectors (GSP3^3 and GSP5^5) fail to give statistically consistent results while the networks with Gaussian RBF projectors (GSGrbf\mathrm{G}_{rbf} and GS2Grbf2\mathrm{G}_{rbf}) give statistically consistent results. Also shown are the mean prediction errors (b) and the valid prediction time (c) from the different ESNs. In addition to statistically consistent results, the networks with Gaussian RBF projectors also show slower growth-rates of the error and hence higher VPTs.

We now assume that only y1y_1 measurements are available (u(t)=y1(x,t)\mathbf{u}(t) = y_1(x,t)), i.e. we do not know the full state-vector. One may argue that we can still form the full state-vector in terms of (y1,y1t)(y_1,y_{1t}) or calculate y2y_2 using (y1,y1t,y1xx)(y_1,y_{1t},y_{1xx}). This will, however, require knowledge of Eq. 3 and noise free measurements of y1y_1 with high spatiotemporal resolution. Neither are usually available in practical situations, nor are they needed by our method (see the results in Figure 5 and Figure 6).

Sparsely observed signals of the KS system and their predictions from the governing equations and the KSy² network, with valid prediction time and autocorrelation plots

Figure 5. (a) Sparsely observed signals of the KS system and their predictions from the governing equations and the KSy2^2 network (dr=2000d_r = 2000 and Ng=1N_g = 1). (b) The VPT of predictions at different levels of sparsity. The observed signal becomes increasingly sparse as dud_u decreases (or dX≈128/dudX \approx128/d_u increases). (c) The VPT for the governing-equations-based predictions decreases rapidly with increasing dXdX, varying with the inverse of the square-root of the root-mean-square-error in the initial condition (e(dX)e(dX)). (d) The VPT for the KSy2^2 predictions are relatively unaffected as long as the autocorrelation between observations at neighbouring spatial locations (Cx(dX)C_x(dX)) remains close to one.

Partially observed noisy signals for the GS system and their predictions from the governing equations and the GS2G_{rbf} network, with VPT values

Figure 6. (a) Partially observed noisy signals for the GS system and their predictions from the governing equations and the GS2GrbfGS2G_{rbf} network (of size dr=1500d_r = 1500 and Ng=4N_g = 4). (b) VPT for the predictions from the governing equations and the GS2GrbfGS2G_{rbf} network. A reduction in predictive accuracy occurs with increasing noise level. The GS2GrbfGS2G_{rbf} network, however, is relatively less affected by the noise. The small values of VPT from both methods demonstrate the challenge of forecasting chaotic systems with partial observability.

With only y1y_1 measurements available, the form of system nonlinearity is not known. The challenge is therefore to select nonlinear projectors Al\mathcal{A}_l that can enable the network to capture the nonlinear dynamics of the GS system from y1y_1 measurements alone. First we try monomial projectors, such as y12y_1^2, y13y_1^3, etc., as used in new-generation-ESNs [32] and often used to approximate nonlinearity in turbulent systems [33]. (GSPn\mathrm{GSP}^{n} refers to an ESN with nonlinear projectors Al=y1l+1\mathcal{A}_l = y_1^{l+1}, l=1,2,…,n−1l = 1,2,\ldots,n-1.) We note from Figure 3 (b) that the growth of E‾\overline{E} slows down slightly with increasing number of monomials but E‾\overline{E} does not saturate to 2\sqrt{2} for any of the GSPn\mathrm{GSP}^{n} networks. Figure 3 (a) shows poor quality of the predictions from the GSPn\mathrm{GSP}^{n} networks. The statistics here is calculated based on the closed-loop predictions from 0≤t≤250000 \leq t \leq25000. We find that adding more monomials or reservoir units does not significantly improve the quality of predictions from the GSPn\mathrm{GSP}^{n} networks (results not shown).

Next, we try Gaussian RBFs as nonlinear projectors. This choice is inspired by the robust approximation properties of RBFs [34] on which RBF networks are based [35, 36]. For each Gaussian RBF projector A=exp⁡(−γ(y1(x,t)−yc(x))2)\mathcal{A} = \exp\left(-\gamma\left(y_1(x,t)-y_c(x)\right)^2\right), there are two parameters: (i) a shape factor γ\gamma and (ii) a space-dependent centre ycy_c. As in RBF networks, we choose the centre in an unsupervised manner and optimize for γ\gamma via a trial-and-correction procedure. In the GS system, y1(x,t)y_1(x,t) varies from 0 to 1, and the degree of nonlinearity increases from 1 to 0. We therefore choose yc(x)=0y_c(x) = 0 as the centre (yc(x)=1y_c(x) = 1 is an equally good choice). Figure 3 shows that the GSGrbf\mathrm{GSGrbf} network, which uses a Gaussian RBF projector exp⁡(−2y12)\exp(-2y_1^2), performs noticeably better, both in terms of short-term accuracy as well as the statistics of the predictions, as compared to any of the GSPn\mathrm{GSP}^{n} networks. The results are further improved, particularly in terms of the statistics, when two Gaussian RBF projectors (exp⁡(−2y12)\exp(-2y_1^2) and exp⁡(−3y12)\exp(-3y_1^2)) are used in the GS2Grbf\mathrm{GS2G}_{rbf} network.

Figure 3 (c) shows the VPT from the GSP5\mathrm{GSP}^{5} and GS2Grbf\mathrm{GS2G}_{rbf} networks, which highlights the significant improvement in the predictive accuracy when Gaussian RBFs are used instead of monomials as nonlinear projectors. The small VPT values from all the networks indicate the difficulty in obtaining accurate predictions from partial observations. This difficulty can be further appreciated from Figure 6 in which the governing equations are used for obtaining the predictions.

Relevance of nonlinear approximation properties of nonlinear projectors

We explore the connection between the nonlinear approximation properties of the projector functions and the performance of the corresponding ESNs. Figure 4 (a) shows the variation of the nonlinear term y1y22(y1)y_1y_2^2(y_1), its mean and mid-range values (mm) with the observed variable y1y_1 in the GS system. Here mm is the mean of the maximum and minimum values of y1y22y_1y_2^2 at a given y1y_1. The error in approximating mm by a nonlinear projector function (A\mathcal{A}) is given as

Nonlinear terms and errors in approximating their mid-range values for the GS and KS systems

Figure 4. Nonlinear terms (a, d) and errors in approximating their mid-range values via Gaussian RBFs (b, e) and monomials (c, f) for the GS system (a-c) and the KS system (d-f). For the GS system, the Gaussian RBFs have relatively lower error, thus explaining the superior performance of the networks with the Gaussian RBF projectors. For the KS system, even when the nonlinearity is a monomial, the Gaussian RBF can approximate the nonlinear term well.

Emid=min⁡a1,a2⟨(a1+a2A−m)2⟩⟨m2⟩,(6)E_{mid} = \min_{a_1,a_2} \frac{\sqrt{\left\langle(a_1+a_2\mathcal{A}-m)^2\right\rangle}}{\sqrt{\left\langle m^2\right\rangle}}, \tag*{(6)}

where A=exp⁡(−γy12)\mathcal{A} = \exp(-\gamma y_1^2) for Gaussian RBFs (the corresponding EmidE_{mid} is shown in subfigure (b)) and A=y1p\mathcal{A} = y_1^p for monomials (the corresponding EmidE_{mid} is shown in subfigure (c)). There are three points to note here. First, the value of γ\gamma at which GSgrbfGS_{\mathit{grbf}} performs best coincides with that at which EmidE_{\mathit{mid}} is minimum (i.e. γ≈2\gamma\approx2), demonstrating that a good nonlinear projector does indeed approximate the system nonlinearity. Second, minimization of the error with respect to the mid-range value (rather than the mean value) indicates that a good nonlinear projector minimizes the chances of maximum error (i.e. L∞L_{\infty}-error) [37]. Third, EmidE_{\mathit{mid}} is lower for the Gaussian RBFs than for the monomials, thus explaining the superior performance of the GSgrbfGS_{\mathit{grbf}} and GS2grbfGS2_{\mathit{grbf}} networks over the GSPnGSP_n networks in Figure 3. Such statistical interpretation indicates that RBFs can be useful in other RNNs as well.

Furthermore, the robustness of RBFs is evidenced by their ability to satisfactorily approximate most types of nonlinearities. For example, A=y2\mathcal{A}=y^2 is the best projector for the KS system for which the nonlinear term itself is y2y^2. However, Gaussian RBFs can still approximate y2y^2 nonlinearity well (see Figure 4 (e)) and the corresponding ESN (KSgrbfKS_{\mathit{grbf}}) predicts the KS system relatively well (see Fig. refKSpreds (b-c)). Here we have only shown the results for Gaussian RBFs; the results are similarly good when multiquadric RBFs are used (see Appendix § 11).

Comparison between model-free and governing-equations-based predictions

Application to Kuramoto–Sivashinsky system with sparse measurements

We now consider a situation where only sparse measurements of the KS system are available. We compare the predictive performance of the governing-equations-based time-marching with that of the KSy2y^2 network. We take the ground truth as the original simulation with N=128N=128 gridpoints. The observed signal (yobs(x,t)y_{\mathrm{obs}}(x,t)), shown in the middle column of Figure 5 (a), consists of measurements at dud_u equispaced points at distance dXL/NdX L/N. This implies that the observations become increasingly sparse with decreasing dud_u (or increasing dXdX).

The governing-equations-based predictions are obtained by time-marching from the initial condition at t=0t=0. The initial condition, say yint(x)y_{\mathrm{int}}(x), is obtained by linearly interpolating yobs(x,t=0)y_{\mathrm{obs}}(x,t=0) from the observed space (consisting of dud_u points) to the numerical grid (consisting of N=128N=128 points). The predictions and the VPT for different levels of sparsity in the observations are shown in Figure 5 (a) and (b), respectively. The accuracy of the predictions decreases with increasing sparsity as indicated by the reduction in VPT. Figure 5 (c) shows that the reduction in VPT is proportional to e(dX)−1\sqrt{e(dX)^{-1}}, where e(dX)=⟨(y(x,t=0)−yint(x,t=0))2⟩e(dX)=\sqrt{\langle(y(x,t=0)-y_{\mathrm{int}}(x,t=0))^2\rangle} is the root-mean-square error in the initial condition. We note that the error in the initial conditions could be reduced by using other interpolation schemes or data assimilation [38, 39]. However, the trend would remain the same because an interpolation scheme cannot entirely compensate for the missing information at smaller scales. We also note that the system parameters and boundary conditions are known perfectly here, which often contribute to error in the governing-equations-based predictions.

The predictions from the KSy2^2 network are obtained by training the network with the sparsely observed data, i.e. the network never sees the original data. The predictions and VPT from the KSy2^2 network for different levels of sparsity in the observations are shown in Figure 5 (a) and (b), respectively. We find that the predictions from the RC-RNN are insensitive to the discretization or interpolation error; a similar level of accuracy is maintained even when dud_u is reduced from NN to N/4N/4. Figure 5 (d) shows that VPT from KSy2^2 is proportional to Cx(dX)\sqrt{C_x(dX)}, where Cx(dX)C_x(dX) is the mean autocorrelation between y(x)y(x) and y(x+dXL/N)y(x+dXL/N) signals. The results show that information about the well correlated elements of the state-vector can be learned from the time history of partial measurements. This suggests that RC-RNNs can unravel the space-time dynamics in agreement with Takens’ embedding theorem [40], which has recently been suggested to be the case also for data-driven discovery of the governing equations from partial measurements [41].

The results show that RC-RNNs can provide excellent predictions up to moderate levels of sparsity even when the governing equations may suffer from high discretization errors. The accuracy of both methods can be improved, such as by using better interpolation schemes for the governing equations or by increasing the reservoir dimension for the RC-RNN. The key point, however, is that RC-RNNs are less affected by the sparsity as compared to governing-equations-based time-marching. When the sparsity is high such that the spatially adjacent measurements are beyond the first zero in the autocorrelation, we find that the RC-RNNs perform poorly. We expect the same limitations for other data-driven methods as well

Application to Gray–Scott system with partial and noisy measurements

We now consider another practical scenario in which only partial and noisy observables are available. We take the ground truth as the original simulation of the GS system. The concentration of species one with measurement noise, say y1obsy_{1obs}, is the observable (shown in the middle column of Figure 6 (a)). The measurement noise is additive, white in space and time, with uniform distribution between [−ξσy1,ξσy1][-\xi\sigma_{y_1}, \xi\sigma_{y_1}], where σy1\sigma_{y_1} is the standard deviation of y1y_1 and ξ\xi is the noise level. Similar to the sparse measurement cases, the governing-equations-based predictions are obtained by time-marching from the initial conditions that are calculated from the noisy measurements of y1y_1 at t=0t = 0, while RC-RNN-based predictions are obtained by training the GS2GrbfGS2G_{rbf} network with the noisy measurements.

The governing equations, their parameters and the boundary conditions are all known perfectly, but the initial condition for y2y_2 is now estimated from the noisy measurements of y1y_1. This introduces high levels of error in the initial condition and, consequently, the accuracy of the governing-equations-based predictions drops drastically with increasing noise-level as shown in Figure 6. The RC-RNNs are fundamentally different in that they predict the future by learning patterns in the time history. By design, therefore, they are robust to small levels of measurement noise. The results in Figure 6 show that although the accuracy of the RC-RNN-based predictions also decrease with increasing noise level, they do not suffer as much as the governing-equations-based predictions.

Conclusions

The main conclusions of this paper are threefold. The first two concern the explicitly added nonlinearity in RC-RNNs. Fig. 2 and Refs. [17, 15] show it to be essential to explicitly add nonlinearities in RC-RNNs to enable them to capture complex system nonlinearities. Firstly, we find it more computationally efficient to add the nonlinearity before the reservoir, such as in the KSy2KS_y^2 network, than after the reservoir, such as in the KSr2KS_r^2 network. Adding the nonlinearity before the reservoir preconditions the network towards the system dynamics and hence approximate the system nonlinearities better. Secondly, we find that the use of RBFs as nonlinear projectors (to explicitly add nonlinearities in RC-RNNs) enables the networks to capture the system nonlinearities even with partial measurements and/or without knowing the governing equations. We show that the robust approximation properties of RBFs, which makes them suitable for nonlinear approximation in numerical schemes and RBF networks [35, 34, 36], also makes them a good choice as nonlinear projectors in the RC-RNNs proposed here. The best nonlinear projectors approximate the mid-range value of the system nonlinearities, thus minimize the L∞L_\infty-error. This statistical interpretation enhances the viability of using RBFs in other RNNs for turbulence prediction.

The third conclusion is on viability of RC-RNNs as an alternative to conventional methods in practical situations. Observations in natural and engineering systems can often have missing data, coarse resolution and noise; thus several ML methods have been developed to extract information from limited and noisy measurements [6, 42]. Prediction of turbulent systems, however, is extremely sensitive to any estimation errors. This is evident from the results in Figure 5 and Figure 6, which show that the governing equations, even with perfectly known boundary conditions and parameters, quickly lose predictive accuracy as the measurements become sparse, incomplete and noisy. We find that the predictive performance of RC-RNNs is relatively less affected by sparsity and noise in the measurements, thus can potentially outperform the governing equations in such cases.

In summary, we have demonstrated the capabilities of our proposed RC-RNN framework to forecast (i) partially observable spatiotemporally chaotic systems with no knowledge of the form of system nonlinearity and (ii) sparsely observable systems or partially observable systems with measurement noise for which even the governing equations become inaccurate. No recourse to the governing equations is required, thus making our RC-RNN framework truly model-free. We emphasize that the partial and sparse observability in our work is fundamentally different from the model-reduction or partial observability discussed in recent studies on turbulence forecasting [12, 14]. In those studies, the extraction of dominant system modes may require noise-free measurements of the full state-vector. Finally, we also highlight the difficulty in predicting noisy and partially observable systems, indicating that much larger reservoirs than those used in this study may be required for useful predictions in real-world applications, such as weather forecasting. Understanding the relationship between the system dynamics and hyperparameters of RC-RNNs and the uncertainty quantification of the predictions would be vital to unlocking the full potential of applying RC-RNNs to practical turbulent systems.

Acknowledgments

We wish to thank Prof. Luca Biferale for his feedback during the preparation of this manuscript. This work was funded by the National Natural Science Foundation of China (Grant Nos. 12002147, 12050410247 and 11988102), the Shenzhen Science and Technology Program (Grant No. KQTD20180411143441009) and the Department of Science and Technology of Guangdong Province (2020B1212030001). We acknowledge support from the Centers for Mechanical Engineering Research and Education at MIT and SUSTech, as well as from the Center for Computational Science and Engineering at SUSTech.. LLKB gratefully acknowledges the funding from the Research Grants Council of Hong Kong (Projects 16210419, 16200220, 16215521).

Details of the reservoir networks

In an echo-state network (ESN), Win∈Rdr×duW_{in} \in\mathbb{R}^{d_r \times d_u} connects the input vector u(t)∈Rdu\mathbf{u}(t) \in\mathbb{R}^{d_u} to the reservoir nodes. Following [17], each reservoir node is connected to a single element from the input vector, while each element of the input vector is connected to dr/dud_r/d_u reservoir nodes. Connecting each reservoir node by a single input element ensures that uncorrelated input elements, such as from two spatially uncorrelated locations, do not get mixed in the reservoir. The non-zero elements of WinW_{in} are randomly chosen from a normal distribution in [−1,1][-1,1]. These elements are then scaled by σ\sigma, which can be a scalar or a vector. In most of the networks used in the present study, σ\sigma is a scalar. In KSGrbf\mathrm{KSGrbf}, GSPn\mathrm{GSP}^{n}, GSGrbf\mathrm{GSGrbf} and GS2Grbf\mathrm{GS2Grbf} networks, σ∈RL+1\sigma\in\mathbb{R}^{L+1} (where LL is the number of nonlinear projectors) such that input from each nonlinear projector is scaled independently.

The adjacency matrix A∈Rdr×drA \in\mathbb{R}^{d_r \times d_r} determines the internal connections between the reservoir nodes. The elements of AA are also randomly chosen from a normal distribution in [−1,1][-1,1], they are then rescaled such that the largest eigenvalue of AA (also known as the spectral radius) is equal to ρ\rho. It is sometimes recommended to make AA sparse by randomly fixing most of its elements to be equal to 0. However, we do not find any advantage in doing that in our case.

The elements of Wout∈Rdv×drW_{out} \in\mathbb{R}^{d_v \times d_r} are optimized to minimize the squared-error between the output v(t)\mathbf{v}(t) and the target signal v^(t)\hat{\mathbf{v}}(t). The optimized elements are obtained by performing the linear regression with Tikhonov regularization:

Wout=V^RT(RRT+βI)−1,(7)W_{out} = \hat{\mathbf{V}}\mathbf{R}^{T}\left(\mathbf{R}\mathbf{R}^{T}+\beta I\right)^{-1}, \tag*{(7)}

where V^=[v^(−T+NtranΔt),v^(−T+NtranΔt+Δt),…,v^(−Δt)]\hat{\mathbf{V}} = [\hat{\mathbf{v}}(-T+N_{tran}\Delta t),\hat{\mathbf{v}}(-T+N_{tran}\Delta t+\Delta t),\ldots,\hat{\mathbf{v}}(-\Delta t)], R=[r(−T+NtranΔt),r(−T+NtranΔt+Δt),…,r(−Δt)]\mathbf{R} = [\mathbf{r}(-T+N_{tran}\Delta t),\mathbf{r}(-T+N_{tran}\Delta t+\Delta t),\ldots,\mathbf{r}(-\Delta t)], β=1e−5\beta= 1\mathrm{e}^{-5} is the regularization parameter and I\mathbf{I} is an identity matrix. The vectors v^(−T),v^(−T+Δt),…,v^(−Δt)\hat{\mathbf{v}}(-T),\hat{\mathbf{v}}(-T+\Delta t),\ldots,\hat{\mathbf{v}}(-\Delta t) and r(−T),r(−T+Δt),…,r(−Δt)\mathbf{r}(-T),\mathbf{r}(-T+\Delta t),\ldots,\mathbf{r}(-\Delta t) are the target signal and reservoir states during the training period, i.e. from t=−Tt=-T to 0, Ntran=50N_{tran}=50 is the number of transient steps discarded during the training and Δt\Delta t is the discrete time step of the network.

Calculations of the mean prediction error

There are two sources of randomness in the calculations of the prediction error (E(t)\mathcal{E}(t)): (i) the initial condition of the system and (ii) the random seed used when drawing the elements of WinW_{in} and AA matrices. Once the network is trained using data from t=−Tt=-T to 0, we calculate the predictions by reinitializing the network for 200 initial conditions. Given that ESNs satisfy the fading memory property, we only need a finite sequence of past inputs, say from tr−NReInitΔtt_r-N_{ReInit}\Delta t to tr−Δtt_r-\Delta t, to obtain the predictions starting from trt_r. We find that a finite sequence of NReInit=10N_{ReInit}=10 past inputs is sufficient to reinitialize the ESNs in our study. During this reinitialization, the activation state is set to zero, i.e. r(tr−NReInitΔt)=0\mathbf{r}(t_r-N_{ReInit}\Delta t)=0, and is then evolved using the inputs u(t−NReInitΔt),u(tr−NReInitΔt+Δt),…,u(tr−Δt)\mathbf{u}(t-N_{ReInit}\Delta t),\mathbf{u}(t_r-N_{ReInit}\Delta t+\Delta t),\ldots,\mathbf{u}(t_r-\Delta t). Such calculations are repeated for 10 random seeds. The mean prediction error (E(t)‾\overline{\mathcal{E}(t)}) is obtained by averaging the prediction error (E(t)\mathcal{E}(t)) over 200 initial conditions and 10 random seeds, i.e. for 2000 calculations.

The mean prediction error (E(t)‾\overline{\mathcal{E}(t)}) presented in the paper is approximately optimized over all the hyperparameters for each network. The main hyperparameters in a network of given size drd_r and the number of parallel units NgN_g are: (i) the interaction length NiN_i, (ii) the spectral radius ρ\rho of AA and (iii) the weight σ\sigma of input elements. When nonlinear projector functions are used, there are additional hyperparameters associated with the projector functions.

Observed signal and predictions from different RC-RNNs

Figure C.7. (a) The observed signal y1y_1 of the Gray-Scott system and (b-d) its predictions from different RC-RNNs. The networks GSGrbfGS_{Grbf} (yc=0y_c = 0) uses exp⁡(−2y12)\exp(-2y_1^2), GSGrbfGS_{Grbf} (yc=1y_c = 1) uses exp⁡(−2(y1−1)2)\exp(-2(y_1 - 1)^2) and GSMrbfGS_{Mrbf} (yc=0y_c = 0) uses 1+3y12\sqrt{1 + 3y_1^2} as nonlinear projectors.

Different radial basis functions

In the main text, we only used Gaussian radial basis functions (RBFs) as nonlinear projector for which we fixed the center at yc(x)=0y_c(x) = 0 to predict the Gray-Scott system for which only concentration of one species, y1(x,t)y_1(x,t), is observable. Figure 2 shows that using Gaussian RBF with center at yc=1y_c = 1 and multiquadric RBF are equally good options, thus showing the robustness of RBFs as nonlinear projectors. A multiquadric RBF is given as 1+γ(y1−yc)2\sqrt{1 + \gamma(y_1 - y_c)^2}, where γ\gamma is the shape factor and ycy_c is the center.

References

  1. [1]G. Hinton, L. Deng, D. Yu, G. Dahl, A. R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, B. Kingsbury, Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, IEEE Signal Processing Magazine 29 (2012) 82–97. doi:10.1109/MSP.2012.2205597.DOI
  2. [2]M. Alber, A. B. Tepole, W. R. Cannon, S. De, S. Dura-Bernal, K. Garikipati, G. Karniadakis, W. W. Lytton, P. Perdikaris, L. Petzold, E. Kuhl, Integrating machine learning and multi-scale modeling—perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences, Digital Medicine 2 (12 2019). doi:10.1038/s41746-019-0193-y.DOI
  3. [3]X. Shi, Z. Gao, L. Lausen, H. Wang, D.-Y. Yeung, W.-K. Wong, W.-C. Woo, Deep learning for precipitation nowcasting: A benchmark and a new model, 2017, pp. 5622–5632. URL http://arxiv.org/abs/1706.03458
  4. [4]Z. Y. Wan, T. P. Sapsis, Reduced-space gaussian process regression for data-driven probabilistic forecast of chaotic dynamical systems, Physica D 345 (2017) 40–55. doi:10.1016/j.physd.2016.12.005. URL http://arxiv.org/abs/1611.01583http://dx.doi.org/10.1016/j.physd.2016.12.005
  5. [5]Z. Y. Wan, P. Vlachas, P. Koumoutsakos, T. Sapsis, Data-assisted reduced-order modeling of extreme events in complex dynamical systems, PLoS ONE 13 (5 2018). doi:10.1371/journal.pone.0197704.DOI
  6. [6]M. Reichstein, G. Camps-Valls, B. Stevens, M. Jung, J. Denzler, N. Carvalhais, Prabhat, Deep learning and process understanding for data-driven earth system science, Nature 566 (2019) 195–204. doi:10.1038/s41586-019-0912-1.DOI
  7. [7]K. Kashinath, M. Mustafa, A. Albert, J. L. Wu, C. Jiang, S. Esmaeilzadeh, K. Azizadehnesheli, R. Wang, A. Chattopadhyay, A. Singh, A. Manepalli, D. Chirila, R. Yu, R. Walters, B. White, H. Xiao, H. A. Tchelepi, P. Marcus, A. Anandkumar, P. Hassanzadeh, Prabhat, Physics-informed machine learning: Case studies for weather and climate modelling, Phil. Trans. R. Soc. A 379 (2021) 20200093. doi:10.1098/rsta.2020.0093.DOI
  8. [8]O. Watt-Meyer, N. D. Brenowitz, S. K. Clark, B. Henn, A. Kwa, J. McGibbon, W. A. Perkins, C. S. Bretherton, Correcting weather and climate models by machine learning nudged historical simulations, Geophysical Research Letters 48 (8 2021). doi:10.1029/2021GL092555.
  9. [9]J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizazdenesheli, P. Hassanzadeh, K. Kashinath, A. Anandkumar, Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators, arXiv.2202.11214 (2 2022). URL http://arxiv.org/abs/2202.11214
  10. [10]C. E. Leith, R. H. Kraichnan, Predictability of turbulent flows, J. Atmos. Sci. 29 (1972) 1041–1058.
  11. [11]E. N. Lorenz, The essence of chaos (1993).
  12. [12]P. R. Vlachas, W. Byeon, Z. Y. Wan, T. P. Sapsis, P. Koumoutsakos, Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 474 (5 2018). doi:10.1098/rspa.2017.0844.DOI
  13. [13]Z. Li, N. Kovachki, K. Azizadehnesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar, Fourier neural operator for parametric partial differential equations, arXiv.2010.08895 (10 2020). URL http://arxiv.org/abs/2010.08895
  14. [14]P. R. Vlachas, G. Arampatzis, C. Uhler, P. Koumoutsakos, Multiscale simulations of complex systems by learning their effective dynamics, Nature Machine Intelligence 4 (2022) 359–366. doi:10.1038/s42256-022-00464-w.DOI
  15. [15]A. Chattopadhyay, P. Hassanzadeh, D. Subramanian, Data-driven predictions of a multiscale lorenz 96 chaotic system using machine-learning methods: Reservoir computing, artificial neural network, and long short-term memory network, Nonlinear Processes in Geophysics 27 (2020) 373–389. doi:10.5194/npg-27-373-2020.DOI
  16. [16]P. R. Vlachas, J. Pathak, B. R. Hunt, S. T. P., M. Girvan, E. Ott, P. Koumoutsakos, Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics, Neural Networks 126 (2020) 191–217. URL https://www.elsevier.com/open-access/userlicense/1.0/
  17. [17]J. Pathak, B. Hunt, M. Girvan, Z. Lu, E. Ott, Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach, Physical Review Letters 120 (1 2018). doi:10.1103/PhysRevLett.120.024102.DOI
  18. [18]V. Yakhot, Large-scale properties of unstable systems governed by the Kuramoto-Sivashinski equation, Physical Review A 24(1) (1981) 642–643.
  19. [19]M. C. Cross, P. C. Hohenberg, Pattern formation outside of equilibrium, Reviews of Modern Physics 65 (1993) 851–1112. doi:10.1103/RevModPhys.65.851.DOI
  20. [20]H. Jaeger, H. Haas, Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science 304 (2004) 78–81.
  21. [21]M. Lukoševičius, A practical guide to applying echo state networks (2012).
  22. [22]V. Bratanov, F. Jenko, D. R. Hatch, M. Wilczek, Nonuniversal power-law spectra in turbulent systems, Phys. Rev. Lett. 111 (2013) 075001.
  23. [23]P. Manneville, Liapounov exponents for the kuramoto-sivashinsky model (1985).
  24. [24]A. K. Kassam, L. N. Trefethen, Fourth-order time-stepping for stiff pdes, SIAM Journal on Scientific Computing 26 (2005) 1214–1233. doi:10.1137/S1064827502410633.DOI
  25. [25]G. Boffetta, M. Cencini, M. Falcioni, A. Vulpiani, Predictability: a way to characterize complexity, Physics Reports 356 (2002) 367–474.
  26. [26]A. J. Majda, N. Chen, Model error, information barriers, state estimation and prediction in complex multiscale systems, Entropy 20 (8 2018). doi:10.3390/e20090644.DOI
  27. [27]T. DelSole, Predictability and information theory. part i: Measures of predictability, J. Atmos. Sci. 61 (2004) 2425–2440.
  28. [28]I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, Curran Associates, Inc., 2014. URL http://www.github.com/goodfeli/adversarial
  29. [29]H. Kim, J. Kim, S. Won, C. Lee, Unsupervised deep learning for super-resolution reconstruction of turbulence, J. Fluid Mech. 910 (2021) A29. doi:10.1017/jfm.2020.1028. URL http://arxiv.org/abs/2007.15324http://dx.doi.org/10.1017/jfm.2020.1028
  30. [30]Y. Nishiura, D. Ueyama, Spatio-temporal chaos for the gray-scott model, Physica D 150 (2001) 137–162.
  31. [31]T. A. Driscoll, N. Hale, L. N. Trefethen, Chebfun Guide, Pafnuty Publications, 2014.
  32. [32]D. J. Gauthier, E. Bollt, A. Griffith, W. A. Barbosa, Next generation reservoir computing, Nature Communications 12 (12 2021). doi:10.1038/s41467-021-25801-2.DOI
  33. [33]S. L. Brunton, J. L. Proctor, J. N. Kutz, W. Bialek, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proceedings of the National Academy of Sciences of the United States of America 113 (2016) 3932–3937. doi:10.1073/pnas.1517384113.DOI
  34. [34]M. D. Buhmann, Radial basis functions, Acta Numerica 9 (2000) 1–38.
  35. [35]F. Girosi, T. Poggio, Networks and the best approximation property, Biol. Cybern. 63 (1990) 169–176.
  36. [36]R. Zemouri, D. Racoceau, N. Zerhouni, Recurrent radial basis function network for time-series prediction, Engineering Applications of Artificial Intelligence 16 (2003) 453–463. doi:10.1016/S0952-1976(03)00063-0.DOI
  37. [37]A. Giloni, M. Padberg, Alternative methods of linear regression, Mathematical and Computer Modelling 35 (2002) 361–374.
  38. [38]J. S. Whitaker, G. P. Compo, J. N. Thé​paut, A comparison of variational and ensemble-based data assimilation systems for reanalysis of sparse observations, Monthly Weather Review 137 (2009) 1991–1999. doi:10.1175/2008MWR2781.1.DOI
  39. [39]J. Brajard, A. Carrassi, M. Bocquet, L. Bertino, Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the lorenz 96 model, Journal of Computational Science 44 (7 2020). doi:10.1016/j.jocs.2020.101171.DOI
  40. [40]F. Takens, Detecting strange attractors in turbulence (1981).
  41. [41]J. Bakarji, K. Champion, J. N. Kutz, S. L. Brunton, Discovering governing equations from partial measurements with deep delay autoencoders, arXiv:2201.05136 (1 2022). URL http://arxiv.org/abs/2201.05136
  42. [42]M. Raissi, A. Yazdani, G. E. Karniadakis, Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations, Science 367 (2020) 1026–1030. URL http://science.sciencemag.org/

Paper details

vlnc:W_tpu64c29
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents