betaFeedback

WorksPhysics and Astronomy

Combining Stochastic Parameterized Reduced-Order Models with Machine Learning for Data Assimilation and Uncertainty Quantification with Partial Observations

Abstract

A hybrid data assimilation algorithm is developed for complex dynamical systems with partial observations. The method starts with applying a spectral decomposition to the entire spatiotemporal fields, followed by creating a machine learning model that builds a nonlinear map between the coefficients of observed and unobserved state variables for each spectral mode. A cheap low-order nonlinear stochastic parameterized extended Kalman filter (SPEKF) model is employed as the forecast model in the ensemble Kalman filter to deal with each mode associated with the observed variables. The resulting ensemble members are then fed into the machine learning model to create an ensemble of the corresponding unobserved variables. In addition to the ensemble spread, the training residual in the machine learning-induced nonlinear map is further incorporated into the state estimation that advances the quantification of the posterior uncertainty. The hybrid data assimilation algorithm is applied to a precipitating quasi-geostrophic (PQG) model, which includes the effects of water vapor, clouds, and rainfall beyond the classical two-level QG model. The complicated nonlinearities in the PQG equations prevent traditional methods from building simple and accurate reduced-order forecast models. In contrast, the SPEKF model is skillful in recovering the intermittent observed states, and the machine learning model effectively estimates the chaotic unobserved signals. Utilizing the calibrated SPEKF and machine learning models under a moderate cloud fraction, the resulting hybrid data assimilation remains reasonably accurate when applied to other geophysical scenarios with nearly clear skies or relatively heavy rainfall, implying the robustness of the algorithm for extrapolation.

Introduction

Complex dynamical systems are ubiquitous in many areas, including geoscience, engineering, neural science, material science, etc. [24, 47, 70, 62] These systems are often high dimensional and strongly nonlinear. Multiscale structures, intermittent instabilities, and non-Gaussian probability density functions (PDFs) are typical features observed in these systems. Modeling and predicting these complex dynamical systems are central scientific problems with significant societal impacts. Since many of these systems are chaotic or turbulent, an accurate estimation of the model states at the initialization stage is vital to facilitate the subsequent forecast. Therefore, developing suitable data assimilation algorithms, which optimally integrate different sources of information to improve the state estimation of a complex dynamical system, becomes an essential prerequisite for reaching skillful forecast results [39, 44, 43, 49]. A typical data assimilation cycle contains two steps. The first step involves a statistical prediction using a given forecast model starting from the previously estimated state. The resulting prior distribution is then corrected based on the statistical input of noisy observation in the second step, leading to the so-called posterior distribution. These two steps are known as ‘forecast’ (or prediction) and ‘analysis’ (filtering or correction), respectively.

One of the fundamental challenges in data assimilation is its high computational cost, especially at the forecast step. In fact, due to the intrinsic high-dimensionality and complicated multiscale nonlinear features, simulating a single realization of the underlying dynamics is already computationally expensive, let alone running the model forward many times when applying the ensemble-based data assimilation methods. Reduced-order models are thus widely used to mitigate such a computational issue. In general, if the governing equation of the underlying system has simple explicit structures, then direct projections of the original system to a reduced order space with suitable closures can be applied for model reduction [35, 67, 33, 53, 56]. Stochastic parameterizations are often incorporated into the resulting reduced-order models to further improve the computational efficiency and accuracy [29, 51, 3, 17, 18, 64, 14]. On the other hand, as the primary goal for the time integration of the model is to seek the forecast statistics, nonlinear statistical reduced-order models have been developed to predict the leading few moments that help reconstruct the forecast PDF [50, 58, 15, 63]. In addition, various machine learning forecast approaches have recently been utilized as surrogate forecast models in data assimilation to produce forecast statistics or forecast ensemble members [60, 28, 68, 71, 69, 10, 9, 52, 57, 12, 55].

In this paper, a hybrid data assimilation algorithm is developed for complex turbulent systems, where the observations contain only a subset of the state variables, known as partial observations. The method combines stochastic parameterized reduced-order models with machine learning to facilitate computational efficiency. The idea of the method is first to apply a spectral decomposition to the entire spatiotemporal fields, followed by developing a machine learning model that builds a nonlinear map between the coefficients of the observed and the unobserved state variables for each spectral mode. Afterward, different strategies are adopted to filter these two sets of state variables. Specifically, a cheap low-order nonlinear stochastic parameterized extended Kalman filter (SPEKF) model [27, 26] is employed as the forecast model in the ensemble Kalman filter (EnKF) [22, 8, 36] to filter the coefficient of each spectral mode associated with the observed state variables. The resulting ensemble members are then fed into the machine learning model to create an ensemble of the unobserved variables associated with the same spectral mode. Notably, due to the turbulent nature and the intrinsic unpredictable components, a training residual typically exists in the optimally calibrated machine learning model, which accounts for the uncertainty in discovering the nonlinear dependence between the observed and unobserved variables [12]. Therefore, in addition to the ensemble spread, the training residual in the machine learning-induced nonlinear map is further incorporated into the state estimation that advances the quantification of the posterior uncertainty. The final posterior state is described by a mixture distribution that can capture the underlying non-Gaussian features.

The hybrid data assimilation approach has several unique merits. First, the spectral decomposition breaks the entire high-dimensional data assimilation problem into many low-dimensional subproblems, each focusing on filtering one primary spectral mode (and a few aliasing modes in the case with sparse observations, if applicable). Coping with such a low-dimensional subproblem facilitates the development of cheap stochastic parameterized models for filtering the observed variables and proper machine learning models for estimating the unobserved states.

Second, the nonlinear SPEKF model is systematically calibrated to reproduce the non-Gaussian statistics of the true signal. Therefore, it can accurately estimate the observed state variables, including recovering intermittency and extreme events, with appropriate uncertainty quantification. The simple but effective stochastic parameterizations in the SPEKF forecast model also play a vital role in compensating for the effect of the complicated nonlinearities in the governing equation of the time series associated with each spectral mode when carrying out the statistical forecast [48, 45, 31, 40]. Such a statistical approximation allows the filtering of different spectral modes independently, significantly reducing the computational cost. Despite the independence of the forecast models, the estimated states of different spectral modes are naturally correlated after the analysis step when the observations are involved in correcting the forecast errors. In addition, spatial dependence between the state variables at different grid points in physical space is automatically recovered after the spatial reconstruction in light of all the spectral modes. Notably, the simple nonlinear SPEKF model is generally more efficient than complicated machine learning models in terms of both model calibration and statistical forecast. This distinguishes the hybrid strategy from utilizing a sophisticated neural network to fully replace the original dynamical model as the forecast system.

Third, reduced-order models are often essential in ensemble-based data assimilation to accelerate the computations. Since there is a lack of observations to directly rectify the error in forecasting the unobserved states, the skill of filtering these states relies heavily on the accuracy of the reduced order models in characterizing the complex nonlinear statistical dependence between the observed and unobserved variables. However, the structures of many dynamical systems, especially those in fluids and geophysics, are too complicated to allow the development of skillful reduced-order models with simple analytic expressions using traditional approaches. Nevertheless, since the primary goal of these reduced-order models is to provide statistical forecast results, it is often unnecessary to explicitly take into account the exact physics in developing these models. Thus, machine learning is an effective surrogate of the traditional parametric reduced-order models in discovering complicated nonlinear dependence between different state variables. Particularly, as the focus of the machine learning is only on a low-dimensional subspace after the spectral decomposition, designing and training the machine learning model becomes much more accessible. Finally, beyond the deterministic forecast, the uncertainty in the machine learning model is incorporated into the estimation of the posterior distribution for these turbulent models. This advances a more accurate quantification of the uncertainty in data assimilation.

The hybrid data assimilation strategy is applied to a two-level, precipitating quasi-geostrophic (PQG) model [65, 21, 37]. Different from the classical QG model [61, 59, 70] (hereinafter referred to as dry QG), the PQG model takes into account the effects of water vapor, clouds, and rainfall. Therefore, it can more realistically describe atmospheric dynamics. It also has the potential to elucidate aspects of the hydrological cycle and the effects of latent heat release on synoptic-scale midlatitude dynamics. The PQG equations include Heaviside nonlinearities due to phase changes, which can potentially have a significant influence on QG turbulence. However, these Heaviside nonlinearities, together with solving a (nonlinear) elliptic equation to recover the streamfunction at each step, prevent using traditional methods for building effective reduced-order models with a simple explicit expression. The machine learning model then becomes crucial to discover the nonlinear dependence between different state variables in facilitating data assimilation.

The rest of the paper is organized as follows. The hybrid data assimilation strategy is presented in § 2. The PQG model, which serves as the test model for the hybrid data assimilation strategy, is summarized in § 3. The data assimilation results are shown in § 4. § 5 includes additional discussions and a conclusion.

Hybrid Data Assimilation Strategy

Consider the following general form of complex dynamical systems:

∂u(x,t)∂t=F1(u(x,t),h(x,t)),(1)\frac{\partial\mathbf{u}(\mathbf{x},t)}{\partial t}=\mathcal{F}_1(\mathbf{u}(\mathbf{x},t),\mathbf{h}(\mathbf{x},t)), \tag*{(1)}
∂h(x,t)∂t=F2(u(x,t),h(x,t)),(2)\frac{\partial\mathbf{h}(\mathbf{x},t)}{\partial t}=\mathcal{F}_2(\mathbf{u}(\mathbf{x},t),\mathbf{h}(\mathbf{x},t)), \tag*{(2)}

where the vectors u(x,t)\mathbf{u}(\mathbf{x},t) and h(x,t)\mathbf{h}(\mathbf{x},t) are the state variables with dimensions S1S_1 and S2S_2, respectively, x\mathbf{x} is the spatial coordinate vector, and tt is the time. On the right-hand side of (1), F1\mathcal{F}_1 and F2\mathcal{F}_2 are complicated nonlinear functions containing differential operators, which may not necessarily have explicit expressions. For the convenience of presenting the hybrid data assimilation framework, periodic boundary conditions are imposed for all the state variables in (1), which facilitates the use of Fourier basis functions in the spectral decomposition. Other spectral decomposition methods, such as the proper orthogonal decomposition (POD) [2] and the autoencoder networks [41], can be adopted for more general situations.

In many practical problems, only a subset of the state variables is observed, known as partial observations. To this end, assume that u\mathbf{u} contains the observed state variables, while there are no observations for h\mathbf{h}. Further, assume that the observational locations are sparse in space at regularly-spaced grid points. More generally, sparse irregularly-spaced observations can be interpolated to the regularly-spaced model grid points. See [49, 32] for the detailed procedure and the mathematical theory of the error estimates.

Denote by u{s1}u^{\{s_1\}} and h{s2}h^{\{s_2\}} a scalar component of u\mathbf{u} and h\mathbf{h}, respectively, where s1∈{1,…,S1}s_1 \in\{1,\ldots,S_1\} and s2∈{1,…,S2}s_2 \in\{1,\ldots,S_2\}. Apply a spectral decomposition to u{s1}(x,t)u^{\{s_1\}}(\mathbf{x},t) and h{s2}(x,t)h^{\{s_2\}}(\mathbf{x},t) with Fourier bases,

u{s1}(x,t)=∑k∈Ku^k{s1}(t)exp⁡(ikx),h{s2}(x,t)=∑k∈Kh^k{s2}(t)exp⁡(ikx),(3)\begin{aligned} u^{\{s_1\}}(\mathbf{x},t) &= \sum_{\mathbf{k}\in\mathcal{K}} \hat{u}_{\mathbf{k}}^{\{s_1\}}(t)\exp(i\mathbf{k}\mathbf{x}),\\ h^{\{s_2\}}(\mathbf{x},t) &= \sum_{\mathbf{k}\in\mathcal{K}} \hat{h}_{\mathbf{k}}^{\{s_2\}}(t)\exp(i\mathbf{k}\mathbf{x}), \tag*{(3)} \end{aligned}

where K\mathcal{K} is a finite index set containing wavevectors. Let u^k(t)=(u^k{s1}(t),…,u^k{S1}(t))\hat{\mathbf{u}}_{\mathbf{k}}(t)=(\hat{u}_{\mathbf{k}}^{\{s_1\}}(t),\ldots,\hat{u}_{\mathbf{k}}^{\{S_1\}}(t)) and h^k(t)=(h^k{s1}(t),…,h^k{S2}(t))\hat{\mathbf{h}}_{\mathbf{k}}(t)=(\hat{h}_{\mathbf{k}}^{\{s_1\}}(t),\ldots,\hat{h}_{\mathbf{k}}^{\{S_2\}}(t)) be the collections of the Fourier coefficients for a fixed wavevector k\mathbf{k}.

A schematic illustration of the algorithm is shown in Figure 1 with the details being discussed in the following subsections.

Figure 1. A schematic illustration of the hybrid data assimilation method.

Filtering the Observed Variables Using Stochastic Parameterized Forecast Models

The exact governing equation of u^k{sj}(t)\hat{u}_{\mathbf{k}}^{\{s_j\}}(t) involves complicated nonlinear interactions with not only all the other components in u^k(t)\hat{\mathbf{u}}_{\mathbf{k}}(t) and h^k(t)\hat{\mathbf{h}}_{\mathbf{k}}(t) but also those in u^k′(t)\hat{\mathbf{u}}_{\mathbf{k}'}(t) and h^k′(t)\hat{\mathbf{h}}_{\mathbf{k}'}(t) for k′≠k\mathbf{k}' \ne\mathbf{k}. Utilizing such an equation as the forecast model of u^k{sj}(t)\hat{u}_{\mathbf{k}}^{\{s_j\}}(t) in data assimilation is computationally expensive for at least two reasons. First, the entire high-dimensional system of u^k(t)\hat{\mathbf{u}}_{\mathbf{k}}(t) and h^k(t)\hat{\mathbf{h}}_{\mathbf{k}}(t) for all k∈K\mathbf{k}\in\mathcal{K} has to be integrated forward at the same time to reach the forecast state of even a single component u^k{sj}(t)\hat{u}_{\mathbf{k}}^{\{s_j\}}(t). Second, the right-hand side of the governing equation of each u^k{sj}(t)\hat{u}_{\mathbf{k}}^{\{s_j\}}(t) contains complicated nonlinear terms, whose calculation at each time step requires a significant amount of computational time.

To this end, a stochastic parameterized extended Kalman filter (SPEKF) model [27, 26] is utilized as the forecast model of each u^k{sj}(t)\hat{u}_{\mathbf{k}}^{\{s_j\}}(t) to reduce the computational cost. The idea of the SPEKF model is to build a statistical approximate forecast model that utilizes simple stochastic parameterizations to compensate for the complicated nonlinear interactions between u^k{sj}(t)\hat{u}_k^{\{s_j\}}(t) and the coefficients of the other spectral modes, such that the statistical forecast of u^k{sj}(t)\hat{u}_k^{\{s_j\}}(t) stays close to that using the exact governing equation. The resulting equation of u^k{sj}(t)\hat{u}_k^{\{s_j\}}(t) depends only explicitly on itself but not the coefficients of other spectral modes. This not only reduces the complexity on the right-hand side of u^k{sj}(t)\hat{u}_k^{\{s_j\}}(t) by a significant amount, but also allows the filtering of different spectral modes independently.

Below, the framework of the SPEKF is presented for a single scalar component of the Fourier coefficient of the observed state variables. For notational simplicity, the superscript in u^k{sj}(t)\hat{u}_k^{\{s_j\}}(t) is omitted, and the scalar Fourier coefficient is simply denoted by u^k(t)\hat{u}_k(t).

The SPEKF Model

The forecast model of u^k(t)\hat{u}_k(t) is approximated by the following SPEKF model:

du^k(t)dt=(−γk(t)+iωk(t))u^k(t)+bk(t)+fk(t)+σkuW˙ku,(4)\frac{\mathrm{d}\hat{u}_k(t)}{\mathrm{d}t} = \left(-\gamma_k(t) + \mathrm{i}\omega_k(t)\right)\hat{u}_k(t) + b_k(t) + f_k(t) + \sigma_k^u\dot{W}_k^u, \tag*{(4)}
dγk(t)dt=−dkγ(γk(t)−γˉk)+σkγW˙kγ,(5)\frac{\mathrm{d}\gamma_k(t)}{\mathrm{d}t} = -d_k^\gamma\left(\gamma_k(t) - \bar{\gamma}_k\right) + \sigma_k^\gamma\dot{W}_k^\gamma, \tag*{(5)}
dωk(t)dt=−dkω(ωk(t)−ωˉk)+σkωW˙kω,(6)\frac{\mathrm{d}\omega_k(t)}{\mathrm{d}t} = -d_k^\omega\left(\omega_k(t) - \bar{\omega}_k\right) + \sigma_k^\omega\dot{W}_k^\omega, \tag*{(6)}
dbk(t)dt=−dkb(bk(t)−bˉk)+σkbW˙kb,(7)\frac{\mathrm{d}b_k(t)}{\mathrm{d}t} = -d_k^b\left(b_k(t) - \bar{b}_k\right) + \sigma_k^b\dot{W}_k^b, \tag*{(7)}

where three additional stochastic processes γk(t)\gamma_k(t), ωk(t)\omega_k(t) and bk(t)b_k(t) are coupled to the governing equation of u^k(t)\hat{u}_k(t). They represent the stochastic damping, stochastic phase and stochastic forcing, respectively. In (3), u^k(t)\hat{u}_k(t) and bk(t)b_k(t) are complex variables while γk(t)\gamma_k(t) and ωk(t)\omega_k(t) are real-valued. The function fk(t)f_k(t) is a known time-periodic function that often represents seasonal effects. The other parameters σku\sigma_k^u, dkγd_k^\gamma, γˉk\bar{\gamma}_k, σkγ\sigma_k^\gamma, dkωd_k^\omega, ωˉk\bar{\omega}_k, σkω\sigma_k^\omega, dkbd_k^b, bˉk\bar{b}_k, σkb\sigma_k^b are all constants, where the three damping coefficients dkγd_k^\gamma, dkωd_k^\omega and dkbd_k^b are positive. In addition, W˙ku\dot{W}_k^u, W˙kγ\dot{W}_k^\gamma, W˙kω\dot{W}_k^\omega and W˙kb\dot{W}_k^b are independent white noises.

It is important to note that if the damping γk\gamma_k, the frequency ωk\omega_k and the additional forcing bkb_k in (3a) are all constants, then u^k\hat{u}_k becomes an Ornstein–Uhlenbeck (OU) process [25], known as the mean stochastic model (MSM), which has linear dynamics and Gaussian statistics. However, intermittency and non-Gaussian features with extreme events and fat tails are often observed in complex turbulent systems. Therefore, γk\gamma_k, ωk\omega_k and bkb_k are parameterized by stochastic processes in (3), which facilitate extra temporal variabilities in uku_k that lead to rich nonlinear dynamical features and non-Gaussian statistics. Specifically, the alternating between positive and negative values in γk\gamma_k corresponds to the damping and the anti-damping of u^k\hat{u}_k, where the latter is crucial in triggering intermittency, extreme events and non-Gaussian distributions. Similarly, the random evolution of the phase ωk\omega_k advances u^k\hat{u}_k to have a wide range of the spectrum that is consistent with the typical characteristics of turbulent signals. Since the primary role of γk\gamma_k, ωk\omega_k and bkb_k is to trigger intermittency and turbulent features of u^k\hat{u}_k, simple structures of these stochastic parameterizations are preferred that allow an efficient numerical integration. Thus, the OU processes are adopted to parameterize the time evolution of these three variables. Note that the interactions between γk\gamma_k and u^k\hat{u}_k as well as ωk\omega_k and u^k\hat{u}_k are both quadratic nonlinear. Therefore the full SPEKF system (3) is nonlinear and u^k\hat{u}_k can have non-Gaussian statistics. Finally, it is worthwhile to remark that despite the nonlinearity in (3), the time evolution of the moments can all be written down using closed analytic formulae, which provide an efficient and accurate statistical forecast of the SPEKF system (3) in data assimilation. Nevertheless, since the SPEKF model (3) only involves four state variables, a direct Monte Carlo simulation with a moderate ensemble size can also be utilized for the forecast, which will not significantly increase the computational cost.

It has been shown in [7] that the SPEKF can compete with or outperform an optimally tuned 3DVAR algorithm in filtering turbulent signals, and it can overcome competing sources of error in a range of dynamical scenarios. It has also been shown that the SPEKF model has much higher skill than the classical Kalman filter using the MSM in filtering and predicting signals with intermittency and extreme events [49, 48, 14, 6, 5].

In the following, the SPEKF model (3) is adopted as the forecast model in the EnKF to filter each spectral mode of the observed state variables, where in total JJ ensemble members are generated, denoted by {u^k(j)(t),j=1,…,J}\{\hat{u}_k^{(j)}(t), j=1,\ldots,J\}. These ensemble members will also be utilized to create a set of inputs for the machine learning to seek the ensemble of the unobserved state variables.

Estimating the Parameters in the SPEKF Model

Determining the model parameters is a prerequisite for applying the SPEKF model as the forecast model in data assimilation. As the large-scale f(t)f(t) is often prescribed and σku\sigma_k^u can be estimated from computing the quadratic variation, the focus here is mainly on estimating the 9 parameters in the three stochastically parameterized processes (5)–(7).

In light of the special structure of the SPEKF model (3), a simple iterative algorithm is developed to advance the estimation of the parameters in (5)–(7). Given a time series of u^k\hat{u}_k and the initial guess of the parameters in (3), an efficient conditional sampling approach with a closed analytic formula can be applied to obtain a time series of γk\gamma_k, ωk\omega_k and bkb_k (see the Appendix for details). Next, the mean, variance, and decorrelation time (integration of the autocorrelation function) of γk\gamma_k are computed from this time series, which are denoted by mkγm_k^\gamma, EkγE_k^\gamma and τkγ\tau_k^\gamma, respectively (similarly for those associated with ωk\omega_k and bkb_k). The three parameters in the OU process, i.e., dkγd_k^\gamma, γˉk\bar{\gamma}_k and σkγ\sigma_k^\gamma, are uniquely determined by these three statistics,

dkγ=1/τkγ,γˉk=mkγ,σkγ=2Ekγ/τkγ.(8)d_k^\gamma= 1/\tau_k^\gamma,\qquad\bar{\gamma}_k = m_k^\gamma,\qquad\sigma_k^\gamma= \sqrt{2E_k^\gamma/\tau_k^\gamma}. \tag*{(8)}

The updated parameters from (4) are then combined with the conditional sampling method to improve the sampled trajectory. Repeating such a procedure a few times leads to the convergence of the estimated parameters.

Filtering Sparse Regularly Spaced Observations with Aliasing

Aliasing comes from the nature of sparse observations in space. In the presence of sparse observations, the noisy observation after transferring to spectral space is given by

v^k=gu^k+g∑k′≠k,k′∈Aku^k′+σko(9)\hat{v}_k = g\hat{u}_k + g\sum_{\substack{k'\ne k, k'\in A_k}}\hat{u}_{k'}+\sigma_k^o \tag*{(9)}

where AkA_k is the index set involving the spectral modes that belong to the same aliasing set as mode kk, σko\sigma_k^o is the observational noise projected to spectral space, and gg is the observational operator. There are many ways of dealing with aliasing, which can be chosen depending on the properties of the underlying dynamics. See [49] for a summary of these strategies.

The specific strategy utilized to filter the PQG model is the so-called reduced Fourier domain Kalman filter (RFDKF) [30]. The RFDKF approximation is based on the intuitive idea that for sufficiently rapid decay in the spectrum of the true turbulent signal, the primary mode contains the most energy, so only this mode should be actively filtered. Thus, RFDKF always trusts the dynamics for all the aliased modes yielding a Kalman gain vector with only the first component (corresponding to the primary mode) being non-zero. Utilizing the RFDKF, the second term on the right-hand side of (9) is replaced by the known equilibrium mean of the aliased modes in the analysis step. Thus, a simple relationship between the observation v^k\hat{v}_k and the primary mode u^k\hat{u}_k appears.

Filtering the Unobserved Variables Using a Machine Learning Model

One of the main difficulties in filtering complex systems with only partial observations is the state estimation of the unobserved variables. Unlike the observed variables, where observations play an important role in directly mitigating the model error in the forecast, the observational information has only an indirect impact on rectifying the forecast error of the unobserved variables. Therefore, the skill of filtering these states heavily relies on the accuracy of the forecast model in characterizing the complex nonlinear dependence between the observed and unobserved variables.

Discovering the Nonlinear Dependence Between Observed and Unobserved Variables Using Machine Learning

To advance the characterization of such a dependence without involving complicated parametric approximate models, machine learning is adopted to develop an effective surrogate model. Given the filtered ensembles of the observed variables for each spectral mode resulting from the EnKF, machine learning aims to find the corresponding posterior ensembles of the unobserved state variables.

The machine learning model can be regarded as a generalization of the linear analysis of linear partial differential equations (PDEs) with at least two sets of state variables u(x,t)\mathbf{u}(\mathbf{x},t) and h(x,t)\mathbf{h}(\mathbf{x},t), where the full solution can be written as a superposition of different eigenmodes,

\begin{pmatrix} \mathbf{u}(\mathbf{x},t) \\ \mathbf{h}(\mathbf{x},t) \end{pmatrix} = \sum_{\mathbf{k}\in\mathcal{K},\alpha\in\mathcal{C}} \hat{v}_{\mathbf{k},\alpha}(t)\exp(\mathrm{i}\mathbf{k}\mathbf{x})\mathbf{r}_{\mathbf{k},\alpha}. \tag*{(10)}\end{pmatrix}

In (6), C\mathcal{C} is the set that contains the indices of different eigenmodes, the Fourier coefficient v^k(t)\hat{v}_{\mathbf{k}}(t) is a scalar, and different state variables are linked via the eigenvectors rk,α=(rk,αu,rk,αh)T\mathbf{r}_{\mathbf{k},\alpha}=(\mathbf{r}^{\mathbf{u}}_{\mathbf{k},\alpha},\mathbf{r}^{\mathbf{h}}_{\mathbf{k},\alpha})^{\mathrm{T}}. For a fixed wavevector, the spectral coefficients of the two state variables are given by the following linear relationship

\begin{pmatrix} \hat{\mathbf{u}}_{\mathbf{k}}(t) \\ \hat{\mathbf{h}}_{\mathbf{k}}(t) \end{pmatrix} = \sum_{\alpha\in\mathcal{C}}\hat{v}_{\mathbf{k},\alpha}(t)\mathbf{r}_{\mathbf{k},\alpha}. \tag*{(11)}\end{pmatrix}

Therefore, h^k(t)\hat{\mathbf{h}}_{\mathbf{k}}(t) is uniquely determined by u^k(t)\hat{\mathbf{u}}_{\mathbf{k}}(t) and rk,α\mathbf{r}_{\mathbf{k},\alpha}.

For complex nonlinear dynamical systems, the above simple relationship is generally not held. Nevertheless, machine learning becomes a natural choice to build a nonlinear relationship between u^k(t)\hat{\mathbf{u}}_{\mathbf{k}}(t) and h^k(t)\hat{\mathbf{h}}_{\mathbf{k}}(t) for each fixed k\mathbf{k}. Then, in light of the filtered solution u^k(t)\hat{\mathbf{u}}_{\mathbf{k}}(t) from the EnKF, the machine learning model provides an ensemble of h^k(t)\hat{\mathbf{h}}_{\mathbf{k}}(t).

Below, a recurrent neural network (RNN) [19] is adopted as the machine learning model to build the link between u^k\hat{\mathbf{u}}_{\mathbf{k}} and h^k\hat{\mathbf{h}}_{\mathbf{k}}:

h^k(j)(t)=RNN⁡(u^k(j)(t−m:t)),(12)\hat{\mathbf{h}}_{\mathbf{k}}^{(j)}(t)=\operatorname{RNN}\left(\hat{\mathbf{u}}_{\mathbf{k}}^{(j)}(t-m:t)\right), \tag*{(12)}

where the input is one ensemble member of u^k(t−m:t)\hat{\mathbf{u}}_{\mathbf{k}}(t-m:t) and the output is that of h^k(t)\hat{\mathbf{h}}_{\mathbf{k}}(t). Running the machine learning model (8) JJ times, an ensemble of the unobserved state variable h^k(t)\hat{\mathbf{h}}_{\mathbf{k}}(t) can be created, which is denoted by {h^k(j)(t),j=1,…,J}\{\hat{\mathbf{h}}_{\mathbf{k}}^{(j)}(t),j=1,\ldots,J\}.

It is essential to highlight that a time series of the observed variable u^k\hat{\mathbf{u}}_{\mathbf{k}} from t−mt-m to tt is utilized to find the corresponding value of the unobserved state variable h^k\hat{\mathbf{h}}_{\mathbf{k}} at time tt. The length of the input can be chosen as one decorrelation time of the true signal. Such a memory effect in the input of the neural network allows us to take into account the dynamics of the observed variable in recovering the unobserved state variable, which is a crucial part that involves more information from physics.

Creating the Input in Machine Learning Model

The RNN (8) aims to capture the nonlinear dependence between u^k\hat{\mathbf{u}}_{\mathbf{k}} and h^k\hat{\mathbf{h}}_{\mathbf{k}}. Therefore, during the training period, the trajectories of the mode k\mathbf{k} from the underlying dynamics (1) are naturally utilized as the input and the output of the RNN. As the input in the RNN (8) requires a trajectory of the observed state variable u^k\hat{\mathbf{u}}_{\mathbf{k}} up to the current time instant, the ensemble members of u^k\hat{\mathbf{u}}_{\mathbf{k}} resulting from the EnKF using the SPEKF can be used as the input during the testing period. Yet, since filtering utilizes only the information from the past, such input may not fully capture the underlying dynamics. To create ensembles that are more dynamically consistent with the truth, an ensemble Kalman smoother (EnKS) [23] is adopted to correct the error in the ensemble members from the EnKF. The resulting ensemble members are then fed into the RNN (8).

Incorporating the Machine Learning Training Residual into the Uncertainty of the Posterior Distribution

Due to the turbulent nature and the intrinsic unpredictable components, the training residual from the machine learning model needs to be taken into account as an additional source of the uncertainty in the state estimation of the unobserved variable h^k(t)\hat{\mathbf{h}}_k(t). The deterministic forecast from the RNN can be regarded as an analog of the mean part of the solution associated with a knowledge-based stochastic model in the context of the classical mean-fluctuation decomposition [54]. In contrast, the training residual mimics the fluctuation that the RNN cannot fully characterize. A separate validation period is included in the training to prevent the over-fitting of the RNN, which will bring about the underestimation of the uncertainty in the machine learning model. An appropriate RNN leads to a comparable level of the residual in such an independent validation period as in the training period for calibrating the RNN.

To compute this additional uncertainty, each ensemble member of the unobserved state variable h^k(j)(t)\hat{\mathbf{h}}_k^{(j)}(t) from (8) is modified by adding a non-Gaussian distribution ϵ\epsilon to the point-wise value,

p(h^k(j)(t))=h^k(j)(t)+ϵ,(13)p\left(\hat{\mathbf{h}}_k^{(j)}(t)\right)=\hat{\mathbf{h}}_k^{(j)}(t)+\epsilon, \tag*{(13)}

where ϵ\epsilon is the distribution of residual in the machine learning model (8) during the training period. One simple way to obtain the uncertainty is to compute

ϵ∼PDF⁡(h^k(t′)−RNN⁡(u^k(t′−m:t′))),for m<t≤T,(14)\epsilon\sim\operatorname{PDF}\left(\hat{\mathbf{h}}_k(t')-\operatorname{RNN}\left(\hat{\mathbf{u}}_k(t'-m:t')\right)\right),\quad\text{for }m<t\leq T, \tag*{(14)}

where TT is the total length of the training period [12]. The posterior state estimate of the unobserved variable h^k(t)\hat{\mathbf{h}}_k(t) is then represented by a non-Gaussian distribution. It is a mixture distribution, where each mixture component is another non-Gaussian distribution associated with one ensemble member computed from (9)–(10). Such uncertainty quantification is utilized in the numerical tests below. Note that (10) assumes that the residual ϵ\epsilon is a constant over time, which is a crude but simple approximation. A more refined approach to determine the uncertainty is to let ϵ\epsilon depend on the input u^k(t′−m:t′)\hat{\mathbf{u}}_k(t'-m:t'). This can be achieved by first partitioning the input u^k(t′−m:t′)\hat{\mathbf{u}}_k(t'-m:t') into several clusters in the training period and then computing the distribution of h^k(t′)−RNN⁡(u^k(t′−m:t′))\hat{\mathbf{h}}_k(t')-\operatorname{RNN}(\hat{\mathbf{u}}_k(t'-m:t')) within each cluster to form a specific ϵ\epsilon. In the testing period, find the cluster that the input belongs to and then add the corresponding value of ϵ\epsilon to h^k(j)(t)\hat{\mathbf{h}}_k^{(j)}(t).

As a final remark, the residual or the uncertainty associated with the RNN depends on the skill of the RNN in discovering the nonlinear relationship between u^k\hat{\mathbf{u}}_k and h^k\hat{\mathbf{h}}_k. A more skillful RNN results in a smaller uncertainty. If the RNN can perfectly recover the underlying dynamics, then such uncertainty vanishes.

Precipitating Quasigeostrophic (PQG) Equations

The model

The PQG model is a recently developed moist version of the quasigeostrophic (QG) model [65, 21, 37] to describe synoptic-scale (≈1000\approx1000 km) dynamics at mid-latitudes. Beyond the classical dry QG dynamics, PQG includes additional physics and dynamics associated with water vapor, clouds, phase changes between vapor and liquid, and rainfall. Thus, the PQG model can more realistically describe large-scale, mid-latitude weather patterns.

In the dry QG model, there is a single prognostic variable called the potential vorticity (PVPV) [61, 59, 70]. All other variables in the systems, namely the horizontal winds and potential temperature, can be diagnostically recovered from PVPV by inverting a linear elliptic operator in so-called PVPV-inversion. With the addition of water, the PQG model requires an additional prognostic variable MM, and all other variables (horizontal winds, potential temperature and water) are found from PVPV-and-MM inversion of a nonlinear elliptic operator. The nonlinear terms in the elliptic operator arise from the presence of phase interfaces separating unsaturated and saturated regions of the flow. These phase boundaries lead to discontinuous coefficients in the elliptic operator, denoted by Heaviside nonlinear functions.

A two-level version of the PQG equations can be derived using a staggered grid in the vertical direction [37], as illustrated in Figure 2. The subscript (⋅)j(\cdot)_j, j=1,2j=1,2 indicates the variables defined at level 1 or level 2, and the subscript (⋅)m(\cdot)_m indicates those at the middle level. The governing equations are given by

Illustration of the two-level setup of the PQG system

Figure 2. Illustration of the two-level setup of the PQG system.

∂PV1∂t+J(ψ1,PV1)−U∂PV1∂x+βv1+v1∂PV1,bg∂y=−LduLdsLLds∂uh∂z⋅∇hθe,1−κΔhψ1−νΔh4PV1,(15)\frac{\partial PV_1}{\partial t}+J(\psi_1,PV_1)-U\frac{\partial PV_1}{\partial x}+\beta v_1+v_1\frac{\partial PV_{1,bg}}{\partial y}=-\frac{L_{du}}{L_{ds}}\frac{L}{L_{ds}}\frac{\partial u_h}{\partial z}\cdot\nabla_h\theta_{e,1}-\kappa\Delta_h\psi_1-\nu\Delta_h^4PV_1, \tag*{(15)}
∂PV2∂t+J(ψ2,PV2)+U∂PV2∂x+βv2+v2∂PV2,bg∂y=−LduLdsLLds∂uh∂z⋅∇hθe,2−νΔh4PV2,(16)\frac{\partial PV_2}{\partial t}+J(\psi_2,PV_2)+U\frac{\partial PV_2}{\partial x}+\beta v_2+v_2\frac{\partial PV_{2,bg}}{\partial y}=-\frac{L_{du}}{L_{ds}}\frac{L}{L_{ds}}\frac{\partial u_h}{\partial z}\cdot\nabla_h\theta_{e,2}-\nu\Delta_h^4PV_2, \tag*{(16)}
DmMmDt+vm∂Mbg∂y=−VrΔzqr,m−νΔh4Mm+E,(17)\frac{D_mM_m}{Dt}+v_m\frac{\partial M_{bg}}{\partial y}=-\frac{V_r}{\Delta z}q_{r,m}-\nu\Delta_h^4M_m+E, \tag*{(17)}

where PV1,2PV_{1,2} and MmM_m are related to a streamfunction ψ1,2\psi_{1,2} by nonlinear, elliptic operators (see (23) and (24) below). From the streamfunction ψj\psi_j, the horizontal winds uh,j=(uj,vj)\mathbf{u}_{h,j}=(u_j,v_j) are obtained from the relations uj=−∂ψj/∂yu_j=-\partial\psi_j/\partial y, vj=∂ψj/∂xv_j=\partial\psi_j/\partial x with mid-level values um=(u1+u2)/Δzu_m=(u_1+u_2)/\Delta z, vm=(v1+v2)/Δzv_m=(v_1+v_2)/\Delta z, where ∇h\nabla_h is the horizontal part of the gradient operator and Δz\Delta z is the distance between levels 1 and 2. The equivalent potential temperature θe\theta_e (a linear combination of potential temperature and water vapor) is found from the streamfunction ψj\psi_j and MmM_m using the relation

θe,m=Hs(LLduψ2−ψ1Δz+qvs,m)+Hu(11+GMLLduψ2−ψ1Δz+11+GMMm),(18)\theta_{e,m}=H_s\left(\frac{L}{L_{du}}\frac{\psi_2-\psi_1}{\Delta z}+q_{vs,m}\right)+H_u\left(\frac{1}{1+G_M}\frac{L}{L_{du}}\frac{\psi_2-\psi_1}{\Delta z}+\frac{1}{1+G_M}M_m\right), \tag*{(18)}

where GmG_m is an O(1)O(1) parameter related to the background profiles, LL is the reference length scale (1000 km), and LduL_{du} is the Rossby radius of deformation associated with the unsaturated background state. The total water mixing ratio is qt,m=qv,m+qr,mq_{t,m}=q_{v,m}+q_{r,m}, where qv,m,qr,mq_{v,m},q_{r,m} are the vapor and liquid components, respectively. In terms of MmM_m and θe,m\theta_{e,m}, qt,m=Mm−GMθe,mq_{t,m}=M_m-G_M\theta_{e,m}. A saturation profile qvs,mq_{vs,m} separates unsaturated regions (water vapor only) from saturated regions (vapor and rain water). In the current two-level model, qvs,mq_{vs,m} may be written in terms of the streamfunction as

qvs,m=qvs0+qvs1ψ2−ψ1Δz,(19)q_{vs,m}=q_{vs}^{0}+q_{vs}^{1}\frac{\psi_{2}-\psi_{1}}{\Delta z}, \tag*{(19)}

where the parameters qvs0q_{vs}^{0} and qvs1q_{vs}^{1} are non-negative constants. The absence or presence of rain qr,mq_{r,m} is diagnosed from MmM_m, θe,m\theta_{e,m} and qvs,mq_{vs,m} according to the relation qr,m=max⁡(0,M−GMθe,m−qvs,m)q_{r,m}=\max(0,M-G_M\theta_{e,m}-q_{vs,m}). The cloud indicator (Heaviside) functions are then

Hs={1if qr>0,0if qr=0,Hu={0if qr>0,1if qr=0.(20)H_s= \begin{cases} 1 & \text{if } q_r>0,\\ 0 & \text{if } q_r=0, \end{cases} \qquad H_u= \begin{cases} 0 & \text{if } q_r>0,\\ 1 & \text{if } q_r=0. \end{cases} \tag*{(20)}

At the top and bottom, PV1,2PV_{1,2}, MmM_m and all associated unknowns obey periodic boundary conditions in the horizontal directions (x,y)(x,y). The top and bottom boundary conditions are derived from a rigid-lid condition (no flow through the boundaries), material invariance of θe\theta_e, and the governing transport equation for total water qt,mq_{t,m} including a rainfall term. A simplified version replaces the latter two conditions with qt,m=θe,m=0q_{t,m}=\theta_{e,m}=0.

As alluded to above, the model assumes a prescribed background state with characteristic length-scale L=1000L=1000 km. This environmental state is partly described by the parameter GmG_m, which is the ratio of vertical variations in potential temperature and total water profiles (both linear). In addition, β\beta characterizes the meridional variation of the Coriolis parameter in a narrow mid-latitude band, UU is the vertical shear of flow in the zonal direction, and the length scales Ldu,LdsL_{du},L_{ds} are the Rossby deformation radii associated with unsaturated and saturated flow regions, respectively. Given parameters U,Gm,L,LduU,G_m,L,L_{du} and qvs1q_{vs}^{1}, the background profiles PVj,bgPV_{j,bg} and Mm,bgM_{m,bg} are

PVj,bg=(−1)j(1+qvs1)1(Δz)2L2Ldu2(2Uy),(21)PV_{j,bg}=(-1)^j(1+q_{vs}^{1})\frac{1}{(\Delta z)^2}\frac{L^2}{L_{du}^2}(2Uy), \tag*{(21)}
Mbg=−(qvs1+GM(1+qvs1))1ΔzLLdu(2Uy),(22)M_{bg}=-(q_{vs}^{1}+G_M(1+q_{vs}^{1}))\frac{1}{\Delta z}\frac{L}{L_{du}}(2Uy), \tag*{(22)}

where qvs1q_{vs}^{1} appears in (19) as the threshold for phase changes. The remaining (constant) parameters in (11)-(13) are the evaporation source term EE which regulates the cloud fraction in this setup, as well as dissipation terms parameterized by κ\kappa (friction at the bottom level) and ν\nu (dissipation of small-scale turbulence).

To close the PQG system, it is necessary to perform PVPV-and-MM inversion to find the streamfunction ψ\psi and associated quantities uh\mathbf{u}_h, θe\theta_e, qtq_t, etc. In the two-level setup, the nonlinear elliptic equations relating PVPV and MM to the streamfunction ψ\psi are given by

PV1=∇h2ψ1+Hs((LLds1Δz)2(ψ2−ψ1)+LduLdsLLds1Δzqvs,m)+Hu((LLdu1Δz)2(ψ2−ψ1)+LLdu1ΔzMm).(23)\begin{aligned} PV_1={}&\nabla_h^2\psi_1+H_s\left(\left(\frac{L}{L_{ds}}\frac{1}{\Delta z}\right)^2(\psi_2-\psi_1)+\frac{L_{du}}{L_{ds}}\frac{L}{L_{ds}}\frac{1}{\Delta z}q_{vs,m}\right)\\ &+H_u\left(\left(\frac{L}{L_{du}}\frac{1}{\Delta z}\right)^2(\psi_2-\psi_1)+\frac{L}{L_{du}}\frac{1}{\Delta z}M_m\right). \tag*{(23)} \end{aligned}
PV2=∇h2ψ2+Hs((LLds1Δz)2(ψ1−ψ2)−LduLdsLLds1Δzqvs,m)+Hu((LLdu1Δz)2(ψ1−ψ2)−LLdu1ΔzMm).(24)\begin{aligned} PV_2={}&\nabla_h^2\psi_2+H_s\left(\left(\frac{L}{L_{ds}}\frac{1}{\Delta z}\right)^2(\psi_1-\psi_2)-\frac{L_{du}}{L_{ds}}\frac{L}{L_{ds}}\frac{1}{\Delta z}q_{vs,m}\right)\\ &+H_u\left(\left(\frac{L}{L_{du}}\frac{1}{\Delta z}\right)^2(\psi_1-\psi_2)-\frac{L}{L_{du}}\frac{1}{\Delta z}M_m\right). \tag*{(24)} \end{aligned}

As discussed in § 3.3, PVPV-and-MM inversion is the most expensive part of PQG numerical solution, and thus a strong motivation for exploring inexpensive and efficient surrogate models as described herein.

Model properties

The PQG model was derived as a distinguished asymptotic limit of the rotating Boussinesq equations with idealized cloud microphysics [65]. As such, the model retains the limiting (small-Rossby number) dynamics of a moist atmosphere under the conditions of strong moist stratification and rapid rotation. Energetics of the continuous PQG model and its two-level version have been analyzed in [65, 37]. In particular, the moist potential energy may be decomposed into three parts representing the familiar unsaturated and saturated potential energies, as well as a moist latent energy that is released upon change of phase.

Initial studies of the two-level PQG system have tested the model’s ability to capture canonical large-scale phenomena associated with important weather events. In [20], purely saturated computations exhibited the formation of atmospheric rivers (ARs). In nature, ARs are long narrow corridors, typically associated with extratropical cyclones, and accounting for most of the poleward transport of water from the tropics to the mid-latitudes. Standard AR-identification algorithms found ARs in the saturated PQG two-level system in the presence of sufficiently strong meridional moisture gradient and precipitation. Variability of the mid-latitude zonal jet was studied in [37], including phase changes of water with small-to-moderate cloud fractions in the range [0,0.2]. Different behaviors were observed depending on cloud fraction, including poleward propagation of the latitude of the jet.

Numerical solver of the PQG equations

Here we give a brief overview of the numerical methods used to solve the system (11)–(20), following [37]. The evolution equations (3.1)–(3.3) are discretized using a pseudospectral method. A third-order Runge Kutta scheme is used for the temporal evolution with time step Δt=0.01\Delta t = 0.01 satisfying the CFL stability criterion. During this portion of the algorithm, computation of the nonlinear terms is the most expensive component, and is achieved using FFTs.

The PVPV-and-MM inversion (19)–(20) is treated in physical space. In particular, a standard centered difference method is coupled with the conjugate gradient method to iteratively solve for the updated streamfunction ψ\psi and updated cloud indicator function HsH_s, the latter which depends on the updated streamfunction itself. The iterative procedure starts with HsH_s evaluated at the previous time step. Given updated PV1,2PV_{1,2}, MM and previous HsH_s, the discretized nonlinear elliptic equations (19)–(20) are inverted to find ψ\psi using conjugate gradient. Then (14)–(15), the relation qr=max⁡(0,M−GMθe,m−qvs,m)q_r = \max(0, M - G_M\theta_{e,m} - q_{vs,m}) and (20) are used to check consistency of the previous and updated HsH_s. If they are not the same, the inversion is repeated starting from PV1,2PV_{1,2}, MM and the updated HsH_s. The procedure is iterated until convergence of HsH_s. This iterative algorithm is the most computationally expensive element of the solution algorithm as a whole, which becomes prohibitive for forecast modeling.

Data Assimilation of PQG with Partial Observations

Setup

Model setup

We consider a standard mid-latitude setting with parameter values that are consistent with previous two-level computations in dry and moist settings [59, 37] (see Table 1). All parameters are fixed except for the evaporation rate EE in (17), which acts as a source of moisture. The value of EE determines a statistical equilibrium in which the evaporation source EE balances the moisture sink from rainfall, on average. Thus, a larger value of EE leads to larger cloud fraction and more rainfall. In the following data assimilation tests, the main focus will be on the case E=0.2E = 0.2, corresponding to 17% cloud fraction on average over the entire domain (see § 4.3). This moderate cloud fraction is consistent with the realistic mid-latitude atmosphere. The data assimilation skill will also be tested on experiments with more extreme cases: E=0.02E = 0.02 and E=0.35E = 0.35 (see § 4.4).

ParametersNNβ\betaGMG_Mqvs0q_{vs}^{0}qvs1q_{vs}^{1}LLLdsL_{ds}LduL_{du}Δz\Delta zVmV_mκ\kappaUU
Values1282.51.01.01.01.01/21/\sqrt{2}1/21/20.51.00.050.1

Table 1. Parameters in the PQG equations.

Data assimilation method

The ensemble transform Kalman filter (ETKF) [4] is adopted as the EnKF method for filtering the observed component of each of the spectral modes (i.e., the time series of the Fourier coefficient), where the SPEKF model is utilized as the forecast model. The data assimilation is carried out in Fourier space.

Partial and sparse observations

The lower layer PV, namely PV1PV_1, is chosen to be the observed state variable. In contrast, there are no direct observations for PV2PV_2, MM, and other state variables. Although the model resolution in the horizontal direction is 128×128128 \times128, only sparse observations at 16×1616 \times16 regularly spaced grid points are available in the following data assimilation tests, mimicking realistic situations of atmospheric observations, usually on coarse-grained mesh grids. The reduced Fourier domain Kalman filter (RFDKF) is utilized to deal with the aliasing issue [30] inside the ETKF, as the energy decays sufficiently fast outside the range of the resolved modes (see Panel (a) of Figure 3). The primary goal is to recover the leading 16×1616 \times16 Fourier modes via data assimilation.

Ensemble size, observational time step and observational noise

The size of the ensemble number is 50, the observational time step is chosen to be Δtobs=0.3\Delta t^{obs} = 0.3, and the observational noise is 25%. Figure 3 shows the spectrum and the decorrelation time of the observed variable, PV1PV_1, which are shown as a function of the absolute value of the wavenumber ∣k∣=k12+k22|k| = \sqrt{k_1^2 + k_2^2}. Note that the observational time step Δtobs=0.3\Delta t^{obs} = 0.3 is smaller than the decorrelation time of modes with ∣k∣≤3|k| \le3 while the decorrelation times of remaining modes in the resolved spatial scale are between 0.3 and 0.1. The data assimilation results with different observational time steps and observational noise will be included in Section 4.4.

Figure 3. Left panel: spectrum of PV1PV_1; right panel: decorrelation time of PV1PV_1. The energy is computed by taking the average over the modes with the same ∣k∣|k|, which is the variable for the x-axis.

Neural network setup

The long short-term memory (LSTM) neural network [34] is adopted here as the RNN. To train the LSTM for each Fourier mode (namely the time series of the Fourier coefficients), 400 time unit data (4×1044 \times10^4 data points) is adopted in the training stage, and 100 time unit data (10410^4 data points) is utilized in the validation stage to prevent overfitting, which retains the balance between the accuracy and computational cost. An empirical calibration is used here to systematically determine the LSTM’s hidden units of each Fourier mode (ki,kj)(k_i, k_j). The number of hidden units = (τi,jdecor/Δt)⋅(∣ki∣+∣kj∣)(\tau_{i,j}^{decor}/\Delta t) \cdot(|k_i| + |k_j|), where τi,jdecor\tau_{i,j}^{decor} is the decorrelation time of the Fourier mode (ki,kj)(k_i,k_j) and Δt\Delta t is step size of PQG simulation. This criterion is motivated by the following two facts. First, the Fourier mode with a larger wave number, i.e., larger ki,kjk_i,k_j values, usually yields more intermittency. Second, a smaller decorrelation time τi,jdecor\tau^{decor}_{i,j} implies a shorter memory of the time series [42].

Comparison with data assimilation using a traditional reduced order forecast model

The hybrid data assimilation algorithm is compared with a traditional reduced-order model-based data assimilation method (hereafter “traditional ROM”). In this traditional ROM method, a spectral representation of the original PQG system with a lower resolution of 16×1616 \times16 grids is developed. This is achieved by first projecting the system onto such a reduced-order basis, where the higher frequencies are truncated. Then the hyper-viscosity is increased to guarantee that the solution is stabilized on such a coarse-grained mesh. Additional stochastic noise is added to the equation of each spectral mode to match the equilibrium variance with the truth. Furthermore, the complicated PVPV-and-MM inversion is replaced by the linear inversion corresponding to dry QG. This reduces the computational cost and prevents the truncation error in the input from being significantly amplified by the Heaviside functions. This reduced-order model is then incorporated into the localized ETKF (LETKF) [38] in physical space for data assimilation. See the Appendix for more details.

This traditional ROM method is fundamentally different from the hybrid strategy. In the traditional ROM, the governing equation of each spectral mode contains the nonlinear interactions between different Fourier modes. In contrast, the SPEKF model in the hybrid strategy utilizes cheap stochastic parameterization to effectively compensate for these complicated nonlinearities. Furthermore, the traditional ROM uses reduced order parametric form to build the connection between the observed and unobserved state variables while the hybrid strategy exploits machine learning to achieve this goal.

Skill scores in assessing data assimilation skill

The following skill scores are utilized to quantify the data assimilation skill. They are the normalized root-mean-square error (RMSE) and pattern correlation (Corr) between the truth and the posterior mean time series. They are defined as follows:

Corr⁡=∑i=1n(uiM−uˉM)(uiref−uˉref)∑i=1n(uiM−uˉM)2∑i=1n(uiref−uˉref)2,RMSE⁡=1std⁡(uref)∑i=1n(uiM−uiref)2n,(25)\begin{aligned} \operatorname{Corr} &= \frac{\sum_{i=1}^{n}(u_i^M-\bar{u}^M)(u_i^{ref}-\bar{u}^{ref})}{\sqrt{\sum_{i=1}^{n}(u_i^M-\bar{u}^M)^2}\sqrt{\sum_{i=1}^{n}(u_i^{ref}-\bar{u}^{ref})^2}},\\ \operatorname{RMSE} &= \frac{1}{\operatorname{std}(u^{ref})}\sqrt{\frac{\sum_{i=1}^{n}(u_i^M-u_i^{ref})^2}{n}}, \tag*{(25)} \end{aligned}

where uiMu_i^M and uirefu_i^{ref} are the posterior mean and the truth, respectively, at time t=tit=t_i, and nn is the total number of the data points to compute these statistics. The time averages of the assimilated and the true time series are denoted by uˉM\bar{u}^M, and uˉref\bar{u}^{ref}, respectively, and std⁡(uref)\operatorname{std}(u^{ref}) is the standard deviation of the true time series urefu^{ref}. The normalization used in the RMSE ensures that the skill score is a non-dimensional quantity. A smaller RMSE and a larger pattern correlation imply a more accurate solution. The filtering result is considered unskillful if the normalized RMSE exceeds 1 or the pattern correlation is below a certain threshold, usually 0.5.

In addition to the time series, the pattern correlation (Corr) between the truth and assimilated spatial fields can be calculated at a fixed time. It is defined as

Corr⁡=∑i=1nx∑j=1ny(ukM(xi,yj)−uˉM)(ukref(xi,yj)−uˉref)∑i=1nx∑j=1ny(ukM(xi,yj)−uˉkM)2∑i=1nx∑j=1ny(ukref(xi,yj)−uˉkref)2(26)\operatorname{Corr} = \frac{\sum_{i=1}^{n_x}\sum_{j=1}^{n_y}(u_k^M(x_i,y_j)-\bar{u}^M)(u_k^{ref}(x_i,y_j)-\bar{u}^{ref})}{\sqrt{\sum_{i=1}^{n_x}\sum_{j=1}^{n_y}(u_k^M(x_i,y_j)-\bar{u}_k^M)^2}\sqrt{\sum_{i=1}^{n_x}\sum_{j=1}^{n_y}(u_k^{ref}(x_i,y_j)-\bar{u}_k^{ref})^2}} \tag*{(26)}

where ukM(xi,yj)u_k^M(x_i,y_j) and ukref(xi,yj)u_k^{ref}(x_i,y_j) are the posterior mean and the truth at time t=tkt=t_k, respectively, at spatial grid (xi,yj)(x_i,y_j), and uˉM\bar{u}^M and uˉref\bar{u}^{ref} are the spatial averages of the assimilated data and the truth at time t=tkt=t_k.

Main data assimilation results

Data assimilation skill of different Fourier modes

Figure 4 shows the posterior mean time series resulting from the hybrid data assimilation algorithm (red), which is compared with the truth generated from the PQG model (11)–(13) (blue). For the largest scale mode (0,1)(0, 1), the posterior mean time series captures the quasi-regular oscillation patterns for both the observed and unobserved state variables. In addition, the posterior uncertainty is quite small, indicating the confidence of the posterior mean estimate. With the increase of ∣k∣|k| (corresponding to smaller scales), the data assimilation skill deteriorates, as expected. Nevertheless, although the decorrelation time of mode (3,4)(3, 4) approaches the observational time step, the recovery of the observed state variable PV1PV_1 remains quite skillful. This is because observations play a direct role in correcting the forecast error. It is worth highlighting that most of the extreme events are captured accurately by the data assimilation solution since the SPEKF forecast model has the mechanism for predicting intermittency with the help of stochastic damping. The SPEKF forecast model is thus more skillful in providing an accurate prior distribution than a simple linear stochastic model (namely, the mean stochastic model) [7]. On the other hand, although the recovered unobserved state variables PV2PV_2 and MM for mode (3,4)(3, 4) are not as accurate as their observed counterpart, the posterior mean time series captures the overall tendency of the truth. Note that the posterior uncertainty of PV2PV_2 and MM for mode (3,4)(3, 4) is still much smaller than the equilibrium variance, which indicates the benefits of state estimation from data assimilation. Finally, if the machine learning training residual is not included, then the posterior uncertainty comes from only the spread of the posterior ensemble members. As is seen from the right column of Figure 4, the 1st and the 99th quantile of the ensemble members do not reach the tail of the posterior PDF, which means the posterior uncertainty will be underestimated without considering the training residual. Such a result validates that the mixture distribution advances the uncertainty quantification of the posterior distribution. Nevertheless, the training residual is not significant, which indicates the skill of machine learning in finding the posterior ensemble members of the unobserved state variables.

(a) Fourier mode (0, 1)

Time series and mixture posterior distributions for Fourier mode (3, 4)

(b) Fourier mode (3, 4)

Figure 4. Time series and mixture posterior distribution of observed variable PV1PV_1 and unobserved variables PV2PV_2 and moisture MM at Fourier modes (0, 1) and (3, 4) with observation time Δtobs=0.3\Delta t^{obs}=0.3 and 25% observation noise. The observations (green dots) are the projected values from physical space to each Fourier mode. Left panel: times series of truth (blue), filter (red), of PV1PV_1, PV2PV_2 and moisture MM, and observation (green) for PV1PV_1; right panel: equilibrium distribution and uncertainty distribution at t=141t=141 for each variable, the green circle denotes the truth value and the yellow crosses denote 1st, 16th, 50th, 84th, and 99th percentile of the point-wise ensemble realization values.

Data assimilation skill of the spatial reconstructed fields for the prognostic variables

Figure 5 shows the reconstructed observed variable PV1PV_{1} and the unobserved variables PV2PV_{2} and MM in physical space. In addition to the hybrid data assimilation strategy, the results from the traditional ROM are also presented for comparison.

Figure 5. Reconstructed spatial fields and pattern correlation skill scores (upper right panel) of observed variable PV1PV_1 (first column) and unobserved variables PV2PV_2 (second column) and moisture MM (third column) at t=63.17t = 63.17, with observation time Δtobs=0.3\Delta t^{obs} = 0.3.

The first column shows the observation, the truth, and the recovered field of PV1PV_{1} using the two methods at a specific time instant t=63.17t = 63.17. The hybrid method almost perfectly reproduces the truth and contains less errors than the observation. The traditional ROM also works reasonably well by accurately recovering the overall patterns with the help of noisy observations. But certain biases in the intermediate scales are found in the results from the traditional ROM. On the other hand, as is shown in the second and the third columns, despite some small-scale errors, the hybrid strategy remains overall skillful in recovering the spatial pattern of the two unobserved prognostic variables. In contrast, the recovered fields using the traditional ROM lead to much larger errors due to the inaccurate coupling relationship between the observed and unobserved variables in the ROM with truncation. Finally, the pattern correlations between the truth and each recovered field using the hybrid strategy at different time instants are shown at the top right corner of Figure 5. The pattern correlation remains robust across time. In particular, the pattern correlation is above 0.8 within the period for the observed variable PV1PV_{1}. The pattern correlation is, on average, 0.65 for the two unobserved variables, namely PV2PV_{2} and MM.

Data assimilation skill of the spatial reconstructed fields for the diagnostic variables

Recall that one of the critical features of the PQG system is the phase change of water and rainfall over the domain, which is characterized by the quantity of rain water qrq_r. In addition, the PQG system models the mid-latitude jet stream with bands of zonal wind related to velocity field uu. Therefore, in addition to the three prognostic variables PV1PV_1, PV2PV_2, and MM, it is crucial to study the accuracy of the recovered fields of these two quantities from the hybrid data assimilation.

Figure 6 shows the rain water qrq_r and zonal wind velocity field u2u_2 at the top level for both the truth and the filtered posterior mean result. Variation of qrq_r from zero to positive values (shades of red) indicated the presence of phase changes, and there are regions of relatively large qr>0q_r > 0 (darker red areas). Here, qrq_r and u2u_2 are calculated by solving the PV-and-M inversion equations (23)–(24), and using the relations above (23)–(24). It is worthwhile highlighting that, due to the sensitive behavior of the output with a small change in the input via the Heaviside nonlinearity in PV-and-M inversion, it is extremely challenging to reach a nearly perfect recovery of qrq_r. In particular, most of the significant features of qrq_r are associated with small scales. Nevertheless, the reconstructed patterns of the qrq_r field based on the data assimilation results reasonably resemble the truth. The pattern correlation is, on average, above the threshold Corr = 0.5 within the entire period, indicating the skillful prediction of the timing and locations of the rainfall. Furthermore, the variability (e.g meandering) of the large-scale, westerly zonal jet is successfully recovered. This is consistent with the pattern correlation skill score in the bottom panel of Figure 6, which is, on average, about 0.8 across time.

Figure 6. Recovered spatial fields and pattern correlation skill scores of rain water qrq_r and zonal velocity at level 2 in the physical space at different time, with Δtobs=0.3\Delta t^{obs}=0.3; the green box in the time series of pattern correlation denotes the time interval of recovered qrq_r and u2u_2 fields.

Figure 7 displays the upper-level zonally averaged wind as a function of time for both the truth and the filter. The reconstructed velocity field successfully captures the location of zonal jets most of the time. Moreover, it recovers the northward propagation of the jet, which was first observed in [37].

Truth and filter, recovered time evolution of the zonally averaged zonal wind

Figure 7. Truth (top) and filter (bottom), recovered time evolution of uˉ2\bar{u}_2, the zonally averaged zonal wind at the top level of atmosphere.

Sensitivity tests

Figure 8 shows skill scores for Fourier modes (0,1)(0, 1), (1,2)(1, 2) and (2,2)(2, 2) with different observational noise levels. For all the modes, increasing the observation noise leads to a larger RMSE and smaller pattern correlation, as expected. For PV1PV_1, due to the availability of direct observations, the data assimilation remains skillful for all these Fourier modes. For the unobserved variables, namely PV2PV_2 and MM, the hybrid data assimilation results get worse from large- to small-scale variables. This is because smaller-scale modes are more intermittent and chaotic with shorter memory, and thus their recovery is more challenging. In addition to the observational noise, the sensitivity test on different observational time steps has also been carried out (not shown here). When the observational time step is reduced from Δtobs=0.3\Delta t^{obs}=0.3 to Δtobs=0.1\Delta t^{obs}=0.1, the recovery of the modes with 8≥∣k∣>38 \ge|k| > 3 is slightly more accurate as the observational time step becomes shorter than the decorrelation time. As a result, the accuracy of the recovered spatial reconstructed fields is slightly enhanced.

Figure 8. Skill scores, RMSE(in blue) and Corr (in red) for different observational noise and fixed observational time Δtobs=0.3\Delta t^{obs}=0.3 with three different Fourier modes.

Two additional sensitivity tests have been implemented. First, if a simple linear regression replaces the RNN for detecting the relationship between the observed and unobserved state variables, then the data assimilation skill for the large-scale modes with ∣k∣≤2|k| \le2 becomes significantly worse. Consequently, the reconstructed spatial fields are more biased even for recovering the large-scale features. Second, suppose the SPEKF model is substituted by a simple linear Gaussian process, namely a complex OU process, as a cheap stochastic parameterization forecast model for the observed variables. In that case, the data assimilation result of PV1PV_1 is less accurate in estimating intermittency and extreme events. This is consistent with the previous results shown in [7], which indicates the importance of adopting the additional stochastic processes. to characterize the non-Gaussian features of these intermittent signals.

Extrapolation

A robust data assimilation method means it remains skillful when carrying out extrapolation within a certain range. In other words, applying the current calibrated forecast model to different dynamical regimes (i.e., other geophysical scenarios) is expected to remain skillful. To this end, the SPEKF model and the LSTM trained in the regime with E=0.2E = 0.2 (the regime studied above) are tested on two other dynamical regimes with E=0.35E = 0.35 and E=0.02E = 0.02, which correspond to 40% and 2% of the cloud fraction, respectively.

Figure 9–Figure 10 show the spatially reconstructed fields of the observed and unobserved variables, PV1PV_1, PV2PV_2 and MM, for the cases of E=0.02E = 0.02 and E=0.35E = 0.35, respectively. In both the extrapolation regimes, the data assimilation skill remains similar to that in Figure 5. The reconstructed observed variable PV1PV_1 field is nearly perfect, and the reconstructed unobserved variables PV2PV_2 and moisture MM accurately capture the large-scale spatial patterns. This is also confirmed from the time series of pattern correlations for PV1PV_1, PV2PV_2 and MM in Figure 9–Figure 10. Among these two cases, the one with E=0.02E = 0.02 has slightly larger errors. Note that E=0.02E = 0.02 leads to almost a dry QG system, which is the extreme case of the PQG and is quite far from the typical features detected in the regime with E=0.2E = 0.2.

Figure 9. Reconstructed spatial fields and pattern correlation skill scores (upper right panel) of observed variable PV1PV_{1} (first column) and unobserved variables PV2PV_{2} (second column) and MM (third column) at t=190t = 190, with observation time Δtobs=0.3\Delta t^{obs} = 0.3; sensitivity test with E=0.35E = 0.35.

Figure 10. Reconstructed spatial fields and pattern correlation skill scores (upper right panel) of observed variable PV1PV_1 (first column) and unobserved variables PV2PV_2 (second column) and MM (third column) at t=172t = 172, with observation time Δtobs=0.3\Delta t^{obs} = 0.3; sensitivity test with E=0.02E = 0.02.

Figure 11 shows the reconstructed qrq_r and u2u_2 in the two extrapolation regimes. In both regimes, the jet structure of u2u_2 is recovered quite accurately in terms of both the strength and the location. The rainwater area qrq_r in the E=0.35E = 0.35 regime is also recovered reasonably well, with an averaged pattern correlation being 0.6. Yet, the detected qrq_r in the E=0.02E = 0.02 regime is far from the truth. This is not surprising as the recovered PV2PV_2 and MM (see Figure 10) are already slightly less accurate. The Heaviside function magnifies such inaccuracies during the reconstruction of qrq_r. In addition, the training regime rarely has time instants that are almost clear skies. Thus, the features in such a case are not well-represented training phases for the SPEKF model and the LSTM network, which causes the deterioration of the data assimilation skill in such an extreme extrapolation regime.

Image not extracted

Figure 11. Recovered spatial fields and pattern correlation skill scores of rain water qrq_r and zonal velocity u2u_2 (level 2) in physical space, at two different times, with Δtobs=0.3\Delta t^{obs}=0.3; the green box in the time series of pattern correlation denotes the time interval of recovered qrq_r and u2u_2 fields; Sensitivity test with E=0.02E=0.02 (left panel) and E=0.35E=0.35 (right panel).

Conclusion and Discussions

In this paper, a new hybrid data assimilation framework is proposed. The new data assimilation combines partial observations with a machine learning model to recover the unobserved variables and quantify the uncertainties. This hybrid data assimilation framework exploits the cheap and low-order but effective SPEKF model as the forecast model in the EnKF for the observed variables. It constructs the RNN machine learning algorithm to recover the unobserved variables with ensembles. In addition to the ensemble estimation, posterior uncertainties are quantified using the mixture distribution with the machine learning residual. Numerical results with the PQG equations show that the proposed data assimilation framework accurately filters observed and unobserved variables and is effective in recovering cloud microphysics quantities and jet streams associated with PQG equations. The calibrated SPEKF and machine learning models are also effective and robust in different dynamical regimes of PQG equations as extrapolation.

There are several future directions that may also provide potential improvement of the current framework. First, recall that the SPEKF model is a data-driven forecast model independent of the exact physical formulation of the time evolution of each spectral mode. More specific feature-based decomposition can be used to improve the characterization of turbulent features for complex systems. For example, a nonlinear POD-type autoencoder [16] can be adopted to replace the Fourier decomposition in suitable applications. Second, only a simple LSTM is utilized for discovering the relationship between observed and unobserved variables. More advanced machine learning tools can be used to describe such a nonlinear relationship more accurately. In addition, some ideas from the rank histogram filter (RHF) [1] can also be helpful to accelerate the computation. Third, extrapolation studies from different viewpoints may have particular interest. For example, the three-dimensional Boussinesq system [66] includes the slowly-evolving QG component and the fast gravity waves. It is essential to explore the data assimilation skill of the slow part of the Boussinesq system for the model calibrated based on a relatively low-cost QG model.

Acknowledgements

The research of L.S. is funded by NSF-DMS-1907667. The research of N.C. is funded by ONR N00014-21-1-2904 and ONR N00014-19-1-2421. C.M. is partially supported as a postdoc research associate under ONR N00014-19-1-2421.

Appendix

Details of the Sampling Formula in the Parameter Estimation of the SPEKF Model

Recall the parameter estimation of the SPEKF model (3) described in § 2.1.2. The algorithm consists of an iterative procedure that alternates between updating the parameters and sampling the trajectories of the three stochastic processes γk(t)\gamma_k(t), ωk(t)\omega_k(t) and bk(t)b_k(t). The update of the parameters is shown in (8). The sampling of the trajectories can be carried out via a closed analytic formula by exploiting the following general conditional sampling framework. Denote by X\mathbf{X} the state variables with observations and by Y\mathbf{Y} the stochastic parameterized processes. The SPEKF model fits into the following general model family with X=u^k\mathbf{X}=\hat{\mathbf{u}}_k and Y=(γk,ωk,bk)T\mathbf{Y}=(\gamma_k,\omega_k,b_k)^{\mathsf{T}},

dXdt=[A0(X,t)+A1(X,t)Y]+B1(X,t)W˙1(t),dYdt=[a0(X,t)+a1(X,t)Y]+b2(X,t)W˙2(t).(27)\begin{aligned} \frac{\mathrm{d}\mathbf{X}}{\mathrm{d}t} &= \left[\mathbf{A}_0(\mathbf{X},t)+\mathbf{A}_1(\mathbf{X},t)\mathbf{Y}\right]+\mathbf{B}_1(\mathbf{X},t)\dot{\mathbf{W}}_1(t),\\ \frac{\mathrm{d}\mathbf{Y}}{\mathrm{d}t} &= \left[\mathbf{a}_0(\mathbf{X},t)+\mathbf{a}_1(\mathbf{X},t)\mathbf{Y}\right]+\mathbf{b}_2(\mathbf{X},t)\dot{\mathbf{W}}_2(t). \tag*{(27)} \end{aligned}

In (27), A0\mathbf{A}_0, a0\mathbf{a}_0, A1\mathbf{A}_1, a1\mathbf{a}_1, B1\mathbf{B}_1 and b2\mathbf{b}_2 are vectors or matrices that can depend nonlinearly on the state variables X\mathbf{X} and time tt while W˙1\dot{\mathbf{W}}_1 and W˙2\dot{\mathbf{W}}_2 are independent white noises. For systems in (27), the conditional distribution p(Y(t)∣X(s),s≤t)∼N(μf,Rf)p(\mathbf{Y}(t)|\mathbf{X}(s),s\le t)\sim\mathcal{N}(\boldsymbol{\mu}_f,\mathbf{R}_f), which is also the posterior distribution of filtering Y\mathbf{Y}, is Gaussian. In addition, given one realization of X(t)\mathbf{X}(t) up to the current time instant tt, the conditional mean μf\boldsymbol{\mu}_f and the conditional covariance Rf\mathbf{R}_f can be solved via the following closed analytic formulae [46],

dμf=(a0+a1μf)+(RfA1∗)(B1B1∗)−1(dX−(A0+A1μf)dt)(28)\mathrm{d}\boldsymbol{\mu}_f=(\mathbf{a}_0+\mathbf{a}_1\boldsymbol{\mu}_f)+(\mathbf{R}_f\mathbf{A}_1^{*})(\mathbf{B}_1\mathbf{B}_1^{*})^{-1}\left(\mathrm{d}\mathbf{X}-(\mathbf{A}_0+\mathbf{A}_1\boldsymbol{\mu}_f)\mathrm{d}t\right) \tag*{(28)}
dRf=[a1Rf+Rfa1∗+b2b2∗−(RfA1∗)(B1B1∗)−1(A1Rf)]dt(29)\mathrm{d}\mathbf{R}_f=\left[\mathbf{a}_1\mathbf{R}_f+\mathbf{R}_f\mathbf{a}_1^{*}+\mathbf{b}_2\mathbf{b}_2^{*}-(\mathbf{R}_f\mathbf{A}_1^{*})(\mathbf{B}_1\mathbf{B}_1^{*})^{-1}(\mathbf{A}_1\mathbf{R}_f)\right]\mathrm{d}t \tag*{(29)}

with ∗^{*} being the complex conjugate transpose. Similarly, given one realization of X(t)\mathbf{X}(t) for t∈[0,T]t\in[0,T], the optimal smoother estimate p(Y(t)∣X(s),s∈[0,T])∼N(μs(t),Rs(t))p(\mathbf{Y}(t)|\mathbf{X}(s),s\in[0,T])\sim\mathcal{N}(\boldsymbol{\mu}_s(t),\mathbf{R}_s(t)) is also Gaussian [11], where the conditional mean μs(t)\boldsymbol{\mu}_s(t) and conditional covariance Rs(t)\mathbf{R}_s(t) of the smoother at time tt satisfy the following backward equations

d←μs=[−a0−a1μs+(b2b2∗)Rf−1(μf−μs)]dt(30)\overleftarrow{\mathrm{d}}\boldsymbol{\mu}_s=\left[-\mathbf{a}_0-\mathbf{a}_1\boldsymbol{\mu}_s+(\mathbf{b}_2\mathbf{b}_2^{*})\mathbf{R}_f^{-1}(\boldsymbol{\mu}_f-\boldsymbol{\mu}_s)\right]\mathrm{d}t \tag*{(30)}
d←Rs=[−(a1+(b2b2∗)Rf−1)Rs−Rs(a1∗+(b2b2∗)Rf)+b2b2∗]dt(31)\overleftarrow{\mathrm{d}}\mathbf{R}_s=\left[-(\mathbf{a}_1+(\mathbf{b}_2\mathbf{b}_2^{*})\mathbf{R}_f^{-1})\mathbf{R}_s-\mathbf{R}_s(\mathbf{a}_1^{*}+(\mathbf{b}_2\mathbf{b}_2^{*})\mathbf{R}_f)+\mathbf{b}_2\mathbf{b}_2^{*}\right]\mathrm{d}t \tag*{(31)}

with μf\boldsymbol{\mu}_f and Rf\mathbf{R}_f being given by (24). Here, the subscript ‘s’ in the conditional mean μs\boldsymbol{\mu}_s and conditional covariance Rs\mathbf{R}_s is an abbreviation for ‘smoother’, which should not be confused with the time variable ss in X(s)\mathbf{X}(s). The notation d←\overleftarrow{\mathrm{d}} corresponds to the negative of the usual difference, which means that the system (25) is solved backward over [0,T][0,T] with the starting value of the nonlinear smoother (μs(T),Rs(T))(\boldsymbol{\mu}_s(T),\mathbf{R}_s(T)) being the same as the filter estimate (μf(T),Rf(T))(\boldsymbol{\mu}_f(T),\mathbf{R}_f(T)). The backward equation takes into account future information. The forward run of (24) and the backward run of (25) collect the past and future observational information, respectively, for the state estimation at time tt.

Associated with the state estimation via the smoother in (25), conditioned on one realization of X(s)\mathbf{X}(s) for s∈[0,T]s\in[0,T], the optimal strategy of sampling the trajectories associated with the unobserved variable Y\mathbf{Y} satisfies the following explicit formula [13],

dY‾=dμ‾s−(a1+(b2b2∗)RF−1)(Y−μs) dt+b2 dWY(t),(32)d\overline{Y}=d\overline{\mu}_s-\left(a_1+\left(b_2b_2^*\right)R_F^{-1}\right)(Y-\mu_s)\,dt+b_2\,dW_Y(t), \tag*{(32)}

where WY(t)W_Y(t) is a Wiener process that is independent from W2(t)W_2(t) in (27). The optimality is in the Bayesian sense.

The closed analytic formula (32) is adopted to sample the trajectories of the three stochastic processes γk(t)\gamma_k(t), ωk(t)\omega_k(t) and bk(t)b_k(t) in the SPEKF model (3).

The traditional ROM for data assimilation

To construct an accurate traditional ROM with the LETKF data assimilation scheme, a truncation of the PQG equations to a coarse resolution grid is applied. Then the hyper-viscosity is increased to guarantee the stability of the solution, which is followed by adding stochastic noise to the truncated system that matches the equilibrium distribution of each mode with the truth. Furthermore, the complicated PV-and-M inversion is replaced by the linear inversion in the dry QG. The resulting system reads:

∂PVr,1∂t+J(ψ~r,1,PVr,1)−U∂PVr,1∂x+βv~r,1+v~r,1∂PV1,bg∂y=−κΔhψ~r,1−ν1(−1)sΔhsPVr,1+Fr,1(33)\frac{\partial PV_{r,1}}{\partial t}+J(\widetilde{\psi}_{r,1},PV_{r,1})-U\frac{\partial PV_{r,1}}{\partial x}+\beta\widetilde{v}_{r,1}+\widetilde{v}_{r,1}\frac{\partial PV_{1,bg}}{\partial y} =-\kappa\Delta_h\widetilde{\psi}_{r,1}-\nu_1(-1)^s\Delta_h^sPV_{r,1}+\mathcal{F}_{r,1} \tag*{(33)}
∂PVr,2∂t+J(ψ~r,2,PVr,2)+U∂PVr,2∂x+βv~r,2+v~r,2∂PV2,bg∂y=−ν2(−1)sΔhsPVr,2+Fr,2,(34)\frac{\partial PV_{r,2}}{\partial t}+J(\widetilde{\psi}_{r,2},PV_{r,2})+U\frac{\partial PV_{r,2}}{\partial x}+\beta\widetilde{v}_{r,2}+\widetilde{v}_{r,2}\frac{\partial PV_{2,bg}}{\partial y} =-\nu_2(-1)^s\Delta_h^sPV_{r,2}+\mathcal{F}_{r,2}, \tag*{(34)}
DmMr,mDt+v~r,m∂Mbg∂y=−νM(−1)sΔhsMr,m+Fr,M(35)\frac{D_mM_{r,m}}{Dt}+\widetilde{v}_{r,m}\frac{\partial M_{bg}}{\partial y} =-\nu_M(-1)^s\Delta_h^sM_{r,m}+\mathcal{F}_{r,M} \tag*{(35)}

where ψ~r,j\widetilde{\psi}_{r,j}, u~r,j\widetilde{u}_{r,j}, v~r,j\widetilde{v}_{r,j} are stream functions, zonal and meridional velocities using the dry QG relation and Fr,1\mathcal{F}_{r,1}, Fr,2\mathcal{F}_{r,2} and Fr,M\mathcal{F}_{r,M} are stochastic forcings that mimic the interactions between the truncated small scales and retained scales. In particular, the dry QG PV and streamfunction relation yields the following:

PVψ=ΔψPV1+PV22ψ=ψ1+ψ22(36)PV_{\psi}=\Delta\psi\frac{PV_1+PV_2}{2} \qquad \psi=\frac{\psi_1+\psi_2}{2} \tag*{(36)}
PVτ=Δτ−kd2τ=PV1−PV22,τ=ψ1−ψ22,(37)PV_{\tau}=\Delta\tau-k_d^2\tau=\frac{PV_1-PV_2}{2}, \qquad \tau=\frac{\psi_1-\psi_2}{2}, \tag*{(37)}

where PVψPV_{\psi} and PVτPV_{\tau} are the potential vorticity in the barotrophic and baroclinic modes respectively while ψ\psi and τ\tau are the corresponding barotrophic and baroclinic streamfunctions; kdk_d is the baroclinic deformation wavenumber which is calculated as kd=8(L/Ldu)2k_d=8(L/L_{du})^2 [59, 37]. Due to this dry QG PV and streamfunction relation, the moisture quantities, EE, VrV_r are removed from the PQG moisture evolution equation which are related to the phase changes of water in the original PQG system. In addition, the stochastic forcing can be represented by Gaussian white noises for each individual Fourier mode:

Fr,1≈∑l=−Nr/2Nr/2∑k=−Nr/2Nr/2σkl(1)W˙ei(kx+ly)+c.c.(38)\mathcal{F}_{r,1}\approx\sum_{l=-N_r/2}^{N_r/2}\sum_{k=-N_r/2}^{N_r/2}\sigma_{kl}^{(1)}\dot{W}e^{i(kx+ly)}+\mathrm{c.c.} \tag*{(38)}
Fr,2≈∑l=−Nr/2Nr/2∑k=−Nr/2Nr/2σkl(2)W˙ei(kx+ly)+c.c.(39)\mathcal{F}_{r,2}\approx\sum_{l=-N_r/2}^{N_r/2}\sum_{k=-N_r/2}^{N_r/2}\sigma_{kl}^{(2)}\dot{W}e^{i(kx+ly)}+\mathrm{c.c.} \tag*{(39)}
Fr,M≈∑l=−Nr/2Nr/2∑k=−Nr/2Nr/2σkl(M)W˙ei(kx+ly)+c.c.(40)\mathcal{F}_{r,M}\approx\sum_{l=-N_r/2}^{N_r/2}\sum_{k=-N_r/2}^{N_r/2}\sigma_{kl}^{(M)}\dot{W}e^{i(kx+ly)}+\mathrm{c.c.} \tag*{(40)}

where, σkl(1)\sigma_{kl}^{(1)}, σkl(2)\sigma_{kl}^{(2)}, σkl(M)\sigma_{kl}^{(M)}, corresponds to the magnitudes of white noise of (k,l)(k,l)th Fourier model of PV1PV_1, PV2PV_2 and moisture.

In addition, to improve the accuracy and robustness of the traditional ROM, the hyperviscosity and the white noise magnitude terms are tuned on an ad-hoc basis. More precisely, a wide range of different viscosity values ν1\nu_1, ν2\nu_2, νM\nu_M as well as σkl(1)\sigma^{(1)}_{kl}, σkl(2)\sigma^{(2)}_{kl}, σkl(M)\sigma^{(M)}_{kl} is run and the optimal set of parameters is chosen that generates the most accurate mean and variance compared with the truth time series. The optimal results are presented as a comparison in § 4.3.

References

  1. [1]Jeffrey L Anderson. A marginal adjustment rank histogram filter for non-Gaussian ensemble data assimilation. Monthly Weather Review, 148(8):3361–3378, 2020.DOI
  2. [2]Gal Berkl oo z, Philip Holmes, and John L Lumley. The proper orthogonal decomposition in the analysis of turbulent flows. Annual review of fluid mechanics, 25(1):539–575, 1993.
  3. [3]Judith Berner, Ulrich Achatz, Lauriane Batte, Lisa Bengtsson, Alvaro De La Camara, Hannah M Christensen, Matteo Colangeli, Danielle RB Coleman, Daan Crommelin, Stamen I D Lopatchiev, et al. Stochastic parameterization: Toward a new view of weather and climate models. Bulletin of the American Meteorological Society, 98(3):565–588, 2017.DOI
  4. [4]Craig H Bishop, Brian J Etherton, and Sharanya J Majumdar. Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects. Monthly weather review, 129(3):420–436, 2001.doi.org/10.1175/1520-0493(2001)129<0420:aswtet>2.0.co;2
  5. [5]Michal Branicki, Nan Chen, and Andrew J Majda. Non-gaussian test models for prediction and state estimation with model errors. Chinese Annals of Mathematics, Series B, 34(1):29–64, 2013.
  6. [6]Michal Branicki and Andrew J Majda. Quantifying uncertainty for predictions with model error in non-Gaussian systems with intermittency. Nonlinearity, 25(9):2543, 2012.DOI
  7. [7]Michal Branicki, Andrew J Majda, and Kody JH Law. Accuracy of some approximate Gaussian filters for the Navier–Stokes equation in the presence of model error. Multiscale Modeling & Simulation, 16(4):1756–1794, 2018.DOI
  8. [8]Gerrit Burgers, Peter Jan Van Leeuwen, and Geir Evensen. Analysis scheme in the ensemble Kalman filter. Monthly weather review, 126(6):1719–1724, 1998.doi.org/10.1175/1520-0493(1998)126<1719:asitek>2.0.co;2
  9. [9]Ashesh Chattopadhyay, Mustafa Mustafa, Pedram Hassanzadeh, Eviatar Bach, and Karthik Kashinath. Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers. arXiv preprint arXiv:2103.09360, 2021.DOI
  10. [10]Ashesh Chattopadhyay, Ebrahim Nabizadeh, Eviatar Bach, and Pedram Hassanzadeh. Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems. arXiv preprint arXiv:2206.04811, 2022.DOI
  11. [11]Nan Chen. Learning nonlinear turbulent dynamics from partial observations via analytically solvable conditional statistics. Journal of Computational Physics, 418:109635, 2020.DOI
  12. [12]Nan Chen and Yingda Li. BAMCAFE: A Bayesian machine learning advanced forecast ensemble method for complex turbulent systems with partial observations. Chaos: An Interdisciplinary Journal of Nonlinear Science, 31(11):113114, 2021.DOI
  13. [13]Nan Chen, Yingda Li, and Honghu Liu. Conditional Gaussian nonlinear system: A fast preconditioner and a cheap surrogate model for complex nonlinear systems. Chaos: An Interdisciplinary Journal of Nonlinear Science, 32(5):053122, 2022.DOI
  14. [14]Nan Chen and Andrew J Majda. Conditional Gaussian systems for multiscale nonlinear stochastic systems: Prediction, state estimation and uncertainty quantification. Entropy, 20(7):509, 2018.DOI
  15. [15]Nan Chen and Di Qi. A physics-informed data-driven algorithm for ensemble forecast of complex turbulent systems. arXiv preprint arXiv:2204.08547, 2022.arxiv.org/abs/2204.08547
  16. [16]Sibo Cheng, Jianhua Chen, Charitos Anastasiou, Panagiota Angeli, Omar K Matar, Yi-Ke Guo, Christopher C Pain, and Rossella Arcucci. Generalised latent assimilation in heterogeneous reduced spaces with machine learning surrogate models. arXiv preprint arXiv:2204.03497, 2022.arxiv.org/abs/2204.03497
  17. [17]HM Christensen, Judith Berner, Danielle RB Coleman, and TN Palmer. Stochastic parameterization and El Niño–Southern oscillation. Journal of Climate, 30(1):17–38, 2017.DOI
  18. [18]Jinqiao Duan and Balasubramanya Nadiga. Stochastic parameterization for large eddy simulation of geophysical flows. Proceedings of the American Mathematical Society, 135(4):1187–1196, 2007.DOI
  19. [19]Samuel Dupond. A thorough review on the current advance of neural network structures. Annual Reviews in Control, 14:200–230, 2019.
  20. [20]Thomas K Edwards, Leslie M Smith, and Samual N Stechmann. Atmospheric rivers and water fluxes in precipitating quasi-geostrophic turbulence. Quarterly Journal of the Royal Meteorological Society, 146:1960–1975, 2020.DOI
  21. [21]Thomas K Edwards, Leslie M Smith, and Samuel N Stechmann. Spectra of atmospheric water in precipitating quasi-geostrophic turbulence. Geophysical & Astrophysical Fluid Dynamics, 114(6):715–741, 2020.DOI
  22. [22]Geir Evensen et al. Data assimilation: the ensemble Kalman filter, volume 2. Springer, 2009.
  23. [23]Geir Evensen and Peter Jan Van Leeuwen. An ensemble Kalman smoother for nonlinear dynamics. Monthly Weather Review, 128(6):1852–1867, 2000.doi.org/10.1175/1520-0493(2000)128<1852:aeksfn>2.0.co;2
  24. [24]Uriel Frisch and Andrei Nikolaevich Kolmogorov. Turbulence: the legacy of AN Kolmogorov. Cambridge university press, 1995.
  25. [25]Crispin W Gardiner et al. Handbook of stochastic methods, volume 3. springer Berlin, 1985.
  26. [26]Boris Gershgorin, John Harlim, and Andrew J Majda. Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation. Journal of Computational Physics, 229(1):32–57, 2010.DOI
  27. [27]Boris Gershgorin, John Harlim, and Andrew J Majda. Test models for improving filtering with model errors through stochastic parameter estimation. Journal of Computational Physics, 229(1):1–31, 2010.DOI
  28. [28]Georg A Gottwald and Sebastian Reich. Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations. Chaos: An Interdisciplinary Journal of Nonlinear Science, 31(10):101103, 2021.DOI
  29. [29]Ian Grooms, Yoonsang Lee, and Andrew J Majda. Ensemble filtering and low-resolution model error: Covariance inflation, stochastic parameterization, and model numerics. Monthly Weather Review, 143(10):3912–3924, 2015.DOI
  30. [30]Marcus J Grote and Andrew J Majda. Stable time filtering of strongly unstable spatially extended systems. Proceedings of the National Academy of Sciences, 103(20):7548–7553, 2006.pubmed.ncbi.nlm.nih.gov/16682626
  31. [31]J Harlim and AJ Majda. Filtering nonlinear dynamical systems with linear stochastic models. Nonlinearity, 21(6):1281, 2008.DOI
  32. [32]John Harlim. Interpolating irregularly spaced observations for filtering turbulent complex systems. SIAM Journal on Scientific Computing, 33(5):2620–2640, 2011.DOI
  33. [33]Jincong He, Pallav Sarma, and Louis J Durlofsky. Use of reduced-order models for improved data assimilation within an EnKF context. In SPE Reservoir Simulation Symposium. OnePetro, 2011.DOI
  34. [34]Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.pubmed.ncbi.nlm.nih.gov/9377276
  35. [35]Ibrahim Hoteit, Dinh-Tuan Pham, and Jacques Blum. A simplified reduced order Kalman filtering and application to altimetric data assimilation in Tropical Pacific. Journal of Marine systems, 36(1-2):101–127, 2002.DOI
  36. [36]Peter L Houtekamer and Herschel L Mitchell. Ensemble Kalman filtering. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 131(613):3269–3289, 2005.
  37. [37]Rentian Hu, Thomas K Edwards, Leslie M Smith, and Samuel N Stechmann. Initial investigations of precipitating quasi-geostrophic turbulence with phase changes. Research in the Mathematical Sciences, 8(1):1–25, 2021.DOI
  38. [38]Brian R Hunt, Eric J Kostelich, and Istvan Szunyogh. Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D: Nonlinear Phenomena, 230(1-2):112–126, 2007.DOI
  39. [39]Eugenia Kalnay. Atmospheric modeling, data assimilation and predictability. Cambridge university press, 2003.
  40. [40]Emily L Kang and John Harlin. Filtering nonlinear spatio-temporal chaos with autoregressive linear stochastic models. Physica D: Nonlinear Phenomena, 241(12):1099–1113, 2012.DOI
  41. [41]Mark A Kramer. Nonlinear principal component analysis using autoassociative neural networks. AIChE journal, 37(2):233–243, 1991.DOI
  42. [42]Tomoki Kurikawa, Omri Barak, and Kunihiko Kaneko. Repeated sequential learning increases memory capacity via effective decorrelation in a recurrent neural network. Physical Review Research, 2(2):023307, 2020.DOI
  43. [43]Boris Khattatov William Lahoz and Richard Menard. Data assimilation. Springer, 2010.
  44. [44]Kody Law, Andrew Stuart, and Kostas Zygalakis. Data assimilation. Cham, Switzerland: Springer, 214, 2015.arxiv.org/abs/1506.07825
  45. [45]Ying Li and Samuel N Stechmann. Predictability of tropical rainfall and waves: Estimates from observational data. Quarterly Journal of the Royal Meteorological Society, 146(729):1668–1684, 2020.DOI
  46. [46]Robert S Liptser and Albert N Shiryaev. Statistics of random processes II: Applications, volume 6. Springer Science & Business Media, 2013.
  47. [47]Andrew Majda and Xiaoming Wang. Nonlinear dynamics and statistical theories for basic geophysical flows. Cambridge University Press, 2006.DOI
  48. [48]Andrew J Majda and Nan Chen. Model error, information barriers, state estimation and prediction in complex multiscale systems. Entropy, 20(9):644, 2018.DOI
  49. [49]Andrew J Majda and John Harlim. Filtering complex turbulent systems. Cambridge University Press, 2012.DOI
  50. [50]Andrew J Majda, Di Qi, and Themistoklis P Sapsis. Blended particle filters for large-dimensional chaotic dynamical systems. Proceedings of the National Academy of Sciences, 111(21):7511–7516, 2014.DOI
  51. [51]PierGianLuca Porta Mana and Laure Zanna. Toward a stochastic parameterization of ocean mesoscale eddies. Ocean Modelling, 79:1–20, 2014.DOI
  52. [52]Romit Maulik, Vishwas Rao, Jiali Wang, Gianmarco Mengaldo, Emil Constantinescu, Bethany Lusch, Prasanna Balaprakash, Ian Foster, and Rao Kotamarthi. AIEADA 1.0: Efficient high-dimensional variational data assimilation with machine-learned reduced-order models. Geoscientific Model Development Discussions, 2022:1–20, 2022.
  53. [53]Marcello Meldi and Alexandre Poux. A reduced order model based on Kalman filtering for sequential data assimilation of turbulent flows. Journal of Computational Physics, 347:207–234, 2017.arxiv.org/abs/1702.06469
  54. [54]Peter Müller. The equations of oceanic motions. Cambridge University Press, 2006.DOI
  55. [55]Suraj Pawar, Shady E Ahmed, Omer San, Adil Rasheed, and Ionel M Navon. Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows. Physics of Fluids, 32(7):076606, 2020.DOI
  56. [56]Benjamin Peherstorfer and Karen Willcox. Dynamic data-driven reduced-order models. Computer Methods in Applied Mechanics and Engineering, 291:21–41, 2015.DOI
  57. [57]Stephen G Penny, Timothy A Smith, T-C Chen, Jason A Platt, H-Y Lin, Michael Goodliff, and Henry DI Abarbanel. Integrating recurrent neural networks with data assimilation for scalable data-driven state estimation. Journal of Advances in Modeling Earth Systems, 14(3):e2021MS002843, 2022.DOI
  58. [58]Di Qi and John Harlim. Machine learning-based statistical closure models for turbulent dynamical systems. Philosophical Transactions of the Royal Society A, 380(2229):20210205, 2022.DOI
  59. [59]Di Qi and Andrew J Majda. Low-dimensional reduced-order models for statistical response and uncertainty quantification: Two-layer baroclinic turbulence. Journal of the Atmospheric Sciences, 73(12):4609–4639, 2016.DOI
  60. [60]Yvonne Ruckstuhl, Tijana Janjić, and Stephan Rasp. Training a convolutional neural network to conserve mass in data assimilation. Nonlinear Processes in Geophysics, 28(1):111–119, 2021.
  61. [61]Rick Salmon. Baroclinic instability and geostrophic turbulence. Geophysical & Astrophysical Fluid Dynamics, 15(1):167–211, 1980.DOI
  62. [62]Rick Salmon. Lectures on geophysical fluid dynamics. Oxford University Press, 1998.
  63. [63]Themistoklis P Sapsis and Andrew J Majda. A statistically accurate modified quasilinear Gaussian closure for uncertainty quantification in turbulent dynamical systems. Physica D: Nonlinear Phenomena, 252:34–45, 2013.DOI
  64. [64]Tapio Schneider, Andrew M Stuart, and Jin-Long Wu. Learning stochastic closures using ensemble Kalman inversion. Transactions of Mathematics and Its Applications, 5(1):tnab003, 2021.arxiv.org/abs/2004.08376
  65. [65]Leslie M Smith and Samuel N Stechmann. Precipitating quasigeostrophic equations and potential vorticity inversion with phase changes. Journal of the Atmospheric Sciences, 74(10):3285–3303, 2017.
  66. [66]Leslie M Smith and Fabian Waleffe. Generation of slow large scales in forced rotating stratified turbulence. Journal of Fluid Mechanics, 451:145–168, 2002.DOI
  67. [67]Răzvan Ștefănescu, Adrian Sandu, and Ionel Michael Navon. POD/DEIM reduced-order strategies for efficient four dimensional variational data assimilation. Journal of Computational Physics, 295:569–595, 2015.DOI
  68. [68]Meng Tang, Yimin Liu, and Louis J Durlofsky. A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems. Journal of Computational Physics, 413:109456, 2020.DOI
  69. [69]Tadashi Tsuyuki and Ryosuke Tamura. Nonlinear data assimilation by deep learning embedded in an ensemble Kalman filter. Journal of the Meteorological Society of Japan. Ser. II, 2022.DOI
  70. [70]Geoffrey K Vallis. Atmospheric and oceanic fluid dynamics. Cambridge University Press, 2017.
  71. [71]L Minah Yang and Ian Grooms. Machine learning techniques to construct patched analog ensembles for data assimilation. Journal of Computational Physics, 443:110532, 2021.DOI

Paper details

vlnc:W_qeyw337b
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents