betaFeedback

WorksPhysics and Astronomy

Ensemble Covariance Propagation with Machine Learning Weather Prediction Models in a Realistic Data Assimilation System

Abstract

Integrating machine learning weather prediction (MLWP) models into classical data assimilation (DA) workflows requires that they accurately evolve ensemble covariances. We evaluate this capability in a two-way coupled DA system, where an ensemble variational (EnVar) component evolves the mean and an ensemble Kalman filter (EnKF) evolves the covariances. The system is cycled over one month and assimilates a wide range of real-world observations, including microwave radiances. Using the MLWP model GraphCast for the EnKF and the physics-based MPAS model for the EnVar yields ensemble spread and covariances comparable to a reference configuration in which MPAS evolves both components. The resulting EnVar mean achieves consistently lower RMSE against GFS analyses, except near the model top, where spurious correlations degrade performance. These results demonstrate that MLWP models can propagate covariance information in a realistic DA cycling system and provide a proof of concept for their integration into future operational ensemble systems.

main.pdf

Key Points:

  • A two-way coupled data assimilation system provides a clean test of covariance propagation with the machine learning model GraphCast

  • GraphCast matches a physics-based system in ensemble spread and covariance structure over a month of cycling with real observations

  • The GraphCast system matches or outperforms a physics-based system except near the model top and is a proof of concept for operational use

Plain Language Summary

Weather forecasts begin from a best estimate of the current atmosphere, which is obtained by combining a short forecast with observations. Forecast centers repeat this procedure every few hours with many slightly different forecasts, whose differences quantify the forecast uncertainty and how much each observation should correct its surroundings. Running these forecasts with a traditional physics-based model is costly. Machine learning (ML) weather models are much cheaper, but earlier attempts to use them in this process gave mixed results, and some forecasts became unstable after a few iterations. A suspected cause is that ML models do not correctly capture how forecast uncertainties evolve. We tested this by letting a physics-based model produce the best estimate of the atmosphere, while the ML model, GraphCast, evolved only the forecast uncertainties. We ran this system for one month with a wide range of real-world observations, including satellite data. It performed as well as, or slightly better than, a system using only the physics-based model at most levels of the atmosphere, showing that ML models can correctly evolve forecast uncertainties. Because this setup resembles the systems used at forecast centers, the results show a practical path to cheaper everyday forecasting with ML models.

Introduction

Machine learning-based weather prediction (MLWP) models have improved rapidly in recent years (see, for example, Pathak et al., 2022; Bi et al., 2023; Bonev et al., 2023; Lam et al., 2023; Kochkov et al., 2024; Price et al., 2025). The best current MLWP models now surpass traditional, physics-based numerical models in many aspects of global medium-range forecasting (Lang et al., 2024) and, once trained, can be run at a fraction of the computational cost.

A natural question is then whether a skillful MLWP model can replace traditional numerical models in the data-assimilation (DA) workflows that provide initial conditions for medium-range forecasts. Several studies, using both variational and ensemble-based DA schemes, have performed idealized experiments with synthetic observations simulated from reanalyses and shown that MLWP models can cycle stably with DA (i.e. repeatedly alternate forecast and assimilation steps, see text S1 in the supplementary material for details) for extended periods (Xiao et al., 2024; Adrian et al., 2025; Kotsuki et al., 2025). When assimilating real observations, cycling DA results with MLWP models have been mixed. Slivinski et al. (2025) show that an Ensemble Kalman Filter (EnKF) assimilating surface pressure observations generates accurate global analyses when forecasts come from a physics-based model, but diverges when MLWP forecasts are used instead. On the other hand, Takeshima et al. [8] and Huang et al. (2026) assimilate the global network of conventional observations and show that EnKFs with MLWP forecasts cycle stably, though they do not compare to the same DA system with forecasts from a physics-based model.

Because modern DA schemes rely on the forecast model to provide covariance information, one requirement for MLWP models used in DA is that they should accurately capture the evolution of perturbations, such as analysis increments or ensemble deviations. It is an open question whether MLWP models do this. They clearly can degrade cross-variable relationships such as geostrophy (Bonavita, 2024) and there is broad consensus that MLWP models trained with a root-mean-square error (RMSE) loss overly damp energy at small scales, with implications for perturbation growth (Lam et al., 2023; Bonavita, 2024; Subich et al., 2025). In addition, gradients of forecast quantities with respect to initial conditions, which are enabled by the autodifferentiability of MLWP models, can exhibit unphysical features (Tian et al., 2026), though other case studies find that MLWP gradients agree well with those from the adjoint of a physics-based numerical model (Baño-Medina et al., 2025). Moreover, and with most direct relevance to the present paper, Slivinski et al. (2025), Kotsuki et al. (2025) and Takeshima et al. [8] all conclude that differences in their experiments between DA with MLWP forecasts or with physics-based numerical forecasts, including the divergence of cycling DA with MLWP models in some cases, stem from flaws in the way MLWP models evolve small perturbations.

This paper explores further the use of MLWP models in existing DA workflows. We begin from a cycling DA system that is credible, if not of operational quality, and that uses a state-of-the-art physics-based numerical model. We configure the system, as is current practice at most operational centers for global weather prediction, to employ an ensemble-variational (EnVar) DA scheme, which minimizes a cost function that includes covariances estimated from a forecast ensemble, together with an EnKF, which produces initial conditions for the forecast ensemble. This configuration allows the model used for ensemble forecasts to be different from that used for the “deterministic” EnVar cycle. We test replacing the physics-based model with a MLWP model for the ensemble forecasts, thus cleanly isolating the effects of the MLWP model to the propagation of covariance information. Our experiments also improve on previous work by using a more comprehensive suite of observations that includes microwave radiances from polar-orbiting instruments, which are the backbone of the global observing network. We demonstrate that using an MLWP model to evolve the ensemble covariances can perform as well as, or better than, using a physics-based model.

The details of the forecast models, the DA system, and the assimilated observations are described in § 3. § 4 compares the performance of the physics-based and MLWP-based DA systems and characterizes the evolution of the deterministic background quality (§ 4.1), RMS-spread statistics (§ 4.2) and single-observation experiments (§ 4.3). Broader implications are discussed in § 5 and additional details and statistics are provided in the supplementary material.

Methods

Forecast models

The two forecast models considered in this study are MPAS (Skamarock et al., 2012) and GraphCast (referred to as GC hereafter, Lam et al., 2023). MPAS is a physics-based model, discretized on an unstructured horizontal mesh with a hybrid terrain-following vertical coordinate, that has been used previously in cycling DA experiments [6, 7 among others]. GC, by contrast, is an MLWP model whose state variables are defined on a structured latitude–longitude mesh and a discrete set of vertical pressure levels. Since the pretrained GC model has a fixed mesh, MPAS is configured to match GC’s model top and horizontal resolution as closely as possible.

Specifically, the MPAS configuration adopted here uses a global domain with a quasiuniform 120 km horizontal mesh, a 22.6 km model top with a gravity wave absorbing layer starting at 18.5 km, and 47 hybrid vertical levels. This model top is lower than in previous studies, and chosen to approximately match the 50 hPa top of GC. For GC, the small configuration with 1∘1^\circ horizontal resolution and 13 vertical pressure levels is used. The two models have comparable horizontal resolution at the equator, but GC has a finer longitudinal grid spacing toward the poles.

Cycling workflow

The DA cycling workflow consists of concurrently running, two-way coupled EnKF and EnVar systems, as illustrated in Figure 1. The EnVar uses the EnKF background ensemble to estimate the background error covariance, while the EnKF analysis ensemble is recentered on the EnVar analysis at each DA cycle. This recentering is central to the design of this study, since it restricts the role of the ensemble forecast model to evolving ensemble perturbations around the mean of the EnVar component. The EnKF system is formulated in gain form (GETKF) with model-space localization in the vertical coordinate [1] and consists of 20 ensemble members. Further details of both DA components are provided in the supplementary material (text S2).

Figure 1. Schematic of the data assimilation cycling workflow for the two-way coupled ensemble (top, orange) and deterministic (bottom, purple) system. Arrows indicate the flow of information, with the following notation: background xbx^{b}, analysis xax^{a}, perturbation δx\delta x, mean xˉ\bar{x}, and ensemble background error covariance BeB_{e}. The subscript i=1,…,ni=1,\ldots,n indexes ensemble members, and the subscript cc denotes the deterministic control state. Configurations C1 and C2 differ in the forecast model for the ensemble component.

The forecast model for each DA component can be chosen independently, and two configurations are considered for this study, as indicated in Figure 1. Configuration C1 uses the physics-based MPAS model for both the EnKF and EnVar forecasts and serves as a physics-based benchmark. Configuration C2 replaces MPAS with GC for the EnKF forecast and retains the physics-based MPAS model for the EnVar forecast. A comparison between configurations C1 and C2 provides a direct test of whether GC can accurately evolve ensemble covariances, because the two configurations differ only in the model used to propagate the ensemble.

The ensemble inflation for each configuration is tuned independently (see text S3 for details), yet the optimal inflation parameters are identical for configurations C1 and C2. This contrasts with previous studies, where MLWP-based ensembles require additional inflation to cycle stably (Kotsuki et al., 2025; Huang et al., 2026).

All data assimilation steps are performed in MPAS-JEDI on the native MPAS mesh, regardless of the choice of ensemble forecast model. In configuration C2, this requires interpolating between the MPAS and GC meshes before and after each ensemble forecast step, and supplementing the GC state with the soil variables it lacks. Details of this procedure are provided in text S4.

Data assimilation experiments

Configurations C1 and C2 are cycled every six hours over a one-month period (2018-04-15T00Z – 2018-05-14T18Z). The initial background at 2018-04-15T00Z is obtained from a 6 h MPAS forecast, initialized from GFS (EnVar component) and GEFS (EnKF), respectively. Subsequent DA cycles generate 6 h forecasts with the designated forecast model, initialized from the latest analysis (MPAS) or the two most recent analyses (GC, which requires two input states).

Each DA component assimilates the same set of observations, which represents a substantial subset of the global observation network, including conventional observations, satellite-derived atmospheric motion vectors, GNSS-RO bending angle profiles, and microwave radiances (see text S5 and Table S1 for details). To the authors’ knowledge, this marks the first assimilation of microwave radiances into an MLWP-based forecast via a traditional radiative transfer model and DA framework, rather than an end-to-end ML-based assimilation approach that maps observations directly to an analysis (as used, for example, in Xu et al., 2025; X. Sun et al., 2025). Observations are ingested in raw form and quality control, observation error modeling, and radiance bias correction are all performed within MPAS-JEDI.

SatelliteSensorClear-sky channel
AquaAMSU-A8
NOAA-15AMSU-A5, 7, 8
NOAA-18AMSU-A5, 6, 7, 8
NOAA-18MHS3, 4, 5
NOAA-19AMSU-A5, 6, 7
NOAA-19MHS3, 4, 5
Metop-AAMSU-A5, 6
Metop-AMHS3, 4, 5
Metop-BAMSU-A8
Metop-BMHS3, 4, 5

Table S1. Assimilated AMSU-A and MHS clear-sky channels

Results

Deterministic component: model-space verification

Because the EnVar system relies on the EnKF ensemble to estimate the background error covariance at each analysis step (see Figure 1), a verification of the 6 h EnVar background provides a global assessment of the EnKF ensemble. This section presents this background verification against the GFS analysis, which is taken as the ground truth.

To simplify the comparison between the two configurations, Figure 2 shows the model-space verification in terms of the relative RMSE difference, rms(C2)/rms(C1) − 1, so that negative values (blue) indicate better performance of C2. The comparison reveals two prominent differences between the two configurations. First, below model level 40, configuration C2 exhibits smaller RMSEs than C1. This improvement persists over the entire month of cycling and grows slowly with time (left column). When binned by model level and latitude (right column), the RMSE reduction is apparent at almost all latitudes and model levels, with the largest improvements over the tropics and southern hemisphere, and smaller improvements over the northern hemisphere. Below model level 40, the MLWP-based system is therefore at least as accurate as the physics-based baseline.

Figure 2. Verification of the 6 h EnVar background against the GFS analysis, shown as the relative RMSE difference between configurations C2 and C1, rms⁡(C2)/rms⁡(C1)−1\operatorname{rms}(\mathrm{C2})/\operatorname{rms}(\mathrm{C1})-1. Negative values (blue) indicate improvement of C2 over C1. Top row: temperature (TT), bottom row: zonal wind (UU). Left column: RMSE averaged horizontally, right column: RMSE averaged over longitude and time. The dashed and dotted horizontal lines indicate, respectively, the approximate location of the GC model top and the highest level in the EnVar analysis that can be directly influenced by observations above the GC model top.

The second, contrasting observation is a pronounced deterioration in configuration C2 that develops above level 40 after approximately two days of cycling and persists throughout the month. Three factors likely contribute. First, the mismatch between the MPAS and GC model tops: the physical height of GC’s 50 hPa top varies with location and time and can be as low as 19 km. When GC forecasts are interpolated to the MPAS mesh, all quantities above the 50 hPa isobar must be extrapolated, which introduces additional error in the ensemble fields compared to the interpolated levels below. The dashed magenta line in Figure 2 marks the lowest MPAS level at which this can occur. Extrapolation errors can also affect the DA at lower levels through vertical ensemble covariances, but only within the 6 km vertical localization distance used in both the EnVar and GETKF systems (dotted line). Second, MPAS applies a gravity wave damping layer that damps vertical motions above 18.5 km (approximately the dashed magenta line), whereas GC has no analogous damping, so the two systems represent different dynamics there. Third, spurious sensitivities near the model top, which have been reported for various MLWP models in both the linear and nonlinear regimes, may also contribute (Baño-Medina et al., 2025; Tian et al., 2026).

We have not isolated the root cause, and a combination of these factors likely contributes to the observed deterioration. Even so, away from the immediate vicinity of the GC model top, the MLWP ensemble performs at least as well as, and in general better than, the MPAS ensemble.

Ensemble component: model-space RMS-spread statistics

Ensemble spread diagnostics provide a second, complementary assessment of the MLWP model’s ability to propagate covariances. Figure 3 compares the spread of the background ensembles against the RMSE of the ensemble mean in model space, with the GFS analysis taken as ground truth. Model levels above the dotted line are potentially affected by extrapolation above the GC model top (see previous section) and are therefore shown muted.

Figure 3. RMS-spread statistics for the EnKF background ensemble. Solid lines show the RMSE of the ensemble mean against the GFS analysis, dashed lines represent the ensemble spread. Black: configuration C1 (MPAS), red: configuration C2 (GC). Results are averaged horizontally and over the last week of the cycling period. Model levels above the dotted horizontal line are potentially affected by spurious correlations near the GC model top.

Below the dotted horizontal line, the two ensembles have very similar spread (dashed lines) across all variables and levels. This has two important implications. First, it demonstrates that GC can maintain an ensemble spread comparable to a physics-based model over many DA cycles. Second, it rules out the possibility that the RMSE differences seen in Figure 2 below this level are driven by systematic differences in how the two systems weight the background against observations. This suggests instead that the lower RMSE of the GC configuration reflects an improved ensemble covariance.

Figure S1 and Figure S2 further show that the two ensemble spreads remain approximately constant over time and have a similar spatial structure, even if local differences exist. For example, configuration C2 typically shows larger spread in the polar regions and lacks some of the small-scale features seen in C1 over the mid-latitudes and tropics. This suggests that the two configurations may weigh the background and observations somewhat differently at specific locations, but they assign comparable weight to each on average.

Figure S1. Time series of RMS-spread statistics for the EnKF background ensemble over the last week of cycling. Solid lines show the RMSE of the ensemble mean against the GFS analysis, dashed lines represent the ensemble spread. Black: configuration C1 (MPAS), red: configuration C2 (GC). Results are averaged horizontally and vertically over the levels not directly affected by spurious correlations near the GC model top (up to model level 34, indicated by the dotted magenta line in Figure 2 and Figure 3).

Figure S2. Ensemble spread at model level 17 at the end of the cycling period (2018-05-15T00). Top row: temperature, bottom row: zonal wind. Left column: configuration C1 (MPAS), right column: C2 (GC).

Comparing the ensemble spread to the RMSE of the mean (solid lines) shows that both systems are under-dispersive. This likely arises from the known bias of the EnKF update, which underestimates the analysis variance, and from unaccounted model error in the forecast step, which are not specific to either forecast model. Under-dispersive ensembles are common in ensemble DA and have been reported in previous MPAS-JEDI systems (see e.g. Figure 1 in Guerrette et al. (2023) and Figure 2 in T. Sun et al. [7]). Nevertheless, the gap between spread and RMSE is generally smaller for configuration C2 (GC), owing to the higher-quality ensemble mean discussed in the previous section.

Ensemble component: covariance structure

As a final, qualitative assessment of the ensemble background error covariance, this section examines its structure through single-observation experiments conducted at the end of the cycling period (2018-05-15T00Z). Results are shown for single temperature observation with a 1 K innovation and a 1 K observation-error standard deviation. We tested multiple observation locations and present here representative results for an observation placed over the North Pacific (40°N, 175°W) at approximately 702 hPa (model level 17).

Figure 4 shows the cross-covariance between zonal wind and the temperature at the measurement location. Both configurations show an anti-cyclonic wind pattern at and above the measurement location. C1 additionally shows a weak cyclonic pattern below the measurement location, which is absent in C2. This anti-cyclonic and cyclonic pattern is consistent with the thermal-wind relation in the mid-latitudes, and a similar structure, with location-specific variations, is found at other measurement locations as well (not shown). Beyond this shared large-scale structure, the spatial arrangement, magnitude, and small-scale details of the two covariances differ. These differences likely reflect a combination of genuine differences between the underlying ensembles and sampling uncertainty from the small ensemble size, and the comparison should therefore be interpreted qualitatively.

Cross-covariance between zonal wind and temperature for C1 (MPAS) and C2 (GC)

Figure 4. Single-observation experiment at the end of the cycling (2018-05-15T00): cross-covariance between zonal wind and temperature, cov⁡(U(r),T(rM))\operatorname{cov}(U(r), T(r_M)), where rMr_M denotes the measurement location. The measurement location is at 40° N, 175° W, 702 hPa and indicated by the × marker. Left column: C1 (MPAS); right column: C2 (GC). Top row: horizontal cross section at the measurement height; bottom row: vertical cross section along 175° W.

The autocovariance of the temperature field with the measurement shows a similar picture (Figure S3): the spatial structure is again similar between the two configurations, but the magnitude in C2 is somewhat smaller than in C1. This magnitude difference is location-dependent, as illustrated in the top row of Figure S2, which shows the temperature variance, i.e. the autocovariance at zero separation, at the measurement level (model level 17) across the globe.

Autocovariance of the temperature field for C1 (MPAS) and C2 (GC)

Figure S3. Single-observation experiment at the end of the cycling period (2018-05-15T00): autocovariance of the temperature field, cov⁡(T(r),T(rM))\operatorname{cov}(T(r), T(r_M)), where rMr_M denotes the measurement location. The measurement location is at 40∘N40^\circ\mathrm{N}, 175∘W175^\circ\mathrm{W}, 702 hPa and indicated by the ×\times marker. Left column: C1 (MPAS); right column: C2 (GC). Top row: horizontal cross section at the measurement height; bottom row: vertical cross section along 175∘W175^\circ\mathrm{W}.

The single-observation experiments indicate that GC can maintain a realistic covariance structure over many DA cycles, consistent with the spread results discussed in the previous section. There is no evidence of grid-scale noise build-up or weakening of the cross-covariances over time, in contrast to the failure mode reported in Slivinski et al. (2025). Although the covariances differ in magnitude, shape, and small-scale details, the lower background errors of configuration C2 (§ 4.1) suggest that the MLWP-based covariances propagate observation information at least as effectively as the physics-based ones.

Conclusions

Integration of MLWP models into classical DA workflows requires that they accurately evolve perturbations, such as analysis increments or ensemble member differences. This study tested their capability to evolve ensemble perturbations that provide ensemble covariance information within a realistic, two-way coupled EnVar–EnKF DA system. In this system, the EnVar component used the EnKF ensemble to estimate background error covariances, and the EnKF analysis ensemble was recentered on the EnVar analysis at each cycle. A baseline configuration (C1) used a physics-based model (MPAS) to evolve both the EnVar and EnKF forecasts, while a comparison configuration (C2) used an MLWP model (GraphCast) to evolve the EnKF members but retained MPAS for the EnVar forecast. This design deliberately limited the MLWP model’s role to evolving ensemble perturbations and the covariance information they provide, so that a comparison of C1 and C2 provided a clean test of this capability. Each DA component assimilated a substantial subset of the global observation network, including conventional observations, satellite-derived atmospheric motion vectors, GNSS-RO bending angle profiles, and microwave satellite radiances.

Configuration C2 cycled stably over an entire month and achieved a performance comparable to, or marginally better than, the physics-based benchmark C1 at most model levels. The exception was near the model top, where artifacts from interpolating between the two model meshes degraded the quality of configuration C2. This degradation could likely be reduced by more closely aligning the meshes of the physics-based and MLWP models, as done, for example, in AICON (Goecke et al., 2026) and MAPCast (Wang & Wang, 2026), or by employing the so-called hybrid EnVar available in JEDI (see e.g. Jung et al., 2024) and relying only on the parametric covariances at levels where the MLWP ensemble would need to be extrapolated. Ensemble spread and single-observation experiments showed that the two systems produce comparable background error covariances on average, with local differences in covariance structure and magnitude. These differences enable configuration C2 to better weigh and propagate observation information, which lead to the better background fit to GFS analyses. Taken together, these results demonstrate that GraphCast can provide physically meaningful covariance information within a realistic DA cycling system.

A unique aspect of this study is the operationally relevant workflow, which separates the evolution of the ensemble mean and covariance. Operational experience suggests that the quality of the background error covariances is often secondary to the quality of the mean forecast (e.g. Buehner et al., 2017; Lorenc et al., 2017; Parrish & Derber, 1992), where the latter uses an even simpler background-covariance model such as 3DVar) and the strong performance of the present configuration may partially reflect the benefit of limiting the MLWP model to the ensemble evolution. A future configuration in which GraphCast also provides the EnVar forecast would offer a controlled test of the mean forecast’s contribution to the overall performance.

The present study extends and contrasts with the existing literature with respect to DA system stability and performance, as well as ensemble inflation. Like the MLWP-based filters of Kotsuki et al. (2025); Takeshima et al. [8]; Huang et al. (2026) and the stabilized filter of Slivinski et al. (2025), the present configuration cycles stably throughout the month-long experiment. This provides further evidence that the diverging filters of Slivinski et al. (2025) based on MLWP model forecasts are the exception rather than the norm. Compared to all of these previous studies, however, the present MLWP-based configuration did not require additional ensemble inflation and is the first to perform as well as, or better than, a physics-based benchmark.

Because the present configuration with its two-way coupled DA components and a wide variety of assimilated observations, including satellite radiances, closely mirrors an operational setup, these results constitute a proof of concept that MLWP models can supply covariance information within future operational ensemble systems as well.

Open Research Section

Observational data from the National Centers for Environmental Prediction (NCEP) used in the data assimilation experiments are available from the NSF NCAR Geoscience Data Exchange at https://doi.org/10.5065/Z83F-N512, https://doi.org/10.5065/39C5-Z211, and https://doi.org/10.5065/DWYZ-Q852. The GFS analyses used for the EnVar initial conditions, model-space verification, and supplementing the GraphCast state are available from NOAA at https://registry.opendata.aws/noaa-gfs-bdp-pds, while the GEFS analyses used for the EnKF initial conditions are available at https://registry.opendata.aws/noaa-gefs.

All atmospheric models, the data assimilation framework, and the workflow engine used in this study are developed openly. The data assimilation framework, the physics-based MPAS model, and the observation conversion tools are provided through MPAS-JEDI (https://github.com/JCSDA/mpas-jedi). GraphCast is run through the Earth2Studio package (https://github.com/NVIDIA/earth2studio), and the data assimilation workflow is orchestrated with MPAS-Workflow (https://github.com/NCAR/MPAS-Workflow).

Conflict of Interest declaration

The authors declare there are no conflicts of interest for this manuscript.

Acknowledgments

The authors thank BJ Jung and Craig Schwartz for helpful discussions. This material is based upon work supported by the NSF National Center for Atmospheric Research (NCAR), which is a major facility sponsored by the U.S. National Science Foundation under Cooperative Agreement No. 1852977. We acknowledge high-performance computing support from the Derecho system (doi:10.5065/qx9a-pg09) provided by NSF NCAR. Additional funding was provided by the United States Air Force under Grant FA8730-21-C-0008. During manuscript preparation, the authors used Claude Sonnet 5.5 (Anthropic) to improve the grammar, clarity, and flow of text written by the authors.

10.1175/1520-0493(2004)132⟨1238:IOIEAO⟩2.0.CO;2

Supporting Information for ”Ensemble Covariance Propagation with Machine Learning Weather Prediction Models in a Realistic Data Assimilation System”

S. Toedtli; D. Ahijevych; W. Wang; C. Snyder

si_template_2019.pdf

Introduction

This supplementary material provides additional details and figures to support the conclusions in the main manuscript. Texts S1–S5 describe, respectively, the concept of cycling in DA systems, the DA components, ensemble inflation, the interpolation between the GC and MPAS meshes, and the assimilated observations. The assimilated microwave sensors and channels are listed in Table S1, while Figure S1 to Figure S3 provide additional figures of ensemble spread and covariance structure.

Corresponding author: Simon Toedtli, [email protected]

Cycling in data assimilation workflows

Data assimilation for weather prediction typically employs “cycling”: DA is performed at a regular cadence, for example every six hours in this study. The inputs to the DA scheme are the most recent observations and a prior forecast from the previous cycle (background), and the output is a new initial condition (analysis). The analysis then initializes the forecast that provides the background for the next cycle.

EnVar and GETKF Components

The configuration of the DA components closely follows previous MPAS-JEDI studies: the EnVar component adopts the clear-sky configuration of Liu et al. [6], while the GETKF component follows the clear-sky configuration of Sun et al. [7]. An in-depth discussion can be found in those references, the following paragraphs highlight a few aspects that are most relevant to this study.

The EnVar component uses a purely ensemble-based background error covariance model. This is a deliberate choice, as it provides the most stringent test for the ensemble forecast models that evolve these covariances. The ensemble-based background error covariance is localized with a Gaspari–Cohn function [4], using a horizontal and vertical full-width of 1200 km and 6 km, respectively, beyond which correlations are set to zero. The 3D-EnVar cost function is solved in incremental form [2] with two outer loops and 60 inner loops, via the Derber–Rosati Preconditioned Conjugate Gradient (DRPCG) algorithm [3].

For the GETKF component, horizontal localization is performed in observation space using the R-localization approach [5] with a localization length scale of 1200 km. Vertical localization, in contrast, is performed in model space by modulating the original ensemble members with the leading eigenvectors of a localization matrix [1]. Here, a Gaspari–Cohn localization matrix with a 6 km full-width is used and the leading 9 eigenvectors are retained, which capture approximately 95% of the variance of the localization matrix.

Ensemble Inflation

Following Sun et al. [7], two posterior ensemble inflation strategies are employed in the present study: relaxation to prior perturbations (RTPP) and relaxation to prior spread (RTPS).

The RTPP scheme [11] replaces the analysis perturbations δxia\delta\mathbf{x}^{a}_{i} of member ii with a linear combination of the analysis and background perturbations, δxib\delta\mathbf{x}^{b}_{i}

δxia←αrtppδxib+(1−αrtpp)δxia(1)\delta\mathbf{x}^{a}_{i} \leftarrow\alpha_{rtpp} \delta\mathbf{x}^{b}_{i} + (1-\alpha_{rtpp}) \delta\mathbf{x}^{a}_{i} \tag*{(1)}

where αrtpp\alpha_{rtpp} is a scalar relaxation parameter.

The RTPS scheme [9], in contrast, relaxes the spread of the analysis ensemble towards the spread of the background ensemble. For each grid cell and variable, jointly indexed by jj, the spread change ratio sjs_j is calculated as

sj=σjb−σjaσja(2)s_j = \frac{\sigma^{b}_{j}-\sigma^{a}_{j}}{\sigma^{a}_{j}} \tag*{(2)}

where σa\sigma^{a} and σb\sigma^{b} are the ensemble standard deviations of the analysis and background, respectively. The analysis perturbations are then updated as

δxia←δxia∘(αrtpss+1)(3)\delta\mathbf{x}^{a}_{i} \leftarrow\delta\mathbf{x}^{a}_{i} \circ(\alpha_{rtps}s+1) \tag*{(3)}

where αrtps\alpha_{rtps} is a relaxation parameter and ∘\circ denotes the Schur product.

Both relaxation parameters are scalars and take on values between α=0\alpha= 0 (no inflation) and 1 (full replacement with the background state). Their optimal values must be determined empirically, and separate tuning experiments were performed for configurations C1 and C2.

Interestingly, the optimal parameters turned out to be identical for both configurations: αrtpp=0.5\alpha_{rtpp} = 0.5 and αrtps=0.75\alpha_{rtps} = 0.75. These settings were used for all experiments shown in the manuscript.

Initial sensitivity tests showed that the filter remains stable over a wide range of parameter settings. However, it is difficult to compare the performance of configurations C1 and C2 if their underlying ensemble spread differs or evolves differently over time. For the chosen inflation parameters, the ensemble spread of the two systems is comparable and remains approximately constant over time (see Figure S1), with one exception: during the first two days of cycling, the spread of configuration C2 initially decreases before recovering to the level shown in Figure S1. We have not investigated this initial decrease, but note that similar behavior appears to be common in MLWP-based ensemble systems (see e.g. Figs. 3 and 5 in Takeshima et al., 2025).

Interpolation Between GraphCast and MPAS Mesh

As mentioned in § 3.2 of the manuscript, all data assimilation steps are performed in MPAS-JEDI on the native MPAS mesh, regardless of the choice of ensemble forecast model. In configuration C2, this requires interpolating between the MPAS and GC meshes before and after each ensemble forecast.

The mapping from the MPAS to the GC mesh requires interpolation from MPAS’s hybrid, terrain-following vertical coordinate to GC’s constant pressure levels, including extrapolation below orography in areas of high terrain. This mapping follows ERA5 conventions (see e.g. [10]), to ensure that the interpolated analyses are consistent with the ERA5 data model used to train GraphCast.

The reverse interpolation, from the GC to the MPAS mesh, uses MPAS’s own initialization system. The interpolation method within the GC domain and extrapolation method above it are variable-specific. Because GC lacks several fields required by MPAS (skin temperature, sea surface temperature, water-equivalent snow depth, sea ice, soil moisture, and soil temperature), these are taken from the GFS analysis valid at the cycle time and combined with the GC output prior to interpolation. Similarly, GC’s static land-sea mask and terrain height are replaced with those from the GFS analysis, to avoid artifacts due to mismatches in model representation. In principle, these missing fields could instead be taken from the deterministic EnVar component, which is also evolved with MPAS in both configurations and therefore includes all of these variables. Once all fields have been interpolated to the MPAS mesh, a single short MPAS time step (dt = 1 min) is performed to compute the diagnostic variables required to evaluate the observation operators in the DA step. This time step introduces a time drift of 4 minutes per day, which could become problematic for GC-based configurations run over much longer periods. However, no apparent detrimental effect from this drift was observed over the month-long experiment presented in this manuscript.

Initial sensitivity experiments explored several model-top arrangements, including a standard 30 km MPAS model top and a lower 19.6 km MPAS model top. Negative interpolation effects were smallest with a 22.6 km model, which is just above the GC model top. This configuration is therefore used throughout the study.

Assimilated observations

The EnKF and EnVar components assimilate the same observations, which represent a substantial subset of the global observation network. These include temperature, specific humidity, and zonal and meridional wind components from aircraft and radiosondes; surface pressure; zonal and meridional wind components from satellite-derived atmospheric motion vectors; Global Navigation Satellite System Radio Occultation (GNSS-RO) bending angle profiles; and selected clear-sky channels from the Advanced Microwave Sounding Unit-A (AMSU-A) and Microwave Humidity Sounder (MHS). The assimilated microwave radiance channels are specified in Table S1. Note that AMSU-A channel 9, which has been assimilated in previous MPAS-JEDI studies, is omitted here because its weight function peaks close to the GC model top and retains nonzero support well above it.

References

  1. [1]Bishop, C. H., J. S. Whitaker, and L. Lei (2017), Gain Form of the Ensemble Transform Kalman Filter and Its Relevance to Satellite Data Assimilation with Model Space Ensemble Covariance Localization, Monthly Weather Review, 145(11), 4575–4592, doi:10.1175/MWR-D-17-0102.1.DOI
  2. [2]Courtier, P., J.-N. Thépeaut, and A. Hollingsworth (1994), A strategy for operational implementation of 4D-Var, using an incremental approach, Quarterly Journal of the Royal Meteorological Society, 120(519), 1367–1387, doi:10.1002/qj.49712051912.DOI
  3. [3]Derber, J., and A. Rosati (1989), A Global Oceanic Data Assimilation System, Journal of Physical Oceanography, 19(9), 1333–1347, doi:10.1175/1520-0485(1989)019⟨1333:AGODAS⟩2.0.CO;2.doi.org/10.1175/1520-0485(1989)019<1333:agodas>2.0.co;2
  4. [4]Gaspari, G., and S. E. Cohn (1999), Construction of correlation functions in two and three dimensions, Quarterly Journal of the Royal Meteorological Society, 125(554), 723–757, doi:10.1002/qj.49712555417.DOI
  5. [5]Hunt, B. R., E. J. Kostelich, and I. Szunyogh (2007), Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter, Physica D: Nonlinear Phenomena, 230(1-2), 112–126, doi:10.1016/j.physd.2006.11.008.DOI
  6. [6]Liu, Z., C. Snyder, J. J. Guerrette, B.-J. Jung, J. Ban, S. Vahl, Y. Wu, Y. Trémolet, T. Auligné, B. Ménétrier, A. Shlyaeva, S. Herbener, E. Liu, D. Holdaway, and B. T. Johnson (2022), Data assimilation for the Model for Prediction Across Scales – Atmosphere with the Joint Effort for Data assimilation Integration (JEDI-MPAS 1.0.0): EnVar implementation and evaluation, Geoscientific Model Development, 15(20), 7859–7878, doi:10.5194/gmd-15-7859-2022.DOI
  7. [7]Sun, T., J. J. Guerrette, Z. Liu, J. Ban, B.-J. Jung, I. Hernandez Banos, and C. Snyder (2025), All-sky AMSU-A radiance data assimilation using the gain-form of Local Ensemble Transform Kalman filter within MPAS-JEDI-2.1.0: implementation, tuning, and evaluation, Geoscientific Model Development, 18(22), 8569–8587, doi:10.5194/gmd-18-8569-2025.DOI
  8. [8]Takeshima, A., K. Shiraishi, A. Okazaki, T. Tsuyuki, and S. Kotsuki (2025), Bridging Artificial Intelligence and Data Assimilation: The Data-driven Ensemble Forecasting System ClimaX-LETKF, doi:10.48550/arXiv.2512.14444, arXiv:2512.14444 [cs.LG].DOI
  9. [9]Whitaker, J. S., and T. M. Hamill (2012), Evaluating Methods to Account for System Errors in Ensemble Data Assimilation, Monthly Weather Review, 140(9), 3078–3089, doi:10.1175/MWR-D-11-00276.1.DOI
  10. [10]Yessad, K. (2019), FULL-POS in the cycle 46t1r1 of ARPEGE/IFS, Tech. rep., Meteo France.
  11. [11]Zhang, F., C. Snyder, and J. Sun (2004), Impacts of Initial Estimate and Observation Availability on Convective-Scale Data Assimilation with an Ensemble Kalman Filter, Monthly Weather Review, 132(5), 1238–1253, doi:10.1175/1520-0493(2004)132⟨1238:IOIEAO⟩2.0.CO;2.doi.org/10.1175/1520-0493(2004)132<1238:ioieao>2.0.co;2

Paper details

vlnc:W_28zktjfc
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Claude Sonnet 5.5

View statement
Looking for related papers…
Contents