In this paper
AbstractIntroductionFESOM2Single precision in FESOM2A model-wide precision parameterOperations kept in double precisionConservative tracer and layer-thickness updatesSalinity representationClimate fidelity over six decadesExperiments and double-precision controlsSingle precision with conservative updatesSingle precision without conservative updatesPerformanceCORE2 hindcastsScaling across meshes and hardwarePrecision sensitivity of an independent implementationDiscussionWhen is reduced precision acceptable?Why short evaluations miss the drift, and how to avoid itReduced precision as a verification toolSingle precision in a coupled modelSingle precision as the default working precisionOutlookConclusionsMesh characteristicsScaling configurationsSingle precision before the conservative updatesGlobal budgets and sea iceSource of the salinity driftMean-state differences and observational biasesCoupled integrationsReferencesWorksPhysics and Astronomy
FESOM2.8 in single precision: climate fidelity over six decades and performance on CPUs and GPUs
Abstract
Halving the floating-point precision of an ocean model can make it substantially faster, but whether the rounding errors stay negligible over climate time scales has rarely been tested. We implement single precision in the global ocean-sea-ice model FESOM2 and compare 60-year hindcasts with double-precision controls. A first implementation lost salt steadily, accumulated a heat-content deficit of 129 ZJ, and shifted major ocean fronts. Controlled experiments trace the drift to the tracer update: small increments added to a large salinity value are rounded away, and the increments themselves are biased. Accumulating tracer tendencies and layer thicknesses in 64 bits, and carrying the rounding error of each tracer update into the next, removes the drift for at most 8% additional runtime. Single- and double-precision simulations then differ in heat content and mean state about as much as two double-precision runs with slightly perturbed initial conditions, sea ice differs by at most 0.2%, and he salinity difference is four orders of magnitude smaller than before. In the coupled model AWI-ESM3, single precision adds no detectable heat source or sink over 60 years. Single precision speeds up the 60-year hindcasts by a factor of 1.35, and the model by up to 1.66 on meshes of up to 5 km resolution; on GPUs, a C++/Kokkos version runs up to 1.27 times faster and needs 43% less memory. Whether reduced precision is safe for climate integrations is decided by long-term budgets, not by short tests.
Introduction
Most ocean models compute in 64-bit floating-point arithmetic, although their discretization, forcing, and parameterization errors exceed the relative rounding error of 32-bit arithmetic, about , by orders of magnitude. The case for 32 bits is one of data movement. Kernels that perform few arithmetic operations per byte they load are limited by memory bandwidth rather than by peak arithmetic throughput [83], and ocean-model kernels are of this kind. Memory has lagged processor speed for three decades [84], and peak arithmetic throughput has outgrown DRAM bandwidth over the last two [22]. Storing a field in 32 instead of 64 bits halves the bytes it moves and the memory it occupies, on CPUs and GPUs alike, and a vector instruction processes twice as many values. Numerical weather prediction has taken this step: the ECMWF Integrated Forecasting System runs its forecasts in single precision with no measurable loss of skill and about 40 % lower cost (Văňа et al., 2017; Lang et al., 2021). Reduced-precision emulation has identified the components that need more precision [19, 17, 34], and single- and mixed-precision dynamical cores run in global atmospheric models [51, 48, 9].
Ocean models have followed more slowly [61]. Mixed-precision versions of NEMO have been derived with emulation-based searches [76, 57], and ECMWF now runs its NEMO ocean–sea-ice forecasts in single precision while keeping double precision for the ocean analysis [32, 60]. The caution has a reason. The ocean integrates small fluxes over decades, and a rounding bias that is invisible in a forecast can accumulate in the heat and salt content. Published evaluations of reduced precision in ocean models cover days to a few simulated years [76, 26, 57], but longer tests show that such drifts occur: single-precision land-surface models underestimate the slow warming of deep soil [27, 18], and single-precision tracer advection in a coarse-resolution ocean model shifted zonal-mean temperature by up to over 300 years [50]. To our knowledge, the heat and salt budgets of an ocean–sea-ice model run entirely in single precision have not been evaluated over multiple decades, nor has the drift been traced to its source and removed.
Here we implement single working precision in FESOM2, an unstructured-mesh global ocean–sea-ice model [16], and ask whether it can be trusted for climate integrations and what it gains. By single precision we mean the model’s default working precision; a small number of operations are kept in 64 bits (Sects. 3.2 and 3.3), so the implementation is mixed precision in the strict sense, with 32 bits as the default. Sixty-year hindcasts, judged against double-precision runs that differ only in round-off or in tiny initial perturbations, show that a straightforward implementation drifts in salinity and heat content. We trace the drift to the way tracers are updated and describe the changes now in FESOM2 that remove it, after which single and double precision differ in the simulated climate not more than two double-precision runs. Storing salinity relative to a reference value, as early versions of the Modular Ocean Model did [56], reduced the drift but is no longer needed. We then measure the speedup on four meshes from nominal to 5 km resolution, for the Fortran model on CPUs and for an independent C++/Kokkos implementation on GPUs, and use the comparison of the two implementations as a further test of numerical consistency. Finally, we repeat the budget test in the coupled model AWI-ESM3.
FESOM2
FESOM2 solves the hydrostatic, Boussinesq primitive equations on an unstructured triangular mesh using a cell-vertex finite-volume discretization [16]. Temperature, salinity, pressure, and sea-surface height are defined at mesh vertices; horizontal velocity is defined at triangle centroids. We use the form of the arbitrary Lagrangian–Eulerian (ALE) vertical coordinate [2]. The single-class dynamic–thermodynamic sea-ice model is based on FESIM [15], shares the ocean’s scalar grid, and uses an elastic–viscous–plastic (EVP) rheology. Previous studies describe the model and its evaluation [66, 67] and computational scalability [39]. FESOM2 is the ocean component of the AWI climate model [74]; here it is run with prescribed atmospheric forcing, except in Sect. 7.4.
The climate experiments (Sect. 4) use the CORE2 mesh (126 858 surface vertices, 47 layers, nominal resolution) and CORE-II interannual forcing for 1948–2007 [44, 14]. The configuration uses K-profile parameterization (KPP) for vertical mixing [45], Gent–McWilliams eddy parameterization and Redi isoneutral diffusion [21, 63], backscatter-augmented viscosity [29], a 900 s timestep, and initial conditions from PHC3.0 [73]. The timestep is half the 1800 s used in production with this mesh, which makes the test harder: the loss of small increments grows as the timestep shrinks (Sect. 7.2). Performance measurements (Sect. 5) span four meshes from CORE2 to the 5-km-class NG5 mesh (Appendix § 9). These measurements and the implementation comparison (Sect. 6) use JRA55-do forcing [78].
Single precision in FESOM2
A model-wide precision parameter
FESOM2 declares its real variables with one kind parameter, . A build option now sets it to 4 bytes and adjusts the compiler flags and the MPI datatype of halo exchanges and reductions accordingly; the default double-precision build is bit-identical to the code before the change. Single precision thus covers the prognostic state, the mesh-derived operators, and the ocean and sea-ice timestepping, with the exceptions described below. It was first released in FESOM 2.8.0 and changed 36 model-source files, mostly literal kinds, interfaces, and input–output declarations that assumed 64-bit reals. It is supported for standalone runs and for coupling to OpenIFS through OASIS; other coupled configurations and the REcoM biogeochemistry model, whose precision parameters have not been converted, are rejected at build time. The model reports its working precision and source revision at startup, and continuous integration builds both precisions.
Operations kept in double precision
A few operations stay in double precision because rounding them would bias the result or cost accuracy for no gain in speed. Input and restart files remain in double precision and are converted on reading. Time means for output are accumulated in 64 bits, because summing thousands of 32-bit samples can bias slowly varying fields such as salinity.
Global sums that close the surface budgets are accumulated and reduced in 64 bits. They add – terms whose small residual results from large cancelling contributions, and global sums are the operations most sensitive to rounding and summation order in parallel climate codes (He and Ding, 2001). The sea-surface-height solver likewise accumulates its conjugate-gradient scalars in 64 bits while keeping vectors and matrices in single precision, as the Krylov solver of the ENDGame dynamical core does [48]. Under the coordinate, the solver’s matrix changes each step by – of its entries, close to the 32-bit relative spacing of ; these changes are accumulated in a 64-bit copy and rounded once per step. Single precision also uses a symmetric Jacobi preconditioner by default, which reduces the number of solver iterations by a third.
The CVMix library [25] has fixed 64-bit interfaces, so its mixing schemes remain in double precision; this study uses FESOM2’s own KPP scheme, which follows the working precision. Finally, the guards against division by zero in KPP are subnormal in 32 bits and were flushed to zero by optimizing compilers, producing 0/0 at the first step; they are now .
Conservative tracer and layer-thickness updates
FESOM2 updates tracers in flux form on an arbitrary Lagrangian–Eulerian vertical grid. The tracer content of a control volume with layer thickness changes by the divergence of the advective and diffusive fluxes,
and each flux through an edge or a layer interface enters the two adjacent control volumes with opposite signs, so that in exact arithmetic the global content changes only through the boundary fluxes. Three roundings break this in single precision. First, a product of flux and timestep that is rounded before it is added to the two control volumes no longer cancels between them. Second, rounding the new state changes the content it represents: at psu, adjacent 32-bit values are psu apart, and a smaller increment is partly or entirely lost. Third, rounding the new layer thickness at every step changes the volume of each control volume, which violates volume conservation and, through , tracer conservation.
A defect of the first kind was found early. The flux-corrected transport (FCT) scheme obtained its low-order tendency as the difference of two states, , which loses small tendencies in single precision. It now computes the tendency from the flux divergence, in single precision since FESOM 2.8.0 and in both precisions since 2.8.1. In five-day coupled experiments, the defect had produced a global heat imbalance of to W m in single precision, compared with at most 0.01 W m in double precision.
The current code avoids all three roundings. Tracer tendencies are accumulated in 64 bits, so that fluxes cancel between control volumes; this includes the advective and diffusive tendencies, the low-order FCT tendency, and the column arrays of the implicit vertical advection and diffusion solvers. The new state is computed in 64 bits, and its rounding error, weighted by the new thickness, is kept in one additional 32-bit array per tracer, , and added at the next update:
where is evaluated in 64 bits and rounds to the nearest 32-bit value. The content represented by the state and the remainder, , thus equals the 64-bit content up to the rounding of the small remainder itself. This is compensated summation ([30]) applied to the thickness-weighted tracer content; the implicit vertical diffusion, which updates the state a second time in each step, updates in the same way. Layer thicknesses are stored in 64 bits. Their halo exchange sends each value as a 32-bit number and its 32-bit remainder, so communication remains in single precision, and output and restart files receive 32-bit copies. Whereas an automatic precision search for NEMO kept tracer advection entirely in double precision [57], FESOM2 keeps its tracer fields and operators in single precision and holds only the tendencies and the thicknesses in 64 bits.
In double precision, these quantities are 64-bit anyway and is not used, so results are unchanged. In single precision, the conservative updates lengthen a timestep by at most 8 % (Sect. 5.2). The remainder is not written to restart files, so each restart discards less than one 32-bit spacing per control volume; in the experiments of Sect. 4.2, this changes the global heat content by at most of its value per restart.
Salinity representation
A runtime option stores salinity as a deviation from a reference value, psu. The Modular Ocean Model timestepped “for purposes of numerical accuracy” ([56]), and atmospheric models use deviations from a reference state for similar reasons ([71] (Rüdisühli et al. 2014)). Over most of the ocean, psu, so representable values are about thirty times closer together than at 35 psu. Without the conservative updates, this representation was the main remedy for the drift of single precision (Sect. 4.3); with them, it no longer changes the results (Sect. 4.2).
The equation of state, the KPP buoyancy flux, the ice–ocean coupling, and other operations that need absolute salinity add the reference value back, and output remains in absolute salinity. Under the nonlinear free surface, freshwater also dilutes the reference part, which enters the surface boundary condition for as a virtual salt flux with constant salinity ([24]); Pacanowski and Griffies [56] describe the sign error that arises without it. Configurations whose salinity operations have not been adapted, such as ice-shelf cavities and icebergs, stop at setup when the option is enabled. In double precision, the option changes results only through round-off, which makes it a control for the climate experiments.
Climate fidelity over six decades
Experiments and double-precision controls
We judge single precision by whether its differences from double precision exceed those between double-precision runs that differ only in round-off or in tiny initial perturbations. Such comparisons are established in port validation and consistency testing ([64, 4, 47, 86]) and have been applied to single precision in a regional climate model ([5]). All experiments are 60-year hindcasts for 1948–2007 in the CORE2 configuration of Sect. 2, on 512 processes and with the same solver preconditioner in both precisions.
The main experiments use the current code with the conservative updates of Sect. 3.3: two single-precision runs, one with absolute salinity and one with the salinity anomaly, and two double-precision runs, one of them started from temperatures perturbed by K. The difference between the two double-precision runs is our measure of chaotic divergence. The anomaly run is compared with the unperturbed absolute-salinity double-precision run, as the two representations differ only in round-off in double precision. The model records its global volume, heat, and salt budgets—content, surface fluxes, and residual—every simulated day.
An earlier set of hindcasts with the first implementation (Sect. 4.3) has its own double-precision control: two runs that differ only in the salinity representation, and hence only in round-off. The chaotic divergence is similar in both sets: the 1988–2007 mean sea-surface temperature (SST) of the double-precision runs differs by 0.0040 RMS in the current code and by 0.0046 in the earlier one. Global integrals use fixed reference layer thicknesses; the neglected surface-height correction is less than of the volume.
Single precision with conservative updates
With conservative updates, single precision is indistinguishable from double precision in heat content, temperature, and mean state (Figure 1). Over 60 years, the heat-content difference stays within 1.5 ZJ, against 1.0 ZJ between the double-precision runs, whereas the first implementation lost 129 ZJ. Sea ice differs slightly more than between the double-precision runs: the annual volume and area of the Arctic and Antarctic ice differ by at most 0.2 %, against 0.07 % (Figure 1f). Separating the heat-content difference into surface supply and internal residual (Appendix § 11.1) confirms that no heat is created or destroyed numerically: the internal residual is at most , compared with a sink of in the first implementation with the salinity anomaly.
Figure 1. Single minus double precision in the 60-year hindcasts before (first implementation) and after the conservative updates of Sect. 3.3. (a–c) Annual differences in volume-mean salinity, ocean heat content, and Arctic sea-ice volume; gray lines are differences between two double-precision runs: salinity anomaly minus absolute salinity before (dark), perturbed minus unperturbed initial temperatures after (light). (d–f) The runs after the updates and the double-precision differences on a finer scale; note the unit in (d). (g–i) 1988–2007 mean SST difference before (g) and after (h) the updates, and between the two double-precision runs after them (i); (h) and (i) share a color scale ten times finer than that of (g).
Salinity is the one quantity in which single precision remains distinguishable. It gains salt at a nearly constant rate, reaching psu in the last decade (Figure 1d), about 14 times the RMS difference between the double-precision runs but 14000 times less than the first implementation lost. The model’s own budget shows that this salt enters through the surface: the surface salt flux supplies all but 0.4 % of the difference in salt content, and the volume and salt budgets close to within of the content, compared with in double precision. We have not identified why single precision takes up slightly more salt at the surface.
The 1988–2007 mean state shows the same picture (Figure 1g–Figure 1i). The single-minus-double-precision SST difference has the amplitude and pattern of the difference between the double-precision runs, an RMS of 0.0046 against 0.0040 , concentrated in the tropical Pacific and the Gulf Stream. It amounts to 0.7 % of the model’s error against HadISST1 [62]; for surface salinity, the corresponding figure is 0.4 % of the error against the World Ocean Atlas 2009 [3]. In the first implementation, the differences were 36 and 30 times the control.
Storing salinity as an anomaly no longer makes a measurable difference. The salinity gain is the same, against psu, and heat content and mean state again differ about as much as the two double-precision runs.
Single precision without conservative updates
The first implementation, without conservative updates, behaved differently (Figure 1a–Figure 1c, Figure 1g; Appendix § 11). With absolute salinity, it lost salt at and accumulated a heat-content deficit of 129 ZJ, mostly through altered surface fluxes; its mean state shifted along the major fronts, and Arctic sea ice thickened. Controlled experiments ruled out the surface freshwater budget, the global sums, the solver, and the domain decomposition, and traced the drift to the tracer update itself. Keeping the state in 64 bits while computing the increments in 32 bits produced a drift of similar size and opposite sign, which showed that the increments, not only their addition to the state, were biased. Storing salinity as an anomaly reduced the drift by a factor
of 32 and the heat-content deficit by 93 %, but left a salinity gain and an internal heat sink of 0.02 W m. In nine-month experiments, compensated summation of the update alone did not remove these residuals; the conservative updates, which also compute the increments and the layer thicknesses in 64 bits, do (Sect. 4.2).
Performance
CORE2 hindcasts
In the 60-year hindcasts of Sect. 4.2, run on four nodes with full output, single precision raises the throughput from 52 to 70 simulated years per day, a speedup of 1.35; the first implementation, without conservative updates, reached 1.41. Storing salinity as an anomaly costs 0.4 % of the runtime.
The gain comes from the ocean (Figure 2). In one simulated month on one node, where single precision runs 1.56 times faster overall, the ocean component gains a factor of 1.76 and its tracer calculations a factor of 1.85, whereas sea ice gains only a factor of 1.14. This is the pattern expected if the ocean timestep is limited by memory traffic, which single precision halves, while the sea-ice subcycles, with short loops and frequent synchronization, are limited by latency.
Figure 2. Component runtimes for one simulated month of the CORE2 hindcast configuration on one node (128 processes) in double and single precision, each the faster of two runs, which differ by at most 0.09 %. Labels give the component speedups.
Scaling across meshes and hardware
To see how the gain depends on resolution and hardware, we measure strong scaling on four meshes, from CORE2 to the 5-km NG5 mesh (Appendix § 9), for the Fortran model on CPUs and for the C++/Kokkos implementation (Sect. 6) on NVIDIA A100 GPUs of the DKRZ Levante system. Each measurement runs double precision, single precision, and single precision without conservative updates for 300 timesteps, twice each in mirrored order; repeated runs differ by a median of 0.3 % (Appendix § 10).
The speedup depends mainly on how much data each process holds (Figure 3). It is largest when the working set of each process far exceeds the cache and shrinks as the working set fits into it. On CORE2, the Fortran speedup falls from 1.66 on half a node to 1.02 on 4096 processes. The eddying DARS and NG5 meshes retain speedups of 1.40–1.64 up to 64 nodes and 1.27–1.40 at 128 nodes; at 256 and 512 nodes, measured with an earlier version without conservative updates, the speedup is 1.18–1.37. The conservative updates cost at most 8 % where computation dominates and less than 1 % at the largest process counts.
Figure 3. Strong scaling in double (solid) and single (dashed) precision for Fortran on CPUs and C++/Kokkos on GPUs. Each node has 128 CPU cores or four A100 GPUs; the 64-process CORE2 point uses half a CPU node. Open symbols: an earlier Fortran version without the conservative updates of Sect. 3.3. (a) Time per step. (b) Single-precision speedup.
On GPUs, the Kokkos implementation gains less, a factor of 1.05–1.27, largest on DARS and smallest on CORE2, where runs on more than two nodes are limited by latency. Single precision also reduces device memory use by 40–43 %, so that NG5 on 16 GPUs needs 21.6 instead of 35.9 GiB per device. On CORE2, one GPU node runs 2.6 times faster than one CPU node with the Fortran model in double precision and 2.0 times faster in single precision, which gains more on CPUs.
For production use, these gains translate into throughput (Figure 4 and Figure 5). At the fastest measured configurations, single precision raises the projected Fortran throughput on DARS from 18 to 22 simulated years per day on 256 nodes and on NG5 from 8.8 to 10 on 512 nodes, both measured without conservative updates; Kokkos on GPUs reaches 8.4 simulated years per day on DARS with 32 nodes. These projections exclude output. Configurations that use the CVMix library, which remains in double precision, may gain less.
Figure 4. Projected throughput in simulated years per day for CORE2 (a) and fArc (b) in double (solid) and single (dashed) precision, for Fortran on CPUs (circles) and Kokkos on GPUs (squares). CORE2 uses an 1800 s timestep; fArc throughput is projected from the 900 s benchmark timestep to the 1200 s production timestep.
Figure 5. As Figure 4, for DARS (a) and NG5 (b), projected from benchmark timesteps of 120 and 180 s to a 240 s production timestep. Open symbols: an earlier Fortran version (see Figure 3).
Precision sensitivity of an independent implementation
FESOM2 is also being developed in C++/Kokkos (Trott et al. [77]) for GPUs, verified kernel by kernel against a C reference that reproduces the long-term statistics of the Fortran model (Koldunov et al. [40]), and in JAX for differentiability, verified kernel by kernel against the Fortran model (Koldunov et al. [41]). An independent implementation also allows a test that double-precision verification cannot provide: whether reducing precision changes two implementations of the same equations in the same way. We compare one-year CORE2 integrations with JRA55-do forcing for 1958, using a Kokkos version without conservative updates. Because the double-precision trajectories of the two implementations diverge chaotically, each is compared with itself: the precision effect is the relative distance between its single- and double-precision December fields, measured against a baseline from five double-precision runs with initial perturbations of K (Table 1).
| Field | Kokkos / Fortran | Fortran / baseline | Kokkos / baseline | Kokkos / Fortran, anomaly |
| SST | 1.6 | 2.4 | 0.35 | |
| temperature | 1.5 | 2.4 | 0.32 | |
| salinity | 1.5 | 2.6 | 0.38 | |
| sea-surface height | 2.7 | 7.0 | 0.33 | |
| ice concentration | 2.4 | 4.1 | 1.4 |
Table 1. Relative distances between single- and double-precision December-1958 fields after one year. Each implementation uses its own double-precision reference. The first numerical column gives the ratio of mean distances, with uncertainty propagated from the standard deviations of four single-precision members per implementation. The next two columns normalize each implementation’s mean distance by its double-precision variability baseline. The last column gives the Kokkos-to-Fortran ratio with the salinity anomaly enabled.
For global temperature and salinity, the precision effect in Kokkos is 0.65–0.83 of that in Fortran, and on GPUs it agrees with the CPU backend within 12 %. Reaching this agreement exposed two defects that double-precision verification had missed: an uninitialized Adams–Bashforth tracer history, and Redi/GM increments added directly to salinity instead of to the accumulated tendency, which rounded them at the spacing of psu. Correcting the latter halved the precision effect, as the advice to form the complete increment before adding it to the state suggests (Croci and Giles [11]; Klöwer et al. [35]). Larger differences remain for sea-ice concentration and between 60 and 70°S, where the Kokkos effect is 2.6 times the Fortran one for SST and 1.9 times for salinity. With the salinity anomaly, these ratios fall to 1.2 and 1.1, which points to rounding of absolute salinity under Antarctic sea ice.
Discussion
When is reduced precision acceptable?
Reduced precision changes a simulation in two ways. Rounding perturbations grow chaotically, so single- and double-precision runs follow different trajectories, which alone does not degrade a climate simulation. A biased numerical source or sink, in contrast, accumulates. Global budgets tell the two apart: salt changes only through the surface, so a salinity difference beyond the controls that is not supplied by the surface flux is numerical, and the heat budget separates a numerical heat source from the response of the surface fluxes. We therefore propose two criteria. Conserved quantities should show no numerical source or sink beyond the spread of the controls over the length of the intended integration, and the simulated climate, including its mean state and variability, should differ from double precision by much less than the model differs from observations; individual trajectories may diverge.
The first implementation failed the first criterion even where it met the second: with the salinity anomaly, its mean-state differences were only 0.028 and 0.021 of the errors in SST and surface salinity, yet it gained salt and lost heat internally. With conservative updates, both criteria are met over 60 years at nominal resolution (Sect. 4.2). The remaining salinity difference exceeds the controls but enters through the surface flux; whether a difference of this size matters depends on the application, for example passive-tracer budgets over centuries.
Why short evaluations miss the drift, and how to avoid it
Several factors may explain why such drifts have not been reported for ocean models. Some earlier models stored salinity relative to a reference value (Sect. 3.4). Mixed-precision designs keep the prognostic state in double precision; the single-precision build of NEMO 4.2, for example, declares temperature, salinity, velocities, and sea-surface height in double precision (The NEMO team, [75]). This removes the rounding of the update but, as our experiments show, not necessarily the bias of increments computed from absolute values (Sect. C2). Evaluations have been short and have compared fields or forecast skill rather than budgets. In budgets, the drift is visible within the first model year (Sect. C1), and a fully single-precision NEMO configuration gained heat and salt relative to double precision within a month (Tintó Prims et al., [76]), but the climatic consequences develop over decades, and the surface fluxes can hide a heat loss for years.
The underlying problem is general: small increments to a large background value are poorly resolved in reduced precision, as documented for soil temperatures in kelvin (Harvey and Verseghy, [27]; Dawson et al., [18]), tracer advection in an ocean model (Nakano et al., [50]), and 16-bit shallow-water models (Klöwer et al., [34]). Because the increment per step scales with the timestep while the spacing of representable values does not, the error grows as the timestep shrinks (Harvey and Verseghy, [27]; Klöwer et al., [35]), so eddy-resolving configurations with timesteps of a few minutes would be more exposed than our CORE2 hindcasts with their 900 s timestep. A reference value reduces the problem, as it did here by a factor of 32. Compensated summation of the update (Kahan, [30]; Møller, [49]; Klöwer et al., [35]; Lai et al., [42]) matched a double-precision state over 300 years in an ocean advection scheme (Nakano et al., [50]), and stochastic rounding (Croci et al., [13]; Paxton et al., [58]; Kimpson et al., [33]) is unbiased in expectation but needs hardware support or emulation. Compensated summation alone, however, corrects only the rounding of the update and leaves biased increments (Sect. C2), and the Kokkos comparison shows that where increments are accumulated matters as well. FESOM2 therefore combines compensated updates with 64-bit tendencies and layer thicknesses, after which the representation of salinity no longer matters. The same structure, a small increment added each step to a large value, appears in dissolved inorganic carbon and alkalinity and in the solver matrix under the coordinate (Sect. 3.2). How prognostic variables are stored and updated is part of reduced-precision model design, not a compiler setting.
Reduced precision as a verification tool
Reduced precision amplifies errors that double precision masks, in particular cancellations and the accumulation of small increments. Here it exposed an uninitialized Adams–Bashforth history and Redi/GM increments added directly to salinity in the C++/Kokkos implementation (Sect. 6), the recovery of the FCT tendency as a difference of two states (Sect. 3.3), and subnormal guards in KPP (Sect. 3.2). The first also affected double precision; the others are specific to reduced precision. Comparing how independent implementations respond to reduced precision, together with long-term budgets, is therefore a sensitive check for models ported to new languages and hardware.
Single precision in a coupled model
Under prescribed atmospheric forcing, the surface fluxes respond to the ocean only through the bulk formulae. In a coupled model, the atmosphere can compensate a numerical heat sink in the ocean, or amplify it. We therefore repeated the comparison in the coupled model AWI-ESM3, in which FESOM 2.8.3 on the CORE2 mesh exchanges fluxes with the atmosphere model OpenIFS and the vegetation model LPJ-GUESS (Appendix § 12). Four 60-year pre-industrial control integrations differ only in the precision of FESOM2 and in a perturbation of the initial ocean temperature by K. OpenIFS runs in single precision, its default, in all four.
The budgets close as in the forced runs. The heat that the model’s own fluxes do not account for is W m in all four runs and does not depend on precision: single and double precision differ by W m, the perturbed runs from their references by in double and W m in single precision. In single precision, the volume and salt residuals remain below of the content. The FCT defect (Sect. 3.3) would have shown up here as a sink of to W m.
The coupled climate is a weaker test, because it is far noisier. With a freely evolving atmosphere, the heat content of two runs from almost the same state drifts apart by up to 54 ZJ within decades, against 1.0 ZJ under prescribed forcing. Within this spread, single and double precision are indistinguishable in heat content, global-mean surface temperatures, the radiation balance at the top of the atmosphere, and sea ice (Table D1).
| SP − DP | DP twin − DP | SP twin − SP | |
| internal heat residual (W m) | |||
| Heat content (ZJ) | +24 | +22 | −32 |
| Global-mean SST (K) | +0.11 | +0.05 | +0.01 |
| Global-mean 2 m temperature (K) | +0.22 | +0.14 | +0.03 |
| Net radiation at the top (W m) | −0.20 | −0.15 | +0.10 |
| Arctic sea-ice volume (%) | −22 | −26 | +0.2 |
| Antarctic sea-ice volume (%) | +5 | +13 | −5 |
Table D1. Coupled AWI-ESM3 integrations: differences between single and double precision and between the perturbed runs and their references. The internal heat residual is the 60-year mean of the part of the heat-content change not supplied by the surface fluxes. All other entries are last-decade (1900–1909) annual means.
Single precision speeds up the tracer calculations of FESOM2 by a factor of 1.8 but its sea ice only by 1.07, the same contrast as in the forced runs (Sect. 5.1). With an unchanged layout, the coupled model runs only 1.25 times faster, because FESOM2 then spends 20 instead of 12% of its time waiting for the atmosphere. We expect the larger gain from giving the ocean fewer processes at the same throughput.
Single precision as the default working precision
The results support making single precision the default working precision of an ocean model, with double precision reserved for the operations that need it. Three properties of current hardware make this the right default rather than an optional optimization.
First, ocean models are limited by memory traffic, not by arithmetic. Kernels that perform few floating-point operations per byte loaded are bounded by memory bandwidth long before peak arithmetic throughput [83], and ocean-model kernels are of this kind. Storing a field in 32 instead of 64 bits halves the bytes it moves and the memory it occupies, and doubles the fraction of the working set that fits in cache. This is the gain that the component timings show (Sect. 5.1): the tracer calculations, which stream large arrays, gain a factor of 1.85, whereas sea ice, limited by latency, gains 1.14.
Second, where arithmetic does matter, 32-bit arithmetic is at least twice as fast. A vector register holds twice as many 32-bit values, so a vectorized loop on the AMD EPYC processors used here processes eight instead of four values per instruction. On the A100 GPUs used here, the ratio of 32- to 64-bit throughput is two [52]; on NVIDIA’s consumer and workstation GPUs of the Ada generation it is 64 [54]; and on NVIDIA’s Blackwell Ultra 64-bit arithmetic was cut outright, to about 1.2 TFLOP per GPU against 37 on its predecessor, while low-precision matrix throughput grew; the announced Rubin generation restores 33 TFLOP , at a 32- to 64-bit ratio of about four (NVIDIA, 2026). On Blackwell Ultra, single precision is not a speedup but the condition for running at all.
Third, these gains address different bottlenecks and do not multiply. A kernel that is bandwidth-bound before conversion remains bandwidth-bound after it, so the speedup of a memory-bound model is close to two at best, and smaller where latency, integer index arrays, retained 64-bit arrays, or synchronization dominate. That is what we measure: 1.02–1.66 on CPUs and 1.05–1.27 on GPUs. The matrix units of recent processors, such as GPU tensor cores and the Arm and x86 matrix extensions, offer far more than twofold throughput for reduced precision, but only for dense matrix products, which the finite-volume loops of an ocean model do not contain, and partly in formats such as TF32 that carry fewer significand bits than IEEE single precision. They matter for solvers and data-driven components (Sect. 7.6), not for the present implementation.
The trend strengthens the argument. Memory has lagged processor speed for three decades [84], and peak arithmetic throughput has outgrown DRAM bandwidth over the last two [22]; between the two GPU generations in use at DKRZ, 64-bit throughput grew 3.5-fold while memory bandwidth grew by a factor of 1.6–2.2 [52, 53]. For a model limited by bytes rather than operations, the number of bytes per value is the one lever that keeps pace with the hardware, and reduced precision makes a larger share of a machine’s capability usable for the same allocation.
Single precision in this sense is mixed precision. The working precision sets the default; the operations that keep 64 bits (Sects. 3.2 and 3.3) are the exceptions, chosen because rounding them creates a bias, and the conservative updates show how few they need to be. Mixed-precision ocean models have so far been derived from 64-bit code by a search over what can be demoted [76, 57], and the single-precision build of NEMO keeps the prognostic state in 64 bits [75]. Holding the state in 32 bits as well captures the memory gain for the whole model and makes the cost of every exception explicit. The recommendation is conditional on validation: the budget diagnostics of Sect. 7.1 decide whether a configuration, resolution, or tracer needs an exception, and the exceptions remain part of the model’s design, documented and tested, not left to the compiler.
Outlook
The speedups found here are of the size reported for NEMO, with about 30 % less time per step in mixed precision [57] and up to 1.8 in single precision, diminishing near the scaling limit [32], and for Veros on GPUs [26]. At kilometre scale, the smaller memory footprint on GPUs may matter as much as the speedup, because it leaves room for larger meshes, ensembles, or additional Earth-system components.
Separating the tracer update into tendency accumulation, layer geometry, and the conservative state update (Sect. 3.3) has made explicit where in the data flow precision matters, and the model-wide kind parameter lets each part be assigned its own. This opens the parts of the model that tolerate less than the working precision. The first candidate is the preconditioner of the sea-surface-height solver: its errors are corrected by the outer iteration as long as residuals and convergence tests are evaluated accurately, and half-precision preconditioning has been demonstrated for geophysical elliptic solvers [1]. For the conjugate-gradient solver of FESOM2 this requires a preconditioner that remains symmetric positive definite, or a flexible variant of the iteration. The second is time integration: mixed-precision Runge–Kutta methods evaluate the costly stages, or all but the few that set the accuracy, in lower precision and retain the order and stability of the scheme [23, 12]. Both must be judged by the budget diagnostics of this paper, and the second meets the small-increment problem head-on, because substeps shrink the increments further (Sect. 7.2). Beyond their own gains, these steps are what allow physics-based models to use processors whose reduced-precision units grow while their 64-bit units do not (Sect. 7.5), and to remain computationally competitive with data-driven climate models, which run in 16- or 32-bit precision on hardware built for them.
Reduced precision also changes the balance between local computation and communication. For a fixed problem size, faster local execution brings the model into the latency-dominated regime at a smaller process count: single precision reduces the volume of the halo exchanges but not their number, their start-up latency, or the number of synchronizations, which is why its speedup falls toward one at the largest process counts (Figure 3b) and why sea ice gains least. Realizing the full benefit therefore calls for aggregating compatible halo exchanges into fewer messages, overlapping them with independent computation, and reducing the synchronizations in the iterative solver. An earlier scaling limit for a single integration does not limit throughput, however. Once additional processes no longer shorten an integration, the same resources run more concurrent ensemble members, each at an efficient process count. ECMWF likewise reinvested the savings of single precision in its ensemble, there in vertical resolution [43]. Climate applications that need ensembles to characterize internal variability and the probabilities of extremes gain in simulated years per node-hour even where communication prevents a single run from going faster.
The conservative updates have been evaluated over 60 years at nominal 1° resolution only. Eddy-rich and kilometre-scale configurations combine shorter timesteps with stronger intrinsic variability, which can mask precision errors [59, 46, 50]. Century-scale conservation and passive tracers remain untested. The biogeochemistry model REcoM still declares its own 64-bit working precision, so FESOM2 builds that include it, such as the carbon-cycle configuration of AWI-ESM3, stay in double precision. Making REcoM follow the working precision is the next step; its dissolved inorganic carbon and alkalinity will need the same conservative updates as salinity (Sect. C2) and their own budget validation. CVMix, too, does not yet follow the working precision. Isolating precision effects requires a fixed domain decomposition, because a different decomposition alone changes the initial global-mean salinity by 0.0013 mpsu. The model-wide precision parameter also allows 16-bit formats to be tested in selected kernels ([34, 35, 1]), which would require the same budget diagnostics.
Conclusions
FESOM2 can run forced climate integrations in single precision. Over 60 years at nominal 1° resolution, its heat content, temperature, and mean state differ from double precision about as much as two double-precision runs with slightly perturbed initial conditions, its sea ice by at most 0.2 %, and the hindcasts run 1.35 times faster. Coupled to an atmosphere, it adds no detectable heat source or sink over 60 years. On meshes from 1° to 5 km, the Fortran model runs up to 1.66 times faster, and on GPUs the C++/Kokkos version needs 40–43 % less memory, which may matter as much as speed at kilometre scale.
This required more than switching the working precision. A first implementation lost salt, developed a heat-content deficit carried by the surface fluxes, and shifted ocean fronts, all from rounding errors of order accumulated over decades. Storing salinity as an anomaly reduced the drift by a factor of 32 but did not remove it. Accumulating tracer tendencies and layer thicknesses in 64 bits and carrying the rounding error of each tracer update into the next did, at up to 8 % additional cost; the remaining salinity difference is 14000 times smaller and enters through the surface flux.
Reduced precision is therefore a matter of numerical design and verification, not a compiler switch. How prognostic variables are updated decides whether small increments survive; conserved budgets reveal a precision-induced source or sink within 1 model year, long before its climatic consequences; and comparing independent implementations exposes errors that double-precision tests miss.
Eddy-rich and kilometre-scale configurations, century-scale conservation, biogeochemical tracers, and 16-bit kernels remain to be evaluated. These steps call for precision-aware Earth-system models, in which working precision, variable representation, and the placement of accumulations are designed together with the numerical algorithms and the hardware.
Code and data availability. FESOM2 is available at https://github.com/FESOM/fesom2 under the GNU General Public License v3.0, with releases archived on Zenodo. Single working precision was introduced in FESOM 2.8.0 ([68]). Pull request 940 was merged on 28 August 2026 (commit ea3bfebe); subsequent changes include the salinity anomaly (977, 986), solver defaults (984, 995, 997), 64-bit accumulation of output means (1032), tracer-conservation correction (1054 in 2.8.0, extended to both precisions by 1056 in 2.8.1), and 64-bit tracer tendencies and layer thicknesses with compensated tracer updates (1206, 1207).
The Fortran source archive ([7]) contains mainline revision 50467fa0 for the final 60-year experiments, the component timings, and the scaling measurements, and revision 27ed9f4f for the earlier six 60-year experiments. Revision 27ed9f4f (19 August 2026) was on the integration branch of pull request 940 before a rebase and is preserved as tag sp-paper-60yr-27ed9f4f at https://github.com/koldunovn/fesom2; revision 50467fa0 is tagged sp-paper-60yr-50467fa0 there. The remaining runs use neighbouring revisions of the same code: mainline revision a62f180 (included in 2.8.0) for the implementation comparison and the Fortran scaling points at 256 and 512 nodes, and earlier revisions with the change under test for the controlled experiments; Cheedela et al. [8] maps each result to its source revision.
The C++/Kokkos implementation is available at https://github.com/koldunovn/fesom_kokkos under the Apache License 2.0. Revisions 010d423, used in Sect. 5, and dfab63c, used in Sect. 6, are archived with their Kokkos 4.4.01 dependency (Trott et al., [77]) in Koldunov and Beyer [37].
Run configurations and job scripts, the partitioning recipe, reduction and plotting scripts, and the derived data used to generate all figures and numerical results (global time series, climatologies, benchmark timings, implementation-comparison statistics, and provenance records) are archived in Cheedela et al. [8] under the MIT licence.
Mesh archives are available for fArc (Wang, [81]), DARS (Sein and Harig, [69]), and NG5 (Koldunov and Harig, [38]). The NG5 vertical-level files used here differ from the archived version at one node and two elements; our copies are included in Cheedela et al. [8]. The CORE2 mesh, PHC3.0-derived initial state (Steele et al., [73]), and 1958 JRA55-do forcing used in Sect. 6 are archived in Koldunov [36]. The 60-year experiments use CORE-II interannual forcing (Large and Yeager, [44]; Yeager and Large, [85]); all other runs use JRA55-do v1.4.0 (Tsujino et al., [78, 79]). HadISST1 (Rayner et al., [62]) and WOA09 salinity (Antonov et al., [3]) were obtained from the ICDC data pool at Universität Hamburg.
The coupled experiments of Sect. 7.4 use AWI-ESM3 version 3.5.1 with FESOM 2.8.3 and the halo exchange of the initial-condition perturbation from FESOM pull request 1214, set up with ESM-Tools (Barbi et al., [6]). Their run configurations and reduction scripts are archived in Cheedela et al. [8].
Raw output from the six 60-year experiments totals about 1.3 TB and exceeds the archive’s capacity. It is retained on DKRZ Levante, is available from the authors on request, and can be regenerated from the archived code, configurations, and input data.
Mesh characteristics
Table A1 lists the four meshes used for performance measurements, with dimensions read from the model’s mesh files. CORE2 is the nominal mesh used in the climate experiments. fArc refines CORE2 to 4.5 km in the Arctic (Wang et al., [82]). DARS follows the local Rossby radius (Sein et al., [70]) and resolves energetically active regions more finely. NG5 has nearly uniform 5 km resolution over most of the globe. The larger meshes also have more vertical levels, so their computational size grows faster than their surface-vertex count.
| Mesh | Vertices | Triangles | Levels | Resolution |
| CORE2 | 126 858 | 244 659 | 48 | nominal 1° |
| fArc | 638 387 | 1 253 306 | 48 | CORE2 + 4.5 km Arctic |
| DARS | 3 160 340 | 6 262 485 | 57 | Rossby-radius scaled |
| NG5 | 7 402 886 | 14 741 520 | 70 | km quasi-uniform |
Table A1. Meshes used in the performance measurements. Vertices and triangles are surface counts; levels are vertical grid levels. Resolution describes the mesh design rather than an area-weighted mean.
Scaling configurations
The measurements run on DKRZ Levante, whose CPU nodes have two 64-core AMD EPYC 7763 processors (128 processes per node) and whose GPU nodes have four NVIDIA A100 devices (one process per device). The Fortran model is compiled with the Intel compiler. All runs use the physical settings of the Kokkos implementation’s default CORE2 configuration on every mesh: linear free surface, biharmonic horizontal viscosity, KPP mixing, standard EVP sea ice, split vertical velocity on fArc, DARS, and NG5, and symmetric sea-surface-height preconditioning in both precisions. They use JRA55-do forcing, PHC3.0 initial conditions, and timesteps of 1800 s for CORE2, 900 s for fArc, 120 s for DARS, and 180 s for NG5.
Each job runs double precision, single precision, and single precision without conservative updates twice, in mirrored order; Kokkos jobs start with a discarded warm-up run. Each run starts from the initial state, advances 300 timesteps without output, and is timed by the implementation’s own step timer; we report the faster of the two runs of each build. Repeated runs differ by a median of 0.3 % and at most 3 %, except for DARS at the two largest process counts, where they differ by up to 12 %; the apparent increase of the DARS speedup between 256 and 512 nodes lies within this spread, and the step time at 512 nodes shows that DARS has passed its useful scaling range there. Speedup is the double-precision time divided by the single-precision time for the same code, mesh, and process count. Absolute times of the two implementations are indicative, because their compilers and timers differ. Throughput is projected to production timesteps of 1800 s for CORE2, 1200 s for fArc, and 240 s for DARS and NG5, neglecting a 1–2 % increase in cost per step at the longer timestep that is measured in the Kokkos implementation and caused by additional solver iterations.
Fortran partitions are generated at each process count with the model’s METIS partitioner [31] and two-dimensional vertex weights, following the recipe supplied with the meshes; on CORE2 at 864 processes, this reproduces the step time of the supplied partition within 2 %. The Kokkos runs use the supplied partitions.
CPU process counts range from 64–4096 for CORE2, 512–8192 for fArc, 1024–65 536 for DARS, and 2048–65 536 for NG5, and GPU counts from 4–64, 8–64, 8–128, and 16–128. The lower limit is set by memory; CPU runs extend to 512 nodes, the allocation limit of the partition. The DARS and NG5 Fortran points at 256 and 512 nodes come from an earlier campaign with a version without conservative updates, which used the same protocol without the third build; two of them, DARS at 32 768 and NG5 at 65 536 processes, use the completed runs of jobs that reached their time limit. At process counts measured in both campaigns, absolute double-precision times differ by −22 to +30 %, most at the highest counts, so the absolute times of the earlier points are indicative, whereas their speedups are ratios within one job. In the DARS job at 16 384 processes, one single-precision run failed at start-up on a network error, and the time of that build is the remaining run.
Single precision before the conservative updates
This appendix documents the 60-year hindcasts with the first implementation (Sect. 4.1), which precede the conservative updates of Sect. 3.3, and the controlled experiments that identified the source of their salinity drift. § 4.3 summarizes them. The double-precision control of these hindcasts is a pair of runs that differ only in the salinity representation, and hence only in round-off.
Global budgets and sea ice
Figure C1 shows the annual global integrals of the first implementation as differences between paired runs. With absolute salinity, single precision loses salt at a steady psu yr and reaches a last-decade deficit of 6.8 mpsu, whereas the control differs by about mpsu and the double-precision run itself changes by 0.0039 mpsu over sixty years; an earlier pair with pre-merge code on 128 processes gave a similar deficit of 5.5 mpsu. The ocean also loses 129 ZJ of heat, and its volume-mean temperature falls by 24 mK, about a third of the double-precision spin-up warming of 77 mK, although SST is slightly higher.
Figure C1. Annual global-integral differences in the 60-year CORE2 hindcasts with the first implementation, without the conservative updates of Sect. 3.3. Orange and magenta show single minus double precision with absolute salinity and with the anomaly, respectively. Gray shows the difference between the double-precision salinity representations.
With the salinity anomaly, the salinity trend reverses sign and shrinks by a factor of 32, to psu yr, and the heat-content and temperature deficits shrink to 8.6 ZJ (a reduction of 93 %) and 1.6 mK. These residuals are still systematic: the salinity difference exceeds that of the control by more than three orders of magnitude, and the temperature and heat-content differences are several times the control.
The heat budget separates the heat-content differences into a surface-flux response and an internal residual. Integrating the monthly surface heat flux and the heat carried by the freshwater flux closes the budget of each double-precision run to within 0.002 W m. With absolute salinity, 123 ZJ of the deficit comes from the surface fluxes and 5.3 ZJ from the internal residual. With the anomaly, the surface fluxes supply 4.7 ZJ more heat than in double precision, while an internal sink removes 13.3 ZJ, or 0.02 W m; the internal residual of the control stays within 0.004 ZJ. The salinity differences and the internal sinks exceed the control by model year 1, whereas the heat-content difference of the anomaly configuration does so only from year 9. § 11.2 attributes most of the internal sink with the anomaly to the rounding of temperature updates rather than to the FCT defect described in Sect. 3.3.
With absolute salinity, Arctic ice volume is 1.0 % larger in the last decade, with a maximum difference of 1.5 %, while ice area is essentially unchanged. Antarctic ice volume and area show a transient response that peaks around 1960, reaching 7.4 % in volume, and subsides by the end of the record. With the anomaly, volume differences remain within 0.22 % in the Arctic and 0.19 % in the Antarctic; the corresponding baseline maxima are 0.05 % and 0.1 %.
Source of the salinity drift
The surface freshwater budget does not explain the freshening. Monthly net fluxes close to within each build’s machine precision, with no systematic bias or trend, and are 4.8 orders of magnitude smaller than the flux required to explain the drift. Controlled 20-year experiments also exclude global budget reductions, the solver, and domain decomposition as its main sources
(Figure C2). Using 64-bit budget reductions or solver operations, changing the decomposition, or adding perturbations smaller than the spacing between representable salinity values yields nearly the same drift as the unmodified single-precision run: −1.87 mpsu at year 20. A double-precision control on a different decomposition remains within 0.0010 mpsu of the reference.
Figure C2. Global-mean salinity differences in controlled experiments with pre-merge code at 512 processes, relative to an earlier double-precision run at 128 processes. Gray: single precision and variants with a different decomposition, 64-bit global sums, 64-bit solver operations, or small state perturbations. Orange: 64-bit tracer state with 32-bit increments. Cyan: double precision with the tracer state rounded to 32 bits after each step (two years). Magenta: salinity stored as . Dark blue: double precision with a different decomposition.
The experiments instead implicate rounding in the salinity update, . At , increments smaller than psu can be lost when added to a 32-bit state, and the calculation of the increments can itself introduce bias. This loss of small increments, known as stagnation ([11]), has biased slowly evolving deep-soil temperatures in single-precision land-surface models ([27, 18]), and single-precision tracer advection in the MRI.COM ocean model lost small tracer fluxes added to the tracer content of grid cells, producing temperature errors that grew over 300 years ([50]). Retaining the state in 64 bits while calculating increments in 32 bits produces a drift of +1.85 mpsu after 20 years: nearly the same magnitude as the unmodified run, but of opposite sign. Conversely, rounding the tracer state to 32 bits after each double-precision timestep produces a drift of −0.14 mpsu in two years. Thus, increasing storage precision alone does not eliminate the bias. This differs from the land-surface, shallow-water, and ocean-advection cases above, in which a higher-precision state largely removes the error ([18, 34, 50]): here the increments are themselves calculated from salinities near 35 psu, and their rounding is biased. The anomaly representation reduces rounding in both the updates and the increments and gives a 20-year difference of 0.064 mpsu, 29 times smaller in magnitude than the unmodified run, with little computational overhead (Sect. 5.1).
Because the salinity differences and internal heat sinks emerge within the first model year (Sect. C1), nine-month restarts from the hindcast states at the end of 1997 can separate their sources. Correcting the FCT defect (Sect. 3.3) reduces the salt loss with absolute salinity by 41 % but changes the residuals with the anomaly by about 1 %; FESOM 2.8.1, which includes the correction, gives an internal heat sink of 0.023 W m and a salinity gain of psu in nine months. Compensated (Kahan) summation ([30]) of the tracer updates alone turns these residuals into a source of 0.0017 W m and a loss of psu, both still above the difference between the double-precision restarts; with absolute salinity, it leaves a gain of psu, as 64-bit storage does. These results motivated computing the increments in 64 bits as well and storing layer thicknesses at that precision (Sect. 3.3).
Mean-state differences and observational biases
We compare the 1988–2007 mean states with both the double-precision baseline and the model’s bias against independent observational products (Figure C3): HadISST1 for SST ([62]) and the World Ocean Atlas 2009 (WOA09) for surface salinity ([3]). PHC3.0 supplies the initial conditions and is not used for this evaluation.
Figure C3. Mean-state differences for 1988–2007 with the first implementation. Left: double-precision bias against observations (SST against HadISST1 and surface salinity against WOA09), with area-weighted root-mean-square error. Remaining columns: single minus double precision with absolute salinity, the same with the salinity anomaly, and the difference between the double-precision salinity representations. These three columns use a common color scale and show area-weighted RMS differences.
With absolute salinity, the root-mean-square (RMS) single-minus-double precision differences are 0.165 °C for SST and 0.0497 psu for surface salinity. These are 0.27 and 0.12 of the respective model errors against observations, but 36 and 30 times the double-precision baseline. The spatial pattern follows major fronts in the North Atlantic Current, Southern Ocean, and Brazil–Malvinas Confluence, indicating a systematic mean-state shift.
With the anomaly, the RMS differences decrease to 0.0172 °C and 0.00883 psu, or 0.028 and 0.021 of the errors against observations. The coherent frontal pattern disappears. The remaining differences are 3.7 and 5.3 times the baseline: small relative to observational biases, but still distinguishable from the double-precision control.
Sea-ice concentration shows a similar response (Figure C4). With absolute salinity, precision-induced differences are 0.029–0.17 of the model error against HadISST1 and up to 28 times the baseline, concentrated near the Antarctic ice edge. With the anomaly, they fall to at most 0.012 of the model error against observations and 2.4 times the baseline. Ice thickness improves less: the residual Arctic volume difference is about a fifth of a percent and remains several times the baseline (Figure C1).
Figure C4. Mean sea-ice concentration differences for 1988–2007 at the seasonal maxima (March in the Northern Hemisphere and September in the Southern Hemisphere). Columns follow Figure C3. RMS differences are calculated over nodes where modeled or observed concentration exceeds 0.05.
Coupled integrations
The coupled experiments of Sect. 7.4 use AWI-ESM3 version 3.5.1, the successor of AWI-CM3 ([74]). FESOM 2.8.3 runs on the CORE2 mesh of Sect. 2 with the same vertical coordinate, mixing, and eddy parameterizations, but with a 1800 s timestep, without surface salinity restoring, and without icebergs. The atmosphere model OpenIFS 48r1 [20] runs at TCo95 resolution with 91 levels and a 1 h timestep, and the dynamic vegetation model LPJ-GUESS ([72]) on the land points of the OpenIFS grid. The coupler OASIS3-MCT ([10]) exchanges fluxes and surface state between atmosphere and ocean every 2 h and between atmosphere and vegetation once a day. The fields that FESOM2 passes to OASIS are 64-bit in both precisions.
All four runs use 1850 forcing and start cold: the ocean at rest from PHC3.0, the atmosphere from its standard initial state, and LPJ-GUESS without a spun-up state. FESOM2 runs on 512 processes, OpenIFS on 192 processes with four threads each, and LPJ-GUESS on 64 processes, 13 nodes in total, with the same layout and coupling weights in all four runs. The two perturbed runs add one Gaussian draw with a standard deviation of K to the initial temperature of each water column, with the same random seed in both precisions. FESOM2 now exchanges the halos of the perturbed fields. Without this exchange, the perturbed runs did not conserve volume, heat, or salt. The model records its budgets every simulated day, as in Sect. 4.1. In double precision, the volume and salt residuals stay below of the content. In single precision, a restart changes the heat content by of its value, as in the forced runs.
Author contributions. SKC implemented single working precision in FESOM2 and ran the 60-year experiments. NK designed the study, implemented single precision in the C++/Kokkos version, and carried out the analysis and the performance measurements. JS ran the coupled experiments. SD and DS contributed to the conceptualization. TJ acquired funding and managed the project. All authors contributed to writing the manuscript.
Competing interests. The authors declare that they have no conflict of interest.
Acknowledgements. We thank the German Climate Computing Center (DKRZ) for access to its computing infrastructure.
Financial support. This work was primarily supported through the core funding of the Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research (AWI), within the Helmholtz Association. SKC is funded by the WarmWorld Better project of the German Federal Ministry of Research, Technology and Space under the funding code 01LK2202A; the responsibility for the content of this publication lies with the authors. The contribution by NK was supported by the TerraDT (Digital Twin of Earth system for Cryosphere, Land surface and related interactions) project, which has received funding from the European Union’s Horizon Europe research and innovation programme under Grant Agreement no. 101187992. SD was supported by project M5 of the Collaborative Research Centre TRR181 “Energy Transfer in Atmosphere and Ocean” funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Projektnummer 274762653. TJ and NK were supported by the EERIE project (Grant Agreement No 101081383) funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor the granting authority can be held responsible for them. The work of SKC and NK has been supported by the European Union’s Destination Earth Initiative and relates to tasks entrusted by the European Union to the European Centre for Medium-Range Weather Forecasts implementing part of this Initiative with funding by the European Union. Views and opinions expressed are those of the authors only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the European Commission can be held responsible for them.
References
- [1]Ackmann, J., Düben, P. D., Palmer, T., and Smolarkiewicz, P. K.: Mixed-Precision for Linear Solvers in Global Geophysical Flows, Journal of Advances in Modeling Earth Systems, 14, e2022MS003 148, https://doi.org/10.1029/2022MS003148, 2022.
- [2]Adcroft, A. and Campin, J.-M.: Rescaled height coordinates for accurate representation of free-surface flows in ocean circulation models, Ocean Modelling, 7, 269–284, https://doi.org/10.1016/j.ocemod.2003.09.003, 2004.
- [3]Antonov, J. I., Seidov, D., Boyer, T. P., Locarnini, R. A., Mishonov, A. V., Garcia, H. E., Baranova, O. K., Zweng, M. M., and Johnson, D. R.: World Ocean Atlas 2009, Volume 2: Salinity, NOAA Atlas NESDIS 69, U.S. Government Printing Office, Washington, D.C., edited by: Levitus, S., 184 pp., 2010.DOI
- [4]Baker, A. H., Hu, Y., Hammerling, D. M., Tseng, Y.-h., Xu, H., Huang, X., Bryan, F. O., and Yang, G.: Evaluating statistical consistency in the ocean model component of the Community Earth System Model (pyCECT v2.0), Geoscientific Model Development, 9, 2391–2406, https://doi.org/10.5194/gmd-9-2391-2016, 2016.
- [5]Banderier, H., Zeman, C., Leutwyler, D., Rüdisühli, S., and Schär, C.: Reduced floating-point precision in regional climate simulations: an ensemble-based statistical verification, Geoscientific Model Development, 17, 5573–5586, https://doi.org/10.5194/gmd-17-5573-2024, 2024.
- [6]Barbi, D., Wieters, N., Gierz, P., Andrés-Martínez, M., Ural, D., Chegini, F., Khosravi, S., and Cristini, L.: ESM-Tools version 5.0: a modular infrastructure for stand-alone and coupled Earth system modelling (ESM), Geoscientific Model Development, 14, 4051–4067, https://doi.org/10.5194/gmd-14-4051-2021, 2021.
- [7]Cheedela, S. K., Koldunov, N. V., Danilov, S., Streffing, J., Sidorenko, D., and Jung, T.: FESOM2 code revisions of “FESOM2.8 in single precision: climate fidelity over six decades and performance on CPUs and GPUs”, Zenodo, https://doi.org/10.5281/zenodo.24014072, 2026a.
- [8]Cheedela, S. K., Koldunov, N. V., Danilov, S., Streffing, J., Sidorenko, D., and Jung, T.: Data, run configurations and scripts for “FESOM2.8 in single precision: climate fidelity over six decades and performance on CPUs and GPUs”, Zenodo, https://doi.org/10.5281/zenodo.24015804, 2026b.
- [9]Chen, S., Zhang, Y., Wang, Y., Liu, Z., Li, X., and Xue, W.: Mixed-precision computing in the GRIST dynamical core for weather and climate modelling, Geoscientific Model Development, 17, 6301–6318, https://doi.org/10.5194/gmd-17-6301-2024, 2024.
- [10]Craig, A., Valcke, S., and Coquart, L.: Development and performance of a new version of the OASIS coupler, OASIS3-MCT_3.0, Geoscientific Model Development, 10, 3297–3308, https://doi.org/10.5194/gmd-10-3297-2017, 2017.
- [11]Croci, M. and Giles, M. B.: Effects of round-to-nearest and stochastic rounding in the numerical solution of the heat equation in low precision, IMA Journal of Numerical Analysis, 43, 1358–1390, https://doi.org/10.1093/imanum/drac012, 2023.
- [12]Croci, M. and Rosilho de Souza, G.: Mixed-precision explicit stabilized Runge–Kutta methods for single- and multi-scale differential equations, Journal of Computational Physics, 464, 111 349, https://doi.org/10.1016/j.jcp.2022.111349, 2022.
- [13]Croci, M., Fasi, M., Higham, N. J., Mary, T., and Mikaitis, M.: Stochastic rounding: implementation, error analysis and applications, Royal Society Open Science, 9, 211 631, https://doi.org/10.1098/rsos.211631, 2022.
- [14]Danabasoglu, G., Yeager, S. G., Bailey, D., Behrens, E., Bentsen, M., Bi, D., Biastoch, A., Bön ing, C., Bozec, A., Canuto, V. M., Cassou, C., Chassignet, E., Coward, A. C., Danilov, S., Diansky, N., Drange, H., Farneti, R., Fernandez, E., Fogli, P. G., Forget, G., Fujii, Y., Griffies, S. M., Gusev, A., Heimbach, P., Howard, A., Jung, T., Kelley, M., Large, W. G., Lebois settier, A., Lu, J., Madec, G., Marsland, S. J., Masina, S., Navarra, A., Nurser, A. J. G., Pirani, A., Salas y Mélia, D., Samuels, B. L., Scheinert, M., Sidorenko, D., Treguier, A.-M., Tsujino, H., Uotila, P., Valcke, S., Voldoire, A., and Wang, Q.: North Atlantic simulations in Coordinated Ocean-ice Reference Experiments phase II (CORE-II). Part I: Mean states, Ocean Modelling, 73, 76–107, https://doi.org/10.1016/j.ocemod.2013.10.005, 2014.
- [15]Danilov, S., Wang, Q., Timmermann, R., Iakovlev, N., Sidorenko, D., Kimmritz, M., Jung, T., and Schröter, J.: Finite-Element Sea Ice Model (FESIM), version 2, Geoscientific Model Development, 8, 1747–1761, https://doi.org/10.5194/gmd-8-1747-2015, 2015.
- [16]Danilov, S., Sidorenko, D., Wang, Q., and Jung, T.: The Finite-volume Sea Ice–Ocean Model (FESOM2), Geoscientific Model Development, 10, 765–789, https://doi.org/10.5194/gmd-10-765-2017, 2017.
- [17]Dawson, A. and Düben, P. D.: rpe v5: an emulator for reduced floating-point precision in large numerical simulations, Geoscientific Model Development, 10, 2221–2230, https://doi.org/10.5194/gmd-10-2221-2017, 2017.
- [18]Dawson, A., Düben, P. D., MacLeod, D. A., and Palmer, T. N.: Reliable low precision simulations in land surface models, Climate Dynamics, 51, 2657–2666, https://doi.org/10.1007/s00382-017-4034-x, 2018.
- [19]Düben, P. D. and Palmer, T. N.: Benchmark Tests for Numerical Weather Forecasts on Inexact Hardware, Monthly Weather Review, 142, 3809–3829, https://doi.org/10.1175/MWR-D-14-00110.1, 2014.
- [20]ECMWF: Release notes for OpenIFS 48r1, https://confluence.ecmwf.int/display/OIFS/Release+notes+for+OpenIFS+48r1, last access: 8 November 2026, 2024.
- [21]Gent, P. R. and McWilliams, J. C.: Isopycnal mixing in ocean circulation models, Journal of Physical Oceanography, 20, 150–155, https://doi.org/10.1175/1520-0485(1990)020<0150:IMIOCM>2.0.CO;2, 1990.
- [22]Gholami, A., Yao, Z., Kim, S., Hooper, C., Mahoney, M. W., and Keutzer, K.: AI and Memory Wall, IEEE Micro, 44, 33–39, https://doi.org/10.1109/MM.2024.3373763, 2024.
- [23]Grant, Z. J.: Perturbed Runge–Kutta Methods for Mixed Precision Applications, Journal of Scientific Computing, 92, 6, https://doi.org/10.1007/s10915-022-01801-2, 2022.
- [24]Griffies, S. M., Harrison, M. J., Pacanowski, R. C., and Rosati, A.: A Technical Guide to MOM4, GFDL Ocean Group Technical Report 5, NOAA/Geophysical Fluid Dynamics Laboratory, Princeton, NJ, available at: https://www.gfdl.noaa.gov/wp-content/uploads/files/model_development/ocean/guide4p0.pdf (version prepared on 4 February 2008; last access: 7 October 2026), 2004.
- [25]Griffies, S. M., Levy, M., Adcroft, A. J., Danabasoglu, G., Hallberg, R. W., Jacobsen, D., Large, W. G., and Ringler, T. D.: Theory and numerics of the Community Ocean Vertical Mixing (CVMix) project, Tech. rep., NOAA/Geophysical Fluid Dynamics Laboratory, Princeton, NJ, available at: https://github.com/CVMix/CVMix-description (last access: 2 August 2026), 2015.
- [26]Häfner, D., Nuterman, R., and Jochum, M.: Fast, cheap, and turbulent — Global ocean modeling with GPU acceleration in Python, Journal of Advances in Modeling Earth Systems, 13, e2021MS002717, https://doi.org/10.1029/2021MS002717, 2021.
- [27]Harvey, R. and Verseghy, D. L.: The reliability of single precision computations in the simulation of deep soil heat diffusion in a land surface model, Climate Dynamics, 46, 3865–3882, https://doi.org/10.1007/s00382-015-2809-5, 2016.
- [29]Juricke, S., Danilov, S., Koldunov, N., Oliver, M., Sein, D. V., Sidorenko, D., and Wang, Q.: A kinematic kinetic energy backscatter parameterization: From implementation to global ocean simulations, Journal of Advances in Modeling Earth Systems, 12, e2020MS002175, https://doi.org/10.1029/2020MS002175, 2020.
- [30]Kahan, W.: Pracniques: further remarks on reducing truncation errors, Communications of the ACM, 8, 40, https://doi.org/10.1145/363707.363723, 1965.
- [31]Karypis, G. and Kumar, V.: A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs, SIAM Journal on Scientific Computing, 20, 359–392, https://doi.org/10.1137/S1064827595287997, 1998.
- [32]Keeley, S., Mogensen, K., Bidlot, J.-R., Alonso Balmaseda, M., and Hatfield, S.: Introduction of a new ocean and sea-ice model based on NEMO4-SI3, ECMWF Newsletter, 180, 24–29, https://doi.org/10.21957/sk4928ds0a, 2024.
- [33]Kimpson, T., Paxton, E. A., Chantry, M., and Palmer, T.: Climate-change modelling at reduced floating-point precision with stochastic rounding, Quarterly Journal of the Royal Meteorological Society, 149, 843–855, https://doi.org/10.1002/qj.4435, 2023.
- [34]Klöwer, M., Düben, P. D., and Palmer, T. N.: Number Formats, Error Mitigation, and Scope for 16-Bit Arithmetics in Weather and Climate Modeling Analyzed With a Shallow Water Model, Journal of Advances in Modeling Earth Systems, 12, e2020MS002246, https://doi.org/10.1029/2020MS002246, 2020.
- [35]Klöwer, M., Hatfield, S., Croci, M., Düben, P. D., and Palmer, T. N.: Fluid Simulations Accelerated With 16 Bits: Approaching 4x Speedup on A64FX by Squeezing ShallowWaters.jl Into Float16, Journal of Advances in Modeling Earth Systems, 14, e2021MS002684, https://doi.org/10.1029/2021MS002684, 2022.
- [36]Koldunov, N. V.: fesom-jax CORE2 one-year setup: mesh, initial state, and JRA55-do forcing (1958) (Version 1.2), Zenodo, https://doi.org/10.5281/zenodo.21770779, 2026.
- [37]Koldunov, N. V. and Beyer, S.: fesom_kokkos at the revisions measured in “FESOM2.8 in single precision: climate fidelity over six decades and performance on CPUs and GPUs”, Zenodo, https://doi.org/10.5281/zenodo.24014074, 2026.
- [38]Koldunov, N. V. and Harig, S.: FESOM2 meshes: NG5 (Version 1.0), Zenodo, https://doi.org/10.5281/zenodo.14926135, 2025.
- [39]Koldunov, N. V., Aizinger, V., Rakowsky, N., Scholz, P., Sidorenko, D., Danilov, S., and Jung, T.: Scalability and some optimization of the Finite-volume Sea ice–Ocean Model, Version 2.0 (FESOM2), Geoscientific Model Development, 12, 3991–4012, https://doi.org/10.5194/gmd-12-3991-2019, 2019.
- [40]Koldunov, N. V., Cheedela, S. K., Danilov, S., Sidorenko, D., Beyer, S., and Jung, T.: An ocean model ported by a large language model: experience and lessons from FESOM2 (Fortran to C to C++/Kokkos), arXiv preprint arXiv:2606.11356, https://doi.org/10.48550/arXiv.2606.11356, 2026a.
- [41]Koldunov, N. V., Danilov, S., Cheedela, S., Sidorenko, D., Beyer, S., Scholz, P., Kuznetsov, I., Streffing, J., Koldunov, A., Pantiŭkhin, D., Loza, S. N., and Jung, T.: FESOM2-JAX v1.0: a differentiable shadow of the ocean–sea-ice model FESOM2, cast onto GPUs, arXiv preprint arXiv:2608.01546, https://doi.org/10.48550/arXiv.2608.01546, 2026b.
- [42]Lai, J., Wang, L., Wu, Q., Yang, Y., and Wang, F.: Enhancing single precision with quasi-double precision: achieving double-precision accuracy in the Model for Prediction Across Scales – Atmosphere (MPAS-A) version 8.2.1, Geoscientific Model Development, 18, 1089–1102, https://doi.org/10.5194/gmd-18-1089-2025, 2025.
- [43]Lang, S. T. K., Dawson, A., Diamantakis, M., Düben, P., Hatfield, S., Leutbecher, M., Palmer, T., Prates, F., Roberts, C. D., Sandu, I., and Wedi, N.: More accuracy with less precision, Quarterly Journal of the Royal Meteorological Society, 147, 4358–4370, https://doi.org/10.1002/qj.4181, 2021.
- [44]Large, W. G. and Yeager, S. G.: The global climatology of an interannually varying air–sea flux data set, Climate Dynamics, 33, 341–364, https://doi.org/10.1007/s00382-008-0441-3, 2009.
- [45]Large, W. G., McWilliams, J. C., and Doney, S. C.: Oceanic vertical mixing: A review and a model with a nonlocal boundary layer parameterization, Reviews of Geophysics, 32, 363–403, https://doi.org/10.1029/94RG01872, 1994.
- [46]Leroux, S., Penduff, T., Bessières, L., Molines, J.-M., Brankart, J.-M., Sérazin, G., Barnier, B., and Terray, L.: Intrinsic and Atmospherically Forced Variability of the AMOC: Insights from a Large-Ensemble Ocean Hindcast, Journal of Climate, 31, 1183–1203, https://doi.org/10.1175/JCLI-D-17-0168.1, 2018.
- [47]Massonnet, F., Ménégoz, M., Acosta, M., Yepes-Arbós, X., Exarchou, E., and Doblas-Reyes, F. J.: Replicability of the EC-Earth3 Earth system model under a change in computing environment, Geoscientific Model Development, 13, 1165–1178, https://doi.org/10.5194/gmd-13-1165-2020, 2020.
- [48]Maynard, C. M. and Walters, D. N.: Mixed-precision arithmetic in the ENDGame dynamical core of the Unified Model, a numerical weather prediction and climate model code, Computer Physics Communications, 244, 69–75, https://doi.org/10.1016/j.cpc.2019.07.002, 2019.
- [49]Møller, O.: Quasi double-precision in floating point addition, BIT, 5, 37–50, https://doi.org/10.1007/BF01975722, 1965.
- [50]Nakano, H., Urakawa, L. S., Aoki, K., Kawakami, Y., and Hirahara, S.: On the speeding up and accuracy of the Second Order Moment (SOM) advection scheme using a mixed-precision method, Ocean Modelling, 194, 102 495, https://doi.org/10.1016/j.ocemod.2024.102495, 2025.
- [51]Nakano, M., Yashiro, H., Kodama, C., and Tomita, H.: Single Precision in the Dynamical Core of a Nonhydrostatic Global Atmospheric Model: Evaluation Using a Baroclinic Wave Test Case, Monthly Weather Review, 146, 409–416, https://doi.org/10.1175/MWR-D-17-0257.1, 2018.
- [52]NVIDIA: NVIDIA A100 Tensor Core GPU, https://www.nvidia.com/en-us/data-center/a100/, last access: 2 November 2026, 2020.
- [53]NVIDIA: NVIDIA H100 Tensor Core GPU, https://www.nvidia.com/en-us/data-center/h100/, last access: 2 November 2026, 2022.
- [54]NVIDIA: NVIDIA Ada GPU Architecture, Whitepaper V2.02, https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf, last access: 8 November 2026, 2023.
- [56]Pacanowski, R. C. and Griffies, S. M.: The MOM 3 Manual, GFDL Ocean Group Technical Report 4, NOAA/Geophysical Fluid Dynamics Laboratory, Princeton, NJ, 680 pp., available at: https://www.gfdl.noaa.gov/wp-content/uploads/files/model_development/ocean/mom3_manual.pdf (draft dated February 2000; last access: 7 October 2026), 1999.
- [57]Paronuzzi-Ticco, S. V., Utrera, G., and Acosta, M. C.: Efficient variable precision reduction in chaotic climate models: Analysis of the NEMO case in the Destination Earth project, Computers & Geosciences, 205, 105 989, https://doi.org/10.1016/j.cageo.2025.105989, 2025.
- [58]Paxton, E. A., Chantry, M., Klöwer, M., Saffin, L., and Palmer, T.: Climate Modeling in Low Precision: Effects of Both Deterministic and Stochastic Rounding, Journal of Climate, 35, 1215–1229, https://doi.org/10.1175/JCLI-D-21-0343.1, 2022.
- [59]Penduff, T., Juza, M., Barnier, B., Zika, J., Dewar, W. K., Treguier, A.-M., Molines, J.-M., and Audiffren, N.: Sea Level Expression of Intrinsic and Forced Ocean Variabilities at Interannual Time Scales, Journal of Climate, 24, 5652–5670, https://doi.org/10.1175/JCLI-D-11-00077.1, 2011.
- [60]Polichtchouk, I., Massart, S., and Kipling, Z.: Upgrade to IFS Cycle 50r1, ECMWF Newsletter, 185, 25–33, https://doi.org/10.21957/ea40-q144, 2025.
- [61]Porter, A. R. and Heimbach, P.: Unlocking the power of parallel computing: GPU technologies for ocean forecasting, State of the Planet, 5-opsr, 23, https://doi.org/10.5194/sp-5-opsr-23-2025, 2025.
- [62]Rayner, N. A., Parker, D. E., Horton, E. B., Folland, C. K., Alexander, L. V., Rowell, D. P., Kent, E. C., and Kaplan, A.: Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century, Journal of Geophysical Research: Atmospheres, 108, 4407, https://doi.org/10.1029/2002JD002670, 2003.
- [63]Redi, M. H.: Oceanic isopycnal mixing by coordinate rotation, Journal of Physical Oceanography, 12, 1154–1158, https://doi.org/10.1175/1520-0485(1982)012<1154:OIMCBR>2.0.CO;2, 1982.
- [64]Rosinski, J. M. and Williamson, D. L.: The Accumulation of Rounding Errors and Port Validation for Global Atmospheric Models, SIAM Journal on Scientific Computing, 18, 552–564, https://doi.org/10.1137/S1064827594275534, 1997.
- [66]Scholz, P., Sidorenko, D., Gurses, O., Danilov, S., Koldunov, N., Wang, Q., Sein, D., Smolentseva, M., Rakowsky, N., and Jung, T.: Assessment of the Finite-volumeE Sea ice–Ocean Model (FESOM2.0) – Part 1: Description of selected key model elements and comparison to its predecessor version, Geoscientific Model Development, 12, 4875–4899, https://doi.org/10.5194/gmd-12-4875-2019, 2019.
- [67]Scholz, P., Sidorenko, D., Danilov, S., Wang, Q., Koldunov, N., Sein, D., and Jung, T.: Assessment of the Finite-volumeE Sea ice–Ocean Model (FESOM2.0) – Part 2: Partial bottom cells, embedded sea ice and vertical mixing library CVMix, Geoscientific Model Development, 15, 335–363, https://doi.org/10.5194/gmd-15-335-2022, 2022.
- [68]Scholz, P., Streffing, J., Sidorenko, D., Hegewald, J., Koldunov, N., Cheedela, S. K., Rackow, T., et al.: FESOM/fesom2: 2.8.0, Zenodo, https://doi.org/10.5281/zenodo.22707597, 2026.
- [69]Sein, D. and Harig, S.: FESOM2 meshes: DARS, Zenodo, https://doi.org/10.5281/zenodo.23140101, 2026.
- [70]Sein, D. V., Koldunov, N. V., Danilov, S., Wang, Q., Sidorenko, D., Fast, I., Rackow, T., Cabos, W., and Jung, T.: Ocean Modeling on a Mesh With Resolution Following the Local Rossby Radius, Journal of Advances in Modeling Earth Systems, 9, 2601–2614, https://doi.org/10.1002/2017MS001099, 2017.
- [71]Skamarock, W. C., Klemp, J. B., Dudhia, J., Gill, D. O., Liu, Z., Berner, J., Wang, W., Powers, J. G., Duda, M. G., Barker, D. M., and Huang, X.-Y.: A Description of the Advanced Research WRF Model Version 4, NCAR Technical Note NCAR/TN-556+STR, National Center for Atmospheric Research, Boulder, CO, https://doi.org/10.5065/1dfh-6p97, 2019.
- [72]Smith, B., Wårlind, D., Arneth, A., Hickler, T., Leadley, P., Siltberg, J., and Zaehle, S.: Implications of incorporating N cycling and N limitations on primary production in an individual-based dynamic vegetation model, Biogeosciences, 11, 2027–2054, https://doi.org/10.5194/bg-11-2027-2014, 2014.
- [73]Steele, M., Morley, R., and Ermold, W.: PHC: A global ocean hydrography with a high-quality Arctic Ocean, Journal of Climate, 14, 2079–2087, https://doi.org/10.1175/1520-0442(2001)014<2079:PAGOHW>2.0.CO;2, 2001.
- [74]Streffing, J., Sidorenko, D., Semmler, T., Zampieri, L., Scholz, P., Andrés-Martínez, M., Koldunov, N., Rackow, T., Kjellsson, J., Goessling, H., Athanase, M., Wang, Q., Hegewald, J., Sein, D. V., Mu, L., Fladrich, U., Barbi, D., Gierz, P., Danilov, S., Juricke, S., Lohmann, G., and Jung, T.: AWI-CM3 coupled climate model: description and evaluation experiments for a prototype post-CMIP6 model, Geoscientific Model Development, 15, 6399–6427, https://doi.org/10.5194/gmd-15-6399-2022, 2022.
- [75]The NEMO team: NEMO release version 4.2.2, Zenodo [code], https://doi.org/10.5281/zenodo.11551599, 2024.
- [76]Tintó Prims, O., Acosta, M. C., Moore, A. M., Castrillo, M., Serradell, K., Cortés, A., and Doblas-Reyes, F. J.: How to use mixed precision in ocean models: exploring a potential reduction of numerical precision in NEMO 4.0 and ROMS 3.6, Geoscientific Model Development, 12, 3135–3148, https://doi.org/10.5194/gmd-12-3135-2019, 2019.
- [77]Trott, C. R., Lebrun-Grandie, D., Arndt, D., Ciesko, J., Dang, V., Ellingwood, N., Gawatri, R., Harvey, E., Hollman, D. S., Ibanez, D., Liber, N., Madsen, J., Miles, J., Poliakoff, D., Powell, A., Rajamanickam, S., Simberg, M., Sunderland, D., Turcksin, B., and Wilke, J.: Kokkos 3: Programming Model Extensions for the Exascale Era, IEEE Transactions on Parallel and Distributed Systems, 33, 805–817, https://doi.org/10.1109/TPDS.2021.3097283, 2022.
- [78]Tsujino, H., Urakawa, S., Nakano, H., et al.: JRA-55 based surface dataset for driving ocean–sea-ice models (JRA55-do), Ocean Modelling, 130, 79–139, https://doi.org/10.1016/j.ocemod.2018.07.002, 2018.
- [79]Tsujino, H., Urakawa, S., Nakano, H., Small, R. J., et al.: input4MIPs.CMIP6.OMIP.MRI.MRI-JRA55-do-1-4-0, Earth System Grid Federation, https://doi.org/10.22033/ESGF/input4MIPs.10842, 2019.
- [81]Wang, Q.: FESOM2 meshes: fArc, Zenodo, https://doi.org/10.5281/zenodo.23139470, 2026.
- [82]Wang, Q., Wekerle, C., Danilov, S., Wang, X., and Jung, T.: A 4.5 km resolution Arctic Ocean simulation with the global multi-resolution model FESOM 1.4, Geoscientific Model Development, 11, 1229–1255, https://doi.org/10.5194/gmd-11-1229-2018, 2018.
- [83]Williams, S., Waterman, A., and Patterson, D.: Roofline: an insightful visual performance model for multicore architectures, Communications of the ACM, 52, 65–76, https://doi.org/10.1145/1498765.1498785, 2009.
- [84]Wulf, W. A. and McKee, S. A.: Hitting the memory wall: implications of the obvious, ACM SIGARCH Computer Architecture News, 23, 20–24, https://doi.org/10.1145/216585.216588, 1995.
- [85]Yeager, S. G. and Large, W. G.: CORE.2 Global Air-Sea Flux Dataset, Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory, https://doi.org/10.5065/D6WH2N0S, 2008.
- [86]Zeman, C. and Schär, C.: An ensemble-based statistical methodology to detect differences in weather and climate model executables, Geoscientific Model Development, 15, 3183–3203, https://doi.org/10.5194/gmd-15-3183-2022, 2022.








