betaFeedback

WorksPhysics and Astronomy

High-level, efficient ocean modeling at all scales with Oceananigans

Abstract

We describe the user interface, governing equations, and numerical methods underpinning the ocean modeling software called "Oceananigans". Oceananigans development has been led by the Climate Modeling Alliance as part of a larger effort to build a trainable climate model with quantifiable uncertainty. Notably, Oceananigans is written in the Julia programming language, is optimized for GPUs, and can efficiently simulate all scales of ocean motion ranging from millimeter-scale turbulence in a small box to planetary-scale ocean circulation. We argue that this combination of capabilities is enabled by a design that combines a basic, easy-to-optimize, GPU-friendly structured finite volume algorithm exposed behind a high-level, programmable user interface. This design negotiates a dual mandate for highest-possible performance (to support state-of-the-art applications) and enhanced accessibility (to facilitate adoption and development). The dual mandate aims ultimately to accelerate the progress of Earth system science.

Key Points:

  • Oceananigans implements a programmable user interface for simulating oceanic motion at all scales.

  • A high-level interface, basic numerics, and GPU acceleration yield accessible code capable of high-fidelity simulations.

  • Combining accessibility with state-of-the-art performance accelerates model development, and thus the progress of Earth system science.

Introduction

Only numerical models can entirely synthesize the vast accumulation of knowledge about the ocean and Earth system. As a result, the capabilities of modeling software rate-limit progress in Earth system science. Since the first general circulation models ran on primitive computers ([94, 18]), advances in hardware, numerical methods, and the approximate parameterization of otherwise unresolved processes have improved the fidelity of ocean simulations ([47]). But the gap between potential and practice in ocean modeling is widening because advances in software and hardware technology have outpaced model development. Today, most ocean modeling software (i)(i) can tackle only a limited subset of ocean modeling problems, (ii)(ii) does not run on the fastest computers (most of which use GPUs), (iii)(iii) relies on outdated user interfaces that are less efficient than modern approaches and unfamiliar to a new generation of programmers.

This paper describes the design of Oceananigans, ocean modeling software written in the Julia programming language ([13]). Oceananigans development was initiated by the Climate Modeling Alliance as part of a larger effort to develop a climate model automatically-calibrated to observations and targeted large eddy simulations, and with quantified uncertainty ([108]). Oceananigans proposes to close the gap between potential and practice in ocean modeling with three interlocking design elements: (i)(i) a single modeling framework that spans the full range of ocean-flavored fluid dynamics from millimeter-scale direct numerical simulation, through meter-scale large eddy simulation, to planetary-scale ocean circulation, (ii)(ii) a structured finite-volume algorithm optimized for GPUs that yields significant performance gains over existing CPU-based codes, and (iii)(iii) a programmable, script-based user interface that leverages modern language features for configuring, running, and analyzing simulations. In this way, Oceananigans is a modeling system for all scales, designed to enable low-cost efficient high resolution simulations, and controlled with a high level interface optimized for productivity. The goal of this paper is not to report on all aspects of the model, which is rapidly evolving, but rather to describe how each of these design elements is allowing fast progress in both the model development and the science it allows. We demonstrate their cooperative utility through a series of examples that progress from idealized two-dimensional turbulence to realistic global ocean simulations.

Interlocking design elements: fast, flexible, and friendly

The evolution of global ocean circulation over millennia is controlled by turbulent mixing with scales that range down to seconds and millimeters. Two distinct systems have evolved to model this huge range of oceanic motion: “GCMs” (general circulation models) for hydrostatic regional-to-global scale simulations, and simpler software for nonhydrostatic large eddy simulations (LESs) with meter-scale grid spacing that are high-fidelity but limited in duration and extent. Compared to LESs, GCMs usually invoke more elaborate numerical methods and parameterizations to cope with the global ocean’s complex geometry and the more significant impacts of unresolved subgrid processes. Oceananigans bridges this divide: beginning as software for LES [97] by refining an approach for hybrid hydrostatic/nonhydrostatic dynamical cores pioneered by MITgcm [75], Oceananigans has grown into a single framework for simulating ocean-flavored fluid dynamics at any scale, from millimeters all the way to the full globe.

A structured finite volume algorithm on C-grids, optimized from the start for GPUs [116], yields significant acceleration over existing CPU-based codes [118]. LES-inspired, minimally dissipative advection schemes [115] achieve up to 2x higher effective resolution than existing methods, automatically adapting to local flow conditions and grid spacing. The resulting modeling system combines regular, structured-grid data layouts, GPU-efficient algorithms, and minimally dissipative advection schemes. Our working hypothesis is that these features enable a resolution-focused approach to accuracy, in which finer grids [22, 60, 23] can offset the need for more complex approaches such as generalized vertical coordinates [111, 68, 93], Lagrangian vertical advection [49, 46], or unstructured horizontal grids [102, 28, 65]. This hypothesis is under ongoing evaluation. Moreover, this recipe achieves a synergy between performance on GPUs, a resolution-based approach to accuracy, and the goal of providing efficient algorithms for modeling oceanic motion at both small nonhydrostatic scales and large hydrostatic scales. Detailed performance analysis is provided by Silvestri et al. [118] and Silvestri et al. [116]; automated benchmarks are run as part of the test suite and are published at numericalearth.github.io/OceananigansBenchmarks, providing up-to-date performance information as the software, hardware, and compiler infrastructure evolve.

Equal to numerical performance for determining the time-to-science (i.e., the amount of time it takes to produce a scientific result) is the productivity that scientists achieve when using modeling software. In 1984, Cox published what we believe is the first description of generalizable ocean modeling software [26, 47], featuring a multi-step workflow of source code modifications followed by namelist configuration. With more than forty years of progress in software engineering and more than a billion times more computational power, most of today’s ocean models bear little resemblance to the Cox model — except, perhaps, for their user interfaces. Oceananigans implements a programmable, script-first user interface for configuring, running, monitoring, and analyzing simulations, inspired primarily by Dedalus [19], with similar approaches taken by SpeedyWeather [62] and Thetis [99, 58]. A central argument of this paper is that such programmable interfaces — which, unlike text-based namelists, provide a seamless one-step workflow for setup, execution, analysis, and visualization [92, 54, 86, 99, 19] (Alnaes et al. 2015) — are effective for accelerating progress in Earth system science.

Programmable interfaces facilitate fast prototyping, collaboration through code sharing, and reproducible simulations.

The use of a modern programming language ties together the different elements of Oceananigans’ design. With Julia, Oceananigans provides a framework wherein new parameterizations [133], new numerical methods [115], and new algorithms for performance optimization [116] may be both prototyped and refined for production without re-implementation. Julia’s flexibility, expressiveness, and compilation to native machine code enable a wide range of performance optimizations, allowing us to achieve state-of-the-art performance on GPUs. Structured-grid algorithms reinforce the productivity benefits of a high-level language: their regular data structures and computational patterns simplify implementation, while the language facilitates rapid prototyping and extension.

We do not argue that Julia is strictly superior to other approaches; this kind of framework may also be implemented with domain-specific languages (DSLs) such as Python’s JAX [17] or C++’s AMReX. Python’s main advantage is its status as the lingua franca of programming: a much larger open-source community with greater resources devoted to its support and development. Some of these advantages are mitigated by Julia-Python interoperability, the advent of agentic code development, and the similarity between Julia and Python syntax. Both languages enable interactivity, extensibility, automatic installation, and portability to laptops and GPUs [17, 25] (Besard et al. 2018). Using a DSL like JAX can have advantages in accelerating development, especially in the initial stages of a project. At the same time, however, DSLs are restrictive, such that using Julia enables a wider range of strategies for performance optimization comparable to C++ with AMReX without the cost of the “two-language problem” and the advantage of flexible model design, algorithm design, and user interface design. Realizing these benefits demands more effort upfront, but can pay off in the long run.

The need for next-generation frameworks is evidenced by the recent proliferation of similar efforts. Regional-mom6 [12] and CROCO-tools [56] provide Python software that automates the configuration of existing models [2, 8], leveraging mature modeling systems with massive user communities and decades of development history, which, in the case of CROCO, support both hydrostatic and nonhydrostatic simulations [9]. New modeling systems are also being developed in modern programming languages: ClimaAtmos [140] and ClimaLand [31] provide atmosphere and land surface modeling in Julia; Veros (hydrostatic ocean simulations, [48]; [81]) and NeuralGCM (hydrostatic atmosphere simulations, [64]; [142]) are written in Python using JAX [17]; and REMORA [61] provides a GPU-capable re-implementation of the Regional Ocean Modeling System (ROMS) built on the AMReX adaptive mesh refinement framework. JAX-based software benefits from the ubiquity of the Python ecosystem, offers portable performance between CPUs, GPUs, and TPUs, and most notably supports automatic differentiation for gradient-based parameter estimation or training of machine learning components [63, 64] and adjoint-based data assimilation [120]. Oceananigans achieves performance portability via KernelAbstractions [25] and differentiability via Enzyme and Reactant [80, 79], which represent newer alternatives to JAX.

Outline of this paper

The purpose of this paper is to explain Oceananigans’ design philosophy and to demonstrate its advantages by providing a sketch of its capabilities, including how its user interface both simplifies basic simulations while enabling complex, creative science. We note that our goal is not to exhaustively document every closure, feature, or technical detail which are rapidly expanding thanks to a growing number of contributors. For comprehensive and up-to-date information, we instead refer the reader to the Oceananigans documentation at clima.github.io/OceananigansDocumentation, which is a living document that evolves with the software. (Benchmarks may also be found on Oceananigans’ performance dashboard and also in Silvestri et al. [118], Silvestri et al. [116]). To that end, this paper proceeds in § 2 by illustrating the basic form of Oceananigans’ programmable interface with two classroom examples: two-dimensional turbulence, and a passive tracer advected by two-dimensional turbulence and forced by user-specified forcing.

§ 3 continues by writing down the Boussinesq governing equations that underpin Oceananigans’ nonhydrostatic and hydrostatic models. We then sketch out Oceananigans capabilities with a series of examples that progress from basic direct numerical simulations of cabbeling, to realistic tidally-forced nonhydrostatic large eddy simulations over a headland, to a 1/61/6th degree hydrostatic, eddying, global ocean simulation. To summarize: Oceananigans supports both nonhydrostatic and hydrostatic simulations on rectilinear and curvilinear grids with or without bathymetry. A suite of pressure solvers, WENO-based advection schemes (C. Shu, [112]), Laplacian diffusivity, and subgrid closures including constant and dynamic Smagorinsky (Smagorinsky, [119]; Lilly, [72]; Bou-Zeid et al., [16]) and Anisotropic Minimum Dissipation (Rozema et al., [106]; Vreugdenhil & Taylor, [129]) support nonhydrostatic direct and large eddy simulations of meter-scale phenomena in periodic, closed, or open domains. Nonhydrostatic simulations are typically run with a rigid lid, but an experimental time-implicit free surface formulation is also supported. Hydrostatic regional to global ocean simulations with an implicit or split-explicit free surface formulation and zz or z∗z^{*} vertical coordinate are supported on rectilinear, latitude-longitude, tripolar (Murray, [82]), and cubed sphere (Adcroft et al., [4]) grids with second-order and WENO-based vector invariant schemes (Silvestri et al., [115]), the Gent–McWilliams parameterization (Gent & Mcwilliams, [39]), horizontal biharmonic diffusivity, and RiRi-based, one-equation (Wagner et al., [133]) and two-equation vertical mixing schemes (Umlauf & Burchard, [125]). Linear, quadratic (Roquet, Madec, Brodeau, & Nycander, [104]), and polynomial (Roquet, Madec, McDougall, & Barker, [105]) equations of state are supported (with arbitrary gravitational direction for nonhydrostatic simulations) as well as traditional, non-traditional, spherical, and β\beta-plane Coriolis forces. Lagrangian particles, Stokes drift, and biogeochemistry are supported.

Oceananigans’ programmable interface facilitates composition with other software, and a growing ecosystem of companion packages has rapidly formed around it — developed largely by students, postdocs, and independent collaborators working outside the core team. That this ecosystem emerged while Oceananigans was still maturing, and before comprehensive validation, suggests that the barrier to extending ocean modeling software in Julia is low enough to justify building on a young platform. The companion package “NumericalEarth” (Wagner, Silvestri, et al., [134]) provides a coupled modeling framework for integrating Oceananigans ocean models together with sea ice simulations and either prescribed or prognostic atmospheres, using fluxes computed according to Monin–Obukhov similarity theory (for example, Edson et al., [36]). NumericalEarth also implements utilities for automatic, script-based downloading of bathymetric data, ocean and sea ice physical and biogeochemical reanalysis products, and atmospheric reanalysis. ClimaSeaIce (Silvestri, Wagner, et al., [117]) provides a sea ice modeling system that uses Oceananigans utilities and abstractions to implement simple sea ice and snow thermodynamics with a viscoplastic rheology. ClimaOcean couples Oceananigans and ClimaSeaIce with the ClimaAtmos (Yatunin et al., [140]) and ClimaLand (Deck et al., [31]) Julia packages for global Earth System simulations.

We conclude in § 5 by describing outstanding problems and questions, outlining ongoing development work, and anticipating the next major innovations in ocean modeling that will someday render the present work obsolete.

using Oceananigans, CUDA # using CUDA allows us to use an Nvidia GPU

# The third dimension is "flattened" to reduce the domain from three to two dimensions.
topology = (Periodic, Periodic, Flat)
architecture = GPU() # CPU() works just fine too for this small example.
x = y = (0, 2π)
grid = RectilinearGrid(architecture; size=(256, 256), x, y, topology)

model = NonhydrostaticModel(grid; advection=WENO(order=9))

ϵ(x, y) = 2rand() - 1 # Uniformly-distributed random numbers between [-1, 1).
set!(model, u=ϵ, v=ϵ)

simulation = Simulation(model; Δt=0.01, stop_time=10)
run!(simulation)

u, v, w = model.velocities
ζ = ∂x(v) - ∂y(u)

using CairoMakie
heatmap(ζ, colormap=:balance, axis=(; aspect=1))

Listing 1: A Julia script that uses Oceananigans and the Julia plotting library CairoMakie to set up, run, and visualize a simulation of two-dimensional turbulence on a Graphics Processing Unit (GPU). The initial velocity field, defined on lines 11-12, consists of random numbers uniformly-distributed between −1-1 and 1. The vorticity ζ=∂xv−∂yu\zeta= \partial_x v - \partial_y u is defined on line 18. The solution is visualized in Figure 1.

Vorticity and passive tracer in two-dimensional turbulence

Figure 1. (a) Vorticity after t=10t = 10 and (b) a passive tracer injected by a moving source at t=2.5t = 2.5 in a simulation of two-dimensional turbulence using an implicitly-dissipative advection scheme. The figures are oriented so that xx varies with the horizontal axis and yy varies with the vertical axis (see listings 1 and 2).

Oceananigans, the library

Oceananigans is a library of tools for building models by writing programs called “scripts”. This design departs from typical monolithic interfaces that ingest lists of flags and parameters from non-executable text files. By blending mathematical symbols with verbose natural language names, Oceananigans syntax tries to enable evocative scripting that approaches the effectiveness of writing for communicating computational science.

Hello, ocean

The process of learning Oceananigans best begins by running simple simulations. Our first example in listing 1 sets up, runs, and visualizes a simulation of two-dimensional turbulence. The 21 lines of listing 1 illustrate one of Oceananigans’ main achievements: a numerical experiment may be completely described by a single script. Moreover, Oceananigans’ interface is modular and concise, so that the complexity of an Oceananigans script is uniquely determined by the complexity of the problem being solved. To execute the code in listing 1, we need to copy it into a file (call this, for example, hello_ocean.jl) and execute it by typing include("hello_ocean.jl") in a Julia session.

Oceananigans scripts are conventionally organized into four sections. The first three define the grid, model, and simulation, and conclude with execution of the simulation. The fourth section, often implemented separately for complex or expensive simulations, performs post-processing and analysis. In listing 1, the grid defined on lines 4–7 determines the problem geometry, grid spacing, and machine architecture. To use a CPU instead of a GPU, one writes CPU() in place of GPU() on line 5: no other changes to the script are required.

Lines 9–12 define the model, which solves the Navier–Stokes equations in two dimensions with a 9th-order Weighted, Essentially Non-Oscillatory (WENO) advection scheme (see Appendix § 8 and also Silvestri et al. [115] for more information about WENO). The velocity components uu and vv are initialized with uniformly distributed random numbers within [−1,1)[-1,1). The model definition can also encompass forcing, boundary conditions, and

function circling_source(x, y, t)
    δ, ω, r = 0.1, 2π/3, 2
    dx = x + r * cos(ω * t)
    dy = y + r * sin(ω * t)
    return exp(-(dx^2 + dy^2) / 2δ^2)
end

forcing = (; c = circling_source)
model = NonhydrostaticModel(; grid, advection=WENO(order=9), tracers=:c, forcing)

Listing 2: Implementation of a moving source of passive tracer with a function in a two-dimensional turbulence simulation. These lines of code replace the model definition on line 9 in listing 1. Julia supports Unicode variable names, including Greek letters.

the specification of additional terms in the momentum and tracer equations such as Coriolis forces or turbulence closures.

Line 14 builds a simulation with a time-step Δt=0.01\Delta t = 0.01 which will run until t=10t = 10. (Time is non-dimensional in this example because the user has not specified physical units and because, by design, NonhydrostaticModel has no default parameters that could impose a unit system. Later examples illustrate how, with dimensional user input, variables like the time step and stop time can acquire user-specified dimensions.) Lines 17-18 analyze the final state of the simulation by computing vorticity, illustrating Oceananigans’ toolbox for composing discrete calculus and arithmetic operations. The same tools may be used to define online diagnostics to be periodically computed and saved to disk while the simulation runs. Line 21 concludes the numerical experiment with a visualization, which is shown in Figure 1.

Incorporating user code

With a programmable interface and aided by Julia’s just-in-time compilation, user functions specifying domain geometry, forcing, boundary conditions, and initial conditions can be incorporated directly into models without modifying source code or using a separate programming environment. To illustrate function-based forcing, we modify listing 1 with code that adds a passive tracer which is forced by a moving source that depends on xx, yy, tt. A visualization of the vorticity and tracer field generated by listings 1 and 2 are shown in Figure 1.

General tasks may be inserted into the time stepping loop by modifying the simulation. This supports things as mundane as printing a summary of the current model status or writing output, to more exotic tasks like nudging state variables or updating a diffusion coefficient based on an externally-implemented model.

Writing simulation output to disk is a well-known bottleneck in high-performance computing (see, for example, [73]). In traditional CPU-based models, I/O can consume a significant fraction of wall-clock time, particularly for high-frequency output of large three-dimensional fields. Some models mitigate this by dedicating a subset of MPI ranks to act as I/O servers that buffer and write data asynchronously while the remaining ranks continue computation ([32]) — an effective but complex solution that requires significant software infrastructure and sufficient nodes to justify the dedicated ranks. The GPU–CPU architecture exploited by Oceananigans offers a simpler alternative: because the CPU is largely idle during GPU computation, it can serve as an asynchronous I/O server. After GPU-to-CPU data transfer, the CPU can write output to disk concurrently with continued GPU computation. As with dedicated I/O servers, overlapping disk writes with computation can reduce output overhead, but does not eliminate data-transfer costs or limits imposed by sustained storage throughput. This approach requires no additional hardware, no dedicated ranks, and works even on a single node with a single GPU.

Abstractions for arithmetic and discrete calculus

Abstractions representing unary, binary, and calculus operators produce a system for building “lazy” expression trees that are evaluated only upon request (for example, if their output should be periodically saved to disk during a simulation). Example calculations representing vorticity, ζ=∂xv−∂yu\zeta= \partial_x v - \partial_y u, speed s=u2+v2s = \sqrt{u^2 + v^2}, and the xx-integral of enstrophy Z=∫02πζ2 dxZ = \int_0^{2\pi} \zeta^2\,\mathrm{d}x are shown in listing 3, alongside an example that uses the same staggered-grid stencil operators found in Oceananigans source code to compute the gradient Richardson number in a user script via KernelFunctionOperation. Oceananigans and Breeze [132] provide built-in diagnostics, while Oceanostics [24] provides additional reusable diagnostics built on these abstractions, so users need not implement each diagnostic in their simulation scripts.

1  u, v, w = model.velocities
3  # "Lazy" expression trees and reductions representing computations:
4  ζ = ∂x(v) - ∂y(u)
5  s = √(u^2 + v^2)
6  Z = Integral(ζ^2, dims=1)

8  # Building and computing a Field that instantiates an AbstractOperation:
9  ζ_field = Field(∂x(v) - ∂y(u))

11 # KernelFunctionOperations: user-defined stencil computations
12 @inline φ²(i, j, k, grid, φ, args...) = φ(i, j, k, grid, args...)^2
14 @inline function Ri(i, j, k, grid, velocities, buoyancy, tracers)
15 ∂z_u2 = ∂zᶠᶜᵃᵃ(i, j, k, grid, φ², ∂zᶠᶜᶠ, velocities.u)
16 ∂z_v2 = ∂zᶜᵃᶜᵃ(i, j, k, grid, φ², ∂zᶜᶠᶠ, velocities.v)
17 S² = ∂z_u2 + ∂z_v2
18 N² = ∂z_b(i, j, k, grid, buoyancy, tracers)
19 Ri = N² / S²
20 return ifelse(S² == 0, zero(grid), Ri)
21 end
23 Ri = KernelFunctionOperation(Center, Center, Face)(Ri, grid,
24     model.velocities, model.buoyancy, model.tracers)
25
26 # Save diagnostics periodically after computing online on GPU
27 output_writer = JLD2Writer(model, (; ζ, Ri), schedule=TimeInterval(1hour))

Listing 3: Illustration of abstractions for expression trees, reductions, and user-defined stencil computations called “AbstractOperations” in Oceananigans. AbstractOperations are “lazy”, in the sense that they represent potential computations but do not instantiate the computation directly. The first half shows simple expression trees for vorticity, speed, and enstrophy. The second half computes the gradient Richardson number using the same staggered-grid interpolation and differentiation operators found in Oceananigans source code; KernelFunctionOperation provides the bridge that wraps these source-code-level operators into the AbstractOperation interface for use in user scripts.

The second half of listing 3 illustrates the staggered-grid interpolation operators Ixaca\mathcal{I}_{x}^{aca}, Iyaca\mathcal{I}_{y}^{aca} and differentiation operators ∂zfcf\partial_{z}^{fcf}, ∂zcff\partial_{z}^{cff} that encode the finite volume staggering of the C-grid. These are the same operators used throughout the Oceananigans source code to build numerical algorithms — for example, the Pacanowski–Philander vertical mixing scheme (Pacanowski & Philander, [85]) is implemented with identical stencil calls. KernelFunctionOperation provides the bridge that exposes these source-code-level operators to user scripts: wrapping a hand-written kernel function into the AbstractOperation interface so that the resulting RiRi object can be saved to disk, reduced, or composed with further operations, just like the simple expression trees above. The example computes the gradient Richardson number Ri=N2/S2Ri = N^2/S^2 and because the user’s kernel calls the same operators that internal parameterizations use, the boundary between “user” and “developer” code is erased. (Note, a more stable RiRi stencil is described by Griffies et al. [45].)

Governing equations, parameterizations, and illustrative examples

Oceananigans implements two “models” for ocean-flavored fluid dynamics: the HydrostaticFreeSurfaceModel, and the NonhydrostaticModel. Each represents a template for equations that govern the evolution of momentum and tracers. Both models are incompressible and make the Boussinesq approximation, which means that the density of the modeled fluid is decomposed into a constant reference ρ0\rho_0 and a small dynamic perturbation ρ′\rho',

ρ(x,t)=ρ0+ρ′(x,t),where ∣ρ′∣≪ρ0,(1)\rho(x,t) = \rho_0 + \rho'(x,t), \qquad\text{where } |\rho'| \ll\rho_0, \tag*{(1)}

and x=(x,y,z)x = (x,y,z) is position and tt is time.

The relative smallness of ρ′\rho' reduces conservation of mass to a statement of incompressibility called the continuity equation,

∇⋅u=0,(2)\nabla\cdot\mathbf{u} = 0, \tag*{(2)}

where

u=defux^+vy^+wz^,(3)\mathbf{u} \mathrel{\overset{\mathrm{def}}{=}} u\hat{x} + v\hat{y} + w\hat{z}, \tag*{(3)}

is the three-dimensional velocity field. Within the Boussinesq approximation, the momentum ρ0u\rho_0\mathbf{u} varies only with the velocity u\mathbf{u}. The effect of density variations is encapsulated by a buoyant acceleration,

b=def−gρ′ρ0,(4)b \mathrel{\overset{\mathrm{def}}{=}} -\frac{g\rho'}{\rho_0}, \tag*{(4)}

where gg is gravitational acceleration. The “buoyancy” bb acts against the direction of gravity.

For gravity directed along −z^-\hat{z}, the total pressure PP, relative to a constant reference pressure, is decomposed into

P=−ρ0gz+ρ0p(x,t),(5)P = -\rho_0gz + \rho_0p(x,t), \tag*{(5)}

where zz is height, −ρ0gz-\rho_0gz is the static contribution to pressure that opposes the gravitational force associated with the reference density ρ0\rho_0, and ρ0p\rho_0p represents the dynamic anomaly. pp is called the kinematic pressure.

The NonhydrostaticModel

The NonhydrostaticModel represents the Boussinesq equations formulated without making the hydrostatic approximation typical to general circulation models. The NonhydrostaticModel has a three-dimensional prognostic velocity field.

The NonhydrostaticModel momentum equation

The NonhydrostaticModel’s momentum equation incorporates advection by a background velocity field, Coriolis forces, surface wave effects via the Craik-Leibovich asymptotic model [27, 53], a buoyancy term allowed to be a nonlinear function of tracers and depth, a stress divergence derived from molecular friction or a turbulence closure, and a user-defined forcing term. Using the Boussinesq approximation in (1) and the pressure decomposition in (5), the generic form of NonhydrostaticModel’s momentum equation is

∂tu=−∇p−(u⋅∇)u−(ub⋅∇)u−(u⋅∇)ub⏟advection−f×u⏟Coriolis+(∇×us)×u+∂tus⏟Stokes forcing−bg^⏟buoyancy−∇⋅τ⏟closure+Fu⏟forcing,(6)\partial_t \boldsymbol{u} = -\nabla p - \underbrace{(\boldsymbol{u}\cdot\nabla)\boldsymbol{u} - (\boldsymbol{u}_b\cdot\nabla)\boldsymbol{u} - (\boldsymbol{u}\cdot\nabla)\boldsymbol{u}_b}_{\text{advection}} - \underbrace{\boldsymbol{f}\times\boldsymbol{u}}_{\text{Coriolis}} + \underbrace{(\nabla\times\boldsymbol{u}_s)\times\boldsymbol{u}+\partial_t\boldsymbol{u}_s}_{\text{Stokes forcing}} - \underbrace{b\hat{\boldsymbol{g}}}_{\text{buoyancy}} - \underbrace{\nabla\cdot\boldsymbol{\tau}}_{\text{closure}} + \underbrace{\boldsymbol{F}_u}_{\text{forcing}}, \tag*{(6)}

where ub\boldsymbol{u}_b is a prescribed and divergence-free “background” velocity field, pp is the kinematic pressure, f\boldsymbol{f} is the background vorticity associated with a rotating frame of reference, us\boldsymbol{u}_s is the Stokes drift profile associated with a prescribed surface wave field, bb is buoyancy, g^\hat{\boldsymbol{g}} is the gravitational unit vector (usually pointing downwards, that is, g^=−z^\hat{\boldsymbol{g}}=-\hat{\boldsymbol{z}}), τ\boldsymbol{\tau} is the stress tensor associated with molecular viscous or subgrid turbulent momentum transport, and Fu\boldsymbol{F}_u is a body force.

To integrate equation (6) while enforcing (2), we use a pressure-correction method that requires solving a three-dimensional Poisson equation to find pp, which can be derived by taking the divergence of (6). This Poisson equation is often a computational bottleneck in curvilinear or irregular domains, and its elimination is the main motivation for making the hydrostatic approximation when formulating the HydrostaticFreeSurfaceModel, as described in § 4.3. For rectilinear grids, we solve the Poisson equation using a direct FFT-based or mixed FFT-tridiagonal solver [109], providing substantial acceleration over MITgcm’s conjugate gradient pressure solver [75]. The FFT-based pressure solver uses the distributed transpose algorithm described in Appendix § 10 and Figure C1, and has been used in simulations on 1,024 GPUs [91]. Detailed scaling results will be presented in a forthcoming study of FFT-based solvers for complex domains. In irregular domains, we either use a masking method that permits an approximate solution of the pressure Poisson equation with the FFT-based method, or an iterative conjugate gradient solver that leverages the FFT-based solver as a preconditioner. The pressure correction scheme is described further in subsection A2.

Figure C1. A schematic showing the distributed Poisson solver procedure with a pencil parallelization that divides the domain in two ranks in both xx and yy. The schematic highlights the data layout in the ranks during each operation.

Advection in the NonhydrostaticModel may be formulated in “flux form”,

advection=uj∂jui+ubj∂jui+uj∂jubi=∂j[(uj+ubj)ui+ujubi],(7)\mathrm{advection}=u_j\partial_j u_i+u_{bj}\partial_j u_i+u_j\partial_j u_{bi}=\partial_j\left[(u_j+u_{bj})u_i+u_j u_{bi}\right], \tag*{(7)}

where we have used the continuity equation (2). We use indicial notation; for example, the ii-th component of the advection term is [(u⋅∇)u]i=uj∂jui[(\boldsymbol{u}\cdot\nabla)\boldsymbol{u}]_i=u_j\partial_j u_i. (See the text surrounding equations (18)–(20) for a discussion of advection term formulation in the HydrostaticFreeSurface model.)

The formulation of the Stokes drift terms in (6) means that u\boldsymbol{u} is the Lagrangian-mean velocity when Stokes drift effects are included (see, for example, Wagner et al. [130]). With a Lagrangian-mean formulation, equations (2) and (6) are consistent only when us\boldsymbol{u}_s is non-divergent — or equivalently, when us\boldsymbol{u}_s is obtained by projecting the divergence out of the usual Stokes drift [126]. A consistent two-way coupling between surface waves and currents that conserves momentum and energy is developed by Vanneste and Young [127]. As discussed by Wagner et al. [130], the Lagrangian-mean formulation of (6) means that closures for LES strictly dissipate resolved kinetic energy, avoiding the inconsistency between resolved and subgrid fluxes affecting typical LES formulated in terms of the Eulerian-mean velocity (see also [89, 131, 71]).

The labeled terms in (6) are controlled by arguments to NonhydrostaticModel invoked in both of listings 1 and 2. For example, “advection” chooses a numerical scheme to approximate the advection term in (6) and (7). As another example, we consider configuring the closure term in (6) to represent (i)(i) molecular diffusion by a constant-coefficient Laplacian ScalarDiffusivity, (ii)(ii) turbulent stresses approximated by the SmagorinskyLilly eddy viscosity model ([119, 72]) for large eddy simulation, or (iii)(iii) omitting it entirely and relying on WENO advection schemes for dissipation and stability ([96, 133]). In these three cases, the closure flux divergence ∇⋅τ=∂mτnm\nabla\cdot\boldsymbol{\tau} = \partial_m \tau_{nm} in indicial notation becomes

−∂mτnm={∂m(ν∂mun)(ScalarDiffusivity)0(nothing)∂m(2CsΔ2∣Σ∣Σnm)(SmagorinskyLilly)(8)-\partial_m \tau_{nm} = \begin{cases} \partial_m(\nu\partial_m u_n) & \text{(ScalarDiffusivity)} \\ 0 & \text{(nothing)} \\ \partial_m\left(2 C_s \Delta^2 \lvert\Sigma\rvert\Sigma_{nm}\right) & \text{(SmagorinskyLilly)} \end{cases} \tag*{(8)}

where ν\nu is the Laplacian viscosity, Σnm=12(∂mun+∂num)\Sigma_{nm} = \frac{1}{2}(\partial_m u_n + \partial_n u_m) is the strain rate tensor, ∣Σ∣\lvert\Sigma\rvert is the magnitude of the strain rate tensor, CsC_s is the SmagorinskyLilly model constant, Δ\Delta scales with the local grid spacing, and νe\nu_e is the eddy viscosity.

ScalarDiffusivity diffusion coefficients may also vary in time- and space or depend on model fields. Other closure options for NonhydrostaticModel include fourth-order ScalarBiharmonicDiffusivity, various Lagrangian-averaged or directionally-averaged flavors of DynamicSmagorinsky ([16]), and the AnisotropicMinimumDissipation turbulence closure ([106, 1, 129, 130]) for large eddy simulations. Vertically-implicit time-discretization is supported with all scalar-type diffusivities, including ScalarDiffusivity and LES closures.

We note that LES may be conducted solely with WENO advection schemes that dissipate grid-scale kinetic energy undergoing a forward cascade from large to small scales. However, no reliable method has yet been developed to diagnose the dissipation of kinetic energy in such “implicit LES” simulations, so, at the time of writing, explicit closures must be included if such diagnostics are required.

We illustrate how NonhydrostaticModel may be used to model fluid motion on the smallest scales in listing 4, which implements a direct numerical simulation of uniform flow past a cylinder with no-slip boundary conditions. The goal of direct numerical simulation is to resolve all scales of motion down to the viscous dissipation scale. For this purpose, we use a ScalarDiffusivity formulated with molecular viscosity, and use a centered second-order advection scheme which adds no additional spurious dissipation. Lines 6–7 of listing 4 embed a cylindrical mask in a RectilinearGrid using a GridFittedBoundary, which generalizes to arbitrary three-dimensional shapes. The no-slip condition is specified on lines 11–12 by invoking ValueBoundaryCondition (a synonym for “Dirichlet” boundary condition): ValueBoundaryCondition enforces a near-wall tangential velocity gradient, which in turn produces a viscous flux associated with the ScalarDiffusivity viscosity (resolving to the same formulation proposed in a seminal paper by Harlow et al., [51]). Other choices include GradientBoundaryCondition (Neumann), MixedBoundaryCondition (Robin condition on a linear combination of the scalar value and gradient), FluxBoundaryCondition (direct imposition of fluxes), OpenBoundaryCondition (for non-trivial boundary-normal velocity

1 r, U, Re, Ny = 1/2, 1, 1000, 2048

3 grid = RectilinearGrid(GPU(), size=(2Ny, Ny), x=(-3, 21), y=(-6, 6),
    topology=(Periodic, Bounded, Flat))

6 cylinder(x, y) = (x^2 + y^2) ≤ r^2
7 grid = ImmersedBoundaryGrid(grid, GridFittedBoundary(cylinder))

9 closure = ScalarDiffusivity(ν=1/Re)

11 no_slip = FieldBoundaryConditions(immersed=ValueBoundaryCondition(0))
12 boundary_conditions = (u=no_slip, v=no_slip)

14 # Implement a sponge layer on the right side of the domain that
15 # relaxes v → 0 and u → U over a region of thickness δ
16 @inline mask(x, y, δ=3, x0=21) = max(zero(x), (x - x0 + δ) / δ)
17 Fu = Relaxation(target=U; mask, rate=1)
18 Fv = Relaxation(target=0; mask, rate=1)

20 model = NonhydrostaticModel(; grid, closure, boundary_conditions, forcing=(u=Fu, v=Fv))

Listing 4: Direct numerical simulation of flow past a cylinder at various Reynolds numbers ReRe. The domain is periodic in xx and a sponge layer on the right side of the domain relaxes the solution to u=u∞x^u = u_\infty\hat{x} with u∞=1u_\infty= 1. The experiment can be converted to a large eddy simulation (thereby sending Re→∞Re \to\infty) by replacing the no-slip boundary conditions with an appropriate drag model and either (i) using an appropriate turbulence closure or (ii) using the WENO(order=9) advection scheme with no turbulence closure. Visualizations of the DNS and LES cases are shown in Figure 2.

Vorticity snapshots showing DNS at Reynolds numbers 100 and 1000 and LES at Re tending to infinity

Figure 2. Vorticity snapshots in simulations of flow around a cylinder. The top two panels show vorticity in direct numerical simulations (DNS) that use a molecular ScalarDiffusivity closure and Centered(order=2) advection with Reynolds numbers (Re\mathrm{Re}) of 100 (top panel) and 1000 (middle panel). The bottom panel shows a large eddy simulation (LES) with no closure and a WENO(order=9) advection scheme. Note that the sponge layer could be replaced with a PerturbationAdvection open boundary condition if desired. The figures are oriented so that xx varies in the horizontal direction and yy varies in the vertical direction.

fields), as well as a helper function BulkDrag (for quadratic drag fluxes, which are a kind of FluxBoundaryCondition).

Results obtained with listing 4 for Re=100Re = 100, Re=1000Re = 1000, and a modified version of listing 4 for large eddy simulation (Re→∞Re \to\infty) are visualized in Figure 2. To adapt listing 4 for LES, the closure is eliminated in favor of a 9th-order WENO advection scheme, and the no-slip boundary condition is replaced with a quadratic drag boundary condition with a drag coefficient estimated from similarity theory using a constant estimated roughness length.

The NonhydrostaticModel tracer conservation equation

The buoyancy term in (6) requires tracers, and can be formulated to use buoyancy itself as a tracer, or to depend on temperature TT and salinity SS. For seawater, a 54-term polynomial approximation TEOS10EquationOfState ([76, 105]) is implemented in the auxiliary package SeawaterPolynomials, along with quadratic approximations to TEOS-10 ([104]) and a LinearEquationOfState. All tracers — either “active” tracers required to compute the buoyancy term, as well as additional user-defined passive tracers — obey the tracer conservation equation

∂tc=−∇⋅[(u+ub+up)c+ucb]+c∇⋅up−∇⋅Jc+Sc+Fc(9)\partial_t c = -\nabla\cdot\left[(\boldsymbol{u}+\boldsymbol{u}_b+\boldsymbol{u}_p)c+\boldsymbol{u}c_b\right] + c\nabla\cdot\boldsymbol{u}_p - \nabla\cdot\boldsymbol{J}_c + S_c + F_c \tag*{(9)}

where cc represents any tracer, cbc_b represents a prescribed background tracer concentration for cc, up\boldsymbol{u}_p is the velocity associated with biogeochemical processes (and which, in general, can have non-zero divergence), Jc\boldsymbol{J}_c is a tracer flux associated with molecular diffusion or subgrid turbulence, ScS_c is a source or sink term associated with biogeochemical processes, such as chemical reactions and biological growth and mortality, (provided, for example, by external packages like OceanBioME; [122]), and FcF_c is a user-defined source or sink. The formulation of tracer advection in (9) leverages ∇⋅u=0\nabla\cdot\boldsymbol{u}=0 and ∇⋅ub=0\nabla\cdot\boldsymbol{u}_b=0.

grid = RectilinearGrid(GPU(), topology = (Bounded, Flat, Bounded),
                       size = (4096, 1024), x = (0, 2), z = (-0.5, 0))

closure = ScalarDiffusivity(ν=1.15e-6, κ=1e-7)

using SeawaterPolynomials: TEOS10EquationOfState
equation_of_state = TEOS10EquationOfState(reference_density=1000)

buoyancy = SeawaterBuoyancy(gravitational_acceleration = 9.81,
                             constant_salinity = 0, # set S=0 and simulate T only
                             equation_of_state)

model = NonhydrostaticModel(; grid, buoyancy, closure, tracers=:T)

Tᵢ(x, z) = z > -0.25 ? 7.55 : 1
Ξᵢ(x, z) = 1e-2 * randn()
set!(model, T=Tᵢ, u=Ξᵢ, v=Ξᵢ, w=Ξᵢ)

Listing 5: Direct numerical simulation of convective turbulence driven by cabbeling between 1- and 7.55-degree freshwater. ν\nu denotes viscosity and κ\kappa denotes the tracer diffusivity. The diffusivity may also be set independently for each tracer.

A simulation with a passive tracer having a user-defined source term is illustrated by listing 2 and Figure 1. For a second example, we consider freshwater cabbeling (see, for example, Bisits et al., [14]). Cabbeling occurs when two water masses of similar density mix to form a new water mass which, due to the nonlinearity of the equation of state, is denser than either of its constituents. Freshwater, for example, is densest at 4 degrees Celsius, while 1- and 7.55-degree water are lighter with roughly the same density. We implement a direct numerical simulation (DNS) in which 7.55-degree water overlies 1-degree water, using the TEOS10EquationOfState provided by the auxiliary package SeawaterPolynomials. The script is shown in listing 5. The resulting density and temperature fields after 1 minute of simulation are shown in Figure 3. Note that the TEOS10EquationOfState typically depends on both temperature and salinity tracers, but listing 5 specifies a constant salinity S=0S = 0 and thus avoids allocating memory for or simulating salinity directly. Also note, DNS is not Oceananigans’ strength due to its second-order finite volume formulation, compared to pseudospectral formulations (Lecoanet et al., [69]). (This contrasts with Oceananigans’ capabilities for large eddy simulation, where simulation error is dominated by the representation of grid-scale dissipation rather than the formal accuracy of the discretization.)

Density and temperature fields in a direct numerical simulation of cabbeling in freshwater

Figure 3. Density and temperature at t=1t = 1 minute in a direct numerical simulation of cabbeling in freshwater. Note that both fields span from x=0x = 0 to x=2x = 2 meters; only the left half of the density field and the right half of the temperature field are shown.

grid = RectilinearGrid(GPU(); size = (1024, 1024, 64),
                       x = (0, 4096), y = (0, 4096), z = (0, 128),
                       topology = (Periodic, Periodic, Bounded))

f, N², Ri = 1e-4, 1e-6, 1
parameters = (f=f, Λ=sqrt(N²/Ri)) # U = Λz, so Ri = N² / ∂z(U)^2 = N² / Λ² and Λ = N / √
    Ri.

@inline U(x, y, z, t, p) = + p.Λ * z
@inline B(x, y, z, t, p) = - p.f * p.Λ * y

background_fields = (u = BackgroundField(U; parameters),
                     b = BackgroundField(B; parameters))

model = NonhydrostaticModel(; grid, background_fields,
                            advection = WENO(order=9), coriolis = FPlane(; f),
                            tracers = :b, buoyancy = BuoyancyTracer())

∆z = minimum_zspacing(grid)
bi(x, y, z) = N² * z + 1e-2 * N² * ∆z * (2rand() - 1)
set!(model, b=bi)

Listing 6: Large eddy simulation of the Eady problem expanded around the background geostrophic shear with Ri=1Ri = 1.

We next consider a large eddy simulation of the Eady problem [35]. In the Eady problem, perturbations evolve around a basic state with constant shear Λ\Lambda in thermal wind balance with a constant meridional buoyancy gradient fΛf\Lambda, such that

u=Λz+u′,b=−fΛy+b′.(10)u = \Lambda z + u', \qquad b = -f\Lambda y + b'. \tag*{(10)}

We use Oceananigans’ BackgroundFields to simulate the nonlinear evolution of (u′,v,w)(u', v, w) and b′b' expanded around UU and BB in a doubly-periodic domain. We impose an initially stable density stratification with b′=N2zb' = N^2z and N2=10−6 s−2N^2 = 10^{-6}\,\mathrm{s}^{-2} superposed with random noise. The Richardson number of the initial condition is Ri=N2/(∂zU)2=N2/Λ2Ri = N^2/(\partial_z U)^2 = N^2/\Lambda^2; we choose mean shear Λ\Lambda so that Ri=1Ri = 1, which guarantees the basic state is unstable to baroclinic instability but stable to symmetric and Kelvin–Helmholtz instability [121]. A portion of the script is shown in listing 6.

Our Eady simulation uses fully-turbulence-resolving resolution with 4 meter horizontal spacing and 2 meter vertical spacing in a 4 km ×\times 4 km ×\times 128 m domain, 1024×1024×641024 \times1024 \times64 grid points, and simulates 30 days on a single Nvidia H100 GPU. On this grid and with 9th-order WENO advection, the model achieves around 19.2 simulated days per day (SDPD), corresponding to 0.84 wall-time-seconds per time-step and a time step size Δt=16.2\Delta t = 16.2 s computed adaptively to target a Courant-Friedrichs-Lewy (CFL) number of 0.7, after the transition to turbulence. Four snapshots of vertical vorticity normalized by ff (the Rossby number) are shown in Figure 4, illustrating the growth of kilometer-scale vortex motions amid bursts of meter-scale three-dimensional turbulence that develop along thin filaments of vertical vorticity and vertical shear. This configuration captures a competition between baroclinic instability, which acts to “restratify” or strengthen boundary layer stratification, and three-dimensional turbulent mixing driven by a forward cascade from kilometer-scale motions [77, 34]. Shear and convection associated with atmospheric storms provide an additional important source of ocean turbulent mixing interacting with kilometer-scale baroclinic instability [15, 21]; these additional processes can be simulated by adding surface fluxes to our Eady configuration.

Surface vertical vorticity in a large eddy simulation of the Eady problem

Figure 4. Surface vertical vorticity in a large eddy simulation of the Eady problem with Ri=1Ri = 1 initially, after t=4.6t = 4.6, 6, 7.7, and 20 days. The grid spacing is 4×4×24 \times4 \times2 meters in x,y,zx,y,z. Part of the script that produces this simulation is shown in listing 6.

Finally, we illustrate Oceananigans’ capabilities for realistic, three-dimensional large eddy simulations in complex geometries by simulating temperature- and salinity-stratified tidal flow past a headland, reminiscent of an extensively observed and modeled flow past Three Tree Point in Puget Sound in the Pacific Northwest of the United States [88, 135]. The bathymetry involves a sloping wedge that juts from a square-sided channel, such that

zb(x,y)=−H(1+y+∣x∣δ),(11)z_b(x,y) = -H \left(1 + \frac{y + |x|}{\delta}\right), \tag*{(11)}

where δ=L/2\delta= L/2 represents the scale of the bathymetry, LL is the half-channel width in yy (the total width is 2L2L), and H=256H = 256 m is the depth of the channel, and z=zb(x,y)z = z_b(x,y) is the height of the bottom. The flow is driven by a tidally-oscillating boundary velocity

U(t)=U2sin⁡(2πtτ2)(12)U(t) = U_2 \sin\left(\frac{2\pi t}{\tau_2}\right) \tag*{(12)}

imposed at the east and west boundaries. Here, τ2=12.421\tau_2 = 12.421 hours is the period of the semi-diurnal lunar tide, and U2=0.1U_2 = 0.1 m s−1^{-1} is the tidal velocity amplitude used in this idealized simulation. The initial temperature and salinity are

T∣t=0=12+4zH ∘C,andS∣t=0=32 g kg−1.(13)T|_{t=0} = 12 + 4\frac{z}{H}\ ^\circ\mathrm{C}, \qquad\text{and} \qquad S|_{t=0} = 32\ \mathrm{g\,kg}^{-1}. \tag*{(13)}

A portion of the script that implements this simulation is shown in listing 7.

The oscillatory and turbulent flow that results from the set up in listing 7, simulated on a single Nvidia H100 GPU, is visualized in figure 5. fig:5a shows the along-channel velocity field, revealing the formation of a turbulent wake downstream of the headland as the tide reverses. fig:5b shows temperature, illustrating how tidal flow over the sloping bathymetry mixes the initially stratified water column. fig:5c shows Ertel potential vorticity, computed using the companion package Oceanostics [24], highlighting filaments of intense vorticity generated by flow separation at the headland tip. This example demonstrates several capabilities working together: immersed boundaries for complex bathymetry, open boundary conditions for tidal forcing, WENO advection for turbulence-resolving simulation, and composition with an external diagnostics package. The simulation uses a grid-fitted immersed boundary method, in which solid boundaries are represented by masking cells that lie within the topography. Oceananigans supports both full-cell (“staircase”) and partial-cell representations of bathymetry within this framework [5, 84].

Hydrostatic model with a free surface

The HydrostaticFreeSurfaceModel solves the hydrostatic, rotating Boussinesq equations with a free surface. The hydrostatic approximation, inherent to the HydrostaticFreeSurfaceModel, means that the vertical momentum equation used by NonhydrostaticModel, z^\hat{z} (6), is replaced by a statement of hydrostatic balance,

∂zp=b,(14)\partial_z p = b, \tag*{(14)}

while the vertical velocity is obtained diagnostically from the continuity equation,

∂zw=−∇h⋅uh.(15)\partial_z w = -\nabla_{\mathrm{h}} \cdot\mathbf{u}_{\mathrm{h}}. \tag*{(15)}

(Above we assumed that gravity points down, i.e., g^=−z^\hat{g} = -\hat{z} in (6)). As a result, time-stepping the HydrostaticFreeSurfaceModel does not require solving a three-dimensional Poisson equation for pressure. Moreover, the HydrostaticFreeSurfaceModel introduces a free surface displacement η\eta, which obeys the linearized equation

∂tη=w∣z=0.(16)\partial_t \eta= w|_{z=0}. \tag*{(16)}
H, L = 256meters, 1024meters
δ = L / 2
x, y, z = (-3L, 3L), (-L, L), (-H, 0)
Nz = 64

grid = RectilinearGrid(GPU(); size=(6Nz, 2Nz, Nz), halo=(6, 6, 6),
                       x, y, z, topology=(Bounded, Bounded, Bounded))

wedge(x, y) = -H *(1 + (y + abs(x)) / δ)
grid = ImmersedBoundaryGrid(grid, GridFittedBottom(wedge))

τ₂ = 12.421hours
U₂ = 0.1 # m/s

@inline U(x, y, z, t, p) = p.U₂ * sin(2π * t / p.τ₂)
@inline U(y, z, t, p) = U(zero(y), y, z, t, p)

open_bc = OpenBoundaryCondition(U; parameters=(; U₂, τ₂),
                                scheme = PerturbationAdvection(inflow_timescale = 2
                                minutes, outflow_timescale = 2minutes))

u_bcs = FieldBoundaryConditions(east = open_bc, west = open_bc)

@inline ambient_temperature(x, z, t, H) = 12 + 4z/H
@inline ambient_temperature(x, y, z, t, H) = ambient_temperature(x, z, t, H)
ambient_temperature_bc = ValueBoundaryCondition(ambient_temperature; parameters = H)
T_bcs = FieldBoundaryConditions(east = ambient_temperature_bc, west =
                                ambient_temperature_bc)

ambient_salinity_bc = ValueBoundaryCondition(32)
S_bcs = FieldBoundaryConditions(east = ambient_salinity_bc, west = ambient_salinity_bc)

model = NonhydrostaticModel(; grid, tracers = (:T, :S),
                            buoyancy = SeawaterBuoyancy(equation_of_state=
                            TEOS10EquationOfState()),
                            advection = WENO(order=9), coriolis = FPlane(latitude=47.5),
                            boundary_conditions = (; T=T_bcs, u = u_bcs, S = S_bcs))

Ti(x, y, z) = ambient_temperature(x, y, z, 0, H)
set!(model, T=Ti, S=32)

Listing 7: Large eddy simulation of flow past a headland reminiscent of Three Tree Point in the Pacific Northwest (see Pawlak et al., [88]; Warner & MacCready, [135]).

Along-channel velocity, temperature, and Ertel potential vorticity in a tidally-forced flow

Figure 5. (a) Along-channel velocity, (b) temperature, and (c) Ertel potential vorticity in a tidally-forced flow past an idealized headland with open boundaries. The tidal flow occurs in the xx-direction and the snapshot depicts the flow just after the tide has turned to the negative-xx direction.

Equation (16) replaces the rigid-lid impenetrability condition w∣z=0=0w|_{z=0}=0 typically applied at top boundaries in the NonhydrostaticModel. The numerical algorithms and computational performance of the HydrostaticFreeSurfaceModel are described in more detail by Silvestri et al. [116].

In the HydrostaticFreeSurfaceModel, the horizontal velocity uh=ux^+vy^\mathbf{u}_h=u\hat{\mathbf{x}}+v\hat{\mathbf{y}} evolves according to

∂tuh=−∇hp−g∇hη−(u⋅∇)uh−f×u−∇⋅τ+Fuh(17)\partial_t\mathbf{u}_h=-\nabla_h p-g\nabla_h\eta-(\mathbf{u}\cdot\nabla)\mathbf{u}_h-\mathbf{f}\times\mathbf{u}-\nabla\cdot\boldsymbol{\tau}+\mathbf{F}_{u_h} \tag*{(17)}

where pp is the hydrostatic kinematic pressure anomaly, η\eta is the free surface displacement, u=ux^+vy^+wz^\mathbf{u}=u\hat{\mathbf{x}}+v\hat{\mathbf{y}}+w\hat{\mathbf{z}} is the three-dimensional velocity, f\mathbf{f} is the background vorticity associated with a rotating frame of reference, τ\boldsymbol{\tau} is the stress associated with subgrid turbulent horizontal momentum transport, and Fuh\mathbf{F}_{u_h} is a body force. As described in Appendix § 6 and by Silvestri, Campin, et al. (2026), equation (17) may be integrated with a predictor-corrector split-explicit method that uses short substeps for the “barotropic mode” (the fast component of the free surface displacement η\eta and vertically-integrated horizontal momentum) and a much slower step for the three-dimensional part of the solution. Implicit and fully explicit methods for the combined time-integration of the momentum (17) and the free surface (A16) are also available.

Momentum advection in HydrostaticFreeSurfaceModel can be formulated in three ways: (i) in the same flux form used by NonhydrostaticModel,

(u⋅∇)uh=∇⋅(uuh)(18)(\mathbf{u}\cdot\nabla)\mathbf{u}_h=\nabla\cdot(\mathbf{u}\mathbf{u}_h) \tag*{(18)}

(ii) in a standard “vector invariant” form that facilitates simulations on curvilinear grids (Adcroft et al. [4]),

(u⋅∇)uh=ζz^×uh+w∂zuh+∇h(12∣uh∣2)(19)(\mathbf{u}\cdot\nabla)\mathbf{u}_h=\zeta\hat{\mathbf{z}}\times\mathbf{u}_h+w\partial_z\mathbf{u}_h+\nabla_h\left(\frac{1}{2}|\mathbf{u}_h|^2\right) \tag*{(19)}

and (iii) a modified vector invariant form,

(u⋅∇)uh=ζz^×uh−uh∂zw+∂z(wuh)+∇h(12∣uh∣2)(20)(\mathbf{u}\cdot\nabla)\mathbf{u}_h=\zeta\hat{\mathbf{z}}\times\mathbf{u}_h-\mathbf{u}_h\partial_z w+\partial_z(w\mathbf{u}_h)+\nabla_h\left(\frac{1}{2}|\mathbf{u}_h|^2\right) \tag*{(20)}

that is used for a WENO-based vector invariant scheme (Silvestri et al., [115]). The WENO-based vector invariant scheme is particularly well-suited for simulating geophysical turbulence subject to strong Coriolis forces, because it selectively dissipates enstrophy and the variance of divergence and does not require an explicit turbulence closure for numerical stability (Silvestri et al., [115]). (Zhang et al. [143] show how the WENO vector invariant methodology may be translated to apply to potential vorticity in a layered formulation, and thereby dissipate potential vorticity.) This contrasts the WENO vector invariant scheme with second-order flux form or standard vector invariant schemes, which produce oscillatory errors and must be paired with an explicit turbulence closure, and with a flux form WENO scheme, which targets the dissipation of kinetic energy rather than enstrophy.

Tracer evolution in the HydrostaticFreeSurfaceModel is governed by the conservation law

∂tc=−∇⋅[(u+up)c]+c∇⋅up−∇⋅Jc+Sc+Fc(21)\partial_t c=-\nabla\cdot\left[(\mathbf{u}+\mathbf{u}_p)c\right]+c\nabla\cdot\mathbf{u}_p-\nabla\cdot\mathbf{J}_c+S_c+F_c \tag*{(21)}

which is identical to the NonhydrostaticModel tracer equation (9), except that background fields are not supported. HydrostaticFreeSurfaceModel also supports prescribing the velocity field u\mathbf{u} in (21), enabling cheap tracer advection and Lagrangian particle trajectory simulations.

Listing 8 implements a two-dimensional simulation of tidally-forced stratified flow over a series of randomly-positioned Gaussian seamounts, using flux form WENO advection

using Oceananigans, Oceananigans.Units

grid = RectilinearGrid(size = (2000, 200), halo = (4, 4),
                       x = (-1000kilometers, 1000kilometers),
                       z = (-2kilometers, 0),
                       topology = (Periodic, Flat, Bounded))

h₀ = 100                  # typical mountain height (m)
δ = 20kilometers          # mountain width (m)
seamounts = 42
W = grid.Lx - 4δ
x₀ = W .* (rand(seamounts) .- 1/2) # mountains' positions ∈ [-Lx/2+2δ, Lx/2−2δ]
h = h₀ .* (1 .+ rand(seamounts))   # mountains' heights ∈ [h₀, 2h₀]

bottom(x) = -grid.Lz + sum(h[s] * exp(-(x - x₀[s])^2 / 2δ^2) for s = 1:seamounts)
grid = ImmersedBoundaryGrid(grid, GridFittedBottom(bottom))

T₂ = 12.421hours # period of M₂ tide constituent
@inline tidal_forcing(x, z, t, p) = p.U₂ * 2π / p.T₂ * sin(2π / p.T₂ * t)
u_forcing = Forcing(tidal_forcing, parameters=(; U₂=0.1, T₂=T₂))

model = HydrostaticFreeSurfaceModel(; grid, tracers=:b, buoyancy=BuoyancyTracer(),
                                    momentum_advection=WENO(), tracer_advection=WENO(),
                                    forcing=(; u=u_forcing))

bᵢ(x, z) = 1e-5 * z
set!(model, b=bᵢ)

Listing 8: Two-dimensional simulation of tidally-forced stratified flow over a superposition of randomly-positioned Gaussian seamounts. Results are shown in Figure 6.

Vertical velocity of an internal wave field excited by tidally-forced stratified flow over a superposition of randomly-positioned Gaussian seamounts

Figure 6. Vertical velocity of an internal wave field excited by tidally-forced stratified flow over a superposition of randomly-positioned Gaussian seamounts, after 16 tidal periods.

schemes to dissipate kinetic energy and tracer variance, as appropriate for a two-dimensional simulation dominated by wave motion. In this simulation, the horizontal flow associated with tidal forcing is forced upwards over the seamounts, perturbing the stable, vertically-stratified buoyancy distribution and forcing the development of internal inertia-gravity waves, sometimes called “internal tides” [38]. The vertical velocity is visualized in Figure 6, showing that the tidal signal is dominated by “mode-one”, whose structure extends smoothly from the top to the bottom of the domain.

Vertical mixing and mesoscale turbulence parameterizations

Similar to the NonhydrostaticModel, the closure tracer fluxes JcJ_c may be three-dimensionally, horizontally, or vertically Laplacian or biharmonic. In addition, a closure is provided that computes both isopycnal “symmetric” diffusive tracer fluxes [100] and skew diffusive tracer fluxes [39, 44], which are typically used to parameterize stirring and vertical form stress, respectively, associated with unresolved mesoscale turbulence (e.g. [141]). The symmetric contribution (often called “Redi”) and skew contribution (often called “GM” after Gent–McWilliams) to the tracer flux may be tapered in the presence of a large isopycnal slope. The GM flux may be formulated as tracer advection, permitting combination with the other velocity components uu and upu_p, but with the downside of invoking additional gradients of the isopycnal slope [44].

Oceananigans provides a number of closures that parameterize vertical fluxes of tracers and momentum associated with unresolved turbulence with scales from O(1)O(1)–O(100)O(100) meters. Depending on the parameterization, the evolution of auxiliary tracers like turbulent kinetic energy and the turbulent kinetic energy dissipation rate may also be simulated. Vertical mixing parameterizations are useful for hydrostatic simulations where vertical mixing is otherwise unresolved due to a coarse horizontal grid spacing (see [67, 128], and references therein). For example, in such regional and global configurations, horizontal grid spacing typically varies from O(100 m)O(100\ \mathrm{m}) to O(100 km)O(100\ \mathrm{km}).

In addition to providing a number of parameterizations useful for hydrostatic ocean modeling on coarse grids, Oceananigans also provides an interface for implementing new parameterizations. This interface, which is invoked by extending the default behavior of closure functions with custom implementations using multiple dispatch, is illustrated in listing 9 with the implementation of the Pacanowski–Philander Richardson-number-based vertical diffusivity with a convective adjustment modification. A complete reference to the closure interface is given in the developer documentation.

To illustrate vertical mixing closures in action, listing 10 implements a simulation of convectively- and wind-driven vertical mixing using both a reference large eddy simulation and in a single column model context. We compare the LES with three parameterizations: CATKE ([133]), which has one additional equation for the evolution of turbulent kinetic energy (TKE); kk-ε\varepsilon ([125]), which has two additional equations for both TKE and TKE dissipation; and our custom implementation of Pacanowski and Philander (1981). Figure 7 plots the result, showing how kk-ε\varepsilon mixes less than CATKE while the Pacanowski–Philander scheme produces smoother diffusivity profiles set by the local shear and stratification. The discrepancy in mixing rates between CATKE and kk-ε\varepsilon is likely due to differences in how the models are calibrated. While all of CATKE’s parameters are jointly calibrated to 35 large eddy simulations (LES) that include surface wave effects ([133]), kk-ε\varepsilon parameters are calibrated one-by-one by referencing laboratory experiments and observations of increasing complexity ([124]). Both CATKE and kk-ε\varepsilon may be optionally substepped, a useful performance optimization for simulations on coarse grids that can otherwise accommodate relatively long baroclinic time steps.

Results from three vertical mixing parameterizations: CATKE, k-ε, and Pacanowski–Philander

Figure 7. Results from three vertical mixing parameterizations — CATKE, kk-ϵ\epsilon, and Pacanowski–Philander — compared against a 3D nonhydrostatic large-eddy simulation (LES) reference of the same wind/buoyancy-forced boundary layer with surface waves (La = 0.3). The LES (black) is run on a 1283128^3 grid with WENO(9) advection; the parameterizations (colored) are evaluated on a 32-level vertical column with Δt=5\Delta t = 5 min. Panels: (a) buoyancy b(z)b(z), (b) zonal velocity u(z)u(z), (c) tracer eddy diffusivity κ(z)\kappa(z) (log scale). The parameterizations are implemented as described in Listings 10 and 9.

Ocean simulations on global spherical shell grids

The HydrostaticFreeSurfaceModel can be used to simulate regional or global ocean circulation on rectilinear grids, latitude-longitude grids, and the tripolar grid ([82]) to cover the entirety of Earth’s global ocean with its current continental configuration. The tripolar grid avoids the coordinate singularity at the geographic North Pole by replacing it with two singularities located over land, typically in northern Canada and Siberia. This is achieved by mapping a family of concentric ellipses obtained from a stereographic projection onto the sphere. The singularities correspond to the foci of the ellipses and can therefore be masked within land regions. As a consequence, the grid contains a northern “fold” or seam connecting the two artificial poles. Oceananigans supports two fold configurations: the seam can lie either at (Face, Face) locations (tracer cell corners) or at (Face, Center) locations (tracer cell edges). Across this fold, grid points on one side are mapped to corresponding indices on the opposite side of the seam using a zipper boundary condition following Murray [82], which pairs fields across the seam while accounting for the appropriate sign changes in vector components. A cubed sphere grid ([107, 103, 98, 4]) that covers the entire sphere and is therefore useful for aquaplanet simulations, is also implemented and currently being validated. To illustrate simulations on

using Oceananigans.TurbulenceClosures
using Oceananigans.TurbulenceClosures: AbstractScalarDiffusivity, VerticalFormulation
using Oceananigans.Operators: ℑxcaa, ℑyaca, ∂zfcf, ∂zcff
using Oceananigans.BuoyancyFormulations: ∂z_b
using KernelAbstractions: @kernel, @index

struct PacanowskiPhilanderDiffusivity{TD, FT} <: AbstractScalarDiffusivity{TD,
    VerticalFormulation, 1}
    ν₀ :: FT; ν₁ :: FT; κ₀ :: FT
    ν_conv :: FT; κ_conv :: FT
    c :: FT; n :: FT
end

function compute_closure_fields!(diffs, closure::PacanowskiPhilanderDiffusivity, model)
    launch!(architecture(model), model.grid, :xyz, _pp_diffusivities!,
            diffs, model.grid, closure, model.velocities, model.tracers, model.buoyancy)
    return nothing
end

# Vertical shear squared at (Center, Center, Face), built from the same
# staggered-grid operators used to compute velocity gradients.
@inline φ²(i, j, k, grid, φ, args...) = φ(i, j, k, grid, args...)^2
@inline shear_squaredᶜᶠ(i, j, k, grid, U) = ℑxcaa(i, j, k, grid, φ², ∂zfcf, U.u) +
                                               ℑyaca(i, j, k, grid, φ², ∂zcff, U.v)

@kernel function _pp_diffusivities!(diffs, grid, closure, velocities, tracers, buoyancy)
    i, j, k = @index(Global, NTuple)
    N² = ∂z_b(i, j, k, grid, buoyancy, tracers)
    S² = max(shear_squaredᶜᶠ(i, j, k, grid, velocities), convert(eltype(grid), 1e-10))
    Ri = max(zero(grid), N²) / S²
    den = 1 + closure.c * Ri
    νz_s = closure.ν₀ + closure.ν₁ / den^closure.n
    κz_s = closure.κ₀ + closure.ν₀ / den + closure.ν₁ / den^(closure.n + 1)
    conv = N² < 0
    diffs.νz[i, j, k] = ifelse(conv, closure.ν_conv, νz_s)
    diffs.κz[i, j, k] = ifelse(conv, closure.κ_conv, κz_s)
end

Listing 9: A partial demonstration of the implementation of the Pacanowski–Philander Richardson-number-based vertical diffusivity as a turbulence closure in Oceananigans. The implementation extends compute_closure_fields!, leveraging the new closure type, and defines a “kernel” that runs on both CPU and GPU using KernelAbstractions syntax. The implementation illustrates the pattern of using the new type to both extend methods as well as to hold parameters of the new component. The full reference implementation, including time-discretization options and show methods, is given in the Oceananigans developer documentation. The closure extends the original Pacanowski–Philander model with a convective-adjustment branch with (νconv,κconv)(\nu_{\mathrm{conv}}, \kappa_{\mathrm{conv}}) when N2<0N^2 < 0, mirroring the pattern used by the built-in RiBasedVerticalDiffusivity.

using Oceananigans
using Oceananigans.Units
using Oceananigans.StokesDrifts: UniformStokesDrift
using Oceananigans.TurbulenceClosures: CATKEVerticalDiffusivity,
    TKEDissipationVerticalDiffusivity

function boundary_layer_simulation(closure, arch=CPU(); les=false, La=0.3, λ=60,
                                   N2=1e-5, Jb=1e-7, τx=-5e-4, f=1e-4)

    u_bcs = FieldBoundaryConditions(top=FluxBoundaryCondition(τx))
    b_bcs = FieldBoundaryConditions(top=FluxBoundaryCondition(Jb))
    model_kw = (; closure, coriolis = FPlane(; f),
                buoyancy = BuoyancyTracer(), tracers=:b,
                boundary_conditions = (u=u_bcs, b=b_bcs))

    if les
        x = y = (-200, 200)
        grid = RectilinearGrid(arch, size=(128, 128, 128); x, y, z=(-200, 0))
        advection = WENO(order=9)

        # Aligned wind/wave Stokes drift: u_star = sqrt(|τx|), us_surf = u_star / La^2,
        # decay scale 1/(2k) where k = 2π/λ is the wavenumber.
        @inline ∂z_us(z, t, p) = 2 * p.k * p.us *
                                 exp(2 * p.k * z)
        stokes_drift = UniformStokesDrift(grid; ∂z_us,
                                          parameters = (k = 2π/λ, us = sqrt(abs(τx)) / La^2))

        model = NonhydrostaticModel(grid; advection, stokes_drift, model_kw...)
        set!(model, b = (x, y, z) -> N2 * z + 1e-6 * rand())
    else
        grid = RectilinearGrid(arch, size=32, z=(-200, 0), topology=(Flat, Flat, Bounded))
        model = HydrostaticFreeSurfaceModel(grid; model_kw...)
        set!(model, b = z -> N2 * z)
    end

    simulation = Simulation(model; ∆t=5minute, stop_time=2days)
    les && conjure_time_step_wizard!(simulation, cfl=0.7)

    run!(simulation)
    return simulation
end

# pp is an instance of the complete custom closure illustrated in listing 9.
function compare_vertical_mixing(pp)
    catke = boundary_layer_simulation(CATKEVerticalDiffusivity())
    kϵ = boundary_layer_simulation(TKEDissipationVerticalDiffusivity())
    pacanowski_philander = boundary_layer_simulation(pp)
    return (; catke, kϵ, pacanowski_philander)
end

Listing 10: Code for comparing three vertical mixing parameterizations (CATKE, kk-ϵ\epsilon, and Pacanowski–Philander) against a 3D nonhydrostatic LES “reference” in the evolution of an initially linearly stratified boundary layer subjected to stationary surface fluxes of buoyancy and momentum, plus a wave-aligned Stokes drift with Langmuir number La=0.3La = 0.3. Results are shown in Figure 7. The argument pp is a fully constructed Pacanowski–Philander closure with the field allocation and accessor methods described in the developer documentation; listing 9 shows its diffusivity kernel. The LES uses boundary_layer_simulation(nothing, GPU(); les=true).

arch = GPU()
Nx, Ny, Nz = size = (4 * 360, 4 * 170, 10)
halo = (7, 7, 7)
H = 3000
latitude, longitude, z = (-85, 85), (0, 360), (-H, 0)

lat_lon_grid = LatitudeLongitudeGrid(arch; size, halo, latitude, longitude, z)
rotated_grid = RotatedLatitudeLongitudeGrid(arch; size, halo, latitude,
                                            longitude, z, north_pole=(70, 55))

# TripolarGrid with "Gaussian islands" over the two north poles
underlying_tripolar_grid = TripolarGrid(arch; size, halo, z)

dϕ, dλ, λ₀, ϕ₀ = 4, 8, 70, 55
isle(λ, ϕ) = ((λ - λ₀)^2 / 2dλ^2 + (ϕ - ϕ₀)^2 / 2dϕ^2) < 1
cylindrical_isles(λ, ϕ) = H * (isle(λ, ϕ) + isle(λ - 180, ϕ) - 1)
tripolar_grid = ImmersedBoundaryGrid(underlying_tripolar_grid, GridFittedBottom(
                                      cylindrical_isles))

momentum_advection = WENOVectorInvariant(order=9)
tracer_advection = WENO(order=7)
coriolis = HydrostaticSphericalCoriolis()
buoyancy = SeawaterBuoyancy(equation_of_state=TEOS10EquationOfState())

grid = lat_lon_grid # rotated_grid, tripolar_grid
free_surface = SplitExplicitFreeSurface(grid, substeps=60)
model = HydrostaticFreeSurfaceModel(; grid, momentum_advection, tracer_advection,
                                    coriolis, free_surface, buoyancy, tracers=(:T, :S))

Tᵢ(λ, ϕ, z) = 30 * (1 - tanh((abs(ϕ) - 45) / 8)) / 2 + rand()
Sᵢ(λ, ϕ, z) = 28 - 5e-3 * z + rand()
set!(model, T=Tᵢ, S=Sᵢ)

simulation = Simulation(model, Δt=2minutes, stop_time=180days)

Listing 11: Near-global simulations of baroclinic instability on three different grids.

global and near-global grids, we simulate baroclinic instability on three spherical grids: a latitude–longitude grid, a tripolar grid with islands placed over the north pole singularities, and a “rotated” latitude–longitude grid wherein the polar grid singularities are rotated from the geographic north pole to 55° N, 70° E, corresponding to one of the tripolar grid’s two north poles. Each simulation runs on a single Nvidia H100 GPU. A portion of the code that produces the simulation is given in listing 11 and the results are visualized in Figure 8.

Visualization of relative vorticity and temperature simulations on three different grids

Figure 8. Visualization of relative vorticity (top row) and temperature (bottom row) in simulations of baroclinic instability on three different grids: a latitude-longitude grid (left), a tripolar grid with cylindrical islands centered on the two north pole singularities (middle), and a “rotated” latitude-longitude grid with a grid pole located at 55∘55^\circN, 70∘70^\circE. Listing 11.

Realistic ocean–sea ice simulations beneath prescribed atmospheric states with NumericalEarth

The coupled modeling package NumericalEarth ([134]) implements an Oceananigans-based framework for coupled modeling including model integration, diagnostics, flux, interfacial state computations — functionality that is not provided by Oceananigans itself. In NumericalEarth, turbulent interfacial fluxes are computed using Monin–Obukhov similarity theory ([78]) following Edson et al. [36] for air-sea fluxes and Grachev et al. [43] for air-ice fluxes. NumericalEarth additionally provides utilities for downloading and interfacing with JRA55 reanalysis data ([123]), building grids based on Earth bathymetry and initializing simulations from datasets including the Estimating the Circulation and Climate of the Ocean (ECCO) state estimate ([37]), Mercator Ocean’s Global Ocean Reanalysis (GLORYS), and EN4 ([42]).

We illustrate NumericalEarth’s capabilities by implementing a coupled ocean and sea ice simulation on a tripolar grid with a nominal horizontal grid spacing of 1/6°. The ocean component is based on Oceananigans’ HydrostaticFreeSurfaceModel, using 9th-order

arch = GPU()
Nx, Ny, Nz = 2160, 1080, 60 # 1/6th degree
z = ExponentialDiscretization(Nz, -6000, 0, mutable=true) # z* vertical coord
underlying_grid = TripolarGrid(arch; z, size=(Nx, Ny, Nz), halo=(7, 7, 7))

# Smoothed ETOPO2022 bathymetry; minimum_depth removes shallow cells whose
# gravity-wave CFL would force an impractically small time step.
bottom_height = NumericalEarth.regrid_bathymetry(underlying_grid;
    dataset = NumericalEarth.ETOPO2022(),
    minimum_depth = 10meters, interpolation_passes = 5, major_basins = 3)
grid = ImmersedBoundaryGrid(underlying_grid, GridFittedBottom(bottom_height);
    active_cells_map = true)

# Build the ocean simulation. Substeps are explicit so the gravity-wave CFL
# (≈ Δx_min / sqrt(g H) ≈ 14 s on subpolar 1/6° cells) is satisfied at our
# production Δt: 100 substeps × 9 s = Δt = 15 min covers it with margin.
free_surface = SplitExplicitFreeSurface(grid; substeps = 100)
ocean = NumericalEarth.ocean_simulation(grid;
    momentum_advection = WENOVectorInvariant(order=9),
    tracer_advection = WENO(order=7),
    equation_of_state = TEOS10EquationOfState(),
    free_surface)

date = CFTime.DateTimeProlepticGregorian(1993, 1, 1)
dataset = NumericalEarth.ECCO4Monthly()
set!(ocean.model, T = NumericalEarth.Metadatum(:temperature; dataset, date),
                    S = NumericalEarth.Metadatum(:salinity; dataset, date))

sea_ice = NumericalEarth.sea_ice_simulation(grid, ocean; advection=WENO(order=5))
set!(sea_ice.model, h = NumericalEarth.Metadatum(:sea_ice_thickness; dataset, date),
                     ℵ = NumericalEarth.Metadatum(:sea_ice_concentration; dataset, date))

# Coupled with JRA55 prescribed atmosphere and prescribed radiation models that
# compute net radiative fluxes at the air-sea and air-ice interfaces.
atmosphere = NumericalEarth.JRA55PrescribedAtmosphere(arch)
radiation = NumericalEarth.JRA55PrescribedRadiation(arch)

coupled_model = NumericalEarth.OceanSeaIceModel(ocean, sea_ice; atmosphere, radiation)

Listing 12: A coupled ocean and sea-ice simulation beneath a prescribed atmospheric state from the JRA55 reanalysis (Tsujino et al. [123]) using NumericalEarth. Ocean temperature and salinity, and sea-ice thickness and concentration, are initialized using the ECCO state estimate (Forget et al. [37]). The free-surface substep count is set explicitly so that the gravity-wave CFL is satisfied at the production time-step.

WENOVectorInvariant advection for momentum and 7th order WENO advection for tracers, CATKE for vertical mixing, the TEOS10EquationOfState, and bathymetry derived from ETOPO22. The sea ice component is based on the Oceananigans-based package ClimaSeaIce, using a 0-layer “slab” thermodynamic formulation with constant heat conductivity and a viscoplastic rheology implemented with the Kimmritz et al. [59] solver. Part of our code, which uses a prescribed atmospheric state and radiation fields derived from the JRA55 reanalysis, and initializes ocean temperature, salinity, sea ice thickness, and sea ice concentration from the ECCO state estimate, is shown in listing 12. Figure 9 shows both the ocean surface speed (black to yellow) and the sea ice speed (red to white colors) generated after 180 days of simulation time on a single Nvidia H100 GPU. On the tripolar grid with a nominal horizontal grid spacing of 1/6° (2160 × 1080 × 60 cells) with Δt=15\Delta t = 15 min after a brief spinup at Δt=5\Delta t = 5 min, the coupled ocean–sea-ice model integrates at approximately 1.9 simulated years per wall-clock day on a single H100; the 180-day integration shown here completes in approximately 6 hours of wall time. For more information about Oceananigans performance on multiple GPUs, see Silvestri et al. [116].

Ocean surface and ice speed from two perspectives in space

Figure 9. Ocean surface and ice speed (masked for ice concentrations less than 10−310^{-3}) in a coupled ocean and sea-ice simulation on a 1/61/6th degree tripolar grid [82] from two perspectives in space. The ocean and sea ice state is initialized from the ECCO state estimate [37] and forced by the JRA55-do atmospheric reanalysis [123]. Air-sea fluxes and air-ice fluxes and skin temperature are computed by fixed point iteration of bulk formula. The air-sea bulk formulation uses [36] stability functions. The air-ice bulk formulation uses the [43] and [87] stability functions for stable and unstable conditions, respectively. The sea ice model uses a 0-layer thermodynamic formulation with constant heat conductivity and a viscoplastic rheology with the solver by [59]. The simulation is spun up with a 5 minute time step for the first 60 days and then integrated for additional 120 days with a 15 minute time step, achieving 1.9 simulated years per day (SYPD) on a single Nvidia H100 GPU.

Synthesis of modeling choices for different problems

Contemplating the computation of ocean-flavored fluid dynamics, the first choice users face is between the nonhydrostatic and hydrostatic models. The NonhydrostaticModel is suited to small-scale dynamics — direct numerical simulations (DNS) and large eddy simulations (LES) of the surface boundary layer, bottom boundary layer, and similar problems — where grid spacing is typically finer than 10–30 m [75]. Due to the availability of fast pressure solvers, rectilinear grids (either directly or immersed in bathymetry) are preferred with NonhydrostaticModel, though latitude-longitude grids are also supported if needed. The HydrostaticFreeSurfaceModel targets larger-scale regional to global problems that usually require curvilinear grids (latitude-longitude or tripolar), though rectilinear grids are also supported and can offer performance advantages, especially in distributed configurations. Additional physics such as Coriolis forces, buoyancy forces, Stokes drift terms, boundary conditions, and forcing term are then selected according to the computational design.

Numerical and parameterization choices depend on the problem. Direct numerical simulations (DNS) require ScalarDiffusivity to model molecular viscosity and diffusion, and is best paired with a Centered(order=2) advection scheme that preserves kinetic energy and tracer variance. For examples of direct numerical simulation, see listings 4 and 5. For large eddy simulation, implicit LES utilizing 9th-order WENO advection scheme and no explicit closure produces simulations that minimize numerical dissipation and permits the coarsest grids (listings 6 and 7). Lower-order WENO schemes (7th or 5th) tend to dissipate more than necessary for well-resolved dynamics, while solutions generated with higher-order WENO schemes (11th and higher) may exhibit spurious oscillations due to inadequate dissipation. Bounds-preserving WENO schemes may be employed to reduce spurious extrema or out-of-bounds values for tracers, such as strictly positive biogeochemical tracers. When explicit viscosity or diffusivity diagnostics are needed, DynamicSmagorinsky can be paired with WENO or a centered scheme, though users should test the combination for their application. For resolving the logarithmic layer near the ocean bottom, even DynamicSmagorinsky may be insufficient, and SmagorinskyLilly can be useful.

For idealized hydrostatic problems such as unforced initial-value problems, WENOVectorInvariant advection is often sufficient on its own. This is used for the baroclinic instability simulations in listing 11. When surface forcing is included and vertical turbulence must be parameterized, we recommend CATKEVerticalDiffusivity with WENOVectorInvariant. This is the numerical configuration used for the 1/6th degree simulation in listing 12. For coarser hydrostatic simulations with unresolved mesoscale eddies, WENO schemes can still be used, albeit at lower order (7th or 5th), and long-running simulations (decades and more) typically additionally require the IsopycnalSkewSymmetricDiffusivity (also called the Gent-McWilliams and Redi diffusivities) and may benefit from area-scaled biharmonic viscosity.

Discussion

This paper describes the GPU-based ocean modeling software called “Oceananigans” written in the high-level Julia programming language. Oceananigans provides a productive script-based user interface and introduces a computational design facilitated by powerful operator abstractions which, coupled with the capabilities of GPUs, aims to reduce the computational cost of simulating oceanic motion from meter to planetary scales. Oceananigans thus realizes a particular design for fast, rapidly developed dynamical cores: simple C-grid WENO numerics for turbulence resolving simulations in a high-level language accelerated by GPUs.

Because Oceananigans can run the large eddy and global ocean–sea-ice simulations presented here on a single GPU, it increases access to these ocean modeling applications. On larger GPU systems, Oceananigans also enables large ensembles of global ocean simulations. For example, using 4,800 GPUs on the Perlmutter supercomputer (National Energy Research Scientific Computing Center, 2025), it is currently possible to complete a 100-member ensemble of century-long global ocean simulations with a horizontal grid spacing of 10 kilometers in 10 days of wall time — thereby resolving mesoscale turbulent mixing, a prominent bias in ocean models and a fundamental process missing from most climate simulations today. These new capabilities address uncertainty in ocean heat and carbon uptake in climate projections.

Oceananigans’ programmable interface has fostered a growing ecosystem of companion packages, many developed independently outside the core team. OceanBioME [122] provides biogeochemistry models for ecosystem dynamics and carbon cycle applications, compatible with both hydrostatic and nonhydrostatic simulations. Oceanostics [24] implements complex diagnostics in Oceananigans syntax for online and offline analysis of simulations. Oceananigans.LagrangianFilter [10] performs online and offline Lagrangian filtering of simulation output to separate slow-evolving flow from high-frequency wave motions. Beyond ocean modeling, Breeze [132] provides a limited-area atmosphere model for LES-to-mesoscale simulations, and Terrarium implements differentiable, GPU-accelerated land and ecosystem models, both built on Oceananigans’ grids, solvers, and advection schemes. NumericalEarth supports coupled atmosphere–ocean simulations with Oceananigans and SpeedyWeather [62]. That these packages were developed by independent researchers — and at early stages of Oceananigans development — is evidence that Julia’s accessibility lowers the barrier to extending and building on ocean modeling software.

Initial work is also under way to couple Oceananigans-based ocean models with prognostic atmosphere models, including the Climate Modeling Alliance atmosphere dynamical core {(citep}(Yatunin et al., 2026)}, SpeedyWeather {(citep}(Klöwer et al., 2024)}, and the Oceananigans-based atmosphere model Breeze {(citep}(Wagner, Constantinou, et al., 2026)}. The DJ4Earth project has used Enzyme {(citep}(Moses et al., 2021)} and Reactant to develop a differentiable adjoint for Oceananigans {(citep}(Moses et al., 2026)}, enabling auto-differentiation to compute the gradients of cost functions that invoke Oceananigans simulations for gradient-based parameter estimation and data assimilation.

Additional directions for future work include:

  • Implementing, developing, and calibrating energy-based parameterizations for mesoscale turbulence (e.g. {(citep}Mak et al., 2018; Jansen et al., 2019)});

  • Implementing, developing, and calibrating more vertical mixing parameterizations including for Langmuir turbulence (for example, building upon {(citep}Reichl & Li, 2019; Harcourt, 2015; Legay et al., 2025; Wagner & Constantinou, 2025)});

  • Characterization of numerical mixing and dissipation in simulations with WENO-based advection schemes, especially for long running climate simulations, and the development of methods for diagnosing implicit kinetic energy and tracer variance dissipation;

  • Higher-order discretizations of viscous and molecular tracer diffusion to improve direct numerical simulation;

  • Further development of ClimaSeaIce, including a multi-layer, multi-category thermodynamic component and elasto-brittle rheology {(citep}(Dansereau et al., 2016)};

  • The development and calibration of parameterizations for the air-sea interface state, air-ice interface state, and coupling (e.g. {(citep}Pelletier et al., 2021)});

  • Further performance optimization especially in multi-GPU configurations, using bespoke GPU features and mixed precision to further optimize GPU performance;

  • Development of new numerical methods for representing bathymetry, such as cut cells (e.g. {(citep}Adcroft et al., 1997; Shaw & Weller, 2016; Yamazaki et al., 2016)});

  • Relaxation of the thin shell approximation for spherical grids to enable deep oceans, such as those encountered on icy moons {(citep}(Kang et al., 2022)};

  • Generation of coupled ocean and sea ice simulation data for training AI ocean emulators (e.g. {(citep}Dheeshjith et al., 2025)});

  • Development and coupling with a GPU-native surface model and coupling with PiCLES, a recently-developed Particle-In-Cell surface wave model {(citep}(Hell et al., 2024)};

  • Development of global coupled atmosphere-ocean configurations with biogeochemistry;

  • Development of open boundary conditions for HydrostaticFreeSurfaceModel with SplitExplicitFreeSurface for regional-scale configurations;

  • Coupling with hybrid physics/ML atmosphere models {(citep}(Kochkov et al., 2024)} and fully-ML atmosphere models like ACE {(citep}Watt-Meyer et al., 2023, 2024)} and GraphCast {(citep}(Lam et al., 2023)}.

Driven by the recent explosion in computationally-intensive applications of machine learning (ML), computational science and technology is advancing at an accelerating rate. To keep pace — to continue to use the world’s fastest computers, to maintain scientific productivity commensurate with other fields, and to enable the next generation of theory and ML-based parameterizations — the development of ocean modeling software must also accelerate. Oceananigans represents a step towards the faster development of ocean modeling software. But substantial and sustained growth in the developer community is still required for Oceananigans to reach its potential.

Time stepping and time discretization

In this section we describe time stepping methods and time discretization options for the NonhydrostaticModel and the HydrostaticFreeSurfaceModel.

Time discretization for tracers

Tracers are stepped forward with similar schemes in the NonhydrostaticModel and the HydrostaticFreeSurfaceModel, each of which includes optional implicit treatment of vertical diffusion terms. Equation (9) is abstracted into two components,

∂tc=Gc+∂z(κz∂zc),(22)\partial_t c = G_c + \partial_z\left(\kappa_z \partial_z c\right), \tag*{(22)}

where, if specified, κz\kappa_z is the vertical diffusivity of cc to be treated with a VerticallyImplicitTimeDiscretization, and GcG_c is the remaining component of the tracer tendency from equation (9). (Vertical diffusion treated with an ExplicitTimeDiscretization is also absorbed into GcG_c.) We apply a semi-implicit time discretization of vertical diffusion to approximate the integral of (A1) from tmt^m to tm+1t^{m+1},

(1−Δt ∂zκz∂z)cm+1=cm+∫tmtm+1Gc dt,(23)\left(1-\Delta t\,\partial_z\kappa_z\partial_z\right)c^{m+1}=c^m+\int_{t^m}^{t^{m+1}}G_c\,\mathrm{d}t, \tag*{(23)}

where Δt:=tm+1−tm\Delta t \mathrel{\mathop:}=t^{m+1}-t^m. The tendency integral ∫tmtm+1Gc dt\int_{t^m}^{t^{m+1}}G_c\,\mathrm{d}t is evaluated either using a “quasi”-second order Adams-Bashforth (QAB2, which is formally first-order) scheme [75, 111], or a low-storage third-order Runge-Kutta (RK3) scheme [138, 114]. For QAB2, the integral in (A2) spans the entire time-step and takes the form

1Δt∫tmtm+1Gc dt≈(32+χ)Gcm−(12+χ)Gcm−1,(24)\frac{1}{\Delta t}\int_{t^m}^{t^{m+1}}G_c\,\mathrm{d}t\approx\left(\frac{3}{2}+\chi\right)G_c^m-\left(\frac{1}{2}+\chi\right)G_c^{m-1}, \tag*{(24)}

where χ\chi is a small parameter, chosen by default to be χ=0.1\chi=0.1. QAB2 requires one tendency evaluation per time-step. For RK3, the indices m=(1,2,3)m=(1,2,3) correspond to substages, and the integral in (A2) takes the form

1Δt∫tmtm+1Gc dt≈γmGcm−ζmGcm−1,(25)\frac{1}{\Delta t}\int_{t^m}^{t^{m+1}}G_c\,\mathrm{d}t\approx\gamma^mG_c^m-\zeta^mG_c^{m-1}, \tag*{(25)}

where γ=(8/15,5/12,3/4)\gamma=(8/15,5/12,3/4) and ζ=(0,17/60,5/12)\zeta=(0,17/60,5/12) for m=(1,2,3)m=(1,2,3) respectively. RK3 requires three evaluations of the tendency GcG_c per time-step. RK3 is self-starting because ζ1=0\zeta^1=0, while QAB2 must be started with a forward-backwards Euler step (the choice χ=−1/2\chi=-1/2 in (A3)). Equation (23) is solved with a tridiagonal algorithm following a second-order spatial discretization of ∂zκz∂zcm+1\partial_z\kappa_z\partial_zc^{m+1} — either once per time-step for QAB2, or once at each of RK3’s three stages.

VerticallyImplicitTimeDiscretization permits longer time-steps when using fine vertical spacing. Listing 13 illustrates the differences between vertically-implicit and explicit time discretization using one-dimensional tracer diffusion with a top-hat diffusivity profile. The results are shown in figure A1.

The pressure correction method for momentum in NonhydrostaticModel

The NonhydrostaticModel uses a pressure correction method for the momentum equation (6) that ensures ∇⋅u=0\nabla\cdot\boldsymbol{u}=0. We rewrite (6) as

∂tu=−∇p+bz^+Ru+∂z(νz∂zu),(26)\partial_t\boldsymbol{u}=-\nabla p+b\hat{\boldsymbol{z}}+\boldsymbol{R}_u+\partial_z\left(\nu_z\partial_z\boldsymbol{u}\right), \tag*{(26)}

where, if specified, νz\nu_z is the vertical component of the viscosity that will be treated with a vertically-implicit time discretization, ∇p\nabla p is the total pressure gradient, and Ru\boldsymbol{R}_u contains the

using Oceananigans
using Oceananigans.TurbulenceClosures:
    ExplicitTimeDiscretization, VerticallyImplicitTimeDiscretization

function tracer_diffusion(∆t, time_discretization)
    grid = RectilinearGrid(size=20, z=(-2, 2), topology=(Flat, Flat, Bounded))
    κ(z, t) = abs(z) < 1
    closure = VerticalScalarDiffusivity(time_discretization; κ)
    model = HydrostaticFreeSurfaceModel(grid; closure, tracers=:c,
                                        velocities=PrescribedVelocityFields())
    set!(model, c=z -> z)
    simulation = Simulation(model; ∆t, stop_time=0.5)
    run!(simulation)
    return model.tracers.c
end

explicit_large = tracer_diffusion(0.5, ExplicitTimeDiscretization())
explicit_small = tracer_diffusion(1e-4, ExplicitTimeDiscretization())
implicit_large = tracer_diffusion(0.5, VerticallyImplicitTimeDiscretization())

Listing 13: Diffusion of an initially linear tracer profile with a top-hat diffusivity. The three simulations shown in Figure A1 differ only in their time step and time discretization.

Simulations of tracer diffusion by a top hat diffusivity profile

Figure A1. Simulations of tracer diffusion by a top hat diffusivity profile using various choices of time-discretization and time-step size. With a long time-step of Δt=0.5\Delta t = 0.5, ExplicitTimeDiscretization is unstable while VerticallyImplicitTimeDiscretization is stable. Yet the vertically-implicit solution depends on the long time-step Δt=0.5\Delta t = 0.5, as revealed by comparison with ExplicitTimeDiscretization using Δt=10−4\Delta t = 10^{-4}.

remaining momentum tendency terms, excluding pressure, buoyancy, and vertically implicit diffusion. We decompose pp into a “hydrostatic anomaly” phyp_{hy} tied to the density anomaly ρ′\rho', and a nonhydrostatic component pnhp_{nh}, such that

p=pnh+phy,where∂zphy=defb.(27)p = p_{nh} + p_{hy}, \qquad\text{where} \qquad\partial_z p_{hy} \mathrel{\mathop{=}^{\mathrm{def}}} b . \tag*{(27)}

Hydrostatic balance cancels the explicit buoyancy term, so that −∇p+bz^=−∇pnh−∇hphy-\nabla p+b\hat{z}=-\nabla p_{nh}-\nabla_h p_{hy}. We define Gu=defRu−∇hphyG_u \mathrel{\mathop{=}^{\mathrm{def}}} R_u-\nabla_h p_{hy}, retaining the horizontal hydrostatic pressure gradient in the tendency used for the predictor. Next, integrating (A5) in time from tmt^m to tm+1t^{m+1} yields

um+1=um+∫tmtm+1[Gu−∇pnh+∂z(νz∂zu)]dt.(28)\mathbf{u}^{m+1} = \mathbf{u}^{m} + \int_{t^m}^{t^{m+1}} \left[G_u-\nabla p_{nh}+\partial_z\left(\nu_z\partial_z\mathbf{u}\right)\right]\mathrm{d}t . \tag*{(28)}

Next we introduce the predictor velocity u~\tilde{\mathbf{u}}, defined such that

(1−Δt ∂zνzm∂z)u~=um+∫tmtm+1Gudt,(29)\left(1-\Delta t\,\partial_z\nu_z^m\partial_z\right)\tilde{\mathbf{u}} = \mathbf{u}^{m}+\int_{t^m}^{t^{m+1}}G_u\mathrm{d}t , \tag*{(29)}

or in other words, defined as a velocity-like field that cannot feel nonhydrostatic pressure gradient ∇pnh\nabla p_{nh}. Equation (A8) uses a semi-implicit treatment of vertical momentum diffusion which is similar but slightly different to the treatment of tracer diffusion in (A2),

∫tmtm+1∂z(νz∂zu)dt≈Δt ∂z(νzm∂zu~).(30)\int_{t^m}^{t^{m+1}}\partial_z\left(\nu_z\partial_z\mathbf{u}\right)\mathrm{d}t \approx\Delta t\,\partial_z\left(\nu_z^m\partial_z\tilde{\mathbf{u}}\right) . \tag*{(30)}

The integral in (A8) is evaluated with the same methods used for tracers — either (A3) for QAB2 or (A4) when using RK3. With a second-order discretization of vertical momentum diffusion, the predictor velocity in (A8) may be computed with a tridiagonal solver.

Introducing a fully-implicit time discretization for pnhp_{nh},

∫tmtm+1∇pnhdt≈Δt ∇pnhm+1,(31)\int_{t^m}^{t^{m+1}}\nabla p_{nh}\mathrm{d}t \approx\Delta t\,\nabla p_{nh}^{m+1}, \tag*{(31)}

and inserting (A10) into (A8), we derive the pressure correction to the predictor velocity,

um+1−u~=−Δt ∇pnhm+1.(32)\mathbf{u}^{m+1}-\tilde{\mathbf{u}}=-\Delta t\,\nabla p_{nh}^{m+1}. \tag*{(32)}

The final ingredient needed to complete the pressure correction scheme is an equation for the nonhydrostatic pressure pnhm+1p_{nh}^{m+1}. For this we form ∇⋅\nabla\cdot(A11) and use ∇⋅um+1=0\nabla\cdot\mathbf{u}^{m+1}=0 to obtain a Poisson equation for pnhm+1p_{nh}^{m+1},

∇2pnhm+1=∇⋅u~Δt.(33)\nabla^2p_{nh}^{m+1}=\frac{\nabla\cdot\tilde{\mathbf{u}}}{\Delta t}. \tag*{(33)}

Boundary conditions for equation (A12) may be derived by evaluating n^⋅\hat{\mathbf{n}}\cdot(A7) on the boundary of the domain.

On RectilinearGrids, we solve (A12) using an eigenfunction expansion of the discrete second-order Poisson operator ∇2\nabla^2 evaluated via the fast Fourier transform (FFT) in equispaced directions [109] plus a tridiagonal solve in variably-spaced directions. With the FFT-based solver, boundary conditions on pnhm+1p_{nh}^{m+1} are accounted for by enforcing n^⋅u~=n^⋅um+1\hat{\mathbf{n}}\cdot\tilde{\mathbf{u}}=\hat{\mathbf{n}}\cdot\mathbf{u}^{m+1} on boundary cells — which is additional and separate from the definition of u~\tilde{\mathbf{u}} in (A8). This alteration of u~\tilde{\mathbf{u}} on the boundary implicitly contributes the appropriate terms that account for inhomogeneous boundary-normal pressure gradients n^⋅∇pnhm+1≠0\hat{\mathbf{n}}\cdot\nabla p_{nh}^{m+1}\ne0 to the right-hand-side of (A12) during the computation of ∇⋅u~\nabla\cdot\tilde{\mathbf{u}}.

A preconditioned conjugate gradient iteration may be used on non-rectilinear grids, including complex domains. For domains that immerse an irregular boundary within a RectilinearGrid, we have implemented an efficient, rapidly-converging preconditioner that leverages the FFT-based solver with masking applied to immersed cells. The FFT-based preconditioner for solving the Poisson equation in irregular domains will be described in a forthcoming paper.

Time discretization of the HydrostaticFreeSurfaceModel

The HydrostaticFreeSurfaceModel uses a geopotential (zz or z∗z^*) vertical coordinate and may be integrated in time using either a fully ExplicitFreeSurface, an ImplicitFreeSurface utilizing a two-dimensional elliptical solve, or a SplitExplicitFreeSurface. The SplitExplicitFreeSurface supports both a linear free surface formulation and a nonlinear free surface formulation with the z∗z^* free-surface-following vertical coordinate [3]; the latter is used for the global simulation shown in § 4.4.1 and described in detail by Silvestri et al. [116]. For brevity, we describe here only the SplitExplicitFreeSurface, which is the most generally useful method. The SplitExplicitFreeSurface substeps the depth-integrated or “barotropic” horizontal velocity Uh\mathbf{U}_{h} along with the free surface displacement η\eta using a short time step while the depth-dependent, “baroclinic” velocities, along with tracers, are relatively stationary.

The barotropic horizontal transport Uh\mathbf{U}_{h} is defined

Uh=def∫−Hηuh dz,(34)\mathbf{U}_{h} \mathrel{\mathop{=}^{\mathrm{def}}} \int_{-H}^{\eta} \mathbf{u}_{h}\,\mathrm{d}z , \tag*{(34)}

where uh=(u,v)\mathbf{u}_{h}=(u,v) is the total horizontal velocity and HH is the depth of the fluid.

Similarly integrating the horizontal momentum equations (17) from z=−Hz=-H to z=ηz=\eta yields an evolution equation for Uh\mathbf{U}_{h},

∂tUh=−g(H+η)∇hη+∫−HηGuh dz,(35)\partial_t\mathbf{U}_{h}=-g(H+\eta)\nabla_h\eta+\int_{-H}^{\eta}\mathbf{G}_{uh}\,\mathrm{d}z , \tag*{(35)}

where Guh\mathbf{G}_{uh} includes all the tendency terms that evolve “slowly” compared to the barotropic mode:

Guh=−∇hp−(u⋅∇)uh−f×u−∇⋅τ+Fh.(36)\mathbf{G}_{uh}=-\nabla_h p-(\mathbf{u}\cdot\nabla)\mathbf{u}_h-\mathbf{f}\times\mathbf{u}-\nabla\cdot\boldsymbol{\tau}+\mathbf{F}_h . \tag*{(36)}

Integrating continuity over the water column, using an impermeable bottom and the surface kinematic condition ∂tη+uh∣z=η⋅∇hη=w∣z=η\partial_t\eta+\mathbf{u}_h|_{z=\eta}\cdot\nabla_h\eta=w|_{z=\eta}, gives

∂tη=−∇h⋅Uh.(37)\partial_t\eta=-\nabla_h\cdot\mathbf{U}_h . \tag*{(37)}

The pair of equations (A14) and (A16) characterize the evolution of the barotropic mode, which involves faster time-scales than the baroclinic mode evolution described by equations (17). To resolve both modes, we use a split-explicit algorithm where the barotropic mode is advanced in time using a smaller time-step than the one used for three-dimensional baroclinic variables. The three-dimensional evolution is discretized in time using the QAB2 scheme in equation (A3) or using a Runge-Kutta scheme; here we describe the coupling of barotropic and baroclinic mode using the QAB2 scheme, for details on the coupling with the RK3 scheme see Silvestri, Campin, et al. (2026). In particular, a predictor three-dimensional velocity is evolved without accounting for the barotropic mode evolution, using the QAB2 scheme in equation (A3). We denote this “predictor” velocity, again, with a tilde as done in section A2.

(1−Δt ∂zνz∂z)u~h−uhm≈∫tmtm+1Guh dt.(38)(1-\Delta t\,\partial_z\nu_z\partial_z)\widetilde{\mathbf{u}}_h-\mathbf{u}_h^m\approx\int_{t_m}^{t^{m+1}}\mathbf{G}_{uh}\,\mathrm{d}t . \tag*{(38)}

We then compute the barotropic mode evolution by sub-stepping MM times the barotropic equations using a forward-backward time-stepping scheme and a time-step Δτ=Δt/N\Delta\tau=\Delta t/N,

ηn+1−ηn=−Δτ ∇h⋅Uhn,(39)\eta^{n+1}-\eta^n=-\Delta\tau\,\nabla_h\cdot\mathbf{U}_h^n , \tag*{(39)}
Uhn+1−Uhn=−Δτ[g(H+η)∇hηn+1−1Δt∫−Hη∫tmtm+1Guh dt dz].(40)\mathbf{U}_h^{n+1}-\mathbf{U}_h^n=-\Delta\tau\left[g(H+\eta)\nabla_h\eta^{n+1}-\frac{1}{\Delta t}\int_{-H}^{\eta}\int_{t_m}^{t^{m+1}}\mathbf{G}_{uh}\,\mathrm{d}t\,\mathrm{d}z\right] . \tag*{(40)}

The slow tendency terms are frozen in time during substepping. The barotropic quantities are averaged within the sub-stepping with

U‾h=∑n=1ManUhnandη‾=∑n=1Manηn,(41)\overline{\mathbf{U}}_h=\sum_{n=1}^{M}a_n\mathbf{U}_h^n \quad\text{and}\quad\overline{\eta}=\sum_{n=1}^{M}a_n\eta^n , \tag*{(41)}
topology = (Bounded, Bounded, Flat)
x = y = (0, 1)
c(x, y) = exp(x) * y

fine_grid = RectilinearGrid(size=(1024, 1024); x, y, topology)
c_fine = CenterField(fine_grid)
set!(c_fine, c)

medium_grid = RectilinearGrid(size=(16, 16); x, y, topology)
c_medium = CenterField(medium_grid)
regrid!(c_medium, c_fine)

coarse_grid = RectilinearGrid(size=(4, 4); x, y, topology)
c_coarse = CenterField(coarse_grid)
regrid!(c_coarse, c_medium)

Listing 14: Finite volume discretization of exye^{xy} on three grids over the unit square. The fields are visualized in Figure B1. The meaning of the “Center” in “CenterField” is discussed below.

Finite volume discretization of $e^{xy}$ on the unit square using 1024 × 1024, 16 × 16, and 4 × 4 cells

Figure B1. Finite volume discretization of exye^{xy} on the unit square using 1024×10241024 \times1024, 16×1616 \times16, and 4×44 \times4 cells.

where MM is the number of substeps per baroclinic step and ana_n are the weights calculated from the provided averaging kernel. The default choice of averaging kernel is the minimal dispersion filters developed by Shchepetkin and McWilliams [111]. The number of substeps MM is calculated to center the averaging kernel at tm+1t^{m+1}. As a result, the barotropic subcycling overshoots the baroclinic step, i.e. M>NM > N with a maximum of M=2NM = 2N. Finally, the barotropic mode is reconciled to the baroclinic mode with a correction step

uhm+1=u~h+1H+η(U‾h−∫−Hηu~h dz)(42)u_h^{m+1} = \widetilde{u}_h + \frac{1}{H+\eta}\left(\overline{U}_h - \int_{-H}^{\eta}\widetilde{u}_h\,\mathrm{d}z\right) \tag*{(42)}

The barotropic variables are then reinitialized for evolution in the next barotropic mode evolution using the time-averaged η‾\overline{\eta} and U‾h\overline{U}_h.

Finite volume spatial discretization

Oceananigans uses a finite volume method in which fields are represented discretely by their average value over small local regions or “finite volumes” of the domain. Listing 14 discretizes c=exyc = e^{xy} on three different grids that cover the unit square. On the finest grid, each cell-averaged value cijfinec_{ij}^{\mathrm{fine}} is computed approximately using set! to evaluate exye^{xy} at the center of each finite volume, where i,ji,j denote the xx and yy indices of the finite volumes. On the medium and coarse grids, the cell averages are computed by averaging or “regridding” fields from the next finer grid. This computation produces three fields with identical integrals over the unit square. For example, integrals are computed exactly by summing discrete fields over all cells,

∫c dx dy=∑i,j1024,1024Vijfinecijfine=∑i,j16,16Vijmediumcijmedium=∑i,j4,4Vijcoarsecijcoarse(43)\int c\,\mathrm{d}x\,\mathrm{d}y = \sum_{i,j}^{1024,1024} V_{ij}^{\mathrm{fine}} c_{ij}^{\mathrm{fine}} = \sum_{i,j}^{16,16} V_{ij}^{\mathrm{medium}} c_{ij}^{\mathrm{medium}} = \sum_{i,j}^{4,4} V_{ij}^{\mathrm{coarse}} c_{ij}^{\mathrm{coarse}} \tag*{(43)}

where VijV_{ij} is the “volume” of the cell with indices i,ji,j (more accurately an “area” in this two-dimensional situation). Figure B1 visualizes the three fields.

The discrete calculus and arithmetic operations required to solve the governing equations of the NonhydrostaticModel and HydrostaticFreeSurfaceModel use the system of “staggered grids” described by Arakawa and Lamb [7]. Both models use “C-grid” staggering, where cells for tracers, pressure, and the divergence of the velocity field ∇⋅u\nabla\cdot\boldsymbol{u} are co-located, and cells for velocity components u=(u,v,w)\boldsymbol{u} = (u,v,w) are staggered by half a cell width in the xx-, yy-, and zz-directions, respectively. Listing 15 illustrates grid construction and notation for a one-dimensional staggered grid with unevenly-spaced cells. Figure B2 visualizes 2- and 3-dimensional staggered grids, indicating the location of certain variables.

Figure B2. Locations of cell centers and interfaces on a two-dimensional (a) and three-dimensional (b) staggered grid. In (a), the red and blue shaded regions highlight the volumes in the dual uu-grid and vv-grid, located at (Face, Center, Center) and (Center, Face, Center), respectively. In (b), the shaded regions highlight the facial areas used in the fluxes computations, denoted with AxA_x, AyA_y, and AzA_z.

1  using Oceananigans
2
3  grid = RectilinearGrid(topology=(Bounded, Flat, Flat), size=4, x=[0, 0.2, 0.3, 0.7, 1])
4
5  u = Field{Face, Center, Center}(grid)
6  c = Field{Center, Center, Center}(grid)
7
8  xnodes(u)      # [0.0, 0.2, 0.3, 0.7, 1.0]
9  xnodes(c)      # [0.1, 0.25, 0.5, 0.85]
10 location(∂x(c)) # (Face, Center, Center)

Listing 15: A one-dimensional staggered grid.

A system of composable operators

A convention for indexing is associated with staggered locations. Face indices are “left” of cell indices. This means that difference operators acting on fields at cells differ from those that act on face fields. To illustrate this we introduce Oceananigans-like difference operators,

1  δxᶠᶜᶜ(i, j, k, grid, c) = c[i, j, k] - c[i-1, j, k]
2  δxᶜᶜᶜ(i, j, k, grid, u) = u[i+1, j, k] - u[i, j, k]

where superscripts denote the location of the result of the operation. For example, the difference δxfcc\delta_x^{fcc} acts on fields located at ccc (meaning cell Center in the xx-, yy-, and zz-directions respectively). Complementary to the difference operators are reconstruction of “interpolation” operators,

1  ℑxᶠᶜᶜ(i, j, k, grid, c) = (c[i, j, k] + c[i-1, j, k]) / 2
2  ℑxᶜᶜᶜ(i, j, k, grid, u) = (u[i+1, j, k] + u[i, j, k]) / 2

The prefix arguments (i, j, k, grid) are more than convention: the prefix enables system for composing operators. For example, defining

1  δxᶠᶜᶜ(i, j, k, grid, f::Function, args...) =
2      f(i, j, k, grid, args...) - f(i-1, j, k, grid, args...)
4  δxᶜᶜᶜ(i, j, k, grid, f::Function, args...) =
5      f(i+1, j, k, grid, args...) - f(i, j, k, grid, args...)

leads to a concise definition of the second-difference operator:

1  δ²xᶜᶜᶜ(i, j, k, grid, f::Function, a...) = δxᶜᶜᶜ(i, j, k, grid, δxᶠᶜᶜ, f, a...)

Operator composition is used throughout Oceananigans source code to implement stencil operations.

Tracer flux divergences, advection schemes, and reconstruction

The divergence of a tracer flux J=Jxx^+Jyy^+Jzz^\boldsymbol{J} = J_x\hat{x} + J_y\hat{y} + J_z\hat{z} is discretized conservatively by the finite volume method via

∇⋅J≈1Vc[δx(AxJx)+δy(AyJy)+δz(AzJz)],(44)\nabla\cdot\boldsymbol{J} \approx\frac{1}{\mathcal{V}_c}\left[\delta_x\left(\mathcal{A}_xJ_x\right) + \delta_y\left(\mathcal{A}_yJ_y\right) + \delta_z\left(\mathcal{A}_zJ_z\right)\right], \tag*{(44)}

where δx,δy,δz\delta_x,\delta_y,\delta_z are difference operators in x,y,zx,y,z, Vc\mathcal{V}_c denotes the volume of the tracer cells, Ax\mathcal{A}_x, Ay\mathcal{A}_y, and Az\mathcal{A}_z denote the areas of the tracer cell faces with surface normals x^\hat{x}, y^\hat{y}, and z^\hat{z}, respectively. Equation (B2) indicates the location of each flux component: fluxes into tracers cell at ccc are computed at the cell faces located at fcc, cfc, and ccf.

The advective tracer flux in (9) is written in “conservative form” using incompressibility (2), and then discretized similarly to (B2) to form

u⋅∇c=∇⋅(uc)≈1Vc[δx(Axu[c]x)+δy(Ayv[c]y)+δz(Azw[c]z)],(45)\boldsymbol{u} \cdot\nabla c = \nabla\cdot(\boldsymbol{u}c) \approx\frac{1}{\mathcal{V}_c}\left[\delta_x\left(\mathcal{A}_x u [c]_x\right) + \delta_y\left(\mathcal{A}_y v [c]_y\right) + \delta_z\left(\mathcal{A}_z w [c]_z\right)\right], \tag*{(45)}

where [c]x[c]_x denotes a reconstruction of cc in the xx-direction from its native location ccc to the tracer cell interface at fcc; [c]y[c]_y and [c]z[c]_z in (B3) are defined similarly.

The advective fluxes uc\boldsymbol{u}c must be computed on interfaces between tracer cells, where the approximate value of cc must be reconstructed. (Velocity components like uu must also be reconstructed on interfaces. Within the C-grid framework, we approximate uu on tracer cell interfaces directly using the values uijku_{ijk}, which represent uu averaged over a region. encompassing the interface.) The simplest kind of reconstruction is Centered(order=2), which is equivalent to taking the average between adjacent cells,

⟨c⟩i=12(ci+ci−1) ,(46)\langle c\rangle_i = \frac{1}{2}(c_i+c_{i-1}) \, , \tag*{(46)}

where ⟨c⟩i\langle c\rangle_i denotes the centered reconstruction of cc on the interface at x=xi−1/2x=x_{i-1/2}. Also in (B4) the j,kj,k indices are implied and we have suppressed the direction xx to lighten the notation. Reconstructions stencils for Center(order=NN) are automatically generated for even NN up to Nmax⁡=12N_{\max}=12, where Nmax⁡N_{\max} is an adjustable parameter in the source code. All subsequent reconstructions are described in the xx-direction only.

Centered schemes should be used when explicit dissipation justified by a physical rationale dominates at the grid scale. In scenarios where dissipation is needed solely for artificial reasons, we find applications for UpwindBiased schemes, which use an odd-order stencil biased against the direction of flow. For example, UpwindBiased(order=1) and UpwindBiased(order=3) schemes are written

u[c]x1={u ci−1if u>0,u ciif u<0,andu[c]x3={u 16(−ci−2+5ci−1+2ci)if u>0,u 16(2ci−1+5ci−ci+1)if u<0,(47)u[c]^1_x = \begin{cases} u\,c_{i-1} & \text{if } u>0,\\ u\,c_i & \text{if } u<0, \end{cases} \qquad\text{and}\qquad u[c]^3_x = \begin{cases} u\,\frac{1}{6}(-c_{i-2}+5c_{i-1}+2c_i) & \text{if } u>0,\\ u\,\frac{1}{6}(2c_{i-1}+5c_i-c_{i+1}) & \text{if } u<0, \end{cases} \tag*{(47)}

where [c]xN[c]^N_x denotes NthN^{\mathrm{th}}-order upwind reconstruction in the xx-direction. (Note that u[c]xN=0u[c]^N_x=0 if u=0u=0.)

The compact form of equations (B5) demonstrates how upwind schemes introduce variance dissipation through numerical discretization. In particular, an UpwindBiased(order=1) reconstruction can be rewritten as a sum of a Centered(order=2) discrete advective flux and a discrete diffusive flux

u[c]x1=uci+ci−12−κ1ci−ci−1Δx ,whereκ1=∣u∣Δx2 .(48)u[c]^1_x=u\frac{c_i+c_{i-1}}{2}-\kappa_1\frac{c_i-c_{i-1}}{\Delta x}\, , \qquad\text{where}\qquad \kappa_1=\frac{|u|\Delta x}{2}\, . \tag*{(48)}

Reordering the UpwindBiased(order=3) advective flux in the same manner recovers a sum of a Centered(order=4) advective flux and a 4th-order hyperdiffusive flux, equivalent to a finite volume approximation of

uc+κ3∂3c∂x3 ,whereκ3=∣u∣Δx312 .(49)uc+\kappa_3\frac{\partial^3 c}{\partial x^3}\, , \qquad\text{where}\qquad \kappa_3=\frac{|u|\Delta x^3}{12}\, . \tag*{(49)}

UpwindBiased reconstruction can be always reordered to expose a sum of Centered reconstruction and a high-order diffusive flux with a velocity-dependent diffusivity. The diffusive operator associated with UpwindBiased(order=1) and UpwindBiased(order=3) is enough to offset the dispersive errors of the Centered component and, therefore, eliminate the artificial explicit diffusion needed for stability purposes. However, this approach does not scale to high order since the diffusive operator associated with a high order UpwindBiased scheme (5th, 7th, and so on), becomes quickly insufficient to eliminate spurious errors associated with the Centered component [41].

The inability to achieve high order and, therefore, low dissipation motivated the implementation of Weighted, Essentially Non-Oscillatory (WENO) reconstruction [112, 113]. WENO is a non-linear reconstruction scheme that combines a set of odd-order linear reconstructions obtained by stencils that are shifted by a value ss relative to the canonical UpwindBiased stencil, using a weighting scheme for each stencil that depends on the smoothness of the reconstructed field cc. Since the constituent stencils are lower-order than the WENO order, this strategy yields a scheme whose order of accuracy adapts depending on the smoothness of the reconstructed field. In smooth regions high-order is retained, while the order quickly decreases in the presence of noisy regions, decreasing the order of the associated diffusive operator. WENO proves especially useful for turbulence-resolving simulations (either at meter or planetary scales) without requiring any additional explicit artificial dissipation [96, 115].

To illustrate how WENO works we consider a fifth-order WENO scheme for u>0u>0,

{c}5=γ−1[c]3,−1+γ0[c]3,0+γ1[c]3,1(50)\{c\}^{5}=\gamma_{-1}[c]^{3,-1}+\gamma_{0}[c]^{3,0}+\gamma_{1}[c]^{3,1} \tag*{(50)}

where the notation [c]3,s[c]^{3,s} denotes an UpwindBiased stencil shifted by ss indices, such that [c]3≔[c]3,0[c]^{3}\mathrel{\mathop{\coloneqq}}[c]^{3,0}. The shifted upwind stencils [c]iN,s[c]_{i}^{N,s} evaluated at index ii are defined

[c]i3,s=16{2ci−1+5ci−ci+1for s=−1,−ci−2+5ci−1+2cifor s=0,2ci−3−7ci−2+11ci−1for s=1.(51)[c]_{i}^{3,s}=\frac{1}{6} \begin{cases} 2c_{i-1}+5c_i-c_{i+1} & \text{for }s=-1,\\ -c_{i-2}+5c_{i-1}+2c_i & \text{for }s=0,\\ 2c_{i-3}-7c_{i-2}+11c_{i-1} & \text{for }s=1. \end{cases} \tag*{(51)}

The nonnegative weights γs(c)\gamma_s(c), with s=−1,0,1s=-1,0,1 and ∑sγs=1\sum_s\gamma_s=1, are determined by a smoothness metric that produces {c}5≈[c]5\{c\}^{5}\approx[c]^{5} when cc is smooth, but limits to the more diffusive {c}5≈[c]3\{c\}^{5}\approx[c]^{3} when cc changes abruptly. Thus WENO adaptively introduces dissipation as needed based on the smoothness of cc, yielding stable simulations with a high effective resolution that require no artificial dissipation. WENO can alternatively be interpreted as adding an implicit hyperviscosity that adapts from low- to high-order depending on the local nature of the solution. To compute the weights γs(c)\gamma_s(c), we use the WENO-Z formulation [11].

The properties of Centered, UpwindBiased, and WENO reconstruction are investigated by listing 16, which simulates the advection of a top hat tracer distribution. The results are plotted in Figure B3.

Advection of a top hat tracer distribution in one dimension using various advection schemes

Figure B3. Advection of a top hat tracer distribution in one dimension using various advection schemes. Centered and UpwindBiased schemes produce oscillations at sharp gradients, while WENO adapts its reconstruction to minimize both oscillations and dissipation.

using Oceananigans

grid = RectilinearGrid(size=128, x=(-4, 8), halo=6, topology=(Periodic, Flat, Flat))
advection = WENO(order=9) # Centered(order=2), UpwindBiased(order=3)
velocities = PrescribedVelocityFields(u=1)
model = HydrostaticFreeSurfaceModel(; grid, velocities, advection, tracers=:c)

top_hat(x) = abs(x) > 1 ? 0 : 1
set!(model, c = top_hat)

simulation = Simulation(model, Δt=1/grid.Nx, stop_time=4)
run!(simulation)

Listing 16: A script that advects a top hat tracer profile in one-dimension with a constant prescribed velocity. We use halo=6 to accommodate schemes up to WENO(order=11).

Discretization of momentum advection

The discretization of momentum advection with a flux form similar to (B3) is more complex than the tracer case because both the advecting velocity and advected velocity require reconstruction. We use the method described by Ghosh and Baeder [40] and Pressel et al. [95], wherein advecting velocities are constructed with a high-order Centered scheme when the advected velocity component is reconstructed with a high-order UpwindBiased or WENO scheme. We have also developed a novel WENO-based method for discretizing momentum advection in the rotational or “vector invariant” form especially appropriate for representing mesoscale and submesoscale turbulent advection on curvilinear grids (Silvestri et al. [115]).

Parallelization

Oceananigans supports distributed computations with slab and pencil domain decomposition. The interior domain is extended using “halo” or “ghost” cells that hold the results of interprocessor boundaries. “halo” cells are updated before the computation of tendencies through asynchronous send / receive operations using the message passing interface (MPI) Julia library (Byrne et al. [20]). For a detailed description of the parallelization strategy of the HydrostaticFreeSurfaceModel; see Silvestri et al. [116]. The NonhydrostaticModel implements the same overlap of communication and computation for halo exchange before the calculation of tendencies. For the FFT-based three-dimensional pressure solver, we implement a transpose algorithm that switches between xx-local, yy-local, and zz-local configurations to compute efficiently the discrete transforms. The transpose algorithm for the distributed FFT solver is shown in Figure C1.

Table of numerical examples

Table D1 provides a list of the numerical experiments in this paper.

DescriptionCodeVisualization
2D turbulence using WENO(order=9) advectionlisting 1fig. 1
2D turbulence with moving tracer sourcelisting 2fig. 1
DNS and LES of flow around a cylinder at various ReRelisting 4fig. 2
DNS of cabbeling in freshwaterlisting 5fig. 3
LES of the Eady problem with WENO(order=9)listing 6fig. 4
Tidally-oscillating flow past Three Tree Pointlisting 7fig. 5
Internal waves generated by tidal forcing over bathymetrylisting 8fig. 6
Comparison of vertical mixing parameterizationslisting 10fig. 7
Near-global baroclinic instability on three gridslisting 11fig. 8
Realistic ocean simulation with NumericalEarthlisting 12fig. 9
Tracer diffusion with various time discretizationslisting 13fig. A1
Visualization of the finite volume discretizationlisting 14fig. B1
One-dimensional advection of a top-hat tracer profilelisting 16fig. B3

Table D1. DNS: Direct Numerical Simulation. LES: Large Eddy Simulation.

Acronyms and abbreviations

Table E1 defines acronyms and abbreviations used in the text, captions, and code examples. Software and dataset names without a standard expansion are described by their purpose.

AbbreviationMeaning
2D, 3DTwo-dimensional, three-dimensional
ACEAi2 Climate Emulator
AIArtificial intelligence
AMReXSoftware framework for block-structured adaptive mesh refinement
CATKEConvective Adjustment and Turbulent Kinetic Energy
CFLCourant–Friedrichs–Lewy
CPUCentral processing unit
CROCOCoastal and Regional Ocean Community model
CUDANVIDIA’s GPU computing platform; accessed through CUDA.jl in the examples
AbbreviationMeaning
DNSDirect numerical simulation
DSLDomain-specific language
ECCOEstimating the Circulation and Climate of the Ocean
EN4Met Office dataset of quality-controlled ocean temperature and salinity profiles and objective analyses, version 4
ETOPO1, ETOPO22Global topography and bathymetry products: the 1-arc-minute ETOPO1 product and the ETOPO 2022 product
FFTFast Fourier transform
GLORYSGlobal Ocean Reanalyses and Simulations
GMGent–McWilliams
GPUGraphics processing unit
I/OInput/output
JAXPython library for array computation and automatic differentiation
JRA55Japanese 55-year Reanalysis (also written JRA-55)
JRA55-doJRA-55-based surface dataset for driving ocean–sea-ice models
LESLarge eddy simulation
MITgcmMassachusetts Institute of Technology general circulation model
MLMachine learning
MOM6Modular Ocean Model, version 6 (used by Regional-mom6)
MPIMessage Passing Interface
PiCLESParticle-in-Cell for Efficient Swell
QAB2Quasi-second-order Adams–Bashforth time-stepping scheme
REMORARegional Model of the Ocean Refined Adaptively
RK3Third-order Runge–Kutta time-stepping scheme
ROMSRegional Ocean Modeling System
SDPDSimulated days per wall-clock day
SYPDSimulated years per wall-clock day
TEOS-10Thermodynamic Equation of Seawater 2010
TKETurbulent kinetic energy
TPUTensor processing unit
WENOWeighted essentially non-oscillatory
WENO-ZWENO reconstruction with the Z weighting formulation

Table E1. Acronyms, abbreviations, and abbreviated software and dataset names.

Open Research Section

Oceananigans is available at the GitHub repository github.com/CliMA/Oceananigans.jl and NumericalEarth is available at github.com/NumericalEarth/NumericalEarth.jl. Oceananigans documentation lives at clima.github.io/OceananigansDocumentation. All the scripts that reproduce the simulations and figures in this paper are available at the GitHub repository github.com/glwagner/OceananigansPaper. Visualizations were made using Makie.jl [29].

Acknowledgments

We thank all six reviewers, Stephen Griffies, and Florian Lemarié for their time and constructive feedback, which helped improve this manuscript. This project is supported by Schmidt Sciences, LLC and by the National Science Foundation grant AGS-1835576. N.C.C. is additionally supported by the Australian Research Council under the Center of Excellence for the Weather of the 21st Century CE230100012 and the Discovery Project DP240101274.

Conflict of interest

The authors have no conflicts of interest to disclose.

Jansen, M. F., Adcroft, A., Khani, S., & Kong, H. (2019). Toward an energetically consistent, resolution aware parameterization of ocean mesoscale eddies. Journal of Advances in Modeling Earth Systems, 11(8), 2844–2860.

Jullien, S., Le Corre, M., Raynaud, S., Gula, J., Sepulveda, A., Le Gentil, S., . . . Collin, J. (2025, March). CROCO Pytools. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.15064146 doi: 10.5281/zenodo.15064146

Kang, W., Mittal, T., Bire, S., Campin, J.-M., & Marshall, J. (2022). How does salinity shape ocean circulation and ice geometry on Enceladus and other icy satellites? Science Advances, 8(29), eabm4665. doi: 10.1126/sciadv.abm4665

Kärnä, T., Kramer, S. C., Mitchell, L., Ham, D. A., Piggott, M. D., & Baptista, A. M. (2018). Thetis coastal ocean model: discontinuous Galerkin discretization for the three-dimensional hydrostatic equations. Geoscientific Model Development, 11(11), 4359–4382.

Kimmritz, M., Danilov, S., & Losch, M. (2016). The adaptive EVP method for solving the sea ice momentum equation. Ocean Modelling, 101, 59–67.

Kiss, A. E., Hogg, A. M., Hannah, N., Boeira Dias, F., Brassington, G. B., Chamberlain, M. A., . . . others (2020). ACCESS-OM2 v1.0: a global ocean–sea ice model at three resolutions. Geoscientific Model Development, 13(2), 401–442.

Klion, H., Hetland, R., Sexton, J., Almgren, A., Grindeanu, I., Hinson, K., & Mahadevan, V. (2025). REMORA: Regional modeling of oceans refined adaptively (built on AMReX). Journal of Open Source Software, 10(110), 7958. doi: 10.21105/joss.07958

Klöwer, M., Gelbrecht, M., Hotta, D., Willmert, J., Silvestri, S., Wagner, G. L., . . . others (2024). SpeedyWeather.jl: Reinventing atmospheric general circulation models towards interactivity and extensibility. Journal of Open Source Software, 9(98), 6323. doi: 10.21105/joss.06323

Kochkov, D., Smith, J. A., Alieva, A., Wang, Q., Brenner, M. P., & Hoyer, S. (2021). Machine learning-accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21), e2101784118.

Kochkov, D., Yuval, J., Langmore, I., Norgaard, P., Smith, J., Mooers, G., . . . Hoyer, S. (2024). Neural general circulation models for weather and climate. Nature, 632, 1060–1066. doi: 10.1038/s41586-024-07744-y

Korn, P., Brüggemann, N., Jungclaus, J. H., Lorenz, S., Gutjahr, O., Haak, H., . . . others (2022). ICON-O: The ocean component of the ICON Earth system model–Global simulation characteristics and local telescoping capability. Journal of Advances in Modeling Earth Systems, 14(10), e2021MS002952.

Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., . . . others (2023). Learning skillful medium-range global weather forecasting. Science, 382(6677), 1416–1421.

Large, W. G., McWilliams, J. C., & Doney, S. C. (1994). Oceanic vertical mixing: A review and a model with a nonlocal boundary layer parameterization. Reviews of Geophysics, 32(4), 363–403. doi: 10.1029/94RG01872

Leclair, M., & Madec, G. (2011). z-Coordinate, an arbitrary Lagrangian–Eulerian coordinate separating high and low frequency motions. Ocean Modelling, 37(3-4), 139–152.

Lecoanet, D., McCourt, M., Quataert, E., Burns, K. J., Vasil, G. M., Oishi, J. S., . . . O’Leary, R. M. (2016). A validated non-linear Kelvin–Helmholtz benchmark for numerical hydrodynamics. Monthly Notices of the Royal Astronomical Society, 455(4), 4274–4288.

Legay, A., Deremble, B., & Burchard, H. (2025). Derivation and implementation of a non-local term to improve the oceanic convection representation within the kk–ϵ\epsilon parameterization. Journal of Advances in Modeling Earth Systems, 17(1), e2024MS004243. doi: 10.1029/2024MS004243

Li, Q. (2026). Large-eddy simulations of stabilizing effects induced by opposing Eulerian shear and Stokes drift shear in an idealized ocean surface boundary layer. Journal of Physical Oceanography, 56(4), 855–871. doi: 10.1175/JPO-D-25-0077.1

Lilly, D. K. (1983). Stratified turbulence and the mesoscale variability of the atmosphere.

References

  1. [1]Abkar, M., Bae, H. J., & Moin, P. (2016). Minimum-dissipation scalar transport model for large-eddy simulation of turbulent flows. Physical Review Fluids, 1(4), 041701.DOI
  2. [2]Adcroft, A., Anderson, W., Balaji, V., Blanton, C., Bushuk, M., Dufour, C. O., . . . Zhang, R. (2019). The GFDL global ocean and sea ice model OM4.0: Model description and simulation features. Journal of Advances in Modeling Earth Systems, 11(10), 3167–3211. doi: 10.1029/2019MS001726DOI
  3. [3]Adcroft, A., & Campin, J.-M. (2004). Rescaled height coordinates for accurate representation of free-surface flows in ocean circulation models. Ocean Modelling, 7(3-4), 269–284.DOI
  4. [4]Adcroft, A., Campin, J.-M., Hill, C., & Marshall, J. (2004). Implementation of an atmosphere–ocean general circulation model on the expanded spherical cube. Monthly Weather Review, 132(12), 2845–2863.DOI
  5. [5]Adcroft, A., Hill, C., & Marshall, J. (1997). Representation of topography by shaved cells in a height coordinate ocean model. Monthly Weather Review, 125(9), 2293–2315.doi.org/10.1175/1520-0493(1997)125<2293:rotbsc>2.0.co;2
  6. [7]Arakawa, A., & Lamb, V. R. (1977). Computational design of the basic dynamical processes of the UCLA general circulation model.DOI
  7. [8]Auclair, F., Benshila, R., Bordois, L., Boutet, M., Brémond, M., Caillaud, M., . . . Zribi, A. (2025). Coastal and Regional Ocean COmmunity model. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.7415055 doi: 10.5281/zenodo.7415055
  8. [9]Auclair, F., Bordois, L., Dansmann, Y., Duhaut, T., Paci, A., Uc, C., & Nguyen, C. (2018). A non-hydrostatic non-boussinesq algorithm for free-surface ocean modelling. Ocean Modelling, 132, 12–29. doi: 10.1016/j.ocemod.2018.07.011DOI
  9. [10]Baker, L. E. (2026). OceananigansLagrangianFilter.jl. Zenodo. Retrieved from https://github.com/loisba ker/OceananigansLagrangianFilter.jl doi: 10.5281/zenodo.1863285DOI
  10. [11]Balsara, D., & Shu, C. (2000). Monotonicity preserving weighted essentially non-oscillatory schemes with increasingly high order of accuracy. Journal of Computational Physics, 160(2), 405–452. doi: 10.1006/jcph.2000.6443DOI
  11. [12]Barnes, A. J., Constantinou, N. C., Gibson, A. H., Kiss, A. E., Chapman, C., Reily, J., . . . Yang, L. (2024). regional-mom6: A Python package for automatic generation of regional configurations for the Modular Ocean Model 6. Journal of Open Source Software, 9(100), 6857. Retrieved from https://doi.org/10.21105/joss.06857 doi: 10.21105/joss.06857
  12. [13]Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2017). Julia: A fresh approach to numerical computing. SIAM review, 59(1), 65–98. doi: 10.1137/141000671DOI
  13. [14]Bisits, J. I., Zika, J. D., & Sohail, T. (2025). Cabbeling as a catalyst and driver of turbulent mixing. Journal of Fluid Mechanics, 1011, A17. doi: 10.1017/jfm.2025.349DOI
  14. [15]Boccaletti, G., Ferrari, R., & Fox-Kemper, B. (2007). Mixed layer instabilities and restratification. Journal of Physical Oceanography, 37(9), 2228–2250. doi: 10.1175/JPO3010.1
  15. [16]Bou-Zeid, E., Meneveau, C., & Parlange, M. (2005). A scale-dependent Lagrangian dynamic model for large eddy simulation of complex turbulent flows. Physics of fluids, 17(2).
  16. [17]Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., … Zhang, Q. (2018). JAX: composable transformations of Python+NumPy programs. Retrieved from http://github.com/jax-ml/jax
  17. [18]Bryan, K. (1969). A numerical method for the study of the circulation of the world ocean. Journal of Computational Physics, 135(2), 154–169.
  18. [19]Burns, K. J., Vasil, G. M., Oishi, J. S., Lecoanet, D., & Brown, B. P. (2020). Dedalus: A flexible framework for numerical simulations with spectral methods. Physical Review Research, 2(2), 023068. doi: 10.1103/PhysRevResearch.2.023068DOI
  19. [20]Byrne, S., Wilcox, L. C., & Churavy, V. (2021). MPI.jl: Julia bindings for the Message Passing Interface. In Proceedings of the JuliaCon Conferences (Vol. 1, p. 68). doi: 10.21105/jcon.00068DOI
  20. [21]Callies, J., & Ferrari, R. (2018). Baroclinic instability in the presence of convection. Journal of Physical Oceanography, 48(1), 45–60. doi: 10.1175/JPO-D-17-0028.1DOI
  21. [22]Chassignet, E. P., & Xu, X. (2017). Impact of horizontal resolution (1/12 to 1/50) on Gulf Stream separation, penetration, and variability. Journal of Physical Oceanography, 47(8), 1999–2021.
  22. [23]Chassignet, E. P., & Xu, X. (2021). On the importance of high-resolution in large-scale ocean models. Advances in Atmospheric Sciences, 38, 1621–1634.
  23. [24]Chor, T., Constantinou, N. C., Bisits, J. I., Wagner, G. L., Ramadhan, A. R., Zheng, Z., & Whitley, V. (2025). Oceanostics.jl. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.8280754 doi: 10.5281/zenodo.8280754
  24. [25]Churavy, V. (2024). KernelAbstractions.jl. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.13773520 doi: 10.5281/zenodo.13773520
  25. [26]Cox, M. D. (1984). A primitive equation, 3-dimensional model of the ocean (Tech. Rep. No. 1). Princeton, NJ: NOAA Geophysical Fluid Dynamics Laboratory.
  26. [27]Craik, A. D., & Leibovich, S. (1976). A rational model for Langmuir circulations. Journal of Fluid Mechanics, 73(3), 401–426.DOI
  27. [28]Danilov, S., Sidorenko, D., Wang, Q., & Jung, T. (2017). The finite-volume sea ice–ocean model (FESOM2). Geoscientific Model Development, 10(2), 765–789.
  28. [29]Danisch, S., & Krumbiegel, J. (2021). Makie.jl: Flexible high-performance data visualization for Julia. Journal of Open Source Software, 6(65), 3349. doi: 10.21105/joss.03349DOI
  29. [31]Deck, K., Braghiere, R. K., Renchon, A. A., Sloan, J., Bozzola, G., Speer, E., … Schneider, T. (2025). ClimaLand: A land surface model for advancing climate modeling with machine learning and data-driven parameterizations. Journal of Advances in Modeling Earth Systems, 18, e2025MS005118. doi: 10.1029/2025MS005118
  30. [32]Dennis, J. M., Edwards, J., Evans, K. J., Guba, O., Lauritzen, P. H., Mirin, A. A., … Worley, P. H. (2012). Cam-se: A scalable spectral element dynamical core for the Community Atmosphere Model. The International Journal of High Performance Computing Applications, 26(1), 74–89. doi: 10.1177/1094342011428142DOI
  31. [34]Dong, J., Fox-Kemper, B., Wenegrat, J. O., Bodner, A. S., Yu, X., Belcher, S., & Dong, C. (2024). Submesoscales are a significant turbulence source in global ocean surface boundary layer. Nature Communications, 15(1), 9566.DOI
  32. [35]Eady, E. T. (1949). Long waves and cyclone waves. Tellus, 1(3), 33–52.DOI
  33. [36]Edson, J. B., Jampana, V., Weller, R. A., Bigorre, S. P., Plueddemann, A. J., Fairall, C. W., … Hersbach, H. (2014). On the exchange of momentum over the open ocean. Journal of Physical Oceanography, 44(9), 1589.DOI
  34. [37]Forget, G., Campin, J.-M., Heimbach, P., Hill, C. N., Ponte, R. M., & Wunsch, C. (2015). ECCO version 4: An integrated framework for non-linear inverse modeling and global ocean state estimation. Geoscientific Model Development, 8(10), 3071–3104.
  35. [38]Garrett, C., & Kunze, E. (2007). Internal tide generation in the deep ocean. Annu. Rev. Fluid Mech., 39(1), 57–87.DOI
  36. [39]Gent, P. R., & Mcwilliams, J. C. (1990). Isopycnal mixing in ocean circulation models. Journal of Physical Oceanography, 20(1), 150–155. doi: 10.1175/1520-0485(1990)020<0150:IMIOCM>2.0.CO;2doi.org/10.1175/1520-0485(1990)020<0150:imiocm>2.0.co;2
  37. [40]Ghosh, D., & Baeder, J. D. (2012). High-order accurate incompressible Navier–Stokes algorithm for vortex-ring interactions with solid wall. AIAA journal, 50(11), 2408–2422.DOI
  38. [41]Godunov, S. K. (1959). A difference scheme for numerical solution of discontinuous solution of hydrodynamic equations. Matematicheskii Sbornik, 47, 271–306. (Translated by US Joint Publications Research Service, JPRS 7226, 1969)
  39. [42]Good, S. A., Martin, M. J., & Rayner, N. A. (2013). EN4: Quality controlled ocean temperature and salinity profiles and monthly objective analyses with uncertainty estimates. Journal of Geophysical Research: Oceans, 118(12), 6704–6716.DOI
  40. [43]Grachev, A. A., Andreas, E. L., Fairall, C. W., Guest, P. S., & Persson, P. O. G. (2007). SHEBA flux–profile relationships in the stable atmospheric boundary layer. Boundary-layer meteorology, 124, 315–333.DOI
  41. [44]Griffies, S. M. (1998). The Gent–McWilliams skew flux. Journal of Physical Oceanography, 28(5), 831–841. doi: 10.1175/1520-0485(1998)028<0831:TGMSF>2.0.CO;2doi.org/10.1175/1520-0485(1998)028<0831:tgmsf>2.0.co;2
  42. [45]Griffies, S. M., Adcroft, A., Danabasoglu, G., et al. (2025). The GFDL-CM4X climate model hierarchy, part I: Model description and thermal properties. Journal of Advances in Modeling Earth Systems, 17. doi: 10.1029/2024MS004861DOI
  43. [46]Griffies, S. M., Adcroft, A., & Hallberg, R. W. (2020). A primer on the vertical lagrangian-remap method in ocean models based on finite volume generalized vertical coordinates. Journal of Advances in Modeling Earth Systems, 12(10), e2019MS001954.DOI
  44. [47]Griffies, S. M., Stouffer, R. J., Adcroft, A. J., Bryan, K., Dixon, K. W., Hallberg, R., & Rosati, A. (2015). A historical introduction to MOM. https://www.gfdl.noaa.gov/wp-content/uploads/files/model_development/ocean/mom_history_v15.09.05.pdf.
  45. [48]Häfner, D., Nuterman, R., & Jochum, M. (2021). Fast, cheap, and turbulent—global ocean modeling with GPU acceleration in Python. Journal of Advances in Modeling Earth Systems, 13(12), e2021MS002717.DOI
  46. [49]Halliwell, G. R. (2004). Evaluation of vertical coordinate and vertical mixing algorithms in the HYbrid-Coordinate Ocean Model (HYCOM). Ocean Modelling, 7(3-4), 285–322.DOI
  47. [51]Harlow, F. H., Welch, J. E., et al. (1965). Numerical calculation of time-dependent viscous incompressible flow of fluid with free surface. Physics of fluids, 8(12), 2182.DOI
  48. [53]Huang, N. E. (1979). On surface drift currents in the ocean. Journal of Fluid Mechanics, 91(1), 191–208.DOI
  49. [54]Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3), 90–95. doi: 10.1109/MCSE.2007.55DOI
  50. [56]Jullien, S., Le Corre, M., Raynaud, S., Gula, J., Sepulveda, A., Le Gentil, S., … Collin, J. (2025, March). CROCO Pytols. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.15064146 doi: 10.5281/zenodo.15064146
  51. [58]Kärnä, T., Kramer, S. C., Mitchell, L., Ham, D. A., Piggot, M. D., & Baptista, A. M. (2018). Thetis coastal ocean model: discontinuous Galerkin discretization for the three-dimensional hydrostatic equations. Geoscientific Model Development, 11(11), 4359–4382.DOI
  52. [59]Kimmritz, M., Danilov, S., & Losch, M. (2016). The adaptive EVP method for solving the sea ice momentum equation. Ocean Modelling, 101, 59–67.DOI
  53. [60]Kiss, A. E., Hogg, A. M., Hannah, N., Boeira Dias, F., Brassington, G. B., Chamberlain, M. A., … others (2020). ACCESS-OM2 v1.0: a global ocean–sea ice model at three resolutions. Geoscientific Model Development, 13(2), 401–442.DOI
  54. [61]Klion, H., Hetland, R., Sexton, J., Almgren, A., Grindeanu, I., Hinson, K., & Mahadevan, V. (2025). REMORA: Regional modeling of oceans refined adaptively (built on AMReX). Journal of Open Source Software, 10(110), 7958. doi: 10.21105/joss.07958DOI
  55. [62]Klöwer, M., Gelbrecht, M., Hotta, D., Willmert, J., Silvestri, S., Wagner, G. L., … others (2024). SpeedyWeather.jl: Reinventing atmospheric general circulation models towards interactivity and extensibility. Journal of Open Source Software, 9(98), 6323. doi: 10.21105/joss.06323
  56. [63]Kochkov, D., Smith, J. A., Alieva, A., Wang, Q., Brenner, M. P., & Hoyer, S. (2021). Machine learning-accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21), e2101784118.DOI
  57. [64]Kochkov, D., Yuval, J., Langmore, I., Norgaard, P., Smith, J., Mooers, G., … Hoyer, S. (2024). Neural general circulation models for weather and climate. Nature, 632, 1060–1066. doi: 10.1038/s41586-024-07744-yDOI
  58. [65]Korn, P., Brüggemann, N., Jungclaus, J. H., Lorenz, S., Gutjahr, O., Haak, H., … others (2022). ICON-O: The ocean component of the ICON Earth system model–Global simulation characteristics and local telescoping capability. Journal of Advances in Modeling Earth Systems, 14(10), e2021MS002952.DOI
  59. [67]Large, W. G., McWilliams, J. C., & Doney, S. C. (1994). Oceanic vertical mixing: A review and a model with a nonlocal boundary layer parameterization. Reviews of Geophysics, 32(4), 363–403. doi: 10.1029/94RG01872DOI
  60. [68]Leclair, M., & Madec, G. (2011). z-Coordinate, an arbitrary Lagrangian–Eulerian coordinate separating high and low frequency motions. Ocean Modelling, 37(3-4), 139–152.DOI
  61. [69]Lecoanet, D., McCourt, M., Quataert, E., Burns, K. J., Vasil, G. M., Oishi, J. S., … O’Leary, R. M. (2016). A validated non-linear Kelvin–Helmholtz benchmark for numerical hydrodynamics. Monthly Notices of the Royal Astronomical Society, 455(4), 4274–4288.DOI
  62. [71]Li, Q. (2026). Large-eddy simulations of stabilizing effects induced by opposing Eulerian shear and Stokes drift shear in an idealized ocean surface boundary layer. Journal of Physical Oceanography, 56(4), 855–871. doi: 10.1175/JPO-D-25-0077.1DOI
  63. [72]Lilly, D. K. (1983). Stratified turbulence and the mesoscale variability of the atmosphere. Journal of the Atmospheric Sciences, 40(3), 749–761. doi: 10.1175/1520-0469(1983) 040<0749:STATMV>2.0.CO;2doi.org/10.1175/1520-0469(1983)040<0749:statmv>2.0.co;2
  64. [73]Lofstead, J., Zheng, F., Liu, Q., Klasky, S., Oldfield, R., Kordenbrock, T., . . . Wolf, M. (2010). Managing variability in the IO performance of petascale storage systems. In Proceedings of the 2010 acm/ieee international conference for high performance computing, networking, storage and analysis. doi: 10.1109/SC.2010.32DOI
  65. [75]Marshall, J., Adcroft, A., Hill, C., Perelman, L., & Heisey, C. (1997). A finite-volume, incompressible Navier Stokes model for studies of the ocean on parallel computers. Journal of Geophysical Research: Oceans, 102(C3), 5753–5766.DOI
  66. [76]McDougall, T. J., & Barker, P. M. (2011). Getting started with TEOS-10 and the Gibbs Seawater (GSW) oceanographic toolbox. SCOR/IAPSO WG, 127(532), 1–28.
  67. [77]Molemaker, M. J., McWilliams, J. C., & Capet, X. (2010). Balanced and unbalanced routes to dissipation in an equilibrated eddy flow. Journal of Fluid Mechanics, 654, 35–63.DOI
  68. [78]Monin, A. A., & Obukhov, A. M. (1954). Basic laws of turbulent mixing in the surface layer of the atmosphere. Contrib. Geophys. Inst. Acad. Sci. USSR, 151(163), e187.
  69. [79]Moses, W. S., Cheng, G., Churavy, V., Gelbrecht, M., Klöwer, M., Kump, J., . . . Heimbach, P. (2026). DJ4Earth: Differentiable, and Performance-Portable Earth System Modeling via Program Transformations. Journal of Advances in Modeling Earth Systems, 18(5), e2025MS005615. doi: 10.1029/2025MS005615DOI
  70. [80]Moses, W. S., Churavy, V., Paehler, L., Hückelheim, J., Narayanan, S. H. K., Schanen, M., & Doerfert, J. (2021). Reverse-mode automatic differentiation and optimization of GPU kernels via Enzyme. In Proceedings of the international conference for high performance computing, networking, storage and analysis (pp. 1–16).DOI
  71. [81]Mrozowska, M. A., Avery, J., Stoustrup, A., Nuterman, R., Johnsen, C.-J., Thormann, A., & Jochum, M. (2025). Bayesian optimization with GPU acceleration for ocean models. Journal of Geophysical Research: Machine Learning and Computation, 2(3), e2024JH000517.DOI
  72. [82]Murray, R. J. (1996). Explicit generation of orthogonal grids for ocean models. Journal of Computational Physics, 126(2), 251–273.DOI
  73. [84]Pacanowski, R. C., & Gnanadesikan, A. (1998). Transient response in a Z-level ocean model that resolves topography with partial cells. Monthly Weather Review, 126(12), 3248–3270. doi: 10.1175/1520-0493(1998)126<3248:TRIALZ>2.0.CO;2doi.org/10.1175/1520-0493(1998)126<3248:triazl>2.0.co;2
  74. [85]Pacanowski, R. C., & Philander, S. G. H. (1981). Parameterization of vertical mixing in numerical models of tropical oceans. Journal of Physical Oceanography, 11(11), 1443–1451.doi.org/10.1175/1520-0485(1981)011<1443:povmin>2.0.co;2
  75. [86]Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., . . . others (2019). PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32.arxiv.org/abs/1912.01703
  76. [87]Paulson, C. A. (1970). The mathematical representation of wind speed and temperature profiles in the unstable atmospheric surface layer. Journal of Applied Meteorology (1962-1982), 857–861.doi.org/10.1175/1520-0450(1970)009<0857:tmrows>2.0.co;2
  77. [88]Pawlak, G., McCready, P., Edwards, K., & McCabe, R. (2003). Observations on the evolution of tidal vorticity at a stratified deep water headland. Geophysical Research Letters, 30(24), 2234. doi: 10.1029/2003GL018092DOI
  78. [89]Pearson, B. (2018). Turbulence-induced anti-Stokes flow and the resulting limitations of large-eddy simulation. Journal of Physical Oceanography, 48(1), 117–122.DOI
  79. [91]Peng, S., Silvestri, S., & Bodner, A. (2026). Submesoscale and boundary layer turbulence under mesoscale forcing in the upper ocean. Retrieved from https://arxiv.org/abs/2601.10441 doi: 10.48550/arXiv.2601.10441
  80. [92]Pérez, F., & Granger, B. E. (2007). IPython: a system for interactive scientific computing. Computing in science & engineering, 9(3), 21–29.DOI
  81. [93]Petersen, M. R., Jacobsen, D. W., Ringler, T. D., Hecht, M. W., & Maltrud, M. E. (2015). Evaluation of the arbitrary Lagrangian–Eulerian vertical coordinate method in the MPAS-Ocean model. Ocean Modelling, 86, 93–113.DOI
  82. [94]Phillips, N. A. (1956). The general circulation of the atmosphere: A numerical experiment. Quarterly Journal of the Royal Meteorological Society, 82(352), 123–164.
  83. [95]Pressel, K. G., Kaul, C. M., Schneider, T., Tan, Z., & Mishra, S. (2015). Large-eddy simulation in an anelastic framework with closed water and entropy balances. Journal of Advances in Modeling Earth Systems, 7(3), 1425–1456.DOI
  84. [96]Pressel, K. G., Mishra, S., Schneider, T., Kaul, C. M., & Tan, Z. (2017). Numerics and subgrid-scale modeling in large eddy simulations of stratocumulus clouds. Journal of Advances in Modeling Earth Systems, 9(2), 1342–1365.DOI
  85. [97]Ramadhan, A., Wagner, G., Hill, C., Campin, J.-M., Churavy, V., Besard, T., . . . Marshall, J. (2020). Oceananigans.jl: Fast and friendly geophysical fluid dynamics on GPUs. Journal of Open Source Software, 5(53). doi: 10.21105/joss.02018DOI
  86. [98]Rančić, M., Purser, R., & Mesinger, F. (1996). A global shallow-water model using an expanded spherical cube: Gnomonic versus conformal coordinates. Quarterly Journal of the Royal Meteorological Society, 122(532), 959–982.DOI
  87. [99]Rathgeber, F., Ham, D. A., Mitchell, L., Lange, M., Luporini, F., McRae, A. T., . . . Kelly, P. H. (2016). Firedrake: automating the finite element method by composing abstractions. ACM Transactions on Mathematical Software (TOMS), 43(3), 1–27.DOI
  88. [100]Redi, M. H. (1982). Oceanic isopycnal mixing by coordinate rotation. Journal of Physical Oceanography, 12(10), 1154–1158.doi.org/10.1175/1520-0485(1982)012<1154:oimbcr>2.0.co;2
  89. [102]Ringler, T., Petersen, M., Higdon, R. L., Jacobsen, D., Jones, P. W., & Maltrud, M. (2013). A multi-resolution approach to global ocean modeling. Ocean Modelling, 69, 211–232.DOI
  90. [103]Ronchi, C., Iacono, R., & Paolucci, P. S. (1996). The “cubed sphere”: A new method for the solution of partial differential equations in spherical geometry. Journal of computational physics, 124(1), 93–114.DOI
  91. [104]Roquet, F., Madec, G., Brodeau, L., & Nycander, J. (2015). Defining a simplified yet “realistic” equation of state for seawater. Journal of Physical Oceanography, 45(10), 2564–2579.DOI
  92. [105]Roquet, F., Madec, G., McDougall, T. J., & Barker, P. M. (2015). Accurate polynomial expressions for the density and specific volume of seawater using the TEOS-10 standard. Ocean Modelling, 90, 29–43.DOI
  93. [106]Rozema, W., Bae, H. J., Moin, P., & Verstappen, R. (2015). Minimum-dissipation models for large-eddy simulation. Physics of Fluids, 27(8).DOI
  94. [107]Sadourny, R. (1972). Conservative finite-difference approximations of the primitive equations on quasi-uniform spherical grids. Monthly Weather Review, 100(2), 136–144.doi.org/10.1175/1520-0493(1972)100<0136:cfaotp>2.3.co;2
  95. [108]Schneider, T., Lan, S., Stuart, A., & Teixeira, J. (2017). Earth system modeling 2.0: A blueprint for models that learn from observations and targeted high-resolution simulations. Geophysical Research Letters, 44(24), 12–396.DOI
  96. [109]Schumann, U., & Sweet, R. A. (1988). Fast Fourier transforms for direct solution of Poisson’s equation with staggered boundary conditions. Journal of Computational Physics, 75(1), 123–137.DOI
  97. [111]Shchepetkin, A. F., & McWilliams, J. C. (2005). The regional oceanic modeling system (ROMS): a split-explicit, free-surface, topography-following-coordinate oceanic model. Ocean modelling, 9(4), 347–404.
  98. [112]Shu, C. (1997). Essentially non-oscillatory and weighted essentially non-oscillatory schemes for hyperbolic conservation laws (ICASE Report No. 97-65). Institute for Computer Applications in Science and Engineering, NASA Langley Research Center.DOI
  99. [113]Shu, C.-W. (2009). High order weighted essentially nonoscillatory schemes for convection dominated problems. SIAM review, 51(1), 82–126.DOI
  100. [114]Silvestri, S., Campin, J.-M., Wagner, G. L., Constantinou, N. C., Lee, X. K., & Ferrari, R. (2026). A low-storage Runge-Kutta framework for non-linear free-surface ocean models. ESS Open Archive, 2026 (0420). doi: 10.22541/essoar.15002225/v1DOI
  101. [115]Silvestri, S., Wagner, G. L., Campin, J.-M., Constantinou, N. C., Hill, C. N., Souza, A., & Ferrari, R. (2024). A new WENO-based momentum advection scheme for simulations of ocean mesoscale turbulence. Journal of Advances in Modeling Earth Systems, 16(7), e2023MS004130. doi: 10.1029/2023MS004130DOI
  102. [116]Silvestri, S., Wagner, G. L., Constantinou, N. C., Hill, C. N., Campin, J.-M., Souza, A. N., … Ferrari, R. (2025). A GPU-based ocean dynamical core for routine mesoscale-resolving climate simulations. Journal of Advances in Modeling Earth Systems, 17(4), e2024MS004465.DOI
  103. [117]Silvestri, S., Wagner, G. L., Constantinou, N. C., Kump, J., Sohail, T., Sloan, J., … Buffo, J. (2026). ClimA/ClimaSea.jl: v0.5.0. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.16143708
  104. [118]Silvestri, S., Wagner, G. L., Hill, C., Ardakani, M. R., Blaschke, J., Campin, J.-M., … Ferrari, R. (2023). Oceananigans.jl: A model that achieves breakthrough resolution, memory and energy efficiency in global ocean simulations. arXiv preprint arXiv:2309.06662. doi: 10.48550/arXiv.2309.06662DOI
  105. [119]Smagorinsky, J. (1963). General circulation experiments with the primitive equations: I. The basic experiment. Monthly weather review, 91(3), 99–164.doi.org/10.1175/1520-0493(1963)091<0099:gcewtp>2.3.co;2
  106. [120]Solvik, K., Penny, S. G., & Hoyer, S. (2025). 4D-Var using hessian approximation and backpropagation applied to automatically differentiable numerical and machine learning models. Journal of Advances in Modeling Earth Systems, 17(4), e2024MS004608.arxiv.org/abs/2408.02767
  107. [121]Stone, P. H. (1971). Baroclinic stability under non-hydrostatic conditions. Journal of Fluid Mechanics, 45(4), 659–671.DOI
  108. [122]Strong-Wright, J., Chen, S., Constantinou, N. C., Silvestri, S., Wagner, G. L., & Taylor, J. R. (2023). OceanBioME.jl: A flexible environment for modelling the coupled interactions between ocean biogeochemistry and physics. Journal of Open Source Software, 8(90), 5669.
  109. [123]Tsujino, H., Urakawa, S., Nakano, H., Small, R. J., Kim, W. M., Yeager, S. G., … others (2018). JRA-55 based surface dataset for driving ocean–sea-ice models (JRA55-do). Ocean Modelling, 130, 79–139.DOI
  110. [124]Umlauf, L., & Burchard, H. (2003). A generic length-scale equation for geophysical turbulence models. Journal of Marine Research, 61. Retrieved from https://elischolar.library.yale.edu/journal_of_marine_research/9DOI
  111. [125]Umlauf, L., & Burchard, H. (2005). Second-order turbulence closure models for geophysical boundary layers. Continental Shelf Research, 25(7-8), 795–827.DOI
  112. [126]Vanneste, J., & Young, W. R. (2022). Stokes drift and its discontents. Philosophical Transactions of the Royal Society A, 380(2225), 20210032.DOI
  113. [127]Vanneste, J., & Young, W. R. (2026). A consistent phase-averaged model of the interactions between surface gravity waves and currents. Journal of Fluid Mechanics. Retrieved from https://arxiv.org/abs/2602.21976 (In press)
  114. [128]Van Roekel, L., Adcroft, A. J., Danabasoglu, G., Griffies, S. M., Kauffman, B., Large, W., … Schmidt, M. (2018). The KPP boundary layer scheme for the ocean: Revisiting its formulation and benchmarking one-dimensional simulations relative to LES. Journal of Advances in Modeling Earth Systems, 10(11), 2647–2685. doi: 10.1029/2018MS001336DOI
  115. [129]Vreugdenhil, C. A., & Taylor, J. R. (2018). Large-eddy simulations of stratified plane Couette flow using the anisotropic minimum-dissipation model. Physics of Fluids, 30(8).DOI
  116. [130]Wagner, G. L., Chini, G. P., Ramadhan, A., Gallet, B., & Ferrari, R. (2021). Near-inertial waves and turbulence driven by the growth of swell. Journal of Physical Oceanography, 51(5), 1337–1351.DOI
  117. [131]Wagner, G. L., & Constantinou, N. C. (2025). Phenomenology of decaying turbulence beneath surface waves. Journal of Fluid Mechanics, 1020, A51. doi: 10.1017/jfm.2025.10649DOI
  118. [132]Wagner, G. L., Constantinou, N. C., Giordano, M., Cheng, K.-Y., Rosevear, M. G., & Bischoff, T. (2026). NumericalEarth/Breeze.jl. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.18050353 doi: 10.5281/zenodo.18050353
  119. [133]Wagner, G. L., Hillier, A., Constantinou, N. C., Silvestri, S., Souza, A., Burns, K. J., … Ferrari, R. (2025). Formulation and calibration of CATKE, a one-equation parameterization for microscale ocean mixing. Journal of Advances in Modeling Earth Systems, 17(4), e2024MS005422.
  120. [134]Wagner, G. L., Silvestri, S., Constantinou, N. C., Strong-Wright, J., Sohail, T., Byrne, S., … Shen, Z. (2026). NumericalEarth/NumericalEarth.jl: v0.2.3. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.19410814 doi: 10.5281/zenodo.19410814
  121. [135]Warner, S. J., & MacCready, P. (2014). The dynamics of pressure and form drag on a sloping headland: Internal waves versus eddies. Journal of Geophysical Research: Oceans, 119(3), 1554–1571.DOI
  122. [138]Williamson, J. H. (1980). Low-storage Runge-Kutta schemes. Journal of Computational Physics, 35(1), 48–56. doi: 10.1016/0021-9991(80)90033-9DOI
  123. [140]Yatunin, D., Byrne, S., Kawczynski, C., Kandla, S., Bozzola, G., Sridhar, A., … Schneider, T. (2026). The Climate Modeling Alliance atmosphere dynamical core: Concepts, numerics, and scaling. Journal of Advances in Modeling Earth Systems, 18(3), e2025MS005014. doi: 10.1029/2025MS005014DOI
  124. [141]Young, W. R. (2012). An exact thickness-weighted average formulation of the Boussinesq equations. Journal of Physical Oceanography, 42(5), 692–707.DOI
  125. [142]Yuval, J., Langmore, I., Kochkov, D., & Hoyer, S. (2024). Neural general circulation models optimized to predict satellite-based precipitation observations. arXiv preprint arXiv:2412.11973.arxiv.org/abs/2412.11973
  126. [143]Zhang, W., Kuo, Y.-H., Silvestri, S., Adcroft, A., Hallberg, R. W., & Griffies, S. M. (2025). A WENO finite-volume scheme for the evolution of potential vorticity in isopycnal ocean models. Authorea Preprints. doi: 10.22541/essoar.175380391.18723979/v1DOI

Paper details

vlnc:W_8q8hq3wk
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents