betaFeedback

WorksPhysics and Astronomy

HydroTuring: A physics probe benchmark testing mass, energy, and momentum conservation in hydrologic models

Abstract

Hydrologic models, including deep learning systems now deployed for operational flood forecasting, are evaluated primarily by how closely their simulated hydrographs match observed streamflow. However, high hydrograph skill does not guarantee physical consistency: in a synthetic calibration experiment, a conceptual model that loses 15% of its water input to an unreported sink still reaches a Nash-Sutcliffe efficiency of 0.93, and no shared benchmark tests whether models obey fundamental conservation laws. Here we introduce HydroTuring (v0.1.0), a black-box benchmark of 37 physics probes that test mass (24 probes), energy (7) and momentum (6) conservation in hydrologic models. Each probe runs a controlled experiment on forcing generated at run time from a recorded seed and applies binary pass or fail criteria validated against 18 physically exact and 59 deliberately broken reference models. Across 32 evaluated models (3 exact references, 7 AI-based models and 22 process-based models), only the 3 exact references pass every probe among the 28 models scored on 5 or more probes. Process-based and AI-based models pass similar shares of their scored probes (76% and 74%), and a pre-trained flood-forecasting LSTM fails 6 of the 11 runoff-only probes it can be scored on. Furthermore, 49% of all 1,184 model-probe pairs remain unscored because models omit the required state or flux variables, while the scored failures occur in perturbation responses and state bounds rather than long-term water-budget closure. HydroTuring provides a reproducible smoke test for model development, peer review and operational deployment under an open community protocol.

Website flood-lab.github.io/HydroTuring | GitHub github.com/Flood-Lab/HydroTuring | Zenodo doi:10.5281/zenodo.23874854

MAINTAINER

PROBE AUTHORS, IN ORDER OF FIRST MERGED PROBE

MODEL CONTRIBUTOR

¹Flood Lab, Department of Civil, Environmental and Architectural Engineering, and Institute of Arctic and Alpine Research, University of Colorado Boulder, Boulder, CO, USA

²South China University of Technology, China

³Department of Electrical Engineering, Columbia University, New York, NY, USA

⁴Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Hong Kong, China

⁵Department of Civil and Environmental Engineering (D.I.C.A.), Politecnico di Milano, Milano 20133, Italy

⁶Independent researcher

⁷Laboratory of Catchment Hydrology and Geomorphology, École Polytechnique Fédérale de Lausanne (EPFL), 1951 Sion, Switzerland

⁸Center for Systems Integration and Sustainability, Department of Fisheries and Wildlife, Michigan State University, East Lansing, MI 48823, USA

⁹Dalian University of Technology, Dalian, China

¹⁰Institute of Mountain Hazards and Environment, Chinese Academy of Sciences, Chengdu, China

¹¹Indian Institute of Technology Mandi, Mandi, Himachal Pradesh, India

¹²School of Civil Engineering and Environmental Science, University of Oklahoma, Norman, OK, USA

¹³State Key Laboratory of Earth Surface Processes and Disaster Risk Reduction, Faculty of Geographical Science, Beijing Normal University, Beijing, China

¹⁴University of Illinois Urbana-Champaign, Urbana, IL, USA

¹⁵Institute of Global Environmental Change, Xi’an Jiaotong University, Xi’an 710049, China

¹⁶Department of Civil and Environmental Engineering, Stanford University, Stanford, CA, USA

¹⁷School of Geography and Planning, Sun Yat-sen University, Guangzhou 510006, China

¹⁸Department of Computer Science and Engineering, State University of New York at Buffalo, Buffalo, NY, USA

¹⁹University of Arizona, Tucson, AZ, USA

*Corresponding author: Zhi Li, [email protected].

Keywords: hydrologic modeling, benchmark, machine learning, conservation laws, water balance, physics-informed learning

Introduction

Machine learning has changed hydrologic modeling. Deep networks trained on large-sample data now reproduce runoff with skill that matches or exceeds process-based models [25, 26] (Nearing et al. 2021). These models are not constrained by the laws of mass, energy and momentum conservation [22] (Reichstein et al. 2019). Their predictions can become unphysical under out-of-sample conditions such as extremely high or low streamflow [17]. Even in a simple conceptual model, a severe conservation failure can hide behind a high hydrograph score.

In a synthetic calibration experiment, we fitted 2 conceptual models to the same daily runoff record. Model A (reference_bucket) conserves water, while model B (reference_leaky) sends 15% of its water input to a sink that its outputs never report. Against the observed streamflow, their Nash-Sutcliffe efficiencies (NSE) are 0.97 and 0.93. Both exceed the 0.75 threshold that a widely cited guideline rates as very good (Moriasi et al., 2007). A hydrograph score cannot tell the 2 apart, although model B loses 15% of every year’s precipitation. This is the long-standing problem of getting the right answer for the wrong reason [24], and it is a case of equifinality [6].

The problem grows with the complexity of the model. A hydrologic system is driven by precipitation, temperature and evaporative demand, and it carries soil, snow and groundwater states. With 6 such dimensions and 10 bins per dimension, a 40-year daily record of 14,610 samples can visit at most 1.5% of the distinct regimes Figure 1a. This is the curse of dimensionality [3]. A model fitted to streamflow alone is therefore constrained on a thin slice of the space it will later be asked to cover. Relying on historical streamflow alone exposes a model to 3 failure modes. First, climate is not stationary (Milly et al., 2008), so a response fitted to the record drifts away from the physical response as conditions leave it Figure 1b. Second, the largest events are rare and carry little weight in squared-error scores [18, 13], so a fitted model can flatten above the largest flow it has seen Figure 1c. Third, a model that gains or loses water at each step accumulates the imbalance in its stores, and the error grows over a long simulation without showing in the streamflow [20]. These limits are not unique to AI models, and process-based models can also fail due to imperfect mass closure. However, deep learning rainfall-runoff models tend to degrade more than well-calibrated process-based models on events beyond their training record [17]. HydroTuring therefore targets hydrologic models as a whole, whether learned, physical or hybrid, because a physical model can also fail a conservation test.

Figure 1. Why a streamflow fit does not establish physics. a, Upper bound on the fraction of distinct regimes that a record can visit, for 14,610 daily samples and 10 bins per dimension. The count is arithmetic, not data. b, c, Schematics of 2 failure modes of a pure streamflow fit. A fit trained on a record departs from a drifting physical response (b), and it stays flat above the largest event in the record (c). Panels b and c are illustrations and carry no data.

Current evaluation standards rely on NSE, the Kling-Gupta efficiency (KGE) and related scores on observed basins [18] (Nash and Sutcliffe 1970). Large-sample datasets made such comparisons routine [1, 27] (Newman et al. 2015). These scores do not test conservation, the response to a perturbation, or behavior outside the record. Physics-informed learning builds constraints into training [22, 20, 4], but no shared test checks whether the trained model honors them. Benchmarks in AI for science report accuracy against public data (Rasp et al., 2024; Takamoto et al., 2022). To our knowledge, no benchmark in hydrology or in Earth system science tests conservation laws on hidden, generated cases. HydroTuring is built to fill this gap.

We propose to treat physics as a test suite. The question changes from how close a model comes to 1 record to what the model must do under any record. The name follows Turing’s imitation game (Turing, 1950). That game judges a machine by its behavior under questions it was not prepared for. HydroTuring asks whether a model behaves like the physical system under questions that no training record answered. We envision that new AI-based and physics-based models will be tested against HydroTuring as a smoke test before they are deployed.

Here we introduce HydroTuring v0.1.0, a probe benchmark for the conservation of mass, energy and momentum in hydrologic models. We define a probe as a controlled experiment with pass or fail criteria on 1 physical law. Our objectives are (1) to define a probe-based benchmark for 3 conservation laws, (2) to run it through a black-box harness that hides the forcing and the tolerances from the model, and (3) to validate every probe against reference models that must pass or must fail. We then report a catalog of 37 probes with the verdicts of 32 models, and we describe an open community protocol that credits probe authors as co-authors. The paper is organized as follows. § 2 reviews related work. § 3 presents the methods: the design principles (§ 3.1), the 3 laws and the probe catalog (§ 3.2), the harness and its protocol (§ 3.3), and the validation of probes against reference models (§ 3.4). § 4 reports the verdicts, and § 5 presents the community protocol. § 6 and § 7 discuss limitations and conclude. We hope that this release attracts more contributors and users from the community.

Background and related work

AI-based hydrologic models

Data-driven rainfall-runoff modeling has been studied for decades. Artificial neural networks were applied to rainfall-runoff and streamflow forecasting in the 1990s [21, 2] (Maier and Dandy 2000). Decision and model trees followed (Solomatine and Dulal, 2003), and then random forests and related ensembles [8] (Shortridge et al. 2016). These models were fitted to 1 basin at a time and used as time-series predictors. Long short-term memory networks [19] carry memory of past states and predict streamflow with high skill when they are trained on many catchments at once [25, 26]. The same approach now supports forecasts of extreme floods in ungauged watersheds at global scale (Nearing et al., 2024). Differentiable models take a different route. They embed a process-based core in a trainable pipeline, so that the physical stores and fluxes remain explicit [16] (Song et al. 2025). Large language models have also entered the workflow, for example to assist the calibration of process-based models [31]. In practice, all of these approaches are evaluated mainly by hydrograph skill on observed basins. The role of hydrologic science in the age of machine learning is an open question (Nearing et al., 2021). A review of data-driven Earth system science argued for hybrid approaches that combine learning with process understanding, and named physical consistency and extrapolation as open challenges (Reichstein et al., 2019).

Physics in machine learning

Physics-based (physics-informed or physics-guided) AI has become prevalent in recent years. Physics-informed neural networks (PINN) add the governing equations to the training loss [22] (Raissi et al. 2019). Graph neural networks can embed the shallow water equations for flood prediction, as in HydroGraphNet (Taghizadeh et al., 2025). Hard-constrained networks enforce analytic constraints by construction [4]. Mass-conserving architectures route water through explicit stores, either inside a recurrent cell [20] or as a perceptron with gates (Wang and Gupta, 2024). Theory-guided data science frames the broader aim [23]. These methods build physics into training. HydroTuring tests it after training, on the model as delivered, without access to its weights. The 2 are complementary. Most of these methods target mass, while HydroTuring probes energy and momentum conservation as well. A mass-conserving architecture should pass the closure probes by construction, and the energy and momentum probes show what construction alone does not guarantee (§ 4).

Evaluation practice in hydrology

The Nash-Sutcliffe efficiency ((Nash and Sutcliffe, 1970)) and the Kling-Gupta efficiency ([18]) are the standard scores. Both aggregate errors over space and time. Neither checks the physics inside the model, so a model can give the right answer for the wrong reason ([24]), and models with different internal budgets can earn the same score ([6]). A score earned on 1 record also says little about transferability to other basins, periods and climates. Guidelines rate model performance by threshold values of these scores ((Moriasi et al., 2007)), and the common metrics are open to misuse when their assumptions are ignored ([13]). Closure of the mass, energy and momentum balances is traditionally a self-check by the model developer. It is rarely reported in a comparable form, and it stays hidden when the model is not open source. The method of multiple working hypotheses aims to expose such structural differences ([11]). The central concern is whether a model gives the right answer for the right reasons. HydroTuring tests the reasons directly, and it needs no access to the source. It asks what the model does when 1 input changes and everything else is held fixed.

Benchmarks in hydrology and AI for science

Large-sample datasets such as CAMELS ([1] (Newman et al. 2015)) and Caravan ([27]) made reproducible comparisons across hundreds of basins routine. WeatherBench 2 does the same for global weather forecasts ((Rasp et al., 2024)). PDEBench provides simulated datasets for scientific machine learning and reports, among its metrics, the deviation of predictions from conserved quantities ((Takamoto et al., 2022)). Agents’ Last Exam is a living benchmark whose task pool grows as new workflows are onboarded, with verifiable outcomes ((Sun et al., 2026)). The community model intercomparison projects, of which CMIP is the best known, show how a shared protocol can be extended by many groups ([15]). Table 1 compares these efforts with HydroTuring on 4 properties. To our knowledge, none of these efforts tests physical laws with generated cases and hidden tolerances.

Behavioral testing

Behavioral testing evaluates a model through the outputs it gives to controlled inputs, without access to its internals. CheckList applies this idea to NLP models, with templates that generate invariance and directional expectation tests ((Ribeiro et al., 2020)). HydroTuring applies the same idea to physical laws. A probe changes one input, such as air temperature, storm depth or the time step, and states what the output must do. Warmer air over the same rain must lower runoff, and the same water delivered at a finer time step must give the same volume.

BenchmarkQuantity evaluatedTest casesPhysical laws tested
CAMELS, Caravan ([1, 27] (Newman et al. 2015))Streamflow skill in observed basinsPublic observed recordsNot tested
WeatherBench 2 ((Rasp et al., 2024))Forecast skill against reanalysis and observationsPublic dataNot tested as laws
PDEBench ((Takamoto et al., 2022))Accuracy of learned PDE solversPublic simulated datasetsPartly, through the error of conserved quantities
Agents’ Last Exam ((Sun et al., 2026))Agent performance on professional workflowsTasks with verifiable outcomes, extended continuouslyNot applicable
CheckList ((Ribeiro et al., 2020))Behavior of NLP modelsTest templatesNot applicable
HydroTuringWhether a model conserves mass, energy and momentumGenerated at run time from a seed, with hidden tolerancesMass, energy and momentum, one law per probe

Table 1. Table 1. Comparison of HydroTuring with related benchmarks. Entries describe what each effort reports in its primary publication. “Generated” means that test cases are produced at run time from a seed, so that no fixed test set exists.

Methods

Design principles

The design of HydroTuring follows 7 principles. Each sets a standard for how a model should be tested to ensure it respects the underlying physics.

(1) Physical Probes. A probe asks 1 physical question and returns a verdict. The question is stated before the test is built, for example whether the same weather delivered at a finer time step gives the same runoff volume over time. A probe returns PASS, FAIL or N/A, and each criterion inside it keeps its own number. The N/A is applied when a particular model cannot be tested for that criterion. For example, if a model does not support sub-daily inputs, the resolution_invariance criterion would be marked N/A. There is no weighted aggregate of probes. A score of 0.8 on a composite cannot say which law a model violates, while a failed resolution_invariance criterion identifies both the violated law and the error magnitude.

(2) Hidden contract. A model is shipped as a Docker container with a manifest and a thin adapter, in any language, and it runs without network access. The harness writes a request, the forcing and the static attributes to a read-only directory, runs the entrypoint once, and reads the outputs from a result file. The model never sees the probe code or the tolerances. This keeps the benchmark open to learned models, process-based models and hybrids on equal terms, and it removes any need to share weights or source.

(3) Hidden forcing. A model is tested on cases it cannot have seen. Each probe generates its forcing at run time from a recorded seed. No dataset is committed to the repository, so there is nothing to memorize or tune against. Each probe runs several seeds, and all must pass. Of the 37 probes, 22 use 3 seeds, 14 use 5, and 1 uses 12. Seeds are recorded, so that any failure can be reproduced. A private suite can be added from outside the repository without changing the public one, which allows an operator to keep further probes undisclosed. (4) Paired cases. Many physical laws constrain how a system responds to a perturbation rather than a single state value. Warmer air over the same rain must lower runoff, and a storm added on a given day cannot change anything before that day. A probe therefore runs a control and 1 perturbed copy of the same case, which differ in 1 input, and it compares the 2 runs. This checks response and invariance, which an absolute closure test cannot reach. A model that ignores an input passes every single-run criterion, and a paired probe is the only test that catches it. Of the 37 probes, 18 compare 2 or more runs of the same case.

(5) Labeled regimes. Generators label the cases they produce. A label column, such as the stretch of a snow season or the days of an isolated storm, is read by the harness and never passed to the model. Criteria use the labels to score a regime separately, so that errors of opposite sign in different regimes cannot cancel, and the report can trace a failure to the regime that produced it.

(6) Strict aggregate. A model passes only when every criterion of every probe that applies to it passes. A single violated law is enough to fail. We adopt this strict rule because physical conservation is binary under a defined tolerance. A model that leaks water in 1 regime fails to conserve water, even if it closes the budget in all others.

(7) Validated probes. A probe is only as good as its ability to separate right from wrong. Every probe must therefore pass an acceptance gate before it is merged. The gate runs exact reference models, which must pass, and deliberately broken reference models, which must fail and must trip the criterion that each was built to trip. If a probe fails an exact model, its tolerance has drifted beyond what exact physics achieves, and the probe is wrong. If a probe passes a broken model, it does not discriminate. We return to the gate in § 3.4.

The three laws and the probe catalog

HydroTuring version 0.1.0 holds 37 probes, merged between 3 September and 16 October 2026: 24 for mass, 7 for energy and 6 for momentum. The probes use 44 distinct criterion types. We group them into 9 families (Figure 2 and Table 2). Section B lists every probe with its question, its criteria and its authors.

Figure 2. Map of the 37 merged probes by law and family. Each entry is 1 probe in suite version 0.1.0. The cross-budget family needs outputs from both the water and the energy ledgers.

LawFamilyProbesRepresentative experimentRepresentative criterion and limit
MassClosure410 years of generated weather, 5 seedsCumulative residual ≤5%\le5\% of precipitation
MassInvariance5Same weather at 1 min, 1 h and 1 dRunoff volumes agree within 10%
MassResponse6Largest storm scaled by 2, 5 and 10Added runoff share between 0.1 and 1.05
MassDynamics52 years without rainRunoff and stores only decline
MassGroundwater and human4380 mm irrigation withdrawalBudget difference accounts for the volume within 5%
EnergySingle budget4Hourly bare-surface budgetResidual within 5% or 2 W m−2^{-2}
EnergyCross budget34 months of drought at fixed net radiationChanges in λE\lambda E, HH and GG sum to zero
MomentumRouting24 large storms followed by dry spellsStore between zero and a 15-day hydrograph
MomentumHydraulics43 steady plateausFriction balance within 5%

Table 2. Probe families in suite v0.1.0. The last column gives 1 representative experiment and its limit. Full definitions are in Section B.

Mass

The governing relation is the water budget of the control volume,

P+G−E−Q−A=ΔSsoil+ΔSsnow+ΔScanopy+ΔSgw+ΔSch,(1)P + G - E - Q - A = \Delta S_{\mathrm{soil}} + \Delta S_{\mathrm{snow}} + \Delta S_{\mathrm{canopy}} + \Delta S_{\mathrm{gw}} + \Delta S_{\mathrm{ch}}, \tag*{(1)}

where PP is precipitation, EE is evapotranspiration including sublimation, QQ is runoff, GG is an exchange that the model declares with the outside, AA is a prescribed human water withdrawal, and ΔS\Delta S is the change in each reported store. The 24 mass probes fall into 5 families.

Closure probes ask whether the budget closes. The reference probe, catchment-closure, forces a lumped catchment with 10 years of daily weather and 5 seeds. It requires the cumulative residual to stay within 5% of cumulative precipitation, every store to remain within its physical range, and evaporation to stay at or below demand. The other 3 closure probes close the budget inside the snowpack across accumulation, storage and melt (snowpack-mass-closure), event by event under rainfall that is scaled toward 100-year depths (extreme-event-closure), and across 12 independent catchments with sampled soil and canopy capacities (ungauged-basin-closure).

Invariance probes ask whether the answer depends on how the modeler set up the domain. A model of the same physical system, discretized at a different spatial or temporal resolution, should give the same result. Each probe changes 1 such setup choice and checks that the output does not move:

  1. The same month of weather delivered at 1-minute, hourly and daily steps must give the same runoff volume within 10% (resolution-invariance).

  1. The same weather on a catchment 10 times larger must give the same depths (area-invariance).

  1. A 28-year shift of the calendar must leave fluxes and stores unchanged to a relative 10−910^{-9} (time-origin-invariance).

  1. A common 250 m shift of all groundwater heads must leave the exchange unchanged (groundwater-datum-invariance).

  1. A periodic climate must forget extra spin-up cycles (spinup-cycle-invariance).

Response probes ask whether a perturbation moves the output in the right direction and by a physical amount. Each probe applies 1 change to the forcing or the initial state and checks the response against a physical bound:

  1. Under identical precipitation, air 3 ∘C3\,^\circ\mathrm{C} warmer must lower runoff and air 3 ∘C3\,^\circ\mathrm{C} cooler must raise it, each by at least 0.1 of the change in evaporative demand (warming-response).

  1. Rain added to a run must appear in the budget and be shared among evaporation, runoff and storage (precipitation-counterfactual).

  1. Rain that would have fallen as snow must yield the same water within 5% (phase-counterfactual).

  1. A 120 mm storm added on 1 day may not lower runoff on any later day (response-nonnegativity).

  1. The same 60 mm storm on a wet catchment must run off more than on a dry one, by at least 2% of the storm and by no more than the 120 mm of extra water (antecedent-monotonicity).

  1. Scaling the largest storm to 10 times its size must not lower runoff, must not add more runoff than rain, and must send at least 10% of the added rain to runoff (extreme-rain).

Dynamics probes ask whether the catchment settles, drains and stays bounded. Each probe tests 1 part of the catchment’s behavior, from the moment of the storm to 50 years of repeated weather:

  1. A storm may not change anything before it falls (causality).

  1. After the rain stops, runoff and every store may only decline, and the total that runs off cannot exceed what the catchment held (dry-down).

  1. Constant weather for 3 years must produce constant stores and runoff within 1% (steady-state).

  1. Over 10 years, runoff must lie between rain minus potential evaporation minus capacity and rain plus the water the stores could release (runoff-bounds).

  1. Repeated weather over 50 years must not show storage drift beyond 2×10−42 \times10^{-4} of precipitation over the final 5-year block (multi-decadal-drift).

Groundwater and human probes ask whether every water term is accounted for. Each probe checks 1 water term that the model declares or prescribes against the budget:

  1. Direct aquifer recharge must preserve storage and the ordering of exchange (aquifer-recharge-ordering).

  1. A declared head-driven exchange must respond to its external head (exchange-response).

  1. In a 2-year aquifer case, a model that reports groundwater-river exchange must account for recharge, both exchange directions and aquifer storage, so that exchange and storage close the water budget (gw-sw-exchange-consistency).

  1. A prescribed irrigation withdrawal of 380 mm over 10 years must leave the budget, and the difference between the natural and the irrigated run must equal the abstracted volume within 5% (human-abstraction).

Energy

The governing relation is the surface energy budget,

Rn=H+λE+G,λE=λ(T)E,(2)R_n = H + \lambda E + G,\qquad\lambda E = \lambda(T)E, \tag*{(2)}

where RnR_n is net radiation, HH is the sensible heat flux, λE\lambda E is the latent heat flux, GG is the ground heat flux and λ(T)\lambda(T) is the temperature-dependent latent heat of the phase change. Seven probes test the energy budget.

Four single-budget probes test whether the energy balance holds by itself: (1) Evaporation on wet days must follow demand and on dry days must follow water (pet-consistency). (2) Hourly residuals must close within each solar day or night, with absolute residuals averaged so that opposite errors cannot cancel (surface-energy-closure). (3) Heat entering a soil layer minus heat leaving it must equal the change in stored heat (soil-heat-storage-consistency). (4) Upward longwave radiation must equal emission at the skin temperature plus reflected downward longwave, within 0.5% or 0.5 W m−2^{-2} (radiation-consistency).

Three cross-budget probes test whether the water and energy budgets agree: (1) A model with a water head and an energy head can close both budgets and still contradict itself, because the inconsistency lies in the coupling between the two budgets. Latent heat must equal evapotranspiration times the latent heat of the phase it underwent, within 0.5% (latent-heat-et-consistency). (2) A drying surface under fixed net radiation must give up its latent heat to the sensible and ground fluxes, and the 3 changes must sum to zero (evaporative-partition). (3) Melt that the water budget reports must be paid for in the energy budget, by the latent heat of fusion and the cold content of the pack (snowmelt-energy-water).

Momentum

Momentum conservation in a channel is expressed by the Saint-Venant equations. The suite does not yet test the full unsteady balance. It tests conditions that any solution of the equations must satisfy. In the steady uniform limit the bed slope S0S_0 equals the friction slope SfS_f, and Manning’s relation links discharge QQ and stage through the channel geometry,

Sf=S0,Q=1nAR2/3S01/2,(3)S_f = S_0,\qquad Q = \frac{1}{n} A R^{2/3} S_0^{1/2}, \tag*{(3)}

where nn is Manning’s roughness, AA the flow area and RR the hydraulic radius. On a mild slope the flow stays subcritical, so the Froude number remains below 1.

Two routing probes test whether routed water stays physical: (1) A routing store may never be negative, and it may never hold more than the cumulative runoff inflow over a 15-day hydrograph (routing-conservation). (2) The lag of a runoff peak must follow a Snyder travel-time scale (Snyder, 1938) within a factor of 2 across 4 catchments and must grow with catchment size (routing-lag-consistency). Four hydraulics probes test the consistency of reported stage and discharge: (1) Stage must be single-valued in discharge and rise with it, with a loop of the right orientation for a flood wave (stage-discharge-monotonic). (2) Discharge and stage on 3 steady plateaus must satisfy the friction balance within 5% (uniform-flow-friction-consistency). (3) The flow must remain subcritical on a mild slope (froude-regime). (4) The celerity of a small pulse, measured between a short and a long reach, must respond to the hydraulic state (wave-celerity-bounds).

Coverage and gaps

Momentum is the least developed law in version 0.1.0. Its 6 probes test consistency conditions and bounds, and no probe yet tests the unsteady momentum balance reach by reach. Three further probes are on the roadmap. Two are in progress: a probe on reach-by-reach mass closure in a branching network, mass/routing-network-closure, and a probe on the release of meltwater from a cold snowpack, energy/snowpack-ripening. The third, momentum/channel-routing-mass, is unclaimed and requires inflow minus outflow minus storage change per reach. An earlier attempt was set aside because the contract then had no variable for water arriving at a reach, so that both sides of the residual came from the model’s own outputs. The contract now carries a prescribed inflow, q_in, which the wave-celerity probe uses (Table 3). We plan to complete version 1 once the suite reaches 50 probes (§ 5).

NameUnitsPhysical meaning
prmm d−1^{-1}Precipitation, echoed exactly as the forcing gave it
evspsblmm d−1^{-1}Evapotranspiration, including sublimation; net of dew and frost where the model nets them
mrromm d−1^{-1}Total runoff
dism3^{3} s−1^{-1}River discharge
gwexmm d−1^{-1}Declared exchange with the outside: regional groundwater, inter-basin transfer
snmmm d−1^{-1}Liquid delivered from the snow module to the ground, rain passed through included, and not an outflow beside mrro
hflsW m−2^{-2}Latent heat flux
hfssW m−2^{-2}Sensible heat flux
hfgW m−2^{-2}Ground heat flux at the actual soil surface, where a deeper flux is corrected for the heat stored above it
mrsommSoil water storage (liquid and ice), reported as an absolute state rather than a tendency
snwmmSnow water equivalent, identically zero and named in zero_stores for a model with no snow module
canopymmCanopy interception storage, identically zero and named in zero_stores for a model with no such store
gwmmGroundwater storage below the soil column
channelmmWater generated as runoff but not yet released by the model’s routing
stagemWater-surface elevation on the case’s datum, read beside the row’s mean discharge (depth is stage minus bed_elevation_m)
Forcing a model receives
prmm d−1^{-1}Precipitation
tas°CNear-surface air temperature in Celsius, unlike the kelvin of ts and tsoil_layer
petmm d−1^{-1}Potential evapotranspiration
rnW m−2^{-2}Prescribed net radiation, which the model accepts as the energy input to the reported budget when declared
abstrmm d−1^{-1}Prescribed human withdrawal, net of return flow
q_inm3^{3} s−1^{-1}Prescribed river or upstream inflow, already a discharge and not multiplied by area or row length

Table 3. Core vocabulary of the /io contract. A model reports a subset of the output variables and declares which. A probe scores a model only if its outputs cover what the probe requires. The full lexicon is in Appendix § 10.

Harness and protocol

Architecture

The harness separates what a model can see from what decides its verdict (Figure 3). It does so by splitting each evaluation between two sides. On the host, a probe generates the forcing and the labels, and it keeps the labels, the criteria and their tolerances to itself. The model runs on the other side, in a sealed container, and receives only the forcing and the static attributes. Because the model never sees the labels or the criteria, it cannot tune its output to the verdict. Two files link the model to the harness. A manifest, model.yaml, declares what the model needs and what it emits, and a thin adapter, ht_adapter.py, translates between this contract and the native model, so that a model needs no change to its own code. Three runners then start the model: Docker, a plain subprocess for models that run natively, and Apptainer for clusters.

Architecture of the HydroTuring harness: Harness on the host, hidden from the model; Probe generator — seeded forcing and labels; Criteria — one law per probe, hidden tolerances; Verdict — PASS, FAIL or N/A with a reason; Model container — manifest, adapter and weights, any language, no network; Acceptance gate — exact references pass, broken references fail; Sandbox — request.json, forcing.csv, static.json (read-only), result.csv, run.json, routing.csv; same contract

Figure 3. Architecture of the HydroTuring harness. The probe generator, the criteria and the tolerances stay on the host. The model runs in a sealed container, reads the forcing and static attributes from a read-only directory, and writes the outputs that the criteria score. Reference models enter through the same contract in the acceptance gate (§ 3.4).

Lexicon and input-output contract

Model variables are named differently in different models. Some models name discharge (cms) as runoff (q), while others use different conventions. Additionally, a hydrologic model may output runoff alone, and a land surface model may output every flux and store. The contract therefore rests on a shared lexicon. In suite 0.1.0 it names 29 output variables, 14 forcing variables and 34 static attributes, each with units, a sign convention and a statement of what a value means over a time step (Table 3 and Section A). Names follow the CMIP conventions where an equivalent exists (Eyring et al., [15]). A model declares the outputs it emits and the inputs it needs or uses, and it must not pad that list. A probe declares the outputs, forcing and static attributes it requires. The harness scores a pair only when the two declarations are compatible.

A run has 3 steps: the harness stages the inputs, the model runs, and the harness checks what the model returns against the contract. In the first step, the harness writes a request, the forcing and the static attributes to a read-only directory, so that the model cannot alter them. On a river network, the run also reads reach-indexed variables from the routing table. In the second step, the entrypoint runs once and writes 1 row per forcing step to the recorded results, together with a provenance file. Spin-up rows are included in these results but are excluded from scoring. In the third step, the harness compares the results with the request. The adapter must emit the exact number of steps that the request names, so a step count mismatch or a missing declared column is a contract error.

Verdicts

A probe returns 1 of 3 verdicts and a reason (Table 4). PASS and FAIL are scientific statements. N/A means that the probe does not apply for this particular model. For example, a streamflow-only model is N/A on every budget probe. Rather than violating conservation, the model cannot be evaluated on that law because it omits the required outputs. We keep this case apart from FAIL because the 2 statements say different things about a model. The reason is stored beside the verdict, together with the measured quantity, so that a reader can see by how much a limit was missed.

VerdictReasonScoredMeaning
PASSOKyesEvery criterion on every seed is within its limit
FAILVIOLATIONyesAt least 1 criterion on 1 seed is outside its limit
N/AINCOMPLETEnoThe model does not report the variables that the probe needs
N/AINCOMPATIBLEnoThe model and the probe disagree on the time step, on an input, or on paired-perturbation support
–ERRORnoThe adapter or the machinery failed, which is operational and makes the run exit with an error code

Table 4. Verdicts and reasons. Only PASS and FAIL enter a model’s score.

A model passes the suite when at least 1 probe could be put to it and every probe it was scored on passes. This is the strict aggregate of § 3.1. Because N/A probes count neither for nor against a model, a pass fraction is meaningless without the number of probes scored. We therefore always report both.

Criteria on the reference probe

Each probe contains criteria, and each criterion returns a binary result and a measured value. The reference probe, catchment-closure, runs 10 generated years of daily weather after 365 days of spin-up, on 5 seeds. Table 5 lists its criteria. The closure criterion divides the cumulative residual of the budget by cumulative precipitation, and it uses a 5% limit as an engineering standard. § 3.4 justifies this limit with the reference models and a sweep over the tolerance.

CriterionQuestionLimit
closureDoes precipitation minus evaporation, runoff and the change of every store stay near zero?Cumulative residual at most 5% of cumulative precipitation
et_plausibleDoes evaporation stay at or below the demand?No step exceeds the potential evaporation supplied
state_boundsDoes every reported store stay in its physical range?No store outside its range
non_degenerateDoes the model produce runoff that responds to rain?Runoff ratio between 0.02 and 0.98 and coefficient of variation at least 0.1
forcing_fidelityDoes the model echo the forcing that it was given?Relative tolerance of 10−610^{-6}

Table 5. Criteria of the reference probe mass/catchment-closure. The last 3 criteria are shared with most mass probes.

Reference models and probe validation

Reference models

The suite carries 77 reference models, 18 exact and 59 broken. An exact reference conserves the law that a probe tests, by construction or through a published scheme. Four exact references serve as common baselines. The model reference_bucket (model A in § 1) is a single bucket that conserves water exactly. The models flex_lumped and flex_topo are 2 conceptual models of the FLEX family ([29]), and sacsma_snow17 couples the Sacramento soil moisture accounting model with the Snow-17 snow model ((Upstream Tech, 2026)). The 14 remaining exact references are written for the probes whose physics they exercise. They include an energy-balance snowpack, a soil column with prescribed heat capacity, a Saint-Venant channel and a lagged-head aquifer cell (Table 11).

ModelProbes requiring a pass
reference_bucket25
flex_lumped25
sacsma_snow1725
flex_topo24
reference_coupled3
reference_saint_venant2
reference_radiative1
reference_snow_energy1
reference_soil_heat1
reference_recharge_order_exact1
reference_driven_exchange1
reference_evolving_exchange1
reference_recharge_exchange1
reference_datum_exact1
reference_exchange_exact1
reference_snyder_router1
reference_rating1
reference_uniform_flow1

Table 11. Exact reference models and the number of probes whose gate requires them to pass. The first four are the common baselines. The others are written for the probes whose physics they exercise.

Each of the 59 broken references carries 1 defect that the suite must detect (Table 12). We use 6 of them throughout the paper. The model reference_leaky (model B in § 1) hides a silent 15% sink, so its budget does not close. The model reference_cheater solves for storage as whatever balances the budget, which closes the budget by construction and drives storage outside its physical range. The model reference_degenerate evaporates all precipitation and produces no runoff. The model reference_saturating caps daily runoff at 25 mm, so it is exact for ordinary storms and flat for extreme ones. The model reference_climatology emits the seasonal mean of runoff and evaporation whatever the weather. The model reference_in_sample leaks water only outside the range for which it was built. A last reference, reference_streamflow_only, is not broken but limited. It reports discharge only, from a store that does not read the temperature. It is N/A on every budget probe, and the warming-response probe catches it.

Reference modelDefect built inProbeCriterion
abstraction_blindThe exact bucket, blind to a prescribed human abstraction.human-abstractionhuman_abstraction
air_emitterNegative control for energy/radiation-consistency.radiation-consistencyradiative_identity
anticipatingThe reference bucket with its runoff smoothed over a centered 7-day window, so every reported value carries 3 days that have not happened yet.causalitycausality
Reference modelDefect built inProbeCriterion
area_leakThe bucket losing a share of its runoff that grows with the catchment area it is given: 0% at 25 km2^{2} and 50% at 2,500 km2^{2}.area-invarianceinvariance
calendarThe bucket model with a recession that drifts with the calendar year.time-origin-invarianceinvariance
cheaterSolves for storage as whatever balances the budget.antecedent-monotonicity, catchment-closure, precipitation-counterfactual, spinup-cycle-invariance, ungauged-basin-closureantecedent-monotonicity, counterfactual-response, state_bounds
climatologyA seasonal climatology baseline: emits the mean runoff and evaporation for the day of the year regardless of the weather, with constant storage.dry-downdry_down
constant_lambdaFully coupled and correct in every other respect, except that the latent heat of vaporization is a constant instead of a function of temperature.evaporative-partition, latent-heat-et-consistencyflux_identity
datum_dependentIntentionally datum-dependent conductance reference.groundwater-datum-invariancedatum_flux_invariance
degenerateThe trivial solution: all precipitation evaporates, nothing runs off, storage never changes.catchment-closure, precipitation-counterfactual, resolution-invariance, runoff-bounds, spinup-cycle-invariance, time-origin-invariance, ungauged-basin-closure, warming-responsecounterfactual-response, non_degenerate, response_sign, runoff_bounds
degree_dayThe same snowpack, melted on air temperature through a degree-day factor and with a surface energy budget that closes around its evaporation alone.snowmelt-energy-watermelt_energy
diurnal_biasThe coupled reference’s water and energy account with only sensible heat biased: the residual is +20 W m−2^{-2} from 06:00 to 18:00 local-solar time, and −20 W m−2^{-2} otherwise.surface-energy-closureenergy_closure_by_phase
energy_leakThe energy-budget counterpart of reference_leaky: coherent between water and energy, but 15% of net radiation never appears in any flux it reports.evaporative-partition, latent-heat-et-consistencyenergy_closure
exchange_sign_errorA lagged-head bookkeeping reference sharing the smoothed-stage mechanism of reference_exchange_exact (declaring aquifer_storage_coefficient in needs_static to match probe.yaml), with a bidirectional river exchange that closes its groundwater balance but reverses a component sign.gw-sw-exchange-consistencyexchange_components
fixed_celerityDeliberately broken routing reference for momentum/wave-celerity-bounds.wave-celerity-boundswave_celerity_bounds
Reference modelDefect built inProbeCriterion
fixed_stepThe reference bucket with its step hard-wired: it claims to run at any resolution and ignores the one it is given, treating every row as a day.resolution-invarianceresolution_invariance
flat_stageThe routed bucket that reports a stage at all times equal to a constant.stage-discharge-monotonicnon_degenerate
frozen_soilThe conductive soil heat reference with reported layer temperature frozen at its initial value.soil-heat-storage-consistencysoil_heat_storage
ground_dodgeLatent heat agrees with the water it evaporated and the energy budget closes, but the sensible heat flux is a fixed share of net radiation that never reads the soil.evaporative-partitionpartition_shift
gw_slow_driftAn exact bucket whose reporter subtracts 0.001 mm d−10.001\ \mathrm{mm}\ \mathrm{d}^{-1} from runoff and accumulates the deficit in groundwater storage.multi-decadal-drifttotal_storage_drift
half_soilThe conductive soil heat reference with only its reported layer temperature response halved.soil-heat-storage-consistencysoil_heat_storage
in_sampleThe bucket model with a leak that only opens outside the range it was built for.extreme-event-closureevent_water_closure
instant_routerDeliberately wrong rainfall-runoff reference for the routing-lag probe.routing-lag-consistencylag_time_bounds
inverse_routerDeliberately wrong geometry-aware reference for the routing-lag probe.routing-lag-consistencyscaling-monotonicity
leakyThe bucket model with a silent 15% sink.catchment-closure, human-abstraction, precipitation-counterfactual, spinup-cycle-invariance, time-origin-invarianceclosure, counterfactual-response
leaky_routerThe bucket with a unit hydrograph whose weights sum to 0.999: 0.1% of every day’s runoff enters the channel store and never leaves.routing-conservationrouting-conservation
no_reflectionNegative control for energy/radiation-consistency.radiation-consistencyradiative_identity
noise_sinkThe exact bucket, conserving water internally, which misreports its evaporation by a seeded ±30%\pm30\% and declares the difference as a regional groundwater exchange.exchange-responseexchange_response
overflowingThe bucket reporting its runoff plus an unphysical 80% of the rain.runoff-boundsrunoff_bounds
overshootingThe bucket with its runoff sharpened by a derivative term: each day’s reported runoff is the bucket’s plus 0.8 times the change since the day before, floored at zero.response-nonnegativityresponse_nonnegativity
rating_driftThe routed bucket with a drifting gauge.stage-discharge-monotonicrating_monotonic
rating_invertedThe routed bucket whose rating is read backwards.stage-discharge-monotonicrating_loop
recharge_blindRecharge-ordering reference that ignores gw_recharge, so its own budget no longer closes.aquifer-recharge-orderingaquifer_recharge_ordering
recharge_overshootRecharge-ordering reference that sends 120% of a recharge day to the river, so its storage falls below the control’s.aquifer-recharge-orderingaquifer_recharge_ordering

Table 12. Deliberately broken reference models (with the reference_ prefix omitted in column 1) and the criterion each is built to trip.

Acceptance gate

A probe enters the suite only when its gate is satisfied. The gate has 2 kinds of check. A must-pass check requires an exact reference to pass the probe. A must-fail check requires a broken reference to fail the probe and to trip the criterion that it was built to trip. The 37 probes carry 116 must-pass and 78 must-fail checks. We re-ran all 37 gates on 2 November 2026 at the commit that tags suite 0.1.0, and every check behaved as declared.

The gate fixes tolerances from 2 sides. An exact model sets the lower bound, because a tolerance tighter than what exact physics achieves would fail honest models. Over 20 random seeds, the exact bucket, flex_lumped and sacsma_snow17 pass catchment-closure on all 20, with residuals below 2.3×10−152.3 \times10^{-15} of precipitation. A broken model sets the upper bound. Over the same seeds, reference_leaky fails the closure criterion on 20 of 20. The model reference_cheater fails the state-bounds criterion on 20 of 20, and reference_degenerate fails the non-degeneracy criterion on 20 of 20 (Table 14).

ProbeModelSeeds passedCriterion trips or measured range
catchment-closurereference_bucket20/20largest residual 2.6×10−162.6 \times10^{-16} of precipitation
catchment-closureflex_lumped20/20largest residual 2.2×10−152.2 \times10^{-15} of precipitation
catchment-closuresacsma_snow1720/20largest residual 2.0×10−152.0 \times10^{-15} of precipitation
catchment-closurereference_leaky0/20closure 20 of 20
catchment-closurereference_cheater0/20state_bounds 20 of 20
catchment-closurereference_degenerate0/20non_degenerate 20, et_plausible 18 of 20
extreme-rainreference_bucket10/10share 0.56 to 1.00
extreme-rainreference_saturating4/10share 0.01 to 0.22

Table 14. Seed sensitivity of 2 probes. Top: mass/catchment-closure on 20 random seeds. Bottom: mass/extreme-rain on 10 random seeds. The column on the right gives the criterion trips over the seeds, or the share of added rain that runs off over the ladder up to 10 times the largest storm, as the range over seeds.

The closure tolerance sits between the 2 bounds. It is 5% of cumulative precipitation, the engineering rule that the probe adopts. Among the 22 evaluated models that report a whole-record residual on catchment-closure, 21 are below 0.2% and the largest is 2.5% (xaj). Any tolerance between 2.6% and 15% therefore separates every evaluated model from reference_leaky, whose sink is 15% by construction (Figure 5a). We keep 5% because it is conventional and because it sits in this empty interval. Any change to it requires the process in § 5, and Section D gives the full sweep.

Figure 5. Closure and failure modes in suite 0.1.0. a, Whole-record closure residuals that the evaluated models report on the closure probes (catchment, ungauged-basin and snowpack), as a share of cumulative precipitation, with 1 dot per model and probe, and the same quantity for the broken reference reference_leaky. The dashed line marks the 5% limit that the 4 closure probes share. b, Number of failed model-probe pairs in which each criterion was tripped. A pair can trip more than 1 criterion. The 131 failed pairs trip 26 distinct criteria, and the 10 most frequent are shown.

A single case can miss a defect. The model reference_saturating fails the extreme-rain criterion on 6 of 10 seeds and passes it on 4. The signal therefore varies from seed to seed. Across the ladder up to 10 times the largest storm, the exact bucket returns between 0.56 and 1.00 of the added rain on every 1 of the 10 seeds. The saturating model returns 0.01 to 0.22 and exceeds the limit of 0.10 on 4 of them. A probe that used 1 seed would pass the saturating model 40% of the time. The probe uses 3 seeds, and a model passes only when all 3 pass. Of the 120 subsets of 3 among the 10 seeds, 116 contain at least 1 seed that exposes the defect. Across the suite, 22 probes use 3 seeds, 14 use 5, and the probe on catchment attributes uses 12.

Results

We evaluated 32 models on the 37 probes of suite 0.1.0. Three are the physically exact references flex_lumped, flex_topo and sacsma_snow17. Seven are AI-based: an LSTM, the differentiable HBV model dhbv2 (Song et al., 2025), and 5 training checkpoints of a mass-conserving perceptron, mcp1 to mcp5 (Wang and Gupta, 2024). The remaining 22 are process-based models, from conceptual bucket models to land surface models, a groundwater model and routing models. They include gr4j (Perrin et al., 2003), xaj (Zhao, 1992), topmodel [5], topkapi [10], vic [32, 34], mhm (Samaniego et al., 2010), summa [12], raven [14], hype [33], prms (Markstrom et al., 2015), noahmp (Niu et al., 2011), cwatm [9], lisflood (Van Der Knijff et al., 2010), modflow6 [28] and mizuroute (Mizukami et al., 2016). Of the 32 models, 29 entered through model proposals § 5. Each model was run in its own container with the manifest and adapter in the repository, and each version is recorded in the archive.

Verdict matrix

The 32 models and 37 probes give 1,184 model-probe pairs (Figure 4). Of these, 474 are PASS, 131 are FAIL and 579 are N/A. Among the N/A pairs, 499 are INCOMPLETE and 80 are INCOMPATIBLE. Consequently, 49% of all model-probe pairs remain unscored due to missing output variables or time-step mismatches rather than physical failures. Mass probes are N/A for 30% of pairs (234 of 768), while energy probes are N/A for 84% (188 of 224) and momentum probes for 82% (157 of 192). Few models report heat fluxes or reach-indexed variables.

Figure 4. Verdict matrix of 32 models on 37 probes in suite 0.1.0. Rows are models. The 3 exact references are on top, and the other models follow in order of the number of probes that they pass. Columns are probes, grouped by law. Green is PASS and orange is FAIL. Light gray is N/A because the model does not report the required variables (INCOMPLETE), and dark gray is N/A because model and probe disagree on an input or a time step (INCOMPATIBLE).

Only the 3 exact references pass every probe that they are scored on (74 of 74 pairs). Among the 28 models that are scored on 5 or more probes, no other model does so. The remaining 4 models, cfe, mizuroute, modflow6 and rvic, are scored on 1 to 3 probes. Three of them pass all of their scored probes, which says little because the sample is small. The 7 AI-based models pass 98 of 133 scored pairs (74%), while the 22 process-based models pass 302 of 398 (76%). Because these pass rates are nearly identical, model class alone does not predict physical consistency. Table 15 gives the standing of every model.

ModelClassAllMassEnergyMomentum
flex_lumpedPhysical reference25/2519/191/15/5
sacsma_snow17Physical reference25/2520/201/14/4
flex_topoPhysical reference24/2419/191/14/4
mcp1AI-based18/2017/191/1–
mcp3AI-based17/2116/191/10/1
mcp2AI-based15/2015/190/1–
dhbv2AI-based15/2113/191/11/1
mcp4AI-based14/2013/191/1–
mcp5AI-based14/2014/190/1–
lstmAI-based5/115/11––
mhmProcess-based21/2219/201/11/1
gr4jProcess-based20/2118/191/11/1
hymodProcess-based20/2118/191/11/1
topkapiProcess-based20/2118/191/11/1
prmsProcess-based20/2218/201/11/1
tankProcess-based19/2018/191/1–
lisfloodProcess-based19/2117/191/11/1
xajProcess-based19/2117/191/11/1
topmodelProcess-based19/2217/191/11/2
wflow_sbmProcess-based19/2216/191/12/2
cwatmProcess-based17/2116/190/11/1
noahmpProcess-based17/2514/203/5–
vicProcess-based16/2315/191/4–
hypeProcess-based15/2313/201/11/2
simhydProcess-based11/2011/190/1–
shudProcess-based9/228/190/11/2
summaProcess-based9/237/191/31/1
ravenProcess-based7/226/200/11/1
modflow6Process-based3/33/3––
cfeProcess-based1/11/1––
mizurouteProcess-based1/1––1/1
rvicProcess-based0/1––0/1

Table 15. Standing of the 32 evaluated models in suite 0.1.0. Entries are probes passed over probes scored. A probe is not scored when the model does not report the variables it needs or cannot run at the probe’s time step (N/A). Rows are ordered by class and then by the number of probes passed. The LSTM is the model behind the case study in § 4.3.

Models fail 25 of the 37 probes. Twelve probes fail no evaluated model, and 3 of them, energy/snowmelt-energy-water, energy/surface-energy-closure and mass/exchange-response, have not yet scored any submitted model. The most discriminating probes are mass/human-abstraction (22 of 27 scored models fail), mass/antecedent-monotonicity (10 of 28), mass/dry-down (9 of 28), mass/resolution-invariance (9 of 29) and mass/response-nonnegativity (8 of 28).

Where models fail

The 131 failed pairs trip 26 distinct criteria (Figure 5b). The 5 most frequent are human_abstraction (22 pairs), state_bounds (21), non_degenerate (16), et_plausible (15) and antecedent_monotonicity (10). Four observations stand out.

First, no evaluated model trips the whole-record closure criterion on any of the 4 closure probes. Among the 22 models that report a residual on catchment-closure, the largest is 2.5% of precipitation. Across the closure probes that report a residual, the largest is 3.9% (xaj on ungauged-basin-closure; Figure 5a). Closure of the long-term water budget holds across the evaluated models.

Second, a closing budget does not imply physical stores and fluxes. Five of 27 scored models fail catchment-closure on another criterion. The model hype lets soil storage reach 363 mm in a 320 mm profile. The models raven and shud evaporate 1.01 and 1.27 times the potential evaporation that the probe supplied. The model simhyd exceeds both its canopy capacity and the potential evaporation, and summa lets canopy storage reach 20.3 mm against a capacity of 2 mm. Although these models close their overall water budgets, they report states or fluxes that exceed physical capacity or atmospheric demand.

Third, closure over the record does not imply closure event by event. Four models fail the event-wise criterion of mass/extreme-event-closure. The model xaj fails on 133 of 1,316 events and shud on 130 of 1,257. The model cwatm fails on 4 of 1,257 events and wflow_sbm on 1 of 1,299. The model xaj passes the whole-record criterion with a residual of 2.5%, so a total over the record can hide events that lose or gain water. Three further models, hype, simhyd and summa, fail the same probe on state bounds.

Fourth, most models do not account for a prescribed human withdrawal. On mass/human-abstraction, 20 of 27 scored models show no change in any budget term when the probe withdraws 380 mm of water over 10 years. Two more account for part of it: lisflood leaves 83% of the withdrawal unaccounted for, and prms leaves 29%. Only 5 scored models close this budget within the 5% limit, of which 3 are the exact references.

Case study: an LSTM

An LSTM flood-forecasting model illustrates how HydroTuring evaluates a model that reports streamflow alone. We ran the publicly released weights of the model lineage behind Nearing et al. (2024), with the training forcing products mapped from the probe’s precipitation, temperature and potential evaporation. Inputs that a probe cannot supply, such as net radiation and surface pressure, are derived from the forcing alone. Catchment attributes are taken from the first year of the generated record or set to the training mean. The model therefore operates as an average catchment under mock inputs. Our evaluation reflects the released weights coupled with this adapter rather than the full operational forecasting system.

The model reports runoff and discharge only. It is N/A on 26 of 37 probes. On 24 of them the probe requires evaporation, storage or heat fluxes, while on 2 the model and the probe disagree on an input or a time step. Rather than violating conservation on those 26 probes, the model cannot be falsified on them because it omits the required outputs. On the 11 probes that require runoff alone, it passes 5 and fails 6 (Table 6).

ProbeVerdictMeasured quantity
resolution-invarianceFAILRunoff differs by 73.8% of precipitation between hourly and daily records of the same month (limit 10%)
extreme-rainFAILAcross the ladder up to 10 times the largest storm, 35 of 526 mm of added rain run off (0.07). The exact bucket returns 0.56 to 1.00
dry-downFAIL2 rainless years yield a near-constant 0.3 mm d−1^{-1} that does not decay (coefficient of variation 0.02, minimum 0.1)
warming-responseFAILRunoff falls by 0.088 per unit of added demand (limit 0.10). The exact bucket falls by 0.20
phase-counterfactualFAILRunoff differs by −8.4% of the rain when snow falls as rain (limit 5%)
response-nonnegativityFAILRunoff falls 0.18 mm d−1^{-1} below the control after 120 mm of rain is added
causalityPASSNothing changes before the added storm. Runoff responds with 0.32 of the added rain
steady-statePASSSettles to 0.63 mm d−1^{-1} under 2.5 mm d−1^{-1} of rain, with no variation in the last year
runoff-boundsPASSRunoff is 0.34 of the rain, inside the bounds 0.12 to 1.05
antecedent-monotonicityPASSThe wetter catchment runs off 16.4 mm more from a 60 mm storm (0.27 of it)
area-invariancePASSScaling the catchment area leaves the depths unchanged

Table 6. Verdicts of the LSTM on the 11 probes that it can be scored on, with the measured quantity on the seed that the report shows. Reference values are from the exact bucket on the same probe where stated.

The pattern is consistent: the model passes probes whose answer follows from its causal recurrent structure or a plausible climatological response, while failing every probe that moves the input outside the training regime. The passes are straightforward. The model uses no future information, settles under constant weather, keeps runoff within the bounds of precipitation, and produces more runoff on a wetter catchment. The failures fall into 3 groups. The first group concerns input extremes. Across the ladder up to 10 times the largest storm, the model returns only 7% of the added rain, even though a catchment with a 320 mm soil capacity cannot absorb 526 mm. Over 2 rainless years, it maintains a baseflow floor learned from its training basins, so the discharge never recedes. The second group concerns perturbation responses. Warming produces the expected sign in both directions on every seed, but its magnitude falls below the 0.10 threshold on 3 of 5 seeds. Allowing the static climate attributes to update with the warmed record yields a stronger response, indicating that part of the apparent climate sensitivity is encoded in the static attributes rather than the dynamic weather forcing. The third group concerns physical consistency.

Hourly records produce a runoff volume that differs by 73.8% of precipitation from daily records of the same weather, because the network accepts arbitrary step sequences without an explicit time-step parameter and treats each hourly rate as a daily depth. Added rain also reduces subsequent daily runoff by 0.18 mm d−1^{-1}.

Differentiable and mass-conserving models

Two other AI-based families provide a contrast. The differentiable model dhbv2 (Song et al., 2025) embeds an explicit HBV core, while the 5 mass-conserving perceptron checkpoints mcp1 to mcp5 (Wang and Gupta, 2024) route water through explicit stores with conservation built into the architecture.

The mass-conserving perceptron closes the water budget by construction, and the probes confirm this property. All 5 checkpoints pass catchment-closure, extreme-event-closure and ungauged-basin-closure, with whole-record residuals below 0.001% of precipitation. However, they fail between 2 and 6 other probes, and the failed probes vary across training epochs within the same architecture. The checkpoint at epoch 27 (mcp1) fails 2 probes, while the checkpoint at epoch 2707 (mcp4) fails 6. In mcp4, runoff rises by 1.17 times the rain added when a storm is scaled by 2, falls below the control after rain is added, and still varies by 11.7% (runoff) and 4.2% (storage) in the final year of constant weather. Four checkpoints fail antecedent-monotonicity: the wetter catchment returns only 0.1 to 0.4 mm more runoff from a 60 mm storm in 3 checkpoints (below the 1.2 mm minimum), and 2.3 mm less in mcp4. Enforcing conservation in the architecture therefore secures that single law while leaving other physical behaviors dependent on training.

The differentiable model passes the 3 closure probes that it is scored on, together with the extreme-rain and warming probes. However, it fails 6 probes in the invariance and response families. Its runoff differs by 12.1% of precipitation between hourly and daily records (limit 10%), likely because the calibrated parameters retain the daily time-step scaling at which they were fitted. Runoff also differs by −34.8% of the rain when snow falls as rain (limit 5%). Added rain lowers subsequent runoff by 0.28 mm d−1^{-1}, and the precipitation-counterfactual probe records 10,040 mm of runoff in the control run against 8,686 mm of rain. Finally, the model fails the area-invariance probe because its channel store departs from the expected scaling when catchment area is rescaled, suggesting that channel parameters calibrated for 1 basin area do not transfer across scales. An explicit physical core does not by itself guarantee the invariances tested by the suite.

Community protocol and governance

A benchmark for physical laws needs more physics beyond what an individual group can supply. Mass, energy and momentum each have specialists who disagree about tolerances, denominators and what counts as a violation. We therefore built HydroTuring as a protocol with explicit steps for contributing, reviewing and crediting work, and we describe it here because the 37 probes in this article exist through it.

Contributing a probe or a model

There are 2 routes. A contributor who wants to add a physical test claims an item from the public roadmap, or proposes a new probe, through an issue. The maintainer labels the proposal as accepted when the probe would discriminate between reference models, and the issue is assigned to the person who will do the work. This early review filters out weak probe designs before contributors invest time in implementation. The contributor then scaffolds the probe from a template with ht init-probe, writes the generator, the criteria and the reference models, and runs the acceptance gate. A contributor who wants a model evaluated proposes it through a separate form. The proposer does not need to package the model. The issue is assigned to whoever will build the container, which is often the maintainer.

Both routes are written for coding agents as well as for people. The repository contains an AGENTS.md file that states the input-output contract, the lexicon and the acceptance rules in compact form. An agent given a model repository and this file can draft an adapter, and the contract review below then checks it. Because a failed probe is a scientific finding rather than a packaging defect, physical failures are never grounds for rejecting a model.

Review and decisions

Every probe merged from 16 September 2026 is reviewed by at least 2 members of a reviewer pool for its law. The probe is merged only after both reviewers approve and the gate is green. The pool currently has 3 reviewers for energy and 3 for mass. Probes merged before 16 September were accepted by the maintainer under the earlier process. Each model undergoes a single review focused on contract compliance rather than physical performance: the adapter must honor the input-output directory, the declared outputs must be honest, and the image must be reproducible.

A scientific steering committee of 4 to 6 people, spanning hydrologic modeling, machine learning for Earth science and operational hydrology, is being formed. It will settle disputes about probe design, decide when a suite version is cut, and approve any change to the tolerance of a merged probe together with the authors of that probe. Such a change silently alters every recorded result, so it bumps the minor version of the suite.

The repository lists 25 contributors beyond the maintainer, namely 18 authors of merged probes and 7 further proposers of accepted models. The 18 probe authors and 1 proposer with 5 accepted proposals are authors of this article. The remaining 6 proposers are credited in the contributor file.

Growth and roadmap

The suite grew from 1 probe on 3 September 2026 to 37 on 16 October (Figure 6). The model pool grew from 1 to 10 by 17 September and to 32 on 31 October. The roadmap lists the probes that we want next, with the template to start from and a note on what each probe discriminates. Two further probes are claimed and in progress, a network-wide mass closure probe and a snowpack ripening probe. One probe remains unclaimed, the reach-by-reach mass balance of a channel. We aim for 50 probes at version 1, with 13 to add, and for more energy and momentum probes in particular.

Figure 6. Growth of HydroTuring from its first probe to suite 0.1.0. Dates are those of the commits that merged each probe and each model into the main branch. The dashed line marks the target of 50 probes for version 1. Of the 32 models, 22 were added on 31 October 2026.

Discussion and limitations

What the verdicts show

Three results stand out. First, evaluated models rarely fail on long-term water-budget closure. All 22 models that report a residual on catchment-closure stay below 2.6% of precipitation; instead, failures occur in the response of the model to a perturbed input and in the physical bounds of reported states and fluxes. Second, enforcing mass conservation in a network architecture secures that law alone without guaranteeing physical behavior on other laws or regimes. Although the 5 mass-conserving perceptron checkpoints close the water budget within 0.001%, they still fail between 2 and 6 probes each. Third, nearly half of all model-probe pairs (49%) cannot be scored, primarily because models omit the variables that a probe requires. Reporting internal states and fluxes is therefore the most direct way to make a model falsifiable.

Use before deployment and during training

We see 2 use cases of HydroTuring. A modeler can run HydroTuring as a smoke test before a model enters operations, in the way a unit test suite precedes a software release. The verdicts name the violated law and give the measured size of the violation. A forecaster or a water manager needs this information to decide whether a model can support drought assessment, irrigation planning or a climate impact study that relies on evaporation and storage. Journals can also run submitted models through the suite as a reproducibility and consistency check, because the suite is open and every verdict can be reproduced from a recorded seed.

The probes can also serve during training. The mass-conserving perceptron checkpoints show that verdicts change from epoch to epoch within 1 architecture. The checkpoint at epoch 27 fails 2 probes, while the checkpoint at epoch 2707 fails 6. A model developer can therefore run the probes on saved checkpoints and select among them based on physical consistency as well as hydrograph skill. Although the probes are black-box experiments and cannot enter a gradient-based loss function directly, they identify which physical constraints a training objective should add.

Limitations

First, passing the benchmark is a necessary condition for physical consistency rather than a sufficient condition for predictive accuracy. A model that passes every probe conserves mass, energy and momentum under the tested conditions, but it may still be inaccurate on an observed basin or violate a law under untested conditions. The suite therefore complements rather than replaces hydrograph evaluation.

Second, public probes can be targeted. A fresh seed prevents a model from memorizing a case, but it does not prevent tuning to the rule that a probe tests. The suite offers 2 defenses. A private probe tree can be added with -probe-root, so that an evaluator can hold out probes whose generators and tolerances are unpublished. Each verdict also reports the measured quantity, so that a model that barely passes can be identified. We therefore regard verdicts from the public suite as a pre-deployment smoke test rather than a certification.

Third, tolerances are conventions. The 5% closure limit, the 10% resolution limit and the other thresholds are engineering choices, and the sweep in § 3.4 shows that the closure verdicts of the evaluated models do not depend on the exact value between 2.6% and 15%. For other criteria, the gate ensures that exact references pass and broken references fail. Changes to a tolerance require the governance process in § 5.

Fourth, the catchment and the weather are synthetic. Most probes run on 1 invented lumped catchment of 250 km2250\ \mathrm{km}^{2} with a 320 mm soil and a 2 mm canopy, under weather from a generator whose climate does not match any gauged basin. Only the ungauged-basin probe samples other attributes. This design is a strength for controlled experiments because the exact physical response is known. However, a physical violation and a distribution shift cannot always be separated. A model trained on real basins that misbehaves in the synthetic catchment may violate physics or may operate far outside its training distribution. The adapter adds a second dependence. In an early evaluation of the differentiable model, 1 reasonable choice for a seasonality attribute changed the coefficient of variation of runoff from 1.1 to 0.07. The probe does not provide this attribute, and that choice changed 2 verdicts. A verdict therefore evaluates a model together with its adapter, and the adapter is documented alongside each model. We plan a real-data track after version 1 is published. It will keep the weather synthetic while replacing the invented catchment with gauged basins whose attributes are drawn from Caravan ([27]). The repository holds a proposal for 4 CAMELS-US basins ([1]) that span an alpine snowpack, a maritime rain regime, a karst aquifer and a lowland agricultural catchment.

Fifth, the spatial scale and scope remain limited. The probes treat a catchment as 1 lumped control volume, apart from the routing and hydraulics probes on a single reach or a small network. They do not test spatial patterns within a basin, even though semi-distributed structure can improve streamflow prediction ([7]). Momentum is the least developed law, and no probe yet tests the full unsteady momentum balance. The energy results also rest on a small sample of models: only 3 evaluated models are scored on the 2 daily cross-budget probes, and all 3 fail them. The repository documentation notes that daily averaging does not remove the mismatch between the net radiation that these probes supply and the radiation that an adapter computes internally. An open issue tracks this fix, and we regard those archived scores as provisional.

Sixth, a black-box test verifies the internal consistency of the variables a model reports rather than the physical realism of its internal states. A model that reports a synthetic store can close its budget on paper. The paired and counterfactual probes make such a post-hoc adjustment harder to sustain because the reported store must respond physically to perturbations, but they cannot rule it out completely. Seventh, each model enters with 1 version and 1 adapter. Verdicts for stochastic models depend on the random draw unless the adapter fixes it, as the LSTM adapter does by taking the exact median of the predictive distribution. Scores are comparable only within 1 suite version.

Conclusion

A hydrograph score cannot distinguish a model that conserves water from one that loses 15% of it, yet the models that issue operational forecasts are judged mainly by that score. We introduced HydroTuring, a benchmark of 37 physics probes for the conservation of mass, energy and momentum in hydrologic models. First, each probe is a controlled experiment on forcing generated at run time, with binary criteria on 1 physical law. Second, the harness runs the model as a black box and keeps the labels and tolerances on the host. Third, every probe is validated against 77 reference models, including 18 that must pass and 59 that must fail. Across 32 evaluated models, only the 3 exact references pass every probe that they are scored on among the 28 models scored on 5 or more probes. A pre-trained flood-forecasting LSTM fails 6 of the 11 probes it can be scored on, while 49% of all model-probe pairs remain unscored because models do not report the variables that the probes require.

These findings support 3 applications. Model developers can run the suite as a pre-deployment smoke test and expose additional internal states and fluxes. Journal editors can verify physical consistency alongside reported hydrograph metrics, and operational agencies can require benchmark verdicts before adopting a model. Under our open community protocol, authors of merged probes become co-authors of the benchmark. We invite the community to contribute the 13 probes needed to reach the 50-probe target for version 1, to propose additional models, and to help build the upcoming real-catchment evaluation track.

Code and data availability

The harness, the 37 probes, the 77 reference models, the model adapters and the results archive are available at https://github.com/Flood-Lab/HydroTuring, and the project website is https://flood-lab.github.io/HydroTuring/. Suite 0.1.0 [30] is archived on Zenodo (https://doi.org/10.5281/zenodo.23874854), and all versions share the concept DOI https://doi.org/10.5281/zenodo.23874853. The harness, probes, reference models and adapters are released under the Creative Commons Attribution-NonCommercial 4.0 license, and each packaged model keeps its own license, which its manifest names. No dataset is distributed, because every probe generates its forcing from a recorded seed. The verdicts, the measured quantities and the seeds of every run reported here are in the results archive in the repository.

Acknowledgments

We thank the contributors who proposed models, reviewed probes and reported issues in the repository.

Schemas and a worked probe

Model manifest

A model declares what it emits and what it needs in model.yaml. The manifest of the exact bucket illustrates the fields. A model that predicts discharge only declares fluxes: [mrro, dis] and states: [], and it is then N/A (INCOMPLETE) on every probe that needs more.

name: reference_bucket
version: "1.1.0"
license: CC-BY-NC-4.0
runner: subprocess
entrypoint: ["python3", "ht_adapter.py"]
timestep: [PT1D, PT1H, PT15M, PT5M, PT1M]
emits:
  fluxes: [pr, snm, evspsbl, mrro, dis, gwex]
  states: [mrso, snw, canopy, channel]
  diagnostics: [stage]
needs_forcing: [pr, tas, pet]
supports:
  perturbation: true
needs_static: [soil_capacity_mm, canopy_capacity_mm]
uses_static: [area_km2, width_m, cross_section_shape, bed_elevation_m,
              slope, manning_n]

Probe definition

A probe is a directory with a definition file, a generator and a README. The definition of the reference probe, mass/catchment-closure, is shown in abridged form. The baselines block is the declaration that the acceptance gate enforces.

id: mass/catchment-closure
law: mass
requires:
  fluxes: [pr, evspsbl, mrro]
  states: [mrso, snw, canopy]
case:
  generator: generate.py       # forcing is generated from the seed at run time
  n_seeds: 5
  timestep: PT1D
  period_years: 10
  spinup_days: 365
  max_runtime_s: 60
criteria:
  - closure:         {denominator: sum_pr, threshold: 0.05,
                      sinks: [evspsbl, mrro]}
  - state_bounds:    {mrso: [0.0, soil_capacity_mm], snw: [0.0, null],
                      canopy: [0.0, canopy_capacity_mm]}
  - et_plausible:    {max_ratio_to_pet: 1.0, pet_variable: pet}
  - non_degenerate:  {runoff_ratio: [0.02, 0.98], min_flux_cv: 0.1,
                      min_response: 0.05, cv_variables: [mrro, evspsbl]}
  - forcing_fidelity: {variables: [pr], rtol: 1.0e-6}
baselines:
  must_pass: [reference_bucket, flex_lumped, flex_topo, sacsma_snow17]
  must_fail:
    reference_leaky: closure
    reference_cheater: state_bounds
    reference_degenerate: non_degenerate

Lexicon

Table 7 lists the output variables of the lexicon, with their group, units and meaning. The forcing variables and static attributes are defined in the same file of the repository.

NameGroupUnitsPhysical meaning
prfluxmm d−1^{-1}Precipitation, echoed exactly as the forcing gave it
evspsblfluxmm d−1^{-1}Evapotranspiration, including sublimation; net of dew and frost where the model nets them
mrrofluxmm d−1^{-1}Total runoff
NameGroupUnitsPhysical meaning
disfluxm3 s−1m^3\,s^{-1}River discharge
gwexfluxmm d−1\mathrm{mm}\,\mathrm{d}^{-1}Declared exchange with the outside: regional groundwater, inter-basin transfer
gw_sw_exchangefluxmm d−1\mathrm{mm}\,\mathrm{d}^{-1}Net river-aquifer exchange inside the control volume, not added to gwex or counted in closure
gw_to_swfluxmm d−1\mathrm{mm}\,\mathrm{d}^{-1}The aquifer losing to the river, a component of gw_sw_exchange
sw_to_gwfluxmm d−1\mathrm{mm}\,\mathrm{d}^{-1}The river losing to the aquifer, a component of gw_sw_exchange
gw_boundaryfluxmm d−1\mathrm{mm}\,\mathrm{d}^{-1}Every other flux across the aquifer’s own boundary, such as a GHB or a well
sblfluxmm d−1\mathrm{mm}\,\mathrm{d}^{-1}The sublimating share of evspsbl, a component of it rather than an addition
snmfluxmm d−1\mathrm{mm}\,\mathrm{d}^{-1}Liquid delivered from the snow module to the ground, rain passed through included, and not an outflow beside mrro
hflsfluxW m−2\mathrm{W}\,\mathrm{m}^{-2}Latent heat flux
hfssfluxW m−2\mathrm{W}\,\mathrm{m}^{-2}Sensible heat flux
hfgfluxW m−2\mathrm{W}\,\mathrm{m}^{-2}Ground heat flux at the actual soil surface, where a deeper flux is corrected for the heat stored above it
rlusfluxW m−2\mathrm{W}\,\mathrm{m}^{-2}Total upward longwave: emission plus reflected downward longwave
hfg_bottomfluxW m−2\mathrm{W}\,\mathrm{m}^{-2}Heat flux through the bottom of the specified soil layer
mrsostatemmSoil water storage (liquid and ice), reported as an absolute state rather than a tendency
snwstatemmSnow water equivalent, identically zero and named in zero_stores for a model with no snow module
canopystatemmCanopy interception storage, identically zero and named in zero_stores for a model with no such store
gwstatemmGroundwater storage below the soil column
channelstatemmWater generated as runoff but not yet released by the model’s routing
tsdiagnosticKRadiative surface (skin) temperature
tsoil_layerdiagnosticKMean temperature of the specified soil layer
stagediagnosticmWater-surface elevation on the case’s datum, read beside the row’s mean discharge (depth is stage minus bed_elevation_m)
lwsnldiagnosticmmEvery non-solid component of snw, so snw minus lwsnl is the ice (part of snw, not additional)
csnowdiagnosticJ m−2\mathrm{J}\,\mathrm{m}^{-2}The pack’s cold content: the positive energy still needed to bring its ice to 0∘C0^\circ\mathrm{C}
reach_q_inroutingm3 s−1m^3\,s^{-1}Total inflow into the identified reach, in routing.csv
q_outroutingm3 s−1m^3\,s^{-1}Flow across the reach’s downstream boundary, in routing.csv
channel_storageroutingm3m^3Absolute water volume in the reach’s routed channel, in routing.csv
prforcingmm d−1\mathrm{mm}\,\mathrm{d}^{-1}Precipitation
tasforcing∘C^\circ\mathrm{C}Near-surface air temperature in Celsius, unlike the kelvin of ts and tsoil_layer
petforcingmm d−1\mathrm{mm}\,\mathrm{d}^{-1}Potential evapotranspiration
rnforcingW m−2\mathrm{W}\,\mathrm{m}^{-2}Prescribed net radiation, which the model accepts as the energy input to the reported budget when declared
rsdsforcingW m−2\mathrm{W}\,\mathrm{m}^{-2}Downward shortwave radiation

Table 7. Full lexicon of the /io contract (schemas/lexicon.yaml, suite 0.1.0). Over a row of several model steps, a variable is a mean, an end-of-row value, or an instantaneous value, as stated in the repository.

NameGroupUnitsPhysical meaning
rldsforcingW m−2^{-2}Downward longwave radiation
hussforcingkg kg−1^{-1}Near-surface specific humidity
psforcingPaSurface air pressure
sfcWindforcingm s−1^{-1}Near-surface wind speed
abstrforcingmm d−1^{-1}Prescribed human withdrawal, net of return flow
gwhforcingmExternal hydraulic head behind the model’s gwex; declaring it asserts gwex responds monotonically to it
gw_rechargeforcingmm d−1^{-1}Direct recharge to the aquifer
sw_stage_mforcingmRiver stage the aquifer exchanges with
q_inforcingm3^{3} s−1^{-1}Prescribed river or upstream inflow, already a discharge and not multiplied by area or row length
area_km2statickm2^{2}Catchment area the fluxes are converted over
elevation_mstaticmCatchment elevation
latitude_degstaticdegrees northLatitude
main_channel_length_kmstatickmLength of the longest channel
centroid_channel_length_kmstatickmChannel length from the outlet to the point nearest the catchment’s centroid
soil_capacity_mmstaticmmWater the soil column can hold
canopy_capacity_mmstaticmmWater the canopy can intercept (0 is a bare surface)
degree_day_factor_mm_per_C_daystaticmm °C−1^{-1} d−1^{-1}Snowmelt per degree-day
baseflow_coefficientstaticd−1^{-1}Fraction of soil water drained to baseflow per day
snow_threshold_degCstatic°CAir temperature dividing snow from rain
epsstatic1Surface emissivity
soil_layer_depth_mstaticmDepth of the prescribed soil layer, from the surface
soil_heat_capacity_arealstaticJ m−2^{-2} K−1^{-1}Effective areal heat capacity of that layer, already including the depth
soil_temperature_initialstaticKInitial temperature of that layer
soil_solid_heat_capacitystaticJ m−3^{-3} K−1^{-1}Volumetric heat capacity of the soil solids
soil_water_heat_capacitystaticJ m−3^{-3} K−1^{-1}Volumetric heat capacity of soil water
soil_pore_air_heat_capacitystaticJ m−3^{-3} K−1^{-1}Volumetric heat capacity of the air in the pores
soil_porositystatic1Volumetric porosity
soil_water_content_initialstaticm3^{3} m−3^{-3}Initial volumetric water content

Table 1 (PDF p. 29).

NameGroupUnitsPhysical meaning
aquifer_top_mstaticmElevation of the aquifer’s top
aquifer_bottom_mstaticmElevation of the aquifer’s bottom
aquifer_initial_head_mstaticmInitial aquifer head
aquifer_specific_yieldstatic1Drainable porosity above the water table
aquifer_storage_coefficientstatic1Storativity, not specific storage
river_conductance_m2_per_daystaticm2m^2 d−1d^{-1}Conductance between the river and the aquifer
river_bottom_offset_mstaticmRiver stage minus riverbed elevation
routing_networkstatic-Object whose reaches member lists the unique string IDs of the reaches
reach_length_mstaticmLength of the reach
width_mstaticmWidth of the rectangular channel bed
bankfull_depth_mstaticmDepth at which the channel is full
bed_elevation_mstaticmElevation of the rigid bed on the datum stage uses
slopestatic1Bed slope
manning_nstatics m−1/3m^{-1/3}Manning’s roughness
cross_section_shapestatic-Name of the cross-section’s shape, such as rectangular

Table 16.

Probe catalog

Table 8 lists the 37 probes of suite 0.1.0 by law and family. Table 9 gives the authors of every probe as recorded in its definition file. Table 10 gives the proposer of every accepted model.

ProbeQuestion the probe asksScored criteriaSeeds
Mass: closure
catchment-closureCatchment water balance closure under synthetic forcingclosure, et_plausible5
snowpack-mass-closureInternal snowpack water-balance closureclosure, snowpack_response5
extreme-event-closureEvent-wise water balance closure under extreme rainfallclosure, event_water_closure5
ungauged-basin-closureWater balance integrity across declared catchment attribute domainsclosure, et_plausible12
Mass: invariance
ProbeQuestion the probe asksScored criteriaSeeds
resolution-invarianceIntegrated runoff must not depend on the step the weather was given atresolution_invariance3
area-invarianceThe same weather on a 10 times larger catchment gives the same depthsinvariance3
time-origin-invarianceIdentical forcing under aligned calendars gives identical water budgetsinvariance, closure3
groundwater-datum-invarianceGroundwater exchange is invariant to a common vertical datum shiftdatum_flux_invariance, exchange_directions, groundwater_balance, exchange_components3
spinup-cycle-invarianceA periodic climate must forget extra spin-up cyclesclosure, spinup_cycle_invariance3
Mass: response
warming-responseWarmer air over the same rain must mean less runoff, and cooler air moreresponse_sign5
precipitation-counterfactualAdded or removed precipitation must be partitioned among the budget termsclosure, counterfactual_response, monotone_response3
phase-counterfactualRain that would have been snow is still the same waterphase_invariance3
response-nonnegativityMore water can never lower the flow, on any dayresponse_nonnegativity3
antecedent-monotonicityThe same storm on a wetter catchment runs off more, and no more than the extra waterantecedent_monotonicity3
extreme-rainMore rain cannot mean less runoff, nor more runoff than was addedmonotone_response3
Mass: dynamics
causalityA catchment cannot respond to rain that has not fallen yetcausality3
dry-downWithout rain, runoff can only fall, and only stored water can draindry_down3
steady-stateUnder constant weather the catchment settles and the budget balancessteady_state3
runoff-boundsRunoff is bounded by rain minus demand below and by rain aboverunoff_bounds5
multi-decadal-driftSlow budget-compensating storage drift over repeated climateclosure, total_storage_drift, et_plausible5
Mass: groundwater and human
aquifer-recharge-orderingAquifer recharge preserves storage and exchange orderingaquifer_recharge_ordering, groundwater_balance, exchange_components3
exchange-responseA declared head-driven exchange must respond to its external headclosure, exchange_response5
gw-sw-exchange-consistencyGroundwater-river exchange and storage close the water budgetexchange_components, groundwater_balance, exchange_directions5
human-abstractionPrescribed irrigation withdrawal must leave the budgetclosure, human_abstraction3
Energy: single budget
pet-consistencyEvaporation follows demand when the soil is wet and water when it is drydemand_consistency3
surface-energy-closureHourly surface budgets cannot hide canceling energy errorsenergy_closure_by_phase5
soil-heat-storage-consistencySoil heat fluxes and layer temperature must agreesoil_heat_storage5

Table 8. Probe catalog for suite version 0.1.0. Criteria shared by most probes (state_bounds, non_degenerate, forcing_fidelity) are listed only when they are the sole criteria. Seeds are the number of generated cases per probe.

ProbeAuthors
mass/catchment-closureZhi Li
mass/snowpack-mass-closureJinlong Hu
mass/extreme-event-closureTaiqi Lian
mass/ungauged-basin-closureShunan Zhou
mass/resolution-invarianceZhi Li
mass/area-invarianceZhi Li
mass/time-origin-invarianceSiavash Shams
mass/groundwater-datum-invarianceZhenjiang Wu
mass/spinup-cycle-invarianceKaihao Long
mass/warming-responseZhi Li
mass/precipitation-counterfactualQingyi Yang
mass/phase-counterfactualZhi Li
mass/response-nonnegativityZhi Li
mass/antecedent-monotonicityZhi Li
mass/extreme-rainZhi Li
mass/causalityZhi Li
mass/dry-downZhi Li
mass/steady-stateZhi Li
mass/runoff-boundsZhi Li
mass/multi-decadal-driftBing Li
mass/aquifer-recharge-orderingZhenjiang Wu
mass/exchange-responseSongkun Yan

Table 9. Authors of each merged probe, as recorded in probe.yaml.

ModelProposed by
lstmZhi Li (CU Boulder)
dhbv2Zhi Li (CU Boulder)
wflow_sbmYuanhang Liu (Independent Researcher)
summaYuanhang Liu (Independent Researcher)
cwatmYuanhang Liu (Independent Researcher)
lisfloodYuanhang Liu (Independent Researcher)
modflow6Yaji Wang (University of Illinois Urbana-Champaign)
hymodXiaoyi Dong (China IPPR International Engineering Co., Ltd., SINOMACH)
xajXiaoyi Dong (China IPPR International Engineering Co., Ltd., SINOMACH)
gr4jYuanhang Liu (Independent Researcher) and Li Zeng (The Hong Kong University of Science and Technology)
topmodelYan Chen (Colorado State University)
tankJiao Wang (University of California Davis)
topkapiJiao Wang (University of California Davis)
cfeXiaoyi Dong (China IPPR International Engineering Co., Ltd., SINOMACH)
simhydHaoyang Su (Fudan University and Shanghai Innovation Institute)
mcp1Yuan-Heng Wang (University of Arizona)
mcp2Yuan-Heng Wang (University of Arizona)
mcp3Yuan-Heng Wang (University of Arizona)
mcp4Yuan-Heng Wang (University of Arizona)
mcp5Yuan-Heng Wang (University of Arizona)
ravenYan Chen (Colorado State University)
hypeYan Chen (Colorado State University)
prmsYan Chen (Colorado State University)
vicValmik Nahata (University of California San Diego)

Table 10. Accepted model proposals and who proposed them. Five accepted proposals count as one probe toward authorship. The model labeled lstm here is listed in the repository under the identifier given in Appendix § 14.§ 3.

ProbeQuestion the probe asksScored criteriaSeeds
radiation-consistencySurface temperature and upward longwave must describe one surfaceradiative_identity5
Energy: cross budget
latent-heat-et-consistencyLatent heat and evapotranspiration must describe one evaporationclosure, energy_closure, flux_identity, et_plausible5
evaporative-partitionEnergy a drying surface stops evaporating must warm the airpartition_shift, closure, energy_closure, flux_identity3
snowmelt-energy-waterMelt reported as water must be paid for out of the energy budgetclosure, melt_energy5
Momentum: routing
routing-conservationA routing store holds runoff for as long as its hydrograph and no longerrouting_conservation3
routing-lag-consistencyRunoff peak lag follows the catchment geomorphic travel-time scalelag_time_bounds, scaling_monotonicity3
Momentum: hydraulics
stage-discharge-monotonicA stage reading rises with discharge, and loops the way a flood wave doesrating_monotonic, rating_loop3
uniform-flow-friction-consistencySteady discharge and stage satisfy the declared open-channel friction balanceuniform_flow_friction5
froude-regimeA mild-sloped reach cannot outrun its own wavefroude_subcritical3
wave-celerity-boundsTransient flood-wave celerity responds to hydraulic statewave_celerity_bounds3

Table 17.

ProbeAuthors
mass/gw-sw-exchange-consistencyYaji Wang
mass/human-abstractionYuanhang Liu
energy/pet-consistencyZhi Li
energy/surface-energy-closureHan Wang
energy/soil-heat-storage-consistencyHan Wang
energy/radiation-consistencyXin Lan
energy/latent-heat-et-consistencyChangming Li
energy/evaporative-partitionChangming Li
energy/snowmelt-energy-waterSiddik Barbhuiya
momentum/routing-conservationZhi Li, Yuanhang Liu
momentum/routing-lag-consistencyBinlan Zhang
momentum/stage-discharge-monotonicYuanhang Liu
momentum/uniform-flow-friction-consistencyMofan Zhang
momentum/froude-regimeYuanhang Liu
momentum/wave-celerity-boundsJingzhi Chen

Table 18.

ModelProposed by
mhmValmik Nahata (University of California San Diego)
shudYuanhang Liu (Independent Researcher)
noahmpValmik Nahata (University of California San Diego)
rvicJiao Wang (University of California Davis)
mizurouteYuanhang Liu (Independent Researcher)

Table 19.

Reference models

Table 11 lists the 18 exact references and the number of probes whose gate requires each to pass. Table 12 lists the 59 broken references, the defect built into each, and the probe and criterion that must catch it.

Table 11. Exact reference models and the number of probes whose gate requires them to pass. The first four are the common baselines. The others are written for the probes whose physics they exercise.

Table 12. Deliberately broken reference models (with the reference_ prefix omitted in column 1) and the criterion each is built to trip. Every row is a must-fail check in the acceptance gate. The gate requires each model to fail and to trip its declared criterion. It requires a model to trip no other criterion only in 2 cases, snow_bypass and fixed_celerity, where the control exists to separate 1 failure mode from another.

Reference modelDefect built inProbeCriterion
restlessThe reference bucket with a recession that keeps its own 30-day clock, whatever the weather.spinup-cycle-invariance, steady-statespinup_cycle_invariance, steady_state
saturatingThe reference bucket with its daily runoff capped at 25 mm: exact for ordinary storms, flat for extreme ones.extreme-rainmonotone_response
shallow_ratingThe exact bucket with a rating borrowed from another reach: its gauge reads Manning normal depth drawn for a section 5 times wider than the one the case declares, so the depth it reports is too shallow for this channel.froude-regimefroude_subcritical
slow_driftAn exact bucket whose reporter subtracts 0.012 mm d−1^{-1} from runoff and accumulates the deficit in soil storage.multi-decadal-driftstate_bounds
snow_bypassDeliberately broken snow model.snowpack-mass-closureclosure
snowlessDeliberately degenerate snow model.snowpack-mass-closuresnowpack_response
spatial_capacityAttribute-dependent fixed internal-capacity fault.ungauged-basin-closurestate_bounds
spatial_etCausal attribute-dependent ET fault.ungauged-basin-closureet_plausible
spatial_forcingCausal attribute-dependent forcing fault.ungauged-basin-closureforcing_fidelity
spatial_frozenCausal attribute-dependent frozen fault.ungauged-basin-closurenon_degenerate
spatial_gainCausal attribute-dependent gain fault.ungauged-basin-closureclosure
spatial_lossCausal attribute-dependent loss fault.ungauged-basin-closureclosure
spatial_negativeCausal attribute-dependent negative fault.ungauged-basin-closurenon_degenerate
steady_sinkreference_token_exchange with its noise removed: evaporation overstated by a constant 0.3 mm d−1^{-1} (13% of precipitation over the record), declared as a steady regional inflow so the budget closes, with a head response 10610^{6} times weaker than the positive controls.exchange-responseexchange_response
streamflow_onlyPredicts discharge and nothing else, like most published rainfall-runoff models.warming-responseresponse_sign
stuck_routerThe bucket with a unit hydrograph whose weights sum to 0.9: 10% of every day’s runoff enters the channel store and never leaves.routing-conservationrouting_conservation
sublimatingThe bucket losing 40% of every snowfall to unreported sublimation.phase-counterfactualphase_invariance
sublimation_blindFully coupled with a temperature-dependent latent heat, but it converts snow sublimation at the latent heat of vaporization instead of sublimation.latent-heat-et-consistencyflux_identity
thirstyThe bucket with evaporation that ignores demand: a fixed 2% of the soil store leaves every day, capped only by available water.pet-consistencydemand_consistency
token_exchangereference_noise_sink with 1 line added: a head-driven trickle 10610^{6} times weaker than the positive controls, folded into the declared exchange and soil storage so the budget closes.exchange-responseexchange_response

Table 37.

Reference modelDefect built inProbeCriterion
two_headA model with decoupled water and energy heads that close their individual budgets without agreeing on latent heat.evaporative-partition, latent-heat-et-consistencyflux_identity, partition_shift
warming_freeThe same snowpack, melting on the energy actually available and reporting its cold content honestly, but charging its surface budget for the fusion alone.snowmelt-energy-watermelt_energy
wrong_roughnessDeliberately inconsistent steady reach.uniform-flow-friction-consistencyuniform_flow_friction
wrong_slopeDeliberately inconsistent steady reach.uniform-flow-friction-consistencyuniform_flow_friction
zero_exchangeIntentionally frozen zero-exchange reference.groundwater-datum-invariancedatum_flux_invariance

Table 20.

Tolerance and seed sensitivity

Table 13 varies the closure tolerance over 4 orders of magnitude and counts the evaluated models that would fail. The residual of the worst model, xaj, is 2.5%, and the verdict on the broken reference is unchanged for every tolerance below 15%. Table 14 shows the seed sensitivity that motivates running several seeds. The 20 seeds for catchment-closure and the 10 seeds for extreme-rain were drawn at random from a recorded generator, and the seed lists are included with the source of this article.

Tolerance (% of precipitation)Evaluated models failing (of 22)reference_leaky
0.014fails
0.12fails
0.51fails
11fails
21fails
5 (adopted)0fails
100fails
200passes

Table 13. Sensitivity of the closure verdicts to the tolerance. Entries are the number of the 22 evaluated models whose whole-record residual on mass/catchment-closure exceeds each limit. The broken reference reference_leaky has a residual of 15% by construction.

Evaluated models and reproducibility

Adapting a model

The repository documents 6 steps for adding a model. First, the proposer opens a model proposal and the maintainer accepts it. Second, the proposer decides which variables the model honestly emits. Third, an adapter translates between the contract and the native model. Fourth, a Dockerfile or a native entrypoint makes the run reproducible. Fifth, a contract check, ht verify-adapter, runs 1 seed on the evaluation window before any physics is scored. Sixth, the contributor runs the suite with ht run and submits the manifest, the adapter and the archived results. The common failure modes are padding the list of emitted variables, which turns an honest limit into a false claim, and declaring a forcing input that the adapter does not consume.

Standing of the evaluated models

Table 15 lists the probes passed over probes scored for every model, split by law.

Reproducibility

All numbers in this article derive from suite 0.1.0, archived on Zenodo, and from the results archive in the repository. The acceptance gate of every probe was re-run on 2 November 2026 with the command ht gate, and the standings of Table 15 reproduce from the archive. The models are listed in the archive under their repository identifiers. The LSTM of this article appears there as google_flood_forecast, version 0.1.0-828dfc5.2, with released weights, and it was run in a Docker container on 8 CPU cores. The mass-conserving perceptron checkpoints mcp1 to mcp5 correspond to epochs 27, 3, 619, 2707 and 2999 of 1 architecture. Versions of all other models are recorded in the archive.

References

  1. [1]Nans Addor, Andrew J. Newman, Naoki Mizukami, and Martyn P. Clark. The camels data set: catchment attributes and meteorology for large-sample studies. Hydrology and Earth System Sciences, 21(10): 5293–5313, 2017. doi: 10.5194/hess-21-5293-2017.DOI
  2. [2]ASCE Task Committee on Application of Artificial Neural Networks in Hydrology. Artificial neural networks in hydrology. I: Preliminary concepts. Journal of Hydrologic Engineering, 5(2):115–123, 2000.
  3. [3]Richard Bellman. Dynamic Programming. Princeton University Press, Princeton, NJ, 1957.
  4. [4]Tom Beucler, Michael Pritchard, Stephan Rasp, Jordan Ott, Pierre Baldi, and Pierre Gentine. Enforcing analytic constraints in neural networks emulating physical systems. Physical Review Letters, 126(9), 2021. doi: 10.1103/PhysRevLett.126.098302.DOI
  5. [5]K. J. Beven and M. J. Kirkby. A physically based, variable contributing area model of basin hydrology. Hydrological Sciences Bulletin, 24(1):43–69, 1979. doi: 10.1080/02626667909491834.
  6. [6]Keith Beven. A manifesto for the equifinality thesis. Journal of Hydrology, 320(1-2):18–36, 2006. doi: 10.1016/j.jhydrol.2005.07.007.DOI
  7. [7]Douglas P. Boyle, Hoshin V. Gupta, Soroosh Sorooshian, Victor Koren, Ziya Zhang, and Michael Smith. Toward improved streamflow forecasts: value of semidistributed modeling. Water Resources Research, 37(11):2749–2759, 2001. doi: 10.1029/2000WR000207.DOI
  8. [8]Leo Breiman. Random forests. Machine Learning, 45(1):5–32, 2001.
  9. [9]Peter Burek, Yusuke Satoh, Taher Kahil, Ting Tang, Peter Greve, Mikhail Smilovic, Luca Guillaumot, Fang Zhao, and Yoshihide Wada. Development of the community water model (cwatm v1.04) – a high-resolution hydrological model for global and regional assessment of integrated water resources management. Geoscientific Model Development, 13(7):3267–3298, 2020. doi: 10.5194/gmd-13-3267-2020.DOI
  10. [10]L. Ciarapica and E. Todini. Topkapi: a model for the representation of the rainfall-runoff process at different scales. Hydrological Processes, 16(2):207–229, 2002. doi: 10.1002/hyp.342.DOI
  11. [11]Martyn P. Clark, Dmitri Kavetski, and Fabrizio Fenicia. Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resources Research, 47(9), 2011. doi: 10.1029/2010WR009827.DOI
  12. [12]Martyn P. Clark, Bart Nijssen, Jessica D. Lundquist, Dmitri Kavetski, David E. Rupp, Ross A. Woods, Jim E. Freer, Ethan D. Gutmann, Andrew W. Wood, Levi D. Brekke, Jeffrey R. Arnold, David J. Gochis, and Roy M. Rasmussen. A unified approach for process-based hydrologic modeling: 1. modeling concept. Water Resources Research, 51(4):2498–2514, 2015. doi: 10.1002/2015WR017198.DOI
  13. [13]Martyn P. Clark, Richard M. Vogel, Jonathan R. Lamontagne, Naoki Mizukami, Wouter J. M. Knoben, Guoqiang Tang, Shervan Gharari, Jim E. Freer, Paul H. Whitfield, Kevin R. Shook, and Simon Michael Papalexiou. The abuse of popular performance metrics in hydrologic modeling. Water Resources Research, 57(9), 2021. doi: 10.1029/2020WR029001.DOI
  14. [14]James R. Craig, Genevieve Brown, Robert Chlumsky, R. Wayne Jenkinson, Georg Jost, Konhee Lee, Juliane Mai, Martin Serrier, Nicholas Sgro, Mahyar Shafii, Andrew P. Snowdon, and Bryan A. Tolson. Flexible watershed simulation with the raven hydrological modelling framework. Environmental Modelling & Software, 129:104728, 2020. doi: 10.1016/j.envsoft.2020.104728.DOI
  15. [15]Veronika Eyring, Sandrine Bony, Gerald A. Meehl, Catherine A. Senior, Bjorn Stevens, Ronald J. Stouffer, and Karl E. Taylor. Overview of the coupled model intercomparison project phase 6 (cmip6) experimental design and organization. Geoscientific Model Development, 9(5):1937–1958, 2016. doi: 10.5194/gmd-9-1937-2016.DOI
  16. [16]Dapeng Feng, Jiangtao Liu, Kathryn Lawson, and Chaopeng Shen. Differentiable, learnable, regionalized process-based models with multiphysical outputs can approach state-of-the-art hydrologic prediction accuracy. Water Resources Research, 58(10), 2022. doi: 10.1029/2022WR032404.DOI
  17. [17]Jonathan M. Frame, Frederik Kratzert, Daniel Klotz, Martin Gauch, Guy Shalev, Oren Gilon, Logan M. Qualls, Hoshin V. Gupta, and Grey S. Nearing. Deep learning rainfall–runoff predictions of extreme events. Hydrology and Earth System Sciences, 26(13):3377–3392, 2022. doi: 10.5194/hess-26-3377-2022.DOI
  18. [18]Hoshin V. Gupta, Harald Kling, Koray K. Yilmaz, and Guillermo F. Martinez. Decomposition of the mean squared error and nse performance criteria: Implications for improving hydrological modelling. Journal of Hydrology, 377(1-2):80–91, 2009. doi: 10.1016/j.jhydrol.2009.08.003.
  19. [19]Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. doi: 10.1162/neco.1997.9.8.1735.DOI
  20. [20]Pieter-Jan Hoedt, Frederik Kratzert, Daniel Klotz, Christina Halmich, Markus Holzleitner, Grey Nearing, Sepp Hochreiter, and Günter Klambauer. MC-LSTM: Mass-conserving LSTM, 2021.arxiv.org/abs/2101.05186
  21. [21]Kuo-In Hsu, Hoshin V. Gupta, and Soroosh Sorooshian. Artificial neural network modeling of the rainfall–runoff process. Water Resources Research, 31(10):2517–2530, 1995.DOI
  22. [22]George Em Karniadakis, Ioannis G. Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021. doi: 10.1038/s42254-021-00314-5.DOI
  23. [23]Anuj Karpatne, Gowtham Atluri, James H. Faghmous, Michael Steinbach, Arindam Banerjee, Auroop Ganguly, Shashi Shekhar, Nagiza Samatova, and Vipin Kumar. Theory-guided data science: A new paradigm for scientific discovery from data. IEEE Transactions on Knowledge and Data Engineering, 29(10):2318–2331, 2017. doi: 10.1109/TKDE.2017.2720168.DOI
  24. [24]James W. Kirchner. Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology. Water Resources Research, 42(3), 2006. doi: 10.1029/2005WR004362.DOI
  25. [25]Frederik Kratzert, Daniel Klotz, Claire Brenner, Karsten Schulz, and Mathew Herrnegger. Rainfall–runoff modelling using long short-term memory (lstm) networks. Hydrology and Earth System Sciences, 22(11):6005–6022, 2018. doi: 10.5194/hess-22-6005-2018.DOI
  26. [26]Frederik Kratzert, Daniel Klotz, Guy Shalev, Günter Klambauer, Sepp Hochreiter, and Grey Nearing. Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets. Hydrology and Earth System Sciences, 23(12):5089–5110, 2019. doi: 10.5194/hess-23-5089-2019.
  27. [27]Frederik Kratzert, Grey Nearing, Nans Addor, Tyler Erickson, Martin Gauch, Oren Gilon, Lukas Gudmundsson, Avinatan Hassidim, Daniel Klotz, Sella Nevo, Guy Shalev, and Yossi Matias. Caravan - a global community dataset for large-sample hydrology. Scientific Data, 10(1), 2023. doi: 10.1038/s41597-023-01975-w.
  28. [28]Christian D. Langevin, Joseph D. Hughes, Edward R. Banta, Richard G. Niswonger, Sorab Panday, and Alden M. Provost. Documentation for the modflow 6 groundwater flow model. Techniques and methods, book 6, chap. a55, U.S. Geological Survey, 2017.DOI
  29. [29]Zhi Li. HydrologicModels: lumped and semi-distributed conceptual models (FLEX variants). https://github.com/chrimerss/HydrologicModels, 2026.
  30. [30]Zhi Li, Changming Li, Siavash Shams, Han Wang, Qingyi Yang, Yuanhang Liu, Taiqi Lian, Xin Lan, Shunan Zhou, Binlan Zhang, Siddik Barbhuiya, Songkun Yan, Bing Li, Jinlong Hu, Yaji Wang, Zhenjiang Wu, Mofan Zhang, Kaihao Long, and Jingzhi Chen. HydroTuring: a conservation-law benchmark for AI hydrologic models, suite version 0.1.0. https://github.com/Flood-Lab/HydroTuring, 2026a. Published 2 November 2026. Archived at Zenodo; concept DOI 10.5281/zenodo.23874853.
  31. [31]Zhi Li, Songkun Yan, and Yang Hong. Do large language models improve hydrologic model calibration? Journal of Hydrology X, 31:100234, 2026b. doi: 10.1016/j.hydroa.2026.100234.DOI
  32. [32]Xu Liang, Dennis P. Lettenmaier, Eric F. Wood, and Stephen J. Burges. A simple hydrologically based model of land surface water and energy fluxes for general circulation models. Journal of Geophysical Research: Atmospheres, 99(D7):14415–14428, 1994. doi: 10.1029/94JD00483.
  33. [33]Göran Lindström, Charlotta Pers, Jörgen Rosberg, Johan Strömqvist, and Berit Arheimer. Development and testing of the hype (hydrological predictions for the environment) water quality model for different spatial scales. Hydrology Research, 41(3-4):295–319, 2010. doi: 10.2166/nh.2010.007.
  34. [34]D. Lohmann, E. Raschke, B. Nijssen, and D. P. Lettenmaier. Regional scale hydrology: I. formulation of the vic-2l model coupled to a routing model. Hydrological Sciences Journal, 43(1):131–141, 1998. doi: 10.1080/02626669809492107.

Paper details

vlnc:W_w24dgv9f
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents