In this paper
AbstractIntroductionBackground and related workAI-based hydrologic modelsPhysics in machine learningEvaluation practice in hydrologyBenchmarks in hydrology and AI for scienceBehavioral testingMethodsDesign principlesThe three laws and the probe catalogMassEnergyMomentumCoverage and gapsHarness and protocolArchitectureLexicon and input-output contractVerdictsCriteria on the reference probeReference models and probe validationReference modelsAcceptance gateResultsVerdict matrixWhere models failCase study: an LSTMDifferentiable and mass-conserving modelsCommunity protocol and governanceContributing a probe or a modelReview and decisionsGrowth and roadmapDiscussion and limitationsWhat the verdicts showUse before deployment and during trainingLimitationsConclusionCode and data availabilityAcknowledgmentsSchemas and a worked probeModel manifestProbe definitionLexiconProbe catalogReference modelsTolerance and seed sensitivityEvaluated models and reproducibilityAdapting a modelStanding of the evaluated modelsReproducibilityReferencesWorksPhysics and Astronomy
HydroTuring: A physics probe benchmark testing mass, energy, and momentum conservation in hydrologic models
Abstract
Hydrologic models, including deep learning systems now deployed for operational flood forecasting, are evaluated primarily by how closely their simulated hydrographs match observed streamflow. However, high hydrograph skill does not guarantee physical consistency: in a synthetic calibration experiment, a conceptual model that loses 15% of its water input to an unreported sink still reaches a Nash-Sutcliffe efficiency of 0.93, and no shared benchmark tests whether models obey fundamental conservation laws. Here we introduce HydroTuring (v0.1.0), a black-box benchmark of 37 physics probes that test mass (24 probes), energy (7) and momentum (6) conservation in hydrologic models. Each probe runs a controlled experiment on forcing generated at run time from a recorded seed and applies binary pass or fail criteria validated against 18 physically exact and 59 deliberately broken reference models. Across 32 evaluated models (3 exact references, 7 AI-based models and 22 process-based models), only the 3 exact references pass every probe among the 28 models scored on 5 or more probes. Process-based and AI-based models pass similar shares of their scored probes (76% and 74%), and a pre-trained flood-forecasting LSTM fails 6 of the 11 runoff-only probes it can be scored on. Furthermore, 49% of all 1,184 model-probe pairs remain unscored because models omit the required state or flux variables, while the scored failures occur in perturbation responses and state bounds rather than long-term water-budget closure. HydroTuring provides a reproducible smoke test for model development, peer review and operational deployment under an open community protocol.
Website flood-lab.github.io/HydroTuring | GitHub github.com/Flood-Lab/HydroTuring | Zenodo doi:10.5281/zenodo.23874854
MAINTAINER
PROBE AUTHORS, IN ORDER OF FIRST MERGED PROBE
MODEL CONTRIBUTOR
¹Flood Lab, Department of Civil, Environmental and Architectural Engineering, and Institute of Arctic and Alpine Research, University of Colorado Boulder, Boulder, CO, USA
²South China University of Technology, China
³Department of Electrical Engineering, Columbia University, New York, NY, USA
⁴Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Hong Kong, China
⁵Department of Civil and Environmental Engineering (D.I.C.A.), Politecnico di Milano, Milano 20133, Italy
⁶Independent researcher
⁷Laboratory of Catchment Hydrology and Geomorphology, École Polytechnique Fédérale de Lausanne (EPFL), 1951 Sion, Switzerland
⁸Center for Systems Integration and Sustainability, Department of Fisheries and Wildlife, Michigan State University, East Lansing, MI 48823, USA
⁹Dalian University of Technology, Dalian, China
¹⁰Institute of Mountain Hazards and Environment, Chinese Academy of Sciences, Chengdu, China
¹¹Indian Institute of Technology Mandi, Mandi, Himachal Pradesh, India
¹²School of Civil Engineering and Environmental Science, University of Oklahoma, Norman, OK, USA
¹³State Key Laboratory of Earth Surface Processes and Disaster Risk Reduction, Faculty of Geographical Science, Beijing Normal University, Beijing, China
¹⁴University of Illinois Urbana-Champaign, Urbana, IL, USA
¹⁵Institute of Global Environmental Change, Xi’an Jiaotong University, Xi’an 710049, China
¹⁶Department of Civil and Environmental Engineering, Stanford University, Stanford, CA, USA
¹⁷School of Geography and Planning, Sun Yat-sen University, Guangzhou 510006, China
¹⁸Department of Computer Science and Engineering, State University of New York at Buffalo, Buffalo, NY, USA
¹⁹University of Arizona, Tucson, AZ, USA
*Corresponding author: Zhi Li, [email protected].
Keywords: hydrologic modeling, benchmark, machine learning, conservation laws, water balance, physics-informed learning
Introduction
Machine learning has changed hydrologic modeling. Deep networks trained on large-sample data now reproduce runoff with skill that matches or exceeds process-based models [25, 26] (Nearing et al. 2021). These models are not constrained by the laws of mass, energy and momentum conservation [22] (Reichstein et al. 2019). Their predictions can become unphysical under out-of-sample conditions such as extremely high or low streamflow [17]. Even in a simple conceptual model, a severe conservation failure can hide behind a high hydrograph score.
In a synthetic calibration experiment, we fitted 2 conceptual models to the same daily runoff record. Model A (reference_bucket) conserves water, while model B (reference_leaky) sends 15% of its water input to a sink that its outputs never report. Against the observed streamflow, their Nash-Sutcliffe efficiencies (NSE) are 0.97 and 0.93. Both exceed the 0.75 threshold that a widely cited guideline rates as very good (Moriasi et al., 2007). A hydrograph score cannot tell the 2 apart, although model B loses 15% of every year’s precipitation. This is the long-standing problem of getting the right answer for the wrong reason [24], and it is a case of equifinality [6].
The problem grows with the complexity of the model. A hydrologic system is driven by precipitation, temperature and evaporative demand, and it carries soil, snow and groundwater states. With 6 such dimensions and 10 bins per dimension, a 40-year daily record of 14,610 samples can visit at most 1.5% of the distinct regimes Figure 1a. This is the curse of dimensionality [3]. A model fitted to streamflow alone is therefore constrained on a thin slice of the space it will later be asked to cover. Relying on historical streamflow alone exposes a model to 3 failure modes. First, climate is not stationary (Milly et al., 2008), so a response fitted to the record drifts away from the physical response as conditions leave it Figure 1b. Second, the largest events are rare and carry little weight in squared-error scores [18, 13], so a fitted model can flatten above the largest flow it has seen Figure 1c. Third, a model that gains or loses water at each step accumulates the imbalance in its stores, and the error grows over a long simulation without showing in the streamflow [20]. These limits are not unique to AI models, and process-based models can also fail due to imperfect mass closure. However, deep learning rainfall-runoff models tend to degrade more than well-calibrated process-based models on events beyond their training record [17]. HydroTuring therefore targets hydrologic models as a whole, whether learned, physical or hybrid, because a physical model can also fail a conservation test.
Figure 1. Why a streamflow fit does not establish physics. a, Upper bound on the fraction of distinct regimes that a record can visit, for 14,610 daily samples and 10 bins per dimension. The count is arithmetic, not data. b, c, Schematics of 2 failure modes of a pure streamflow fit. A fit trained on a record departs from a drifting physical response (b), and it stays flat above the largest event in the record (c). Panels b and c are illustrations and carry no data.
Current evaluation standards rely on NSE, the Kling-Gupta efficiency (KGE) and related scores on observed basins [18] (Nash and Sutcliffe 1970). Large-sample datasets made such comparisons routine [1, 27] (Newman et al. 2015). These scores do not test conservation, the response to a perturbation, or behavior outside the record. Physics-informed learning builds constraints into training [22, 20, 4], but no shared test checks whether the trained model honors them. Benchmarks in AI for science report accuracy against public data (Rasp et al., 2024; Takamoto et al., 2022). To our knowledge, no benchmark in hydrology or in Earth system science tests conservation laws on hidden, generated cases. HydroTuring is built to fill this gap.
We propose to treat physics as a test suite. The question changes from how close a model comes to 1 record to what the model must do under any record. The name follows Turing’s imitation game (Turing, 1950). That game judges a machine by its behavior under questions it was not prepared for. HydroTuring asks whether a model behaves like the physical system under questions that no training record answered. We envision that new AI-based and physics-based models will be tested against HydroTuring as a smoke test before they are deployed.
Here we introduce HydroTuring v0.1.0, a probe benchmark for the conservation of mass, energy and momentum in hydrologic models. We define a probe as a controlled experiment with pass or fail criteria on 1 physical law. Our objectives are (1) to define a probe-based benchmark for 3 conservation laws, (2) to run it through a black-box harness that hides the forcing and the tolerances from the model, and (3) to validate every probe against reference models that must pass or must fail. We then report a catalog of 37 probes with the verdicts of 32 models, and we describe an open community protocol that credits probe authors as co-authors. The paper is organized as follows. § 2 reviews related work. § 3 presents the methods: the design principles (§ 3.1), the 3 laws and the probe catalog (§ 3.2), the harness and its protocol (§ 3.3), and the validation of probes against reference models (§ 3.4). § 4 reports the verdicts, and § 5 presents the community protocol. § 6 and § 7 discuss limitations and conclude. We hope that this release attracts more contributors and users from the community.
Background and related work
AI-based hydrologic models
Data-driven rainfall-runoff modeling has been studied for decades. Artificial neural networks were applied to rainfall-runoff and streamflow forecasting in the 1990s [21, 2] (Maier and Dandy 2000). Decision and model trees followed (Solomatine and Dulal, 2003), and then random forests and related ensembles [8] (Shortridge et al. 2016). These models were fitted to 1 basin at a time and used as time-series predictors. Long short-term memory networks [19] carry memory of past states and predict streamflow with high skill when they are trained on many catchments at once [25, 26]. The same approach now supports forecasts of extreme floods in ungauged watersheds at global scale (Nearing et al., 2024). Differentiable models take a different route. They embed a process-based core in a trainable pipeline, so that the physical stores and fluxes remain explicit [16] (Song et al. 2025). Large language models have also entered the workflow, for example to assist the calibration of process-based models [31]. In practice, all of these approaches are evaluated mainly by hydrograph skill on observed basins. The role of hydrologic science in the age of machine learning is an open question (Nearing et al., 2021). A review of data-driven Earth system science argued for hybrid approaches that combine learning with process understanding, and named physical consistency and extrapolation as open challenges (Reichstein et al., 2019).
Physics in machine learning
Physics-based (physics-informed or physics-guided) AI has become prevalent in recent years. Physics-informed neural networks (PINN) add the governing equations to the training loss [22] (Raissi et al. 2019). Graph neural networks can embed the shallow water equations for flood prediction, as in HydroGraphNet (Taghizadeh et al., 2025). Hard-constrained networks enforce analytic constraints by construction [4]. Mass-conserving architectures route water through explicit stores, either inside a recurrent cell [20] or as a perceptron with gates (Wang and Gupta, 2024). Theory-guided data science frames the broader aim [23]. These methods build physics into training. HydroTuring tests it after training, on the model as delivered, without access to its weights. The 2 are complementary. Most of these methods target mass, while HydroTuring probes energy and momentum conservation as well. A mass-conserving architecture should pass the closure probes by construction, and the energy and momentum probes show what construction alone does not guarantee (§ 4).
Evaluation practice in hydrology
The Nash-Sutcliffe efficiency ((Nash and Sutcliffe, 1970)) and the Kling-Gupta efficiency ([18]) are the standard scores. Both aggregate errors over space and time. Neither checks the physics inside the model, so a model can give the right answer for the wrong reason ([24]), and models with different internal budgets can earn the same score ([6]). A score earned on 1 record also says little about transferability to other basins, periods and climates. Guidelines rate model performance by threshold values of these scores ((Moriasi et al., 2007)), and the common metrics are open to misuse when their assumptions are ignored ([13]). Closure of the mass, energy and momentum balances is traditionally a self-check by the model developer. It is rarely reported in a comparable form, and it stays hidden when the model is not open source. The method of multiple working hypotheses aims to expose such structural differences ([11]). The central concern is whether a model gives the right answer for the right reasons. HydroTuring tests the reasons directly, and it needs no access to the source. It asks what the model does when 1 input changes and everything else is held fixed.
Benchmarks in hydrology and AI for science
Large-sample datasets such as CAMELS ([1] (Newman et al. 2015)) and Caravan ([27]) made reproducible comparisons across hundreds of basins routine. WeatherBench 2 does the same for global weather forecasts ((Rasp et al., 2024)). PDEBench provides simulated datasets for scientific machine learning and reports, among its metrics, the deviation of predictions from conserved quantities ((Takamoto et al., 2022)). Agents’ Last Exam is a living benchmark whose task pool grows as new workflows are onboarded, with verifiable outcomes ((Sun et al., 2026)). The community model intercomparison projects, of which CMIP is the best known, show how a shared protocol can be extended by many groups ([15]). Table 1 compares these efforts with HydroTuring on 4 properties. To our knowledge, none of these efforts tests physical laws with generated cases and hidden tolerances.
Behavioral testing
Behavioral testing evaluates a model through the outputs it gives to controlled inputs, without access to its internals. CheckList applies this idea to NLP models, with templates that generate invariance and directional expectation tests ((Ribeiro et al., 2020)). HydroTuring applies the same idea to physical laws. A probe changes one input, such as air temperature, storm depth or the time step, and states what the output must do. Warmer air over the same rain must lower runoff, and the same water delivered at a finer time step must give the same volume.
| Benchmark | Quantity evaluated | Test cases | Physical laws tested |
| CAMELS, Caravan ([1, 27] (Newman et al. 2015)) | Streamflow skill in observed basins | Public observed records | Not tested |
| WeatherBench 2 ((Rasp et al., 2024)) | Forecast skill against reanalysis and observations | Public data | Not tested as laws |
| PDEBench ((Takamoto et al., 2022)) | Accuracy of learned PDE solvers | Public simulated datasets | Partly, through the error of conserved quantities |
| Agents’ Last Exam ((Sun et al., 2026)) | Agent performance on professional workflows | Tasks with verifiable outcomes, extended continuously | Not applicable |
| CheckList ((Ribeiro et al., 2020)) | Behavior of NLP models | Test templates | Not applicable |
| HydroTuring | Whether a model conserves mass, energy and momentum | Generated at run time from a seed, with hidden tolerances | Mass, energy and momentum, one law per probe |
Table 1. Table 1. Comparison of HydroTuring with related benchmarks. Entries describe what each effort reports in its primary publication. “Generated” means that test cases are produced at run time from a seed, so that no fixed test set exists.
Methods
Design principles
The design of HydroTuring follows 7 principles. Each sets a standard for how a model should be tested to ensure it respects the underlying physics.
(1) Physical Probes. A probe asks 1 physical question and returns a verdict. The question is stated before the test is built, for example whether the same weather delivered at a finer time step gives the same runoff volume over time. A probe returns PASS, FAIL or N/A, and each criterion inside it keeps its own number. The N/A is applied when a particular model cannot be tested for that criterion. For example, if a model does not support sub-daily inputs, the resolution_invariance criterion would be marked N/A. There is no weighted aggregate of probes. A score of 0.8 on a composite cannot say which law a model violates, while a failed resolution_invariance criterion identifies both the violated law and the error magnitude.
(2) Hidden contract. A model is shipped as a Docker container with a manifest and a thin adapter, in any language, and it runs without network access. The harness writes a request, the forcing and the static attributes to a read-only directory, runs the entrypoint once, and reads the outputs from a result file. The model never sees the probe code or the tolerances. This keeps the benchmark open to learned models, process-based models and hybrids on equal terms, and it removes any need to share weights or source.
(3) Hidden forcing. A model is tested on cases it cannot have seen. Each probe generates its forcing at run time from a recorded seed. No dataset is committed to the repository, so there is nothing to memorize or tune against. Each probe runs several seeds, and all must pass. Of the 37 probes, 22 use 3 seeds, 14 use 5, and 1 uses 12. Seeds are recorded, so that any failure can be reproduced. A private suite can be added from outside the repository without changing the public one, which allows an operator to keep further probes undisclosed. (4) Paired cases. Many physical laws constrain how a system responds to a perturbation rather than a single state value. Warmer air over the same rain must lower runoff, and a storm added on a given day cannot change anything before that day. A probe therefore runs a control and 1 perturbed copy of the same case, which differ in 1 input, and it compares the 2 runs. This checks response and invariance, which an absolute closure test cannot reach. A model that ignores an input passes every single-run criterion, and a paired probe is the only test that catches it. Of the 37 probes, 18 compare 2 or more runs of the same case.
(5) Labeled regimes. Generators label the cases they produce. A label column, such as the stretch of a snow season or the days of an isolated storm, is read by the harness and never passed to the model. Criteria use the labels to score a regime separately, so that errors of opposite sign in different regimes cannot cancel, and the report can trace a failure to the regime that produced it.
(6) Strict aggregate. A model passes only when every criterion of every probe that applies to it passes. A single violated law is enough to fail. We adopt this strict rule because physical conservation is binary under a defined tolerance. A model that leaks water in 1 regime fails to conserve water, even if it closes the budget in all others.
(7) Validated probes. A probe is only as good as its ability to separate right from wrong. Every probe must therefore pass an acceptance gate before it is merged. The gate runs exact reference models, which must pass, and deliberately broken reference models, which must fail and must trip the criterion that each was built to trip. If a probe fails an exact model, its tolerance has drifted beyond what exact physics achieves, and the probe is wrong. If a probe passes a broken model, it does not discriminate. We return to the gate in § 3.4.
The three laws and the probe catalog
HydroTuring version 0.1.0 holds 37 probes, merged between 3 September and 16 October 2026: 24 for mass, 7 for energy and 6 for momentum. The probes use 44 distinct criterion types. We group them into 9 families (Figure 2 and Table 2). Section B lists every probe with its question, its criteria and its authors.
Figure 2. Map of the 37 merged probes by law and family. Each entry is 1 probe in suite version 0.1.0. The cross-budget family needs outputs from both the water and the energy ledgers.
| Law | Family | Probes | Representative experiment | Representative criterion and limit |
| Mass | Closure | 4 | 10 years of generated weather, 5 seeds | Cumulative residual of precipitation |
| Mass | Invariance | 5 | Same weather at 1 min, 1 h and 1 d | Runoff volumes agree within 10% |
| Mass | Response | 6 | Largest storm scaled by 2, 5 and 10 | Added runoff share between 0.1 and 1.05 |
| Mass | Dynamics | 5 | 2 years without rain | Runoff and stores only decline |
| Mass | Groundwater and human | 4 | 380 mm irrigation withdrawal | Budget difference accounts for the volume within 5% |
| Energy | Single budget | 4 | Hourly bare-surface budget | Residual within 5% or 2 W m |
| Energy | Cross budget | 3 | 4 months of drought at fixed net radiation | Changes in , and sum to zero |
| Momentum | Routing | 2 | 4 large storms followed by dry spells | Store between zero and a 15-day hydrograph |
| Momentum | Hydraulics | 4 | 3 steady plateaus | Friction balance within 5% |
Table 2. Probe families in suite v0.1.0. The last column gives 1 representative experiment and its limit. Full definitions are in Section B.
Mass
The governing relation is the water budget of the control volume,
where is precipitation, is evapotranspiration including sublimation, is runoff, is an exchange that the model declares with the outside, is a prescribed human water withdrawal, and is the change in each reported store. The 24 mass probes fall into 5 families.
Closure probes ask whether the budget closes. The reference probe, catchment-closure, forces a lumped catchment with 10 years of daily weather and 5 seeds. It requires the cumulative residual to stay within 5% of cumulative precipitation, every store to remain within its physical range, and evaporation to stay at or below demand. The other 3 closure probes close the budget inside the snowpack across accumulation, storage and melt (snowpack-mass-closure), event by event under rainfall that is scaled toward 100-year depths (extreme-event-closure), and across 12 independent catchments with sampled soil and canopy capacities (ungauged-basin-closure).
Invariance probes ask whether the answer depends on how the modeler set up the domain. A model of the same physical system, discretized at a different spatial or temporal resolution, should give the same result. Each probe changes 1 such setup choice and checks that the output does not move:
The same month of weather delivered at 1-minute, hourly and daily steps must give the same runoff volume within 10% (resolution-invariance).
The same weather on a catchment 10 times larger must give the same depths (area-invariance).
A 28-year shift of the calendar must leave fluxes and stores unchanged to a relative (time-origin-invariance).
A common 250 m shift of all groundwater heads must leave the exchange unchanged (groundwater-datum-invariance).
A periodic climate must forget extra spin-up cycles (spinup-cycle-invariance).
Response probes ask whether a perturbation moves the output in the right direction and by a physical amount. Each probe applies 1 change to the forcing or the initial state and checks the response against a physical bound:
Under identical precipitation, air warmer must lower runoff and air cooler must raise it, each by at least 0.1 of the change in evaporative demand (warming-response).
Rain added to a run must appear in the budget and be shared among evaporation, runoff and storage (precipitation-counterfactual).
Rain that would have fallen as snow must yield the same water within 5% (phase-counterfactual).
A 120 mm storm added on 1 day may not lower runoff on any later day (response-nonnegativity).
The same 60 mm storm on a wet catchment must run off more than on a dry one, by at least 2% of the storm and by no more than the 120 mm of extra water (antecedent-monotonicity).
Scaling the largest storm to 10 times its size must not lower runoff, must not add more runoff than rain, and must send at least 10% of the added rain to runoff (extreme-rain).
Dynamics probes ask whether the catchment settles, drains and stays bounded. Each probe tests 1 part of the catchment’s behavior, from the moment of the storm to 50 years of repeated weather:
A storm may not change anything before it falls (causality).
After the rain stops, runoff and every store may only decline, and the total that runs off cannot exceed what the catchment held (dry-down).
Constant weather for 3 years must produce constant stores and runoff within 1% (steady-state).
Over 10 years, runoff must lie between rain minus potential evaporation minus capacity and rain plus the water the stores could release (runoff-bounds).
Repeated weather over 50 years must not show storage drift beyond of precipitation over the final 5-year block (multi-decadal-drift).
Groundwater and human probes ask whether every water term is accounted for. Each probe checks 1 water term that the model declares or prescribes against the budget:
Direct aquifer recharge must preserve storage and the ordering of exchange (aquifer-recharge-ordering).
A declared head-driven exchange must respond to its external head (exchange-response).
In a 2-year aquifer case, a model that reports groundwater-river exchange must account for recharge, both exchange directions and aquifer storage, so that exchange and storage close the water budget (gw-sw-exchange-consistency).
A prescribed irrigation withdrawal of 380 mm over 10 years must leave the budget, and the difference between the natural and the irrigated run must equal the abstracted volume within 5% (human-abstraction).
Energy
The governing relation is the surface energy budget,
where is net radiation, is the sensible heat flux, is the latent heat flux, is the ground heat flux and is the temperature-dependent latent heat of the phase change. Seven probes test the energy budget.
Four single-budget probes test whether the energy balance holds by itself: (1) Evaporation on wet days must follow demand and on dry days must follow water (pet-consistency). (2) Hourly residuals must close within each solar day or night, with absolute residuals averaged so that opposite errors cannot cancel (surface-energy-closure). (3) Heat entering a soil layer minus heat leaving it must equal the change in stored heat (soil-heat-storage-consistency). (4) Upward longwave radiation must equal emission at the skin temperature plus reflected downward longwave, within 0.5% or 0.5 W m (radiation-consistency).
Three cross-budget probes test whether the water and energy budgets agree: (1) A model with a water head and an energy head can close both budgets and still contradict itself, because the inconsistency lies in the coupling between the two budgets. Latent heat must equal evapotranspiration times the latent heat of the phase it underwent, within 0.5% (latent-heat-et-consistency). (2) A drying surface under fixed net radiation must give up its latent heat to the sensible and ground fluxes, and the 3 changes must sum to zero (evaporative-partition). (3) Melt that the water budget reports must be paid for in the energy budget, by the latent heat of fusion and the cold content of the pack (snowmelt-energy-water).
Momentum
Momentum conservation in a channel is expressed by the Saint-Venant equations. The suite does not yet test the full unsteady balance. It tests conditions that any solution of the equations must satisfy. In the steady uniform limit the bed slope equals the friction slope , and Manning’s relation links discharge and stage through the channel geometry,
where is Manning’s roughness, the flow area and the hydraulic radius. On a mild slope the flow stays subcritical, so the Froude number remains below 1.
Two routing probes test whether routed water stays physical: (1) A routing store may never be negative, and it may never hold more than the cumulative runoff inflow over a 15-day hydrograph (routing-conservation). (2) The lag of a runoff peak must follow a Snyder travel-time scale (Snyder, 1938) within a factor of 2 across 4 catchments and must grow with catchment size (routing-lag-consistency). Four hydraulics probes test the consistency of reported stage and discharge: (1) Stage must be single-valued in discharge and rise with it, with a loop of the right orientation for a flood wave (stage-discharge-monotonic). (2) Discharge and stage on 3 steady plateaus must satisfy the friction balance within 5% (uniform-flow-friction-consistency). (3) The flow must remain subcritical on a mild slope (froude-regime). (4) The celerity of a small pulse, measured between a short and a long reach, must respond to the hydraulic state (wave-celerity-bounds).
Coverage and gaps
Momentum is the least developed law in version 0.1.0. Its 6 probes test consistency conditions and bounds, and no probe yet tests the unsteady momentum balance reach by reach. Three further probes are on the roadmap. Two are in progress: a probe on reach-by-reach mass closure in a branching network, mass/routing-network-closure, and a probe on the release of meltwater from a cold snowpack, energy/snowpack-ripening. The third, momentum/channel-routing-mass, is unclaimed and requires inflow minus outflow minus storage change per reach. An earlier attempt was set aside because the contract then had no variable for water arriving at a reach, so that both sides of the residual came from the model’s own outputs. The contract now carries a prescribed inflow, q_in, which the wave-celerity probe uses (Table 3). We plan to complete version 1 once the suite reaches 50 probes (§ 5).
| Name | Units | Physical meaning |
pr | mm d | Precipitation, echoed exactly as the forcing gave it |
evspsbl | mm d | Evapotranspiration, including sublimation; net of dew and frost where the model nets them |
mrro | mm d | Total runoff |
dis | m s | River discharge |
gwex | mm d | Declared exchange with the outside: regional groundwater, inter-basin transfer |
snm | mm d | Liquid delivered from the snow module to the ground, rain passed through included, and not an outflow beside mrro |
hfls | W m | Latent heat flux |
hfss | W m | Sensible heat flux |
hfg | W m | Ground heat flux at the actual soil surface, where a deeper flux is corrected for the heat stored above it |
mrso | mm | Soil water storage (liquid and ice), reported as an absolute state rather than a tendency |
snw | mm | Snow water equivalent, identically zero and named in zero_stores for a model with no snow module |
canopy | mm | Canopy interception storage, identically zero and named in zero_stores for a model with no such store |
gw | mm | Groundwater storage below the soil column |
channel | mm | Water generated as runoff but not yet released by the model’s routing |
stage | m | Water-surface elevation on the case’s datum, read beside the row’s mean discharge (depth is stage minus bed_elevation_m) |
| Forcing a model receives | ||
pr | mm d | Precipitation |
tas | °C | Near-surface air temperature in Celsius, unlike the kelvin of ts and tsoil_layer |
pet | mm d | Potential evapotranspiration |
rn | W m | Prescribed net radiation, which the model accepts as the energy input to the reported budget when declared |
abstr | mm d | Prescribed human withdrawal, net of return flow |
q_in | m s | Prescribed river or upstream inflow, already a discharge and not multiplied by area or row length |
Table 3. Core vocabulary of the /io contract. A model reports a subset of the output variables and declares which. A probe scores a model only if its outputs cover what the probe requires. The full lexicon is in Appendix § 10.
Harness and protocol
Architecture
The harness separates what a model can see from what decides its verdict (Figure 3). It does so by splitting each evaluation between two sides. On the host, a probe generates the forcing and the labels, and it keeps the labels, the criteria and their tolerances to itself. The model runs on the other side, in a sealed container, and receives only the forcing and the static attributes. Because the model never sees the labels or the criteria, it cannot tune its output to the verdict. Two files link the model to the harness. A manifest, model.yaml, declares what the model needs and what it emits, and a thin adapter, ht_adapter.py, translates between this contract and the native model, so that a model needs no change to its own code. Three runners then start the model: Docker, a plain subprocess for models that run natively, and Apptainer for clusters.

Figure 3. Architecture of the HydroTuring harness. The probe generator, the criteria and the tolerances stay on the host. The model runs in a sealed container, reads the forcing and static attributes from a read-only directory, and writes the outputs that the criteria score. Reference models enter through the same contract in the acceptance gate (§ 3.4).
Lexicon and input-output contract
Model variables are named differently in different models. Some models name discharge (cms) as runoff (q), while others use different conventions. Additionally, a hydrologic model may output runoff alone, and a land surface model may output every flux and store. The contract therefore rests on a shared lexicon. In suite 0.1.0 it names 29 output variables, 14 forcing variables and 34 static attributes, each with units, a sign convention and a statement of what a value means over a time step (Table 3 and Section A). Names follow the CMIP conventions where an equivalent exists (Eyring et al., [15]). A model declares the outputs it emits and the inputs it needs or uses, and it must not pad that list. A probe declares the outputs, forcing and static attributes it requires. The harness scores a pair only when the two declarations are compatible.
A run has 3 steps: the harness stages the inputs, the model runs, and the harness checks what the model returns against the contract. In the first step, the harness writes a request, the forcing and the static attributes to a read-only directory, so that the model cannot alter them. On a river network, the run also reads reach-indexed variables from the routing table. In the second step, the entrypoint runs once and writes 1 row per forcing step to the recorded results, together with a provenance file. Spin-up rows are included in these results but are excluded from scoring. In the third step, the harness compares the results with the request. The adapter must emit the exact number of steps that the request names, so a step count mismatch or a missing declared column is a contract error.
Verdicts
A probe returns 1 of 3 verdicts and a reason (Table 4). PASS and FAIL are scientific statements. N/A means that the probe does not apply for this particular model. For example, a streamflow-only model is N/A on every budget probe. Rather than violating conservation, the model cannot be evaluated on that law because it omits the required outputs. We keep this case apart from FAIL because the 2 statements say different things about a model. The reason is stored beside the verdict, together with the measured quantity, so that a reader can see by how much a limit was missed.
| Verdict | Reason | Scored | Meaning |
| PASS | OK | yes | Every criterion on every seed is within its limit |
| FAIL | VIOLATION | yes | At least 1 criterion on 1 seed is outside its limit |
| N/A | INCOMPLETE | no | The model does not report the variables that the probe needs |
| N/A | INCOMPATIBLE | no | The model and the probe disagree on the time step, on an input, or on paired-perturbation support |
| – | ERROR | no | The adapter or the machinery failed, which is operational and makes the run exit with an error code |
Table 4. Verdicts and reasons. Only PASS and FAIL enter a model’s score.
A model passes the suite when at least 1 probe could be put to it and every probe it was scored on passes. This is the strict aggregate of § 3.1. Because N/A probes count neither for nor against a model, a pass fraction is meaningless without the number of probes scored. We therefore always report both.
Criteria on the reference probe
Each probe contains criteria, and each criterion returns a binary result and a measured value. The reference probe, catchment-closure, runs 10 generated years of daily weather after 365 days of spin-up, on 5 seeds. Table 5 lists its criteria. The closure criterion divides the cumulative residual of the budget by cumulative precipitation, and it uses a 5% limit as an engineering standard. § 3.4 justifies this limit with the reference models and a sweep over the tolerance.
| Criterion | Question | Limit |
| closure | Does precipitation minus evaporation, runoff and the change of every store stay near zero? | Cumulative residual at most 5% of cumulative precipitation |
| et_plausible | Does evaporation stay at or below the demand? | No step exceeds the potential evaporation supplied |
| state_bounds | Does every reported store stay in its physical range? | No store outside its range |
| non_degenerate | Does the model produce runoff that responds to rain? | Runoff ratio between 0.02 and 0.98 and coefficient of variation at least 0.1 |
| forcing_fidelity | Does the model echo the forcing that it was given? | Relative tolerance of |
Table 5. Criteria of the reference probe mass/catchment-closure. The last 3 criteria are shared with most mass probes.
Reference models and probe validation
Reference models
The suite carries 77 reference models, 18 exact and 59 broken. An exact reference conserves the law that a probe tests, by construction or through a published scheme. Four exact references serve as common baselines. The model reference_bucket (model A in § 1) is a single bucket that conserves water exactly. The models flex_lumped and flex_topo are 2 conceptual models of the FLEX family ([29]), and sacsma_snow17 couples the Sacramento soil moisture accounting model with the Snow-17 snow model ((Upstream Tech, 2026)). The 14 remaining exact references are written for the probes whose physics they exercise. They include an energy-balance snowpack, a soil column with prescribed heat capacity, a Saint-Venant channel and a lagged-head aquifer cell (Table 11).
| Model | Probes requiring a pass |
reference_bucket | 25 |
flex_lumped | 25 |
sacsma_snow17 | 25 |
flex_topo | 24 |
reference_coupled | 3 |
reference_saint_venant | 2 |
reference_radiative | 1 |
reference_snow_energy | 1 |
reference_soil_heat | 1 |
reference_recharge_order_exact | 1 |
reference_driven_exchange | 1 |
reference_evolving_exchange | 1 |
reference_recharge_exchange | 1 |
reference_datum_exact | 1 |
reference_exchange_exact | 1 |
reference_snyder_router | 1 |
reference_rating | 1 |
reference_uniform_flow | 1 |
Table 11. Exact reference models and the number of probes whose gate requires them to pass. The first four are the common baselines. The others are written for the probes whose physics they exercise.
Each of the 59 broken references carries 1 defect that the suite must detect (Table 12). We use 6 of them throughout the paper. The model reference_leaky (model B in § 1) hides a silent 15% sink, so its budget does not close. The model reference_cheater solves for storage as whatever balances the budget, which closes the budget by construction and drives storage outside its physical range. The model reference_degenerate evaporates all precipitation and produces no runoff. The model reference_saturating caps daily runoff at 25 mm, so it is exact for ordinary storms and flat for extreme ones. The model reference_climatology emits the seasonal mean of runoff and evaporation whatever the weather. The model reference_in_sample leaks water only outside the range for which it was built. A last reference, reference_streamflow_only, is not broken but limited. It reports discharge only, from a store that does not read the temperature. It is N/A on every budget probe, and the warming-response probe catches it.
| Reference model | Defect built in | Probe | Criterion |
abstraction_blind | The exact bucket, blind to a prescribed human abstraction. | human-abstraction | human_abstraction |
air_emitter | Negative control for energy/radiation-consistency. | radiation-consistency | radiative_identity |
anticipating | The reference bucket with its runoff smoothed over a centered 7-day window, so every reported value carries 3 days that have not happened yet. | causality | causality |
| Reference model | Defect built in | Probe | Criterion |
area_leak | The bucket losing a share of its runoff that grows with the catchment area it is given: 0% at 25 km and 50% at 2,500 km. | area-invariance | invariance |
calendar | The bucket model with a recession that drifts with the calendar year. | time-origin-invariance | invariance |
cheater | Solves for storage as whatever balances the budget. | antecedent-monotonicity, catchment-closure, precipitation-counterfactual, spinup-cycle-invariance, ungauged-basin-closure | antecedent-monotonicity, counterfactual-response, state_bounds |
climatology | A seasonal climatology baseline: emits the mean runoff and evaporation for the day of the year regardless of the weather, with constant storage. | dry-down | dry_down |
constant_lambda | Fully coupled and correct in every other respect, except that the latent heat of vaporization is a constant instead of a function of temperature. | evaporative-partition, latent-heat-et-consistency | flux_identity |
datum_dependent | Intentionally datum-dependent conductance reference. | groundwater-datum-invariance | datum_flux_invariance |
degenerate | The trivial solution: all precipitation evaporates, nothing runs off, storage never changes. | catchment-closure, precipitation-counterfactual, resolution-invariance, runoff-bounds, spinup-cycle-invariance, time-origin-invariance, ungauged-basin-closure, warming-response | counterfactual-response, non_degenerate, response_sign, runoff_bounds |
degree_day | The same snowpack, melted on air temperature through a degree-day factor and with a surface energy budget that closes around its evaporation alone. | snowmelt-energy-water | melt_energy |
diurnal_bias | The coupled reference’s water and energy account with only sensible heat biased: the residual is +20 W m from 06:00 to 18:00 local-solar time, and −20 W m otherwise. | surface-energy-closure | energy_closure_by_phase |
energy_leak | The energy-budget counterpart of reference_leaky: coherent between water and energy, but 15% of net radiation never appears in any flux it reports. | evaporative-partition, latent-heat-et-consistency | energy_closure |
exchange_sign_error | A lagged-head bookkeeping reference sharing the smoothed-stage mechanism of reference_exchange_exact (declaring aquifer_storage_coefficient in needs_static to match probe.yaml), with a bidirectional river exchange that closes its groundwater balance but reverses a component sign. | gw-sw-exchange-consistency | exchange_components |
fixed_celerity | Deliberately broken routing reference for momentum/wave-celerity-bounds. | wave-celerity-bounds | wave_celerity_bounds |
| Reference model | Defect built in | Probe | Criterion |
| fixed_step | The reference bucket with its step hard-wired: it claims to run at any resolution and ignores the one it is given, treating every row as a day. | resolution-invariance | resolution_invariance |
| flat_stage | The routed bucket that reports a stage at all times equal to a constant. | stage-discharge-monotonic | non_degenerate |
| frozen_soil | The conductive soil heat reference with reported layer temperature frozen at its initial value. | soil-heat-storage-consistency | soil_heat_storage |
| ground_dodge | Latent heat agrees with the water it evaporated and the energy budget closes, but the sensible heat flux is a fixed share of net radiation that never reads the soil. | evaporative-partition | partition_shift |
| gw_slow_drift | An exact bucket whose reporter subtracts from runoff and accumulates the deficit in groundwater storage. | multi-decadal-drift | total_storage_drift |
| half_soil | The conductive soil heat reference with only its reported layer temperature response halved. | soil-heat-storage-consistency | soil_heat_storage |
| in_sample | The bucket model with a leak that only opens outside the range it was built for. | extreme-event-closure | event_water_closure |
| instant_router | Deliberately wrong rainfall-runoff reference for the routing-lag probe. | routing-lag-consistency | lag_time_bounds |
| inverse_router | Deliberately wrong geometry-aware reference for the routing-lag probe. | routing-lag-consistency | scaling-monotonicity |
| leaky | The bucket model with a silent 15% sink. | catchment-closure, human-abstraction, precipitation-counterfactual, spinup-cycle-invariance, time-origin-invariance | closure, counterfactual-response |
| leaky_router | The bucket with a unit hydrograph whose weights sum to 0.999: 0.1% of every day’s runoff enters the channel store and never leaves. | routing-conservation | routing-conservation |
| no_reflection | Negative control for energy/radiation-consistency. | radiation-consistency | radiative_identity |
| noise_sink | The exact bucket, conserving water internally, which misreports its evaporation by a seeded and declares the difference as a regional groundwater exchange. | exchange-response | exchange_response |
| overflowing | The bucket reporting its runoff plus an unphysical 80% of the rain. | runoff-bounds | runoff_bounds |
| overshooting | The bucket with its runoff sharpened by a derivative term: each day’s reported runoff is the bucket’s plus 0.8 times the change since the day before, floored at zero. | response-nonnegativity | response_nonnegativity |
| rating_drift | The routed bucket with a drifting gauge. | stage-discharge-monotonic | rating_monotonic |
| rating_inverted | The routed bucket whose rating is read backwards. | stage-discharge-monotonic | rating_loop |
| recharge_blind | Recharge-ordering reference that ignores gw_recharge, so its own budget no longer closes. | aquifer-recharge-ordering | aquifer_recharge_ordering |
| recharge_overshoot | Recharge-ordering reference that sends 120% of a recharge day to the river, so its storage falls below the control’s. | aquifer-recharge-ordering | aquifer_recharge_ordering |
Table 12. Deliberately broken reference models (with the reference_ prefix omitted in column 1) and the criterion each is built to trip.
Acceptance gate
A probe enters the suite only when its gate is satisfied. The gate has 2 kinds of check. A must-pass check requires an exact reference to pass the probe. A must-fail check requires a broken reference to fail the probe and to trip the criterion that it was built to trip. The 37 probes carry 116 must-pass and 78 must-fail checks. We re-ran all 37 gates on 2 November 2026 at the commit that tags suite 0.1.0, and every check behaved as declared.
The gate fixes tolerances from 2 sides. An exact model sets the lower bound, because a tolerance tighter than what exact physics achieves would fail honest models. Over 20 random seeds, the exact bucket, flex_lumped and sacsma_snow17 pass catchment-closure on all 20, with residuals below of precipitation. A broken model sets the upper bound. Over the same seeds, reference_leaky fails the closure criterion on 20 of 20. The model reference_cheater fails the state-bounds criterion on 20 of 20, and reference_degenerate fails the non-degeneracy criterion on 20 of 20 (Table 14).
| Probe | Model | Seeds passed | Criterion trips or measured range |
catchment-closure | reference_bucket | 20/20 | largest residual of precipitation |
catchment-closure | flex_lumped | 20/20 | largest residual of precipitation |
catchment-closure | sacsma_snow17 | 20/20 | largest residual of precipitation |
catchment-closure | reference_leaky | 0/20 | closure 20 of 20 |
catchment-closure | reference_cheater | 0/20 | state_bounds 20 of 20 |
catchment-closure | reference_degenerate | 0/20 | non_degenerate 20, et_plausible 18 of 20 |
extreme-rain | reference_bucket | 10/10 | share 0.56 to 1.00 |
extreme-rain | reference_saturating | 4/10 | share 0.01 to 0.22 |
Table 14. Seed sensitivity of 2 probes. Top: mass/catchment-closure on 20 random seeds. Bottom: mass/extreme-rain on 10 random seeds. The column on the right gives the criterion trips over the seeds, or the share of added rain that runs off over the ladder up to 10 times the largest storm, as the range over seeds.
The closure tolerance sits between the 2 bounds. It is 5% of cumulative precipitation, the engineering rule that the probe adopts. Among the 22 evaluated models that report a whole-record residual on catchment-closure, 21 are below 0.2% and the largest is 2.5% (xaj). Any tolerance between 2.6% and 15% therefore separates every evaluated model from reference_leaky, whose sink is 15% by construction (Figure 5a). We keep 5% because it is conventional and because it sits in this empty interval. Any change to it requires the process in § 5, and Section D gives the full sweep.
Figure 5. Closure and failure modes in suite 0.1.0. a, Whole-record closure residuals that the evaluated models report on the closure probes (catchment, ungauged-basin and snowpack), as a share of cumulative precipitation, with 1 dot per model and probe, and the same quantity for the broken reference reference_leaky. The dashed line marks the 5% limit that the 4 closure probes share. b, Number of failed model-probe pairs in which each criterion was tripped. A pair can trip more than 1 criterion. The 131 failed pairs trip 26 distinct criteria, and the 10 most frequent are shown.
A single case can miss a defect. The model reference_saturating fails the extreme-rain criterion on 6 of 10 seeds and passes it on 4. The signal therefore varies from seed to seed. Across the ladder up to 10 times the largest storm, the exact bucket returns between 0.56 and 1.00 of the added rain on every 1 of the 10 seeds. The saturating model returns 0.01 to 0.22 and exceeds the limit of 0.10 on 4 of them. A probe that used 1 seed would pass the saturating model 40% of the time. The probe uses 3 seeds, and a model passes only when all 3 pass. Of the 120 subsets of 3 among the 10 seeds, 116 contain at least 1 seed that exposes the defect. Across the suite, 22 probes use 3 seeds, 14 use 5, and the probe on catchment attributes uses 12.
Results
We evaluated 32 models on the 37 probes of suite 0.1.0. Three are the physically exact references flex_lumped, flex_topo and sacsma_snow17. Seven are AI-based: an LSTM, the differentiable HBV model dhbv2 (Song et al., 2025), and 5 training checkpoints of a mass-conserving perceptron, mcp1 to mcp5 (Wang and Gupta, 2024). The remaining 22 are process-based models, from conceptual bucket models to land surface models, a groundwater model and routing models. They include gr4j (Perrin et al., 2003), xaj (Zhao, 1992), topmodel [5], topkapi [10], vic [32, 34], mhm (Samaniego et al., 2010), summa [12], raven [14], hype [33], prms (Markstrom et al., 2015), noahmp (Niu et al., 2011), cwatm [9], lisflood (Van Der Knijff et al., 2010), modflow6 [28] and mizuroute (Mizukami et al., 2016). Of the 32 models, 29 entered through model proposals § 5. Each model was run in its own container with the manifest and adapter in the repository, and each version is recorded in the archive.
Verdict matrix
The 32 models and 37 probes give 1,184 model-probe pairs (Figure 4). Of these, 474 are PASS, 131 are FAIL and 579 are N/A. Among the N/A pairs, 499 are INCOMPLETE and 80 are INCOMPATIBLE. Consequently, 49% of all model-probe pairs remain unscored due to missing output variables or time-step mismatches rather than physical failures. Mass probes are N/A for 30% of pairs (234 of 768), while energy probes are N/A for 84% (188 of 224) and momentum probes for 82% (157 of 192). Few models report heat fluxes or reach-indexed variables.
Figure 4. Verdict matrix of 32 models on 37 probes in suite 0.1.0. Rows are models. The 3 exact references are on top, and the other models follow in order of the number of probes that they pass. Columns are probes, grouped by law. Green is PASS and orange is FAIL. Light gray is N/A because the model does not report the required variables (INCOMPLETE), and dark gray is N/A because model and probe disagree on an input or a time step (INCOMPATIBLE).
Only the 3 exact references pass every probe that they are scored on (74 of 74 pairs). Among the 28 models that are scored on 5 or more probes, no other model does so. The remaining 4 models, cfe, mizuroute, modflow6 and rvic, are scored on 1 to 3 probes. Three of them pass all of their scored probes, which says little because the sample is small. The 7 AI-based models pass 98 of 133 scored pairs (74%), while the 22 process-based models pass 302 of 398 (76%). Because these pass rates are nearly identical, model class alone does not predict physical consistency. Table 15 gives the standing of every model.
| Model | Class | All | Mass | Energy | Momentum |
flex_lumped | Physical reference | 25/25 | 19/19 | 1/1 | 5/5 |
sacsma_snow17 | Physical reference | 25/25 | 20/20 | 1/1 | 4/4 |
flex_topo | Physical reference | 24/24 | 19/19 | 1/1 | 4/4 |
mcp1 | AI-based | 18/20 | 17/19 | 1/1 | – |
mcp3 | AI-based | 17/21 | 16/19 | 1/1 | 0/1 |
mcp2 | AI-based | 15/20 | 15/19 | 0/1 | – |
dhbv2 | AI-based | 15/21 | 13/19 | 1/1 | 1/1 |
mcp4 | AI-based | 14/20 | 13/19 | 1/1 | – |
mcp5 | AI-based | 14/20 | 14/19 | 0/1 | – |
lstm | AI-based | 5/11 | 5/11 | – | – |
mhm | Process-based | 21/22 | 19/20 | 1/1 | 1/1 |
gr4j | Process-based | 20/21 | 18/19 | 1/1 | 1/1 |
hymod | Process-based | 20/21 | 18/19 | 1/1 | 1/1 |
topkapi | Process-based | 20/21 | 18/19 | 1/1 | 1/1 |
prms | Process-based | 20/22 | 18/20 | 1/1 | 1/1 |
tank | Process-based | 19/20 | 18/19 | 1/1 | – |
lisflood | Process-based | 19/21 | 17/19 | 1/1 | 1/1 |
xaj | Process-based | 19/21 | 17/19 | 1/1 | 1/1 |
topmodel | Process-based | 19/22 | 17/19 | 1/1 | 1/2 |
wflow_sbm | Process-based | 19/22 | 16/19 | 1/1 | 2/2 |
cwatm | Process-based | 17/21 | 16/19 | 0/1 | 1/1 |
noahmp | Process-based | 17/25 | 14/20 | 3/5 | – |
vic | Process-based | 16/23 | 15/19 | 1/4 | – |
hype | Process-based | 15/23 | 13/20 | 1/1 | 1/2 |
simhyd | Process-based | 11/20 | 11/19 | 0/1 | – |
shud | Process-based | 9/22 | 8/19 | 0/1 | 1/2 |
summa | Process-based | 9/23 | 7/19 | 1/3 | 1/1 |
raven | Process-based | 7/22 | 6/20 | 0/1 | 1/1 |
modflow6 | Process-based | 3/3 | 3/3 | – | – |
cfe | Process-based | 1/1 | 1/1 | – | – |
mizuroute | Process-based | 1/1 | – | – | 1/1 |
rvic | Process-based | 0/1 | – | – | 0/1 |
Table 15. Standing of the 32 evaluated models in suite 0.1.0. Entries are probes passed over probes scored. A probe is not scored when the model does not report the variables it needs or cannot run at the probe’s time step (N/A). Rows are ordered by class and then by the number of probes passed. The LSTM is the model behind the case study in § 4.3.
Models fail 25 of the 37 probes. Twelve probes fail no evaluated model, and 3 of them, energy/snowmelt-energy-water, energy/surface-energy-closure and mass/exchange-response, have not yet scored any submitted model. The most discriminating probes are mass/human-abstraction (22 of 27 scored models fail), mass/antecedent-monotonicity (10 of 28), mass/dry-down (9 of 28), mass/resolution-invariance (9 of 29) and mass/response-nonnegativity (8 of 28).
Where models fail
The 131 failed pairs trip 26 distinct criteria (Figure 5b). The 5 most frequent are human_abstraction (22 pairs), state_bounds (21), non_degenerate (16), et_plausible (15) and antecedent_monotonicity (10). Four observations stand out.
First, no evaluated model trips the whole-record closure criterion on any of the 4 closure probes. Among the 22 models that report a residual on catchment-closure, the largest is 2.5% of precipitation. Across the closure probes that report a residual, the largest is 3.9% (xaj on ungauged-basin-closure; Figure 5a). Closure of the long-term water budget holds across the evaluated models.
Second, a closing budget does not imply physical stores and fluxes. Five of 27 scored models fail catchment-closure on another criterion. The model hype lets soil storage reach 363 mm in a 320 mm profile. The models raven and shud evaporate 1.01 and 1.27 times the potential evaporation that the probe supplied. The model simhyd exceeds both its canopy capacity and the potential evaporation, and summa lets canopy storage reach 20.3 mm against a capacity of 2 mm. Although these models close their overall water budgets, they report states or fluxes that exceed physical capacity or atmospheric demand.
Third, closure over the record does not imply closure event by event. Four models fail the event-wise criterion of mass/extreme-event-closure. The model xaj fails on 133 of 1,316 events and shud on 130 of 1,257. The model cwatm fails on 4 of 1,257 events and wflow_sbm on 1 of 1,299. The model xaj passes the whole-record criterion with a residual of 2.5%, so a total over the record can hide events that lose or gain water. Three further models, hype, simhyd and summa, fail the same probe on state bounds.
Fourth, most models do not account for a prescribed human withdrawal. On mass/human-abstraction, 20 of 27 scored models show no change in any budget term when the probe withdraws 380 mm of water over 10 years. Two more account for part of it: lisflood leaves 83% of the withdrawal unaccounted for, and prms leaves 29%. Only 5 scored models close this budget within the 5% limit, of which 3 are the exact references.
Case study: an LSTM
An LSTM flood-forecasting model illustrates how HydroTuring evaluates a model that reports streamflow alone. We ran the publicly released weights of the model lineage behind Nearing et al. (2024), with the training forcing products mapped from the probe’s precipitation, temperature and potential evaporation. Inputs that a probe cannot supply, such as net radiation and surface pressure, are derived from the forcing alone. Catchment attributes are taken from the first year of the generated record or set to the training mean. The model therefore operates as an average catchment under mock inputs. Our evaluation reflects the released weights coupled with this adapter rather than the full operational forecasting system.
The model reports runoff and discharge only. It is N/A on 26 of 37 probes. On 24 of them the probe requires evaporation, storage or heat fluxes, while on 2 the model and the probe disagree on an input or a time step. Rather than violating conservation on those 26 probes, the model cannot be falsified on them because it omits the required outputs. On the 11 probes that require runoff alone, it passes 5 and fails 6 (Table 6).
| Probe | Verdict | Measured quantity |
| resolution-invariance | FAIL | Runoff differs by 73.8% of precipitation between hourly and daily records of the same month (limit 10%) |
| extreme-rain | FAIL | Across the ladder up to 10 times the largest storm, 35 of 526 mm of added rain run off (0.07). The exact bucket returns 0.56 to 1.00 |
| dry-down | FAIL | 2 rainless years yield a near-constant 0.3 mm d that does not decay (coefficient of variation 0.02, minimum 0.1) |
| warming-response | FAIL | Runoff falls by 0.088 per unit of added demand (limit 0.10). The exact bucket falls by 0.20 |
| phase-counterfactual | FAIL | Runoff differs by −8.4% of the rain when snow falls as rain (limit 5%) |
| response-nonnegativity | FAIL | Runoff falls 0.18 mm d below the control after 120 mm of rain is added |
| causality | PASS | Nothing changes before the added storm. Runoff responds with 0.32 of the added rain |
| steady-state | PASS | Settles to 0.63 mm d under 2.5 mm d of rain, with no variation in the last year |
| runoff-bounds | PASS | Runoff is 0.34 of the rain, inside the bounds 0.12 to 1.05 |
| antecedent-monotonicity | PASS | The wetter catchment runs off 16.4 mm more from a 60 mm storm (0.27 of it) |
| area-invariance | PASS | Scaling the catchment area leaves the depths unchanged |
Table 6. Verdicts of the LSTM on the 11 probes that it can be scored on, with the measured quantity on the seed that the report shows. Reference values are from the exact bucket on the same probe where stated.
The pattern is consistent: the model passes probes whose answer follows from its causal recurrent structure or a plausible climatological response, while failing every probe that moves the input outside the training regime. The passes are straightforward. The model uses no future information, settles under constant weather, keeps runoff within the bounds of precipitation, and produces more runoff on a wetter catchment. The failures fall into 3 groups. The first group concerns input extremes. Across the ladder up to 10 times the largest storm, the model returns only 7% of the added rain, even though a catchment with a 320 mm soil capacity cannot absorb 526 mm. Over 2 rainless years, it maintains a baseflow floor learned from its training basins, so the discharge never recedes. The second group concerns perturbation responses. Warming produces the expected sign in both directions on every seed, but its magnitude falls below the 0.10 threshold on 3 of 5 seeds. Allowing the static climate attributes to update with the warmed record yields a stronger response, indicating that part of the apparent climate sensitivity is encoded in the static attributes rather than the dynamic weather forcing. The third group concerns physical consistency.
Hourly records produce a runoff volume that differs by 73.8% of precipitation from daily records of the same weather, because the network accepts arbitrary step sequences without an explicit time-step parameter and treats each hourly rate as a daily depth. Added rain also reduces subsequent daily runoff by 0.18 mm d.
Differentiable and mass-conserving models
Two other AI-based families provide a contrast. The differentiable model dhbv2 (Song et al., 2025) embeds an explicit HBV core, while the 5 mass-conserving perceptron checkpoints mcp1 to mcp5 (Wang and Gupta, 2024) route water through explicit stores with conservation built into the architecture.
The mass-conserving perceptron closes the water budget by construction, and the probes confirm this property. All 5 checkpoints pass catchment-closure, extreme-event-closure and ungauged-basin-closure, with whole-record residuals below 0.001% of precipitation. However, they fail between 2 and 6 other probes, and the failed probes vary across training epochs within the same architecture. The checkpoint at epoch 27 (mcp1) fails 2 probes, while the checkpoint at epoch 2707 (mcp4) fails 6. In mcp4, runoff rises by 1.17 times the rain added when a storm is scaled by 2, falls below the control after rain is added, and still varies by 11.7% (runoff) and 4.2% (storage) in the final year of constant weather. Four checkpoints fail antecedent-monotonicity: the wetter catchment returns only 0.1 to 0.4 mm more runoff from a 60 mm storm in 3 checkpoints (below the 1.2 mm minimum), and 2.3 mm less in mcp4. Enforcing conservation in the architecture therefore secures that single law while leaving other physical behaviors dependent on training.
The differentiable model passes the 3 closure probes that it is scored on, together with the extreme-rain and warming probes. However, it fails 6 probes in the invariance and response families. Its runoff differs by 12.1% of precipitation between hourly and daily records (limit 10%), likely because the calibrated parameters retain the daily time-step scaling at which they were fitted. Runoff also differs by −34.8% of the rain when snow falls as rain (limit 5%). Added rain lowers subsequent runoff by 0.28 mm d, and the precipitation-counterfactual probe records 10,040 mm of runoff in the control run against 8,686 mm of rain. Finally, the model fails the area-invariance probe because its channel store departs from the expected scaling when catchment area is rescaled, suggesting that channel parameters calibrated for 1 basin area do not transfer across scales. An explicit physical core does not by itself guarantee the invariances tested by the suite.
Community protocol and governance
A benchmark for physical laws needs more physics beyond what an individual group can supply. Mass, energy and momentum each have specialists who disagree about tolerances, denominators and what counts as a violation. We therefore built HydroTuring as a protocol with explicit steps for contributing, reviewing and crediting work, and we describe it here because the 37 probes in this article exist through it.
Contributing a probe or a model
There are 2 routes. A contributor who wants to add a physical test claims an item from the public roadmap, or proposes a new probe, through an issue. The maintainer labels the proposal as accepted when the probe would discriminate between reference models, and the issue is assigned to the person who will do the work. This early review filters out weak probe designs before contributors invest time in implementation. The contributor then scaffolds the probe from a template with ht init-probe, writes the generator, the criteria and the reference models, and runs the acceptance gate. A contributor who wants a model evaluated proposes it through a separate form. The proposer does not need to package the model. The issue is assigned to whoever will build the container, which is often the maintainer.
Both routes are written for coding agents as well as for people. The repository contains an AGENTS.md file that states the input-output contract, the lexicon and the acceptance rules in compact form. An agent given a model repository and this file can draft an adapter, and the contract review below then checks it. Because a failed probe is a scientific finding rather than a packaging defect, physical failures are never grounds for rejecting a model.
Review and decisions
Every probe merged from 16 September 2026 is reviewed by at least 2 members of a reviewer pool for its law. The probe is merged only after both reviewers approve and the gate is green. The pool currently has 3 reviewers for energy and 3 for mass. Probes merged before 16 September were accepted by the maintainer under the earlier process. Each model undergoes a single review focused on contract compliance rather than physical performance: the adapter must honor the input-output directory, the declared outputs must be honest, and the image must be reproducible.
A scientific steering committee of 4 to 6 people, spanning hydrologic modeling, machine learning for Earth science and operational hydrology, is being formed. It will settle disputes about probe design, decide when a suite version is cut, and approve any change to the tolerance of a merged probe together with the authors of that probe. Such a change silently alters every recorded result, so it bumps the minor version of the suite.
The repository lists 25 contributors beyond the maintainer, namely 18 authors of merged probes and 7 further proposers of accepted models. The 18 probe authors and 1 proposer with 5 accepted proposals are authors of this article. The remaining 6 proposers are credited in the contributor file.
Growth and roadmap
The suite grew from 1 probe on 3 September 2026 to 37 on 16 October (Figure 6). The model pool grew from 1 to 10 by 17 September and to 32 on 31 October. The roadmap lists the probes that we want next, with the template to start from and a note on what each probe discriminates. Two further probes are claimed and in progress, a network-wide mass closure probe and a snowpack ripening probe. One probe remains unclaimed, the reach-by-reach mass balance of a channel. We aim for 50 probes at version 1, with 13 to add, and for more energy and momentum probes in particular.
Figure 6. Growth of HydroTuring from its first probe to suite 0.1.0. Dates are those of the commits that merged each probe and each model into the main branch. The dashed line marks the target of 50 probes for version 1. Of the 32 models, 22 were added on 31 October 2026.
Discussion and limitations
What the verdicts show
Three results stand out. First, evaluated models rarely fail on long-term water-budget closure. All 22 models that report a residual on catchment-closure stay below 2.6% of precipitation; instead, failures occur in the response of the model to a perturbed input and in the physical bounds of reported states and fluxes. Second, enforcing mass conservation in a network architecture secures that law alone without guaranteeing physical behavior on other laws or regimes. Although the 5 mass-conserving perceptron checkpoints close the water budget within 0.001%, they still fail between 2 and 6 probes each. Third, nearly half of all model-probe pairs (49%) cannot be scored, primarily because models omit the variables that a probe requires. Reporting internal states and fluxes is therefore the most direct way to make a model falsifiable.
Use before deployment and during training
We see 2 use cases of HydroTuring. A modeler can run HydroTuring as a smoke test before a model enters operations, in the way a unit test suite precedes a software release. The verdicts name the violated law and give the measured size of the violation. A forecaster or a water manager needs this information to decide whether a model can support drought assessment, irrigation planning or a climate impact study that relies on evaporation and storage. Journals can also run submitted models through the suite as a reproducibility and consistency check, because the suite is open and every verdict can be reproduced from a recorded seed.
The probes can also serve during training. The mass-conserving perceptron checkpoints show that verdicts change from epoch to epoch within 1 architecture. The checkpoint at epoch 27 fails 2 probes, while the checkpoint at epoch 2707 fails 6. A model developer can therefore run the probes on saved checkpoints and select among them based on physical consistency as well as hydrograph skill. Although the probes are black-box experiments and cannot enter a gradient-based loss function directly, they identify which physical constraints a training objective should add.
Limitations
First, passing the benchmark is a necessary condition for physical consistency rather than a sufficient condition for predictive accuracy. A model that passes every probe conserves mass, energy and momentum under the tested conditions, but it may still be inaccurate on an observed basin or violate a law under untested conditions. The suite therefore complements rather than replaces hydrograph evaluation.
Second, public probes can be targeted. A fresh seed prevents a model from memorizing a case, but it does not prevent tuning to the rule that a probe tests. The suite offers 2 defenses. A private probe tree can be added with -probe-root, so that an evaluator can hold out probes whose generators and tolerances are unpublished. Each verdict also reports the measured quantity, so that a model that barely passes can be identified. We therefore regard verdicts from the public suite as a pre-deployment smoke test rather than a certification.
Third, tolerances are conventions. The 5% closure limit, the 10% resolution limit and the other thresholds are engineering choices, and the sweep in § 3.4 shows that the closure verdicts of the evaluated models do not depend on the exact value between 2.6% and 15%. For other criteria, the gate ensures that exact references pass and broken references fail. Changes to a tolerance require the governance process in § 5.
Fourth, the catchment and the weather are synthetic. Most probes run on 1 invented lumped catchment of with a 320 mm soil and a 2 mm canopy, under weather from a generator whose climate does not match any gauged basin. Only the ungauged-basin probe samples other attributes. This design is a strength for controlled experiments because the exact physical response is known. However, a physical violation and a distribution shift cannot always be separated. A model trained on real basins that misbehaves in the synthetic catchment may violate physics or may operate far outside its training distribution. The adapter adds a second dependence. In an early evaluation of the differentiable model, 1 reasonable choice for a seasonality attribute changed the coefficient of variation of runoff from 1.1 to 0.07. The probe does not provide this attribute, and that choice changed 2 verdicts. A verdict therefore evaluates a model together with its adapter, and the adapter is documented alongside each model. We plan a real-data track after version 1 is published. It will keep the weather synthetic while replacing the invented catchment with gauged basins whose attributes are drawn from Caravan ([27]). The repository holds a proposal for 4 CAMELS-US basins ([1]) that span an alpine snowpack, a maritime rain regime, a karst aquifer and a lowland agricultural catchment.
Fifth, the spatial scale and scope remain limited. The probes treat a catchment as 1 lumped control volume, apart from the routing and hydraulics probes on a single reach or a small network. They do not test spatial patterns within a basin, even though semi-distributed structure can improve streamflow prediction ([7]). Momentum is the least developed law, and no probe yet tests the full unsteady momentum balance. The energy results also rest on a small sample of models: only 3 evaluated models are scored on the 2 daily cross-budget probes, and all 3 fail them. The repository documentation notes that daily averaging does not remove the mismatch between the net radiation that these probes supply and the radiation that an adapter computes internally. An open issue tracks this fix, and we regard those archived scores as provisional.
Sixth, a black-box test verifies the internal consistency of the variables a model reports rather than the physical realism of its internal states. A model that reports a synthetic store can close its budget on paper. The paired and counterfactual probes make such a post-hoc adjustment harder to sustain because the reported store must respond physically to perturbations, but they cannot rule it out completely. Seventh, each model enters with 1 version and 1 adapter. Verdicts for stochastic models depend on the random draw unless the adapter fixes it, as the LSTM adapter does by taking the exact median of the predictive distribution. Scores are comparable only within 1 suite version.
Conclusion
A hydrograph score cannot distinguish a model that conserves water from one that loses 15% of it, yet the models that issue operational forecasts are judged mainly by that score. We introduced HydroTuring, a benchmark of 37 physics probes for the conservation of mass, energy and momentum in hydrologic models. First, each probe is a controlled experiment on forcing generated at run time, with binary criteria on 1 physical law. Second, the harness runs the model as a black box and keeps the labels and tolerances on the host. Third, every probe is validated against 77 reference models, including 18 that must pass and 59 that must fail. Across 32 evaluated models, only the 3 exact references pass every probe that they are scored on among the 28 models scored on 5 or more probes. A pre-trained flood-forecasting LSTM fails 6 of the 11 probes it can be scored on, while 49% of all model-probe pairs remain unscored because models do not report the variables that the probes require.
These findings support 3 applications. Model developers can run the suite as a pre-deployment smoke test and expose additional internal states and fluxes. Journal editors can verify physical consistency alongside reported hydrograph metrics, and operational agencies can require benchmark verdicts before adopting a model. Under our open community protocol, authors of merged probes become co-authors of the benchmark. We invite the community to contribute the 13 probes needed to reach the 50-probe target for version 1, to propose additional models, and to help build the upcoming real-catchment evaluation track.
Code and data availability
The harness, the 37 probes, the 77 reference models, the model adapters and the results archive are available at https://github.com/Flood-Lab/HydroTuring, and the project website is https://flood-lab.github.io/HydroTuring/. Suite 0.1.0 [30] is archived on Zenodo (https://doi.org/10.5281/zenodo.23874854), and all versions share the concept DOI https://doi.org/10.5281/zenodo.23874853. The harness, probes, reference models and adapters are released under the Creative Commons Attribution-NonCommercial 4.0 license, and each packaged model keeps its own license, which its manifest names. No dataset is distributed, because every probe generates its forcing from a recorded seed. The verdicts, the measured quantities and the seeds of every run reported here are in the results archive in the repository.
Acknowledgments
We thank the contributors who proposed models, reviewed probes and reported issues in the repository.
Schemas and a worked probe
Model manifest
A model declares what it emits and what it needs in model.yaml. The manifest of the exact bucket illustrates the fields. A model that predicts discharge only declares fluxes: [mrro, dis] and states: [], and it is then N/A (INCOMPLETE) on every probe that needs more.
name: reference_bucket
version: "1.1.0"
license: CC-BY-NC-4.0
runner: subprocess
entrypoint: ["python3", "ht_adapter.py"]
timestep: [PT1D, PT1H, PT15M, PT5M, PT1M]
emits:
fluxes: [pr, snm, evspsbl, mrro, dis, gwex]
states: [mrso, snw, canopy, channel]
diagnostics: [stage]
needs_forcing: [pr, tas, pet]
supports:
perturbation: true
needs_static: [soil_capacity_mm, canopy_capacity_mm]
uses_static: [area_km2, width_m, cross_section_shape, bed_elevation_m,
slope, manning_n]
Probe definition
A probe is a directory with a definition file, a generator and a README. The definition of the reference probe, mass/catchment-closure, is shown in abridged form. The baselines block is the declaration that the acceptance gate enforces.
id: mass/catchment-closure
law: mass
requires:
fluxes: [pr, evspsbl, mrro]
states: [mrso, snw, canopy]
case:
generator: generate.py # forcing is generated from the seed at run time
n_seeds: 5
timestep: PT1D
period_years: 10
spinup_days: 365
max_runtime_s: 60
criteria:
- closure: {denominator: sum_pr, threshold: 0.05,
sinks: [evspsbl, mrro]}
- state_bounds: {mrso: [0.0, soil_capacity_mm], snw: [0.0, null],
canopy: [0.0, canopy_capacity_mm]}
- et_plausible: {max_ratio_to_pet: 1.0, pet_variable: pet}
- non_degenerate: {runoff_ratio: [0.02, 0.98], min_flux_cv: 0.1,
min_response: 0.05, cv_variables: [mrro, evspsbl]}
- forcing_fidelity: {variables: [pr], rtol: 1.0e-6}
baselines:
must_pass: [reference_bucket, flex_lumped, flex_topo, sacsma_snow17]
must_fail:
reference_leaky: closure
reference_cheater: state_bounds
reference_degenerate: non_degenerate
Lexicon
Table 7 lists the output variables of the lexicon, with their group, units and meaning. The forcing variables and static attributes are defined in the same file of the repository.
| Name | Group | Units | Physical meaning |
| pr | flux | mm d | Precipitation, echoed exactly as the forcing gave it |
| evspsbl | flux | mm d | Evapotranspiration, including sublimation; net of dew and frost where the model nets them |
| mrro | flux | mm d | Total runoff |
| Name | Group | Units | Physical meaning |
dis | flux | River discharge | |
gwex | flux | Declared exchange with the outside: regional groundwater, inter-basin transfer | |
gw_sw_exchange | flux | Net river-aquifer exchange inside the control volume, not added to gwex or counted in closure | |
gw_to_sw | flux | The aquifer losing to the river, a component of gw_sw_exchange | |
sw_to_gw | flux | The river losing to the aquifer, a component of gw_sw_exchange | |
gw_boundary | flux | Every other flux across the aquifer’s own boundary, such as a GHB or a well | |
sbl | flux | The sublimating share of evspsbl, a component of it rather than an addition | |
snm | flux | Liquid delivered from the snow module to the ground, rain passed through included, and not an outflow beside mrro | |
hfls | flux | Latent heat flux | |
hfss | flux | Sensible heat flux | |
hfg | flux | Ground heat flux at the actual soil surface, where a deeper flux is corrected for the heat stored above it | |
rlus | flux | Total upward longwave: emission plus reflected downward longwave | |
hfg_bottom | flux | Heat flux through the bottom of the specified soil layer | |
mrso | state | mm | Soil water storage (liquid and ice), reported as an absolute state rather than a tendency |
snw | state | mm | Snow water equivalent, identically zero and named in zero_stores for a model with no snow module |
canopy | state | mm | Canopy interception storage, identically zero and named in zero_stores for a model with no such store |
gw | state | mm | Groundwater storage below the soil column |
channel | state | mm | Water generated as runoff but not yet released by the model’s routing |
ts | diagnostic | K | Radiative surface (skin) temperature |
tsoil_layer | diagnostic | K | Mean temperature of the specified soil layer |
stage | diagnostic | m | Water-surface elevation on the case’s datum, read beside the row’s mean discharge (depth is stage minus bed_elevation_m) |
lwsnl | diagnostic | mm | Every non-solid component of snw, so snw minus lwsnl is the ice (part of snw, not additional) |
csnow | diagnostic | The pack’s cold content: the positive energy still needed to bring its ice to | |
reach_q_in | routing | Total inflow into the identified reach, in routing.csv | |
q_out | routing | Flow across the reach’s downstream boundary, in routing.csv | |
channel_storage | routing | Absolute water volume in the reach’s routed channel, in routing.csv | |
pr | forcing | Precipitation | |
tas | forcing | Near-surface air temperature in Celsius, unlike the kelvin of ts and tsoil_layer | |
pet | forcing | Potential evapotranspiration | |
rn | forcing | Prescribed net radiation, which the model accepts as the energy input to the reported budget when declared | |
rsds | forcing | Downward shortwave radiation |
Table 7. Full lexicon of the /io contract (schemas/lexicon.yaml, suite 0.1.0). Over a row of several model steps, a variable is a mean, an end-of-row value, or an instantaneous value, as stated in the repository.
| Name | Group | Units | Physical meaning |
rlds | forcing | W m | Downward longwave radiation |
huss | forcing | kg kg | Near-surface specific humidity |
ps | forcing | Pa | Surface air pressure |
sfcWind | forcing | m s | Near-surface wind speed |
abstr | forcing | mm d | Prescribed human withdrawal, net of return flow |
gwh | forcing | m | External hydraulic head behind the model’s gwex; declaring it asserts gwex responds monotonically to it |
gw_recharge | forcing | mm d | Direct recharge to the aquifer |
sw_stage_m | forcing | m | River stage the aquifer exchanges with |
q_in | forcing | m s | Prescribed river or upstream inflow, already a discharge and not multiplied by area or row length |
area_km2 | static | km | Catchment area the fluxes are converted over |
elevation_m | static | m | Catchment elevation |
latitude_deg | static | degrees north | Latitude |
main_channel_length_km | static | km | Length of the longest channel |
centroid_channel_length_km | static | km | Channel length from the outlet to the point nearest the catchment’s centroid |
soil_capacity_mm | static | mm | Water the soil column can hold |
canopy_capacity_mm | static | mm | Water the canopy can intercept (0 is a bare surface) |
degree_day_factor_mm_per_C_day | static | mm °C d | Snowmelt per degree-day |
baseflow_coefficient | static | d | Fraction of soil water drained to baseflow per day |
snow_threshold_degC | static | °C | Air temperature dividing snow from rain |
eps | static | 1 | Surface emissivity |
soil_layer_depth_m | static | m | Depth of the prescribed soil layer, from the surface |
soil_heat_capacity_areal | static | J m K | Effective areal heat capacity of that layer, already including the depth |
soil_temperature_initial | static | K | Initial temperature of that layer |
soil_solid_heat_capacity | static | J m K | Volumetric heat capacity of the soil solids |
soil_water_heat_capacity | static | J m K | Volumetric heat capacity of soil water |
soil_pore_air_heat_capacity | static | J m K | Volumetric heat capacity of the air in the pores |
soil_porosity | static | 1 | Volumetric porosity |
soil_water_content_initial | static | m m | Initial volumetric water content |
Table 1 (PDF p. 29).
| Name | Group | Units | Physical meaning |
aquifer_top_m | static | m | Elevation of the aquifer’s top |
aquifer_bottom_m | static | m | Elevation of the aquifer’s bottom |
aquifer_initial_head_m | static | m | Initial aquifer head |
aquifer_specific_yield | static | 1 | Drainable porosity above the water table |
aquifer_storage_coefficient | static | 1 | Storativity, not specific storage |
river_conductance_m2_per_day | static | Conductance between the river and the aquifer | |
river_bottom_offset_m | static | m | River stage minus riverbed elevation |
routing_network | static | - | Object whose reaches member lists the unique string IDs of the reaches |
reach_length_m | static | m | Length of the reach |
width_m | static | m | Width of the rectangular channel bed |
bankfull_depth_m | static | m | Depth at which the channel is full |
bed_elevation_m | static | m | Elevation of the rigid bed on the datum stage uses |
slope | static | 1 | Bed slope |
manning_n | static | s | Manning’s roughness |
cross_section_shape | static | - | Name of the cross-section’s shape, such as rectangular |
Table 16.
Probe catalog
Table 8 lists the 37 probes of suite 0.1.0 by law and family. Table 9 gives the authors of every probe as recorded in its definition file. Table 10 gives the proposer of every accepted model.
| Probe | Question the probe asks | Scored criteria | Seeds |
| Mass: closure | |||
catchment-closure | Catchment water balance closure under synthetic forcing | closure, et_plausible | 5 |
snowpack-mass-closure | Internal snowpack water-balance closure | closure, snowpack_response | 5 |
extreme-event-closure | Event-wise water balance closure under extreme rainfall | closure, event_water_closure | 5 |
ungauged-basin-closure | Water balance integrity across declared catchment attribute domains | closure, et_plausible | 12 |
| Mass: invariance | |||
| Probe | Question the probe asks | Scored criteria | Seeds |
| resolution-invariance | Integrated runoff must not depend on the step the weather was given at | resolution_invariance | 3 |
| area-invariance | The same weather on a 10 times larger catchment gives the same depths | invariance | 3 |
| time-origin-invariance | Identical forcing under aligned calendars gives identical water budgets | invariance, closure | 3 |
| groundwater-datum-invariance | Groundwater exchange is invariant to a common vertical datum shift | datum_flux_invariance, exchange_directions, groundwater_balance, exchange_components | 3 |
| spinup-cycle-invariance | A periodic climate must forget extra spin-up cycles | closure, spinup_cycle_invariance | 3 |
| Mass: response | |||
| warming-response | Warmer air over the same rain must mean less runoff, and cooler air more | response_sign | 5 |
| precipitation-counterfactual | Added or removed precipitation must be partitioned among the budget terms | closure, counterfactual_response, monotone_response | 3 |
| phase-counterfactual | Rain that would have been snow is still the same water | phase_invariance | 3 |
| response-nonnegativity | More water can never lower the flow, on any day | response_nonnegativity | 3 |
| antecedent-monotonicity | The same storm on a wetter catchment runs off more, and no more than the extra water | antecedent_monotonicity | 3 |
| extreme-rain | More rain cannot mean less runoff, nor more runoff than was added | monotone_response | 3 |
| Mass: dynamics | |||
| causality | A catchment cannot respond to rain that has not fallen yet | causality | 3 |
| dry-down | Without rain, runoff can only fall, and only stored water can drain | dry_down | 3 |
| steady-state | Under constant weather the catchment settles and the budget balances | steady_state | 3 |
| runoff-bounds | Runoff is bounded by rain minus demand below and by rain above | runoff_bounds | 5 |
| multi-decadal-drift | Slow budget-compensating storage drift over repeated climate | closure, total_storage_drift, et_plausible | 5 |
| Mass: groundwater and human | |||
| aquifer-recharge-ordering | Aquifer recharge preserves storage and exchange ordering | aquifer_recharge_ordering, groundwater_balance, exchange_components | 3 |
| exchange-response | A declared head-driven exchange must respond to its external head | closure, exchange_response | 5 |
| gw-sw-exchange-consistency | Groundwater-river exchange and storage close the water budget | exchange_components, groundwater_balance, exchange_directions | 5 |
| human-abstraction | Prescribed irrigation withdrawal must leave the budget | closure, human_abstraction | 3 |
| Energy: single budget | |||
| pet-consistency | Evaporation follows demand when the soil is wet and water when it is dry | demand_consistency | 3 |
| surface-energy-closure | Hourly surface budgets cannot hide canceling energy errors | energy_closure_by_phase | 5 |
| soil-heat-storage-consistency | Soil heat fluxes and layer temperature must agree | soil_heat_storage | 5 |
Table 8. Probe catalog for suite version 0.1.0. Criteria shared by most probes (state_bounds, non_degenerate, forcing_fidelity) are listed only when they are the sole criteria. Seeds are the number of generated cases per probe.
| Probe | Authors |
| mass/catchment-closure | Zhi Li |
| mass/snowpack-mass-closure | Jinlong Hu |
| mass/extreme-event-closure | Taiqi Lian |
| mass/ungauged-basin-closure | Shunan Zhou |
| mass/resolution-invariance | Zhi Li |
| mass/area-invariance | Zhi Li |
| mass/time-origin-invariance | Siavash Shams |
| mass/groundwater-datum-invariance | Zhenjiang Wu |
| mass/spinup-cycle-invariance | Kaihao Long |
| mass/warming-response | Zhi Li |
| mass/precipitation-counterfactual | Qingyi Yang |
| mass/phase-counterfactual | Zhi Li |
| mass/response-nonnegativity | Zhi Li |
| mass/antecedent-monotonicity | Zhi Li |
| mass/extreme-rain | Zhi Li |
| mass/causality | Zhi Li |
| mass/dry-down | Zhi Li |
| mass/steady-state | Zhi Li |
| mass/runoff-bounds | Zhi Li |
| mass/multi-decadal-drift | Bing Li |
| mass/aquifer-recharge-ordering | Zhenjiang Wu |
| mass/exchange-response | Songkun Yan |
Table 9. Authors of each merged probe, as recorded in probe.yaml.
| Model | Proposed by |
lstm | Zhi Li (CU Boulder) |
dhbv2 | Zhi Li (CU Boulder) |
wflow_sbm | Yuanhang Liu (Independent Researcher) |
summa | Yuanhang Liu (Independent Researcher) |
cwatm | Yuanhang Liu (Independent Researcher) |
lisflood | Yuanhang Liu (Independent Researcher) |
modflow6 | Yaji Wang (University of Illinois Urbana-Champaign) |
hymod | Xiaoyi Dong (China IPPR International Engineering Co., Ltd., SINOMACH) |
xaj | Xiaoyi Dong (China IPPR International Engineering Co., Ltd., SINOMACH) |
gr4j | Yuanhang Liu (Independent Researcher) and Li Zeng (The Hong Kong University of Science and Technology) |
topmodel | Yan Chen (Colorado State University) |
tank | Jiao Wang (University of California Davis) |
topkapi | Jiao Wang (University of California Davis) |
cfe | Xiaoyi Dong (China IPPR International Engineering Co., Ltd., SINOMACH) |
simhyd | Haoyang Su (Fudan University and Shanghai Innovation Institute) |
mcp1 | Yuan-Heng Wang (University of Arizona) |
mcp2 | Yuan-Heng Wang (University of Arizona) |
mcp3 | Yuan-Heng Wang (University of Arizona) |
mcp4 | Yuan-Heng Wang (University of Arizona) |
mcp5 | Yuan-Heng Wang (University of Arizona) |
raven | Yan Chen (Colorado State University) |
hype | Yan Chen (Colorado State University) |
prms | Yan Chen (Colorado State University) |
vic | Valmik Nahata (University of California San Diego) |
Table 10. Accepted model proposals and who proposed them. Five accepted proposals count as one probe toward authorship. The model labeled lstm here is listed in the repository under the identifier given in Appendix § 14.§ 3.
| Probe | Question the probe asks | Scored criteria | Seeds |
| radiation-consistency | Surface temperature and upward longwave must describe one surface | radiative_identity | 5 |
| Energy: cross budget | |||
| latent-heat-et-consistency | Latent heat and evapotranspiration must describe one evaporation | closure, energy_closure, flux_identity, et_plausible | 5 |
| evaporative-partition | Energy a drying surface stops evaporating must warm the air | partition_shift, closure, energy_closure, flux_identity | 3 |
| snowmelt-energy-water | Melt reported as water must be paid for out of the energy budget | closure, melt_energy | 5 |
| Momentum: routing | |||
| routing-conservation | A routing store holds runoff for as long as its hydrograph and no longer | routing_conservation | 3 |
| routing-lag-consistency | Runoff peak lag follows the catchment geomorphic travel-time scale | lag_time_bounds, scaling_monotonicity | 3 |
| Momentum: hydraulics | |||
| stage-discharge-monotonic | A stage reading rises with discharge, and loops the way a flood wave does | rating_monotonic, rating_loop | 3 |
| uniform-flow-friction-consistency | Steady discharge and stage satisfy the declared open-channel friction balance | uniform_flow_friction | 5 |
| froude-regime | A mild-sloped reach cannot outrun its own wave | froude_subcritical | 3 |
| wave-celerity-bounds | Transient flood-wave celerity responds to hydraulic state | wave_celerity_bounds | 3 |
Table 17.
| Probe | Authors |
mass/gw-sw-exchange-consistency | Yaji Wang |
mass/human-abstraction | Yuanhang Liu |
energy/pet-consistency | Zhi Li |
energy/surface-energy-closure | Han Wang |
energy/soil-heat-storage-consistency | Han Wang |
energy/radiation-consistency | Xin Lan |
energy/latent-heat-et-consistency | Changming Li |
energy/evaporative-partition | Changming Li |
energy/snowmelt-energy-water | Siddik Barbhuiya |
momentum/routing-conservation | Zhi Li, Yuanhang Liu |
momentum/routing-lag-consistency | Binlan Zhang |
momentum/stage-discharge-monotonic | Yuanhang Liu |
momentum/uniform-flow-friction-consistency | Mofan Zhang |
momentum/froude-regime | Yuanhang Liu |
momentum/wave-celerity-bounds | Jingzhi Chen |
Table 18.
| Model | Proposed by |
mhm | Valmik Nahata (University of California San Diego) |
shud | Yuanhang Liu (Independent Researcher) |
noahmp | Valmik Nahata (University of California San Diego) |
rvic | Jiao Wang (University of California Davis) |
mizuroute | Yuanhang Liu (Independent Researcher) |
Table 19.
Reference models
Table 11 lists the 18 exact references and the number of probes whose gate requires each to pass. Table 12 lists the 59 broken references, the defect built into each, and the probe and criterion that must catch it.
Table 11. Exact reference models and the number of probes whose gate requires them to pass. The first four are the common baselines. The others are written for the probes whose physics they exercise.
Table 12. Deliberately broken reference models (with the reference_ prefix omitted in column 1) and the criterion each is built to trip. Every row is a must-fail check in the acceptance gate. The gate requires each model to fail and to trip its declared criterion. It requires a model to trip no other criterion only in 2 cases, snow_bypass and fixed_celerity, where the control exists to separate 1 failure mode from another.
| Reference model | Defect built in | Probe | Criterion |
restless | The reference bucket with a recession that keeps its own 30-day clock, whatever the weather. | spinup-cycle-invariance, steady-state | spinup_cycle_invariance, steady_state |
saturating | The reference bucket with its daily runoff capped at 25 mm: exact for ordinary storms, flat for extreme ones. | extreme-rain | monotone_response |
shallow_rating | The exact bucket with a rating borrowed from another reach: its gauge reads Manning normal depth drawn for a section 5 times wider than the one the case declares, so the depth it reports is too shallow for this channel. | froude-regime | froude_subcritical |
slow_drift | An exact bucket whose reporter subtracts 0.012 mm d from runoff and accumulates the deficit in soil storage. | multi-decadal-drift | state_bounds |
snow_bypass | Deliberately broken snow model. | snowpack-mass-closure | closure |
snowless | Deliberately degenerate snow model. | snowpack-mass-closure | snowpack_response |
spatial_capacity | Attribute-dependent fixed internal-capacity fault. | ungauged-basin-closure | state_bounds |
spatial_et | Causal attribute-dependent ET fault. | ungauged-basin-closure | et_plausible |
spatial_forcing | Causal attribute-dependent forcing fault. | ungauged-basin-closure | forcing_fidelity |
spatial_frozen | Causal attribute-dependent frozen fault. | ungauged-basin-closure | non_degenerate |
spatial_gain | Causal attribute-dependent gain fault. | ungauged-basin-closure | closure |
spatial_loss | Causal attribute-dependent loss fault. | ungauged-basin-closure | closure |
spatial_negative | Causal attribute-dependent negative fault. | ungauged-basin-closure | non_degenerate |
steady_sink | reference_token_exchange with its noise removed: evaporation overstated by a constant 0.3 mm d (13% of precipitation over the record), declared as a steady regional inflow so the budget closes, with a head response times weaker than the positive controls. | exchange-response | exchange_response |
streamflow_only | Predicts discharge and nothing else, like most published rainfall-runoff models. | warming-response | response_sign |
stuck_router | The bucket with a unit hydrograph whose weights sum to 0.9: 10% of every day’s runoff enters the channel store and never leaves. | routing-conservation | routing_conservation |
sublimating | The bucket losing 40% of every snowfall to unreported sublimation. | phase-counterfactual | phase_invariance |
sublimation_blind | Fully coupled with a temperature-dependent latent heat, but it converts snow sublimation at the latent heat of vaporization instead of sublimation. | latent-heat-et-consistency | flux_identity |
thirsty | The bucket with evaporation that ignores demand: a fixed 2% of the soil store leaves every day, capped only by available water. | pet-consistency | demand_consistency |
token_exchange | reference_noise_sink with 1 line added: a head-driven trickle times weaker than the positive controls, folded into the declared exchange and soil storage so the budget closes. | exchange-response | exchange_response |
Table 37.
| Reference model | Defect built in | Probe | Criterion |
| two_head | A model with decoupled water and energy heads that close their individual budgets without agreeing on latent heat. | evaporative-partition, latent-heat-et-consistency | flux_identity, partition_shift |
| warming_free | The same snowpack, melting on the energy actually available and reporting its cold content honestly, but charging its surface budget for the fusion alone. | snowmelt-energy-water | melt_energy |
| wrong_roughness | Deliberately inconsistent steady reach. | uniform-flow-friction-consistency | uniform_flow_friction |
| wrong_slope | Deliberately inconsistent steady reach. | uniform-flow-friction-consistency | uniform_flow_friction |
| zero_exchange | Intentionally frozen zero-exchange reference. | groundwater-datum-invariance | datum_flux_invariance |
Table 20.
Tolerance and seed sensitivity
Table 13 varies the closure tolerance over 4 orders of magnitude and counts the evaluated models that would fail. The residual of the worst model, xaj, is 2.5%, and the verdict on the broken reference is unchanged for every tolerance below 15%. Table 14 shows the seed sensitivity that motivates running several seeds. The 20 seeds for catchment-closure and the 10 seeds for extreme-rain were drawn at random from a recorded generator, and the seed lists are included with the source of this article.
| Tolerance (% of precipitation) | Evaluated models failing (of 22) | reference_leaky |
| 0.01 | 4 | fails |
| 0.1 | 2 | fails |
| 0.5 | 1 | fails |
| 1 | 1 | fails |
| 2 | 1 | fails |
| 5 (adopted) | 0 | fails |
| 10 | 0 | fails |
| 20 | 0 | passes |
Table 13. Sensitivity of the closure verdicts to the tolerance. Entries are the number of the 22 evaluated models whose whole-record residual on mass/catchment-closure exceeds each limit. The broken reference reference_leaky has a residual of 15% by construction.
Evaluated models and reproducibility
Adapting a model
The repository documents 6 steps for adding a model. First, the proposer opens a model proposal and the maintainer accepts it. Second, the proposer decides which variables the model honestly emits. Third, an adapter translates between the contract and the native model. Fourth, a Dockerfile or a native entrypoint makes the run reproducible. Fifth, a contract check, ht verify-adapter, runs 1 seed on the evaluation window before any physics is scored. Sixth, the contributor runs the suite with ht run and submits the manifest, the adapter and the archived results. The common failure modes are padding the list of emitted variables, which turns an honest limit into a false claim, and declaring a forcing input that the adapter does not consume.
Standing of the evaluated models
Table 15 lists the probes passed over probes scored for every model, split by law.
Reproducibility
All numbers in this article derive from suite 0.1.0, archived on Zenodo, and from the results archive in the repository. The acceptance gate of every probe was re-run on 2 November 2026 with the command ht gate, and the standings of Table 15 reproduce from the archive. The models are listed in the archive under their repository identifiers. The LSTM of this article appears there as google_flood_forecast, version 0.1.0-828dfc5.2, with released weights, and it was run in a Docker container on 8 CPU cores. The mass-conserving perceptron checkpoints mcp1 to mcp5 correspond to epochs 27, 3, 619, 2707 and 2999 of 1 architecture. Versions of all other models are recorded in the archive.
References
- [1]Nans Addor, Andrew J. Newman, Naoki Mizukami, and Martyn P. Clark. The camels data set: catchment attributes and meteorology for large-sample studies. Hydrology and Earth System Sciences, 21(10): 5293–5313, 2017. doi: 10.5194/hess-21-5293-2017.DOI
- [2]ASCE Task Committee on Application of Artificial Neural Networks in Hydrology. Artificial neural networks in hydrology. I: Preliminary concepts. Journal of Hydrologic Engineering, 5(2):115–123, 2000.
- [3]Richard Bellman. Dynamic Programming. Princeton University Press, Princeton, NJ, 1957.
- [4]Tom Beucler, Michael Pritchard, Stephan Rasp, Jordan Ott, Pierre Baldi, and Pierre Gentine. Enforcing analytic constraints in neural networks emulating physical systems. Physical Review Letters, 126(9), 2021. doi: 10.1103/PhysRevLett.126.098302.DOI
- [5]K. J. Beven and M. J. Kirkby. A physically based, variable contributing area model of basin hydrology. Hydrological Sciences Bulletin, 24(1):43–69, 1979. doi: 10.1080/02626667909491834.
- [6]Keith Beven. A manifesto for the equifinality thesis. Journal of Hydrology, 320(1-2):18–36, 2006. doi: 10.1016/j.jhydrol.2005.07.007.DOI
- [7]Douglas P. Boyle, Hoshin V. Gupta, Soroosh Sorooshian, Victor Koren, Ziya Zhang, and Michael Smith. Toward improved streamflow forecasts: value of semidistributed modeling. Water Resources Research, 37(11):2749–2759, 2001. doi: 10.1029/2000WR000207.DOI
- [8]Leo Breiman. Random forests. Machine Learning, 45(1):5–32, 2001.
- [9]Peter Burek, Yusuke Satoh, Taher Kahil, Ting Tang, Peter Greve, Mikhail Smilovic, Luca Guillaumot, Fang Zhao, and Yoshihide Wada. Development of the community water model (cwatm v1.04) – a high-resolution hydrological model for global and regional assessment of integrated water resources management. Geoscientific Model Development, 13(7):3267–3298, 2020. doi: 10.5194/gmd-13-3267-2020.DOI
- [10]L. Ciarapica and E. Todini. Topkapi: a model for the representation of the rainfall-runoff process at different scales. Hydrological Processes, 16(2):207–229, 2002. doi: 10.1002/hyp.342.DOI
- [11]Martyn P. Clark, Dmitri Kavetski, and Fabrizio Fenicia. Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resources Research, 47(9), 2011. doi: 10.1029/2010WR009827.DOI
- [12]Martyn P. Clark, Bart Nijssen, Jessica D. Lundquist, Dmitri Kavetski, David E. Rupp, Ross A. Woods, Jim E. Freer, Ethan D. Gutmann, Andrew W. Wood, Levi D. Brekke, Jeffrey R. Arnold, David J. Gochis, and Roy M. Rasmussen. A unified approach for process-based hydrologic modeling: 1. modeling concept. Water Resources Research, 51(4):2498–2514, 2015. doi: 10.1002/2015WR017198.DOI
- [13]Martyn P. Clark, Richard M. Vogel, Jonathan R. Lamontagne, Naoki Mizukami, Wouter J. M. Knoben, Guoqiang Tang, Shervan Gharari, Jim E. Freer, Paul H. Whitfield, Kevin R. Shook, and Simon Michael Papalexiou. The abuse of popular performance metrics in hydrologic modeling. Water Resources Research, 57(9), 2021. doi: 10.1029/2020WR029001.DOI
- [14]James R. Craig, Genevieve Brown, Robert Chlumsky, R. Wayne Jenkinson, Georg Jost, Konhee Lee, Juliane Mai, Martin Serrier, Nicholas Sgro, Mahyar Shafii, Andrew P. Snowdon, and Bryan A. Tolson. Flexible watershed simulation with the raven hydrological modelling framework. Environmental Modelling & Software, 129:104728, 2020. doi: 10.1016/j.envsoft.2020.104728.DOI
- [15]Veronika Eyring, Sandrine Bony, Gerald A. Meehl, Catherine A. Senior, Bjorn Stevens, Ronald J. Stouffer, and Karl E. Taylor. Overview of the coupled model intercomparison project phase 6 (cmip6) experimental design and organization. Geoscientific Model Development, 9(5):1937–1958, 2016. doi: 10.5194/gmd-9-1937-2016.DOI
- [16]Dapeng Feng, Jiangtao Liu, Kathryn Lawson, and Chaopeng Shen. Differentiable, learnable, regionalized process-based models with multiphysical outputs can approach state-of-the-art hydrologic prediction accuracy. Water Resources Research, 58(10), 2022. doi: 10.1029/2022WR032404.DOI
- [17]Jonathan M. Frame, Frederik Kratzert, Daniel Klotz, Martin Gauch, Guy Shalev, Oren Gilon, Logan M. Qualls, Hoshin V. Gupta, and Grey S. Nearing. Deep learning rainfall–runoff predictions of extreme events. Hydrology and Earth System Sciences, 26(13):3377–3392, 2022. doi: 10.5194/hess-26-3377-2022.DOI
- [18]Hoshin V. Gupta, Harald Kling, Koray K. Yilmaz, and Guillermo F. Martinez. Decomposition of the mean squared error and nse performance criteria: Implications for improving hydrological modelling. Journal of Hydrology, 377(1-2):80–91, 2009. doi: 10.1016/j.jhydrol.2009.08.003.
- [19]Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. doi: 10.1162/neco.1997.9.8.1735.DOI
- [20]Pieter-Jan Hoedt, Frederik Kratzert, Daniel Klotz, Christina Halmich, Markus Holzleitner, Grey Nearing, Sepp Hochreiter, and Günter Klambauer. MC-LSTM: Mass-conserving LSTM, 2021.arxiv.org/abs/2101.05186
- [21]Kuo-In Hsu, Hoshin V. Gupta, and Soroosh Sorooshian. Artificial neural network modeling of the rainfall–runoff process. Water Resources Research, 31(10):2517–2530, 1995.DOI
- [22]George Em Karniadakis, Ioannis G. Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021. doi: 10.1038/s42254-021-00314-5.DOI
- [23]Anuj Karpatne, Gowtham Atluri, James H. Faghmous, Michael Steinbach, Arindam Banerjee, Auroop Ganguly, Shashi Shekhar, Nagiza Samatova, and Vipin Kumar. Theory-guided data science: A new paradigm for scientific discovery from data. IEEE Transactions on Knowledge and Data Engineering, 29(10):2318–2331, 2017. doi: 10.1109/TKDE.2017.2720168.DOI
- [24]James W. Kirchner. Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology. Water Resources Research, 42(3), 2006. doi: 10.1029/2005WR004362.DOI
- [25]Frederik Kratzert, Daniel Klotz, Claire Brenner, Karsten Schulz, and Mathew Herrnegger. Rainfall–runoff modelling using long short-term memory (lstm) networks. Hydrology and Earth System Sciences, 22(11):6005–6022, 2018. doi: 10.5194/hess-22-6005-2018.DOI
- [26]Frederik Kratzert, Daniel Klotz, Guy Shalev, Günter Klambauer, Sepp Hochreiter, and Grey Nearing. Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets. Hydrology and Earth System Sciences, 23(12):5089–5110, 2019. doi: 10.5194/hess-23-5089-2019.
- [27]Frederik Kratzert, Grey Nearing, Nans Addor, Tyler Erickson, Martin Gauch, Oren Gilon, Lukas Gudmundsson, Avinatan Hassidim, Daniel Klotz, Sella Nevo, Guy Shalev, and Yossi Matias. Caravan - a global community dataset for large-sample hydrology. Scientific Data, 10(1), 2023. doi: 10.1038/s41597-023-01975-w.
- [28]Christian D. Langevin, Joseph D. Hughes, Edward R. Banta, Richard G. Niswonger, Sorab Panday, and Alden M. Provost. Documentation for the modflow 6 groundwater flow model. Techniques and methods, book 6, chap. a55, U.S. Geological Survey, 2017.DOI
- [29]Zhi Li. HydrologicModels: lumped and semi-distributed conceptual models (FLEX variants). https://github.com/chrimerss/HydrologicModels, 2026.
- [30]Zhi Li, Changming Li, Siavash Shams, Han Wang, Qingyi Yang, Yuanhang Liu, Taiqi Lian, Xin Lan, Shunan Zhou, Binlan Zhang, Siddik Barbhuiya, Songkun Yan, Bing Li, Jinlong Hu, Yaji Wang, Zhenjiang Wu, Mofan Zhang, Kaihao Long, and Jingzhi Chen. HydroTuring: a conservation-law benchmark for AI hydrologic models, suite version 0.1.0. https://github.com/Flood-Lab/HydroTuring, 2026a. Published 2 November 2026. Archived at Zenodo; concept DOI 10.5281/zenodo.23874853.
- [31]Zhi Li, Songkun Yan, and Yang Hong. Do large language models improve hydrologic model calibration? Journal of Hydrology X, 31:100234, 2026b. doi: 10.1016/j.hydroa.2026.100234.DOI
- [32]Xu Liang, Dennis P. Lettenmaier, Eric F. Wood, and Stephen J. Burges. A simple hydrologically based model of land surface water and energy fluxes for general circulation models. Journal of Geophysical Research: Atmospheres, 99(D7):14415–14428, 1994. doi: 10.1029/94JD00483.
- [33]Göran Lindström, Charlotta Pers, Jörgen Rosberg, Johan Strömqvist, and Berit Arheimer. Development and testing of the hype (hydrological predictions for the environment) water quality model for different spatial scales. Hydrology Research, 41(3-4):295–319, 2010. doi: 10.2166/nh.2010.007.
- [34]D. Lohmann, E. Raschke, B. Nijssen, and D. P. Lettenmaier. Regional scale hydrology: I. formulation of the vic-2l model coupled to a routing model. Hydrological Sciences Journal, 43(1):131–141, 1998. doi: 10.1080/02626669809492107.




