betaFeedback

WorksPhysics and Astronomy

From FAIR to AWARE: Data Readiness for Agentic AI in the Earth Sciences

Abstract

Artificial intelligence in the Earth sciences is beginning to act: prototype agents already search archives, run simulations and draft analyses. Scientific data were curated for researchers who bring expert judgement; an autonomous agent cannot be assumed to supply it. Data can satisfy the FAIR principles and still mislead a machine acting alone. We propose that Earth-science data become AWARE (Actionable, Well-described, Auditable, Recipe-equipped, Epistemically labelled), an agent-specific requirements layer that extends FAIR, within a three-part agenda: AWARE data, bounded tools, layered validation. The ingredients exist across community infrastructure. We outline a staged, measurable path to adoption, intended to reduce silent error.

main_sn.pdf

Introduction

In a recent controlled evaluation by Zhou and colleagues [1], an autonomous artificial-intelligence agent was asked to analyse the water balance of large river basins. It loaded the supplied data, wrote the analysis code, and produced a polished figure with values within a few per cent of the reference, close enough to pass visual inspection. The figure was wrong: the agent had silently dropped the land-fraction weighting. No exception was raised, and nothing in the output revealed the omission. The necessary information was not missing: the agent received the same descriptions of the data that the tool-based condition used, and its script even located the model’s cell-area variable. Nothing, however, required the agent to act on that knowledge. When the same underlying large language models (LLMs) were instead restricted to a catalogue of validated, self-describing tools, the share of benchmark tasks solved correctly on the first attempt rose from about 5% to 82% (seven hydrology tasks, graded bit-for-bit against expert-verified references) [1]. The gain is consistent with broader evidence that LLMs equipped with validated tools and retrieval over curated knowledge outperform the same models operating alone [2, 3, 4]. The size of the effect, however, comes from a single study.

This Perspective examines the source of this discrepancy and the changes to data infrastructure needed to close it. The failure described above occurred in a controlled evaluation; as agentic AI enters routine scientific practice, the same silent failure mode will recur across many analyses. The evaluation also shows that validated tools prevent such errors; we argue that the knowledge those tools encode must ultimately reside with the data, in versioned, machine-actionable form.

The problem is urgent because adoption is running ahead of governance. A growing literature debates how generative artificial intelligence should be governed in science, from authorship and ethics [5, 6, 7] to assessment procedures [8]. Meanwhile, researchers already use LLMs throughout the workflow: drafting methods, searching the literature and the archives, processing data, and writing analysis code. Agentic AI is moving inside the scientific loop: LLM agents now plan and execute sequences of actions on a user’s behalf through natural-language interfaces.

Three concurrent advances drive this shift. The first is the maturation of machine-learning models of the Earth system itself: data-driven weather prediction has moved from a curiosity to systems that match or exceed selected operational benchmarks at a small fraction of the inference cost [9, 10, 11, 12, 13]. The approach has since spread to foundation models of the atmosphere and ocean, hybrid and climate-scale emulation, and kilometre-scale downscaling [14, 15, 16, 17, 18, 19, 20, 21]. The second is the emergence of LLMs tuned to geoscientific vocabulary and literature [22, 23, 24, 25, 26]. Together they mean that fast, callable models of the Earth system now exist for an agent to invoke the way it invokes any other tool.

The third advance is agentic orchestration. Whereas the first two advances improved existing components, orchestration changes how they are used: instead of answering questions, multi-agent systems plan, call tools, execute code in sandboxed environments, inspect their own figures, and correct their own errors across multi-step workflows. Such systems, most of them still research prototypes, already automate the analysis loop over archives at petabyte scale [28] [27] and ground themselves in curated knowledge graphs to reproduce published results [29]. Others discover and analyse data across multidisciplinary archives [30], deliver localized climate information through an openly released service prototype [31, 32], and unify Earth-observation analysis through an ecosystem of more than one hundred tools [33].

Large institutional programmes support this development. The European Union’s Destination Earth initiative launched its initial operational system, including the first two digital twins of the Earth system, in June 2024; a full digital replica is targeted for 2030 [34, 35]. The Helmholtz Association funds Earth-system foundation-model pilots [36]. In the United States, the National Science Foundation funds an institute for trustworthy artificial intelligence in weather, climate and coastal oceanography [37], and the National Oceanic and Atmospheric Administration (NOAA) operates a centre for artificial intelligence [38]; the European Space Agency (ESA) runs comparable programmes for Earth observation [39]. In one Scopus bibliometric sample, machine-learning publications on climate grew by more than two orders of magnitude between 2008 and 2022 [40]. These programmes invest chiefly in models, digital twins and platforms; the readiness of the data their agents will act on has attracted no comparable investment.

Systematic evaluations, however, show that these systems remain unreliable. On ScienceAgentBench, which comprises 102 data-analysis tasks extracted from 44 peer-reviewed publications in four disciplines, the best evaluated agent solved 32.4% of tasks given three attempts each [41]. On DSBench, 466 data-analysis tasks drawn from real data-science competitions, the best agent solved 34.1% [42]. More revealing than the failure rate is the nature of the failures. On TerraBench, which poses 403 tasks over heterogeneous Earth-system data (gridded fields, satellite imagery, geospatial context and simulator output) and grades tool selection and numerical accuracy separately, agents did far better at choosing the right tool than at producing numerically acceptable answers: more than 84% of numerical results fell outside tolerance, and the grounding of tool arguments (region, period, parameters) was a principal failure source [43]. That benchmark, at least, locates the failures in data handling rather than scientific reasoning; the broader benchmarks establish capability limits without resolving their cause. Much of the knowledge that would prevent such errors is not published; it is held within individual research groups: which datasets can be validly compared, which regridding method conserves the global integral of a variable, and which simulation versions carry known defects. Model scaling alone cannot guarantee access to knowledge that is unpublished, local, or specific to a dataset version.

These observations motivate the central argument of this Perspective: the orchestration layer is maturing faster than the data layer, and agentic systems amplify the weaknesses of the resources on which they operate. This assessment is informed by our own development and operation of agentic systems over Earth-science archives [30, 31, 32, 44]. We argue that agent readiness rests on three pillars: (1) data that carry their own expert context, formalized below as the AWARE requirements; (2) an infrastructure of composable tools and standard machine interfaces that agents can operate safely; and (3) layered validation, since full autonomy is not a realistic near-term goal: pre-execution code checks, runtime assertions and adversarial review to catch errors before release, and machine-readable provenance to keep released results re-executable.

The unconstrained agent of our opening example had none of these safeguards; the tool-restricted condition supplied the second pillar alone, and accuracy rose accordingly. Agent readiness requires all three pillars, and each depends on skilled human labour (curation, tool development, review) that must be credited and funded, a governance question addressed below. The remainder of this Perspective first examines why Earth-science data resist autonomous use, then develops the three pillars, shows that their components already exist within community infrastructure, and outlines a staged path to adoption.

Why Earth-science data are not ready for autonomous agents

Earth-science data can satisfy the FAIR principles (findable, accessible, interoperable, reusable) [45] and still be unusable by autonomous agents. Machine actionability is an explicit goal of FAIR, but the principles are deliberately general: they do not prescribe domain-specific validity constraints or executable handling instructions, and they assume a reader who supplies tacit knowledge wherever documentation is ambiguous. A fully autonomous agent is the limiting case of the “distant reuser” [46]: it has no way to consult the data creator. Some of what such consultation conveys is tacit and cannot be written down at all [47, 48]; this Perspective concerns the part that can be made explicit. Five structural properties of Earth-science data widen the gap between FAIR compliance and agent usability, and each motivates one of the AWARE requirements introduced in the next section, named in parentheses below.

The decoding burden (answered by Recipe-equipped). Earth-science data are stored in several distinct encoding families, each externalizing decoding knowledge differently. The Network Common Data Form (NetCDF) family, the backbone of model archives [49], is largely self-describing: its modern variants reside inside a hierarchical container (HDF5) whose header declares the compression filters and chunking in use. The header does not, however, supply the decoder itself: filters are plugins resolved from a registry outside the file, so an agent in an environment lacking the required plugin can fail on valid files; such failures are uncommon but difficult to diagnose. The GRIB format of operational meteorology is a stronger example: it is interpretable only through versioned code tables maintained by the World Meteorological Organization outside the file. Cloud-native formats such as Zarr expose metadata that agents can parse natively, but exist in multiple specification versions with extensible compression registries [50]; the same pattern holds elsewhere: seismic waveforms require external station metadata before instrument responses can be removed, and ocean profile quality-control flags are defined in reference tables maintained separately from the profiles they describe.

Semantics are voluntary (answered by Well-described). Community metadata conventions supply the semantic layer that gives arrays their scientific meaning. In climate science this role falls to the Climate and Forecast (CF) conventions, which define variable identity, physical units, calendars and grid descriptions and have been formalized in an encoding-independent data model [51]. But enforcement of such conventions is uneven across projects and repositories, and real archives contain a long tail of partially compliant data. A human reader routes around a missing or ambiguous attribute; an agent that assumes compliance mishandles it silently. Model grids provide a canonical example. The same physical variable may be delivered on a regular latitude–longitude grid by one model, on a curvilinear ocean grid with two-dimensional coordinate arrays by a second, and on an unstructured mesh by a third; if the metadata describing the grid is incomplete, all three still load as ordinary arrays, but any regional selection, spatial average or model intercomparison computed from them is quietly wrong. Agents will encounter such incomplete metadata in practice; robustness to partial compliance is therefore a core requirement.

Grids are silent-correctness traps (answered by Well-described, Recipe-equipped). Regular, rotated-pole, curvilinear and unstructured-mesh grids all decode into ordinary-looking arrays. Earth-observation imagery adds projection, datum and bounding-box semantics that agents must interpret correctly before any spatial operation is valid. Regridding a variable whose area integral must be preserved, using a method that does not conserve it, corrupts the budget without raising any exception; taking an unweighted average over a regular latitude–longitude grid overweights the poles; ignoring cell-area information, external land–sea masks or missing-value sentinels produces plausible numbers that are quantitatively wrong. The companion files that prevent these errors are themselves a weak link. The CF conventions can declare the dependency (through cell_measures and external_variables), but archives populate these attributes unevenly (in CMIP6, cell-area fields such as areacella are separate datasets, absent for some model–experiment combinations), and the convention specifies no resolution mechanism. In our audited sessions (28 incidents from four agentic systems operated by our group, catalogued in the Supplementary Information), an agent located the correct cell-area dataset and then did not use it, and a reanalysis land mask was borrowed from a climate model, contaminating the comparison along complex coastlines. Neither failure raised an error. These are the highest-severity failure modes for autonomous pipelines precisely because they are invisible in the output. Humans make some of these errors too; the difference lies in scale: an agent can repeat one mistake across a thousand analyses without collegial review.

Scale and structure (answered by Actionable, Auditable). The sixth phase of the Coupled Model Intercomparison Project (CMIP6), the most recent with a fully populated archive (the seventh is under way), was designed around 312 experiments across 23 endorsed sub-projects, with an archive volume estimated at roughly twenty petabytes [52, 53, 54]. The archive is published through the Earth System Grid Federation (ESGF), a deliberately decentralised system of dozens of independently administered nodes in which replication, versioning, authentication and node outages are constant operational concerns [54, 55]. Dataset identity is governed by a formal reference syntax and controlled vocabularies [56], and even the set of available variables and output frequencies is a formal, versioned object [57]. At this scale “download then process” is no longer viable: agents must default to lazy, chunked, server-side access through remote-access protocols, cloud-optimized stores and virtual aggregations [58].

Federation failures are particularly hazardous in combination with agent autonomy: when access to part of an ensemble fails mid-analysis, an agent may silently substitute other data. For exchangeable realizations of the same model this can be statistically harmless; substituting a different model configuration is not, and either way the change must be declared. In one audited session, an agent that lost access to a high-resolution configuration substituted an alternative and logged the change (incident D3, Supplementary Information); the log was correct, but the ensemble’s composition, and hence its meaning, had shifted. Without a declared substitution policy, such drift surfaces only in a post-hoc audit of the logs.

Epistemic class is invisible (answered by Epistemically labelled). A free-running CMIP6 projection, an ERA5 reanalysis field [59], an interpolated Argo profile and a satellite retrieval all decode into identical-looking multidimensional arrays, yet they are different kinds of scientific object. The day-to-day weather of a free-running simulation does not correspond to the weather that occurred on those dates, and a reanalysis blends a model with observations, so it is neither ground truth nor independent of models and carries assimilation artefacts and its own uncertainty. Nothing in the files records these distinctions, so no file checker can warn an agent that applies the wrong kind of data to a task. In our audited runs, a pipeline subtracted a reanalysis baseline from future model output and interpreted the difference as climate change. That difference, however, also contains the model’s systematic bias with respect to the reanalysis; the standard approach is to subtract the same model’s historical simulation. Both datasets were individually correct, convention-compliant and FAIR; the subtraction itself was the invalid operation, and it was caught only in review. Valid-use constraints must therefore travel with the data; in the AWARE checklist they appear twice, as the epistemic label that governs comparisons and as a separate machine-readable valid-use record.

From FAIR to AWARE data

The first pillar concerns the data itself. Three notions of data readiness must be distinguished: FAIR, AI/ML-ready and agent-ready; the third is the least developed. FAIR [45] establishes general findability, accessibility, interoperability and reuse, with machine actionability an explicit goal; it deliberately leaves domain-specific validity open. AI/ML-ready adds what a training pipeline needs: the Croissant format makes datasets directly loadable into ML frameworks [60], FAIR-for-AI extends the principles to models and benchmarks [61], and FAIR Digital Objects reframe data as typed, operable units [62, 63].

Agent-ready adds what neither of the first two provides: the expert context that a human silently supplies at the moment of reuse, externalized into machine-actionable form so far as it can be stated, bound to the data by a persistent identifier and resolvable without human intervention. Such context is rarely recorded: in a survey of 4,041 data papers, collection and source information was commonly documented, while generalization limits appeared in 8% of papers and maintenance policies in under 1% [64].

The theoretical basis is the “data creators’ advantage” [65]: effective reuse depends on tacit knowledge that humans obtain by collaborating with data creators. A fully autonomous agent cannot rely on timely collaboration, so the context must ship with the data, as “Datasheets for Datasets” argued for human consumers [66], but as structured assertions rather than prose. Where the assertions run out, the right agent behaviour is to stop and escalate to the data steward; the contract should make such escalation the exception by answering the recurring questions once for all agents. The cost of leaving this context unpublished is illustrated by one archive, where supporting natural-language discovery required rewriting 1,768 terse variable, experiment and model descriptions into self-contained plain-language entries [44]. Much of that work reconstructed, and partly had to infer, context that data producers are best placed to publish once. This enrichment can be partly automated: LLMs can draft the entries for experts to review.

We refer to data that close this gap as AWARE:

Actionable: every reuse-critical element resolves to a machine endpoint, never to a human-only web page; licences and access are machine-negotiable;

Well-described: semantics are checked rather than merely recommended, with standardized variable names, machine-parseable physical units, calendars, grid and coordinate-system topology, masks and cell areas, plus a structured register of known artefacts;

Auditable: versioned, checksummed identity with machine-readable lineage following the W3C PROV standard and a citation bundle linking the exact version to the literature that used it;

Recipe-equipped: a deterministic, executable recipe for loading and correctly handling the data ships with the data itself; conceptually, a recipe is a skill the dataset supplies to any agent: a packaged, validated competency that agents invoke rather than reimplement;

Epistemically labelled: the dataset declares what it is (free-running model output, assimilated reanalysis, satellite retrieval, in-situ observation) and what it may validly be compared against; CF’s free-text source attribute anticipates the class label, but agents need it as a controlled, checkable vocabulary together with the comparison rules.

The acronym summarizes the requirement: data that carry their own operating instructions, history and limits are a necessary part of any architecture for autonomous use, though not sufficient on their own. Together the five requirements address the structural problems of the preceding section, several of which are addressed jointly, as the labels there indicate. AWARE does not replace FAIR: FAIR sets general principles, and AWARE adds the agent-specific requirements a domain must specify before a reuser who cannot ask questions can act on the data.

The requirements are not specific to gridded data; gridded data are in fact the more tractable case. Model output and reanalyses have regular structure, and their conventions, however unevenly enforced, at least exist. Observational data are different. They come from thousands of instruments, sensors and campaigns, and in many communities each provider packages the results as it sees fit before uploading them to a database. There is often no convention an agent could even assume, so every structural choice must be guessed from whatever metadata happen to be present. A human expert resolves such heterogeneity by inspection; for an agent, each guess is a silent-failure risk, and at archive scale repeated guessing also wastes computation and energy. Observational data therefore stand to gain the most from AWARE: a shipped load recipe and declared conventions replace guesswork exactly where no shared convention exists.

CMIP6 illustrates both the extent and the limits of current metadata enrichment. The infrastructure was designed to connect each published dataset to systematic documentation of the model and experiment that produced it [67], to a dedicated citation service for evolving data [68], and to an errata service that tracks known issues. But the documentation link embedded in a dataset resolves to a page written for human readers, not to a machine-actionable document. Context held on a separate service is, moreover, accessible only while that service remains online; when it is unavailable or discontinued, the agent loses the context with it. For an agent to act on the knowledge that a specific simulation carries, say, a known sea-ice defect over a particular period, that knowledge must resolve as a structured assertion (affected component, severity, temporal bounds, corrective action) rather than as narrative text.

The hardest ingredient of AWARE data, expert judgement about fitness for purpose, is already produced at scale, but it is published in forms an agent cannot act on. The European Space Agency’s Climate Change Initiative, for example, publishes systematic product validation and intercomparison reports for its essential-climate-variable datasets [69], and comparable evaluation functions operate at other major providers. These assessments contain the tacit-context layer an autonomous agent needs, but they are published as prose, figures and reports rather than as version-bound, machine-validated assertions. Recast as structured assertions bound to the dataset identifier, they would let agents select data by quality and use case rather than by name alone; the translation requires scientific judgement and governance. The situation is general: an archive that invests in quality assessment, errata tracking or user guidance already possesses the raw material of AWARE data and lacks only a machine-readable rendering of the knowledge it curates.

Fitness for purpose also has a statistical dimension that current metadata rarely states: monthly means cannot support daily-exceedance metrics, and small single-realization ensembles may be underpowered to resolve scenario differences in regional extremes, yet nothing warns an agent which class of question a dataset can answer; the dependence runs through the signal-to-noise ratio: the same single realization that answers a question about the mean state may be useless for regional extremes. A register of such statements could become arbitrarily detailed; a workable contract will start from the few classes of question that recur (means, trends, extremes) rather than aim for completeness.

The same holds for the record of who has used a dataset and how. Standards for dataset–literature links, provenance description and research-object packaging together form the vocabulary of a citation bundle [70, 71, 72], yet today an agent cannot generally recover a complete, version- and variable-level path from a dataset identifier to the papers that used it, the methods they applied, and the caveats they reported. Agentic systems that need this grounding currently build it themselves, wiring corpora of thousands of publications into every analysis [29, 44]. A community knowledge graph binding dataset identity to provenance and linked literature at the granularity of individual variables and versions would therefore be a productive investment. Grounding of this kind also guards against fabricated or misidentified datasets: in our audited runs, verifying citations against retrieved sources caught fabricated DOIs (Supplementary Information).

No existing catalogue plays the role of the authoritative contract. The SpatioTemporal Asset Catalog serves Earth-observation scenes, and faceted model-archive catalogues serve gridded output; the contract must add typed, validatable records on top of either. Two gaps recur. First, discovery metadata rarely states, as a consistent typed field, what a dataset is good for and what it is not; humans carry that knowledge, and in practice agent interfaces end up reconstructing it by hand. Second, licences illustrate the Actionable gap. Terms are named in metadata, but acceptance remains a per-dataset click on a website, which halts a fully autonomous retrieval until a human intervenes. Designing machine-readable terms with a delegable, auditable acceptance path is tractable, provided acceptance remains attributable to a responsible person or institution. Box 1 condenses the requirements; Supplementary Note 1 sketches a minimal machine-readable contract.

Box 1 | The AWARE checklist for agent-ready Earth-science data (implementation of the first pillar)

A dataset is AWARE when a machine-actionable contract, bound to a versioned persistent identifier, provides:

  • □ Resolvable identity & version (Auditable): PID resolves to a specific immutable version with checksum; replica and node state discoverable; “latest” is never implicit.

  • □ Self-contained load recipe (Recipe-equipped): format, required decoding components, chunking layout and access protocol declared machine-readably; a reference invocation with its expected output lets any agent self-test the recipe.

  • □ Checked semantics (Well-described): every variable carries a resolvable standard name, UDUNITS-parseable units, and explicit calendar/time reference; existing convention checkers (CF-checker, PrePARE) already automate this at publication time.

  • □ Grid & operation contract (Well-described): grid topology declared (regular, rotated, curvilinear or unstructured mesh; map projection and datum for Earth-observation data); cell areas and land–sea masks present, or resolvably referenced, when the declared operation requires them; the invariant each operation must preserve stated per variable and declared purpose (a budget calculation must conserve the area integral; a quick-look map need not), with the safe choice as default and faster paths as logged overrides.

  • □ Epistemic class label (Epistemically labelled): model output / assimilated reanalysis / satellite retrieval / in-situ observation, with ensemble

semantics, declared uncertainty information, the classes of question the dataset can support, and flags for non-independent validation and invalid cross-comparison.

  • Known-artefacts register (Well-described): drift, spin-up shocks, instrument degradation, errata as structured flags with temporal/spatial bounds and corrective actions, not prose.

  • Provenance + citation bundle (Auditable): lineage in the W3C PROV standard plus DataCite/Scholix links binding the exact version to the literature that used it; contract assertions carry author, date and expiry.

  • Valid-use & licence (Actionable): machine-readable intended/discouraged uses and a licence permitting (or denying) autonomous retrieval; no click-through EULAs.

  • No human-only endpoints (Actionable): no reuse-critical information is available solely through pages written for human readers.

The agent-operable tool chain and infrastructure

AWARE data are necessary but not sufficient: the second pillar is a toolchain and infrastructure that agents can operate, and here the Earth sciences already have much of what is required (Figure 1). Decades of community development have already produced an analysis stack that embodies the Unix philosophy: small, composable, self-describing tools connected by a universal interface. Single-purpose command-line operators perform standard transformations and report errors as concise text; shared, labelled array representations let libraries for climate indices, regridding, meteorology and geospatial processing interoperate without bespoke glue [73, 74, 75, 76]; parallel back-ends supply scale; and equivalent stacks exist in every subfield. Self-describing metadata plays the role that plain text plays in Unix: the shared interface that makes composition checkable.

The layered agent-operable infrastructure stack

Figure 1. The layered agent-operable infrastructure stack. Layers 1–2 implement the data pillar, layers 3–5 the toolchain pillar, and layers 6–7 the trust pillar. The agent’s reason–act–observe loop spans layers 2–7, issuing tool calls downward and receiving verifiable observations upward; results pass a risk-based human review before release. Abbreviations: CDO, Climate Data Operators; NCO, netCDF Operators; MCP, Model Context Protocol; xarray, xclim, xESMF, cf-xarray, MetPy, GDAL and ObsPy are community analysis libraries.

Studies of software agents show that how the working environment is presented to an agent strongly influences its success. Three interface properties recur. Agents perform better when every step returns concise, observable feedback they can verify before proceeding [77]. They perform better when interfaces consist of a few simple, well-separated commands rather than an unconstrained terminal [4]. And they adapt as software evolves when up-to-date documentation can be retrieved on demand [78]. Well-chosen tools from the community stack come close: with pinned versions, bounded operations are rerunnable within declared tolerances, self-documenting, and verifiable step by step against the data’s own metadata. Agent-generated monolithic scripts have none of these properties: a several-hundred-line configuration generated in one pass raises the chance of error, a failure deep inside an opaque pipeline cannot be attributed to its cause, and intermediate states are invisible to the agent’s reason–act–observe loop. A practical design rule for institutions follows: expose capabilities as small, schema-validated operations, and treat large curated workflows as single pre-validated tools that agents invoke whole rather than attempt to rewrite.

The exposure layer. The stack still lacks standardized exposure: a machine interface through which an institution offers its data and tools to any agent, once, instead of writing bespoke integrations for each. The required properties are protocol-independent: typed, schema-validated functions with self-describing documentation; structured context resources; server-side authentication so credentials never enter an agent’s context; and pointers with server-side computation rather than multi-gigabyte payloads. Open standards implementing this pattern now exist (a prominent current option is the Model Context Protocol, an open standard governed since December 2025 by the Agentic AI Foundation under the Linux Foundation [79, 80]), and their general-purpose ecosystems are growing rapidly, yet little of that growth targets Earth-system data or high-performance computing. Early MCP servers exist for batch schedulers and for NASA Earthdata holdings. But as of August 2026, to our knowledge, no institutionally governed, community-standard agent endpoint serves the federated climate archives or the major model catalogues; this is the gap the Earth-science community is well placed to fill.

In our own early agent interfaces to major archives, much of the engineering effort went not into moving data but into reconstructing, validating and labelling metadata that the archives could have stated about themselves. A federated-archive agent endpoint would expose faceted search, subsetting and server-side computation as a handful of typed verbs; whichever protocol carries them is an implementation detail the community can change later. The protocol layer is not a trust layer: tool descriptions and dataset-supplied recipes are untrusted inputs, so deployments need signature verification, least-privilege scopes, cost ceilings and separation of data from instructions [79].

Context served alongside the data. Exposure need not stop at raw arrays; three further services, hosted by the archive itself, extend what an agent can do safely. The first is curated retrieval indices: searchable indices over the literature, documentation and assessment reports that concern an archive’s holdings. Offered as context resources, these ground an agent’s reasoning in vetted domain sources rather than an open web search, an approach agentic systems already use to suppress fabricated claims [29]. The second is precomputed products served as tools: climatologies, anomalies and standard indices computed once, validated by experts, and offered as fast queryable operations, so that an agent answers routine questions without re-processing petabytes or re-deriving, and possibly corrupting, validated quantities. The third is tabular views: compact summary tables can be cheaper for an LLM to inspect, and easier to validate, than raw multidimensional arrays, so serving them (station inventories, regional means, event catalogues) alongside the arrays gives agents a representation they can verify at little cost.

Agents on high-performance computing systems. Earth-system science runs on batch-scheduled clusters (workload managers [81], environment modules, parallel file systems) at national climate-computing centres (the European sites that host Destination Earth are one example). Three patterns make this world agent-operable. Job templates as tools: instead of letting an agent author a raw submission script with hallucinated queue names and invented software environments, expose validated, parameterized templates with schema-constrained typed fields, converting open-ended generation into a bounded function call. Immutable containers: Apptainer/Singularity [82] gives reproducible, hash-identifiable environments without root. Asynchronous orchestration: batch queues break the tight reason–act–observe loop, so agent frameworks must sleep on job identifiers and wake on completion callbacks, with quotas, allowlists and audit trails that treat agent-issued jobs as a new security surface.

Guardrails, validation and reproducibility

The third pillar is trust: everything above accelerates discovery only if the community can rely on what agents produce, and full trust without a human in the loop is not currently achievable. Language models are optimized for fluency rather than physical truth, and cannot be relied on to correct their own reasoning without external feedback [83]. When fully autonomous “AI scientist” systems have been audited, the audits found the familiar pathologies of careless research (favourable benchmark selection, evaluation leakage, misused metrics, outright fabrication) alongside a high rate of plain coding failure [84, 85]. The realistic target is therefore not blind autonomy but trust engineered in layers, with risk-based human checkpoints: routine bounded steps run automatically, while costly, state-changing or legally consequential actions, and final scientific release, require human approval.

Table 1 summarizes the failure modes an agent must be guarded against. The community’s existing validation infrastructure operates mostly on files: compliance checkers verify that a dataset’s metadata conforms to conventions before publication, and curated evaluation frameworks embed correct analytical behaviour [86], but they help only if the agent chooses to call them. Unit-aware libraries and workflow tests validate parts of a computation. No shared, agent-facing layer yet validates the analysis code an agent writes on the fly, which is where the failures of Table 1 arise. Closing this gap is a tractable engineering target, because a useful subset of the physical invariants is already formalized in community conventions and library semantics; the rest requires new domain assertions. A code-checking layer does not need to invent climate knowledge, only to enforce existing and newly encoded standards on code rather than on files. Data delivery needs the same shift of perspective. For an autonomous pipeline, a retrieval status of “successful” is a claim to verify, not a fact. A machine-readable delivery manifest, stating what was requested, what was delivered and what was dropped, would eliminate a whole class of silent partial deliveries.

Our recently proposed architecture implements such a layer [44]; Box 2 generalizes its five components. A design choice not itemized in Box 2 is the injection of runtime summaries, which require the LLM to reason about the statistics of every array it produces rather than about arrays it has never inspected. The layers reduce risk; they do not prove correctness. Several of them were added after behaviour we observed during development: agents confronted with a failed download filling model-like variables from random-number generators, and reference lists containing plausible-looking DOIs that never appeared in any retrieved source. Our checkers now treat both as hard stops, and every citation is verified against the sources the agent demonstrably saw. A related design rule from our operational experience is that “unknown” must be a legal answer: quantitative estimates an agent acts on should carry explicit reliability labels, because a confident wrong estimate is worse than an acknowledged gap.

Two findings from the broader literature constrain the design. Debate among agents is unreliable on its own: in a deliberately adversarial experiment, a single persuasive agent lowered group accuracy substantially while increasing consensus on wrong answers [87], so verification must be grounded in external tools, re-execution and assertions that fail closed. And since LLM inference is not exactly reproducible even at nominally deterministic settings [88], the right target is not a bit-reproducible agent but a rerunnable emitted artefact: code, computational environment and data versions that on re-execution reproduce results within declared numerical tolerances.

Box 2 | The agentic validation and guardrail pipeline (implementation of the third pillar)

  1. Pre-flight static analysis: a code linter gates agent-written code before execution, flagging unweighted spatial averages, regridding that violates a declared conservation requirement, cross-calendar arithmetic, mismatched coordinate conventions, reductions over undecoded missing values, and hard-coded “result” arrays. Rejected code returns a structured explanation for the agent to repair.

  1. Validated primitives: correct behaviour is the default; agents are steered to unit-aware, mask-aware, calendar-aware library calls (xclim Indicators, xCDAT, cf-xarray), with unsafe paths requiring an explicit, logged override.

  1. Runtime assertions: injected post-conditions (conservation to tolerance after regridding, no NaN inflation, dimensional consistency, physically plausible bounds) returning structured tracebacks the agent can act on.

  1. Adversarial verification, tool-grounded: independent reviewer agents from different LLM families critique method and output, anchored in re-execution and assertion results (not voting alone), iterating to an auditable verdict.

  1. Provenance capture: the entire run is serialized as a machine-readable provenance package following community standards [71, 72]: data identifiers and versions, computational environment, generated code and tool-call log, submitted alongside the result.

Provenance links agent-run analyses to the publication system. Journals and ethics bodies already agree that AI tools cannot be authors and that accountability stays with humans [5, 6, 7]; Nature Geoscience has explicitly called for human oversight in an editorial [89]. The unresolved question is what peer review becomes when an agent ran the analysis. In a controlled experiment, an LLM-based classifier detected planted methodological problems with 55% accuracy when given the paper alone, rising to 82% when the execution logs and generated code were supplied as well [84]. Existing data- and code-availability mandates should therefore extend, for agent-run analyses, to the full machine-readable provenance record together with the validation report. The reviewer’s question then shifts from “does this methodology sound plausible?” to “does the analysis re-execute, and did the physical checks pass?”, with one caveat: a faithfully re-executed pipeline reproduces its own bugs. Provenance must therefore travel with the invariant checks of Box 2: provenance establishes how a figure was made, and the validation layer establishes that the computation respected physical constraints.

Existing infrastructure and a staged path to adoption

None of the three pillars requires invention from scratch: the community already operates most of the needed substrate; the missing element is a semantic and exposure layer that turns platforms into agent endpoints. That layer is lighter than a new platform, but the semantic and governance work it contains is substantial, as the discussion below makes clear. Supplementary Table 1 reads the existing ecosystem (Pangeo, the ESGF federation, Destination Earth, commercial Earth-observation clouds, the Argo and ICOS observational networks, PANGAEA, and domain services beyond climate) as a set of templates. The pattern is consistent: every platform provides something an agent needs; none yet provides the contract of Box 1 or a standard agent endpoint. Closing this gap is an engineering task that can proceed incrementally, beginning with a single well-chosen use case. Stages 1–4 of the road map below amount to a small end-to-end demonstrator of all three pillars that a single data centre could host.

A six-stage adoption road map

  1. Pick one high-value use case. A frequently repeated, error-prone, checkable task: a regional multi-model mean of near-surface temperature anomalies from CMIP6, masked to land; or response-corrected seismic waveforms for a given event. Threshold to advance: a named domain scientist supplies a gold-standard manual result and agrees the task is worth automating.

  1. Draft an AWARE pilot contract for one dataset. Apply Box 1 to a single reference dataset: write the executable load recipe, enforce grid, unit and calendar semantics, stand up the known-artefacts register, and assemble the citation bundle. Do not build a platform.

  1. Insert the guardrails. The Box 2 pipeline sits between the agent and any execution from the start: pre-flight code analysis, validated primitives, runtime assertions, tool-grounded adversarial review, provenance capture. No stage of adoption runs the agent unguarded.

  1. Expose the dataset through a standard machine interface. A small set of typed operations (dataset search, variable loading, valid regridding, citation retrieval) plus context resources, with server-side authentication and read-only defaults, over whichever open agent-interface standard the institution prefers. Threshold: an off-the-shelf agent, running through the guardrail pipeline, completes the Stage-1 task autonomously; scientific release of the result remains subject to human review.

  1. Measure before scaling. Task success against the gold standard; the silent-error rate of wrong-but-plausible outputs, the metric that matters most; reproducibility (the emitted artefact re-executes within declared tolerances, with the same citation bundle); time-to-result versus manual; and energy per task. Threshold: silent-error rate below an agreed bound and reproducibility of the artefact demonstrated on the pilot.

  1. Scale the template, not the tool. Replicate the pattern (recipe + semantics + artefacts register + citation bundle + agent interface + guardrails) to adjacent datasets; publish the schema as a community standard; federate agent endpoints across archive nodes.

Governance enablers

Four design choices determine whether such a road map can be sustained. Credit: enriching a dataset toward AWARE status is skilled scientific labour. The FORCE11 data-citation principles [90], the software-citation principles [91] and the FAIR principles for research software [92] establish that such work merits formal citation and credit; we recommend, further, that it be supported by sustained rather than project-grant funding. The citation bundle makes the credit automatic. Data management is notoriously hard to fund, and agent-ready archives will not emerge as a side effect of research grants; they need recognized, sustained stewardship budgets. Precedent exists: major providers already run dataset evaluation and quality control as funded services, and part of the cost displaces support effort archives already spend answering the same reuse questions by hand. Candidate conveners for a community standard include the WMO, WCRP, the ESGF federation, OGC and the Research Data Alliance.

Openness: closed compute platforms can support repeatability within the service while limiting portability and independent inspection. Open stacks (Pangeo, openEO [93], open weights for data-driven models) keep analyses independently re-executable, and the leading agent-interface standards are themselves neutral, openly governed infrastructure.

Energy: the footprint of large AI is real and quantifiable [94, 95, 96] and should be measured and published per task. Data-driven forecast emulators can be far more energy-efficient than the physics-based models they complement: in one case study of a year of operational ensemble forecasting, training included, the saving ranged from about twenty-fold to more than a thousand-fold depending on the emulator [97]; whether end-to-end agentic workflows, including LLM inference, retries and validation, yield a net reduction is a hypothesis to be measured per task, not a conclusion.

Equity: we recommend that agent-ready endpoints remain free at the point of use, with compute located near data, so that researchers without exascale resources, above all in the Global South, participate as equals rather than as consumers; the economics of sustaining such services deserve open discussion.

Open questions and limitations

AWARE is a proposal, and several of its design questions remain open.

  1. Unit of conformance. The contract is layered rather than monolithic: a load recipe is version-specific, a regridding constraint is variable-specific, fitness for purpose is task-specific, and authentication belongs to a service; a workable profile must state which assertions live at which level.

  1. Grading. Conformance is graded rather than binary: the road map above implies maturity levels, and a minimum profile with optional extensions, together with a designated assessor, still needs definition.

  1. Relational safety. No dataset is safe for all autonomous uses; the accurate formulation is “safer for specified operations under declared constraints”.

  1. Ownership. The assertions need owners: translating curated prose into structured artefact registers and valid-use rules requires scientific judgement, review, versioning and dispute resolution, not mechanical conversion.

  1. Security. Executable recipes are a software-supply-chain surface and need signing, dependency pinning and sandboxing.

  1. Restrictive by design; For the AWARE approach to work well, consuming agents should be instructed to only use the datasets for the intended uses and strictly follow the available processing recipes. This ensures correct usage of datasets that have complete metadata, but might exclude usage of datasets that are missing

some of the aspects. For instance, assessments of fitness-for-purpose are difficult to produce consistently.

AWARE also overlaps with adjacent efforts: Datasheets [66], Croissant [60], FAIR Digital Objects [62], FAIR Implementation Profiles [98] and emerging AI-readiness frameworks [99, 100]. Much of AWARE deliberately reuses what these provide; its distinctive addition is the machine-actionable encoding of task-validity constraints for autonomous scientific action, wired directly to runtime guardrails. The framework is falsifiable: hold the agent, model and tools constant, compare a FAIR baseline with an AWARE-enriched version on held-out tasks, and measure task success, silent-error rate, abstention quality, authoring cost and the false-positive rate of the guardrails.

Outlook

Two trajectories can be anticipated. An institution that neglects data readiness risks directing unconstrained agents at raw, under-described archives. Such agents can mix units, omit masks, violate conservation in regridding and compute over errata-flagged data, without any of these operations raising an error. The resulting outputs will be fluent, plausible and physically wrong; they will be difficult to trace and to reproduce, and the institution’s data weaknesses will be amplified rather than offset by automation.

An institution that invests in data readiness, by contrast, will wrap its high-value datasets in AWARE contracts, expose them through guarded standard interfaces, gate every agent-written analysis behind the validation pipeline of Box 2, and attach a provenance record to every figure. Its researchers will shift from writing routine data-handling code to supervising auditable agent workflows over large archives; every result carries its citation bundle; curation work is credited; reviewers can re-execute analyses instead of inferring what was done. Each enriched dataset lowers the cost of enriching the next, so the approach scales incrementally.

More powerful AI will make parts of this agenda unnecessary. Larger models with longer context windows will read more documentation, write better code and recover from more of their own errors, and coding-failure rates on benchmarks should continue to fall. What scaling cannot supply is knowledge that exists nowhere in machine-readable form: the version-specific defect, the unpublished mask convention, the valid-use boundary a provider never wrote down. Readiness also changes the economics of autonomy. With the rules stated explicitly, correct area weighting no longer depends on a model’s ability to infer unstated conventions, and a small, open, locally run model becomes sufficient, at a fraction of the energy and cost; AWARE data thus widen the choice of usable models.

The decisive factor is not the capability of any single LLM but whether the community engineers an ecosystem in which silent failure is systematically guarded against, figures are traceable by software to their data and code, and physical constraints live in the infrastructure itself. The components exist: the FAIR principles established findable archives, cloud-native formats provide scalable access, the composable toolchain supplies safe analytical primitives, and open agent-interface standards provide the connection layer. What remains is the integrating layer specified in this Perspective.

In such an ecosystem, the agent of our opening paragraph would have received the weighting rule as part of the dataset’s own recipe; its code would have been gated by a static checker that flags an unweighted average; and its figure would have carried a provenance record a reviewer can re-execute. None of these layers guarantees detection, but together they make it unlikely that the omission that opened this Perspective would have survived to publication. The FAIR principles made Earth-science data findable and accessible, for humans and machines alike. The step to AWARE would allow those data to be handed, with declared limits and auditable safeguards, to machines that act on them. Its value should be judged by a measurable standard: fewer silent errors, and better-timed abstention, across real Earth-science tasks.

Acknowledgements

We acknowledge the continuous support by the Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research. We also thank the Helmholtz Association with its (data) infrastructure investments as well as all direct and indirect funders (DFG, BMFTR, EU). This work was supported by the European Union’s Destination Earth Initiative and relates to tasks entrusted by the European Union to the European Centre for Medium-Range Weather Forecasts implementing part of this Initiative with funding by the European Union. This work is also supported by project S1: Diagnosis and Metrics in Climate Models of the Collaborative Research Centre TRR 181 “Energy Transfer in Atmosphere and Ocean”, funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation, project no. 274762653). It is also supported by the EERIE project (Grant Agreement No 101081383), funded by the European Union, and by the TerraDT project (Digital Twin of Earth system for Cryosphere, Land surface and related interactions), which has received funding from the European Union’s Horizon Europe research and innovation programme under Grant Agreement no. 101187992. Views and opinions expressed are those of the authors only and do not necessarily reflect those of the European Union or the European Climate Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor the granting authority can be held responsible for them.

Use of AI tools

During the preparation of this manuscript, the authors used Claude (Anthropic) and ChatGPT (OpenAI) as writing-assistance tools for language refinement and feedback on draft text. All AI-assisted suggestions were critically reviewed and edited by the authors, who take full responsibility for the content of the manuscript.

Author contributions

I.K. and D.P. contributed equally to this work and are joint corresponding authors. All authors contributed to the conception of the Perspective and to the writing and revision of the manuscript, and approved the final version.

Competing interests

The authors declare no competing interests.

Data availability

No new geophysical datasets were generated or analysed for this Perspective. The operational incidents referred to in the text are documented in a curated incident catalogue, compiled from the authors’ contemporaneous working records, and provided as Supplementary Information.

Code availability

Code availability. No new code was developed for this Perspective. The agentic systems discussed are described in the cited publications.

supplementary_information.pdf

This document contains Supplementary Table 1, Supplementary Note 1 (a minimal illustrative AWARE contract) and Supplementary Note 2 (the AWARE incident catalogue).

FailureHow an agent triggers itExisting check / gap
Wrong units / conversionHard-codes a factor; adds fields with mismatched unitsxclim/MetPy dimensional checks (pint) if used; gap: raw numpy/xarray code
Undeclared averaging domainAverages an ocean variable without declaring domain and mask semantics, mixing land, ocean or missing cellsMasking utilities exist if used; gap: no check that a domain declaration was required
Non-conservative regriddingApplies bilinear to precipitation/flux fields, breaking global integralsxESMF offers conservative methods; gap: nothing flags a flux regridded bilinearly
Unweighted spatial meansComputes a plain average over latitude and longitude, overweighting the polesCommunity libraries provide weighted statistics; gap: agent must choose the weighted call
Calendar / time-axis bugsCasts idealized model calendars (360-day, no-leap) into civil-calendar timestamps; days are dropped or duplicatedCalendar-aware libraries exist; gap: no shared-calendar assertion before differencing
Coordinate conventionsSubsets lon −100... − 80 on a 0–360 dataset; N–S latitude inversioncf-xarray axis detection; gap: wrapped or misaligned subsetting can pass silently through downstream code
Fill-value mishandlingAggregates over undecoded missing-value sentinels (e.g. 102010^{20})File-format decoding and validated libraries; gap: raw array extraction bypasses metadata
Vertical-coordinate confusionPlots terrain-following model levels as if they were constant-pressure surfacesMetadata for the vertical transformation exists; gap: agents rarely execute it autonomously
Projection / datum errors (Earth observation); instrument response (seismology)Mixes map projections or vertical datums; skips instrument-response removalGeospatial and seismological libraries handle it if invoked; gap: no pre-flight geospatial lint
Model year vs. observed date; reanalysis as truthAligns model “1998” with the observed El Niño; validates a model against ERA5 as ground truthgap: conceptual, uncatchable by file linters; requires epistemic labels (Box 1) + guardrails
Error propagationChains six transformations; a step-1 units slip corrupts the final figureProvenance records the chain; gap: no end-to-end invariant checking

Table 1. Potential silent failure modes in agent-driven Earth-science analysis. Depending on library and input, each can complete without raising an exception and yield plausible but wrong results.

A minimal AWARE contract (illustrative sketch)

The example below sketches, in YAML form, what a minimal machine-readable AWARE contract for one dataset version could contain. It is illustrative, not a specification: field names, vocabularies and serialization are open design questions (see Open questions and limitations in the main text).

# AWARE contract -- minimal illustrative example
dataset: cmip6.CMIP.AWI.AWI-CM-1-1-MR.historical.r1i1p1f1.Omon.tos
version: v20200212          # immutable; checksum: sha256:...
epistemic_class: free-running_model_output
ensemble: {member: r1i1p1f1, single_realization: true}
load_recipe:
  format: netcdf4
  companion_files: [areacello@v20200212]
operations:
  spatial_mean: {requires: [cell_areas, ocean_mask], weighting: area}
  regridding:    {preserve: area_integral, source_grid: curvilinear}
valid_comparisons:
  - {against: observations, mode: distributional, not: year_by_year}
  - {against: reanalysis, caveat: non_independent_bias_bearing}
known_artefacts:
  - {type: spinup_drift, variable: tos, period: [1850, 1870],
     action: exclude_or_detrend}
licence:  {id: CC-BY-4.0, autonomous_retrieval: permitted}
citation_bundle: doi:10.xxxx/xxxxx
owner:     {institution: AWI, contact: [email protected]}
PlatformAgent-relevant strengthLimit
PangeoUniform CF-aware xarray/Dask/Zarr data model over petabytes; a widely used community substrate for agent-callable analysisEcosystem, not a managed service; no agent interface, auth or guardrails
Earth System Grid Federation (∼20\sim20 PB of model output across dozens of nodes)Standardised dataset identity and controlled vocabularies; machine-queryable; the primary source of CMIP model outputFile-download paradigm; node heterogeneity; metadata does not include load recipes
Destination EarthKilometre-scale digital twins on European supercomputers; programmatic extraction of data subsets from live simulation streamsYoung; access governance maturing; not yet a general agent endpoint
Microsoft Planetary ComputerPetabytes of catalogued Earth-observation data; Earth Copilot provides an open-source, deployable MCP integration over its STAC APIVendor cloud; observation-centric, not model output
Google Earth Engine [Gorelick et al., 2017]Massive catalogue co-located with compute; low entry barrierClosed backend; analyses are not portable outside the platform
NASA Earthdata / EOSDISCommon Metadata Repository: large, structured, machine-queryableDAAC silos; variable metadata richness
Argo (∼4,000\sim4{,}000 floats) / ICOS (∼180\sim180 stations)Standardized QC, netCDF, DOIs, free real-time access; mature observational networksPoint data need careful gridding; QC-flag semantics require domain knowledge
PANGAEA [Felden et al., 2023]Multidisciplinary FAIR pioneer; DOI per dataset; curation depth (the substrate of PANGAEA GPT)Labour-intensive curation; not cloud-native array access
EarthScope/IRIS, Sea-DataNet/EMODnet, OpenTopographyStandardised domain services beyond climate (FDSN web services for seismic waveforms, harmonised ocean and lidar/topography access); proof the template generalises field-wideAgent tooling nascent; QC-flag and instrument-response semantics still human-mediated

Table 1. Illustrative examples of platforms the community already operates, read as templates for agent readiness. Entries are the authors’ illustrative reading as of 2026, not systematic evaluations; counts and service status change over time.

The AWARE incident catalogue

Purpose and provenance

This catalogue documents the operational incidents referred to in the Perspective as observations from “our audited sessions”. Each incident was observed during the development and operation of four agentic systems built by the authors’ group between 2023 and 2026, and recorded in working notes, session logs, commit histories or post-hoc reviews: an autonomous research agent for CMIP6 analysis over the ESGF federation (CMIP-Forge); ClimSight, an openly released climate-service system answering localized climate questions ([31, 32]); its extended risk-analysis variant; and PANGAEA-GPT, a multi-agent system for discovery and analysis over the PANGAEA archive ([30]). Incidents are reported at the level of what happened and what it implies; internal paths, credentials and personal identifiers are removed. Nothing here is hypothetical. Incidents were included when they were recorded contemporaneously or could be traced to a specific session, log, commit or published artefact. They illustrate failure modes and are not a frequency sample, so no rates should be inferred from their counts. All entries were documented in 2026; incidents of agent behaviour were observed with then-current frontier models, and entries describing long-lived data-handling defects state their lifetimes explicitly. Per-system provenance is given in the closing section “How this catalogue was compiled”.

Each entry lists: the observed behaviour, how it was detected, the consequence had it gone undetected, and the AWARE requirement (Actionable, Well-described, Auditable, Recipe-equipped, Epistemically labelled) or guardrail it motivates.

Discovery and vocabulary

A1 — Invented identifiers. When an agent did not know an exact controlled-vocabulary token (model name, variant label), it produced plausible-looking fabrications often enough that a validation layer now checks every proposed identifier against the live ESGF vocabulary before any search. Motivates: Well-described; guardrail.

A2 — Wrong hemisphere ranked first. A natural-language request for an Antarctic ice-core record returned a Greenland core (NGRIP) as the top archive hit: the search matched on ice-core vocabulary and ignored the hemisphere, and the wrong dataset survived to output, caught only in a benchmark review. Detected: benchmark evaluation. If undetected: an analysis of the wrong hemisphere delivered under the requested name. Motivates: Well-described (spatial coverage as a machine-checkable facet); guardrails.

A3 — One quantity, three names and two units. In one observational pipeline the same physical quantity (total precipitation) has no single name: the store lacks the conventional short name the model expects, the tool requires two components to be summed instead, a legacy helper still advertises the conventional name, and the intermediate output re-exposes it in a different unit. The tool documentation warns about it in four places, the recovery prompt documents the mirror-image failure, and agents still stumble in both directions. Motivates: Well-described (one resolvable vocabulary per dataset); Recipe-equipped.

A4 — Right vocabulary, wrong ocean. A query for Pacific soundscape data was correctly expanded to “passive acoustic monitoring”, and 26 of the 28 returned datasets were from the Weddell Sea; the one clearly Pacific dataset ranked 27th. Because the semantic expansion is exactly right, the result set reads as a domain-expert answer while the ocean basin is wrong in almost all of it. Detected: benchmark judging, scored as a failure of spatial fidelity. Motivates: Well-described (spatial coverage as a first-class, machine-checkable facet).

A5 — Right instrument, wrong decade. A request for Argo profiles from 2019 returned high-quality records from 2006: exactly the right instrument and variables, thirteen years off, and nothing in the presentation of the results signalled the mismatch. Detected: pilot evaluation. If undetected: the hardest kind of error to catch by eye. Motivates: Well-described (temporal coverage as a checkable facet).

Requests and delivery

B1 — Empty delivery, confident answer. In one logged session the climate-data extraction failed and the variable table arrived empty; the pipeline continued to a fluent final answer with no model data behind it and no user-visible flag. Detected: reading the session log. If undetected: the archetypal case of a language model answering from priors while the data slot is empty. Motivates: Actionable (machine-readable delivery manifest); guardrails.

B2 — Broken retriever presented as an empty library. A dependency change silently broke the literature-retrieval component; every affected session reported “no relevant documents found” and answered without literature grounding. The crash was recorded as an empty search result, not as an outage, so no alarm was raised. Detected: post-hoc log review (28 identical errors). If undetected: answers with misattributed provenance. Motivates: Auditable (dependency and delivery state as first-class signals); guardrails.

B3 — Empty hazard record reads as safety. For a storm-surge-prone coastal city the natural-hazards retrieval returned an empty table (“no disasters recorded for this location”), an artefact of the source’s geocoding coverage; the phrasing survives into answers, where a lay reader takes absence of records as low hazard. Motivates: Epistemically labelled (coverage and completeness declared); Well-described.

B4 — Access through a human-minted token. A projection data service used by the system authenticates through a token that a human must create out of band and periodically refresh; when the token lapsed mid-run, retrieval failed and the advertised dataset was silently absent from analyses until an operator intervened. Fully automated operation is impossible by construction. Motivates: Actionable (machine-negotiable access).

B5 — A billing wall, then invented fields. Every discovery path to a cloud model archive failed in turn, ending at a requester-pays billing gate that no agent can pass without a human payment step. The run then continued into fabrication of the fields it could not download (the failure family of E1). Detected: a human reading the final response. Motivates: Actionable (machine-negotiable access with declared cost); guardrails.

B6 — A cache key without the subset. An observational retrieval cached files under variable, years and bounding box, but not the requested months: a June–August file was later served, as a reported cache hit, for an all-months request, and the truncated record became the observational baseline of a multi-model comparison. The cache-hit message did not display the months at all, so the omission was invisible in everything the agent read. Detected: a developer, from the collision pattern. Motivates: Auditable (content-addressed identity); Recipe-equipped.

B7 — Two regions, one cache file. Another cache keyed the bounding box at one-decimal precision, so two nearby but different boxes mapped to the same file: the second request received the first region’s data while every report described the requested region. The same tool feeds the observational baseline used to rank climate models for a site, so a collided cache can silently decide which models count as best. Detected: adversarial audit, reproduced numerically; still open at the time of writing. Motivates: Auditable (identity that encodes the full request).

B8 — Truncated series at high confidence. A retrieval guard validated only the start date, so a request ending in the future proceeded: label-based time slicing silently returned only the available steps, the tool reported success at high confidence with the full requested coverage, and the cache stored the truncated file under the full range, making the gap permanent. Detected: adversarial audit. Motivates: Auditable (delivery manifests); guardrails (fail on partial coverage).

Reading and decoding

C1 — Companion files located but not used. In one audited CMIP6 session the agent correctly found the cell-area dataset (areacello) for an unstructured-mesh ocean model, then fell back to cosine-of-latitude weighting anyway; the result was acceptable only by luck of the mesh geometry. In another run, a land mask for the reanalysis reference was borrowed from one of the compared climate models, quietly contaminating the baseline along complex coastlines (the borrowed-mask shortcut was even admitted in a code comment). Motivates: Well-described (declared companion dependencies), Recipe-equipped.

C2 — The units attribute lies. For one model’s precipitation files the declared unit is not what the numbers are; the handling code says so in a comment and applies a hand-derived conversion, and the repository preserves two earlier, mutually contradictory interpretations of the same files. Nothing in the data identifies the correct reading; the knowledge lives in a code comment. Motivates: Well-described (checked, trustworthy units); Recipe-equipped.

Scientific validity

D1 — Bias subtraction read as climate signal. An ensemble pipeline computed future change as model future minus reanalysis baseline and interpreted the difference as climate change; systematic model bias entered the result as apparent signal (spurious warming; most of an apparent regional drying). Both datasets were individually correct, convention-compliant and FAIR; the invalid object was the subtraction. Caught by external review; the fix (difference each model against its own historical run) is textbook practice written nowhere near the data. Motivates: Epistemically labelled; guardrails.

D2 — Chronology treated as event alignment. An agent’s first instinct was to rank free-running historical simulations against reanalysis year by year, punishing models for the phase of internal variability rather than physics; corrected to distributional comparison only by a review step. Motivates: Epistemically labelled.

D3 — Ensemble silently degraded by substitution. After losing access to a high-resolution configuration mid-analysis, an agent substituted an alternative configuration and logged the change; the log was correct, but the ensemble’s composition, and hence its meaning, had shifted, visible only in a post-hoc audit. Motivates: Auditable (explicit substitution policy).

D4 — Numbers below the resolution floor. A drought frequency of 0.30 events/year, derived from a three-model, single-member, monthly, box-averaged ensemble without bias correction, was printed to three decimals; the same value appeared under both emission scenarios, and the drying signal was slightly stronger under the lower-emission pathway, marking the metric as noise. Review prescribed attaching “n=3n=3, single-member, box-mean, no bias correction” to every projected number. Motivates: Epistemically labelled (statistical fitness for purpose).

D5 — Question class unsupported by the table. A case brief required daily-scale metrics (wet-bulb exceedance days, cooling degree days); the assembled ensemble was monthly-mean, so none were computable, yet a report asserted daily-scale conclusions anyway. The pipeline now pre-checks that the required table exists and emits an explicit low-confidence “unavailable” record otherwise, because a silent empty result reads downstream as “the effect does not exist”. Motivates: Well-described, Epistemically labelled.

D6 — Signed longitude on a 0–360 grid. A nearest-cell lookup queried a global grid with the user’s signed longitude, so every negative longitude snapped to cells near the prime meridian at the same latitude: queries for the entire Western Hemisphere returned plausible temperatures from the wrong continent, blended by distance weighting as if local. The defect lived in the default data path for around fourteen months. Detected: eventually, by inspection; nothing ever crashed. Motivates: Well-described (declared coordinate conventions); Recipe-equipped; guardrails.

D7 — Transposed coordinates rationalized by the model. A biodiversity query passed latitude and longitude in the wrong order for more than two years, so species lists came from the wrong part of the planet; instead of flagging the absurdity, the language model rationalized it (a deep-sea fish “signals proximity to marine systems” for an inland garden question). Detected: code inspection; the wrong outputs had repeatedly passed casual review. Motivates: Well-described; guardrails (plausibility checks). Fluent post-hoc rationalization defeats human spot checks.

D8 — The selected ensemble quietly changed class. Ten models were selected for a regional study, three of them high-resolution configurations chosen precisely because they resolve the region’s orography; when a follow-up session retrieved the data, those three had been silently replaced by their standard-resolution siblings, and the result was presented as “the ten selected models”. Smaller substitutions were footnoted; the resolution downgrade was not. Detected: post-hoc reading of the session pair. Motivates: Auditable (declared substitution policy); Epistemically labelled (resolution class as part of dataset identity).

Fabrication and honesty

E1 — Random numbers as model output. Confronted with a failed download, agents were caught filling variables named like model output from random-number generators; the static checker now treats the pattern as a hard stop, and the rule has fired in practice. Motivates: guardrails.

E2 — Fabricated references. In one audited literature-review session, 7 of 21 cited DOIs had never appeared in any search result the agent had actually seen; some carried characteristic formatting corruption from training memory. Every DOI in agent output is now verified against the sources the agent demonstrably retrieved. Motivates: guardrails, Auditable.

E3 — Fakes dressed in scholarship. Early development revealed synthetic stand-in climatologies and invented lookup tables published at high confidence while citing real datasets they were never derived from. Synthetic fallbacks are now banned outright: a tool that cannot reach its real source must return an explicit low-confidence failure naming the missing piece. Motivates: Epistemically labelled; guardrails.

E4 — Retrieval without analysis. On a cheap model tier, one agent retrieved the same temperature field five times and computed nothing from it; 35% of one run’s tool calls were duplicates or identical failed retries. The gate “an agent with three or more retrievals and zero derived statistics fails the run” encodes the rule that downloading files is not evidence. Motivates: guardrails.

E5 — Fabricated precision. An answer stated a distance “approximately 392 kilometres from the nearest coastline” for an inland city, though no such quantity was among the supplied data; the value, roughly right and stated as fact, came from the model’s priors and was indistinguishable in the output from data-backed numbers. Motivates: guardrails (provenance labels on every quantitative claim).

How this catalogue was compiled

This section answers, per system, over what period the evidence was collected, how much agent activity the records cover, how failures were recorded, and which catalogue entries each system contributed. It was compiled in 2026 from per-system audit notes, each reconstructed directly from the project’s repository, logs and review records; every figure below is traceable to a named record in those notes. Counts are floors, not totals: transcript retention limits and unrecorded early phases affect all systems.

CMIP-Forge (autonomous CMIP6 research agent). Developed from 2024 to 2026, with instrumented per-turn session logging in 2026. Records cover 260 per-turn transcripts across 117 session identifiers, seven curated end-to-end use-case archives, a structured hallucination audit (8 use cases, 27 queries), an independent numerical re-verification of one full run against the raw NetCDF files, and senior re-reads of the seven published use cases. Failures were recorded in session transcripts, audit documents and the guardrail code they motivated. Contributed incidents: A1, B5, B6, C1, D2, D3, D8, E2, and E1 (shared with the risk-analysis system).

ClimSight (openly released climate-service system; system descriptions published in [31, 32]). Developed and operated from 2023 to 2026. Little formal logging was kept during operation; the evidence base is a dedicated mining pass (2026) over the repository and its run copies: application logs retained in run snapshots, dated fix commits in the git history, the published question–answer dataset, deployment notes and an internal review document. The pass distinguished documented incidents (a wrong output or record exists) from failures inferred only from defensive code; only documented incidents were admitted here. Contributed incidents: A3, B1–B4, C2, D6, D7, E5.

ClimSight risk-analysis variant (multi-agent climate-risk assessment). Developed and operated in 2026; internal R&D only. At least 68 complete end-to-end runs (a floor: fixture and aborted runs share the storage scheme); one fully audited run comprised 2,340 orchestration events and 268 evidence cards. Failures were recorded in per-session memory notes, second-model review verdicts required for every substantive change, a 37-test adversarial regression suite encoding closed fabrication incidents, and, in the final development phase, a deterministic per-run report gate. Contributed incidents: B7, B8, D1, D4, D5, E3, E4, and E1 (shared with CMIP-Forge).

PANGAEA-GPT (multi-agent discovery and analysis over the PANGAEA archive; Pantiukhin et al. [30]). Developed since 2024. A systematic evaluation campaign preserved per-run records (53 runs with outcomes and error counts), four hand-annotated validation scenarios and dated working notes; runtime logs survive only for a short late window, and no incident log was kept during early development. Contributed incidents: A2, A4, A5.

The one incident observed in two systems (E1) is counted once. The underlying audit notes, which name the specific repositories, logs and record files behind every figure above, are retained by the authors and available on reasonable request.

References

  1. [1]Zhou, T., Qian, Y. & Leung, L. R. Can we trust LLMs for complex Earth system model analysis? Silent failure and evidence from module-grounded benchmarking. EGUShpere [preprint], egusphere-2026-2237, under review for Geosci. Model Dev. (2026).
  2. [2]Schick, T. et al. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems 36 (2023). ArXiv:2302.04761.arxiv.org/abs/2302.04761
  3. [3]Lewis, P. et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems 33 (2020). ArXiv:2005.11401.arxiv.org/abs/2005.11401
  4. [4]Yang, J. et al. SWE-agent: Agent-Computer interfaces enable automated software engineering. Advances in Neural Information Processing Systems 37 (2024). ArXiv:2405.15793.arxiv.org/abs/2405.15793
  5. [5]Nature Portfolio. Artificial intelligence (AI) — editorial policy. https://www.nature.com/nature-portfolio/editorial-policies/ai (accessed 12 August 2026).
  6. [6]COPE Council. Authorship and AI tools — position statement (2023).
  7. [7]International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. Updated January 2026; Section V, artificial intelligence. https://www.icmje.org/recommendations/ (accessed 12 August 2026) (2026).
  8. [8]Buck, H. J., Al Khourdajie, A., Asayama, S. & Geden, O. Potential futures for the IPCC’s approach to artificial intelligence. Communications Earth & Environment 7, 372 (2026).DOI
  9. [9]Lam, R. et al. Learning skillful medium-range global weather forecasting. Science 382, 1416–1421 (2023).DOI
  10. [10]Bi, K. et al. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619, 533–538 (2023).DOI
  11. [11]Pathak, J. et al. FourCastNet: A global data-driven high-resolution weather model using adaptive Fourier neural operators. arXiv:2202.11214 (preprint) (2022).arxiv.org/abs/2202.11214
  12. [12]Price, I. et al. Probabilistic weather forecasting with machine learning. Nature 637, 84–90 (2025).DOI
  13. [13]Alet, F. et al. Skillful joint probabilistic weather forecasting from marginals. arXiv:2506.10772 (preprint) (2025).arxiv.org/abs/2506.10772
  14. [14]Bodnar, C. et al. A foundation model for the Earth system. Nature 641, 1180–1187 (2025).DOI
  15. [15]Nguyen, T., Brandstetter, J., Kapoor, A., Gupta, J. K. & Grover, A. ClimaX: A foundation model for weather and climate. Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR 202, 25904–25938 (2023).arxiv.org/abs/2301.10343
  16. [16]Schmude, J. et al. Prithvi WxC: Foundation model for Weather and Climate. arXiv:2409.13598 (preprint) (2024).arxiv.org/abs/2409.13598
  17. [17]Kochkov, D. et al. Neural general circulation models for weather and climate. Nature 632, 1060–1066 (2024).DOI
  18. [18]Watt-Meyer, O. et al. ACE2: accurately learning subseasonal to decadal atmospheric variability and forced responses. npj Clim. Atmos. Sci. 8, 205 (2025).
  19. [19]Chen, L. et al. A machine learning model that outperforms conventional global subseasonal forecast models. Nature Communications 15, 6425 (2024).DOI
  20. [20]Allen, A. et al. End-to-end data-driven weather prediction. Nature 641, 1172–1179 (2025).DOI
  21. [21]Mardani, M. et al. Residual corrective diffusion modeling for km-scale atmospheric downscaling. Communications Earth & Environment 6, 124 (2025).arxiv.org/abs/2309.15214
  22. [22]Thulke, D. et al. ClimateGPT: Towards AI synthesizing interdisciplinary research on climate change. arXiv:2401.09646 (preprint) (2024).arxiv.org/abs/2401.09646
  23. [23]Bi, Z. et al. OceanGPT: A large language model for ocean science tasks. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) 3357–3372 (2024).
  24. [24]Deng, C. et al. K2: A foundation language model for geoscience knowledge understanding and utilization. Proceedings of the 17th ACM International Conference on Web Search and Data Mining (WSDM ’24) 161–170 (2024).arxiv.org/abs/2306.05064
  25. [25]Lin, Z. et al. GeoGalactica: A scientific large language model in geoscience. arXiv:2401.00434 (preprint) (2024).arxiv.org/abs/2401.00434
  26. [26]Webersinke, N., Kraus, M., Bingler, J. A. & Leippold, M. ClimateBert: A pre-trained language model for Climate-Related text. arXiv:2110.12010 (preprint) Guo, Z. et al. A Self-Evolving AI agent system for climate science. arXiv:2507.17311v3 (preprint) (2025).
  27. [28]Kim, H. et al. ClimateAgent: Multi-Agent orchestration for complex climate data science workflows. arXiv:2511.20109 (preprint); to appear in Transactions on Machine Learning Research (2026) (2025). TMLR version of record lists Li and Kim as joint first authors with Li first — re-check author order before submission.arxiv.org/abs/2511.20109
  28. [29]Jaber, A. et al. AutoClimDS: Climate data science agentic AI — a knowledge graph is all you need. arXiv:2509.21553 (preprint) (2025).arxiv.org/abs/2509.21553
  29. [30]PantiukhIn, D., Shapkin, B., Kuznetsov, I., Jost, A. A. & Koldunov, N. Accelerating earth science discovery via multi-agent LLM systems. Frontiers in Artificial Intelligence 8, 1674927 (2025).
  30. [31]Koldunov, N. & Jung, T. Local climate services for all, courtesy of large language models. Communications Earth & Environment 5, 13 (2024).DOI
  31. [32]Kuznetsov, I. et al. Transforming climate services with LLMs and multi-source data integration. npj Climate Action 4, 97 (2025).DOI
  32. [33]Feng, P. et al. Earth-Agent: Unlocking the full landscape of Earth observation with agents. International Conference on Learning Representations (ICLR) (2026). ArXiv:2509.23141.arxiv.org/abs/2509.23141
  33. [34]European Commission. Destination Earth. https://digital-strategy.ec.europa.eu/en/policies/destination-earth (accessed 12 August 2026).
  34. [35]Doblas-Reyes, F. J. et al. The Destination Earth digital twin for climate change adaptation. Geosci. Model Dev. 19, 2821–2848 (2026).
  35. [36]Helmholtz Association. Helmholtz foundation model initiative (HFMI) — pilot projects. https://hfmi.helmholtz.de/pilot-projects/ (accessed 12 August 2026).
  36. [37]McGovern, A. et al. NSF AI institute for research on trustworthy AI in weather, climate, and coastal oceanography (AI2ES). Bull. Amer. Meteor. Soc. 103, E1658–E1668 (2022).DOI
  37. [38]National Oceanic and Atmospheric Administration. NOAA center for artificial intelligence. https://www.noaa.gov/ai (accessed 21 August 2026).
  38. [39]European Space Agency. ESA φ-lab: artificial intelligence for Earth observation. https://philab.esa.int/ (accessed 21 August 2026).
  39. [40]Ajibade, S.-S. M. et al. A research landscape bibliometric analysis on climate change for last decades: Evidence from applications of machine learning. Heliyon 9, e20297 (2023).DOI
  40. [41]Chen, Z. et al. ScienceAgentBench: Toward rigorous assessment of language agents for Data-Driven scientific discovery. International Conference on Learning Representations (ICLR) (2025). ArXiv:2410.05080.arxiv.org/abs/2410.05080
  41. [42]Jing, L. et al. DSBench: How far are data science agents from becoming data science experts? International Conference on Learning Representations (ICLR) (2025). ArXiv:2409.07703.arxiv.org/abs/2409.07703
  42. [43]Nguyen, D. T. et al. TerraBench: Can agents reason over heterogeneous Earth-System data? arXiv:2606.13148v2 (preprint) (2026).arxiv.org/abs/2606.13148
  43. [44]Pantiukhin, D., Shapkin, B., Kuznetsov, I., Jung, T. & Koldunov, N. CMIP-Forge: An agentic system that retrieves, computes, and Self-Reviews climate science. arXiv:2606.17076 (preprint) (2026).arxiv.org/abs/2606.17076
  44. [45]Wilkinson, M. D. et al. The FAIR guiding principles for scientific data management and stewardship. Scientific Data 3, 160018 (2016).
  45. [46]Borgman, C. L. & Groth, P. From data creator to data reuser: Distance matters. Harvard Data Science Review 7 (2025). ArXiv:2402.07926.DOI
  46. [47]Polanyi, M. The Tacit Dimension (Doubleday, Garden City, NY, 1966).
  47. [48]Collins, H. Tacit and Explicit Knowledge (University of Chicago Press, Chicago, 2010).DOI
  48. [49]Rew, R. K. & Davis, G. P. NetCDF: An interface for scientific data access. IEEE Comput. Graph. Appl. 10, 76–82 (1990).DOI
  49. [50]Zarr Development Team. Zarr storage specification, version 2 (superseded) and Zarr core specification, version 3.1. https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html; https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html (accessed 12 August 2026).
  50. [51]Hassell, D., Gregory, J., Blower, J., Lawrence, B. N. & Taylor, K. E. A data model of the Climate and Forecast metadata conventions (CF-1.6) with a software implementation (cf-python v2.1). Geosci. Model Dev. 10, 4619–4646 (2017).DOI
  51. [52]Eyring, V. et al. Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci. Model Dev. 9, 1937–1958 (2016).
  52. [53]Balaji, V. et al. Requirements for a global data infrastructure in support of CMIP6. Geosci. Model Dev. 11, 3659–3680 (2018).DOI
  53. [54]Petrie, R. et al. Coordinating an operational data distribution network for CMIP6 data. Geosci. Model Dev. 14, 629–644 (2021).DOI
  54. [55]Cinquini, L. et al. The Earth system grid federation: An open infrastructure for access to distributed geospatial data. Future Generation Computer Systems 36, 400–417 (2014).
  55. [56]Taylor, K. E., Juckes, M., Balaji, V. et al. CMIP6 model output metadata requirements, data reference syntax (DRS) and controlled vocabularies (CVs). PCMDI; Zenodo, v1 (2018).DOI
  56. [57]Juckes, M. et al. The CMIP6 data request (DREQ, version 01.00.31). Geosci. Model Dev. 13, 201–224 (2020).DOI
  57. [58]Cimadevilla, E., Lawrence, B. N. & Cofiño, A. S. The Earth system grid federation (ESGF) virtual aggregation (CMIP6 v20240125). Geosci. Model Dev. 18, 2461–2478 (2025).DOI
  58. [59]Hersbach, H. et al. The ERA5 global reanalysis. Q. J. R. Meteorol. Soc. 146, 1999–2049 (2020).
  59. [60]Akhtar, M. et al. Croissant: A metadata format for ML-Ready datasets. Advances in Neural Information Processing Systems 37, Datasets and Benchmarks Track (2024). ArXiv:2403.19546.DOI
  60. [61]Huerta, E. A. et al. FAIR for AI: an interdisciplinary and international community building perspective. Scientific Data 10, 487 (2023).
  61. [62]Schultes, E. & Wittenburg, P. Manolopoulos, Y. & Stupnikov, S. (eds) FAIR principles and digital objects: Accelerating convergence on a data infrastructure. (eds Manolopoulos, Y. & Stupnikov, S.) Data Analytics and Management in Data Intensive Domains (DAMDID/RCDL 2018), Vol. 1003 of Communications in Computer and Information Science, 3–16 (Springer, 2019).DOI
  62. [63]De Smedt, K., Koureas, D. & Wittenburg, P. FAIR digital objects for science: From data pieces to actionable knowledge units. Publications 8, 21 (2020).DOI
  63. [64]Giner-Miguelez, J., Gómez, A. & Cabot, J. On the readiness of scientific data papers for a fair and transparent use in machine learning. Scientific Data 12, 61 (2025).
  64. [65]Pasquetto, I. V., Borgman, C. L. & Wofford, M. F. Uses and reuses of scientific data: The data creators’ advantage. Harvard Data Science Review 1 (2019).DOI
  65. [66]Gebru, T. et al. Datasheets for datasets. Communications of the ACM 64, 86–92 (2021).arxiv.org/abs/1803.09010
  66. [67]Pascoe, C. et al. Documenting numerical experiments in support of the Coupled Model Intercomparison Project Phase 6 (CMIP6). Geosci. Model Dev. 13, 2149–2167 (2020).DOI
  67. [68]Stockhause, M. & Lautenschlager, M. CMIP6 data citation of evolving data. Data Science Journal 16, 30 (2017).DOI
  68. [69]Hollmann, R., Merchant, C. J., Saunders, R. et al. The ESA climate change initiative: satellite data records for essential climate variables. Bulletin of the American Meteorological Society 94, 1541–1552 (2013).DOI
  69. [70]Burton, A. et al. The Scholix framework for interoperability in Data-Literature information exchange. D-Lib Magazine 23 (2017).DOI
  70. [71]Moreau, L. & Missier, P. PROV-DM: The PROV data model. W3C Recommendation, 30 April 2013. https://www.w3.org/TR/2013/REC-prov-dm-20130430/ (2013).
  71. [72]Leo, S. et al. Recording provenance of workflow runs with RO-Crate. PLOS ONE 19, e0309210 (2024).DOI
  72. [73]Hoyer, S. & Hamman, J. xarray: N-D labeled arrays and datasets in Python. J. Open Res. Softw. 5, 10 (2017).DOI
  73. [74]Bourgault, P. et al. xclim: xarray-based climate data analytics. J. Open Source Softw. 8, 5415 (2023).DOI
  74. [75]May, R. M. et al. MetPy: A meteorological Python library for data analysis and visualization. Bull. Amer. Meteor. Soc. 103, E2273–E2284 (2022).DOI
  75. [76]Vo, T. et al. xCDAT: A Python package for simple and robust analysis of climate data. J. Open Source Softw. 9, 6426 (2024).DOI
  76. [77]Yao, S. et al. ReAct: Synergizing reasoning and acting in language models. International Conference on Learning Representations (ICLR) (2023). ArXiv:2210.03629.arxiv.org/abs/2210.03629
  77. [78]Patil, S. G., Zhang, T., Wang, X. & Gonzalez, J. E. Gorilla: Large language model connected with massive APIs. Advances in Neural Information Processing Systems 37 (2024). ArXiv:2305.15334.arxiv.org/abs/2305.15334
  78. [79]Model Context Protocol. Model context protocol specification. https://modelcontextprotocol.io/specification/latest (revision 2026-07-28; accessed 12 August 2026) (2026).
  79. [80]Linux Foundation. Linux Foundation announces the formation of the Agentic AI Foundation, anchored by new project contributions including Model Context Protocol (MCP). Press release, 9 December 2025. https://blog.modelcontextprotocol.io/posts/2025-12-09-mcp-joins-agentic-ai-foundation/ (2025).
  80. [81]Yoo, A. B., Jette, M. A. & Grondona, M. SLURM: Simple Linux utility for resource management. Job Scheduling Strategies for Parallel Processing (JSSPP 2003), Lecture Notes in Computer Science 2862, 44–60 (2003).DOI
  81. [82]Kurtzer, G. M., Sochat, V. & Bauer, M. W. Singularity: Scientific containers for mobility of compute. PLOS ONE 12, e0177459 (2017).DOI
  82. [83]Huang, J. et al. Large language models cannot Self-Correct reasoning yet. International Conference on Learning Representations (ICLR) (2024). ArXiv:2310.01798.arxiv.org/abs/2310.01798
  83. [84]Luo, Z., Kasirzadeh, A. & Shah, N. B. The more you automate, the less you see: Hidden pitfalls of AI scientist systems. arXiv:2509.08713v2 (preprint); NeurIPS 2025 AI4Science Workshop (2025).arxiv.org/abs/2509.08713
  84. [85]Beel, J., Kan, M.-Y. & Baumgart, M. Evaluating Sakana’s AI scientist: Bold claims, mixed results, and a promising future? arXiv:2502.14297v3 (preprint) (2025).DOI
  85. [86]Righi, M. et al. Earth system model evaluation tool (ESMValTool) v2.0 — technical overview. Geosci. Model Dev. 13, 1179–1199 (2020).DOI
  86. [87]Kraidia, I., Qaddara, I., Almutairi, A., Alzaben, N. & Belhouari, S. B. When collaboration fails: persuasion-driven adversarial influence in multi-agent large language model debate. Scientific Reports 16, 11640 (2026).DOI
  87. [88]Yuan, J. et al. Understanding and mitigating numerical sources of nondeterminism in LLM inference. arXiv:2506.09501v2 (preprint) (2025).arxiv.org/abs/2506.09501
  88. [89]Nature Geoscience. Adapting to AI. Nature Geoscience 17, 481 (2024). Editorial.
  89. [90]Data Citation Synthesis Group. Joint declaration of data citation principles. Martone, M. (ed.). San Diego, CA: FORCE11 (2014).
  90. [91]Smith, A. M., Katz, D. S. & Niemeyer, K. E. Software citation principles. PeerJ Computer Science 2, e86 (2016).
  91. [92]Barker, M. et al. Introducing the FAIR principles for research software. Scientific Data 9, 622 (2022).DOI
  92. [93]Schramm, M. et al. The openEO API — harmonising the use of Earth observation cloud services using virtual data cube functionalities. Remote Sensing 13, 1125 (2021).
  93. [94]Strubell, E., Ganesh, A. & McCallum, A. Energy and policy considerations for deep learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics 3645–3650 (2019).arxiv.org/abs/1906.02243
  94. [95]Patterson, D. et al. Carbon emissions and large neural network training. arXiv:2104.10350 (preprint) (2021).arxiv.org/abs/2104.10350
  95. [96]Luccioni, A. S., Viguier, S. & Ligot, A.-L. Estimating the carbon footprint of BLOOM, a 176B parameter language model. Journal of Machine Learning Research 24, 1–15 (2023).arxiv.org/abs/2211.02001
  96. [97]Rieutord, T., Mignon, C., McAulfield, E. & Gleeson, E. Energy and carbon footprint considerations for data-driven weather forecasting models. Weather 81, 114–119 (2026).DOI
  97. [98]Schultes, E. et al. Grossmann, G. & Ram, S. (eds) Reusable FAIR implementation profiles as accelerators of FAIR convergence. (eds Grossmann, G. & Ram, S.) Advances in Conceptual Modeling (ER 2020 Workshops), Vol. 12584 of Lecture Notes in Computer Science, 138–147 (Springer, 2020).
  98. [99]Majithia, N., Carey-Wilson, T., Simperl, E. & Shadbolt, N. An actionable framework for AI-ready data. AI Magazine 47, e70054 (2026).DOI
  99. [100]Nature Machine Intelligence. Thinking and rethinking data AI readiness. Nature Machine Intelligence 8, 1013 (2026). Editorial.DOI

Paper details

vlnc:W_wtp4qtdj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Claude, ChatGPT

View statement
Looking for related papers…
Contents