WorksPhysics and Astronomy
The Agentic Observer Hackathon: Autonomous Decision-making in Simulated Astronomical Surveys
Abstract
Astronomical surveys must allocate observing time while responding to changing conditions and competing scientific priorities. We present the Agentic Observer Hackathon (AOH), a community challenge for studying autonomous survey operations. Within a synthetic multi-object survey with prescribed exposure responses and assignment constraints, decision systems select pointings, assign targets and choose exposure configurations while responding to observing requests, instrumental faults and data loss. We evaluate systems submitted by 102 teams on campaigns reserved for evaluation and analyse target completion in a reference campaign. The leading systems combine numerical exposure planning with language models for interpreting operational information and supporting longer-term decisions. Exploratory tests show improved fault reporting with model access in several systems, while revised scientific priorities produce substantial changes in ranking. The reference campaign traces target completion and diagnoses errors in the baseline's inference of prescribed quality bands. The released simulator and reference strategies provide common survey conditions for comparing exposure allocation, target selection and recovery from interruptions.
Introduction
Astronomical surveys allocate finite observing time among targets with different exposure requirements and scientific priorities. Each observation contributes to the survey sample and changes the targets that remain to be observed. The choice of the next field and exposure duration therefore depends on survey progress, target visibility and the prevailing observing conditions. Over a campaign, these choices determine the depth, sky coverage and completeness of the resulting sample [5, 2, 29].
Corresponding author: Ting Tan and Huanyuan Shan
Email: [email protected], [email protected]
Operational surveys couple scheduling to measured conditions. The Rubin scheduler was designed to balance survey objectives [24, 2], DES uses the obstack scheduler to select exposures from survey progress and observing conditions [25], and the Legacy Surveys adjust exposure times as conditions change [5]. DESI selects fields, assesses ex-
ends_mid=1 posure completeness and updates target lists automatically (E. F. Schlafly et al. [29]). AstroSA and DeepSurveySim provide environments for comparing scheduling methods (H. Xie et al. [40]; M. Voetberg & B. Nord [36]). Offline reinforcement-learning studies have also compared deep -network scheduling variants on simulated Stone Edge Observatory campaigns (F. Terranova et al. [34]).
Language-model agents combine reasoning with actions (S. Yao et al. [41]), and benchmarks such as ScienceWorld and AgentBench evaluate their behaviour in interactive environments (R. Wang et al. [39]; X. Liu et al. [21]). These capabilities make language-model agents promising for astronomical observation planning, where decisions must respond to changing conditions. StarWhisper Telescope integrates observation planning, telescope control and image processing, with astronomer review of observing plans (C. Wang et al. [37]). FORMA uses a multi-agent workflow to verify classifications and redshifts in astronomical spectra (H. Wang et al. [38]). Alongside these applications, shared evaluation environments would help compare the capabilities of agentic systems on astronomical tasks. For survey operations, a common simulation would allow observing decisions to be evaluated under the same catalogues, weather histories and events. The use of shared simulations for method evaluation has precedents in astronomy, including GREAT3 and SKA Science Data Challenge 3a, which provide common datasets and assessment procedures (R. Mandelbaum et al. [22]; A. Bonaldi et al. [3]).
We present the Agentic Observer Hackathon (AOH), a community challenge in which participants develop decision systems for a synthetic simulation of multi-object spectroscopic observations. The simulation combines field selection, target assignment and exposure planning with changing weather, observing requests and instrumental events (Figure 1). We evaluate 102 submitted systems on four held-out campaigns and examine their survey outcomes and run-to-run variability. The leading systems combine numerical exposure planning with language models for interpreting operational information and supporting longer-term decisions. A reference campaign relates exposure choices and instrument performance to target completeness, while exploratory tests examine fault reporting and adaptation to revised scientific priorities. We release the simulator, reference implementations and public campaign data to support future survey planning and the development of observing decision systems.
Figure 1. Survey operations represented in AOH. (a) Stages of a survey, with the simulated observing operations highlighted and the preceding stages supplying fixed task inputs. (b) The six simulation modules, labelled by subsection, and their relationships to observation planning. Arrows show inputs, execution and feedback. Dashed boxes identify downstream survey stages outside the simulated scope; solid boxes in panel (b) denote implemented components. The two-level layout is visually inspired by Figure 5 of R. Mandelbaum et al. [22].
Section 2 describes the simulated survey and its response models. Section 3 presents the task design, decision interface, evaluation objective and reference methods. Section 4 reports survey outcomes, comparisons across held-out campaigns, system designs and exploratory tests. Sections 5 and 6 discuss the implications for survey operations and summarise the conclusions.
Simulation of survey operations
For AOH, we define a common survey environment with shared rules for observing geometry, target assignment and exposure response. All systems evaluated on a given task receive the same catalogue and observing calendar, and encounter the same weather and event histories. We model coupled field selection, target assignment and exposure planning in a synthetic multi-object spectroscopic survey. DESI provides context for the target classes and the organisation of survey operations (DESI Collaboration et al. [10]; E. F. Schlafly et al. [29]). Each task specifies a sky footprint, target catalogue, observing calendar and assignment grid. Atmospheric conditions, instrumental performance and operational events evolve during the campaign (Figure 1). The target properties and observing conditions determine each exposure response, and the valid observation history determines survey progress.
Table 1 summarises the models and evaluation choices used to compare observing strategies. Target strengths and observing conditions set exposure requirements (§ 2.1 and § 2.5), while science weights, penalties and operational rewards define the campaign objective (§ 3.3). Decision systems use bulletins, forecasts and exposure feedback to revise their plans as conditions change, new requests arrive or operational events affect observations (§ 2.3 and § 2.4). Note that the response, weather and scoring coefficients are set specifically for the challenge. Adapting the simulation to real surveys requires calibrating target responses against spectra and exposure measurements, and weather and instrument models against site monitoring and operational records.
| Component | Basis | Role in the simulation |
| Sky geometry | Astronomical methods | Computes visibility and airmass within prescribed observing limits and calendars. |
| Target classes and feature strength | Prescribed models | Class fractions set the DESI-inspired target mix; sets relative exposure requirements. |
| Exposure quality and response | Prescribed models | Maps target strength, exposure duration and conditions to the capped response . |
| Weather and disturbances | Prescribed models | Generates seasonal and correlated weather, closures and changes in instrument efficiency. |
| Class weights and completion | Evaluation choices | Weights , required status and penalties set priorities; a response threshold defines completion. |
| Programme matching | Evaluation choices | Band thresholds and multipliers reward selection of prescribed quality labels. |
| Fault and request rewards | Evaluation choices | Rewards correct fault reports and fulfilled requests; penalises false reports. |
Table 1. Astronomical basis and prescribed choices in the AOH simulation.
Sky coverage and target population
We include several target populations to compare how observing time is shared among objects with different exposure requirements and scientific priorities. The catalogue adopts the broad classes used in DESI target selection (A. D. Myers et al. [23]; C. Hahn et al. [12]; R. Zhou et al. [42]; A. Raichoor et al. [28]; E. Chaussidon et al. [7]). It contains stars and four extragalactic populations: bright, predominantly low-redshift galaxies from the Bright Galaxy Survey (BGS); luminous red galaxies (LRG), dominated by older stellar populations; star-forming emission-line galaxies (ELG); and quasars (QSO), luminous accreting galactic nuclei.
For an observed target, the exposure needed to reach a specified signal-to-noise ratio depends on its apparent magnitude and spectrum, the telescope collecting area, instrumental throughput, background and observing conditions [29]. In the simulation, the dimensionless feature strength sets the exposure needed to reach unit response at mean observing quality , where and define the exposure scale (Section 2.5). Larger strengths therefore require shorter exposures. We draw from class-dependent lognormal distributions, which give positive strengths and a continuous range of exposure requirements. Their prescribed medians give stars and BGS galaxies shorter reference exposures than LRGs, ELGs and quasars (Table B3).
Each target also has a prescribed science weight , which sets its contribution to the survey score. Higher weights for quasars and lower weights for stars and BGS galaxies create a trade-off between exposure demand and adopted scientific priority (Table B3). Class priorities also enter DESI target assignment [29]. Our strength distributions and weights are prescribed for the challenge; a real survey requires strengths calibrated against its spectra and exposure measurements, and weights set by its scientific objectives. A required subset is selected independently of class, strength and sky position. A penalty for required targets that remain incomplete gives this subset an additional completion priority.
We construct the survey footprints from separated spherical polygons with irregular boundaries and specified areas. Their locations and areas vary between tasks, giving different seasonal visibility from the observing sites (Section 3.1). Each catalogue contains a diffuse population and angular overdensities. These density variations and irregular boundaries make field placement affect how many targets can be assigned in an exposure. We retain targets with an observing window during the survey season. Figure 2 shows the final test catalogues E–H and the public footprints; Appendix § 9.§ 3 describes their construction.
Figure 2. Sky distributions of the final test catalogues E–H in an equal-area Mollweide projection. Orange points show the randomly selected required subset; blue points show the remaining catalogue targets. All targets contribute to the science score. The score also penalises required targets that remain incomplete at termination. Each panel gives the total and required-target counts. The coloured dashed outlines show the public A–D footprints, shared with A1–D1. The black solid curve marks the Galactic plane (). Grey shading marks as a reference for proximity to the Galactic plane. Right ascension increases to the left, with a central meridian of . The E–H catalogues come from the registered final scenario bundles; the public footprints were verified against the release on 2026 October 10.
Observing windows and target assignment
The observing site and season set the accessible sky and the available night time. We use astronomical darkness, with polar altitude below , and require targets **Note—**Astronomical methods use established geometry and analytic approximations. Prescribed models specify synthetic target populations, exposure responses and observing conditions. Evaluation choices define priorities, completion criteria and rewards.
to remain above throughout an exposure. The reference catalogue selection requires at least one continuous 900-s window satisfying these limits during the season. Thus every accepted target has a geometric observing opportunity, while weather and competition for observing time determine whether it is completed. Requested exposures last 60–3600 s; those starting within a night window end at the earliest of the requested end, the end of that night and the end of the campaign. Exposure and waiting share the available night time; slews, target reassignment and readout have zero simulated duration. The calendar construction and treatment of gaps are specified in Appendix § 9.
We use a square grid of cells on the tangent plane to impose finite spatial capacity: one target per cell per exposure. Nearby targets compete for cells, so dense regions can require several visits. The final E–H tasks use 9–100 cells (Table B2). For comparison, DESI uses 5000 robotic fibre positioners with finite, overlapping patrol regions and collision constraints ([9, 30]). These constraints affect assignment completeness ([33]). In AOH, the commanded altitude–azimuth pointing defines the initial field centre, including any scenario pointing offset, and the field then follows sidereal tracking. At exposure start, the grid axes follow increasing local altitude and azimuth. Target-to-cell assignments and the grid orientation relative to the tracked field are held fixed during that exposure. Targets must remain above the altitude limit throughout. Appendix § 9 gives the projection and geometric eligibility conditions.
| Card | (s) | Requests | Faults | Stress | |||
| L1 | 16 | 6.4 | 0.5 | 900 | 2 | 1 | no |
| L2 | 16 | 6.4 | 0.5 | 900 | 2 | 1 | no |
| L3 | 16 | 6.4 | 0.5 | 900 | 2 | 1 | yes |
| L4 | 16 | 6.4 | 0.5 | 900 | 2 | 1 | yes |
| 16 | 6.4 | 0.5 | 900 | 2 | 1 | no | |
| 16 | 6.4 | 0.5 | 900 | 2 | 1 | no | |
| 16 | 6.4 | 0.5 | 900 | 2 | 1 | yes | |
| 16 | 6.4 | 0.5 | 900 | 2 | 1 | yes | |
| A | 16 | 6.4 | 0.5 | 900 | 6 | 1 | no |
| B | 25 | 1.6 | 0.8 | 1000 | 6 | 1 | no |
| C | 9 | 6.3 | 0.5 | 900 | 6 | 1 | no |
| D | 100 | 6.0 | 0.5 | 900 | 6 | 3 | yes |
| A1 | 16 | 6.4 | 0.5 | 900 | 11 | 104 | no |
| B1 | 25 | 1.6 | 0.8 | 1000 | 12 | 131 | no |
| C1 | 9 | 6.3 | 0.5 | 900 | 12 | 54 | no |
| D1 | 100 | 6.0 | 0.5 | 900 | 14 | 267 | yes |
| E | 100 | 6.0 | 0.5 | 900 | 6 | 46 | no |
| F | 9 | 6.3 | 0.5 | 900 | 6 | 42 | no |
| G | 25 | 1.6 | 0.8 | 1000 | 6 | 32 | no |
| H | 16 | 6.4 | 0.5 | 900 | 6 | 29 | yes |
Table B2. Assignment, exposure and operational settings of the 20 tasks.
Atmospheric conditions and operational events
We describe observing conditions by seeing, transparency, sky quality and instrument efficiency, together with intervals when the site is closed. Seeing is measured in arcseconds; the other factors are dimensionless. A larger sky-quality factor represents a more favourable background. We generate synthetic atmospheric histories with a seasonal mean and autoregressive fluctuations ([4]). Seasonal variation changes observing opportunities across the campaign, while correlated fluctuations make recent conditions informative for the next exposure. Their amplitudes and persistence set how rapidly the required exposure time and preferred programme change. All policies evaluated on a given task use the same weather and event histories. Appendix § 9.§ 4 gives the weather and event models.
Operational events include rain, clouds, haze and cold waves, launch-related closures, terrain obstructions, earthquakes and instrumental faults. An event can affect the whole site or a horizon sector, changing target visibility or exposure quality. Earthquakes reduce instrument efficiency, which then recovers over successive nights. A correct report of an active reportable fault repairs the instrument. Tasks with additional disturbances also include pointing offsets and loss of earlier observations. These events test responses to temporary losses, report-triggered repair and replacement observations, together with adjustments to pointing. We use the local task L3 as a diagnostic example because its complete scenario products and archived actions allow us to follow conditions, decisions and target completion through observing requests, instrument losses and data loss. Figure 3 shows this sequence; § 4.2 analyses the survey outcome, and Appendix § 11.§ 4 gives the event timings and effects.
Figure 3. Evolution of the 37-night L3 campaign with the baseline model (§ 3.4) and language-model advice disabled. (a,b) Nightly median seeing, transparency and instrument efficiency over open intervals; shading shows the interquartile ranges. Vertical guides mark the instrumental events and fault repair. (c) Realised events above the divider and released information below it. Grey bars mark weather and launch events, with hatching for directional effects. Triangles mark earthquakes, the orange bar the active fault, the blue circle its repair, and the diamond the data-loss notification. Blue ticks mark information issue times; orange windows mark observing requests. (d) Nightly exposure time by programme. Left bars show the programmes selected by the baseline model; right bars show the programme giving the largest weighted bonus for each exposure. The reference uses the full simulated conditions at fixed pointing, assignments and duration (Appendix § 11.§ 4). Hatched segments denote zero-response ties. The solid and dotted lines show night-window and open time, respectively. Exposure-action time includes closed intervals and subsequently invalidated exposures. (e) Retained completion for all 8700 targets and the 435 required targets, with threshold .
Observing information and time-dependent requests
The decision system receives bulletins, forecasts and exposure feedback as the campaign advances. Bulletins identify event categories and approximate directions, forecasts describe upcoming weather and launch events, and exposure outcomes provide information about instrument performance. Records become available at their assigned issue times; observing decisions therefore use the catalogue, information released so far and previous outcomes. Tasks A1–D1 and E–H additionally provide free-text duty logs.
Time-sensitive observing programmes allocate telescope time in response to external triggers [1]. We represent changing priorities through observing requests with deadlines, making the value of an exposure depend on when it is taken. Each request specifies its targets, issue time, deadline, response threshold, minimum completed-target count and reward. To satisfy a request, the relevant valid exposures must lie wholly within its issue-to-deadline window. These requests compete with the ongoing survey for the remaining observing time, and their completion can change if supporting observations are later lost. § 3 gives the scoring rules; Appendix § 9.§ 4 describes request generation and the reference information cadence.
Exposure response and programme matching
For faint, background-limited point-source imaging, effective exposure scales with atmospheric transmission squared and inverse PSF FWHM squared at fixed sky brightness ([26]). DESI computes effective time using reference spectra and source profiles, with seeing entering through the fraction of light captured by a fibre (E. F. Schlafly et al. [29]; J. Guy et al. [11]). AOH uses a prescribed response with first-power transparency and inverse first-power seeing. The challenge response grows linearly with condition-weighted exposure up to unity.
For target in exposure , we define the prescribed mean observing quality as
where is the actual exposure duration and is a dimensionless normalisation. The factors , and describe instrument efficiency, transparency and sky quality; is seeing in units of 1 arcsec, is airmass, is the airmass exponent, and is the lunar factor. The indicator is unity for an available sightline and zero during closure or obstruction. We divide the exposure into intervals of duration and evaluate the factors at their midpoints , with . All factors include the relevant events and repairs. Closed or obstructed intervals have zero response and contribute to the elapsed duration . The product is the challenge effective exposure used below. Within each open interval, efficiency, transparency and sky quality multiply the time contribution, while seeing and the airmass factor enter in the denominator.
The airmass uses the functional form of F. Kasten & A. T. Young [19], evaluated with the simulator’s geometric zenith angle and normalised at the zenith (Appendix § 9.§ 2). Physical scattered-moonlight models calculate a contribution that adds to the sky background (K. Krisciunas & B. E. Schaefer [20]; A. Jones et al. [15]). In AOH, is a dimensionless multiplicative quality penalty determined by lunar illumination, altitude and target–Moon separation. Its amplitude and angular scale are challenge parameters. The normalisation sets the exposure scale, while controls the airmass dependence. Appendix § 9 specifies these functions and the numerical integration, and Table A1 lists the shared settings.
| Quantity | Symbol | Value | Role in the simulation |
| Observing geometry and time | |||
| Solar-altitude boundary (deg) | — | Defines astronomical darkness. | |
| Minimum target altitude (deg) | — | Enforced throughout the actual exposure. | |
| Weather interval (s) | — | Discretises conditions and usable night boundaries. | |
| Requested exposure duration (s) | — | – | Integer seconds; actual duration is limited by night and campaign boundaries. |
| Assignment-cell gap (deg) | Contiguous square cells; at most one target per cell per exposure. | ||
| Exposure-transition overhead (s) | — | The next decision follows the end of the exposure. | |
| Integration interval bound (s) | Midpoint quadrature, with additional splits at relevant boundaries. | ||
| Exposure response | |||
| Quality normalisation | Sets the dimensionless quality scale. | ||
| Airmass exponent | Controls the response dependence on airmass. | ||
| Lunar penalty amplitude | Sets the amplitude of the prescribed lunar factor. | ||
| Lunar-altitude exponent | Controls the lunar-altitude dependence. | ||
| Lunar angular scale (deg) | Sets the decay scale with target–Moon separation. | ||
| Observing-programme assignment | |||
| DARK band | Matching programme gives multiplier . | ||
| BRIGHT band | Matching programme gives multiplier . | ||
| BACKUP band | Matching programme gives multiplier . | ||
| Mismatched programme | Retains the unenhanced science-weighted response. |
Table A1. Common simulation parameters for the final E–H tasks.
For a geometrically eligible assignment, the raw response is
where and are the reference strength and exposure time for the task. A target with reaches after an effective exposure . Capping the response gives each target a finite return, creating a trade-off between deepening an exposure and moving to other targets. For a desired response , equation 2 gives the condition . At fixed , this corresponds to a duration threshold . The requested duration is an allowed integer number of seconds, and the actual exposure must reach within the observing window. When conditions change during an exposure, the effective exposure is calculated from equation 1.
The science-weighted contribution is . DARK, BRIGHT and BACKUP are interface labels for the high, intermediate and low bands of a synthetic quality statistic. Selecting the matching label earns a fixed score multiplier (Table A1). We assign the band using site atmosphere, sky quality, airmass and the lunar factor (equation (6)). This makes band identification and instrument-efficiency estimation distinct parts of the challenge. Airmass and lunar separation vary across a field, so each target has its own band. Instrument efficiency and directional event effects enter the exposure response in equation 1; at fixed site conditions and geometry, an efficiency loss can reduce the response while preserving the band. In DESI operations, observing programmes specify target populations and effective-depth requirements (E. F. Schlafly et al. [29]).
Survey progress and recovery
We retain the best valid responses so that each progress measure remains traceable to an exposure, including after data loss. For the set of its valid exposures, the retained science contribution and raw response are
with for an empty history. The two maxima are calculated independently and can correspond to different exposures. We adopt as the challenge completion criterion: at least one valid exposure must reach half the reference response. Targets completed below saturation can gain further response up to unity. Further visits can increase or and retain a usable measurement if an earlier exposure is lost. DESI combines spectra across exposures (J. Guy et al. [11]); § 4.7 discusses an extension based on cumulative depth.
A data-loss event invalidates a contiguous sequence of completed exposures. We recalculate , and request completion from the surviving observations and return the updated progress to the decision system. A request reward is withdrawn if the remaining exposures fail its completion criterion. The time spent on lost exposures remains part of the campaign duration, and replacement observations use the remaining observing time.
Task design, evaluation objective and reference methods
We compare policies within the shared environment of § 2 using a common decision interface and survey objective. Each task fixes the inputs and conditions for every evaluated system; tasks vary the available sky, observing season, assignment geometry and operational conditions. The objective combines retained target response with completeness, coverage and operational outcomes. Released reference methods provide common starting points and comparisons. § 4.1 specifies the execution budgets, repeated evaluations and score aggregation.
Task design and evaluation roles
The suite comprises five families of four tasks, separating local development, platform practice, public competition, duty-log extensions and held-out evaluation (Table 2). Development tasks provide complete scenario products for offline testing. Practice and competition tasks expose their static inputs and release observing information during a run. The held-out E–H scenarios remained on the evaluation side during the competition, with their catalogues and operational information supplied to frozen submissions through the prescribed interface. This separation tests policies on campaigns beyond those used for development.
| Card | Site | Lat. (°) | First night | Nights | Targets | Required |
| L1–L4: local development | ||||||
| L1 | Paranal | −24.62 | 2026-11-01 | 32 | 7 800 | 390 |
| L2 | Paranal | −24.62 | 2026-11-08 | 36 | 8 600 | 430 |
| L3 | Paranal | −24.62 | 2026-11-15 | 37 | 8 700 | 435 |
| L4 | Paranal | −24.62 | 2026-11-22 | 38 | 8 800 | 440 |
| α–δ: platform practice | ||||||
| α | Paranal | −24.62 | 2026-10-04 | 38 | 10 000 | 500 |
| β | Paranal | −24.62 | 2026-10-18 | 38 | 9 600 | 480 |
| γ | Paranal | −24.62 | 2026-10-09 | 38 | 9 900 | 495 |
| δ | Paranal | −24.62 | 2026-10-24 | 38 | 9 400 | 564 |
| A–D: public competition | ||||||
| A | Paranal | −24.62 | 2026-10-01 | 123 | 30 000 | 1 500 |
| B | Cape Town | −33.92 | 2027-06-01 | 183 | 50 000 | 2 500 |
| C | Mauna Kea | +19.82 | 2026-12-01 | 121 | 30 000 | 1 500 |
| D | Nemo1 | −45.00 | 2027-01-01 | 365 | 30 000 | 1 500 |
| A1–D1: duty-log extension | ||||||
| A1 | Paranal | −24.62 | 2026-10-01 | 123 | 30 000 | 1 500 |
| B1 | Cape Town | −33.92 | 2027-06-01 | 183 | 50 000 | 2 500 |
| C1 | Mauna Kea | +19.82 | 2026-12-01 | 121 | 30 000 | 1 500 |
| D1 | Nemo1 | −45.00 | 2027-01-01 | 365 | 30 000 | 1 500 |
| E–H: held-out evaluation | ||||||
| E | Ali | +32.54 | 2027-01-01 | 181 | 30 000 | 1 500 |
| F | Gavdosa2 | +34.84 | 2027-10-01 | 183 | 28 351 | 1 418 |
| G | Sahara2 | +18.94 | 2027-06-01 | 214 | 35 000 | 1 750 |
| H | Kunlun2 | −80.42 | 2027-04-20 | 128 | 30 000 | 1 500 |
Table 2. Observing campaigns, target catalogues and evaluation roles of the 20 tasks.
Across tasks, changes in site and season alter target visibility and observing time, while catalogue size and assignment geometry change the demand on each field and exposure. The public A–D tasks span two to four footprint components and approximately 4000–8000 deg. A1–D1 share the corresponding catalogues, footprints and night calendars, while adding duty-log records and using different realised weather, events and observing requests. Their comparison therefore includes changes in information, observing conditions and time-dependent priorities.
The final E–H tasks contain 28 351–35 000 targets over 128–214 nights, with approximately 5% designated as required. They combine different sites, seasons and assignment geometries (Figure 2). Task G pairs a small assignment field with the exposure normalisation of public task B; task H is a winter campaign at high southern latitude with pointing offsets and data loss enabled. All four include observing requests, instrumental faults and short duty-log notes. These configurations exercise the allocation of time and spatial capacity, the use of operational information and recovery under new campaign conditions. Table 2 gives the campaign and catalogue parameters; Appendix § 10.§ 1 gives the assignment and operational settings.
Decision interface and available information
At each observing decision, the decision system allocates the remaining observing time using the target catalogue, forecasts and feedback from earlier exposures. Initialisation supplies the public task inputs and scoring parameters; subsequent decisions use the current simulation time, released bulletins and forecasts, new messages, request progress and previous action outcomes. The system can choose a pointing, target-to-cell assignments, observing programme and exposure duration, wait for a later opportunity, report an instrumental fault, or end the campaign. Observations and waiting advance simulated time, with zero transition overhead between exposures. Reports consume computation time and have zero simulated duration.
Exposure feedback identifies geometrically accepted targets and gives their rounded programme-weighted scores. The raw response, quality and programme multiplier are evaluated internally, so policies must infer the relevant conditions and target progress from the available feedback. The reported cumulative target utility is ; the final score additionally accounts for completeness and operational outcomes. Every accepted action and its result are archived for reconstruction of the observing history. This defines a sequential decision problem with partial information, in which each observation both contributes to the survey and informs later choices. Appendix § 10.§ 2 gives the formal decision-process description and feedback fields.
Survey objective and observing trade-offs
To compare observing strategies, we score each campaign by its retained target response, completion of the required subset, coverage across right-ascension bands and operational outcomes. This score defines a common set of survey priorities for the benchmark. Using and from § 2.6, the settled campaign score is
where counts required targets with , measures the uniformity of completion fractions across populated 10-degree right-ascension bands, is the total observing-request reward, and is the net fault-report settlement. We use Jain’s index for (R. K. Jain et al. [14]); its definition and boundary conventions are given in Appendix § 10.§ 3. Using completion fractions accounts for differing numbers of catalogue targets between bands. Jain’s index is unchanged when all completion fractions are multiplied by the same positive factor, so it measures their balance; the target-response term rewards the observations themselves. Two-dimensional
**Note—**L1–L4 provide complete scenario products for local development. Practice and competition tasks provide static inputs and sequential observing information; A1–D1 add duty logs. E–H supply new campaigns to frozen submissions. First night gives the calendar’s night-date label; observing intervals are recorded in UTC. L3 begins at 2026 November 16, 00:30 UTC. Values are from the archived task products, using the A1–D1 versions active at the close of the competition. Assignment and operational settings are listed in Table B2. 2Simulated site at the stated coordinates. 1Fictional site.
completion maps show the finer spatial structure of the resulting sample.
The coefficients set the relative value of required completion and spatial balance. Each incomplete required target costs 50 points, while the uniformity penalty is bounded by 200. This gives required targets a strong completion incentive alongside the science-weighted return from the full catalogue. We use this scoring configuration for the reference campaign and held-out evaluation. The exploratory tasks modify selected scoring parameters (§ 4.6). Appendix § 10.§ 3 compares the scales of the individual terms.
Time-dependent requests add priorities within specified intervals. A request earns its stated reward when enough requested targets reach the required response using valid exposures wholly within the issue-to-deadline window; an incomplete request contributes zero. The released cards use a response threshold of 0.5. Under the standard scoring configuration, a correct report of
an active, reportable instrumental fault earns 100 and triggers repair. Two false reports are free at the start of a run and after each correct report; subsequent false reports cost 150 each until the next correct report. This allowance makes some diagnostic probes free of score penalties. The reward and penalty values set a common trade-off between detecting faults and making false reports. Data loss changes the valid target history and can withdraw completed-request rewards, as described in § 2.6. The final objective therefore reflects both the observing plan and the effectiveness of monitoring and recovery.
Reference methods and comparison protocol
We compare participant systems with two baseline models, denoted Baseline 1 and Baseline 2, that differ in how they estimate observing conditions and plan exposures. Each is implemented in Python, TypeScript and Rust. Baseline 1 selects exposures with a rule-based planner using target priority, remaining visibility and feedback from earlier observations. It searches candidate pointings around selected targets, fills assignment cells with accessible neighbours, selects duration using estimated gain per unit time, and chooses the observing programme from predicted target bands. Notices guide waiting and directional avoidance, and persistent response degradation can trigger a fault report. An optional language model provides advice to this planner.
Baseline 2 adds explicit estimation of observing conditions and instrument health and a joint search over pointing, assignment, duration and programme (Figure 4). Its planner balances expected gain against a cost for observing time. Optional language-model modules interpret operational text and forecasts and assess evidence for instrumental faults; their outputs modify parts of the numerical decision loop, with rule-based fallbacks. The two baselines therefore provide distinct combinations of state estimation, planning and model assistance. Their planning heuristics are described in Appendix § 10.§ 4; the common evaluation objective remains equation (4).
Figure 4. Decision loop of the baseline model. Exposure feedback updates estimates of observing conditions, instrument health and sky geometry. A rule-based planner jointly selects pointing, target assignment, exposure duration and programme using expected gain and an observing-time cost. Optional language-model modules process duty logs, forecasts and quality history; dashed arrows indicate where their outputs modify the loop, with rule-based fallbacks. Baseline 1 uses a simpler exposure-selection planner with optional language-model advice.
The final comparison uses the Python implementations of both baseline models published with the final leaderboard. Each has three scored runs on each held-out task, with the same score aggregation as the participant systems (§ 4.1). Their scores are reported in § 4.3. We use Baseline 2 as the baseline model for the L3 reference campaign (§ 4.2).
Evaluation and survey outcomes
We apply the common interface and objective of § 3 to examine survey outcomes under fixed task conditions. The L3 reference campaign links individual decisions to target completion. The E–H evaluation compares submitted systems across four held-out campaigns and repeated runs. We then examine representative strategies and exploratory tests of operational responses and revised survey priorities.
Evaluation sample and protocol
AOH brought together an international field of teams from multiple countries. The online competition ran from 2026 October 5 to 7, following online training on October 2–3. Teams of one to three members developed and tested their observing systems on the public tasks, then submitted them through the competition platform. Final submissions were evaluated on the four held-out campaigns E–H using a common scoring procedure. The published final leaderboard contains 102 ranked teams.
The participation and submission rules are available on the challenge website.3
Award eligibility required language-model-driven technology in at least two stages, such as interpretation, planning, action selection or adaptation. Deterministic components and external model services were permitted. This eligibility rule shapes the evaluated sample.
Each submission was fixed to a source commit and container image, with the submitted system isolated from the held-out scenario products and scorer. The published computation budget was 900 s of normalised time per run, with a 30-min elapsed-time cap. External inference services and external compute were permitted, so total resource use also depends on each system’s external services. Model-service use and cost remain relevant properties of the comparison ([18]). Task validity, outcome validity and transparent reporting are complementary requirements for agentic evaluation ([43]). Appendix § 11.§ 2 defines the computation accounting, and Appendix § 11 describes the execution and archival procedures.
Under the regular final protocol, each frozen submission is evaluated three times on each of E–H. The final score is the equally weighted mean of the four per-task means. Repeats measure variation of the submitted system on a fixed task, including stochastic decisions and service behaviour. Platform-attributable failures trigger replacement evaluations; the competition rules assign zero to team-attributable evaluation failures. Runs that reach a simulator termination code, including the elapsed-time limit, settle the observing history accumulated so far. The teams ranked second and 73rd were finalised using a different selection of valid runs. The results below use the published scores; § 4.5 describes these exceptions and Appendix § 11.§ 3 gives the full accounting.
Survey outcomes in a reference campaign
We run the baseline model on local task L3 with language-model advice disabled. We analyse its complete archive to relate exposure choices and operational responses to retained target completion. The policy completes the 37-night calendar, spanning 2026 November 16, 00:30 UTC to December 22, 08:15 UTC in simulated observing time, with 957 exposures. Of 8700 input targets, 6474 reach , including 434 of the 435 required targets: completion fractions of 74.4% and 99.8%, respectively. Both observing requests are met. The policy reports and repairs an instrumental fault eight days after its onset.
Figure 3(c) shows the events and information released during the campaign. The sharp efficiency loss around day 17 follows the first earthquake on December 3; an independent instrumental fault starts on December 7. Panel (d) compares the executed programme labels with the maximum weighted response bonus at fixed pointing, assignments and duration. Before the first earthquake, 18 of 582 choices fall below this reference, compared with 127 of 210 between that event and fault repair. After repair, five of 165 do so, across a period that includes a second earthquake. The baseline uses saturated exposure feedback to estimate the prescribed quality band separately from response quality. Its default fallback and fit selection also depend on response quality (Appendix § 10.§ 4). Instrument losses therefore bias band estimates within this feedback-based estimator. Appendix § 11.§ 4 gives the event times, state estimates and fault reports.
Programme labels identify bands of the synthetic quality statistic in equation (6). On November 23, near full Moon, 525 of 558 target–exposure records occupy the high band, labelled DARK, under the prescribed lunar factor and site conditions. On December 9, near new Moon, all 146 records occupy the intermediate band, labelled BRIGHT; nine of the 16 baseline exposures instead select BACKUP. These examples test the configured band rule and its inference from exposure feedback. The post-hoc reference compares the resulting score multipliers for each executed exposure.
Figure 5 shows target completeness as a function of position, class and feature strength. Completion generally increases with feature strength, as expected from the shorter exposure time needed at fixed observing quality. The modest decline in the highest bins coincides with an increasing fraction of low-weight stars. The highest bin contains 229 stars (): 197 are complete, including all 15 required targets; 19 have no surviving observation and 13 remain below threshold.
Figure 5. Target coverage in the L3 reference run of Figure 3 (baseline model, language-model advice disabled). (a) Completion fraction in equal-area cells of 2.5° in right ascension by 0.025 in , drawn at their celestial-coordinate limits. Light-grey cells contain no input targets; dark blue marks populated cells with zero completion. Overall, 6 474 of 8 700 targets are complete. (b) Completion versus feature strength , with total and required-target counts (, ) above each bin. At fixed observing quality, larger requires a shorter exposure to reach the threshold. (c) Fractions of each class with no surviving valid exposure record, with a valid record but incomplete, and complete; input counts appear above the bars. Among the 7 983 targets with valid records, 11 have zero response in every retained exposure.
Performance across held-out campaigns
The baseline models score 24 134 for Baseline 1 and 27 101 for Baseline 2 (Figure 6; Table D5).
Figure 6. Performance on the four held-out task cards. Shaded boxes span the 25th–75th percentiles of the 102 participant task means, and horizontal bars mark their medians (Weibull quartiles). Coloured symbols show the two baseline models, Baseline 1 and Baseline 2; error bars give the sample standard deviation of three runs. Symbols are offset horizontally for clarity.
| Entry | E | F | G | H | Final |
| Baseline 1 | 30 408 | 22 015 | 23 609 | 20 502 | 24 134 |
| Baseline 2 | 33 078 | 25 346 | 20 297 | 29 683 | 27 101 |
| 1 Pleiades | 37 543 | 31 005 | 31 412 | 34 343 | 33 576 |
| 2 starwing | 37 194 | 30 722 | 30 307 | 33 981 | 33 051 |
| 3 MiXingJiao | 37 312 | 30 206 | 29 441 | 33 435 | 32 599 |
| 4 CCB | 37 056 | 29 905 | 28 207 | 32 528 | 31 924 |
| 5 Argelander | 36 388 | 29 716 | 27 695 | 33 432 | 31 808 |
| 6 cloudbrain | 36 969 | 29 502 | 27 216 | 33 035 | 31 680 |
Table D5. Baseline models and the six highest-ranked submissions: per-task means and final scores reconstructed from the official run records.
The 102 ranked teams have a median final score of 27 047 and an interquartile range of 22 548–27 864 (Figure D3 in Appendix § 12.§ 1). Seventy-four teams exceed the published Baseline 1 entry, and 46 exceed Baseline 2, which has the higher aggregate reference score. The highest-ranked system scores 33 576 and leads on every task; the first three teams lie within 977 points and the first ten within 3 045. Table D5 gives the detailed scores.
Figure D3. Final results reconstructed from the official per-run score export. (a) Cumulative distribution of the 102 participant final scores, with the two baseline models. (b) Participant task medians and interquartile ranges (Weibull convention), compared with the same references. (c) 90% rank intervals from 2000 resamplings of the three counted runs per team and task; the diagonal marks the published rank. For the two exceptional entries, resampling is conditional on their selected best three runs.
Median team scores range from 20 462 on G to 33 251 on E (Figure D3b). The relative ordering of teams is broadly consistent across the four tasks, with tie-corrected Kendall’s . Each task combines a different catalogue, observing geometry, season and event history.
Representative observing strategies
We describe representative observing strategies using the organisers’ archived code-review report of 2026 October 9. The teams are identified by their published final ranks.
Figure 7 summarises representative methods and the task demands examined in the final and exploratory evaluations. Labels R1, R2, etc. identify the published final E–H ranks. The methods in its two columns can coexist within a system. All six leading entries use handwritten exposure planners, validate model outputs in code and fall back to rules. Their numerical components include greedy search (R1), a seasonal linear programme that sets observing-time prices (R2), Bayesian filtering (R1 and R6), and offline-trained condition and fault estimators (R3). Language models support strategy selection, weekly planning and extraction of facts from duty logs. Change-point detection (R2), per-exposure response inversion (R4) and reporting rules (R5) provide additional routes to fault diagnosis and recovery.
Figure 7. Numerical and language-model components of selected AOH systems. Rank labels R1, R2, etc. refer to the final E–H leaderboard. The upper row shows observing methods described in the organisers’ October 9 code-review report. The middle row shows fault-detection methods, model-assisted reporting gains measured for R7, R17 and R26 on X, and the notice-processing tasks Y and Z. The lower row gives the model-access settings for the same 27 selected systems in the transfer tests, using existing fallbacks when access is blocked. X1 changes climate, Y1 changes scientific priorities and scoring rules, and Z1 changes catalogue scale and operational disturbances. A system can use components from both columns. § 4.4 and § 4.6 describe the methods and results.
Appendix § 12.§ 1 gives the system-level descriptions; § 4.6 reports the tests summarised in the lower rows of the figure.
Run-to-run variability
Repeat variation limits how finely the final ranking can be interpreted. In 2000 bootstrap resamplings (random seed 20261024), the three counted repeats are sampled with replacement independently for each team and task; the final mean scores are then ranked. This gives 90% rank intervals spanning at most two positions for each top-ten team, but a median width of 20 positions for ranks 31–60 (Figure D3c). Across the 408 team–task groups, the median sample standard deviation is 307 points. Propagating the four task variances to each final mean gives a median standard error of 149 points across teams. The median gap between adjacent ranks is 92 points, and 42 teams lie between 26 500 and 28 000. Eighteen of 101 adjacent pairs differ by more than 1.96 combined standard errors. The middle of the ranking is therefore better interpreted as performance groups than as a precise ordering.
Two finalisations depart from the regular averaging protocol. The teams ranked second and 73rd were scored using the three highest-scoring valid runs on each task. Our summaries retain this published selection. Their bootstrap intervals describe resampling variability conditional on these selected runs. Selecting the highest observed outcomes can introduce selection effects; an analogous issue for model variants on Chatbot Arena is analysed by S. Singh et al. [31]. Here the selected units are repeated runs of submitted systems. The candidate counts and selection rule are therefore part of the evaluation record (Appendix § 11.§ 3).
The intervals quantify run-to-run variability on the four fixed task realisations. Service conditions can vary between replacement attempts and can be shared by concurrently launched repeats. Of the 1 200 counted runs for the 100 regularly finalised teams, 427 have a non-zero replacement counter; in 103 of their 400 task groups the three counted runs have no common execution-time overlap. These features can correlate or shift outcomes and limit interpretation of the nominal intervals. Detailed timing and termination records are summarised in Appendix § 11.§ 3.
Exploratory stress and transfer tests
Additional tests examine responses to operational text and changes in campaign conditions and scientific priorities (Figure 7). They use the frozen final versions of the 27 highest-ranked teams, with model calls routed to a single model and one run per tested configuration. Model-enabled runs are compared with runs that block model access and invoke each system’s existing fallback. The X task also supplies a structured-summary comparison. Appendix § 13 gives the task definitions and detailed outcomes.
In X, a guide-camera failure is announced in a noisy, multilingual duty log. Across 25 paired submissions, the median score difference between model-enabled and model-blocked runs is approximately +3 400. R7, R17 and R26 gain approximately 64 000–72 000 points with model access, correctly reporting 69–86 faults compared with 0–4 when access is blocked. The six highest-ranked systems score 26 000–33 000 with model access blocked; the severe efficiency loss makes the fault detectable through exposure feedback. Y uses subtle throughput changes and decoys, while Z adds engineering notices with revisions and references to earlier nights. The median score differences are approximately +400 on Y and −100 on Z. In these two tasks, most systems report few of the announced services.
| Rules & numerical planning | LLM-assisted systems | |
| Survey operations E–H | Greedy search (R1)<br>Linear programming (R2)<br>Bayesian filtering (R1, R6)<br>Learned estimators (R3) | Strategy selection (R1)<br>Weekly planning (R2, R4)<br>Log extraction (R1, R3, R6) |
| Operational text<br>X: Noisy logs<br>Y: Subtle faults<br>Z: Versioned notices | Change-point detection (R2)<br>Response inversion (R4)<br>Reporting rules (R5) | LLM-assisted reporting<br>(R7, R17, R26 on X) |
| Transfer tests<br>X1: Climate<br>Y1: Priorities<br>Z1: Scale | Model blocked<br>(R1–R27) | Model enabled<br>(R1–R27) |
Table 7. R denotes final rank. Methods can coexist within a system; examples are illustrative.
The transfer tests change climate (X1), scientific priorities and scoring rules (Y1), or catalogue scale together with operational disturbances (Z1). Each task supplies its new parameters in the public configuration. For model-enabled runs, the Spearman rank correlations with the final evaluation are , 0.38 and 0.87 for X1, Y1 and Z1, respectively. The largest changes occur on Y1, where large required-target penalties accompany the largest rank declines.
Discussion
We discuss the implications of these results for survey planning and operational diagnostics, together with the requirements for adapting the simulation to a specific observatory.
Scientific priorities. The final comparison shows that 46 of 102 systems improve on both baselines under the common survey objective. For the 27 selected systems, model-enabled transfer tests give rank correlations with the final evaluation of 0.38 for Y1, 0.85 for X1 and 0.87 for Z1. Y1 changes several scientific and scoring settings together, including the required-target fraction, the missed-target penalty and the false-report allowance. X1 changes climate, while Z1 changes catalogue scale and operational disturbances. A policy comparison for an observatory should therefore begin with its target priorities, depth and cadence requirements, which determine the value of each observation.
Survey diagnostics. The L3 diagnostics illustrate the challenge priorities at target level. The baseline completes 74.4% of all targets and 99.8% of the required subset, while meeting both observing requests. Its completion maps and class fractions locate the targets with no retained observation or a response below threshold. The archived sequence also traces band-selection errors to the baseline’s use of weighted response feedback. This finding concerns inference of the prescribed scoring bands. A facility study would compare inferred conditions with independent measurements of throughput, sky background and image quality, following the measured inputs used in exposure control E. F. Schlafly et al. [29]. On X, three systems correctly report more faults and achieve higher scores with model access. The large efficiency loss is also visible in exposure feedback, and the six leading systems retain high scores with model access blocked. These simulated results support further comparisons of diagnostic policies using both measured exposure outcomes and operational notices.
Completion criteria. The completion rule and time accounting determine exposure allocation. For one target at fixed observing quality, with all exposure records valid, the largest-single-exposure rule gives at least as much retained raw response when a fixed integration time is concentrated in one visit as when it is divided among shorter visits, provided that the single visit satisfies the duration and visibility limits. Zero transition overhead independently removes the time cost of slews, reassignment and readout. These assumptions shape the value of revisiting a field. DESI splits long observations to manage cosmic rays and changing geometry, and combines the spectra in its data processing [29, 11]. A facility model would use accumulated depth and measured overheads, together with the relevant exposure-splitting constraints. Imaging applications would also include filter choice and cadence.
Observatory applications. A facility comparison would combine target responses and site histories calibrated against exposure and monitoring data with an explicit model of feasible actions. OpSim models survey execution [8], while MAF evaluates observing histories through science metrics [16]. For a fibre spectrograph, feasible assignments depend on patrol regions and collision constraints [9, 30, 33]. Calibration exposures, measured slew and readout costs, and safety requirements would enter the observing model; Galactic extinction and crowding would enter target selection. Policies could then be compared on the same catalogue and condition histories using completeness, achieved depth, cadence and recovery time. Recording actions and outcomes would allow changes in survey performance to be traced to observing decisions. The released simulator and reference strategies provide a starting point for these facility-specific comparisons.
Conclusions
We have presented AOH, a simulation of survey observations with target assignment, exposure planning, time-dependent observing requests and recovery from operational interruptions. Among 102 systems evaluated on four held-out campaigns, 46 score above both baseline models. The leading system scores 33576 and has the highest mean score on each campaign. The six leading systems combine numerical exposure planners with language models for planning and interpretation of operational records.
The L3 reconstruction follows retained target completion through changing conditions and diagnoses errors in the baseline’s inference of synthetic quality bands. This provides a diagnostic example alongside the comparison across held-out campaigns. Exploratory tests show more correct fault reports with model access in several systems and substantial changes in ranking under revised scientific priorities. A facility application requires a science objective, calibrated exposure and operating models, and constraints appropriate to its instrument. The released simulation and reference strategies support comparisons of observing decisions under these specified models.
ACKNOWLEDGMENTS
AOH is supported by the GOSIM open-source community.
Generative-AI assistance was used to organise and edit this draft and to assist reference and implementation checks.
AUTHOR CONTRIBUTIONS
Haosong Wang contributed to manuscript writing, simulation development, and challenge engineering and organisation. Ting Tan and Huanyuan Shan co-led the challenge and contributed to manuscript writing and simulation development. Yao Li contributed to manuscript writing and challenge engineering and organisation. Dezi Liu, Xiangkun Liu, Yifei Luo, Wei Wang, Zonghuan Wu and Zhongxu Zhai contributed to challenge organisation throughout the project, provided scientific advice and reviewed the manuscript.
DATA AVAILABILITY
The AOH simulator, scoring code, reference implementations and documentation are available at https://github.com/gosimfoundation/hackathon-survey26. The repository also contains public replay data, including target catalogues, observing conditions and action histories. Released task bundles are listed at https://create.gosim.org/survey26/platform/resources, and the final leaderboard and campaign replays are available at [https://create.gosim.org/survey26/platform/](https://create.gosim.org/survey26/platform/ leaderboard/final-hidden. Our score analysis uses the official per-run export of 2026 October 11. Teams can access their full execution logs through the result repositories linked to their platform accounts.
Simulation geometry and response models
Table A1 summarises the settings shared by the final E–H tasks. We first describe observing geometry and exposure integration, then the generation of catalogues, weather and events.
**Note—**Entries describe the released task settings and shared numerical conventions. The response quantities are defined in equations (1), (2) and (6); the lunar factor is defined in Appendix § 9. Site, calendar and catalogue size vary by task (Table 2), as do assignment area and the response scales , (Table B2). Weather-generation coefficients belong to each scenario’s adopted parameter set.
Observing geometry and calendar
Target altitude is calculated from celestial coordinates and local sidereal time [32]. Solar and lunar positions use the simulator’s analytic approximations. Model twilight crossings are refined with a one-second stopping tolerance; dusk is rounded forward and dawn backward to an epoch-aligned 900-s grid. This retains complete weather intervals within astronomical darkness and gives a date-dependent night length. Conditions are stored as constant values within each interval. Exposure requests use integer-second durations. An exposure starting in a calendar gap ends at the earlier of its requested end and the campaign end, with gap time contributing zero response.
For assignment-cell side and gap , the grid pitch is and its outer side is in tangent-plane units; the final E–H configurations use . The gnomonic projection follows M. R. Calabretta & E. W. Greisen [6], with axes along increasing local altitude and azimuth at exposure start and a scale factor of . Cell membership is evaluated at exposure start and held fixed throughout the exposure. The projection applies to angular separations smaller than . A geometrically eligible assignment requires the actual field centre to lie in the altitude range , the target to occupy its assigned cell at exposure start, and its altitude to remain at or above throughout the actual exposure. The altitude limit corresponds to approximately in the adopted airmass model.
Exposure quality and numerical integration
We adopt the functional form of F. Kasten & A. T. Young [19] for the airmass,
where is the numerical value of the geometric zenith angle in degrees. The published approximation uses apparent zenith angle; inserting the geometric angle is a simulator convention. The division by normalises the airmass at the zenith.
The lunar factor is
where is lunar altitude, is target–Moon angular separation, and is the adopted illuminated fraction from Sun–Moon elongation . The E–H values of , , and are listed in Table A1; and use the same angular units. Trigonometric arguments are converted consistently to radians in the implementation. The factor reduces the response as the illuminated Moon rises or approaches the target. Its separable form assigns distinct controls to loss amplitude, altitude weighting and angular decay, with .
For programme selection, the condition-based quality is
Here , and are the stored site factors, including site-wide event effects, and is one for an open site interval and zero for closed intervals or calendar gaps. This prescribed statistic assigns the synthetic quality band used by the scoring rule. The DARK, BRIGHT and BACKUP thresholds and matching multipliers are listed in Table A1; an unmatched programme has .
Equations 1 and A2 use separate midpoint partitions. The full-response partition splits at weather boundaries and the start and end times of directional events. The programme partition splits at weather boundaries. Both subdivide each segment into pieces of at most 120 s. Directional-sector membership is evaluated at each piece midpoint; a target crossing a sector within a piece is therefore represented by midpoint sampling. Closed pieces contribute zero numerator with their duration retained in . The target altitude constraint is checked throughout the complete actual exposure before scoring.
Site-wide event factors modify the stored conditions before clipping to their configured bounds. Directional effects are evaluated from target altitude and azimuth at the integration midpoints; closures and obstructions give zero response there. Earthquake recovery is stored in a separate effect table and can extend beyond the initial event interval.
Footprints and target catalogues
For region , we choose a centre and place boundary vertices at equally spaced position angles , where . The angular distance of each vertex from the centre is
Here sets the angular scale, and set the amplitude and phase of harmonic , and is the number of harmonics. The amplitudes and phases are drawn using a fixed random seed, with amplitude bounds that keep all radii positive. With zero amplitudes, the vertices lie at equal angular distances from the centre; the harmonic terms introduce boundary variations, with larger describing finer angular structure. We join adjacent vertices by great-circle arcs and adjust until the spherical polygon matches its assigned area within the prescribed tolerance.
For each region , we sum the signed solid angles of triangles formed by its centre and adjacent vertices [35, 13]. We take the absolute sum, replace areas above by their complement to , and convert steradians to square degrees to obtain . Starting from the angular radius of a spherical cap with the assigned area , we update the radial scale as . We reconstruct the vertices and recompute the spherical area after each update, stopping when the relative area error falls below the prescribed component tolerance. The area ratio is dimensionless, with both areas expressed in the same units.
For a diffuse proposal, we select a footprint component in proportion to its area and draw a position in its enclosing disc, with angular radius and position angle , where and are independent uniform variates on . This radial law samples area uniformly in the plane of angular offsets. Clustered proposals use independent Gaussian east–north offsets around centres sampled within the footprint. We map each proposal onto the sphere and retain positions that lie within the footprint and have an observing window during the season. The spherical mapping and rejection cuts determine the final surface density and mixture fractions.
For a catalogue of size , class counts follow the specified fractions, with rounding corrections that preserve ; class labels are randomly assigned to the accepted targets. For class , we draw , where is the median feature strength and its logarithmic scatter [27]. All targets of class receive the science weight . The generator selects distinct target indices with equal probability as required targets, where is the specified fraction.
Weather and operational events
For weather variable , let clip to its configured bounds. The seasonal mean is , with . Here is the local night’s day of year, is the phase day, is the nominal value, is the amplitude, and is for seeing and for transparency and sky quality. The nightly state and intra-night value evolve as
Here indexes nights and indexes intervals within night . The coefficients set the persistence, and set the fluctuation scales. The variables and are independent standard normals, so the innovation standard deviations are . This scaling makes the configured fluctuation scale the stationary standard deviation of the corresponding constant-mean, unclipped autoregression. We initialise the states with and . The nightly update centres the previous state on the current night’s seasonal mean. Seasonal evolution, clipping and initialisation determine the resulting variances and correlations. This construction gives bounded weather histories with variations within and between nights. Background instrument-efficiency fluctuations follow a uniform distribution. Each task stores the resulting conditions, including event effects, for use by all evaluated policies.
For E–H, the nominal seeing is 1.10 arcsec, the seasonal amplitude is 0.15 arcsec, and the fluctuation scales are arcsec and arcsec. The persistence coefficients are and , with conditions stored every 900 s. The within-night seeing innovation therefore has standard deviation arcsec per interval. Site-wide event factors and closures modify these background conditions.
The availability chain is initialised in the open state before the first interval of each night, and a transition is drawn for each interval. The open-to-closed probability is , where and are the baseline and seasonal amplitudes. The closed-to-open probability is .
The earthquake efficiency loss is
where is the event magnitude parameter, the reference magnitude, the magnitude-scaling coefficient, the initial loss cap, the onset night and the recovery time in nights. The cap bounds the initial loss, and sets the number of nights for an e-fold decrease. These parameters control the severity and duration of reduced efficiency for testing adaptation. The efficiency multiplier is ; configured coefficients optionally also scale seeing, transparency and sky quality. Effects begin at the event timestamp and continue while their amplitude reaches or exceeds the activity threshold. All coefficients belong to the versioned scenario configuration.
Standard event bulletins use eight compass sectors or a site-wide designation; the initial bulletin also identifies terrain sectors. The reference forecast cadence and horizon are seven days, covering weather and launch events by night. Earthquake effects appear in current bulletins, while exposure feedback and duty logs provide information about instrument performance. Request generation screens targets for visibility and exposure requirements at nominal quality unity. This identifies candidate observing opportunities; realised conditions and assignments determine whether the issued request is achieved.
Task configurations and decision protocol
Complete task configurations
Table B2 gives the assignment, exposure and operational settings for the 20 tasks. Table 2 gives their observing campaigns, target catalogues and evaluation roles.
**Note—**Values are from the archived task products listed in Table 2. is the total assignment area in of tangent-plane coordinates. Requests counts observing requests with a completion reward; A1–D1 and E–H also provide free-text duty logs. Faults counts reportable instrumental faults. Stress marks cards with pointing offsets and the loss of earlier exposure records enabled. All cards use .
The five task families use the information-access rules described in § 3.1. Figure 2 shows the sky distributions of the final E–H catalogues.
Table B3 gives the target populations in E–H. The four catalogues share the same prescribed class fractions and science weights; integer target counts introduce small rounding differences in the realised fractions. Feature strengths are independently drawn for each catalogue, producing the empirical medians listed in the table.
| Class | Prescribed fraction (%) | Median : E | F | G | H | |
| Star | 8 | 0.30 | 4.012 | 4.008 | 3.958 | 3.996 |
| BGS | 24 | 0.45 | 1.256 | 1.257 | 1.250 | 1.260 |
| LRG | 20 | 1.00 | 0.349 | 0.350 | 0.349 | 0.348 |
| ELG | 34 | 1.00 | 0.552 | 0.552 | 0.549 | 0.551 |
| QSO | 14 | 1.70 | 0.184 | 0.181 | 0.179 | 0.180 |
| Targets | 30 000 | 28 351 | 35 000 | 30 000 | ||
| Required | 1 500 | 1 418 | 1 750 | 1 500 |
Table B3. Synthetic target populations in the final test catalogues E–H. Note—Computed from the registered final scenario bundles. The four cards use the same prescribed class fractions and science weights; the final four columns give empirical medians of the dimensionless feature strength , rounded to three decimal places.
Decision process and feedback
The decision system maintains its internal state throughout a campaign. At decision epochs , the task can be described as a partially observable Markov decision process [17]. The campaign has a fixed end time, while the number and spacing of decisions depend on the chosen actions. The state includes the clock, task inputs, complete observation record, telescope and instrument status, request and report accounting, and the unreleased weather and event schedule. The observation contains the information available to the system at . A policy maps its observation–action history to an action
Here specifies the pointing, assigns at most one target to each of the task’s cells, is the observing programme, and is the requested duration. The transition applies the action, advances the clock by its actual duration and updates the observation record and operational state. The episode ends at campaign termination or stop, with the settled score of equation (4) as its terminal return. Including the complete observation and settlement state retains the dependence of completeness, uniformity and request outcomes on earlier actions.
At each decision point the interface provides the current time, latest bulletin and forecast, newly issued messages, active-request progress, the sum of best target scores, the previous action result and the remaining computation budget. A report returns its diagnostic outcome and reward or penalty. Ordinary exposure feedback provides geometrically accepted target identities and rounded programme-weighted scores; the raw factors used by the scorer remain internal. State updates following data loss communicate the surviving target progress. Accepted actions are associated with their decision points and retained together with their outcomes.
Uniformity and operational settlement
In the adopted benchmark setting, a single missing required target costs 50, compared with a largest individual target contribution of , and the uniformity penalty is bounded by 200. These scales make required-target completion valuable relative to the marginal contribution of an individual target; the total target-utility term still accumulates over the full catalogue.
For each populated 10-degree right-ascension band , let be the fraction of catalogue targets with . Across the populated bands, Jain’s index is
We set when all completion fractions vanish; equal non-zero fractions give . Empty catalogue bands are excluded. Thus the statistic compares completion fractions across populated bands without weighting by their target counts. The uniformity penalty is . Two-dimensional completion maps, such as Figure 5a, complement this right-ascension statistic by resolving the angular structure of the selected sample.
For each observing request, the scorer takes the largest eligible raw response of each requested target using valid exposures wholly contained in the issue-to-deadline interval, including its endpoints. It counts targets reaching the request’s threshold and awards the specified reward if the required count is reached. The request contribution is recomputed after any supporting data loss. Fault reports are processed against the active reportable fault and the false-report allowance. Correct reports reset that allowance; up to 32 consecutive report actions are permitted. All report rewards and penalties are included in .
A data-loss trigger is processed at the first subsequent decision point, after completion of any exposure in progress. For completed exposures and configured fractional endpoints , , the invalidated action-index range is . The implementation preserves the original ledger entries and updates their validity flags.
Reference-policy heuristics
The released Python implementation of Baseline 1 ranks visible targets by science value, required status and remaining visibility, then searches fields around selected anchor targets. After selecting a pointing and assignment, its duration search uses weighted increases in squared response estimates and a bonus of 60 for crossing the required-target completion threshold. Among candidate durations, it can prefer one that reaches a needed request-target threshold if its heuristic gain rate is at least 90% of the best rate. The squared-response gain increases the relative value of larger predicted responses, while the threshold bonus favours completing required targets. These quantities rank candidate actions; the settled score uses equation (4), including the required-target penalty of 50. For programme selection, Baseline 1 scales its lunar and airmass model by an empirical response scale divided by a fixed reference efficiency of 0.95. The scale is inferred from exposure feedback after dividing out target strength, duration and modelled lunar and airmass factors. It therefore retains changes in instrument efficiency. The scorer’s programme-band quality in equation (6) uses atmospheric factors and target geometry.
Baseline 2 separately estimates the response scale used for exposure planning and the quality scale used for programme selection. Saturated target responses constrain programme matching through their observed score multipliers. With fewer than four such samples in the preceding two hours, the default implementation falls back to the response scale divided by 0.95; otherwise, equally consistent band fits are selected by proximity to that same estimate. Band inference consequently retains a dependence on instrument efficiency when feedback is sparse or ambiguous. The ratio of the two inferred scales supplies an instrument-efficiency proxy; the policy combines changes in this proxy with earthquake notices and the false-report allowance when deciding whether to report a fault. Its joint planner searches pointing, target assignment, duration and programme using incremental target value and an explicit cost of observing time. This cost balances the gain from the current exposure against time available for subsequent fields. Optional model stages provide night-level advice and fault assessment, with a further check before costly reports; these stages are disabled in the local L3 diagnostic run. The released planner also adjusts its search effort to the remaining computation budget. The archived actions preserve the diagnosed trajectory; a fresh policy execution also depends on measured computation costs. Source versions, settings and model outputs, when used, are needed to reproduce the evaluated configuration.
Competition, execution and reconstruction records
Competition sample and public rankings
The final leaderboard publishes 102 ranked participant entries and two baseline models. Its frozen data export, generated on 2026 October 11 at 10:37:50 UTC, provides run identifiers, per-run scores, task means, totals and execution metadata. The final-score analyses use this export.
Public resources, rules and the final leaderboard are linked in Data Availability.
Compute accounting
For decision window , let be elapsed time, the measured CPU-counter increment, and the calibrated duration ratio relative to a reference machine. The default metered implementation charges
The calibration uses a rolling median duration for a fixed workload, with the submitted process paused during re-calibration. A larger corresponds to a slower machine. If CPU metering becomes unavailable, the affected and subsequent windows use as their CPU surrogate. The elapsed-time cap begins at the first decision window and includes subsequent between-turn work. The public setting is 900 s charged time and 1800 s elapsed time; configuration and environment overrides belong in the archived execution record.
Figure C2. Regular held-out evaluation protocol. Each frozen submission is evaluated three times on each of four tasks, with platform-attributable failures replaced. The final score is the mean of the four three-run means. Exceptions and the timing of counted replacement runs are described in Appendix § 11.§ 3.
Execution, termination and exceptional finalisations
Figure C1 outlines the evaluation platform. Each submission is fixed to a source commit and container image. The submitted system and the simulator run in separate jobs and communicate through the interface in § 3.2. The simulator accesses the held-out scenario products and scorer. At termination, the platform publishes the settled score and archives the decision trace, exposure ledger and logs in a team-specific result repository.
Figure C1. Evaluation platform. Teams submit source versions, the scheduler queues isolated evaluations, and each submitted system exchanges decisions with the simulator, which accesses the held-out scenario. The platform archives the score, decision record and logs and publishes the score and replay. Colours distinguish the team side (blue), organiser platform (grey), isolated evaluation (orange) and outputs (green).
Each task runs in an isolated Linux container with two CPU cores and 2 GB of memory, subject to the computation accounting in Appendix § 11.§ 2. The initial three repeats for a task are launched concurrently; counted replacements may start later. Platform-attributable failures trigger replacement evaluations under the published rules.
A run is scored whenever it ends with a termination code, including runs stopped by the elapsed-time cap; the engine settles the observing history accumulated so far. Of the 1 224 runs selected for the published final ranking, 1 115 reached the end of the campaign calendar, 41 reached the elapsed-time cap, 28 were ended by the submitted system and five ended with an execution error. The remaining 35 have no termination code in the public export, although they have scored results.
The entries ranked second and 73rd are marked as exceptions to the averaging rule. Their published task means are reproduced by selecting the three highest-scoring valid runs per card, from 15–21 and 24–31 candidates per card, respectively. All other ranked entries contain three scored runs per card. These selections reproduce all 408 task means and all 102 final scores.
Of the 1 200 counted runs for the 100 regularly finalised teams, 427 have a non-zero replacement counter. In 103 of their 400 task groups the three counted runs have no common execution-time overlap; their median spread in start times is 9.3 h. The metadata record replacement counts and execution times. Shared service conditions and changes between attempts can correlate or shift outcomes, as discussed in § 4.5.
The platform archives complete decision traces, exposure ledgers and logs in team-specific result repositories. Public campaign replays provide released action histories, catalogues and observing conditions. The score distributions and repeatability statistics reported here are reconstructed from the official per-run score export; the target-level diagnostics in § 4.2 use the archived local L3 campaign.
Reference-campaign reconstruction
The L3 reference campaign uses the Python implementation of the baseline model with language-model advice disabled, default adaptive search settings and the card’s 900-s normalised computation budget. The archived runner, policy and task inputs are identified by cryptographic checksums. The system receives the catalogue, configuration and sequential information through the standard interface and completes 37 nights with 957 exposures. Replaying the archived actions reproduces the messages, target responses, validity flags and score components. Reconstructing the retained target maxima also reproduces the completion fractions and right-ascension uniformity penalty.
For the programme comparison in Figure 3(d), we hold each archived exposure fixed and evaluate its additional weighted response under each programme :
where contains the geometrically accepted assignments and ranges over DARK, BRIGHT and BACKUP. Raw responses, target weights and exposure geometry are unchanged. The prescribed matching bonuses are 20%, 12% and 6%, respectively. The total weighted response of this fixed exposure is . The second term is independent of programme, so maximising also maximises the exposure’s weighted response. The frozen scorer reproduces the archived multipliers for all 11 238 target–exposure records. Accounting for six-decimal response rounding gives 907 unique choices and 50 zero-response ties. The baseline model attains a maximum in 807 exposures, including the ties. This retrospective calculation uses the full simulated conditions and keeps the executed exposure sequence fixed.
The first earthquake begins at 03:45 UTC on December 3, initially multiplying the background efficiency by 0.164; the efficiency loss decays exponentially with a four-night time-scale. An instrumental fault starts at 07:00 UTC on December 7 with multiplier 0.443 and persists until a correct report at 07:00 UTC on December 15. The second earthquake begins at 06:15 UTC on December 16 with initial multiplier 0.649. These event and repair times define the stages in Table C4. Figure 3(b) shows nightly efficiency statistics, while panel (c) marks the exact event times.
| Stage | Exposures | Median | Median (s) | BACKUP choices | Below maximum bonus |
| Before first earthquake | 582 | 0.946 | 600 | 4 | 18 |
| Earthquake recovery | 90 | 0.473 | 300 | 39 | 53 |
| Instrumental fault | 120 | 0.369 | 600 | 70 | 74 |
| After fault repair | 10 | 0.930 | 2400 | 0 | 0 |
| After second earthquake | 155 | 0.798 | 900 | 0 | 5 |
Table C4. Stages of the L3 campaign with the baseline model, separated by instrumental-event and repair times in Figure 3(c).
All three events act on instrument efficiency and preserve the programme band at fixed atmosphere and geometry. Atmospheric events can change the band independently: a site-wide haze event overlaps December 7–11. On December 9, the observed targets have –, within BRIGHT. On November 23, –, giving 525 DARK and 33 BRIGHT records. These counts follow the synthetic quality thresholds and provide checks on the configured band assignments.
Programme mismatches concentrate between the first earthquake and fault repair (Table C4). The baseline model’s fallback and tie-breaking rules retain a dependence on the response scale (Appendix § 10.§ 4). Its fault diagnostic requires recent evidence and applies an earthquake-related reporting hold and an additional decline criterion. The policy makes a false report on December 4, repairs the fault on December 15 and makes another false report on December 17. Both false reports fall within the free allowance, giving a total reporting reward of 100. Following repair, five of the remaining 165 programme choices fall below the fixed-exposure reference.
The data-loss notification at 04:02:30 UTC on December 3 invalidates 12 target records from one earlier exposure, reducing the completed-target count from 4681 to 4677. At termination, 7983 targets have surviving exposure records and 6474 are complete, including 434 of 435 required targets. Each observing request completes six of its eight targets within its window and earns 100. Exposure-action time includes 12.33 h in closed intervals and exceeds open time on 19 nights. Closed intervals consume simulated time and contribute zero response, explaining why the exposure bars in Figure 3(d) can exceed the open-time curve.
**Note—**Exposures are grouped by start time; an exposure crossing a boundary remains in its starting stage. Efficiency medians use open environmental slots starting in each stage, including repair. The last two columns describe the baseline model. Below-maximum choices use equation C8 with all other exposure properties fixed. The 50 zero-response ties (38 before the first earthquake and 12 during the fault) count as attaining the maximum.
Published scores and representative systems
Detailed scores and illustrative systems
Table D5 gives the task-level scores of the two baseline models and six leading teams. Figure D4 compares the three leading systems using the organisers’ October 9 code-review report. The first maintains a belief state over observing conditions and faults. The second uses a seasonal linear programme to assign observing-time costs and a change-point detector for faults. The third combines a rule-based planner with two estimators trained offline and uses a language model to convert duty-log text into checked structured facts. These examples illustrate different ways to combine state estimation, numerical planning and operational information.
Figure D4. The three highest-ranked decision systems, mapped onto four operations at each decision step. The colour of each box shows what does the work in that step: a language model, a model trained offline by the team, or rules and numerical optimisation. The descriptions follow the organisers’ archived October 9 code-review report; rank labels refer to the published final leaderboard.
**Note—**Each task score is the mean of three runs, and the final score is their equal-weight mean over E–H. For starwing the counted runs are the three highest-scoring valid candidates per task (Appendix § 11.§ 3). Values are rounded only for display.
Exploratory observing tasks
These tests use the frozen submissions of the 27 highest-ranked teams, with one run per tested configuration. Model-enabled runs route calls to a common model service. Paired runs block model access and invoke each system’s existing fallback. The tests examine operational-text interpretation and adaptation to changes in survey conditions and priorities.
Operational-text tests
**Guide-camera failures ().—**Task X adds noisy operational logs to the public text-heavy tasks. The logs announce guide-camera failures and normal calibration periods using three languages, varied time formats, corrections, rumours and unrelated messages. A third configuration adds the same information as a one-line structured summary. At fixed observing plan, reporting each genuine fault at its announced start time gives a score of approximately +27 700; the same plan with fault reporting disabled gives approximately −70 400. Parsers developed for A1–D1 score approximately −68 000 to −69 000 on the new wording.
For model-enabled submissions, the Spearman correlation with the final ranking is . The original top ten have a median score of approximately +30 200, compared with +6 700 for ranks 11–27. Across 25 paired submissions, the median model-access difference is approximately +3 400, and the median difference between the structured-summary and original model-enabled configurations is approximately +200. R7, R17 and R26 gain approximately 64 000–72 000 points with model access (Figure 7). The severe efficiency loss also produces a clear signature in exposure feedback: the six leading systems score 26 000–33 000 with model access blocked.
Subtle faults (Y) and revised notices (Z).—Task Y announces a fibre-positioner service that lowers throughput by 15% for 30 min. Positioner self-tests produce the same throughput change and serve as decoys. Task Z announces atmospheric-dispersion-corrector services through engineering tickets. Tickets can be revised, postponed to the next open night, linked to earlier notices or withdrawn; informal operator messages add irrelevant information. Each task contains 96 announced service events and six background instrumental faults.
At fixed observing plan, correct interpretation of the service notices gains approximately 8700 points on Y and 8900 on Z relative to the control with fault reporting disabled. Reporting every throughput decrease loses more than 13 000 points in these comparisons. Across the 27 submissions, the model-enabled rank correlations with the final evaluation are for Y and 0.53 for Z. Two systems make at least 60 correct fault reports on Y; all systems remain below this count on Z. The median model-access differences are approximately +400 and −100 points, respectively.
Changes in climate, scientific priorities and scale
The transfer tasks provide their revised parameters in the public configuration. X1 adopts a monsoon climate. Y1 changes several scientific and scoring settings, including more than doubling the required-target fraction, multiplying its missed-target penalty by five and charging every false report. It also changes class weights, programme multipliers, uniformity and request settings. Z1 uses 40 000 targets over approximately 9000 deg, with additional faults, data loss and pointing offsets.
For model-enabled runs, the rank correlations with the final evaluation are , 0.38 and 0.87 for X1, Y1 and Z1, respectively. The largest rank changes occur under Y1’s combined configuration, where large penalties for incomplete required targets accompany the largest declines.
References
- [1]Andreoni, I., Margutti, R., Banovetz, J., et al. 2024, Rubin ToO 2024: Envisioning the Vera C. Rubin Observatory LSST Target of Opportunity program, https://arxiv.org/abs/2411.04793
- [2]Bianco, F. B., Ivezić, Ž., Jones, R. L., et al. 2022, ApJS, 258, 1, doi: 10.3847/1538-4357/ac3e72
- [3]Bonaldi, A., Hartley, P., Braun, R., Purser, S., et al. 2025, MNRAS, 543, 1092, doi: 10.1093/mnras/staf1466DOI
- [4]Box, G. E. P., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. 2015, Time Series Analysis: Forecasting and Control, 5th edn. (Hoboken, NJ: John Wiley & Sons)
- [5]Burleigh, K. J., Landriau, M., Dey, A., et al. 2020, AJ, 160, 61, doi: 10.3847/1538-3881/ab939b
- [6]Calabretta, M. R., & Greisen, E. W. 2002, A&A, 395, 1077, doi: 10.1051/0004-6361:20021327DOI
- [7]Chaussidon, E., Yèche, C., Palanque-Delabrouille, N., et al. 2023, ApJ, 944, 107, doi: 10.3847/1538-4357/acb3c2DOI
- [8]Delgado, F., Saha, A., Chandrasekharan, S., et al. 2014, in Proc. SPIE, Vol. 9150, Modeling, Systems Engineering, and Project Management for Astronomy VI, 915015, doi: 10.1117/12.2056898
- [9]DESI Collaboration, Abareshi, B., Aguilar, J., et al. 2022, AJ, 164, 207, doi: 10.3847/1538-3881/ac882b
- [10]DESI Collaboration, Aghamousa, A., Aguilar, J., et al. 2016, The DESI Experiment Part I: Science, Targeting, and Survey Design, https://arxiv.org/abs/1611.00036
- [11]Guy, J., Bailey, S., Kremin, A., et al. 2023, AJ, 165, 144, doi: 10.3847/1538-3881/acb212DOI
- [12]Hahn, C., Wilson, M. J., Ruiz-Macias, O., et al. 2023, AJ, 165, 253, doi: 10.3847/1538-3881/acff8
- [13]Jacobson, A., Kavan, L., & Sorkine-Hornung, O. 2013, ACM Transactions on Graphics, 32, 33:1. https://igl.ethz.ch/projects/winding-number/
- [14]Jain, R. K., Chiu, D.-M. W., & Hawe, W. R. 1984, A Quantitative Measure of Fairness and Discrimination for Resource Allocation in Shared Computer Systems, Tech. Rep. DEC-TR-301, Digital Equipment Corporation. https://www.cs.wustl.edu/~jain/papers/ftp/fairness.pdf
- [15]Jones, A., Noll, S., Kausch, W., Szyszka, C., & Kimeswenger, S. 2013, A&A, 560, A91, doi: 10.1051/0004-6361/201322433DOI
- [16]Jones, R. L., Yoachim, P., Chandrasekharan, S., et al. 2014, in Proc. SPIE, Vol. 9149, Observatory Operations: Strategies, Processes, and Systems V, 91490B, doi: 10.1117/12.2056835
- [17]Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. 1998, Artificial Intelligence, 101, 99, doi: 10.1016/S0004-3702(98)00023-X
- [18]Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., & Narayanan, A. 2025, Transactions on Machine Learning Research. https://openreview.net/forum?id=Zv4uFzMviVZ
- [19]Kasten, F., & Young, A. T. 1989, Appl. Opt., 28, 4735, doi: 10.1364/AO.28.004735DOI
- [20]Krisciunas, K., & Schaefer, B. E. 1991, PASP, 103, 1033, doi: 10.1086/132921DOI
- [21]Liu, X., Yu, H., Zhang, H., et al. 2024, in International Conference on Learning Representations. https://arxiv.org/abs/2308.03688
- [22]Mandelbaum, R., Rowe, B., Bosch, J., et al. 2014, ApJS, 212, 5, doi: 10.1088/0067-0049/212/1/5DOI
- [23]Myers, A. D., Moustakas, J., Bailey, S., et al. 2023, AJ, 165, 50, doi: 10.3847/1538-3881/aca5f9DOI
- [24]Naghib, E., Yoachim, P., Vanderbei, R. J., Connolly, A. J., & Jones, R. L. 2019, AJ, 157, 151, doi: 10.3847/1538-3881/aaefce
- [25]Neilsen, Jr., E. H., Annis, J. T., Diehl, H. T., et al. 2019, Dark Energy Survey’s Observation Strategy, Tactics, and Exposure Scheduler, Tech. Rep. FERMILAB-TM-2714-AE-CD-PPD, Fermi National Accelerator Laboratory, doi: 10.2172/1574836DOI
- [26]Neilsen, Jr., E. H., Bernstein, G., Gruendl, R., & Kent, S. 2016, Limiting magnitude, τ, t_eff, and image quality in DES Year 1, Tech. Rep. FERMILAB-TM-2610-AE-CD, Fermi National Accelerator Laboratory, doi: 10.2172/1250877DOI
- [27]NIST/SEMATECH. 2012, e-Handbook of Statistical Methods, doi: 10.18434/M32189
- [28]Raichoor, A., Moustakas, J., Newman, J. A., et al. 2023, AJ, 165, 126, doi: 10.3847/1538-3881/acb213DOI
- [29]Schlafly, E. F., Kirkby, D., Schlegel, D. J., et al. 2023, AJ, 166, 259, doi: 10.3847/1538-3881/ad0832DOI
- [30]Silber, J. H., Fagrel ius, P., Fanning, K., et al. 2023, AJ, 165, 9, doi: 10.3847/1538-3881/ac9ab1DOI
- [31]Singh, S., Nan, Y., Wang, A., et al. 2025, in Advances in Neural Information Processing Systems, Vol. 38, doi: 10.52202/085713-2620
- [32]Smart, W. M. 1977, Textbook on Spherical Astronomy, 6th edn., ed. R. M. Green (Cambridge: Cambridge University Press), doi: 10.1017/CBO9781139167574DOI
- [33]Smith, A., He, J.-h., Cole, S., et al. 2019, Monthly Notices of the Royal Astronomical Society, 484, 1285, doi: 10.1093/mnras/stz059
- [34]Terranova, F., Voetberg, M., Nord, B., & Pagul, A. 2023, in Machine Learning and the Physical Sciences Workshop at NeurIPS. https://arxiv.org/abs/2311.18094
- [35]van Oosterom, A., & Strackee, J. 1983, IEEE Transactions on Biomedical Engineering, BME-30, 125, doi: 10.1109/TBME.1983.325207
- [36]Voetberg, M., & Nord, B. 2023, DeepSurveySim: Simulation Software and Benchmark Challenges for Astronomical Observation Scheduling, https://arxiv.org/abs/2312.09092
- [37]Wang, C., Zhang, Y., Li, Y., et al. 2025, Communications Engineering, 4, 184, doi: 10.1038/s44172-025-00520-4
- [38]Wang, H., Tan, T., Yao, J., et al. 2026, Executable verification through formalized expert reasoning in astronomical spectroscopy, https://arxiv.org/abs/2607.06128
- [39]Wang, R., Jansen, P., Côté, M.-A., & Ammanabrolu, P. 2022, in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (Association for Computational Linguistics), 11279–11298, doi: 10.18653/v1/2022.emnlp-main.775
- [40]Xie, H., Kang, Z., & Jiang, X. 2024, Astron. Comput., 47, 100806, doi: 10.1016/j.ascom.2024.100806DOI
- [41]Yao, S., Zhao, J., Yu, D., et al. 2023, in International Conference on Learning Representations. https://arxiv.org/abs/2210.03629
- [42]Zhou, R., Dey, B., Newman, J. A., et al. 2023, AJ, 165, 58, doi: 10.3847/1538-3881/aca5fbDOI
- [43]Zhu, Y., Jin, T., Pruksachatkun, Y., et al. 2025a, in Advances in Neural Information Processing Systems, Vol. 38, doi: 10.52202/085713-5547










