betaFeedback

WorksComputer Science

When evaluation becomes intervention in clinical AI

Abstract

AI judges, models that grade another system's output against a rubric, extend clinical review to the scale at which clinical AI writes. When teams use judge assessments to select a prompt or train a model, the judge becomes part of the intervention. Selection can then favor errors the judge overlooks. We describe what a judge's report must contain and the independent clinical test each judge-driven change needs.

Clinical AI and the problem of trust

A clinical model can be trusted only as far as it has been checked against ground truth, the best available way to establish the condition it assesses.[1] For the risk calculators that have long guided practice, that reference was an outcome already present in the data. The American College of Surgeons NSQIP surgical risk calculator was built from 1,414,006 patients at 393 hospitals and predicts outcomes such as death and postoperative complications from 21 preoperative factors.[2] Whether each patient died or had a complication was recorded, so every prediction could be checked against what actually happened. Other tasks lack a recorded outcome, and experts must supply the ground truth. In one study, a model that read retinal photographs for diabetic retinopathy was trained on 128,175 graded images and validated against the majority decision of at least seven board-certified ophthalmologists.[3] Labeling at that scale was expensive, but it gave each image one agreed answer to check the model against.

Language models changed what has to be checked. A model that answers a medical question writes a paragraph, and there is no single correct paragraph to compare it with.[4] Agents are going further by using tools, acting in the health record, and talking directly with patients, as one diagnostic agent did with patients before their primary care visits.[5] For open-ended clinical text, the ground truth is clinical review: clinicians read each output against the source evidence and judge whether it is correct, complete, and safe.

Clinicians cannot review this text at the pace it is produced. One vendor's documentation system alone drafts notes for more than a million encounters a week.[6] To keep pace, the models that write clinical text are now also used to judge it, against clinician-designed rubrics and vetted examples.[7, 8] A modest set of clinician judgments can calibrate a judge that then assesses thousands of cases.[9] Judge assessments are also acted on. When one open-weight judge stood in for an expert panel in rating clinical texts, the preferred output changed in 41.7% of pairwise comparisons.[10] Teams use judge assessments deliberately to choose which version goes live or what a model is trained on. They also use them informally: a team that compares versions by their scores and keeps the higher-scoring one is selecting by the judge, whether or not it calls it that. Either way, a judge whose scores select a prompt or train a model has become part of the intervention it was built to measure.

Consider an AI scribe whose note leaves out that the patient was told to seek emergency care if chest pain returns. A judge whose rubric does not assess follow-up instructions could give the note a passing score, and the note would enter the record without the chest-pain instruction. Across eight judge designs, LLM judges distinguished an intact note from the same note with a fact removed little better than chance, though they detected added or altered content well.[11] If the team then keeps the prompt that earns the highest scores, or retrains the scribe on the judge's verdicts, the omission may persist or become more common in future notes. Trusting this judge would take two kinds of evidence. The judge would have to meet prespecified thresholds for detecting clinically important errors. Independent review would also have to measure missed errors among passing notes. A scribe tuned on its scores would have to write notes that are better on what matters clinically, without adding harm.

Existing guidance for clinical AI judges covers how to validate a judge and govern it over its lifecycle.[12] We build on that guidance by separating validation of a judge's measurements from independent testing of the changes its scores drive. We organize the evidence by how a judge's output is used: to observe or inform, to shape or select, or to supply training labels or rewards.

What an AI judge is, and the roles it plays

Defining an AI judge

We use the term AI judge for a model that grades another AI system's output against stated criteria and returns an assessment that people or systems then act on.[13, 14, 15, 16] In clinical AI, the criteria are usually a clinical rubric. The machine-learning literature calls this LLM-as-a-judge, a term introduced with studies of chatbot evaluation.[7] When the judge uses tools to inspect the steps an agent took as well as its final answer, it is called agent-as-a-judge.[17]

Not every automated check is a judge. Directly verifiable signals, such as dose arithmetic or whether a record reaches a specified state, can be checked against a preset rule. A judge is needed where checking an output requires interpretation. Whether a note captured what mattered in a visit, such as a medication the patient stopped because of side effects, can be decided only by reading the source evidence against clinical criteria. We focus on judges that apply written criteria a clinician can check. Reward models score outputs without written criteria, and when their scores select a version or train a model, the change needs the same independent test.[18]

How a judge's output is used

Consider again the AI scribe. A judge reviewing its draft notes might score each note on a rubric covering accuracy, completeness, and tone, or rank several drafts. For a single note, the judge's output may route the note to a clinician or trigger a revision before the clinician sees it. Across many notes, the scores describe how the scribe performs. They can also be used to select its prompt, modify its harness (the instructions, tools, and retrieval that surround the model), or train the model itself.[18, 19, 20, 21, 22, 23, 24]

A judge's output can serve three overlapping roles (Figure 1). It can observe or inform, as when it benchmarks a system or provides scores alongside cases a clinician is already reviewing, without changing which cases receive review or their priority. It can shape or select, as when it sets which cases clinicians review and in what order, blocks, ranks, or revises an output, or picks the prompt or configuration that goes live. It can also supply labels or rewards that change the model's learned parameters, or weights.

Roles of an AI judge and the evidence each requires

Figure 1. Roles of an AI judge and the evidence each requires. One judge can hold several roles. Every role requires a report that shows what the judge's scores mean. When the scores change a workflow, configuration, or model, the change must also be tested against clinically relevant performance measures assessed independently of the judge, with evidence scaled to clinical risk, reach, and reversibility.

The roles differ in who or what supplies the evidence for a change. If a clinician reviews the underlying case and decides what to fix, the clinician directs the repair. If the judge's verdict is enough to make the change, the judge drives it, whether it acts on its own or a person carries out its decision. Routing is the exception: even when a clinician directs every repair, a score that decides which cases receive review is a shape-or-select use.

What a judge must show

Agreement with clinicians shows how closely a judge follows their judgments on cases like those tested, under the stated rubric. Once the judge's output decides what the system says or which version runs, the claim changes: the resulting change must meet criteria set in advance for clinically relevant performance and harm. Measurement theory treats the consequences of using a score as part of its validity, so that claim needs its own evidence, independent of both the judge and the process that selected the change.[25, 26]

Defining and reporting what a judge measures

Defining the task and locking the judge

A single score hides which criteria were met.[13] Calibration can help a judge apply its criteria consistently but cannot supply a criterion the rubric omits. Rubrics can narrow during construction toward what can be labeled reliably.[27] In the scribe example, a missing follow-up criterion leaves the dropped chest-pain instruction untested, however well the judge is calibrated. Adding the criterion may not be enough: judges told to check completeness still detected removed facts near chance, and detection partly recovered only when each fact in the transcript was checked against the note.[11] Teams should lock the judge's full configuration before testing (Box 1). In one audit of three commercial scribes, changing only the judge's instruction raised the share of candidate errors it confirmed from 9.3% to 79.0%.[28] A vendor update to the underlying model creates a new version of the judge even when the product name is unchanged.[29, 30]

Box 1 | What a report on an AI judge should contain

Scope and configuration

  • Task, patient population, language, care setting, and agent role.

  • Locked configuration: model version, full prompts, rubric, example answers, input preparation, decoding settings, tools, and rule for combining item scores.

  • Criteria considered and dropped during rubric construction.

  • For multi-step encounters: how much earlier context the judge saw, whether it reviewed actions and tool calls as well as the transcript, and the step that introduced each error.

  • For panels: each judge's model, version, role, and model family, and how often the judges disagree and make the same errors, measured against independently collected clinician labels.

Comparison with clinicians

  • Matched-baseline design, blinding, and adjudication method set in advance, with the original ratings retained.

  • For each criterion, whether clinician disagreement signals an error or a defensible difference, and how the reference label was set.

  • Results by criterion, separately for cases where clinicians agreed and cases where they split, and whether different defensible labels would change the conclusions.

  • For yes-or-no criteria: prevalence in clinician ratings, the full two-by-two table with the positive class stated, and counts of indeterminate labels.

  • Sensitivity, specificity, predictive values, and an agreement coefficient with confidence intervals, or positive and negative percent agreement when clinician ratings serve only as a comparative method.

  • Rules for excluded cases, invalid outputs, and indeterminate labels, and whether verdicts were pooled across criteria.

  • Thresholds for the intended use, set in advance.

Samples

  • What reviewers could see, such as the transcript, the recording, or the record of the agent's actions.

  • Results with counts, separately for the matched baseline, the random sample, and enriched high-risk samples.

  • Random review of unflagged cases and of agreements, with auditors blinded to the judge's verdict.

Training labels and monitoring, where they apply

  • Source of each label, rater qualifications, blinding, inter-rater agreement, and adjudication.

  • Audit size and frequency, failure prevalence, trigger thresholds, and which changes the monitoring can detect and how quickly.

Multi-step encounters and panels

In a multi-step encounter, an error introduced at one step can affect care through later actions, and how much earlier context the judge sees changes what it finds.[31] A correct final answer can also follow an unsafe path, so the judge may need the record of actions and tool calls as well as the transcript.[32, 33]

A team can rely on one judge or several (Figure 2). Judges in a separate scoring panel assess outputs independently before their scores are combined, which may reduce errors specific to one judge and reveal disagreement.[34, 35, 36, 37] In one evaluation of a pediatric decision-support system, multi-model panels agreed with physicians no more closely than the best single judge.[38] Deliberation lets judges exchange reasons on contested cases but may lead them to converge on an incorrect judgment. Selective routing sends uncertain or high-risk cases to a specialist, a second judge, or a clinician, but it depends on the first judge recognizing which cases need escalation.

Judge architectures and shared sources of error

Figure 2. Judge architectures and shared sources of error. A single judge, separate scoring panel, deliberative panel, and selective routing can be used alone or in combination. Adding judges may reduce errors specific to one judge, but errors the judges share pass through every arrangement. JJ, J1J_1, J2J_2 and J3J_3 denote judges, and in d, J1J_1 is the first-stage judge and J2J_2 a second judge. Σ\Sigma, combination of separate scores. S, specialist review. C, clinician review.

Judges from different model families may still share errors,[34, 39, 40, 41] and shared reference answers that raise agreement[42] could also carry one error across a panel. In the same pediatric evaluation, judges agreed with one another more than with physicians.[38]

Comparing the judge with clinicians

A fair comparison needs a matched baseline, in which several clinicians independently apply the same rubric to the same in-scope cases. On the first pass, they should be blinded to the judge's score and confidence and to one another's ratings, with the method for resolving disagreements set in advance. This baseline helps distinguish disagreements specific to the judge from variation already present among clinicians.[43] In one audit of 124 urology consultations, three academic urologists applying the same guidelines to the same transcripts reached only slight to fair agreement on which flags from an AI auditor were meaningful, so the auditor's measured accuracy depended on whose judgment served as the reference.[44]

Criteria differ in what clinician disagreement means. For some, such as whether the emergency instruction was preserved, the note either kept it or did not, and disagreement signals an error to resolve through adjudication. For others, such as whether a note is complete enough, clinicians can disagree for defensible reasons, and the disagreement is information to keep. For the first type, the judge is measured against the adjudicated label. For the second, the claim is that the judge falls within the range of clinician judgments, and the report should show where it falls.

For each yes-or-no criterion, the two-by-two table shows what the judge missed, which aggregate agreement can conceal (Figure 3). In a pediatric decision-support evaluation, judges agreed with physicians on an ordinal safety scale, yet given a pass/fail prompt they passed 89–100% of the answers physicians rated as unsafe.[38] In one study of patient-facing answers, judges separated complete from incomplete responses at near chance, and when judge and clinicians agreed an answer was incomplete, they rarely named the same omission.[16] When adjudicated clinician labels form a reference standard set in advance and independently of the judge, sensitivity is the proportion of reference failures detected, and it should be reported with specificity, predictive values, and an agreement coefficient.[45, 46, 47, 48] Otherwise, positive and negative percent agreement replace diagnostic accuracy measures, and predictive values cannot be estimated.[49] A judge suits its intended use only if these estimates meet thresholds set in advance.

High agreement can hide missed failures

Figure 3. High agreement can hide missed failures. Each square is one AI-written note, labeled safe or unsafe by clinicians and by a judge for one safety item, whether an emergency-care instruction was preserved. a, In both panels a judge disagrees with clinicians on 10 of 100 notes, with five missed unsafe notes and five false alarms. When unsafe notes are common, sensitivity is 90%, and when they are rare, the same five misses leave it at 0%, while agreement stays at 90%. b, Two judges agree with clinicians on 95 of 100 notes. Judge A misses all five unsafe notes, and judge B catches all five with five false alarms. κ\kappa, Cohen's kappa; AC1, Gwet's AC1. The examples are constructed, with clinician labels designated as the reference; sensitivity is relative to those labels. Readers need failure counts and sensitivity to see what a judge missed.[45, 46, 47, 48]

Ongoing review and a complete report

After validation, random review of unflagged cases estimates the proportion of those cases containing missed problems. Targeted high-risk samples should be analyzed separately. Clinicians can adjudicate uncertain cases and panel disagreements, but a judge cannot refer failures it does not recognize. Adjudication therefore cannot replace random review. When disagreement triggers adjudication, a random sample of agreements should also be reviewed, because both sides may have made the same error.

Auditors should be blinded to the judge's verdict: in a mammography reader study, incorrect AI suggestions roughly halved experienced radiologists' accuracy.[50] A report is complete when an independent reader can reconstruct the evaluation from it (Box 1).[30]

Using a judge's output to change the system

How a system learns to please its judge

When a team uses judge scores to select prompts or train a model, it favors outputs the judge rewards. If the judge rewards an answer despite a clinical error, selection or training may make that error more common, and the judge's score can keep improving while clinically relevant performance plateaus or declines.[51, 52, 53] Known judge preferences for longer answers, particular positions, agreement with the user, or their own model family become directions in which the system moves.[39, 40, 41, 54]

In a medical rubric-training study, rubric-based judges preferred the trained model, whereas rubric-free judges rated its answers lower for factual correctness.[52] In a second clinical-consultation study, a reward that summed rubric items let the model delay the decisive instruction, such as calling emergency services, and recover the points with added advice. Appropriateness on held-out physician criteria fell below the untrained model.[55] Neither study measured harm to patients.

The same thing can happen without retraining the model. A team that tries many versions of a prompt and keeps the one the judge scores highest is also teaching the system to please the judge. Automated tools now run this search at scale and can match the gains of retraining.[56]

Keeping the test outside the loop

Methods that target evaluator bias, such as judges from different model families[35] or scores corrected for answer length,[41] leave clinical benefit untested, and shared errors can persist.[36] In one study, a prompt that cut judged unsafe overconfidence by 24.7 points also cut correct diagnoses by 2 to 58 points depending on the model, and a judge from another model family put the safety gain at 13.1 points.[57] A judge that guides a change should not supply the only evidence for the changed system's clinical performance, utility, or patient benefit.

The changed system should be compared head to head with the version it would replace, on a sealed set of cases that never informed development. Assessors should be independent of judge development and candidate selection, and blinded to system identity and judge scores. Acceptance or non-inferiority criteria should be set in advance, and harms and subgroup failures assessed separately. Each measure should match the claim, whether output quality, clinical utility, or patient outcomes.

A sealed set stops being independent once its results start guiding revisions, and rotating cases alone does not prevent this. Release decisions should rely on cases withheld from candidate selection or on a prespecified protocol that limits feedback from repeated testing.

Evidence should scale with plausible harm, the patients and decisions reached, clinician oversight, and how quickly a change can be reversed. A narrow, low-risk change may be supported by a blinded offline comparison and monitored rollout. Broad, high-risk changes that are difficult to reverse need prospective evaluation in the intended workflow. Prespecified rollback criteria should reverse deployment when clinical performance falls short, even if judge scores improve.

Where training material comes from

A judge's output can change a system by updating its weights, through fine-tuning, direct preference optimization (DPO), or reinforcement learning,[18, 19] or by rewriting its harness of instructions, examples, and tools.[56] A judge queried during training gives the model repeated chances to exploit errors in the reward. A fixed set of AI-generated labels leaves an auditable record, but its systematic errors may be hard to find. Hybrid schemes can reserve difficult cases for clinicians, provided random audits include AI-labeled cases and the source of each label is recorded.[58]

Several clinical models have been trained on judge scores, with benchmark gains and blinded clinician ratings in two reports.[21, 22, 23, 24, 59] Among these studies, none reports a prospective comparison isolating judge-generated from clinician-generated training labels using clinical measures independent of the training judge. The closest, ClinAlign, compared physician-revised with automatically drafted rubrics, on benchmarks only.[24]

For high-risk uses involving open-ended clinical judgments, we recommend that judge-trained candidates remain investigational until independent prospective evaluation supports the intended use. When the safety claim is that judge labels can replace clinician labels without material loss, we propose an additional comparison with an otherwise matched model trained on clinician-authored or independently adjudicated labels. The two models should share the base model, cases, label quantity, training methods, compute, and evaluation conditions, and the comparison needs a clinically justified non-inferiority margin. For a purchased model, where no matched counterpart can be built, the comparison is with current care, and the effect of the label source remains unresolved. Adopters should ask vendors where training labels came from.

Monitoring after release

After deployment, both the clinical system and the judge can change, and monitoring has to tell the two apart.

Watching the system and the judge

The clinical system changes when its model, instructions, or tools are updated, or when the record templates it reads change. The judge changes when its developer updates the underlying model, when its rubric is revised, or when it is recalibrated on new clinician labels, and a shifting case mix affects both.[60, 61] Regression sets can detect the return of known failures, and re-scoring a fixed set of clinician-labeled reference cases can reveal changes in how the judge behaves on those cases.[62] Fixed cases cannot show whether the judge is still reliable on the current case mix, so monitoring should pair them with clinician review of fresh cases. For a judge in the live pathway, the random and high-risk samples described earlier should cover outputs it rejects as well as those it accepts, because harm surveillance cannot rely on agreement with a judge whose rubric may be blind to the harm.

Detecting change and responding to it

Rising disagreement between the judge and clinicians on random audits may signal drift, a change in the judge's behavior over time. It may also reflect changes in the clinical system or the case mix. For the scribe, clinicians on audit might find more notes missing follow-up instructions while the judge's scores hold steady, a gap that only clinician review would reveal.

Risk-adjusted cumulative-sum (CUSUM) charts track accumulated departures from expected clinical performance,[63] and sequential probability-ratio tests, applied retrospectively to the Bristol and Shipman mortality data, weigh accumulating evidence of a change.[64] We propose applying them to how often clinicians disagree with the judge on audited cases. A threshold crossing, a known failure reappearing, or a report of serious harm should prompt review of the system, judge, rubric, and case mix, proportionate to the risk. Disabling a judge that blocks unsafe outputs or routes urgent cases may remove a safeguard, so the fallback should be verified before release.

Before a judge’s output changes care

The evidence base is young, as 19 of the 66 references are preprints, and several claims rest mainly on them, including the measured reward hacking in models trained on medical rubrics,[52, 53] shared errors among judges from different model families,[34] and methods for separating judge drift from system drift.[60, 62] Two references describe one vendor's own practice.[6, 8] Our specific evidence requirements are proposals whose feasibility and benefits need testing, and the size of the reported problems should be rechecked as peer-reviewed evidence accumulates.

FDA's guidance on predetermined change control plans (PCCPs) calls for justified reference standards and sequestered test data but sets no validation thresholds for LLM judges.[65] FDA's 2026 discussion paper on generative AI devices extends adjudicator independence to LLM adjudicators, suggests that modified devices could be rebenchmarked, and asks how a machine-based supervisory agent should itself be evaluated.[66] It does not say what evidence a judge-selected change needs. For a judge-selected update covered by an authorized PCCP, we propose that manufacturers document independent clinical evaluation of the change under the plan's Modification Protocol. Journals should require the judge report alongside any claim made for a judge-trained model.

Much of that evidence is still missing. In one review of 49 healthcare studies that used judges, 13 involved no human expert validation, and those that did used a median of three experts.[12] A judge may identify a promising change. Independent clinical evidence should decide whether it reaches patients.

Data availability

No datasets were generated or analyzed for this Perspective.

Acknowledgements

This work received no specific funding.

Use of AI tools

Claude (Anthropic) was used to assist with literature screening, revising the text and checking references against their sources. It was not used to draft the manuscript. The authors reviewed all output and take full responsibility for the content.

Author contributions

A.A.M. and J.K.J. conceived the Perspective, drafted the manuscript and revised it. A.C., S.S., J.P., R.I., J.K., K.S., Y.S. and L.K. contributed to the literature synthesis and revised the manuscript. L.M.A. and K.A. revised the manuscript critically for important intellectual content. A.M.C. and J.K.J. supervised the work. All authors reviewed and approved the final version.

Competing interests

S.S. and J.K.J. are authors of a cited benchmark,[33] S.S. is an author of a cited training study,[22] and K.S. is an author of a cited study.[42] J.K.J. is a co-founder of aprentis, a company developing infrastructure for accountable clinical AI autonomy. All other authors declare no financial or non-financial competing interests.

Correspondence

Correspondence to Anthony Abou Mrad ([email protected]).

References

References

  1. [1]Chen, P.-H. C., Mermel, C. H. & Liu, Y. Evaluation of artificial intelligence on a reference standard based on subjective interpretation. Lancet Digit. Health 3, e693–e695 (2021). https://doi.org/10.1016/S2589-7500(21)00216-8
  2. [2]Bilimoria, K. Y. et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J. Am. Coll. Surg. 217, 833–842 (2013).DOI
  3. [3]Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).DOI
  4. [4]Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023).DOI
  5. [5]Brodeur, P. G. et al. Conversational diagnostic artificial intelligence in ambulatory primary care: a prospective feasibility study. Lancet 408, 1732–1744 (2026).DOI
  6. [6]Oberst, M., Liang, D. & Lipton, Z. C. Pioneering the science of AI evaluation. Abridge white paper https://www.abridge.com/ai/science-ai-evaluation (2025).
  7. [7]Zheng, L. et al. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. Adv. Neural Inf. Process. Syst. 36 (2023).arxiv.org/abs/2306.05685
  8. [8]Chen, C., Khan, S., Oberst, M. & Chouldechova, A. Continual monitoring of note quality at scale: validating and using LLM judges. Abridge Engineering Blog https://tech.abridge.com/blog/continual-monitoring-validating-and-using-llm-judges (2026).
  9. [9]Feng, C. et al. Noisy but valid: robust statistical evaluation of LLMs with imperfect judges. In Proc. International Conference on Learning Representations (2026).
  10. [10]Pakull, T. M. G. et al. Cross-paradigm audit of large language model judges for clinical text evaluation. npj Digit. Med. 9, 850 (2026). https://doi.org/10.1038/s41746-026-03443-2
  11. [11]Fox, S., Markham, L., Lail, R. & Karotsieris, M. LLM judges verify presence, not absence: omission blindness in AI clinical notes and what recovers it. Preprint at arXiv https://arxiv.org/abs/2608.31016 (2026).
  12. [12]Li, C. et al. A scoping review of LLM-as-a-judge in healthcare and the MedJUDGE framework. Preprint at arXiv https://arxiv.org/abs/2604.25933 (2026).
  13. [13]Arora, R. K. et al. HealthBench: evaluating large language models towards improved human health. Preprint at arXiv https://arxiv.org/abs/2505.08775 (2025).
  14. [14]Li, L. et al. LLM-as-a-Judge in healthcare: a scoping analysis of applications, methods, and human alignment. Preprint at arXiv https://arxiv.org/abs/2605.25273 (2026).
  15. [15]Croxford, E. et al. Evaluating clinical AI summaries with large language models as judges. npj Digit. Med. 8, 640 (2025).DOI
  16. [16]DeLucia, A. et al. Same verdict, different reasons: LLM-as-a-judge and clinician disagreement on medical chatbot completeness. Preprint at arXiv https://arxiv.org/abs/2604.16383 (2026).
  17. [17]Zhuge, M. et al. Agent-as-a-judge: evaluate agents with agents. Proc. Mach. Learn. Res. 267, 80569–80611 (2025).arxiv.org/abs/2410.10934
  18. [18]Ouyang, L. et al. Training language models to follow instructions with human feedback. Adv. Neural Inf. Process. Syst. 35, 27730–27744 (2022).arxiv.org/abs/2203.02155
  19. [19]Rafailov, R. et al. Direct preference optimization: your language model is secretly a reward model. Adv. Neural Inf. Process. Syst. 36, 53728–53741 (2023).arxiv.org/abs/2305.18290
  20. [20]Lee, H. et al. RLAIF vs. RLHF: scaling reinforcement learning from human feedback with AI feedback. Proc. Mach. Learn. Res. 235, 26874–26901 (2024).arxiv.org/abs/2309.00267
  21. [21]Zhang, W. et al. RubricsTree: scalable and evolving open-ended evaluation of personal health agents across health memory and medical skills. Preprint at arXiv https://arxiv.org/abs/2606.18203 (2026).
  22. [22]Liévin, V. et al. ResidencyRL: reinforcement learning in simulated clinical environments. Preprint at arXiv https://arxiv.org/abs/2608.07418 (2026).
  23. [23]Nagar, A. et al. CLR-voyance: reinforcing open-ended reasoning for inpatient clinical decision support with outcome-aware rubrics. Preprint at arXiv https://arxiv.org/abs/2605.09584 (2026).
  24. [24]Lyu, S. et al. ClinAlign: scaling healthcare alignment from clinician preference. In Findings of the Association for Computational Linguistics: ACL 2026 16727–16747 (2026).
  25. [25]Messick, S. Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. Am. Psychol. 50, 741–749 (1995).
  26. [26]Jacobs, A. Z. & Wallach, H. Measurement and fairness. In Proc. ACM Conference on Fairness, Accountability, and Transparency 375–385 (2021).DOI
  27. [27]Valentine, M. A. et al. Constructing a classification scheme, and its consequences: a field study of learning to label data for computer vision in a hospital intensive care unit. Proc. ACM Hum.-Comput. Interact. 8, CSCW2, 490 (2024). https://doi.org/10.1145/3687029
  28. [28]Fox, S., Markham, L., Lail, R. & Karotsieris, M. One note in three: a verified census of three deployed AI scribes, and the instrument that counted it. Preprint at https://arxiv.org/abs/2608.31017 (2026).
  29. [29]Kellogg, K. C. et al. Large language models require a new form of oversight: capability-based monitoring. npj Digit. Med. 9, 375 (2026).DOI
  30. [30]Yang, Z., Hou, Y. & Yang, X. When the judge changes, so does the measurement: auditing LLM-as-judge reliability. Preprint at arXiv https://arxiv.org/abs/2607.08535 (2026).
  31. [31]Hassoon, A. & Dredze, M. Innovation-residual auditing of autonomous analysis agents: localization, detection limits, error control, and identifiability. Preprint at arXiv https://arxiv.org/abs/2608.05490 (2026).
  32. [32]Ruhrberg Estévez, S., Ferber, D., van der Schaar, M. & Kather, J. N. How to benchmark medical AI agents. PLoS Med. 23, e1005170 (2026).DOI
  33. [33]Schmidgall, S. et al. AgentClinic: a multimodal benchmark for tool-using clinical AI agents. npj Digit. Med. 9, 499 (2026).DOI
  34. [34]Mukherjee, S., Hamna, H., Bali, K. & Sitaram, S. The geometry of LLM-as-judge: why inter-LLM consensus is not human alignment. Preprint at arXiv https://arxiv.org/abs/2606.03043 (2026).
  35. [35]Verga, P. et al. Replacing judges with juries: evaluating LLM generations with a panel of diverse models. Preprint at arXiv https://arxiv.org/abs/2404.18796 (2024).
  36. [36]Eisenstein, J. et al. Helping or herding? Reward model ensembles mitigate but do not eliminate reward hacking. In Proc. First Conference on Language Modeling (2024).arxiv.org/abs/2312.09244
  37. [37]Bedi, S. et al. Holistic evaluation of large language models for medical tasks with MedHELM. Nat. Med. 32, 943–951 (2026).
  38. [38]Gershon, R. et al. Large language models as judges for clinical generative AI evaluation. npj Digit. Med. https://doi.org/10.1038/s41746-026-03248-3 (2026).
  39. [39]Panicksssery, A., Bowman, S. R. & Feng, S. LLM evaluators recognize and favor their own generations. Adv. Neural Inf. Process. Syst. 37, 68772–68802 (2024).
  40. [40]Wang, P. et al. Large language models are not fair evaluators. In Proc. 62nd Annual Meeting of the Association for Computational Linguistics 9440–9450 (2024).arxiv.org/abs/2305.17926
  41. [41]Dubois, Y., Galambosi, B., Liang, P. & Hashimoto, T. B. Length-controlled AlpacaEval: a simple way to debias automatic evaluators. In Proc. First Conference on Language Modeling (2024).arxiv.org/abs/2404.04475
  42. [42]Shi, K. et al. References improve LLM alignment in non-verifiable domains. In Proc. International Conference on Learning Representations (2026).arxiv.org/abs/2602.16802
  43. [43]Lovchinsky, I. et al. Discrepancy ratio: evaluating model performance when even experts disagree on the truth. In Proc. International Conference on Learning Representations (2020). https://openreview.net/forum?id=Byg-wJSYDS
  44. [44]Misrai, V., Bruchon, A., Campan, A., Loubes, J.-M. & Piau, A. Auditing what was said: the epistemic promise and limits of ambient AI in clinical practice. npj Digit. Med. https://doi.org/10.1038/s41746-026-03192-2 (2026).
  45. [45]Cohen, J. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960).DOI
  46. [46]Feinstein, A. R. & Cicchetti, D. V. High agreement but low kappa: I. The problems of two paradoxes. J. Clin. Epidemiol. 43, 543–549 (1990).pubmed.ncbi.nlm.nih.gov/2348207
  47. [47]Gwet, K. L. Computing inter-rater reliability and its variance in the presence of high agreement. Br. J. Math. Stat. Psychol. 61, 29–48 (2008).
  48. [48]Lee, C., Zeng, T., Jeong, J., Sohn, J.-y. & Lee, K. How to correctly report LLM-as-a-judge evaluations. Proc. Mach. Learn. Res. 306 (2026).arxiv.org/abs/2511.21140
  49. [49]US Food and Drug Administration. Statistical guidance on reporting results from studies evaluating diagnostic tests: guidance for industry and FDA staff. FDA https://www.fda.gov/media/71147/download (2007).
  50. [50]Dratsch, T. et al. Automation bias in mammography: the impact of artificial intelligence BI-RADS suggestions on reader performance. Radiology 307, e222176 (2023).DOI
  51. [51]Gao, L., Schulman, J. & Hilton, J. Scaling laws for reward model overoptimization. Proc. Mach. Learn. Res. 202, 10835–10866 (2023).arxiv.org/abs/2210.10760
  52. [52]Mahmoud, A. et al. Reward hacking in rubric-based reinforcement learning. Preprint at arXiv https://arxiv.org/abs/2605.12474 (2026).
  53. [53]Wang, X. et al. Reproducing, analyzing, and detecting reward hacking in rubric-based reinforcement learning. Preprint at arXiv https://arxiv.org/abs/2606.04923 (2026).
  54. [54]Sharma, M. et al. Towards understanding sycophancy in language models. In Proc. International Conference on Learning Representations (2024).arxiv.org/abs/2310.13548
  55. [55]Liu, M. et al. Scoring higher, answering worse: mitigating reward hacking in rubric-based RL via protocol-level rubrics. Preprint at arXiv https://arxiv.org/abs/2609.38847 (2026).
  56. [56]Agrawal, L. A. et al. GEPA: reflective prompt evolution can outperform reinforcement learning. In Proc. International Conference on Learning Representations (2026).arxiv.org/abs/2507.19457
  57. [57]Afrasyab, K. Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs. Preprint at arXiv https://arxiv.org/abs/2607.18086 (2026).
  58. [58]Miranda, L. J. V. et al. Hybrid preferences: learning to route instances for human vs. AI feedback. In Proc. 63rd Annual Meeting of the Association for Computational Linguistics 7162–7200 (2025).
  59. [59]Yang, Z. et al. Health-SCORE: towards scalable rubrics for improving health-LLMs. Preprint at arXiv https://arxiv.org/abs/2601.18706 (2026).
  60. [60]Kore, A. et al. Empirical data drift detection experiments on real-world medical imaging data. Nat. Commun. 15, 1887 (2024).
  61. [61]Finlayson, S. G. et al. The clinician and dataset shift in artificial intelligence. N. Engl. J. Med. 385, 283–286 (2021).DOI
  62. [62]Li, Y. Who drifted: the system or the judge? Anytime-valid attribution in LLM evaluation pipelines. Preprint at arXiv https://arxiv.org/abs/2606.15474 (2026).
  63. [63]Steiner, S. H., Cook, R. J., Farewell, V. T. & Treasure, T. Monitoring surgical performance using risk-adjusted cumulative sum charts. Biostatistics 1, 441–452 (2000).pubmed.ncbi.nlm.nih.gov/12933566
  64. [64]Spiegelhalter, D., Grigg, O., Kinman, R. & Treasure, T. Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int. J. Qual. Health Care 15, 7–13 (2003).pubmed.ncbi.nlm.nih.gov/12630796
  65. [65]US Food and Drug Administration. Marketing submission recommendations for a predetermined change control plan for artificial intelligence-enabled device software functions: guidance for industry and Food and Drug Administration staff. FDA https://www.fda.gov/media/166704/download (2025).
  66. [66]US Food and Drug Administration. Considerations for the regulation of generative AI-enabled medical devices: discussion paper and request for feedback. FDA https://www.fda.gov/medical-devices/digital-health-center-excellence/considerations-regulation-generative-ai-enabled-medical-devices-discussion-paper-and-request (2026).

Paper details

vlnc:W_a32swemq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 6

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Claude

View statement
Looking for related papers…
Contents