betaFeedback

WorksComputer Science

Knowing What You Know: A Trait Theory of Reviewer Value in Human-AI Cognitive Systems

Abstract

When a capable model and a human reviewer cooperate on the same decision, which human still adds value? We argue this is a question about a single cognitive trait, metacognitive calibration: the alignment between a reviewer's behavioral confidence and the actual correctness of their judgments. We develop a trait theory of reviewer value to make it precise. The theory makes five claims: value is governed by calibration rather than by activity, speed, agreement, or tenure; calibration leaves observable traces in behavioral telemetry; those traces are non-diagnostic on their own, because the fast, low-revision signature of a calibrated expert is indistinguishable from that of a confident novice until it is gated by outcomes; human calibration and a model's self-knowledge are the same construct, so the joint system should defer to whichever agent is better calibrated on each item; and individual calibration is causally identifiable only above the level of the single judgment. We ground each claim in a six-month production window of an AI-assisted accounting system (541621 reviewer events, 483 reviewers), and find that a purely behavioral expertise profile inverts the ground truth, scoring an overconfident cohort (5.2 percent correction survival) higher than genuine experts (99.2 percent); that reviewer quality is flat across tenure, so the trait is not learned on the job; and that the same calibration logic that separates good from bad reviewers, applied to the model, concentrates 92.5 percent of human effort on genuine errors. We close with the boundary the theory must respect: under majority-vote aggregation, individual causal value is unidentifiable.

Introduction

A growing class of cognitive systems pairs a capable statistical model with a human who reviews its output. The model proposes; the human confirms or corrects; the corrections become the next round of training data. Medical-imaging triage, content moderation, and the AI-assisted bookkeeping system we study here all share this architecture. As the model improves, an awkward question surfaces: most of the human reviews now agree with the model. In the system we analyze, expert bookkeepers verify model-categorized financial transactions, and over a recent six-month window they confirmed the model’s prediction unchanged 71% of the time. The humans are expensive and finite. Which of them is the joint system still relying on, and for what?

The instinctive answers are about activity: keep the reviewers who do the most work, who agree with their peers, who have been here longest, or who decide fastest and rarely change their minds. We will show that every one of these answers is wrong, and that they are wrong for the same reason. The property that actually governs a reviewer’s value to the joint system is not how much they do or how confidently they do it, but how well their confidence tracks their correctness, their metacognitive calibration [8, 9]. A reviewer who is fast and decisive and right is the most valuable agent in the system; a reviewer who is fast and decisive and wrong is the most dangerous, because the system, retrained on their confident corrections, inherits their error. Behaviorally, before we look at outcomes, the two are identical. This paper develops that observation into a trait theory of reviewer value, and tests it. We make three contributions to the study of human-AI cognitive systems.

A Trait Theory (Section 3). We state five claims about how a single human trait, metacognitive calibration, governs value in a model-human loop: that it dominates activity, agreement, and tenure as a predictor of quality; that it is partially observable in behavioral telemetry; that telemetry is non-diagnostic until conditioned on outcomes; that the human’s calibration and the model’s self-knowledge are one construct, which reframes division of labor as a calibration comparison; and that individual calibration is causally identifiable only above the single judgment.

Empirical Grounding (Sections 4-7). On 541,621 reviewer events we show that a behavioral expertise profile, built from exactly the telemetry signals trait psychology would recommend (review time, revision rate, response consistency), inverts when validated against outcomes: the cohort it ranks as most expert survives at 5.2%, the cohort it ranks lower survives at 99.2%. We show reviewer quality is flat across tenure (there is no onboarding curve), so calibration is a stable trait, not an acquired skill. And we show that the identical calibration construct, estimated for the model rather than the human, produces a self-knowledge signal that routes human attention to genuine errors with 92.5% precision.

The Identifiability Boundary (Section 8). We are explicit about what the theory cannot claim. Under the majority-vote aggregator that the deployed system actually uses, an individual reviewer’s causal contribution washes out: removing one reviewer rarely changes the aggregated label, so per-reviewer causal value is unidentifiable, and the only individual-level signal that survives is volume. The theory is therefore predictive at the individual level and causal only at the level of reviewer segments.

Our aim is explanatory in the sense the cognitive-systems paradigm values [16]: not a new metric that edges out a baseline, but an account of why the obvious ways of valuing human reviewers fail, organized around a cognitive trait that humans and models share. The account is, we think, the more useful for being falsifiable, and we report one place where our own first attempt at it falsified itself.

Background

Annotator Quality and Label Aggregation. A long line of work estimates annotator reliability to aggregate noisy labels, from the confusion-matrix model of Dawid & Skene [7] to learning-from-crowds methods [21, 24, 14]. These approaches score how accurately an annotator reproduces a latent ground-truth label. Our setting differs in two ways that matter for a cognitive-systems reading. First, there is no static ground truth: the “correct” category is itself partly what the system is trying to learn, and a capable model already supplies a strong prior, so the relevant question is not annotator accuracy in isolation but annotator value relative to a model that is often right. Second, recent work on annotator disagreement argues that variation among annotators is signal, not noise [20, 25, 11, 10, 6], but it studies disagreement among humans, whereas our central disagreement is between a human and a model.

Metacognition and Calibration. The trait at the center of our theory is studied under two names in two literatures that rarely meet. In cognitive psychology, metacognition is the monitoring of one’s own knowing [8], and its measurement, how well confidence discriminates correct from incorrect judgments, is a mature subfield [9]. The Dunning-Kruger effect [15] is precisely a calibration failure: low performers hold high confidence. In machine learning, calibration is the agreement between a model’s predicted probability and its empirical accuracy [12]. We treat these as one construct applied to two kinds of agent, which lets us ask a question neither literature poses on its own: in a human-model team, whose calibration should the team trust on a given item?

Behavioral-Trait Inference and Human-AI Teaming. That stable individual traits are recoverable from behavioral telemetry is established for personality [22, 1, 23]. A cognitive-systems account of personality goes further, arguing that descriptive trait models should be grounded in the structures and processes that generate behavior rather than merely cataloguing it [18]; our outcome-gating result (§ 5) is in that spirit, since the raw behavioral trait is descriptive and becomes explanatory of value only when tied to what produces correctness. We borrow the method (infer a latent trait from logged behavior), but apply it to a trait, calibration, that the personality literature does not foreground, and we report a cautionary result about doing so naively. Grounding such trait attributions in the structures and processes that generate the behavior is harder still for the machine agent, where it collides with the interpretability problem in modern machine learning [13]. Our measurement strategy sidesteps this deliberately: both traits are estimated at the telemetry level, the human’s from logged review behavior gated by outcomes (Appendix § 12) and the model’s from logged features of its predictions (Appendix § 13), so no claim in this paper requires opening either agent’s internal machinery. Finally, work on complementary human-AI performance [2] and scalable oversight [3, 4] asks when a team beats either member alone; our contribution is to locate the answer in a measurable trait of the human member and its counterpart in the model.

A Trait Theory of Reviewer Value

We consider a cognitive system in which, for each task instance, a model emits a prediction y^\hat{y} with internal confidence p^\hat{p}, and a human reviewer either confirms y^\hat{y} or corrects it to some y′y'. A correction is later judged to have survived if it is not overturned by a different reviewer. Let the reviewer be indexed by aa. We use correction survival as the operational proxy for whether a reviewer’s judgment was correct, because it is the only outcome the deployed system actually adjudicates; we return to its limits in § 10.

The theory comprises five claims.

Claim 1 (Calibration Primacy). The value a reviewer adds to the joint system is governed primarily by their metacognitive calibration: the degree to which the reviewer corrects the model when and only when the model is wrong, and not by their activity volume, decision speed, agreement with peers, or tenure. A maximally valuable reviewer overrides the model exactly on the items the model gets wrong and defers on the rest. The two ways to fail are symmetric and both costly: a reviewer who confirms model errors leaves them in the training data; a reviewer who “corrects” items the model had right injects errors. Calibration is the single axis along which both failures are reductions.

Claim 2 (Telemetry Observability). Calibration is partially observable in behavioral telemetry, without access to outcomes. The signals are temporal and sequential: how long a reviewer deliberates before deciding (review time), how often they revise their own past decisions (self-correction rate), how stable their behavior is across time (temporal consistency), and the run-length structure of their confirm/correct sequence. A well-calibrated reviewer and a poorly-calibrated one differ, on average, in these traces.

Claim 3 (Non-diagnosticity of Raw Telemetry). Behavioral telemetry alone cannot distinguish a calibrated expert from a confident novice. The same signature (fast decisions, few revisions, high consistency) is produced by a reviewer who is quick because they are right and by one who is quick because they do not deliberate. Telemetry measures decisiveness, and decisiveness is expertise only when it is accompanied by correctness. A behavioral trait therefore becomes diagnostic of value only when it is conditioned on outcomes; the raw trait, uncorrected, will systematically over-rank the overconfident. This claim is the theory’s sharpest, most falsifiable prediction, and § 5 tests it directly.

Claim 4 (Agent Symmetry). The human reviewer’s calibration and the model’s self-knowledge (its estimate of whether its own prediction is correct) are instances of one construct. Writing P(IK)P(\mathrm{IK}) for a calibrated estimate of the probability the model already knows the answer, the joint system’s division-of-labor question becomes a calibration comparison: route a task to the human only where the better-calibrated judgment is the human’s, i.e. where the model’s self-knowledge is low. Calibration is thus not only how we should value humans but how the system should allocate them.

Claim 5 (Identifiability Bound). Individual calibration is causally identifiable only above the level of the single judgment. When the system aggregates multiple reviewers’ judgments on recurring task groups by majority vote, removing any one reviewer rarely changes the aggregate, so the causal contribution of an individual reviewer’s calibration is not identified from outcome data; what remains identifiable is the causal value of reviewer segments. The theory is therefore predictive at the individual level and causal at the segment level, and a careful account must not confuse the two.

These claims are deliberately ordered from the substantive to the cautionary. Claims 1 and 4 say calibration is what matters and that it is shared by both agents. Claims 2 and 3 say it is observable but treacherously so. Claim 5 fences off the inference the data cannot support. The rest of the paper grounds each in a production system.

Setting and Data

The Cognitive System. We study a deployed AI-assisted accounting platform. Each financial transaction is assigned a predicted categorization over several fields (transaction type, vendor, chart-of-account, payment processor) by a production model. A human bookkeeper then reviews the prediction and either confirms it or corrects one or more fields. Corrections re-enter the model’s training signal. The reviewer population is professional and recurring: a given bookkeeper typically serves a fixed set of client businesses over months. We take the unit of analysis to be the reviewer-business pair aa, since a bookkeeper’s calibration is specific to the client whose books they know.

Data. We extract every human verification event over a six-month window (2025-12-11 to 2026-06-11): 541,621 events from 483 reviewers across 1,184 businesses, of which 71% confirm the model and 29% correct it. Restricting to pairs with at least 20 verification events to allow stable trait estimation yields 3,579 reviewer-business pairs. All data are operational logs generated under existing employment agreements; every statistic we report is aggregate, and no individual is identifiable from any reported number.

Outcome Measure. A correction survives if it is not subsequently overturned by a different reviewer (self-reverts and automated reverts do not count). Survival is the system’s own revealed judgment of whether a correction was right, and it is the dependent variable against which we validate every behavioral trait below.

Trait Telemetry. From the same logs we compute the behavioral signals of Claim 2 for each pair: mean review time (seconds between the task being surfaced and the decision), self-correction rate (fraction of the pair’s own decisions it later revised), temporal consistency (inverse coefficient of variation of the pair’s period-to-period accuracy on a fixed benchmark), and a run-length consistency score over the confirm/correct sequence. We assemble these into a behavioral expertise profile (the construct Claim 3 predicts will be non-diagnostic on its own), defined in Appendix A. We emphasize that these are precisely the telemetry features a trait-inference approach [22] would select; the exercise is to see what they capture when the latent trait is calibration rather than personality.

Behavioral Telemetry Inverts Without Outcome Gating

Claim 3 predicts that a behavioral expertise profile, uncorrected by outcomes, will over-rank the overconfident. We test it directly. The behavioral profile (call it BTP, the behavioral trait profile) rewards fast, low-revision, consistent reviewing. Across the 3,579 pairs it captures something real and distinct: it is negatively correlated with a reviewer’s self-correction rate (r=−0.74r = -0.74), which is what a decisiveness measure should do, and it is nearly uncorrelated with the rest of the value composite, so it is not redundant. The question is what that “something” is.

It is not expertise. Pairs scoring high on the behavioral profile survive their corrections at 68.8%, while low-profile pairs survive at 98.7%: the behavioral trait points the wrong way. The mechanism is visible when we split the low-revision reviewers (those the profile most rewards) by their actual correction survival (Table 1). Among reviewers who rarely revise themselves, there are two populations a behavioral measure cannot tell apart: genuine experts, whose corrections survive 99.2% of the time, and an overconfident cohort, whose corrections survive 5.2% of the time. The behavioral profile scores the overconfident cohort higher (76.2 vs. 70.1): they are faster and revise even less, because they are not deliberating at all. This is the Dunning-Kruger pattern [15] recovered from production telemetry, and it is exactly the failure Claim 3 predicts.

Cohort (low self-revision)nnBehavioral profileCorrection survival
Expert (survival ≥85%\ge85\%)1,21170.199.2%
Overconfident (survival <85%< 85\%)57776.25.2%

Table 1. Among low-revision reviewers, correction survival, not behavior, separates genuine right-first-time experts from the overconfident. A purely behavioral trait inverts: it scores the overconfident cohort higher.

The remedy the theory prescribes is to gate the behavioral trait on outcomes: count behavioral decisiveness as expertise only for reviewers whose corrections survive. Once gated, the surviving signal is genuine, but it is now carried by the outcome condition, not by the telemetry: the behavioral component alone correlates only weakly (r=0.07r = 0.07) with catching errors on genuinely hard cases. We read this as confirmation, not refutation, of the theory: a behavioral trait is necessary infrastructure for identifying calibrated expertise (it tells you who is decisive), but it is not sufficient, because decisiveness must be conditioned on correctness before it means anything. Reporting this inversion, rather than quietly dropping the behavioral profile, is the point: it is the cleanest evidence we have for Claim 3.

Calibration Is a Trait, Not a Tenure Effect

If calibration were skill acquired on the job, reviewer quality should climb with experience, and the right policy would be to retain and train rather than to select. Claim 1 instead treats calibration as a comparatively stable individual trait. We test this by measuring correction survival as a function of reviewer tenure, defined as all-time weeks since the pair’s first verification (Table 2).

Tenure (weeks)0-11-22-44-88-1616+
nn (events)26,1955,5477,97710,52812,30344,897
Survival98.6%99.1%98.7%99.2%98.5%99.4%

Table 2. Correction survival is essentially flat across reviewer tenure: there is no onboarding curve. Calibration behaves as a stable trait rather than an acquired skill.

Survival never leaves the 98.5–99.4% band, from a reviewer’s first week to beyond their sixteenth. There is no onboarding cliff and no steady climb. Whatever separates the 5.2%-survival overconfident cohort of the previous section from the 99.2%-survival experts, it is present from week one and does not develop with exposure. This is consistent with calibration being a trait the selection policy should measure and route on, not a deficit that more reviewing will cure, and it sharpens Claim 1’s contrast with the tenure-based heuristic, which Table 2 shows would add no discriminating information at all.

The Same Calibration, Estimated for the Model, Allocates the Humans

Claim 4 holds that the model’s self-knowledge and the human’s calibration are the same construct, and that the joint system should spend human attention where the model’s calibration is poorest. We make this operational by learning P(IK)P(\mathrm{IK}), a calibrated estimate of the probability the model’s prediction will be confirmed, from logged features alone (gradient-boosted trees with isotonic calibration; Appendix § 13). Two results follow.

First, model self-knowledge is a genuine, distinct signal, not a relabeling of the model’s raw confidence. P(IK)P(\mathrm{IK}) is well calibrated (expected calibration error 0.037) and predicts human correction far better than the model’s own confidence score p^\hat{p} (AUROC 0.845 vs. 0.662, on held-out businesses; post-hoc calibration of p^\hat{p} alone repairs its calibration but not its ranking, AUROC 0.672). Its correlation with raw confidence is only moderate (r=0.39r = 0.39): a model can be confident and wrong, and P(IK)P(\mathrm{IK}) is exactly the metacognitive layer that knows the difference, the machine analogue of the human calibration trait of § 5.

Second, allocating by this calibration signal concentrates human effort on real errors. Routing the lowest-P(IK)P(\mathrm{IK}) 20% of held-out-business transactions to humans and auto-confirming the rest sends reviewers a set that is 92.5% genuine corrections: human attention is spent almost entirely where the model is actually wrong. The comparison that isolates calibration as the operative property is the head-to-head against routing by the model’s raw confidence at the same 20% human budget: confidence-routing leaves 36.3% of real corrections in the auto-confirmed set, whereas P(IK)P(\mathrm{IK})-routing leaves 29.1%: raw confidence misdirects 1.25× as many genuine errors into auto-acceptance. The difference is not the amount of information but its calibration, which is precisely Claim 4’s content carried over to the model side.

The symmetry is the conceptual payoff. The human failure mode of § 5 (high confidence, low correctness) is the same failure P(IK)P(\mathrm{IK}) corrects for on the model: high p^\hat{p}, low actual correctness. A single trait, metacognitive calibration, both ranks the humans and, estimated for the model, decides which humans the system needs.

What the Theory Cannot Claim: An Identifiability Boundary

Claim 5 fences off an inference the data do not support, and we state it plainly because the temptation to overreach here is strong. It is natural to want to assign each individual reviewer a causal value (how much worse would the system be without this person?) and to validate the calibration trait against it. We attempted exactly this, estimating each pair’s marginal contribution to a faithful reproduction of the production model via Monte-Carlo Shapley values over the held-out test set. The estimator is sound: the resulting values are well spread (116 of 120 sampled pairs nonzero, range −0.12 to 1.0). Yet they correlate with no quality dimension (not the calibration trait, not survival, not any AVSM dimension), and significantly with only one thing: raw label volume (r=0.24r = 0.24, p<0.002p < 0.002).

This is not a measurement failure; it is structural, and it follows from the system’s aggregation rule. The deployed model learns a categorization per recurring (business, pattern) group by effective majority vote over the reviewers who have touched that group. Within such a group, calibrated reviewers converge on the same correct answer, so swapping one of them in or out almost never changes the aggregated label. The individual reviewer’s identity washes out of the outcome; the only per-reviewer quantity that still moves the aggregate is how many distinct groups they touch, i.e. volume. Under majority-vote aggregation, individual causal value is unidentifiable, and any framework, including ours, that claims to have validated an individual-level quality trait against causal outcomes is, in this regime, measuring volume.

What is identifiable is value at the level of reviewer segments. Removing an entire behavioral segment (say, the cluster of reviewers who almost never correct the model) changes the coverage of the aggregator and therefore moves measurable outcomes. In our setting, counterfactually retraining the model with the redundant “rubber-stamp” segment removed shifts confirmation by only −0.31 percentage points, and pruning to the high-calibration segment raises correction survival from 98.4% to 99.0%: the low-value segments are causally redundant, and the high-value segment causally improves quality, both at the segment grain. This is the precise sense in which the trait theory is causal at the segment level and only predictive at the individual level. We regard stating this boundary as part of the contribution: a cognitive-systems account of human value should be honest about the aggregation regime that determines what “value” can even mean.

Discussion

One Trait, Two Agents, One Allocation Rule. The throughline is that a single cognitive construct, calibration (the alignment of confidence with correctness), answers three questions usually treated separately: which human reviewers are good (§ 5), whether quality is selected or trained (§ 6), and how the system should split work between human and model (§ 7). The unification is possible only because we treat the human’s metacognition ([8]) and the model’s self-knowledge ([12]) as the same thing measured on two agents, a move squarely in the cognitive-systems tradition of seeking unified accounts of intelligent behavior in humans and machines ([17]). For a cognitive-systems audience, the claim of interest is that a joint human-AI system has a shared metacognitive currency, and that its design reduces to comparing the two agents’ calibration item by item.

Why the Activity Heuristics Fail. Each rejected heuristic fails because it correlates with calibration only spuriously. Volume is the residual that survives aggregation, not a quality signal (§ 8). Agreement rewards conformity, which an overconfident clique supplies as readily as an expert one. Speed and low revision (the behavioral profile) measure decisiveness, which is expertise’s signature only after the fact (§ 5). Tenure does not move quality at all (§ 6). The heuristics are not merely weaker than calibration; they are confounds that a careful account has to actively strip out.

Implications for Human-AI Teaming. If calibration is the load-bearing trait, the design levers change. Selection should measure calibration, not seniority; allocation should route by the model’s calibration deficit, not by reviewer availability; and training, if quality is a stable trait, has limited leverage over the trait itself but could plausibly target its expression, e.g. slowing the overconfident cohort’s decisions. The calibration-training literature is consistent with this division: explicit, repeated outcome feedback can improve calibration ([19, 5]), and that is exactly the channel that ordinary tenure in our system does not supply (§ 6). Time on the job leaves the trait where it was; a deliberate outcome-feedback regime remains an open, and more hopeful, lever for the overconfident cohort. Crucially, because telemetry is non-diagnostic (Claim 3), none of these levers can be driven by behavioral signals alone; they all require an outcome channel (here, correction survival) to gate the behavior.

Limitations and Scope

Survival as a Correctness Proxy. Our outcome measure is whether a correction is overturned by another reviewer. This inherits the reviewer pool’s own calibration: a correction that survives in a pool of overconfident reviewers is not guaranteed correct. We mitigate this by requiring the overturning reviewer to be different, and by the fact that the high- and low-survival cohorts separate so sharply (99.2% vs. 5.2%) that pool-level noise cannot explain the gap, but a fully external ground truth would be stronger.

Single Domain. All results come from one accounting platform. The trait theory is stated to be domain-general, but its calibration thesis should be retested where the model is weaker, where reviewers are non-professional, or where corrections are not adjudicable by a survival signal. We expect the non-diagnosticity result (Claim 3) to be the most portable and the identifiability bound (Claim 5) to depend on the aggregator: a non-aggregating system, where a single judgment determines the label, would make individual causal value identifiable and is the natural next test.

Generalizability of Agent Symmetry. Claim 4 identifies the human’s calibration and the model’s self-knowledge as one construct, and it is fair to ask how far that identification travels. The symmetry as stated needs three ingredients: both agents’ judgments admit a confidence signal (behavioral for the human, P(IK)P(\mathrm{IK}) for the model); a shared outcome channel adjudicates correctness for both; and the two agents face the same decisions, so that their calibrations are comparable item by item. Settings with all three (medical-image triage, content moderation, code review of machine-generated changes) should support the same construction. The symmetry weakens where a meaningful machine confidence is missing or degenerate (e.g. the verbalized confidence of a generative model), where the human’s confidence has no behavioral expression in the logs, or where the two agents decide different things, so that item-level comparison is undefined. We also note what the symmetry does and does not assert: it is a claim about the construct, not the measurement route (the human trait is estimated from behavior gated by outcomes, while P(IK)P(\mathrm{IK}) is trained on outcomes directly), so cross-domain tests should expect the construct, not our particular estimators, to transfer.

Observational, Not Interventional, at the Individual Level. The individual-level results are predictive. The only causal evidence (§ 8) is at the segment level and via counterfactual retraining, not a live intervention. A randomized routing experiment that varies the calibration-based allocation is designed but, being a change to a production system affecting real human workload, is deferred to a separate study; we make no live-system claims here.

Conclusion

We argued that the question “which human does a capable model still need?” is, at bottom, a question about one cognitive trait shared by both agents: metacognitive calibration. The trait theory we proposed predicts that activity, agreement, speed, and tenure are confounds rather than signals; that behavioral telemetry can see calibration but cannot, on its own, tell a calibrated expert from a confident novice; that the model’s self-knowledge is the same construct and should allocate the humans; and that individual causal value dissolves under majority-vote aggregation. On six months of production data each prediction held, including the one that embarrassed our own first behavioral profile by ranking an overconfident, 5%-survival cohort above genuine experts. We offer the theory less as a scoring recipe than as an account of why human value in a model-human loop is so easy to mismeasure, and of the single trait that, once you condition it on outcomes, stops the mismeasurement.

Disclosure of LLM Usage

A large language model was used as an assistive tool during the preparation of this manuscript, limited to drafting and editing prose, formatting LaTeX\LaTeX, and checking references. It was not used to generate research ideas, design the study, produce or analyze data, or compute any reported result; all empirical findings derive from the authors’ own analysis of the production logs. The authors reviewed and verified the entire manuscript and take full responsibility for its content.

Behavioral Trait Profile: Telemetry Proxies

The behavioral expertise profile of § 5 scores each reviewer-business pair aa from logged behavior only, with no access to outcomes:

BTP(a)=0.4 rtime(a)+0.3(1−sca)+0.3 ca.\mathrm{BTP}(a) = 0.4\,r_{\mathrm{time}}(a) + 0.3\left(1-\mathrm{sc}_{a}\right) + 0.3\,c_{a}.

The proxies and their ranges are given in Table 3. The weights are fixed a priori and not tuned on any outcome; the point of § 5 is precisely that any purely behavioral weighting inherits the expertise-overconfidence confound, so the specific weights are not load-bearing.

ProxyComputationRange / weight
rtimer_{\mathrm{time}} review-time scorestep function of mean review seconds: <10< 10s →0.5\to0.5; 30-300s →1.0\to1.0; 300-600s →0.7\to0.7; >600> 600s →0.4\to0.4[0,1][0, 1], weight 0.4
sca\mathrm{sc}_{a} self-correction rateevents the pair later unverified itself, over total verifications; enters as 1−sca1-\mathrm{sc}_{a}weight 0.3
cac_{a} consistencymin⁡(runlen‾/5,1)\min(\overline{\mathrm{runlen}}/5, 1) over the run-length encoding of the per-event confirm/correct sequence[0,1][0, 1], weight 0.3

Table 3. Behavioral telemetry proxies in the expertise profile. All are computable from event logs without outcome labels.

The fast/decisive signature (rtimer_{\mathrm{time}} peaking at moderate speeds, low sca\mathrm{sc}_{a}, high cac_{a}) is produced equally by calibrated experts and by the overconfident, which is why the profile must be gated on correction survival before it discriminates value (Claim 3). The survival/P(IK)P(\mathrm{IK})-gated re-specification counts the behavioral score only for reviewers whose corrections survive and concentrate on low-P(IK)P(\mathrm{IK}) (genuinely hard) cases.

Model Self-Knowledge: P(IK)P(\mathrm{IK}) Estimation

P(IK)P(\mathrm{IK}) is a calibrated estimate of the probability the model’s prediction is confirmed by a human, learned from logs. Implementation: gradient-boosted trees (300 estimators, learning rate 0.05, max-leaf-nodes 31, L2L_{2} 1.0) with five-fold isotonic calibration. Features: the model’s raw confidence p^\hat{p}; vendor-, category-, and business-level confirmation-rate priors (fit on the training fold only to avoid leakage, with unseen vendors, categories, or businesses falling back to the training-fold global rate); the absolute amount zz-score against the vendor’s history; and binary flags for whether a vendor and a chart-of-account were predicted. Prediction-to-review latency is excluded: it is realized only after the routing decision, so a deployed gate could not observe it. The label is confirmed (P(IK)=1P(\mathrm{IK}) = 1) versus corrected (P(IK)=0P(\mathrm{IK}) = 0); non-contributor pairs (zero corrections) are excluded. The split is business-grouped 80/20 so that no business appears in both train and test. Expected calibration error is the sample-weighted 15-bin estimate on the held-out businesses (0.037). The routing results of Section 7 score only the held-out businesses’ transactions and route exactly the lowest 20% of P(IK)P(\mathrm{IK}) scores by rank (deterministic tie-breaking), so P(IK)P(\mathrm{IK})- and confidence-routing review identical volumes on the same held-out window.

Ethics and Data Handling

All data are operational logs from a commercial platform, generated by professional reviewers under existing employment agreements in the ordinary course of their work. We report only aggregate statistics; no cohort in this paper is small enough, or keyed to enough attributes, to identify an individual reviewer, and no reviewer names, identifiers, or per-person scores appear. The calibration trait is proposed as an input to work allocation (routing tasks to where they add value), not to individual evaluation or sanction; we note in § 9 that, given Claim 3, behavioral telemetry must never be used for individual assessment without an outcome channel, precisely because it over-ranks the overconfident.

References

  1. [1]Ashton, M. C., & Lee, K. (2009). The HEXACO–60: A short measure of the major dimensions of personality. Journal of Personality Assessment, 91, 340–345.DOI
  2. [2]Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2021). Does the whole exceed its parts? the effect of AI explanations on complementary team performance. CHI Conference on Human Factors in Computing Systems.arxiv.org/abs/2006.14779
  3. [3]Bowman, S. R., et al. (2022). Measuring progress on scalable oversight for large language models. arXiv preprint arXiv:2211.03540.arxiv.org/abs/2211.03540
  4. [4]Burns, C., et al. (2024). Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. Proceedings of the 41st International Conference on Machine Learning. JMLR.org.arxiv.org/abs/2312.09390
  5. [5]Carpenter, J., Sherman, M. T., Kievit, R. A., Seth, A. K., Lau, H., & Fleming, S. M. (2019). Domain-general enhancements of metacognitive ability through adaptive training. Journal of Experimental Psychology: General, 148, 51–64.DOI
  6. [6]Chang, J. C., Amershi, S., & Kamar, E. (2017). REVOLT: Collaborative crowdsourcing for labeling machine learning datasets. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 2334–2346).DOI
  7. [7]Dawid, A. P., & Skene, A. M. (1979). Maximum likelihood estimation of observer error-rates using the EM algorithm. Journal of the Royal Statistical Society: Series C, 28, 20–28.
  8. [8]Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34, 906–911.DOI
  9. [9]Fleming, S. M., & Lau, H. C. (2014). How to measure metacognition. Frontiers in Human Neuroscience, 8, 443.DOI
  10. [10]Geva, M., Goldberg, Y., & Berant, J. (2019). Are we modeling the task or the annotator? an investigation of annotator bias in natural language understanding datasets. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (pp. 1161–1166).arxiv.org/abs/1908.07898
  11. [11]Gordon, M. L., Zhou, K., Patel, K., Hashimoto, T., & Bernstein, M. S. (2021). The disagreement deconvolution: Bringing machine learning performance metrics in line with reality. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.DOI
  12. [12]Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. International Conference on Machine Learning (ICML).arxiv.org/abs/1706.04599
  13. [13]Harbison, B., & Goel, A. K. (2026). LEX-EC: A lexical evidence-channel audit framework for zero-shot LLM personality classification in black-box settings. CoRR, abs/2607.24435.arxiv.org/abs/2607.24435
  14. [14]Hovy, E., & Lavid, J. (2010). Towards a ‘science’ of corpus annotation: A new methodological challenge for corpus linguistics. International Journal of Translation, 22, 13–36.
  15. [15]Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121–1134.pubmed.ncbi.nlm.nih.gov/10626367
  16. [16]Langley, P. (2012a). The cognitive systems paradigm. Advances in Cognitive Systems, 1, 3–13.
  17. [17]Langley, P. (2012b). Intelligent behavior in humans and machines. Advances in Cognitive Systems, 2, 3–12.
  18. [18]Langley, P. (2017). A cognitive systems analysis of personality and conversational style. Advances in Cognitive Systems, 5, 1–12.
  19. [19]Lichtenstein, S., & Fischhoff, B. (1980). Training for calibration. Organizational Behavior and Human Performance, 26, 149–171.
  20. [20]Plank, B. (2022). The ‘problem’ of human label variation: On ground truth in data, modeling and evaluation. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (pp. 10671–10682).arxiv.org/abs/2211.02570
  21. [21]Raykar, V. C., Yu, S., Zhao, L. H., Valadez, G. H., Florin, C., Bogoni, L., & Moy, L. (2010). Learning from crowds. Journal of Machine Learning Research, 11, 1297–1322.DOI
  22. [22]Sinha, P., Dey, L., Mitra, P., & Basu, A. (2015). Mining HEXACO personality traits from enterprise social media. Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (pp. 140–147). Lisboa, Portugal: Association for Computational Linguistics. From https://aclanthology.org/W15-2920/.
  23. [23]Sinha, P., Dey, L., Mitra, P., & Thomas, D. (2020). A hierarchical clustering algorithm for characterizing social media users. Companion Proceedings of the Web Conference 2020 (pp. 353–362). New York, NY, USA: Association for Computing Machinery. From https://doi.org/10.1145/3366424.3383296.
  24. [24]Snow, R., O’Connor, B., Jurafsky, D., & Ng, A. (2008). Cheap and fast—but is it good? evaluating non-expert annotations for natural language tasks. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (pp. 254–263).
  25. [25]Uma, A., Fornaciari, T., Hovy, D., Paun, S., Plank, B., & Poesio, M. (2021). Learning from disagreement: A survey. Journal of Artificial Intelligence Research, 72, 1385–1470.DOI

Paper details

vlnc:W_hufskmmn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents