betaFeedback

WorksComputer Science

RADE: Risk-Adjusted Deployment Efficiency, A Framework for Routing, Memory, and Escalation in Safe LLM Systems

Abstract

A deployed language model does not merely answer questions. It spends computation, consumes time, consults evidence, exposes memory, invites or avoids human review, and sometimes makes mistakes whose consequences are radically unequal. Yet LLM systems are still commonly compared as if the relevant object were a single model and the relevant outcome were a single benchmark score. This paper argues that the unit of analysis should instead be the deployment policy: the mechanism that decides when to retrieve, when to remember, when to call a larger model, when to verify, and when not to answer at all. We introduce risk-adjusted deployment efficiency (RADE), a system-level objective that scores the utility a deployment delivers net of inference, retry, oversight, and priced expected failure cost, while treating harms that cannot be priced as hard budgets alongside constraints on latency, privacy, and resources. The framework brings selective routing, retrieval and governed memory, efficient specialization, uncertainty estimation, abstention, and human escalation into a common account. We then examine five deployment regimes in which the same nominal LLM capability leads to different rational architectures, and propose a reporting protocol based on risk-coverage behavior, end-to-end cost, update drift, memory exposure, evidence quality, and escalation load. The central claim is modest but consequential: frontier capability should be treated as a resource to allocate, not a default state to maintain. A safe and sustainable system is not the one that always uses the strongest model; it is the one that spends capability where the reduction in risk or increase in utility justifies the cost.

Introduction

Consider a deployed assistant at nine o’clock on an ordinary morning. The first request asks it to rewrite a meeting note. The second asks for the current return policy of a supplier whose documentation changes often. The third asks whether an account action should be executed on behalf of a customer. To a benchmark, these may all appear to be instances of language generation. To a responsible operator, they are different species of problem. The first can often be handled locally and cheaply. The second demands fresh evidence. The third may require authorization checks, a verifier, and possibly a human. The interesting engineering question is therefore not, "Which model answers all three best?" It is, "What should the system spend on each request, and what should it refuse to spend?"

That question becomes more important as LLMs move from standalone prototypes into production infrastructure. In a persistent deployment, a model is only one actor in a longer chain. An answer may depend on retrieved documents, a memory store, a tool call, a routing decision, a safety policy, a versioned model, and a person who reviews uncertain cases. A fluent completion can still be wrong because the evidence was stale, the wrong memory was retrieved, the request was routed to an underpowered model, or a model update silently changed behavior that had previously been validated. Frontier capability continues to improve, while responsible-AI evaluation and reporting remain uneven (Stanford Institute for Human-Centered Artificial Intelligence 2026). Larger and more instructable models are not uniformly more reliable, particularly when uncertainty is poorly expressed [31]; semantic uncertainty can reveal confidently generated confabulations that surface fluency hides [6].

The usual story of progress begins with scale. Scaling laws help explain why additional parameters, data, and compute can improve average capability [12, 11]. But deployment changes the object being optimized. A larger model may improve first-pass accuracy and simultaneously increase latency, marginal cost, memory pressure, and dependence on a remote service. A smaller model may be inexpensive per call and expensive per completed task if it triggers frequent retries and human review. A human checker may reduce severe failures and make the system unusably slow. A retrieval layer may improve freshness and introduce a new route for poisoning or stale evidence. The cheapest model call is not necessarily the cheapest answer, and the most accurate model is not necessarily the safest system.

Model-level measures such as cost per call and benchmark accuracy do not determine what a completed answer costs or how safely a deployed system behaves. This paper addresses that mismatch with a deployment-first view: we treat a deployed LLM system as a policy for allocating capability, evidence, memory, verification, and human attention. Prior work has already supplied many of the mechanisms: cascades and learned routing can shift traffic between models

Copyright © 2027, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. [4, 22]; retrieval can externalize part of the knowledge burden [19, 2]; sparse computation and serving systems can reduce active compute and memory overhead [7, 16]; and uncertainty methods can help identify cases that should be rejected or escalated [29, 6]. What these lines of work lack in common is not another component but a shared question: when is each component that improves an answer worth invoking, given the likelihood and consequence of that answer being wrong?

Statement of Contributions

This paper makes four contributions. First, it introduces risk-adjusted deployment efficiency (RADE), a constrained net-value objective that subtracts inference, retry, human-review, and priced expected failure costs from delivered utility, while enforcing explicit budgets on harms that cannot be priced and constraints on latency, privacy, and resources. Second, it recasts routing, retrieval, memory, verification, and escalation as parts of one control policy whose task is to allocate expensive capability selectively. Third, it uses five representative deployment regimes to show why the same model can be a sensible choice in one setting and a poor choice in another. Fourth, it proposes a deployment-oriented evaluation protocol centered on risk-coverage behavior, end-to-end cost, update drift, memory governance, evidence quality, and escalation load.

The Deployment Policy as the Unit of Analysis

The vocabulary of modern LLM evaluation is still strongly model-centric. We ask how many parameters a model has, what score it obtains, how much a token costs, or whether one model beats another on a benchmark. Those are useful facts. They are not yet a deployment decision.

Suppose a system contains two language models: a small local model and a larger general model. It also has a search index, a persistent memory store, a deterministic calculator, and a human-review queue. One could ask which language model is better, but that misses most of the policy. The real system chooses among paths. It may answer locally, retrieve first, call the larger model, verify the draft, invoke a tool, ask a clarifying question, or abstain. The object of interest is therefore not a model mm but a system ss that maps a request and its context to a sequence of actions.

This shift sounds semantic until failures become asymmetric. If the first request in our opening example is mishandled, the user may rewrite a sentence manually. If the third request is mishandled, an unauthorized action may be executed. Averaging the two into one accuracy number erases the fact that one error is cheap and the other expensive. A deployment metric must therefore know something about consequence, not merely frequency.

We represent a deployment profile as

d=(D,B,Tmax⁡,Pmax⁡,ϵ,H),(1)d = (\mathcal{D}, B, T_{\max}, P_{\max}, \epsilon, H), \tag*{(1)}

where D\mathcal{D} is the deployment distribution, BB is a resource budget, Tmax⁡T_{\max} is a latency constraint, Pmax⁡P_{\max} is a privacy or exposure constraint, ϵ\epsilon contains tolerated probabilities for critical failure classes, and HH describes the available human oversight. The important feature of dd is that it describes the world around the model. The same model placed in a different world can rationally lead to a different system.

Risk-Adjusted Deployment Efficiency

A useful accounting identity follows from a simple observation: a deployment pays a price for more than inference. It pays when a weak answer triggers a retry. It pays when a person must review the result. It pays, sometimes overwhelmingly, when a consequential failure occurs.

For each failure class kk, a factual error, an unauthorized action, a privacy disclosure, an unsafe clinical suggestion, or another domain-specific event, let pk(s,d)p_k(s,d) be its probability per request under system ss and deployment profile dd. Failure classes differ in whether their consequences can be priced. We therefore partition them into K$\mathcal{K}_{\$}, classes whose consequence Lk(d)L_k(d) admits a defensible common unit (for example, a refund issued in error, wasted staff time, or a contractual penalty), and Ksev\mathcal{K}_{\mathrm{sev}}, classes that admit only a severity ordering (for example, a severe clinical error or the disclosure of sensitive personal data). For k∈K$k \in\mathcal{K}_{\$}, Lk(d)L_k(d) measures the damage caused by the failure itself, excluding the value of the task that was not delivered, which is accounted for separately below. We define the expected risk-adjusted cost per request as

CRA(s,d)=Cinfer+Cretry+Creview+∑k∈K$pk(s,d)Lk(d).(2)C_{\mathrm{RA}}(s,d) = C_{\mathrm{infer}} + C_{\mathrm{retry}} + C_{\mathrm{review}} + \sum_{k \in\mathcal{K}_{\$}} p_k(s,d)L_k(d). \tag*{(2)}

The distinction in Equation (2) is elementary but often lost in model comparisons. Inference cost is paid directly; failure cost is paid probabilistically. In a low-consequence task, the former may dominate. In a high-consequence task, a small change in the latter can dwarf everything else. Classes in Ksev\mathcal{K}_{\mathrm{sev}} are deliberately absent from the sum: adding a severity score to a monetary cost would produce a number with no meaningful unit.

Let U(s,d)U(s,d) denote the expected utility per request delivered by requests that end in a correct completion under the deployment profile. A request counts toward UU whether it is completed on the first path or after escalation to a stronger model or a human reviewer; the cost of that escalation is already carried by CretryC_{\mathrm{retry}} or CreviewC_{\mathrm{review}}. Incorrect outputs and requests that end without a correct completion contribute nothing to UU; the damage a failure causes enters only through LkL_k. A failure therefore registers in two places for two distinct reasons, once as value not delivered and once as damage done, but no single loss is counted twice. We define risk-adjusted deployment efficiency as a net value,

RADE⁡(s,d)=U(s,d)−λCRA(s,d),(3)\operatorname{RADE}(s,d) = U(s,d) - \lambda C_{\mathrm{RA}}(s,d), \tag*{(3)}

where λ>0\lambda> 0 converts cost into utility units (λ=1\lambda= 1 when utility is itself monetary). The deployment problem is to maximize Equation (3) subject to hard conditions such as

pk(s,d)≤ϵk∀k∈Kcritical,(4)p_k(s,d) \le\epsilon_k \qquad\forall k \in\mathcal{K}_{\mathrm{critical}}, \tag*{(4)}
E[T(s,d)]≤Tmax⁡,(5)\mathbb{E}[T(s,d)] \le T_{\max}, \tag*{(5)}
P(s,d)≤Pmax⁡,(6)P(s,d) \le P_{\max}, \tag*{(6)}
Cresource(s,d)≤B,(7)C_{\mathrm{resource}}(s,d) \le B, \tag*{(7)}

where T(s,d)T(s,d) is the end-to-end latency of a request, P(s,d)P(s,d) is the extent to which private data is sent or retained beyond the boundary the deployment permits, for example by routing it to an external service or storing it in persistent memory, and Cresource(s,d)C_{\mathrm{resource}}(s,d) is the computational capacity the system requires, such as accelerator compute and memory; each is bounded by the corresponding element of the deployment profile dd. The privacy constraint is therefore a structural limit on exposure, distinct from the probability of a disclosure failure, which Equation (4) bounds. The set Kcritical\mathcal{K}_{\mathrm{critical}} contains every class in Ksev\mathcal{K}_{\mathrm{sev}} together with any priced class whose probability must also be capped. The damage caused by classes in Ksev\mathcal{K}_{\mathrm{sev}} therefore never enters the objective. A severe harm is treated as a budget to respect, not a cost to trade. We keep the word efficiency in its allocative sense: a system that maximizes Equation (3) inside the constraints cannot reallocate capability, evidence, or oversight to deliver more net value. The difference form is chosen over a ratio such as U/CRAU/C_{\mathrm{RA}} for two reasons. First, the zero point and scale of utility are conventions. Replacing UU with aU+baU+b for a>0a>0 leaves the ranking produced by Equation (3) unchanged once λ\lambda is rescaled by aa, whereas a shift bb in the zero point can reorder systems ranked by a ratio. Second, a ratio rewards systems that do very little very cheaply. Consider system s1s_1, which completes 20%20\% of requests correctly at $0.01 per request, and system s2s_2, which completes 90%90\% at $0.10, with each correct completion worth $1 and no priced failures. The ratio scores s1s_1 at 20 and s2s_2 at 9; the net value scores s1s_1 at 0.19 and s2s_2 at 0.80. Only the second ranking matches the choice an operator would make. When no exchange rate is agreed, λ\lambda should be treated as a reporting parameter: results should be shown across a range of values, or as the frontier of UU against CRAC_{\mathrm{RA}} among feasible systems, so that readers can see whether a ranking depends on the choice.

What a completed answer costs is captured by a secondary measure. Let A(s,d)A(s,d) denote the probability that a request ends in a correct completion. The price of an answer is

π(s,d)=CRA(s,d)A(s,d),(8)\pi(s,d) = \frac{C_{\mathrm{RA}}(s,d)}{A(s,d)}, \tag*{(8)}

the expected risk-adjusted spend per correct completion. Because π\pi is itself a ratio, it inherits the pathology above: in the example, s1s_1 pays $0.05 per correct answer and s2s_2 pays about $0.11. It is therefore descriptive rather than an objective, and should be reported only alongside A(s,d)A(s,d) or compared among systems that meet a declared minimum completion rate.

RADE is not proposed as a new training loss. It is an accounting frame for deployment. The constraints in Equations (4)-(7) come first: a high net value should not compensate for violating a hard privacy or severe-failure bound. Only among systems inside the declared safety envelope does the comparison in Equation (3) become meaningful.

This view explains several choices that appear contradictory when judged at the model level. A cheap model that frequently induces retries can cost more than a stronger model on hard queries. A frontier model used for every request can be wasteful if most requests are routine. Human review can be valuable precisely because it is expensive: it should be reserved for cases in which the expected reduction in harm justifies the delay and labor. Retrieval can save the cost of retraining for new facts, but it must earn that saving by providing evidence that is fresh enough and trustworthy enough to improve the decision.

Work on model cascades already points in this direction. FrugalGPT shows that heterogeneous model combinations can improve the cost-quality frontier [4]; RouteLLM learns to route between stronger and weaker models using preference data [22]. RADE adds a condition that becomes crucial in deployment: the router should care not only about the probability of being wrong, but also about the consequence of being wrong.

Deployment Regimes and Binding Constraints

The easiest way to see why a universal "best model" is a poor abstraction is to walk the same technology through different regimes. Table 1 summarizes five representative regimes. The point is not that all deployments fit neatly into these categories. The point is that the binding constraint changes.

RegimeInfrastructureBinding constraintFailure budgetRational deployment priority
Research assistantCloud, multimodelMarginal cost and evidence freshnessModerate; users can often verifyRoute routine work cheaply; spend on retrieval and verification for factual claims.
Enterprise on-premiseLocal GPUs / private cloudPrivacy, hardware capacity, maintainabilityLow for disclosure and policy violationsPrefer bounded local models, private retrieval, governed memory, and controlled escalation.
High-volume action supportCloud, tool-usingUnauthorized action and review loadVery low for irreversible actionsSeparate language generation from authorization; verify before execution.
Offline clinical assistantEdge / hospital networkPrivacy, latency, evidence stalenessExtremely low for severe clinical errorsConstrain scope, surface evidence freshness, abstain aggressively, escalate to a clinician.
General knowledge workCloud plus optional localHeterogeneous tasks and long-horizon driftTask-dependentRoute dynamically, govern persistent memory, and regression-test behavior across updates.

Table 1. Five representative deployment regimes. The relevant "failure budget" concerns the tolerated probability and severity of consequential failure, not ordinary benchmark error.

In the research-assistant regime, abstention is often cheap. A user can ask a follow-up, open the cited source, or rephrase the request. The system can therefore trade some automation for lower cost. In the enterprise regime, privacy may dominate: sending every query to a remote frontier model can be unacceptable even if that model is more capable on average. In high-volume support, the decisive boundary is not between a good sentence and a bad sentence; it is between text generation and authorization to act. An assistant may be permitted to draft a refund explanation while being forbidden to issue the refund without a separate check.

The clinical regime makes the point most sharply. Local execution can improve privacy and availability, but a perfectly private model with stale evidence can still be dangerous. The binding risk may therefore shift from disclosure to freshness. In general knowledge work, no single risk dominates for long: the system must cope with varied tasks, persistent memory, changing models, and repeated updates.

A deployment-first framework asks, in each regime, what the system must know before it spends capability. Model size becomes one control knob among several.

Selective Computation as a Safety Policy

Selective computation is usually introduced as an efficiency technique: send easy requests down a cheap path and difficult requests down an expensive one. That description is incomplete. A production router is also a safety policy because it decides which requests are allowed to receive which level of competence, evidence, and oversight.

Figure 1 shows the architecture implied by this view. A low-cost gate first estimates something richer than difficulty. It asks whether the request is routine or high consequence, whether current evidence is required, whether private memory is relevant, whether a tool call would create an irreversible effect, and whether the system can recognize the limits of its own competence. From there the request is routed to a small model, a larger model, or a tool, drawing on retrieval and memory where needed. The final decision is not simply which model generates the answer, but whether that answer is backed by enough evidence and verification to be released at all.

A deployment-first control architecture

Figure 1. A deployment-first control architecture. A low-cost gate estimates complexity and consequence; retrieval and governed memory provide context; heterogeneous execution paths produce candidate outputs; and verification determines whether a verified answer is released or the case is withheld and escalated. The figure is architectural rather than an empirical benchmark result.

Severity-Aware Routing

Learned routing and cascades already show that not every query requires the same model [4, 22]. A safety-aware router adds two requirements. First, it should route partly on failure severity, not only predicted answer quality. Second, escalation should remain possible after generation. A cheap path may look adequate at the start and become inadequate when retrieval fails, evidence conflicts, or a verifier detects uncertainty.

Several efficiency techniques can be understood through the same lens. Sparse expert models separate total capacity from active computation [7]. State-space architectures such as Mamba explore alternatives to quadratic attention for long sequences (Gu and Dao 2023). Speculative decoding drafts with a cheaper model and verifies with a stronger one [18]. The internal mechanisms differ, but the systems principle is common: expensive computation should be conditional rather than automatic.

Retrieval and Evidence

The second request in our opening example asks for a policy that changes over time. Here, parametric memory is a poor place to put operational truth. Retrieval-augmented generation and retrieval-augmented pretraining show how external corpora can supply knowledge without encoding every update into model weights [19, 2]. The deployment advantage is not merely freshness.

Retrieval creates an evidence path that can be inspected, versioned, and, at least in principle, verified.

But retrieval is not proof. A stale document, a poisoned index, or a badly matched passage can produce an answer that looks more grounded because it contains citations. Retrieved evidence should therefore inform verification as well as generation: evidence is an input to judgment, not a substitute for it.

Infrastructure and Safety Margins

Systems work is often treated as an implementation detail that begins after model evaluation. In deployment, it can change which safety measures remain available. Long-context serving consumes substantial key-value-cache memory; PagedAttention improves memory management and serving throughput by reducing fragmentation and enabling more flexible allocation [16]. This matters because a system running close to its resource limit may respond to load by truncating context, disabling a verifier, shrinking retrieval, or creating queues that defeat latency guarantees. Resource headroom is therefore not merely an efficiency concern. It is part of the margin within which safety mechanisms continue to function.

Memory, Continual Learning, and Governance

A deployed assistant is expected to learn in a peculiar sense of the word. It should remember a user’s preference tomorrow, reflect a changed policy next week, and still retain capabilities that were validated last month. These demands are often grouped under "continual learning," but they happen on different timescales and should not all be solved by changing model weights.

The danger of collapsing these timescales is familiar. Continual fine-tuning can produce catastrophic forgetting or distribution-specific capability loss [8, 20, 14]. Even successive releases of general-purpose models can exhibit behavioral drift despite improvements in aggregate capability [5]. A system that updates frequently without remembering what it used to do is not learning in the operational sense; it is changing. Complementary Learning Systems theory offers a useful analogy: fast episodic adaptation and slow semantic consolidation should occur on different timescales [21, 15]. For LLM deployment, the analogy suggests three layers.

Fast memory. Recent facts, user preferences, task state, and organization-specific knowledge can live outside the base model and be retrieved only when relevant. Systems such as MemGPT make this separation explicit [23]. The advantage is reversibility: a memory item can be corrected or deleted without retraining the model.

Slow consolidation. Stable patterns may eventually justify a parameter update, but such updates should be less frequent and surrounded by regression tests. Classical continual-learning methods such as Elastic Weight Consolidation and Synaptic Intelligence illustrate the general principle of protecting important representations while incorporating new information [13, 30]. Whether these exact methods are the right mechanism for modern LLMs is less important here than the architectural lesson: fast local change need not imply immediate global change.

A governance layer between the two. Memory is useful because it preserves information across time; that is also why it can become dangerous. Training-data extraction has shown that memorized text can be recoverable from language models [3], and recent work demonstrates black-box extraction attacks against agent memory [27]. Persistent memory should therefore be treated as a governed data store. It needs write policies, retrieval scopes, provenance, retention rules, expiry, deletion, and adversarial tests. A memory subsystem with no policy is not personalization; it is uncontrolled persistence.

This reframing changes the practical meaning of continual learning. When new information arrives, the safest first move may be to update retrieval or episodic memory, observe how the information is used, and consolidate only what proves stable. The system gains plasticity without paying the full stability cost on every update.

Uncertainty, Abstention, and Escalation

A capable model can produce an answer to almost any well-formed prompt. A reliable system must sometimes resist that capability.

For high-consequence deployment, average correctness is not enough. The system must distinguish cases in which it is competent from cases in which it should seek more evidence or hand control elsewhere. Larger models can remain overconfident or become less reliable along dimensions that aggregate benchmarks do not reveal [31]. RLHF can also amplify verbalized overconfidence through reward-model biases [17]. This is why uncertainty estimation matters most at the boundary between action and abstention.

Let u(x)u(x) denote an uncertainty or risk score for request xx. Suppose the system answers only when u(x)≤τu(x) \le\tau. We can then define coverage and selective risk as

κ(τ)=Pr⁡(u(x)≤τ),\kappa(\tau) = \Pr(u(x) \le\tau),
Rsel(τ)=E[ℓ(y^,y)∣u(x)≤τ].(9)R_{\mathrm{sel}}(\tau) = \mathbb{E}[\ell(\hat{y},y) \mid u(x) \le\tau]. \tag*{(9)}

A useful score should reduce selective risk as coverage falls, especially on shifted or high-severity subsets. Free-form generation makes this difficult because many strings can express the same underlying claim. Semantic entropy addresses part of the problem by measuring uncertainty over meaning clusters and improves rejection behavior for confabulation-prone questions [6]. Other work on confidence elicitation shows that verbal or probability-like confidence is not automatically calibrated [29].

Abstention is not synonymous with refusal. The system has a repertoire of non-answers: ask for missing information, retrieve another source, call a more capable model, invoke a deterministic tool, request human review, or block an irreversible action. The operational problem is therefore not "How often should the model refuse?" but "Which escalation should follow which uncertainty?"

The economics return here. If a first-pass answer costs cic_i, triggers a retry with probability qrq_r at cost crc_r, triggers human review with probability qhq_h at cost chc_h, and incurs failure class kk with probability pkp_k, then the expected risk-adjusted cost per request, the request-level form of Equation (2), is

ci+qrcr+qhch+∑k∈KsevpkLk.(10)c_i + q_r c_r + q_h c_h + \sum_{k \in\mathcal{K}_{\mathrm{sev}}} p_k L_k. \tag*{(10)}

A more expensive first-pass model can be cheaper overall if it sufficiently reduces retries, review, and priced failures. It can also be the only admissible choice if it is the one that keeps the classes in Ksev\mathcal{K}_{\mathrm{sev}} within their budgets in Equation (4). A cheap model plus a good router and verifier can also dominate an expensive model used blindly. What matters is the policy, not the sticker price.

Bounded Specialization

Efficiency arguments often end with a recommendation to use smaller models. That conclusion is too easy. Smaller models are attractive in privacy-sensitive, on-premise, and edge settings because they reduce serving requirements. Compute-optimal training already shows that parameter count alone is an incomplete measure of capability [11]; domain-adaptive pretraining shows that in-domain data can produce large gains without requiring a universally larger model [10].

But specialization buys competence by narrowing the world in which competence is expected. A domain model may be excellent inside its distribution and brittle just outside it. The safer pattern is therefore bounded specialization: define the region in which the smaller model is trusted, monitor for shift, and preserve an escalation path to a broader model or a person.

The same logic applies to systems optimization. Quantization, pruning, batching, and cache management should not be evaluated only by throughput. An optimization that produces a tiny average accuracy loss can still be harmful if that loss is concentrated in rare high-severity cases. A system can become faster and less efficient in the RADE sense, or faster and inadmissible if the loss pushes a class in Ksev\mathcal{K}_{\mathrm{sev}} past its budget in Equation (4). Safety evaluation must therefore be stratified by consequence, not only by frequency.

Evaluation and Reporting Protocol

Once the unit of analysis moves from model to policy, the evaluation report must change with it. A deployment paper should let a reader reconstruct the operational trade-off: what was automated, what was escalated, what the system spent, what changed after updates, and where the dangerous failures occurred.

Table 2 gives a minimum reporting set. None of the rows is exotic. Their importance comes from reporting them together.

DimensionReportQuestion it answers
Task utilityAccuracy / task success / application utility on a representative deployment distributionDoes the system perform useful work, including on important subgroups or scenarios?
Selective reliabilityRisk-coverage curve, abstention rate, uncertainty or calibration qualityDoes the system know which cases to escalate?
Failure severityError taxonomy and frequency by severity classAre rare consequential failures hidden by many harmless successes?
Resource costTokens, active-compute proxy, memory footprint, latency, and monetary cost where availableWhat does a completed task actually consume?
Oversight loadHuman-review rate, retry rate, escalation destination, time-to-resolutionHow much hidden labor is required to make the system reliable?
Update stabilityRegression-suite delta, forgetting rate, policy drift after updatesDoes improvement in one version silently break validated behavior elsewhere?
Memory governanceWrite/read policy, retention, deletion test, adversarial extraction or leakage rateWhat persistent information can be stored, retrieved, and exposed?
Evidence qualityRetrieval recall/precision, source freshness, verifier pass/fail behaviorAre answers supported by evidence that is relevant and current?

Table 2. A minimum reporting set for risk-adjusted evaluation of deployed LLM systems.

End-to-End Evaluation

Component metrics can be individually strong while the system fails. A router may predict average model preference accurately and still send exactly the wrong high-severity cases to the cheap path. A verifier may have a strong aggregate discrimination score and still miss the small class of errors that triggers irreversible actions. End-to-end evaluation must therefore sample from the deployment distribution and include adversarial, shifted, and high-severity slices.

Risk-Coverage and Cost-Risk Curves

Any selective system has a threshold somewhere, even if the threshold is hidden in a prompt or business rule. Reporting risk as a function of coverage reveals whether the system can give up automation gracefully as the required safety level rises. Cost-risk curves play the same role for routing and verification: they show whether an additional model call, retrieval step, or human check buys enough risk reduction to justify its cost.

Update Stability and Behavioral Drift

Static evaluation is inadequate when models, indexes, tools, and policies change repeatedly. Each update should be tested against a persistent regression set and a fresh distribution-shift set. For discrete, safety-relevant outcomes, a simple behavioral drift statistic between consecutive versions tt and t+1t+1 of the system is

Dt,t+1=Pr⁡x∼D[gt(x)≠gt+1(x)],(11)D_{t,t+1} = \Pr_{x \sim\mathcal{D}}\left[g_t(x) \ne g_{t+1}(x)\right], \tag*{(11)}

where gtg_t maps a request to the outcome chosen by version tt, such as answer, abstain, escalate, or authorize action, and the probability is estimated in practice on the regression set. A nonzero value is not itself a failure, since an update may change outcomes for the better; it identifies the requests that need inspection. The statistic also cannot detect a change in the content of an answer when the outcome type stays the same, so it should be paired with task-specific scoring of correctness. Its purpose is narrower: to make silent policy change visible.

Design Trade-offs

A framework is useful when it clarifies a trade-off, not when it pretends to abolish one.

Stability and plasticity. Fast adaptation is valuable, yet shared-parameter updates can damage unrelated behavior [8, 20]. Separating fast episodic updates from slower consolidation reduces the tension but does not remove it.

Memory and privacy. Persistent memory improves continuity because it preserves information that would otherwise be forgotten. The same persistence creates the exposure risk. Memory extraction results therefore make access policy part of the threat model rather than a user-interface detail [27].

Capability and efficiency. A smaller or quantized model reduces resource cost and usually narrows the region in which it should be trusted. The right question is not whether the small model can replace the large one everywhere, but whether the system can recognize the region in which the cheaper path remains inside the declared failure budget.

Automation and oversight. Human review can reduce some failure probabilities while adding delay, labor, and new opportunities for error. A system that sends most requests to people may be safe and economically pointless. The objective is not maximum automation; it is oversight that either earns its cost in net value or is needed to satisfy the constraints, and no more.

Freshness and attack surface. Retrieval and tools provide current information and useful actions by extending the system beyond the model. Every extension also enlarges the dependency graph. Source integrity, retrieval poisoning, tool authorization, and fallback behavior therefore belong in the same evaluation report as answer quality.

Implications for Research and Standards

The deployment-first view changes what counts as an informative result.

For architecture research, parameter count and benchmark score should be joined by active computation, memory footprint, and the risk profile at the chosen operating point. Selective computation becomes especially valuable when it is conditioned not merely on difficulty but on consequence and uncertainty.

For continual-learning research, improvement on the newest distribution should be reported together with behavior retained from earlier validated versions. Memory systems should state their write, retrieval, retention, and deletion semantics and be tested against extraction attacks.

For uncertainty research, correlation between confidence and correctness is only an intermediate result. An uncertainty score becomes operationally meaningful when it sends the right cases to the right escalation path at an acceptable coverage and oversight cost. Risk-coverage curves should therefore be interpreted alongside the consequence model of the target deployment.

For sustainability research, reliability is part of the resource story. A cheap model that causes frequent retries, review, or incidents may consume more resources than a stronger model used selectively. Conversely, a routing-and-verification policy can make frontier capability available without paying for it on every request.

For standards and certification, the framework suggests questions that are concrete enough to audit: What are the declared failure classes and budgets? Which requests may trigger autonomous action? How fresh must evidence be? Which component owns the decision to abstain? What memory can be written and deleted? What happens when an update changes the answer/escalate policy? These questions are more operationally meaningful than the undifferentiated claim that a model is "safe".

Ethical Considerations

RADE is a ledger, and every ledger is kept from someone’s point of view. The formalism in Section does not by itself say whose costs are counted, whose benefits are maximized, or who decides where its lines fall. Left unstated, the default is the operator’s point of view, and that default is where most of the ethical risk lies.

Whose harm counts. If LkL_k records only what a failure costs the operator, such as refunds, fines, or legal exposure, then harm borne by users and third parties disappears from (2). A deployment can then look efficient precisely because the people it harms are cheap to compensate or have no claim at all. We therefore read LkL_k as the damage to everyone a failure affects, not as operator liability, and UU as the value delivered to the people the system serves, for which revenue is at best a proxy. The examples of priced classes in Section (a refund issued in error, wasted staff time, a contractual penalty) are operator-side costs chosen because they are easy to price; under this reading they are parts of LkL_k, not the whole of it, and LkL_k must also include what the failure costs the affected user or third party, such as time lost correcting it or a loss incurred by acting on it. Payments that only move money between parties, such as an erroneous refund or a penalty paid to a client, should count at the net damage they reflect rather than at face value, since counting them at face value would record a transfer as a loss. We also read the requirement in Section that a priced class admit a "defensible" common unit strictly: a unit is not defensible merely because a number can be attached to the harm. Where damage falls on people who did not accept the risk, or can be priced only by assumption, the class belongs in Ksev\mathcal{K}_{\mathrm{sev}} by default. The partition then carries ethical weight: harms that should not be bought are governed by budgets that act as side constraints, not as prices.

Who sets the parameters. The partition between K$\mathcal{K}_{\$} and Ksev\mathcal{K}_{\mathrm{sev}}, the losses LkL_k, the budgets ϵk\epsilon_k, and the exchange rate λ\lambda determine most of what RADE will recommend, and the operator has an interest in each: looser budgets and lower losses both make more deployments look acceptable. These parameters should therefore not be set by the operator alone. Depending on the domain, they can come from sector regulation, as in clinical settings, from external oversight by independent third parties [25], or from the participation of the communities that bear the risk [1]. At a minimum, they should be published with their justification, so that the choices behind a favorable score can be contested.

Averages hide who bears the risk. The probabilities pkp_k are averages over the deployment distribution D\mathcal{D}. A system can satisfy every budget on average while a particular group experiences a much higher failure rate; language models are already known to perform worse for some languages and social groups [28]. The budgets in Equation 4 should therefore hold when D\mathcal{D} is restricted to each affected group, not only over D\mathcal{D} as a whole. A related risk is specific to value-maximizing deployment. When utility is measured in money, it tends to track willingness to pay, and a policy chosen to maximize Equation 3 will tend to spend more capability on requests worth more. Differences in speed or convenience may be a legitimate commercial choice; differences in safety are not. The budgets for Ksev\mathcal{K}_{\mathrm{sev}} must hold for low-value and high-value requests alike.

Costs that move rather than disappear. Some savings in the ledger are transfers. Abstention costs the operator little, but the unfinished work moves to the user. That is acceptable where users can verify and follow up, as in the research-assistant regime of Section , and less so where they cannot. Escalation moves work onto human reviewers, whose attention is finite and who tend to over-rely on automated suggestions, an effect observed in experts as well as novices [24]. The oversight-load row of Table 2 exists to keep this labor visible.

A score is not a license. A favorable RADE does not show that a harm is acceptable; it shows that the assumptions about harm have been written down. The framework’s ethical value lies in that disclosure. An explicit ledger can be audited, challenged, and corrected, while an implicit one cannot. Treating the score as a justification rather than a disclosure would fall into a version of what Selbst et al. (2019) call the formalism trap: treating a question that remains contested in its social context as if a formal quantity could settle it.

Scope and Limitations

This paper is a conceptual and integrative contribution rather than a new empirical benchmark. RADE does not solve the moral or methodological problem of assigning costs to harm. It makes the problem explicit instead: harms without a defensible common unit are placed in Ksev\mathcal{K}_{\mathrm{sev}} and governed by probability budgets rather than being forced into (2). The partition itself, the budgets ϵk\epsilon_k, and the exchange rate λ\lambda remain value judgments. Some consequences are non-monetary, delayed, difficult to compare, or distributed across people, and these choices should be declared and justified rather than treated as technical defaults.

The five deployment regimes are illustrative, not exhaustive. They do not cover every domain, organizational form, or cultural context. The framework also relies on routers, uncertainty estimates, verifiers, and risk labels that can fail under distribution shift. No single gate should therefore be interpreted as proof of safety. High-severity applications still require independent controls and human accountability.

Finally, resource and latency comparisons depend on hardware and serving assumptions. Empirical applications of RADE should report enough infrastructure detail for the cost terms to be interpreted correctly. The framework is intended to make those assumptions visible, not to hide them behind one universal score.

Conclusion

Return to the three requests from the beginning. The meeting note, the changing supplier policy, and the customer action do not deserve the same path merely because all three can be written as prompts. One needs inexpensive language generation. One needs evidence. One needs control.

That is the broader point. A deployed LLM system is not a model waiting to answer; it is a policy for deciding how much computation, evidence, memory, verification, and human attention a request deserves. RADE formalizes the choice of that policy as a risk-adjusted allocation problem. It asks the system to satisfy its declared safety constraints first and then to maximize the net value of useful work after operational and priced expected failure costs.

Under this view, scale remains valuable, but its role changes. Frontier capability becomes something to invoke deliberately when its expected reduction in risk or increase in utility justifies the marginal cost. Smaller models, retrieval, memory, tools, verifiers, and human review become complementary instruments rather than competitors in a model leaderboard. The measure of progress is no longer capability per parameter, nor even capability per token. It is the quality of the decisions a system can make about when to spend capability, when to seek evidence, and when to stop.

References

  1. [1]Birhane, A.; Isaac, W.; Prabhakaran, V.; Díaz, M.; Elish, M. C.; Gabriel, I.; and Mohamed, S. 2022. Power to the People? Opportunities and Challenges for Participatory AI. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization.DOI
  2. [2]Borgeaud, S.; Mensch, A.; Hoffmann, J.; Cai, T.; Rutherford, E.; Millican, K.; van den Dries sche, G.; Lespiau, J.-B.; Damoc, B.; Clark, A.; et al. 2022. Improving language models by retrieving from trillions of tokens. In Proceedings of the International Conference on Machine Learning (ICML).arxiv.org/abs/2112.04426
  3. [3]Carlini, N.; Tramer, F.; Wallace, E.; Jagielski, M.; Herbert-Voss, A.; Lee, K.; Roberts, A.; Brown, T.; Song, D.; Erlingsson, Ú.; et al. 2021. Extracting training data from large language models. In USENIX Security Symposium.arxiv.org/abs/2012.07805
  4. [4]Chen, L.; Zaharia, M.; and Zou, J. 2023a. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv preprint arXiv:2305.05176.arxiv.org/abs/2305.05176
  5. [5]Chen, L.; Zaharia, M.; and Zou, J. 2023b. How is ChatGPT’s behavior changing over time? arXiv preprint arXiv:2307.09009.arxiv.org/abs/2307.09009
  6. [6]Farquhar, S.; Kossen, J.; Kuhn, L.; and Gal, Y. 2024. Detecting Hallucinations in Large Language Models Using Semantic Entropy. Nature, 630: 625–630.DOI
  7. [7]Fedus, W.; Zoph, B.; and Shazeer, N. 2022. Switch Transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120): 1–39.arxiv.org/abs/2101.03961
  8. [8]French, R. M. 1999. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4): 128–135.
  9. [10]Gururangan, S.; Marasović, A.; Swayamdipta, S.; Lo, K.; Beltagy, I.; Downey, D.; and Smith, N. A. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8342–8360.arxiv.org/abs/2004.10964
  10. [11]Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; de Las Casas, D.; Hendricks, L. A.; Welbl, J.; Clark, A.; et al. 2022. Training compute-optimal large language models. In Advances in Neural Information Processing Systems (NeurIPS).arxiv.org/abs/2203.15556
  11. [12]Kaplan, J.; McCandlish, S.; Henighan, T.; Brown, T. B.; Chess, B.; Child, R.; Gray, S.; Radford, A.; Wu, J.; and Amodei, D. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.arxiv.org/abs/2001.08361
  12. [13]Kirkpatrick, J.; Pascanu, R.; Rabinowitz, N.; Veness, J.; Desjardins, G.; Rusu, A. A.; Milan, K.; Quan, J.; Ramalho, T.; Grabska-Barwinska, A.; et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13): 3521–3526.DOI
  13. [14]Kotha, S.; Springer, J. M.; and Raghunathan, A. 2024. Understanding catastrophic forgetting in language models via implicit inference. International Conference on Learning Representations (ICLR).arxiv.org/abs/2309.10105
  14. [15]Kumaran, D.; Hassabis, D.; and McClelland, J. L. 2016. What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends in Cognitive Sciences, 20(7): 512–534.DOI
  15. [16]Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J.; Zhang, H.; and Stoica, I. 2023. Efficient memory management for large language model serving with Paged Attention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP).arxiv.org/abs/2309.06180
  16. [17]Leng, J.; Huang, C.; Zhu, B.; and Huang, J. 2025. Taming overconfidence in LLMs: Reward calibration in RLHF. In International Conference on Learning Representations (ICLR).arxiv.org/abs/2410.09724
  17. [18]Leviathan, Y.; Kalman, M.; and Matias, Y. 2023. Fast inference from transformers via speculative decoding. arXiv preprint arXiv:2211.17192.arxiv.org/abs/2211.17192
  18. [19]Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; Küttler, H.; Lewis, M.; Yih, W.-t.; Rocktäschel, T.; Riedel, S.; and Kiela, D. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems (NeurIPS).arxiv.org/abs/2005.11401
  19. [20]Luo, Y.; Yang, Z.; Meng, F.; Li, Y.; Zhou, J.; and Zhang, Y. 2023. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747.arxiv.org/abs/2308.08747
  20. [21]McClelland, J. L.; McNaughton, B. L.; and O’Reilly, R. C. 1995. Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102(3): 419–457.
  21. [22]Ong, I.; Almahairi, A.; Wu, W.; Chiang, W.-L.; Wu, T.; Gonzalez, J. E.; Kadav, A.; and Stoica, I. 2024. RouteLLM: Learning to Route LLMs with Preference Data. arXiv preprint arXiv:2406.18665.arxiv.org/abs/2406.18665
  22. [23]Packer, C.; Wooders, S.; Lin, K.; Fang, V.; Patil, S. G.; Stoica, I.; and Gonzalez, J. E. 2023. MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560.arxiv.org/abs/2310.08560
  23. [24]Parasuraman, R.; and Manzey, D. H. 2010. Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors, 52(3): 381–410.
  24. [25]Raji, I. D.; Xu, P.; Honigsberg, C.; and Ho, D. E. 2022. Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. Selbst, A. D.; boyd, d.; Friedler, S. A.; Venkatasubramanian, S.; and Vertesi, J. 2019. Fairness and Abstraction in Sociotechnical Systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68.
  25. [27]Wang, B.; He, W.; Zeng, S.; Xiang, Z.; Xing, Y.; Tang, J.; and He, P. 2025. Unveiling Privacy Risks in LLM Agent Memory. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 25241–25260.arxiv.org/abs/2502.13172
  26. [28]Weidinger, L.; Uesato, J.; Rauh, M.; Griffin, C.; Huang, P.-S.; Mellor, J.; Glaese, A.; Cheng, M.; Balle, B.; Kasirzadeh, A.; et al. 2022. Taxonomy of Risks Posed by Language Models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 214–229.DOI
  27. [29]Xiong, M.; Hu, Z.; Lu, X.; Li, Y.; Fu, J.; He, J.; and Hooi, B. 2024. Can LLMs express their uncertainty? An empirical evaluation of confidence elicitation in LLMs. In International Conference on Learning Representations (ICLR).arxiv.org/abs/2306.13063
  28. [30]Zenke, F.; Poole, B.; and Ganguli, S. 2017. Continual learning through synaptic intelligence. In Proceedings of the International Conference on Machine Learning (ICML).arxiv.org/abs/1703.04200
  29. [31]Zhou, L.; Schellaert, W.; Martínez-Plumed, F.; Moros-Daval, Y.; Ferri, C.; and Hernández-Orallo, J. 2024. Larger and more instructable language models become less reliable. Nature, 634(8032): 61–68.DOI

Paper details

vlnc:W_fc36qsgr
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 5

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents