Cross-Primitive Consistency in Jev

https://github.com/67L1/PrimaCons-CRPR

Introduction

Jev has attracted growing interest as an approach to structured decision-making with LLMs. Recent studies have investigated its capabilities, implementations, and extensions [1, 12, 16, 33], alongside emerging applications in clinical evaluation [11, 15, 20, 41, 45] and safety moderation [31, 42, 47, 58]. Unlike conventional LLM applications that generate free-form text, Jev represents decision tasks through typed querying primitives and obtains structured predictions directly from model outputs. For example, Noul formulates a binary decision, whereas Choice presents multiple candidate answers.

These primitives provide different interfaces for expressing decision tasks, allowing a model to return structured outcomes without generating and subsequently parsing a natural language response. This design reduces inference overhead and supports efficient decision-making across diverse applications. The combination of structured outputs and efficient inference has made Jev an increasingly relevant approach to deploying LLMs in decision-oriented settings.

Research on Jev has primarily examined its predictive capabilities, calibration, robustness, and performance on specific tasks [1, 40, 42, 53]. However, evaluating decisions under a single querying primitive leaves an important aspect of model behavior unexamined: whether the same semantic state yields consistent decisions when expressed through different primitives. This issue is closely related to consistency in LLMs, which has been studied under various forms of input variation [3, 13, 44, 50], but cross-primitive consistency in Jev remains unexplored. Unlike comparisons between arbitrary inputs, this setting requires the underlying decision semantics to remain fixed while the querying interface changes. When the resulting decisions disagree, the discrepancy cannot be explained by a change in the intended task alone. As illustrated in Figure 1, a Noul query returns “False” (the response is factual), whereas a semantically equivalent Choice query selects “Hallucinated” (the response is hallucinated). Such contradictions reveal a dimension of decision reliability that conventional accuracy evaluations cannot capture. A systematic investigation of cross-primitive consistency is therefore necessary to understand how querying interfaces affect Jev’s decisions and whether the resulting inconsistencies can be mitigated. To enable such an investigation, we introduce PRIMACONS, a benchmark that evaluates cross-primitive consistency along four complementary axes: Complement Consistency, Label Hierarchy Consistency, Abstention Alignment Consistency, and Label-Description Consistency. These axes examine decision invariance under binary isomorphism, taxonomic coherence, procedure invariance, description invariance, and projection invariance, respectively. PRIMACONS is constructed from HaluEval [25], PKU-SafeRLHF [21], AbstentionBench [23], and AG News [67], with separate calibration and test splits. By preserving the semantics of the queries while varying the relevant querying primitive or presentation choice, PRIMACONS supports controlled evaluation across multiple forms of decision inconsistency. We quantify discrepancies using probability gaps, total variation, and complementary decision-level metrics. Evaluations of JevK5-2B, JevK5-4B, and JevK5-9B reveal consistency violations across all four axes. Specifically, further analysis identifies the following four recurring patterns: cross-primitive discrepancies predict decision failure with a ROC-AUC of 0.91; inconsistency increases with finer label granularity and is accompanied by probability-mass inflation; the magnitude and direction of disagreement depend on the querying primitive; and text augmentation can exacerbate cross-primitive discrepancies. These findings demonstrate that the cross-primitive inconsistency is systematic and associated with identifiable properties of the decision task and its formulation. Building on these findings, we propose the Cross-Primitive Residual Calibrator (CPRC) to mitigate cross-primitive inconsistency. CPRC freezes the backbone and attaches a lightweight residual network that predicts logit corrections, adjusting decision distributions without updating the full model. Its training objective combines cross-entropy with a consistency penalty, jointly encouraging predictive accuracy and alignment across semantically equivalent queries. We evaluate CPRC using a leave-one-domain-out (LODO) protocol to assess its effectiveness on both observed and held-out domains. In the in-domain setting, CPRC reduces $G_{n^{+},n^{-}}$ from 0.183 to 0.087 (-52.6%) and improves accuracy from 77.67% to 98.48% (+20.81%). Under LODO evaluation, it achieves a reduction of $TV(p^{r},p^{n})$ from 0.439 to 0.182 (-58.6%), and improves accuracy on raw labels from 84.90% to 87.42% (+2.52%). These results demonstrate that cross-primitive consistency can be improved through lightweight calibration while maintaining or enhancing predictive performance across domains. Our contributions are threefold:

• We introduce PRIMACONS, a four-axis benchmark for systematically evaluating cross-primitive consistency in Jev while controlling for the semantics of the queries.

• We conduct an empirical study across 3 Jev models, revealing consistency violations across all axes and characterizing systematicity, granularity amplification, primitive asymmetry, and sensitivity to text augmentation.

• We propose CPRC, a lightweight calibration method that combines the predictive supervision with a cross-primitive consistency objective, improving decision consistency and accuracy across different domains.

PrimaCons

This section introduces PRIMACONS, a Cross-Primitive Consistency Benchmark, designed to evaluate Jev’s internal consistency when semantic states remain constant while calling primitives change. PRIMACONS encompasses four test scenarios, each targeting a distinct dimension of consistency: i) complement consistency (§ 2.1), ii) label hierarchy consistency (§ 2.2), iii) abstention alignment consistency (§ 2.3), and iv) label-description consistency (§ 2.4).

Axis I: Complement Consistency

PRIMACONS’s axis I tests whether positive and negative inquiries about the same proposition yield complementary probabilities, preserving binary isomorphism. To evaluate this, we adopt HaluEval [25], a hallucination benchmark in which a model must judge whether a response contains hallucinated content given a provided context. We split it into a balanced 6,000-sample calibration set and a 6,000-sample test set. For each record, we construct a state $s_i$ by concatenating the provided context and a target response. We then subject it to three primitives: a positive Noul (querying the presence of hallucination), a reversed Noul (querying factual correctness), and a binary Choice. We quantify consistency by the probability gap among the three views.

Axis II: Label Hierarchy Consistency

PRIMACONS’s axis II tests whether the coarse-grained decision of a sample agrees with what its fine-grained category decisions jointly imply. We adopt PKU-SafeRLHF [21], a safety alignment dataset that labels each text as toxic or not across multiple granularities. We retain the samples labeled as unsafe, and select 6,000 calibration samples and 4,000 test samples. We employ the original 12 fine-grained safety categories, which are then paired two-by-two into medium-level parent categories and ultimately projected to a single coarse label of whether the sample is safe. Instead of changing textual phrasing, we perturb the decision granularity. We quantify consistency between the fine-grained predictions at different levels and the coarse-grained predictions.

Axis III: Abstention Alignment Consistency

PRIMACONS’s axis III investigates whether offering an “unable to answer” option alters the model’s confidence about answerability. To assess this, we employ AbstentionBench [23], a benchmark in which a model is given a context and a question and judge the question’s answerability. We establish a 6,000-sample calibration set and a 6,000-sample test set, each covering an equal number of answerable and unanswerable samples. We test two primitives: a three-option Choice and a binary Noul. For answerable questions, the Choice options contain the correct answer, a distractor, and an “unable to answer” option. For unanswerable questions, they contain two distractors and an “unable to answer” option. For both types, Noul queries whether the question can be answered. We evaluate consistency by computing the probability gap and decision conflict rate between Choice’s and Noul’s abstention selections.

Axis IV: Label-Description Consistency

PRIMACONS’s axis IV measures distribution robustness when choice labels are augmented with text descriptions. Since the textual state is unchanged, neither description augmentation nor primitive should shift the distribution. Adopting the four AG News categories [67], we construct a 6,000-sample calibration set and a 6,000-sample test set. For each sample, we quantify consistency at two levels: i) between a Choice primitive with raw category labels and one with description-augmented labels, and ii) between Choice’s probability distribution and a normalized distribution derived from four category-specific Noul queries. We measure their consistency with Total Variation ($TV$).

Consistency on PrimaCons

Evaluation Metrics

We conduct experiments on JevK5-2B, JevK5-4B and JevK5-9B using the calibration split of PRIMACONS. For each evaluation axis, consistency metrics are introduced, with the exhaustive equations delegated to the Appendix.

Axis I: Complement Consistency. We measure the probability gap ($G$) among the positive Noul ($p_m^{n+}$), reversed Noul ($p_m^{n-}$), and Choice ($p_m^c$) primitives. Because the positive and reversed Nouls query the proposition and its complement respectively, we express both in the probability that the proposition holds, with $p_m^{n+}$ taken directly and $p_m^{n-}=1-p$ converted to the same direction for comparison. The pairwise inconsistency is measured over $M$ samples:

Axis II: Label Hierarchy Consistency. We quantify the consistency between Choice and Noul at three granularity levels: coarse ($C$, 2-class), medium ($M$, 6-class), and fine ($F$, 12-class). For each level $L\in\{C,M,F\}$ with $Y_L$ categories, $l_n^C$ denote Choice probability vector. Noul vector $l_n^N$ is constructed by concatenating the complement and raw probabilities at the coarse level ($l_n^N=(1-p_n^N,p_n^N)$), and by normalizing category-specific queries at the medium and fine levels. We then measure the level-specific disagreement $D_L$ using the sample-averaged $TV$.

Main Results and Analyses

capturing the cross-primitive divergence at each respective tier of the hierarchy.

Axis III: Abstention Alignment Consistency. We assess divergence using an abstention probability gap ($G_A$) and a hard-label conflict rate ($K$). Let $a_n^C$ be the probability the Choice primitive assigns to its abstention option, and $a_n^N$ the unanswerability probability reported by the Noul primitive, with $\hat{y}_n^C$ and $\hat{y}_n^N$ denoting the corresponding discrete decisions. We compute the probability gap as:

and the conflict rate as:

capturing how often the two primitives yield contradictory abstention decisions.

Axis IV: Label-Description Consistency. We quantify distribution shifts using $TV$. For each sample $i$, let $p_i^r$ and $p_i^d$ denote the $C=4$ class probability vectors of the raw-label and description-augmented Choice, respectively. For $p_i^n$, we obtain a per-category probability $\tilde{p}_{ic}$ from a Noul query on each category $c$ and normalize:

We then define the sample-averaged $TV$ as:

and report it for all three pairs from $\{p^r,p^d,p^n\}$.

Main Results and Analyses

On the calibration set, we present the results for each axis and find that JevK5 violates the corresponding logical invariants across all four axes, indicating that its output probabilities are sensitive to how the primitive is formulated.

Axis I: Complement Consistency. Binary Isomorphism requires that, in formal logic, the probability that a factual proposition is true ($p$) and the probability that it is false ($1-p$) sum to 1. As shown in Table 1, JevK5 violates this principle. A probability gap of $G_{n+,n-}=0.1360$ remains between the Positive Noul and the Reversed Noul, and the Choice primitive aligns with neither Noul view ($G_{n+,c}=0.1132$, $G_{n-,c}=0.0689$). This inconsistency indicates that JevK5 does not give a unified answer to the same proposition when it is asked in its positive and reversed forms, nor across the two primitives. The equivalent logical state fails to yield complementary probabilities.

Model	$G_{n^{+},n^{-}} \downarrow$	$G_{n^{+},c} \downarrow$	$G_{n^{-},c} \downarrow$
JevK5–2B	0.1858	0.2471	0.1127
JevK5–4B	0.1360	0.1132	0.0689
JevK5–9B	0.0892	0.0972	0.0686

To further investigate the violation of Binary Isomorphism, we analyze the sample-wise probability dispersion and its correlation with actual prediction errors. As shown in Figure 3, the Positive Noul $p^{n^{+}}$ and the Reversed Noul $p^{n^{-}}$ should coincide on the diagonal $y=x$ under perfect Binary Isomorphism. Instead, the predictions are far from this line, clustering in the top-left and bottom-right corners. It indicates that JevK5 produces severely conflicting probability estimates for the same state depending solely on the phrasing polarity. Besides, this cross-primitive disagreement is a strong symptom of JevK5’s unreliability. We adopt ROC-AUC to test whether the residual $\lvert p^{n^{+}}-p^{n^{-}}\rvert$ can rank the decision failure, and it reaches 0.91. This shows that when

JevK5’s decisions conflict across equivalent views, the conflict can be a signal of decision failure.

Axis II: Label Hierarchy Consistency. Taxonomic Coherence requires that when labels form a hierarchy, the aggregate probability mass of fine-grained categories perfectly explains the overarching coarse-grained prediction. As shown in Table 2, JevK5 violates this principle. At the coarse level, $TV$ between Choice and Noul is relatively low at $D_C = 0.0870$. However, as the category space breaks down into 6 medium and 12 fine categories, the divergence spikes to $D_M = 0.5329$ and $D_F = 0.6323$. This reveals that while primitives might agree on top-level abstractions, they lose consistency when distributing probability mass across fine-grained categorical spaces.

Model	$D_C \downarrow$	$D_M \downarrow$	$D_F \downarrow$
JevK5-2B	0.2945	0.4837	0.4890
JevK5-4B	0.0870	0.5329	0.6323
JevK5-9B	0.1300	0.3650	0.5107

To further investigate the violation of Taxonomic Coherence, we analyze sample-wise probability residuals across hierarchical levels. Let $\mathcal{L} = \{C, M, F\}$ denote the coarse, medium, and fine levels. We define the mass residual between two levels $X$ and $Y$ as

where $X$ is coarser than $Y$. As shown in Figure 4a, the $e^{CM}$ and $e^{CF}$ should coincide on the diagonal $y=x$ under perfect Taxonomic Coherence. Instead, the predictions deviate significantly from this line, clustering densely above it. This indicates that JevK5 produces larger divergence as the category granularity becomes finer. Figure 4b examines whether each Medium category’s probability is consistent with the sum of its two Fine probabilities. The average residuals for all Medium categories fall below 0, demonstrating that JevK inflates rather than partitions a fixed probability as granularity increases and fails to maintain stable decision states across hierarchical levels.

Axis III: Abstention Alignment Consistency. Procedure Invariance requires that a judgment be invariant to the elicitation procedure.

Selecting among a set of options and querying a single proposition in binary form should recover the same judgment [citation]. Here, Choice elicits abstention by selecting from an option set, whereas the Noul elicits it by querying answerability. Both are procedures for eliciting the same judgment of the question’s answerability. As shown in Table 3, JevK5 violates this principle. There is an average abstention probability gap of $G_A = 0.2614$ between the Choice and Noul primitives. More critically, the hard-label conflict rate reaches $K = 26.69\%$, accompanied by an accuracy gap of 0.1083. This indicates that JevK5’s decisions are not invariant to the elicitation procedure. The same judgment flips depending on whether it is elicited by selecting from an option set or by a binary query.

Model	$G_A \downarrow$	$K$ (%) $\downarrow$	Acc. Gap (%) $\downarrow$
JevK5-2B	0.1928	36.54	9.66
JevK5-4B	0.2614	26.69	10.83
JevK5-9B	0.2130	17.18	11.72

To further investigate the violation of Conservation of Uncertainty, we analyze the sample-wise abstention probabilities across primitives. Let $a^C$ and $a^N$ denote the abstention probabilities derived from the Choice and Noul, respectively. As shown in Figure 5, $a^C$ and $a^N$ should coincide on the diagonal $y=x$ under perfect Conservation of Uncertainty. Instead, the predictions scatter widely away from this line, yielding an average probability gap of 0.2614. By partitioning the decision space into four quadrants, we find that 24.74% of samples fall in the top-left quadrant, where Noul leans toward abstention while Choice does not. Symmetrically, 4.45% fall in the bottom-right quadrant. This indicates that JevK5’s abstention decision is highly susceptible to whether it is queried via Choice or Noul.

Axis IV: Label-Description Consistency. Description Invariance requires that rephrasing the options should not change the decision distribution. If two labels denote the same category, augmenting them with descriptive text should preserve JevK5’s judgment. As shown in Table 4, adding descriptive text shifts the Choice distribution ($\overline{TV}(p^r,p^d)=0.1013$). It indicates that even within the same primitive, the neutral descriptive text perturbs the decision distribution. Another requirement concerns the Projection Invariance between binary and multi-class views. Each category-specific Noul query answers a binary question and thus lives in $\{0,1\}^{K}$, whereas the Choice returns a point in the probability simplex $\Delta^{K-1}$. Normalizing the $K$ Noul probabilities is a projection from the former to the latter, and the projected distribution should recover the same decision state as the one reported by Choice. As shown in Table 4, shifting from Choice to the Noul triggers a relocation of probability mass, yielding $\overline{TV}(p^r,p^n)=0.2405$. Because JevK5 already violates description invariance, the descriptive text further amplifies this drift, raising the gap to $\overline{TV}(p^d,p^n)=0.2694$ when the same comparison is made with the description-augmented Choice. This gap indicates that the decision state is not preserved under the $\{0,1\}^{K}\to\Delta^{K-1}$ projection. To further investigate the violation of Description Invariance and Projection Invariance, we examine the sample-wise probability distributions across different primitives and choices. As shown in Figure 6, Choice and Noul probabilities should coincide on the diagonal $y=x$ if both invariances held. Instead, the predictions deviate markedly, exhibiting an asymmetric crossing pattern: when the Choice probability is low, the Noul probability tends to be higher, whereas when the Choice probability is high, Noul becomes more conservative. This indicates that the two primitives interpret the same probability space differently. Furthermore, the cross symbols (×), which denote the distribution means, reveal that for World, Sports, and Sci&Tech, the mean of the described choice lies farther from the diagonal than that of the raw choice. This demonstrates that adding descriptive text exacerbates cross-primitive inconsistency rather than

Model	$\overline{TV}(p^r,p^d) \downarrow$	$\overline{TV}(p^r,p^n) \downarrow$	$\overline{TV}(p^d,p^n) \downarrow$
JevK5-2B	0.0917	0.4386	0.4716
JevK5-4B	0.1013	0.2405	0.2694
JevK5-9B	0.0439	0.2227	0.2450

anchoring the semantics, showing that JevK5’s probability simplex is unstable to surface-level perturbations.

Ways to Mitigate Inconsistency

Calibrator Architecture. To mitigate cross-primitive inconsistency and improve Jev’s reliability, Cross-Primitive Residual Calibrator (CPRC) is proposed. CPRC is a residual calibrator that aligns the predictions of multiple primitives on the same semantic state (cf. Figure 7). Keeping the backbone model frozen, we append a lightweight MLP residual network. JevK5 consists of $L$ layers. For each primitive, this calibrator receives the PCA-projected hidden state from the $(L-1)$-th layer, the raw probabilities, and view-level metadata (e.g., predictive entropy). Separate binary and multiclass heads predict logit corrections, which we add directly to the original logits:

where $z_i^{\mathrm{raw}}$ is the raw logit, $x_i$ the concatenated calibrator input, $f_\theta$ is CPRC, and $z_i$ the calibrated logit.

Training and Evaluation Protocol. To prevent data leakage, we isolate training and evaluation, selecting PCA dimensions and hyperparameters exclusively on the calibration split before evaluating on the test split. Furthermore, to assess cross-domain generalization, we adopt a leave-one-domain-out (LODO) protocol. We train the calibrator on the combined calibration sets of three axes while holding out the remaining axis for testing. We target Axis IV as the held-out test domain, as it involves more diverse comparisons, making it suitable for evaluating generalization.

Training Objective. We optimize the calibrator using a joint objective:

where $\alpha$ weights the consistency penalty. For binary decisions, $\mathcal{L}_{\mathrm{CE}}$ is the binary cross-entropy

and for multiclass decisions, it is the standard cross-entropy

where $N$ is the number of calibration samples, $y_i$ is the ground-truth label, $p_i$ is the calibrated probability assigned to the positive class, and $p_{i,y_i}$ is the calibrated probability assigned to the ground-truth class $y_i$. The consistency penalty $\mathcal{L}_{\mathrm{cons}}$ targets the semantic divergence to each axis: i) the pairwise probability gap for Axis I (cf. Eq. (1)); ii) the $TV$ between Choice and Noul distributions at three granularities for Axis II (cf. Eq. (2)); iii) the probability gap between Choice abstention and Noul answerability for Axis III (cf. Eq. (3)–(4)); iv) and the $TV$ between Choice and Noul category-wise predictions for Axis IV (cf. Eq. (5)–(6)).

Calibration Results. Beyond measuring the consistency metrics of each axis, we additionally report accuracy.

In-Domain Evaluation. Table 5 shows the CPRC’s results on Axis I, II, and III. CPRC mitigates cross-primitive inconsistency across all three axes. For example, on JevK5–2B in Axis I, $G_{n^{+},n^{-}}$ drops from 0.183 to 0.087 (-52.56%) while Choice accuracy rises from 65.03% to 79.00% (+13.97%). On JevK5–4B in Axis II, the fine-level divergence $D_F$ drops from 0.633 to 0.355 (-43.95%) while accuracy rises from 95.20% to 99.65% (+4.45%). On JevK5–2B in Axis II, $D_M$ drops from 0.481 to 0.130 (-73.03%) while accuracy rises from 77.67% to 98.48% (+20.81%).

Model	Axis I				Axis II				Axis III
Model	$G_{n^{+},n^{-}} \downarrow$	$G_{n^{+},c} \downarrow$	$G_{n^{-},c} \downarrow$	Acc	$D_C \downarrow$	$D_M \downarrow$	$D_F \downarrow$	Acc	$G_A \downarrow$	$K(\%) \downarrow$	AG $\downarrow$	Acc
JevK5–2B	0.1832	0.2442	0.1106	65.03	0.2923	0.4806	0.4867	77.67	0.1884	36.15	8.77	77.81
JevK5–2B (CPRC)	0.0869	0.0962	0.0643	79.00	0.0666	0.1296	0.1573	98.48	0.2260	18.37	6.92	87.96
JevK5–4B	0.1361	0.1132	0.0688	71.77	0.0870	0.5330	0.6325	95.20	0.2619	26.74	11.03	85.63
JevK5–4B (CPRC)	0.0577	0.0619	0.0511	82.80	0.0842	0.2815	0.3545	99.65	0.2018	14.84	9.90	93.58
JevK5–9B	0.0861	0.0931	0.0654	71.22	0.1292	0.3618	0.5085	93.77	0.2103	17.27	11.51	91.79
JevK5–9B (CPRC)	0.0567	0.0633	0.0575	80.98	0.0890	0.1756	0.2534	99.13	0.1802	12.83	9.73	95.67

Out-of-Domain Generalization. To evaluate whether CPRC learns fundamental corrections rather than memorizing domain-specific statistics, we test it under the LODO protocol, with Axis IV as the unseen target domain. The results show that CPRC generalizes its consistency improvements to this unseen domain (Table 6). For example, on JevK5–2B, $\overline{TV}(p^{r},p^{n})$ drops from 0.439 to 0.182 (-58.57%) while raw-label accuracy rises from 84.90% to 87.42% (+2.97%). The same model also reduces $\overline{TV}(p^{d},p^{n})$ from 0.474 to 0.237 (-49.98%) while description-augmented accuracy rises from 86.80% to 87.90% (+1.27%). On JevK5–9B, $\overline{TV}(p^{r},p^{n})$ drops from 0.225 to 0.094 (-58.42%) while raw-label accuracy rises from 88.02% to 89.75% (+1.96%). Averaged over 6,000 test instances, CPRC introduces only a +0.75% additional latency overhead, which is negligible.

Model	Consistency ($\overline{TV} \downarrow$)
Model	$\overline{TV}(p^{r},p^{d})$	$\overline{TV}(p^{r},p^{n})$	$\overline{TV}(p^{d},p^{n})$
JevK5–2B	0.0825	0.4391	0.4736
JevK5–2B (CPRC)	0.0988	0.1819	0.2369
JevK5–4B	0.0907	0.2407	0.2696
JevK5–4B (CPRC)	0.0663	0.1719	0.2083
JevK5–9B	0.0447	0.2251	0.2477
JevK5–9B (CPRC)	0.0513	0.0936	0.1262

Related Work

System-One Decision Model reduces the cost and improves the speed of using LLMs as decision models through System One-style single-pass inference. Existing Jev-related methods are broadly categorized into three streams. The first category is Empirical Evaluation and Capability Distillation. These methods benchmark system limits or distill decision priors to optimize inference efficiency, e.g., probing zero-shot reasoning boundaries across diverse tasks [10, 30, 48, 65] and building Jev-style decision models via direct logit readout or low-rank adaptation [16, 32, 33, 39, 56, 60]. The second category is Domain-Specific Adaptation. These methods fine-tune decision layers to target specialized scenarios, e.g., automating clinical and medical evaluations [15, 41] and integrating multilingual or normative knowledge [5, 27, 59]. The third category is Adversarial Vulnerability and Safety Assessment. These methods perturb inputs or exploit contextual shifts to expose decision fragility, e.g., executing black-box adversarial edits and prompt injections [19, 31, 61] and demonstrating susceptibility to natural context flips [63], alongside deploying them as zero-shot safety detectors [18]. Despite delineating external performance boundaries, these methods only evaluate input-output pairs without verifying internal coherence, introducing reliability risks when semantically equivalent queries yield contradictory decisions. Unlike these methods, we construct PRIMACONS, which quantifies Jev’s internal consistency along four axes, and we further propose CPRC, a residual calibrator that mitigates the observed inconsistency with the frozen Jev model. Consistency of LLMs refers to the invariance of their outputs under semantically equivalent inputs. Existing studies are broadly categorized into three streams. The first category is Contextual and Semantic Perturbation Robustness. These methods inject surface-level noise to expose output instability, e.g., mutating prompts while preserving logical semantics to detect behavioral drift [8, 9, 14, 29, 52, 54] and evaluating reasoning degradation under adversarial inputs, irrelevant distractors, or counterfactual evidence [24, 35, 43, 46, 64, 66, 68, 69]. The second category is Internal Representation and Quantization Sensitivity. These methods perturb weights and activations to analyze collapse during low-bit compression, e.g., isolating activation spikes and positional encoding artifacts [6, 17, 22, 49], tracking how quantization shrinks decision margins [34, 34, 38, 51, 55],

Model	Raw Labels	Desc Labels
JevK5–2B	84.90	86.80
JevK5–2B (CPRC)	87.42	87.90
JevK5–4B	84.08	86.97
JevK5–4B (CPRC)	84.12	87.05
JevK5–9B	88.02	89.03
JevK5–9B (CPRC)	89.75	90.07

and probing rescaling invariance, feedback exposure, and value-magnitude sensitivity in low-bit rounding [2, 26, 28, 62]. The third category is Intermediate Reasoning Trace Faithfulness. These methods audit multi-step generation pipelines to detect logical disconnects, e.g., exposing rationales that fabricate evidence for predetermined answers [4, 64], optimizing intermediate token probabilities via consistency decoding or confidence-weighted aggregation, probing commitment misalignment [28], answer-reasoning disconnect [9, 37, 38], and semantic drift [7, 36, 57].

Our work builds on the unique architecture of decision models, in which the context is decoupled from the querying primitive. It raises a question that single-primitive architecture cannot capture: whether the same state yields a consistent decision across primitives, which directly reflects the model’s decision reliability. We quantify this cross-primitive consistency, grounding each axis in a distinct theoretical requirement (binary isomorphism, taxonomic coherence, procedure invariance, description invariance, and projection invariance) and propose CPRC, a calibrator that aligns the probability distributions across primitives.

Conclusion

System-One decision models bypass reasoning to accelerate inference. Their reliability is therefore hard to audit, yet it can still be assessed through consistency. Cross-primitive consistency offers one such lens. Therefore, we introduce PRIMACONS to quantify it, revealing that current decision models violate different invariants across complementary, hierarchical, abstention, and label-description dimensions. To mitigate this inconsistency, we propose CPRC, which appends a lightweight residual network and optimizes the network while keeping the backbone frozen via a joint consistency objective. Validated under a LODO protocol, CPRC curtails inconsistency and boosts accuracy across unseen domains. We hope this work offers a lens for assessing decision-model reliability and that the proposed calibrator helps make such models deployable in high-risk scenarios.