WorksComputer Science
GT-FENG: General Theory of Factual Evaluation and Normative Grounding in Language Models *Author(s)
Abstract
Whether a language model's answer is correct often depends on context: the same value can be right under one jurisdiction, date, or legal definition and wrong under another. We introduce GT-FENG (General Theory of Factual Evaluation and Normative Grounding), the first general framework to unify contextual/temporal correctness and rule-based error attribution across law, policy, and other rule-governed domains. GT-FENG checks answers against the applicable rules and identifies documented rules that match incorrect answers. It shows that when multiple rules give the same value, the answer alone cannot tell which rule the model used. In Medicaid income-limit answers from two models, documenting the asked state's own records reduces the number of wrong answers that match only other states' values by 68-78%, and more than half of stable wrong answers then match two or more documented sources. GT-FENG provides a foundation for evaluating language models in rule-governed domains and for future benchmarks and evaluation toolkits. We release GT-FENG as an open-source Python package, gtfeng.
Introduction
People ask language models about rules: the income limit for a public benefit, the deadline for a tax filing, or the duties of an employer. The correct answer depends on the jurisdiction, date, and definition specified in the question.
Standard factuality evaluations check answers against reference answers or sources [8] (Wei et al. 2024), but do not show which documented rules match a wrong answer. When asked for Oklahoma’s parent income standard in July 2025, Claude Sonnet 5.5 answered 41%, a wrong value that equals Iowa’s current standard and two earlier Oklahoma standards. A count of cross-state errors would attribute this answer to Iowa, and a count of outdated answers would attribute it to Oklahoma’s past.
We introduce GT-FENG (General Theory of Factual Evaluation and Normative Grounding), the first general framework to unify contextual/temporal correctness and rule-based error attribution across law, policy, and other rule-governed domains. GT-FENG defines contextual correctness (§ 3), finds the documented rules that match a wrong answer (§ 4), and shows the limits of this attribution (§ 5). We apply it to Medicaid eligibility answers from two models
Related Work
Many factual questions have no answer that is true everywhere and at all times ([4]). Zhang and Choi (2021) found that roughly 16.5% of information-seeking questions in NQ-Open have answers that depend on when or where the question is asked, and Jiang et al. [7] found that a considerable share of samples in widely used factuality benchmarks are now outdated. In law and policy, correctness often depends on which rules apply at a given place and time (Zheng et al., 2025). Dahl et al. [3] documented legal hallucinations across jurisdictions, courts, and time periods, and Yang et al. (2026) built CrossLex to test whether models recognize that the same facts call for different rules in China, California, and Germany. Several benchmarks focus on time. Prior et al. [11] studied stale and recency-biased answers to German statutory questions, Cymbler et al. [2] showed that legal retrieval systems cite the version currently in force when an earlier or later version applies, Sobhani et al. [13] tested version resolution on Bangladeshi customs instruments, and Nicolaysen [9] classified answers about numeric values in Norwegian law as correct, outdated, wrong, or uninterpretable against each provision’s dated history. These studies treat jurisdictional and temporal validity as part of correctness.
General factuality frameworks check answers against evidence [12]. Iqbal et al. [6] built OpenFactCheck, a unified framework that checks the claims in a response and compares fact-checking systems. Xu et al. (2025) added error attribution to evaluation and assigned each failed answer to one of 15 secondary error categories with a fine-tuned judge model. Alvarez et al. [1] identified cross-source conflation in tool-using agents, where a claim is supported by the evidence but attributed to the wrong source. These frameworks ask whether an answer is supported or why it failed. GT-FENG asks a narrower, rule-based attribution question: which documented rules set the value that a wrong answer gives.
Work on normative documents treats version and scope as part of the answer. The LegalRuleML standard represents legal rules together with their jurisdiction, authority, and temporal validity [10]. Wang et al. (2026) found that a governed system resolving version and scope through explicit rules before generation outperformed a hosted retrieval service on questions over normative documents, and Ulizko et al. (2026) introduced a benchmark for normative reasoning when the supplied evidence is inconsistent or defeasible. Feng [5] linked wrong Medicaid answers to other states’ values and to the asked state’s own earlier values and conventions, and found that the number of cross-state matches depends on how completely the asked state’s own records are documented. Although prior work addresses contextual correctness, rule applicability, and error attribution, it does not provide a unified account of how wrong answers correspond to documented rules, how such correspondences become ambiguous, or how attribution changes with the reference set. GT-FENG brings these elements together by checking answers against the applicable rule, identifying all matching rules within a specified reference set, and formalizing the resulting attribution ambiguity and reference-set dependence.
GT-FENG: General Theory
Contextual Correctness
We call a fact whose value depends on the jurisdiction, the date, and the legal definition in use a normative fact: its value is set by a rule, and the rule applies only within its context.
Definition 1 (Contextual correctness). An answer A to a question about quantity Q in jurisdiction J at time T under definition D is contextually correct when A equals the value of the rule that applies at (Q, J, T, D). Correctness is therefore a relation between an answer and a context. The same A can be correct in one context and wrong in another.
Indexed Normative Facts
GT-FENG indexes each rule by four coordinates: the quantity q, the jurisdiction j, the time t, and the definition d (the reporting convention or legal definition under which the value is stated). The gold value G(q, j, t, d) is the value that the rule sets at that index.
Value types. G can return a number (an income limit), a Boolean (whether a duty applies), a category (which program covers a person), or a set (the conditions that qualify a person). Equality is decided by a comparison rule fixed for each domain before scoring: numbers after normalization to the unit and precision of the source, Booleans and categories exactly, and sets as sets.
Further coordinates. The four coordinates cover the cases studied here. A domain can add coordinates, such as the issuing authority or the population a rule covers. The definitions and propositions below apply in the same way, and a source is then defined by every coordinate except time.
Normative Grounding
Definition 2 (Grounding). A wrong answer A is grounded in the rule at index (q, j, t, d) when that index is documented in R and G(q, j, t, d) = A.
Grounding is a documented correspondence between an answer and a rule. It states which documented rules agree with the answer. It makes no claim about the process inside the model that produced the answer, and Proposition 3 shows that no procedure using only the question, the answer, and R can make such a claim when two sources share a value.
Evaluation and Attribution
A question asks for one index, written (Q, J, T, D), and the model returns an answer A. An item is included in the analysis when every independent repeat returns the same answer.
| Symbol | Name | Meaning |
| , , , | Quantity, Jurisdiction, Time, Definition | coordinates the question asks about |
| , , , | (same) | any coordinate value |
| Gold | , the value set by the rule | |
| Answer | the model’s answer | |
| Reference | documented indices | |
| Matches | indices in whose gold value equals | |
| Shift | coordinates in which a match differs | |
| Family | a set of shifts | |
| Unmatched | wrong answer with no match | |
| Observed | observed count | |
| Null | count expected under the local perturbation baseline | |
| Excess | ||
| Increments | offsets used to build nearby values | |
| Perturbations | number of random perturbations for the -value |
Table 1. GT-FENG notation. Each symbol is the initial of its name.
Factual Evaluation
Rule 1 (Correct). If , the answer is correct.
Rule 2 (Matches). Otherwise, the match set contains every index in that satisfies both conditions:
(this excludes the correct rule itself)
Rule 3 (Unmatched). If is empty, the answer is labeled U.
Normative Grounding
Rule 4 (Shift). The shift of a match is the set of coordinates in which it differs from , written as a string of coordinate letters. A match with has shift J. A match with has shift T. A match with and has shift JT.
Rule 5 (Label and ambiguity). If every match in has the same shift , the answer is labeled S. Otherwise the answer is labeled ambiguous, and the label lists every shift present in $M. GT-FENG distinguishes two kinds of ambiguity:
Shift ambiguity: contains matches with two or more different shifts. Oklahoma’s parent standard for July 2025 is not 41%. At reference level (§ 6), 41% is Iowa’s current standard (shift J), Oklahoma’s 2018 standard (shift T), and Oklahoma’s 2022 standard under the other disregard convention (shift TD), so the label is ambiguous {J, T, TD}.
Source ambiguity: contains two or more distinct sources. Indices with the same quantity, jurisdiction, and definition count as one source, because they describe one rule over several periods. GPT-5.6 Sol answers 305% for Massachusetts’s pregnancy limit (gold 205%). At R3, 305% is the current limit of both Missouri and West Virginia. Both matches have shift J, so the label is J and the answer is source-ambiguous.
Shifts are computed over every matching index, and sources are counted after indices of the same quantity, jurisdiction, and definition are merged. The two kinds of ambiguity are counted separately. A single source can have two shifts: if another state set the answer’s value in both 2018 and 2025, its two indices form one source with shifts J and JT.
An item is grounded in a family of shifts (an error category) when is nonempty and every shift present in belongs to . can be any set of shifts, such as {J, JD} for cross-jurisdiction errors or {T, TD} for outdated values.
Rule 6 (Reference levels). The evaluator reports labels at nested reference sets R1 R2 R3 . R1 contains the indices that a naive check would use, and the last level contains the most complete documentation available.
Rule 7 (Local perturbation baseline). For numeric values and a family , is the number of wrong items grounded in . For each wrong item, the evaluator replaces by every nearby value , where runs over a fixed set of increments I. excludes increments equal to a known definition shift, and increments that reach the correct value are dropped for that item. is the sum over wrong items of the share of nearby values that are grounded in . The excess , measured relative to the specified local perturbation baseline, is the number of grounded items beyond those that nearby values produce. The p-value is the share of random perturbations (each taking one increment per item) for which the number of grounded items reaches .
Standard Report
At each reference level, a GT-FENG evaluation reports seven quantities: (1) correct items; (2) items labeled U; (3) single-shift labels, by shift; (4) shift-ambiguous items; (5) source-ambiguous items; (6) the mean number of distinct sources in among matched items; and (7) for each family of interest. Reporting the same quantities makes the results comparable.
Theoretical Properties
Write for the match set computed with reference set .
Proposition 1 (Monotonicity). Let . For every item, . Hence (a) an item labeled U under is labeled U under , and (b) an item that is shift-ambiguous or source-ambiguous under stays so under .
Proof. Rule 2 selects indices from by a condition that does not depend on , so a larger yields a larger or equal . For (a), empty implies empty. For (b), two matches in also belong to , with their shifts unchanged. □
Proposition 2 (Upper bound under full coverage). Let be a family of shifts and . Assume coverage: contains every index of whose shift belongs to . Then every item grounded in under is grounded in under , and the number of items grounded in cannot increase from to $R2.
Proof. Take an item grounded in under . Every match in has a shift in , so by coverage every match lies in , and . Proposition 1 gives . The two sets are equal, so the item is grounded in under . □
The coverage assumption is the only condition. It holds for whenever lists every other jurisdiction’s value at time under every definition, and it holds for joint families such as whenever lists every other jurisdiction’s full history under every definition.
Proposition 3 (Attribution non-identifiability). Suppose contains matches from two distinct sources, with indices and . Consider two accounts of the answer: the model drew on the first source, or the model drew on the second. Any attribution procedure that uses only the question, the answer, and returns the same output under both accounts, so a procedure that names a single source is wrong under at least one of them.
Proof. Both sources set the value A, so both accounts produce the same observable answer. The question and are fixed. An attribution procedure is a function of these observations, so it returns the same output under both accounts. If that output names one source, it names the wrong source under the other account. □
Proposition 3 defines the limit of attribution from answers. Under source ambiguity, is the most that a procedure using only the question, the answer, and can return. A precedence order, such as checking the asked jurisdiction’s own values first, makes a reporting choice among the rules in , and GT-FENG records in full alongside any such choice.
Instantiation: Medicaid Income Eligibility
We instantiate GT-FENG on the Medicaid eligibility data of Feng [5]. is one of three quantities (pregnancy, infant, parent). is one of 51 jurisdictions (the 50 states and the District of Columbia). is July 2025, and earlier times are the MACStats editions 2018–2024. is the treatment of the 5-point income disregard (included or excluded). Federal anchors (100%, 133%, 138%) are matched as a separate fixed set. A match to an anchor carries the label federal, a match to another number in the asked state’s data book carries the label Q (Table 2), and both labels lie outside .
lists every index with shift J or JD, so the coverage assumption of Proposition 2 holds for at every later level.
The counts reproduce the four protocol levels of the original study exactly (37, 16, 10, 8 and 47, 32, 25, 15). Each column moves in the direction Propositions 1 and 2 require: U and O never rise, and both kinds of ambiguity never fall. At R3, about half of all wrong answers are shift-ambiguous (33
| Level | Indices added | Shifts added |
| R1 | other states, July 2025, both definitions | J, JD |
| R2 | asked state: other definition; editions 2021, 2023, 2024; federal anchors | D, T, TD, federal |
| R3 | asked state: every edition 2018–2024 | T, TD |
| R4 | asked state: every number in its data-book rows 2018–2025, with and without the disregard, other than values of the asked quantity | shifts containing Q, recorded as Q |
| Level | Indices added | Shifts added |
| R1 | other states, July 2025, both definitions | J, JD |
| R2 | asked state: other definition; editions 2021, 2023, 2024; federal anchors | D, T, TD, federal |
| R3 | asked state: every edition 2018–2024 | T, TD |
| R4 | asked state: every number in its data-book rows 2018–2025, with and without the disregard, other than values of the asked quantity | shifts containing Q, recorded as Q |
Table 5.
| Model | Level | Shift-ambiguous | Source-ambiguous | Sources per matched item | ||
| Claude | R1 | 25 | 13 | 30 | 3.1 | 37 |
| Claude | R2 | 15 | 28 | 30 | 3.2 | 16 |
| Claude | R3 | 11 | 33 | 35 | 3.2 | 10 |
| Claude | R4 | 6 | 35 | 36 | 3.2 | 8 |
| GPT | R1 | 21 | 21 | 33 | 2.6 | 47 |
| GPT | R2 | 17 | 29 | 36 | 2.9 | 32 |
| GPT | R3 | 15 | 32 | 38 | 3.0 | 25 |
| GPT | R4 | 9 | 40 | 41 | 3.0 | 15 |
Table 2. Reference levels for Medicaid. Each level contains the one above it.
Table 3. Quantities (2) and (4)–(7) of the GT-FENG standard report for stable wrong items (Claude Sonnet 5.5: 62 items; GPT-5.6 Sol: 68 items). Wrong items include answers under the other disregard convention (11 and 6), which have shift D. O counts items grounded in .
of 62 for Claude, 32 of 68 for GPT), and more than half are source-ambiguous (35 and 38). Under Proposition 3, these items have a documented set of supporting rules and no single identifiable source.
Generality
GT-FENG applies to facts whose value is set by a rule in context. Table 4 shows how the same coordinates and rules apply to four types of question. The rows illustrate the representation and are not empirical results.
| Question type | Q | J | T | D | Value type | Example of a grounded error |
| Benefit eligibility | income limit for a group | state | effective date | income counting convention | number | the limit of another state (J) |
| Statutory deadline | days allowed for a filing | country or state | statute version | calendar or business days | number | the period of the repealed version (T) |
| Legal duty | whether an employer must provide paid leave | state or city | effective date | definition of a covered employer | Boolean | the rule for a larger employer class (D) |
| Program conditions | conditions that qualify a person | state | program year | statutory or administrative list | set | last year’s list (T) |
Table 4. GT-FENG coordinates across value types (illustrative).
Value types differ in how often grounding is informative. A Boolean question has two possible answers, so whenever documents many rules, almost every wrong answer is grounded in many of them and is source-ambiguous. A numeric question with many distinct values allows finer attribution.
Conclusion
We introduced GT-FENG, a general framework for evaluating answers whose correctness depends on the rule that applies. GT-FENG checks each answer against the rule that applies and finds the documented rules that match a wrong answer. We show that when two rules share a value, the answer alone
cannot tell which rule a model used. In Medicaid eligibility answers from two models, documenting the asked state’s own records reduces the number of wrong answers that match only other states’ values from 37 to 8 and from 47 to 15, and at the most complete level more than half of stable wrong answers match two or more documented sources. Evaluations in law, policy, and other rule-governed domains can report these correspondences, their ambiguity, and their dependence on the reference set.
Limitations
The empirical instance covers one policy domain, three quantities, two models, and 62 and 68 stable wrong answers; the other value types in Section 7 are illustrations. GT-FENG matches answers only to documented rules, so an answer that equals an undocumented rule is labeled U. It reports which rules agree with an answer and makes no claim about the process inside the model. Reference levels and families of shifts are chosen by the evaluator, and counts depend on these choices, which is why the standard report includes every level. Numbers are compared exactly after normalization, so near misses are unmatched. Items whose repeats disagree are left out.
References
- [1]Ander Alvarez, Santhiya Rajan, Alessandro Genuardi, Oliver Wirjadi, Samuel Mugel, and Román Orús. 2026. ProvenanceGuard: Source-aware factuality verification for MCP-based LLM agents. arXiv preprint arXiv:2606.18037.arxiv.org/abs/2606.18037
- [2]Rose Cymbler, Daniel Guez, and Laurent Fabre. 2026. Temporal misgrounding in legal RAG: A versioned-corpus benchmark for French tax law. arXiv preprint arXiv:2608.09393.arxiv.org/abs/2608.09393
- [3]Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E. Ho. 2024. Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis, 16(1):64–93.DOI
- [4]Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eischlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. 2022. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10:257–273.
- [5]Jiayu Feng. 2026. Right number, wrong state? measuring cross-jurisdiction substitution in LLM recall of state policy. arXiv preprint arXiv:2610.09458.arxiv.org/abs/2610.09458
- [6]Hasan Iqbal, Yuxia Wang, Minghan Wang, Georgi Georgiev, Jiahui Geng, Iryna Gurevych, and Preslav Nakov. 2024. OpenFactCheck: A unified framework for factuality evaluation of LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 219–229, Miami, Florida, USA. Association for Computational Linguistics.
- [7]Xunyi Jiang, Dingyi Chang, Julian McAuley, and Xin Xu. 2026. When benchmarks age: Temporal misalignment through large language model factuality evaluation. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 500–512, Rabat, Morocco. Association for Computational Linguistics.
- [8]Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannah Hajishirzi. 2023. FACtScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12076–12100, Singapore. Association for Computational Linguistics.
- [9]Eirik Botten Nicolaysen. 2026. norregel: a preregistered benchmark of dated numeric values in Norwegian law. arXiv preprint arXiv:2610.29207.
- [10]OASIS. 2021. LegalRuleML core specification version 1.0. https://docs.oasis-open.org/legalruleml/legalruleml-core-spec/v1.0/os/legalruleml-core-spec-v1.0-os.html. OASIS Standard, 30 August 2021.
- [11]Max Prior, Andreas Schultz, and Matthias Grabmair. 2026. Asking for an old friend: Diagnosing and mitigating temporal failure modes in LLM-based statutory question answering. arXiv preprint arXiv:2605.23497.arxiv.org/abs/2605.23497
- [12]Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter. 2023. Measuring attribution in natural language generation models. Computational Linguistics, 49(4):777–840.
- [13]Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Fahmid Hasan Chowdhury, Md Adnan Arefeen, Farig Sadeque, Md. Faizul Bari, and Swakkhar Shatabda. 2026. Time present and time past: Benchmarking large language models on temporally evolving document understanding. arXiv preprint arXiv:2608.08512.arxiv.org/abs/2608.08512