WorksComputer Science
Tracing How Language Models Integrate Temporal Anchors into Factual Recall
Abstract
Factual recall in language models is typically framed as retrieving a factual association: a subject–relation pair maps to a single object. However, many facts are context-dependent: the same pair maps to different objects depending on a condition such as time. We introduce anchored factual recall, in which the model receives an explicit conditioning expression, an anchor, and must select the object valid under it. This raises questions that single-object recall does not: where the anchor enters the computation, what it changes, and how that change selects the answer. Focusing on temporal anchors, we answer them with causal interventions on four open-weight models on English prompts and identify two routes. In a subject-mediated route, the anchor resolves the final subject-token representation to the queried year, and the relation redistributes the readout of this state across attention heads. In a direct route] the final position attends to the anchor, and upper-layer MLPs turn this signal into a preference for the time-appropriate object. The two routes are parallel channels: their contributions add up, most queries use both, and their balance depends on the query, e.g., on the relation, consistently across models. This reframes mechanistic accounts of factual recall from retrieving a single stored object to selecting among context-dependent alternatives.
Introduction
Language models (LMs) encode large amounts of factual knowledge in their parameters, and a growing body of work characterizes how this knowledge is retrieved during inference [17, 19, 2, 14]. An influential account by Geva et al. [5] describes recall for prompts such as “The capital of France is __” in three stages: the model enriches the subject representation with attributes, propagates relation information to the final position, and extracts the target attribute there. Related studies support key parts of this picture [13, 9] (Yu et al. 2023).
This account addresses facts represented as (subject, relation, object) triples, in which the subject and relation determine a single object, so recall amounts to retrieving that object. However, many real-world facts pose a different problem: “Cristiano Ronaldo played for __” has no unique answer until “in 2010” or “in 2020” resolves it to “Real Madrid” or “Juventus.” Here several objects are valid for the same subject and relation, and the model must select among them according to a condition. Selection raises questions unique to it, which do not arise in the recall of triple-form facts: where the condition enters the computation (Q1), what it changes (Q2), and how that change makes one object preferred over its competitors (Q3).
We study anchored factual recall, in which the model receives an explicit conditioning expression, an anchor, alongside the subject and relation: given (anchor, subject, relation), it must recall the object. We focus on temporal anchors, a central case in which the correct object changes over time [21] (Zhao et al. 2024). Work on time-conditioned recall has almost exclusively examined model outputs [4, 1, 8]. Park et al. [16] identify attention heads specialized for temporal knowledge, but how the anchor’s information is routed and transformed to select one object over its competitors remains untraced.
We answer Q1–Q3 with causal interventions across four models on English prompts, and find the following (Figure 1). Q1 (where): the anchor enters at two points: the subject span uses it in early-to-middle layers, and the final position accesses it directly from the middle layers onward, defining a subject-mediated route and a direct route. Q2 (what): the final subject-token state becomes specific to the queried year and selectively supports the object valid at that year. Q3 (how): on the subject-mediated route, the relation sets which attention heads read the subject state out, and these heads causally support the prediction. On the direct route, the final position reads the anchor, and upper-layer MLPs convert this signal into a preference for the time-appropriate object. The two routes are parallel channels: their contributions add up, most queries use both, and the balance between them varies with the query, consistently across models.

Figure 1. Two routes integrate the temporal anchor into factual recall. Subject-mediated: the anchor resolves the subject representation to 2020, and the relation sets how the final position reads it. Direct: the final position reads the anchor itself, and MLPs turn it into a preference for Juventus. The two routes are used jointly on most queries, with a balance that varies across queries.
Together, these results reframe the standard picture of factual recall from retrieving an object to selecting one among competing objects, a selection that the model carries out through two parallel, jointly used routes. To our knowledge, this is the first account of the information flow by which a temporal anchor selects among the competing facts an LM stores: where its information enters, how it is routed and transformed, and how the two routes combine. More broadly, it provides a starting point for understanding how LMs recall knowledge whose validity depends on the conditions under which it is queried.
Task Definition
Anchored Factual Recall
Mechanistic studies of factual recall typically consider facts represented as triples , in which the object is treated as determined by the subject and relation . This formulation supports the study of how an LM retrieves a fixed factual association from a subject–relation prompt, but it does not represent cases in which the valid object depends on an additional condition.
We instead consider anchored facts, represented as quadruples , where an anchor conditions which object is valid. For a fixed subject–relation pair, different anchors may select different objects. Writing for the map from a query to its gold object, this means for some . In temporally anchored factual recall, the anchor is a year, and the model must select the object valid at that time. Anchored factual recall therefore poses a different mechanistic problem from fixed-association recall: the model must not only retrieve factual information, but also use the anchor to resolve competition among context-dependent alternatives.
Prior work on unanchored factual recall provides causal tools for tracing how internal components contribute to a prediction ([5, 13, 9] (Yu et al. 2023)). We build on this intervention-based methodology, but do not assume that a computational decomposition developed for subject–relation prompts applies to anchored factual recall.
Scope and Notation
Anchored factual recall is a general formulation; we study its temporal case, in which the anchor is a year and the model must select the object valid at that time. Because we ask how the anchor selects the object, we need queries in which the object is predicted last and only the anchor varies. We thus use fixed English anchor-first queries,
In [year], [subject] [relation].
Our findings are scoped to this setting.
Extending the positional decomposition of Geva et al. (2022) with an anchor, we partition each tokenized query into four spans: the anchor span ; the subject span ; the last-position span , the final query token alone; and the relation span , the relation tokens preceding it. For example, for the query “In 2010, Cristiano Ronaldo played for”, covers “2010”, covers “Cristiano Ronaldo”, covers “played”, and is “for”; the remaining tokens (“In” and the comma) belong to no span. Thus, realizes the full relation expression, and the spans occur in the fixed order . Following the same source–target convention, we write to denote attention from query positions in the target span to key–value positions in the source span . For example, denotes attention from the final position to the subject span, whereas denotes attention from the relation span to the anchor span.
Experimental Setup
Data. We use TempLAMA ([4])1, a dataset of temporally scoped factual queries built from Wikidata, in which each subject–relation template is paired with a year between 2010 and 2020 and the gold object may vary across years. It represents time-scoped facts as quadruples : in our setting, the anchor is a year, and the valid object may change across years for a fixed subject–relation pair . We group instances by subject–relation pair and restrict the analysis to groups for which the gold object changes across years and the model correctly predicts the gold object for at least two years whose gold objects differ. For every retained group, we include all year-specific instances the model answers correctly. This restriction focuses our analysis on cases in which the model demonstrably exhibits time-sensitive factual recall.2 We count a prediction as correct if its greedy continuation contains the canonical gold object or its accepted aliases under a normalized, word-bounded matching criterion. We keep only years with a single gold object.
Prompts. Each prompt consists of five in-context demonstrations followed by the query defined in §2; each demonstration is a completed query of the same form, ending with its gold object. For each retained instance, we randomly sample five demonstrations that share neither year, subject, nor relation with it, so that no demonstration leaks the answer or its query pattern. The sampled demonstrations are fixed for that instance and reused across all analyses of that instance and all year-swapped contrast runs derived from it.
Models. We benchmarked several open-weight 7–8B models on TempLAMA and retained the four highest-scoring, ensuring that enough instances survive the filter above: Llama-3.1-8B (Grattafiori et al. [6]), Qwen2.5-7B (Qwen et al. [18]), 2023) and CounterFact (Geva et al. [5]).
Ministral-8B (Liu et al. (2026a))3, and DeepSeek-R1-Distill-Llama-8B (DeepSeek-AI [3]).
Reference–contrast pairs. To isolate the anchor’s effect, most interventions compare a query with a copy differing only in the year. For each instance, which we call the reference, we sample one contrast year from the same group whose gold object differs from the reference’s, and replace only the year in the query; demonstrations and all other tokens are unchanged. We keep a pair only if the model answers both prompts correctly under these shared demonstrations. We call the two forward passes the reference run and the contrast run, and write and for the first tokens of the canonical reference and contrast objects; pairs in which the two share the first token are excluded. All interventions apply to the query segment only; their layer scope is given in Appendix § 7.§ 2.
Mechanistic Analysis
Our analysis follows the three questions from §1: where the model uses the anchor (Q1; §4.1), what the anchor changes (Q2; §4.2), and how that change is turned into a preference for the time-appropriate object (Q3; §4.3).
Q1: Where Does the Anchor Enter?
A: Temporal information enters through two routes: a subject-mediated route (subject span, early-to-middle layers) and a direct route (final position, middle layers onward).
Method. We first ask at which positions, and at which layers, the anchor’s information enters the recall computation. An entry point is a position whose attention to the anchor the prediction depends on; to find these, we follow Geva et al. [5] and apply attention knockout to each edge for : over a window of adjacent layers centered at , we block all heads at from attending to the anchor span and measure the change in the reference-token probability,
where and are the next-token probabilities of at the final position in the knockout run and in the unmodified run of the same prompt. A negative indicates that attention from to the anchor causally supports the reference prediction in the tested layer window.
Result. Figure 2 shows for the three edges. Across all four models, blocking produces the earliest substantial decrease in the reference-token probability, with the effect concentrated in the early-to-middle layers. Blocking has little effect in the earliest layers but produces a later decrease from the middle of the network onward. By contrast, blocking has a much smaller effect: a shallow dip in the middle layers. Although the magnitude and layer range of each effect vary across models, this layer-wise ordering is consistent across the models.

Figure 2. Q1: the anchor enters the computation at two points. Attention knockout of degrades the prediction in early-to-middle layers, knockout of from the middle layers onward, whereas knockout of has little effect.
Temporal information thus becomes causally relevant at two points in the recall computation. At the subject span, where blocking degrades the prediction early, the subject tokens read the anchor; at the final position, where blocking degrades it later, the anchor is read directly, without passing through the subject. The relation span, by contrast, shows little sign of being a major carrier of temporal information: blocking costs little. One possible reason is that the subject and the anchor together already constrain the relation to the subject’s time-varying attributes, leaving the relation span less to gain from reading the anchor. We call the paths through the subject span and through the final position the subject-mediated route and the direct route. Here route means a path by which anchor information reaches the prediction, not a separate set of components. §4.2 and §4.3.1 characterize the first, § 4.3.2 the second, and §4.3.3 how they relate.
Q2: What Does the Anchor Change?
A: The anchor changes the final subject-token representation into a largely entity-specific state that causally and selectively supports the year-appropriate object.
Method. We apply activation patching (Meng et al., [13]; Wang et al., [23]): at each layer , we replace the residual-stream state at the final subject token, where subject information accumulates (Geva et al., [5]; Meng et al., [13]), in the contrast run with that of the reference run. Because the two prompts differ only in the anchor, any change in the prediction reflects anchor-dependent information in the patched state. We measure the logits of the first continuation tokens of three classes of temporal candidates: the reference object , the contrast object , and other, the mean over the group’s remaining gold objects (computed for groups that have at least one besides and ). We report the change in each logit relative to the unmodified contrast run, so that zero corresponds to no effect of the patch. If the subject state did not depend on the anchor, patching would change none of the three logits. If it is specific to the reference anchor, it should raise , lower , and leave the other candidates unchanged: the first two show that the patch replaces one year’s preference with another’s, and the third that this replacement does not spill over to the objects of other years.
Result. Across all four models, patching the reference subject state into the contrast run selectively shifts the prediction toward the reference-year object (Figure 3). The logit of increases, the logit of decreases, and the mean logit of the remaining temporal candidates changes little. The effect emerges and reaches its maximum in the middle layers. Rather than broadly increasing all objects associated with the pair, the intervention selectively favors the reference-year object, suggesting that the patched representation carries anchor-dependent information that downstream computation uses to distinguish among temporal alternatives.

Figure 3. The subject state selectively supports the year-appropriate object. Patching the reference subject state into the contrast run raises the logit of the reference-year object , lowers the logit of the contrast-year object , and leaves the mean logit of the other temporal candidates largely unchanged.
One alternative explanation is that the anchor simply adds a generic year-dependent component to the subject state, shared across entities. We test this possibility in Appendix § 8.§ 1 by estimating such a shared year offset from other subjects. Across all four models, this offset recovers at most about a quarter of the effect produced by the true reference-year subject state. The anchor-dependent change is therefore largely specific to the queried subject, rather than a generic representation of the year. Taken together with the selective patching effect above, this suggests that the anchor resolves the subject representation toward the object that is valid for that subject at the queried time.
Q3: How Is the Answer Selected?
A: Through both routes: on the subject-mediated route, the relation redistributes readout of the time-conditioned subject state across attention heads, and relation-sensitive heads causally support the prediction (§4.3.1); on the direct route, upper-layer MLPs turn the anchor signal into a preference for the year-appropriate object (§ 4.3.2). The two routes are parallel channels, jointly used on most queries, and the balance between them varies with the query (e.g., the relation), consistently across models (§4.3.3).
Subject-Mediated Route: The Relation Reconfigures the Head-Wise Readout from the Subject State
We begin with the subject-mediated route. The previous section showed that the anchor makes the subject state selective for the object valid at the queried time. The final position can use this state only by attending to it, so we next ask how the relation controls that attention: which heads read from the subject, and whether they matter.
Method. Let denote the attention weight from the final position to the final subject token in head of layer for prompt . We compare two swaps while holding the demonstrations fixed. A relation swap replaces the relation expression with another relation having the same number of tokens, so that token positions remain aligned. A time swap changes only the anchor year.
(i) Redistribution across heads. We ask whether changing the relation changes which heads read from the subject. Within each layer, we normalize the last-subject attention weights across heads,
and measure the total-variation distance between the resulting distributions:
This normalization removes changes in the total attention paid to within a layer and isolates changes in how that readout is distributed across heads. A larger distance under the relation swap than the time swap indicates that the head-wise allocation of subject readout depends on the queried relation rather than merely on the temporal anchor.
(ii) Cross-fitted ablation. A change in attention weights is descriptive: a head can change its readout without affecting the answer. We therefore ask whether heads whose readout from the subject is particularly sensitive to the relation are causally important for the prediction.
We split subject–relation groups into five folds. On the discovery folds, each layer–head pair receives a relation-sensitivity score
where base, rel, and time denote the original, relation-swapped, and time-swapped prompts. The score is high when a head’s attention to changes more when the relation is swapped than when only the anchor year is swapped. Unlike the redistribution analysis above, this score uses the unnormalized weights: causal relevance may depend not only on how subject readout is redistributed across heads, but also on how much a head reads from the subject in absolute terms. We select the top 10% of layer–head pairs without using answer logits.
We then evaluate the selected pairs on held-out correctly answered base prompts. Instead of ablating a whole head, we remove its contribution from the final subject token to the final position :
where is the head’s output projection and its value vector at . This contribution is recomputed during the intervened forward pass, so ablations in later layers operate on the state produced by earlier interventions. We report the top-1 flip rate, the fraction of prompts for which the originally correct top-ranked next token is no longer top-ranked after ablation. As controls, we ablate time-sensitive layer–head pairs selected by the reversed score (time swap minus relation swap) and contribution-matched random pairs from the same layers (Appendix § 7.§ 2).
Result. (i) Redistribution. Changing the relation alters the head-wise distribution of last-subject readout far more than changing only the anchor year (Figure 4). Across all four models, the distance under the relation swap exceeds that under the time swap at every layer. Changing the relation thus redistributes the readout of the time-conditioned subject state across attention heads.

Figure 4. The relation redistributes last-subject readout across attention heads. Per-layer total-variation distance between head-normalized last-subject attention distributions (Eq. (2)): swapping the relation changes the distribution far more than swapping the anchor year.
(ii) Ablation. The cross-fitted ablation shows that relation-sensitive subject readout is causally important for the answer (Figure 5). Removing the last-subject contribution of the selected layer–head pairs changes the originally correct top-1 next-token prediction in approximately 63% of Llama prompts, 52% of Qwen prompts, 52% of Ministral prompts, and 67% of DeepSeek prompts. By contrast, ablating time-sensitive pairs changes the prediction in only 2–5% of prompts, and contribution-matched random pairs in 8–12%.

Figure 5. Relation-sensitive heads causally support the correct prediction. Top-1 flip rate after ablating the last-subject contribution of relation-sensitive, time-sensitive, or contribution-matched random heads (selected on discovery folds; §4.3.1). Error bars: 95% bootstrap CIs.
For every model, the paired difference between relation-sensitive and contribution-matched random ablation is positive. With §4.2, these results support a division of labor along the subject-mediated route: the anchor makes the subject state selective for the object valid at that time, the relation determines how that state is read out at the final position, and relation-sensitive heads causally support the prediction.
Direct Route: Final-Position MLPs Convert the Anchor Signal into an Answer Preference
We now turn to the direct route. The knockout analysis showed that the final position reads directly from the anchor, but not how the resulting signal becomes selective for the correct temporal object. We therefore ask two questions: whether final-position MLPs are necessary for the answer-relevant effect of this direct signal (i), and whether their outputs become aligned with the reference-over-contrast answer direction (ii).
Method. (i) Value injection with MLP pinning. We first transfer only the anchor-derived attention signal from the reference run to the contrast run. For each layer , let denote the contribution of the anchor positions to the attention output at for prompt . We compute
where and are the reference and contrast prompts, and add this difference to the attention output at in the contrast run at every layer simultaneously. Thus, the contrast run receives the reference–contrast difference carried specifically along , while the rest of the computation begins from the contrast prompt.
We quantify how much this intervention restores the reference answer using the logit difference
where is the final-position logit of token , and the recovery score
Here, corresponds to the unmodified contrast run and to the reference-run value; we leave the score unclipped. Injection with the MLPs free to respond gives .
To test whether final-position MLPs are responsible for this recovery, we repeat the same injection while fixing every final-position MLP output to the value it had in the unmodified contrast run. The remaining recovery is . We define
the additional recovery when the final-position MLPs are allowed to respond to the injected anchor signal. Each reference–contrast pair therefore yields one value of , , and . If the final-position MLPs mediate the effect of the direct anchor signal, pinning them should remove the recovery, so that is near zero and accounts for most of ; if they do not, should remain close to .
(ii) Direct logit attribution (DLA) (nostalgebraist, 2020). The intervention above tests whether the MLPs mediate the effect of the direct anchor signal. We next ask what answer preference their outputs write into the residual stream. For each layer, we project the final-position MLP output onto the reference-versus-contrast logit direction
where is the unembedding matrix. Let denote the layer- MLP output after the normalization scaling used for attribution. We measure the anchor-dependent change in its contribution to the logit margin as
A positive value means that the MLP output favors over more in the reference run than in the contrast run. Because this comparison uses the unmodified runs, it captures all anchor-dependent information reaching the final-position MLPs, not only information arriving through the direct route; the value-injection experiment above provides the route-specific causal test.
Result. (i) Value injection. Injecting the anchor signal recovers part of the reference–contrast logit margin, and almost all of this recovery passes through the MLPs: – against across models (Figure 6). What the final position reads from the anchor is therefore not yet answer-selective: without the MLPs’ response, it barely moves the logits, and it favors the year-appropriate object only after the final-position MLPs transform it.

Figure 6. The direct route acts through final-position MLPs. Value-injection recovery decomposed into MLP-mediated () and non-MLP () components: pinning the MLPs removes nearly all of the recovery.
(ii) DLA. shows where the final-position MLPs become answer-selective. In all four models, it stays near zero through the lower and middle layers and turns positive over a band of upper layers (Figure 7). The MLPs in this band write an output favoring the reference-year object more in the reference run than in the contrast run, whereas earlier MLPs write little anchor-dependent preference. The anchor-dependent preference is thus written late, in the upper layers, rather than accumulating gradually across the network.

Figure 7. Upper-layer MLPs are answer-selective. Direct logit attribution: the reference-minus-contrast difference in each final-position MLP’s contribution to the logit margin turns positive over a band of upper layers.
Together, (i) and (ii) show how the direct route works: attention carries the anchor signal to the final position (§4.1), the final-position MLPs are necessary for its effect on the answer, and the preference for the year-appropriate object is written in the upper layers. The recovery, however, is under half of the reference–contrast logit margin (), so this route does not determine the answer alone; how it combines with the subject-mediated route is the question of §4.3.3.
Route Combination: Approximately Additive Effects with Query-Dependent Balance
The preceding sections examine each route on its own. Here we ask how their contributions combine: are their effects additive (i), and how do their relative contributions vary across queries (ii)?
Method. We control which positions can read the anchor. At all layers, we block attention to the anchor span from every later position except those we choose. This gives four conditions: both (the subject tokens and the final position can read the anchor), subject only, direct only (the final position), and none. In the unmodified run, full, every position can read it.
For each condition we compute a margin: the logit of the correct object minus that of a competitor. The competitor is the gold object of another year of the same group (the first such year in the dataset), and each object is scored by the maximum over the first tokens of its aliases. The gain of a condition is its margin minus the margin under none. We write , , , and for the mean gains over instances under full, both, subject only, and direct only. If the two routes combine additively, the two single-route gains should sum to the gain when both routes are available:
Result. (i) Additivity. The effects of the two routes are approximately additive. Each route raises the margin when available alone, and the two gains sum closely to the gain when both routes are available (Figure 8a). is within 5% of in Llama, Qwen, and Ministral, with no significant difference after Bonferroni correction, and 7% below it in DeepSeek, a significant difference. The gains also add up within individual instances (Appendix § 8.§ 2.1). These results suggest that the two routes contribute largely in parallel rather than one replacing the other. Together, the two routes account for most of the anchor effect available in the unmodified model: is 0.69–0.90 across models. The remaining effect requires allowing positions other than the subject and final position to attend to the anchor.4

Figure 8. The two routes are parallel channels whose contributions add up, and most instances use both. (a) Gain in the logit margin over the none condition: both routes (left) vs. the two single-route gains, stacked (right). (b) Instances whose margin is raised by at least one route alone, by which routes raise it.
(ii) Variation across queries. Both routes can independently support the prediction on most queries, but their relative contributions vary. Counting a route as contributing when its single-route gain is above zero, 73–82% of the instances with a positive gain from at least one route have positive gains from both (Figure 8b; stricter criteria in Appendix § 8.2.4). Thus, for most queries, either route alone carries answer-relevant anchor information rather than the prediction depending exclusively on one route. The balance between the two gains varies across instances (Appendix § 8.§ 2.2), and the models agree on how it varies. For subject–relation groups analyzed in two models, is positively correlated in all six model pairs (Spearman’s –). This cross-model agreement suggests that the relative contribution of the two routes reflects properties of the queried fact rather than model-specific noise. One such property is the relation. We measure the relative subject-route contribution as . For position held (P39), this share is 64–75% in all four models; for member of sports team (P54), it is 20–46%, indicating a larger direct-route contribution (difference significant in every model, ). The route-specific interventions show the same pattern. Subject-state patching (§4.2) is stronger for P39, raising the logit of by 0.59–0.77 of the reference–contrast difference (peak over layers), compared with 0.29–0.50 for P54. Conversely, value injection (§ 4.3.2) is stronger for P54, with –0.68, compared with 0.16–0.23 for P39. The relation therefore explains part of the variation in the balance between the two routes; which other query properties matter remains open.
Related Work
Mechanistic studies of factual recall. Prior work traces how Transformers retrieve factual associations from subject–relation prompts ([5]), localizing factual information and analyzing how attention, MLPs, and residual-stream representations contribute to predictions ([13, 9]). Assuming a fixed object per subject–relation pair, these studies leave open where competition among objects is resolved; we find it resolved both in the subject representation and at the final position, with the two contributing approximately additively. Our question also differs from whether time is represented ([7]) or how context competes with parametric memory ([10] (Yu et al. 2023)): we investigate how an explicit temporal anchor selects among time-dependent facts.
Temporal factual knowledge. TempLAMA and related work evaluate whether LM predictions change appropriately across timestamps or event times ([4, 20, 1, 24]), assessing temporal knowledge at the output level. Park et al. [16] go further and localize attention heads specialized for temporal knowledge; a single illustrative causal-tracing example there also hints that time is merged into the subject representation. We establish this route across four models, show that the resulting state is largely entity-specific and selectively supports the year-appropriate object (§4.2), and identify a second, direct route and how the two combine (§4.1, §4.3.3). Concurrently, van Adrichem et al. [22] characterize the mechanisms supporting an implicitly assumed current year, whereas we study an explicit anchor that selects among stored facts.
Conclusion
We introduced anchored factual recall, where an explicit condition determines which object is valid for a subject–relation pair, and used temporal anchors to study how LMs implement this selection. Across four open-weight models, we identified a subject-mediated route, in which the anchor resolves the final subject-token representation to the queried year and the relation redistributes last-subject readout across attention heads. Cross-fitted ablation shows that the last-subject contribution of relation-sensitive heads is causally important for the model’s originally correct next-token prediction. We also identified a direct route, in which the final position accesses the anchor directly and final-position MLPs transform this signal into a preference for the time-appropriate object. The two routes act as parallel channels: their contributions add up, most queries use both, and the balance between them varies across queries in the same way across models. Anchored factual recall thus reframes the mechanistic study of factual knowledge as a problem of selecting among competing context-dependent facts, not merely retrieving a fixed association.
Limitations
Our experiments cover one setting: English, one dataset (TempLAMA), one prompt format, and years as anchors. We chose a single controlled setting so that reference and contrast prompts differ only in the anchor, and our claims are about this setting only. We have not tested other prompt formats, other temporal expressions, such as dates or events, or non-temporal anchors, such as locations, so whether the two routes hold for anchored factual recall in general remains open.
The prompt format matters for the findings themselves. The subject-mediated route requires the anchor to precede the subject, because under causal attention the subject can read only what comes before it. We could not vary this order within English without a confound: placing the anchor after the subject while keeping the object last either changes the syntactic construction or yields sentences that are rare in natural English, so a change in route usage could not be separated from a reaction to unusual input. Testing the role of word order therefore needs a language in which both orders are natural, and we state our claims for English only.
All four models have 7–8B parameters. Smaller models answer too few time-varying facts correctly to pass our filter, and larger ones are costly to analyze: our interventions take about 150 times the forward passes of ordinary prediction per example. Whether the two routes, and the way they combine, hold at other scales remains open.
Experimental Details
Dataset
We evaluate on the TempLAMA test set, which contains 34,962 instances and is publicly available at https://github.com/google-research/language/tree/master/language/templama.
Implementation Details
Reference–contrast construction. Most analyses use temporal reference–contrast pairs that differ only in the anchor.
The relation-readout analysis additionally uses two swaps. A length-matched relation swap holds the anchor, subject, and demonstrations fixed while replacing the complete relation expression with that of another relation. Each relation has one fixed replacement with the same number of tokens (e.g., “played for” → “was attending”, “held the position of” → “was the chair of”), so that all token positions stay aligned. A time swap holds the subject, relation, and demonstrations fixed while replacing only the anchor year.
These swaps are used for the descriptive readout analysis and for cross-fitted head selection. The causal ablation itself is evaluated on the original, unmodified, correctly answered prompt.
Few-shot prompting. All prompts use a fixed shot format. Demonstrations are sampled at random (fixed seed) from instances whose year, subject, and relation all differ from those of the target, so that no demonstration leaks the target answer or its exact query pattern. Demonstrations are drawn independently per target instance.
Intervention scope. Attention knockout is applied over a centered window of 9 adjacent layers: an edge centered at layer is removed at that layer and its four neighbors on each side, following the windowed-knockout practice of Geva et al. [5].
The reported curve is indexed by the window center. Subject-state activation patching is applied at a single layer at a time, yielding one value per layer. Value injection is applied at all layers simultaneously, each layer receiving its own reference-minus-contrast difference, and MLP pinning to all final-position MLPs; the recovery decomposition therefore yields one value per pair. The route-isolation analysis (§4.3.3) and the anchor blocking of the shared-year-offset control (§B.1) apply attention knockout at all layers. The relation-sensitive-head ablation is the exception: it jointly intervenes on a cross-fitted set of layer–head pairs across multiple layers, as detailed below.
Cross-fitted relation-sensitive-head ablation. We partition subject–relation groups into five folds. For each held-out fold, head selection and control construction use only the examples in the remaining four discovery folds. We average the relation-sensitivity score over the discovery examples and rank all finite layer–head scores across the analyzed layers. We target the top 10% of layer–head pairs, with a minimum of eight selected heads. When at least eight positively scored candidates are available, selection is restricted to positive scores. To preserve a disjoint same-layer pool for the random control, at most half of the heads in any layer may be selected. Time-sensitive heads are selected using the reversed score and are matched to the relation-sensitive set in their per-layer head counts. For each fold, we additionally construct 20 contribution-matched random head sets. Each random set contains the same number of heads in each layer as the relation-sensitive set and excludes the selected relation-sensitive heads. For each selected head, a random match is sampled without replacement from a pool of up to four unselected heads in the same layer whose mean unablated contribution norms on the discovery folds are closest to that of the selected head. The reported matched-random effect is the mean over these 20 sets. Evaluation is performed on the original prompts in the held-out fold. For every selected layer–head pair, we recompute its current contribution from the final subject token to the final query position during the modified forward pass, (attention weight, output projection, and value vector of head in layer ; Eq. (3)), and subtract it from the attention output. All selected pairs are ablated jointly in the same forward pass, so interventions in later layers operate on the state produced by the earlier interventions. We report the top-1 flip rate defined in §4.3.1. Uncertainty is estimated with 2,000 percentile bootstrap resamples of subject–relation groups, yielding 95% confidence intervals. The same cluster bootstrap is applied to the condition means and to the paired differences between relation-sensitive and contribution-matched random ablation and between relation-sensitive and time-sensitive ablation.
Computational resources. All experiments were conducted on a single NVIDIA A40 GPU (48GB).
Additional Analyses
Control: The Effect Is Not a Shared Year Offset
Method. Suppose, for each pair of years, the anchor adds an entity-independent vector to the subject representation; we call this a shared year offset. The reference-year state would then be reproduced by adding the offset to the contrast-year state:
where and are the reference and contrast years of comparison and does not depend on the entity.
For each comparison, we estimate as the mean difference between the reference-year and contrast-year final subject-token states of other subjects with the same pair of years. Subjects are assigned to five folds and the estimate for a comparison uses only subjects from the other folds, with states first averaged within subject. We then run the contrast prompt and set the final subject-token state at layer either to the true reference-year state or to the contrast-year state plus the estimated offset. In both cases, attention to the anchor from the subject, the final position, and the remaining query positions is blocked in all layers, so that neither condition can recover the answer by reading the anchor elsewhere. We report the recovery of the logit difference between and , measured relative to the same run with the contrast-year state and normalized by the difference between the unmodified reference and contrast runs.
Result. A shared year offset reproduces only a small part of the effect of the reference-year state (Figure 9). At the layer where the two differ most, the true state recovers 0.37, 0.30, 0.38, and 0.24 of the logit difference in Llama, Qwen, Ministral, and DeepSeek, whereas the offset recovers 0.08, 0.08, 0.10, and −0.01; the difference is significant in every model. In each model, the offset thus accounts for at most about a quarter of the effect (21%, 25%, 26%, and 0%). The true state is most effective at the layers where subject-state patching peaks (Figure 3). The selective effect observed under patching therefore primarily reflects a state specific to the entity, rather than a year component common to all entities.

Figure 9. The effect of the anchored subject state is not a shared year offset. Recovery of the logit difference between and when the final subject-token state of the contrast run is set to the reference-year state, or to the contrast-year state plus the mean reference-minus-contrast difference of other subjects with the same pair of years. No query position can attend to the anchor in either condition. Shaded bands are 95% bootstrap intervals over subject–relation groups.
Route Usage
Additivity of the two routes
The two routes together account for , 0.79, 0.90, and 0.69 of the anchor’s effect in Llama, Qwen, Ministral, and DeepSeek.
We test (§4.3.3) with a bootstrap over subject–relation groups. The uncorrected -values are 0.049 for Llama, 0.39 for Qwen, 0.10 for Ministral, and for DeepSeek; with a Bonferroni correction for the four models (), only DeepSeek differs significantly.
Within instances, the deviation has median −0.01, 0.00, 0.00, and −0.02 and interquartile range [−0.10, 0.04], [−0.05, 0.05], [−0.03, 0.07], and [−0.14, 0.02] for Llama, Qwen, Ministral, and DeepSeek. Its absolute value has a median of 0.05–0.07 and is below 0.25 in 85%, 91%, 82%, and 81% of the instances. For comparison, has a median of 0.34–0.81 and interquartile range [0.18, 1.67], [0.00, 0.96], [0.02, 1.30], and [0.00, 1.37].
Route usage across instances
We compare, for each reference–contrast comparison , how much each route contributes according to the route-specific interventions of §4.2 and § 4.3.2. For the subject-mediated route, is the peak increase, over layers, in the logit of under subject-state patching, normalized by the difference in that logit between the reference and contrast runs. For the direct route, is the MLP-mediated recovery of value injection. The two scores are negatively correlated across instances in all four models (Figure 10): Spearman’s for Llama, −0.62 for Qwen, −0.42 for Ministral, and −0.51 for DeepSeek, all . The share of the two routes thus varies across instances. This does not conflict with additivity: and are both fractions of the same reference–contrast difference, so when the two contributions add up to a roughly fixed total, a larger share of one implies a smaller share of the other. This describes how the usage of the two routes varies across instances; it does not imply that one route increases its contribution when the other is weakened, which we test next.

Figure 10. Route usage varies across instances. Subject-mediated recovery () vs. direct-route recovery () per reference–contrast comparison.
Does weakening one route strengthen the other?
We also tested the interaction between the routes by intervention. We replace the states of the subject tokens, at all layers, with the states they have when no position can read the anchor, and measure the MLP-mediated recovery of value injection (§ 4.3.2). is the recovery with the final-position MLPs free minus the recovery with them pinned to their contrast-run outputs; it is therefore also defined without value injection, where it measures the part of the replacement’s own effect that passes through these MLPs. The replacement raises , but the same replacement without value injection raises it by the same amount (0.17–0.30 across models), so the increase is caused by the replacement itself, not by a stronger direct route. After subtracting this control, the change in the direct route’s contribution is −0.015 [−0.031, 0.002] in Llama, −0.045 [−0.103, 0.003] in Qwen, 0.010 [−0.012, 0.033] in Ministral, and 0.004 [−0.052, 0.092] in DeepSeek (95% bootstrap intervals over subject–relation groups). Weakening the subject-mediated route therefore does not strengthen the direct route, consistent with the additivity observed under route isolation (§4.3.3).
Route usage by instance and by relation
Figure 8b counts a route as contributing when its gain is above zero. If we require a gain above 0.1–0.3 instead, both routes raise the margin in 42–66% of the instances raised by at least one route. In every model, instances raised by both routes are more frequent than they would be if the two routes acted on instances independently of each other (by 4–9 percentage points at a threshold of zero; all 95% intervals above zero).
The share of the subject-mediated route, , differs between position held (P39) and member of sports team (P54) by 0.29–0.44 across models (all ; bootstrap over subjects, Bonferroni-corrected for the four models). Among the relations with fewer instances, the share is 14–34% for owned by (P127); three models, below 50% in each with a 95% bootstrap interval over instances) and about 50% for employer (P108).
References
- [1]Wenhu Chen, Xinyi Wang, and William Yang Wang. 2021. A dataset for answering time-sensitive questions. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).arxiv.org/abs/2108.06314
- [2]Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493–8502.
- [3]DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. Preprint, arXiv:2501.12948.DOI
- [4]Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. 2022. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10:257–273.
- [5]Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12216–12235, Singapore. Association for Computational Linguistics.arxiv.org/abs/2304.14767
- [6]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Koreney, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Diedue Hupke, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Is han Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, J enya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnston, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuy Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Goket, Virginie Do, Vish Vogeti, Vitor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Ayaushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changan Wang, Changyku Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgencel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspergren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Sapkale, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanachandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuq iang Zhang, Shuq iang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijay Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaochen Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. 2024. The Llama 3 herd of models. Preprint, arXiv:2407.21783.
- [7]Wes Gurnee and Max Tegmark. 2024. Language models represent space and time. In The Twelfth International Conference on Learning Representations.arxiv.org/abs/2310.02207
- [8]David Herel, Vojtech Bartek, Jiri Jirak, and Tomas Mikolov. 2025. Time awareness in large language models: Benchmarking fact recall across time. Preprint, arXiv:2409.13338.arxiv.org/abs/2409.13338
- [9]Evan Hernandez, Arnab Sen Sharma, Tal Haklay, Kevin Meng, Martin Wattenberg, Jacob Andreas, Yonatan Belinkov, and David Bau. 2024. Linearity of relation decoding in transformer language models. In Proceedings of the 2024 International Conference on Learning Representations.arxiv.org/abs/2308.09124
- [10]Gaotang Li, Yuzhong Chen, and Hanghang Tong. 2025. Taming knowledge conflicts in language models. In Forty-second International Conference on Machine Learning.arxiv.org/abs/2503.10996
- [13]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. Advances in neural information processing systems, 35:17359–17372.arxiv.org/abs/2202.05262
- [14]Ayana Niwa, Masahiro Kaneko, and Kentaro Inui. 2025. Rectifying belief space via unlearning to harness LLMs’ reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 25060–25075.
- [16]Yein Park, Chanwoong Yoon, Jungwoo Park, Minbyul Jeong, and Jaewoo Kang. 2025. Does time have its place? temporal heads: Where language models recall time-specific information. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16616–16643, Vienna, Austria. Association for Computational Linguistics.arxiv.org/abs/2502.14258
- [17]Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 2463–2473.arxiv.org/abs/1909.01066
- [18]Qwen, ;, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115.
- [19]Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 5418–5426.arxiv.org/abs/2002.08910
- [20]Jungbin Son and Alice Oh. 2023. Time-aware representation learning for time-sensitive question answering. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 70–77, Singapore. Association for Computational Linguistics.arxiv.org/abs/2310.12585
- [21]Qingyu Tan, Hwee Tou Ng, and Lidong Bing. 2023. Towards benchmarking and improving the temporal reasoning capability of large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14820–14835, Toronto, Canada. Association for Computational Linguistics.
- [22]Suze van Adrichem, Aditi Bhaskar, Diyi Yang, Christopher Potts, and Jing Huang. 2026. Do language models consistently encode the current year? Preprint, arXiv:2608.15507.
- [23]Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The International Conference on Learning Representations.arxiv.org/abs/2211.00593
- [24]Yifan Wei, Yisong Su, Huanhuan Ma, Xiaoyan Yu, Fangyu Lei, Yuanzhe Zhang, Jun Zhao, and Kang Liu. 2023. MenatQA: A new dataset for testing the temporal comprehension and reasoning abilities of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1434–1447.arxiv.org/abs/2310.05157