WorksComputer Science
Reading or Remembering? Measuring Temporal Contamination in Language Models
Abstract
Language models are now used to read central-bank text for economic forecasting, but most were trained on text written after the documents they read. A model reading a 2021 FOMC statement may already know that inflation rose in 2022, so part of its apparent skill can be memory of the outcome, a temporal form of test-set contamination. We measure this memory on FOMC statements and minutes from 2000 to 2026 by comparing modern open models with chronologically consistent models, which are trained only on text from before a fixed date. Asked about past GDP outcomes given only a date, models that cannot know the answer still reach an AUC of up to 0.64 from priors on dates and eras, so chance is the wrong baseline. Against these memory-free models on the same questions, most modern models recall which years had strong growth. Membership-inference scores sort documents by era and document type rather than by whether a model saw them, as exact counts in an open pretraining corpus show. In real-time nowcasts, every lookahead premium we estimate on US GDP is smaller than the forecast comparison can detect at macroeconomic sample sizes, and none on GDP or inflation survives correction for multiple testing. Lookahead audits should therefore measure memory directly against memory-free models on the same items.
Introduction
Language models are now used to read central-bank communication for forecasters, and most of the models in use were trained on text written after the period they are asked to read. A model that has seen the commentary of 2021 and 2022 does not need to infer from a 2021 FOMC statement that inflation is about to rise; it can remember that it did. In machine-learning terms this is temporal contamination, a form of test-set contamination that runs through time: the outcomes a model is evaluated on appear in text written after its inputs. Lookahead of this kind has been documented in financial and economic prediction [21, 19, 15] (Sarkar and Vafa 2024; Lopez-Lira et al. 2025), and contamination by release date has been found in benchmark results (Roberts et al., 2024). Chronologically consistent models, trained only on text available before a fixed date, now exist to remove it [27] (Yan et al. 2026).
We want to measure, for a forecasting task, how much of a model’s apparent skill is memory, and whether that memory reaches the forecast when the forecast is made as it would be in practice: in real time, on the data available that day, and scored against the first published estimate. The usual checks are poorly suited to this. Scoring a model against chance assumes that a model which has not seen the outcome guesses at random. Membership-inference scores assume that the texts a model saw and the texts it did not see differ only in that respect, an assumption that fails when the two come from different periods [14]. And a comparison of forecast accuracy with and without the suspect model needs more data than a macroeconomic series provides.
We build such a measurement for the FOMC’s statements and minutes from 2000 to 2026 and for real-time nowcasts of US GDP and CPI growth from 2012 to 2024, and we score each part of it against a reference other than chance. Date-only probes give a model a date and ask about an outcome it could know only from later text. The reference is a set of readers that cannot know the outcome, scored on the same items, and the inference allows for outcomes that cluster in time and for priors on calendar words. A ladder of fixed check-points from one chronologically consistent family asks whether a model does better on outcomes its training could have contained than on outcomes it could not. Membership scores ask whether a model saw a document; we check them against checkpoints with known cutoffs and against exact counts in one model’s indexed training corpus. Finally, a real-time nowcaster measures what the readings are worth, with a reader behind a temporal gate as the control, two hard-data baselines, the professional nowcasts, and a power analysis.
The first three of our six findings are positive results. The other three mark what these data cannot resolve, and we report them with intervals and, for the forecast, the smallest effect the test could detect.
Readers that cannot know the outcome do not score 0.5. On the GDP probe they reach AUCs up to about 0.64 from priors on dates and eras, so every claim of memory below compares a model with these readers on the same items.
The modern models remember outcomes that the memory-free readers cannot know. Against the gated reader, six of the seven score higher after adjusting for multiple comparisons. A date retrieves which years grew strongly.
Membership scores track era and document type rather than membership. Exact span counts in OLMo-2’s indexed corpus date the end of its FOMC text to 2 November 2022, a year before its model card’s cutoff, and within a year and a document type Min-K% does not separate the documents found from those not found.
The probes cannot resolve finer distinctions. Within a year there are too few mixed outcomes to tell whether a model remembers individual quarters. The modern models’ margins over ChronoGPT-2024, a small chronologically consistent model whose training also runs past every outcome, do not survive adjustment for multiple comparisons, so whether the larger models remember more than the small one stays open. The checkpoint ladder’s intervals all include zero and are wider than that gap, which leaves open whether the small models remember at all.
The forecast comparison lacks the power to detect a lookahead premium. The leaky readers’ GDP premiums over the gated reader are a few percent of RMSE, below the 8.8 to 13.1 percent the test detects with 80% power, and their sign depends on whether 2020 or 2021 is left out. They are also measured on a GDP nowcaster that does not beat a running mean of past growth.
Text adds no skill that survives correction for multiple testing, either to the real-time nowcaster or to the SPF, GDPNow and New York Fed nowcasts.
A lookahead audit of a forecasting system should therefore rest neither on forecast accuracy, which at macroeconomic sample sizes cannot resolve premiums of a few percent, nor on membership scores compared across eras, which confound membership with time, but on date-only probes scored against memory-free readers on the same items, and on exact corpus counts where a corpus is indexed. We release the corpus, every score and every script.
Related work
Memorisation and contamination detection
Memorisation grows log-linearly with model capacity, duplication and prompt length ([4]). Detection methods ask whether a text was in the training corpus: Min-K% Prob averages the log-probabilities of a text’s least likely tokens (Shi et al., 2024), Min-K%++ normalises them by the next-token distribution (Zhang et al., 2025), and guided prompts flag contamination when a completion matches a reference instance ([22]). Contamination must be measured benchmark by benchmark (Sainz et al., 2023), and answers to Stack Overflow questions are often collages of memorised text whether or not the question predates training ([6]); Xu et al. (2024) and Ravaut et al. (2025) survey the methods. Duan et al. [14] find that membership inference barely beats chance on Pile-trained models and trace reported successes to members and non-members drawn from different periods. Our probes face the same confound, because answer priors track eras. We ask whether a model knows an outcome that postdates a document, and score each leaky reader against a gated reader and a persistence rule on the same items rather than against 0.5.
Temporal generalisation and knowledge cutoffs
Perplexity on text from after the training period worsens with distance from it (Lazaridou et al., 2021); conditioning on timestamps improves recall of time-dependent facts [12]; GPT pass rates on Codeforces and Project Euler problems reveal contamination by release date (Roberts et al., 2024); and effective cutoffs often differ from reported ones [7]. We instead use known cutoffs as the treatment: within one chronologically consistent family, a ladder of fixed checkpoints shares architecture, size, tokenizer and recipe, and each checkpoint’s cutoff divides the outcomes it could have seen from those it could not.
Chronologically consistent language models StoriesLM continues each checkpoint on the following year’s news from 1900 (Sarkar, 2024), and Time Machine GPT releases point-in-time models [13]. ChronoBERT and ChronoGPT reach Sharpe ratios in next-day return prediction comparable to a much larger Llama model [27], with instruction-tuned versions following [26]. DatedGPT trains twelve 1.3B-parameter models with annual cutoffs from 2013 to 2024 and finds a return-prediction premium of 26.4 basis points per standard deviation for models whose training covers the outcome period (Yan et al., 2026). These papers evaluate on asset returns. We use ChronoGPT-Instruct as a gate inside a macroeconomic nowcaster, use fixed checkpoints of ChronoGPT and DatedGPT as ungated controls, and measure the premium on 150 real-time origins over 50 quarters together with the smallest premium the test can detect.
Lookahead bias in economic and financial uses of language models Lookahead appears when models predict risk factors from earnings calls and election winners from biographies (Sarkar and Vafa, 2024); anonymising headlines improves GPT sentiment trading in-sample, so distraction can outweigh lookahead [21]; models recall economic series from before their cutoff even when told to respect a historical boundary (Lopez-Lira et al., 2025); and the probability of a non-unknown answer given only a firm, ticker and date collapses after the cutoff [19]. Telling a model to ignore what it knows leaves a 52% gap to true ignorance (Li et al., 2026), no modern model passes the first fake-date test of Eliseev and Seleznev [15] for macroeconomic forecasts, and the PaLM inflation forecasts of Faria-e Castro and Leibovici [17] are in-sample for 2019–23. MACROCAST avoids leakage by pretraining on synthetic series [5], and Ludwig et al. (2026) state the general no-leakage condition. Most of this evidence concerns firm-level outcomes or a model forecasting a series directly. We study a model reading central-bank text as one input to a real-time nowcaster, price lookahead as a change in RMSE against first release, and find that the memory a probe detects does not appear as resolvable forecast skill.
Text in macroeconomic nowcasting News text adds information that hard indicators miss [30, 16] (Thorsrud 2020). Central-bank text has been read with topic models [23, 24], with machine learning on staff documents (Aruoba and Drechsel, forthcoming), and with encoders trained further on central-bank text [18]. Our bridge-factor nowcaster follows the real-time nowcasting literature [20, 2, 28], in which data vintages change results [32] (Crossshore 2011) and model nowcasts beat the SPF on headline inflation [31]. We drop the two pandemic quarters, which is acceptable for estimation though it understates forecast uncertainty (Lenza and Primiceri, 2022). These text measures come from topic models, dictionaries or models trained on a chosen corpus; a general-purpose language model brings a corpus that extends past the forecast period, so we make the reader’s training cutoff the variable under test.
Setup
Figure 1 shows the pipeline at one forecast origin and Figure 2 places every window and checkpoint on one calendar.

Figure 1. The pipeline at one forecast origin . A temporal gate routes each FOMC document to the ChronoGPT checkpoint whose training ended the year before it; ungated readers see every document. An output gate turns next-token label scores into a five-dimensional policy signal and abstains when three wordings of a question disagree in sign. The same readers answer date-only memory probes. At each origin the nowcaster takes one vintage of the Philadelphia Fed Real-Time Data Set (RTDSM), the eleven indicators as published by that date. It fills the unpublished months, the ragged edge, with a second-order autoregression on each indicator’s monthly changes (AR(2) fill) and reduces the target’s block of indicators to three principal components (block PCA). A ridge regression of the target on these factors and any text signal, the ridge bridge, gives the nowcast . Every nowcast is scored against the first release, the first published estimate of the quarter.

Figure 2. The study on one calendar. ChronoGPT checkpoints end on 31 December of each year from 1999 to 2024; a document is read by the checkpoint that ends the year before it. The modern readers were released in 2024, so their training text covers most of the test window. Four fixed ChronoGPT checkpoints, and three DatedGPT checkpoints on the probes, form the ungated ladder. Specification selection uses only origins before the test window.
Corpus The corpus is every FOMC statement and set of minutes released from February 2000 to January 2026, 429 documents (221 statements and 208 minutes). The document list and the dates come from the Federal Reserve’s own index: a statement is dated by its press release, a set of minutes by its release (about three weeks after the meeting since 2005, and after the following meeting before that). Every reader sees the same excerpts, cut by one fixed rule: a statement runs from the release header to the vote (or to the page footer when there is no recorded vote); minutes contribute the first 2,500 characters of the discussion of the outlook and the first 2,500 of the discussion of policy. Excerpts are at most 5,000 characters. Appendix § 7.§ 3 documents the data cleaning, including the removal of one leak.
Real-time data and targets Monthly indicators come from the Philadelphia Fed Real-Time Data Set (RTDSM) [11], which keeps every vintage of each series, that is, the series as it stood on a given date before later revisions. We use eleven series with monthly vintages, covering industrial production, capacity utilisation, payroll employment, hours, housing starts, three real consumption series and three price indices. Each forecast origin uses exactly one vintage.
tage and is dated the last day of the vintage month, by which time every release in it is public. Targets are first-release quarter-on-quarter growth at an annual rate, , for real GDP and for the quarterly average of headline CPI; the first release is the first published estimate of a quarter, before any revision. The test window has 150 monthly origins over 50 quarters from 2012 to 2024; 2020 Q2 and Q3 are excluded from estimation and scoring.
Readers Table 4 in Appendix § 7.§ 1 lists the readers. The gated reader is the ChronoGPT-Instruct family [27, 26], about 1.55 billion parameters, with a checkpoint ending on 31 December of every year from 1999 to 2024; a document dated in year goes to the checkpoint ending in . Four of these checkpoints are also run ungated on every document: those ending in 2011, 2015, 2019 and 2024. They form a ladder: each has the same architecture and recipe as the gated reader, and each could have seen outcomes up to its own cutoff and no later. Seven modern open models are run ungated. A second chronologically consistent family, DatedGPT (Yan et al., 2026), about 1.3 billion parameters with checkpoints ending in each year from 2013 to 2024, answers the probes only, both gated and as three fixed checkpoints (2016, 2020, 2024); its gated reader has no checkpoint for the 2000–2011 training window of the nowcaster, and we did not run its fixed checkpoints on the documents. A dictionary reader is reported for reference; its phrase lists were written with knowledge of later Fed language, so it is not memory-free.
| Reader | Parameters | Training text ends | Role |
| ChronoGPT-Instruct, 26 checkpoints | 1.55B | 31 Dec of each year, 1999–2024 | gated |
| ChronoGPT-Instruct, fixed checkpoint | 1.55B | 31 Dec 2011, 2015, 2019, 2024 | ungated ladder |
| DatedGPT-Instruct, 12 checkpoints | 1.34B | 31 Dec of each year, 2013–2024 | gated, probes only |
| DatedGPT-Instruct, fixed checkpoint | 1.34B | 31 Dec 2016, 2020, 2024 | ungated ladder, probes only |
| Qwen2.5-Instruct (Qwen Team, 2024) | 1.5B, 3B, 7B, 14B | undocumented; released Sep 2024 | ungated |
| OLMo-2-1124-Instruct (Team OLMo et al., 2024) | 7B, 13B | Dec 2023 (base model card); FOMC text to November 2022 | ungated |
| Mistral-7B-Instruct-v0.3 (Jiang et al., [29]) | 7B | undocumented; released May 2024 | ungated |
| Dictionary of policy phrases | none | phrases written after the sample | reference |
Table 4. Readers. Release dates are the public Hugging Face release of the checkpoint used. For OLMo-2 we measure in its indexed stage-1 pretraining corpus when FOMC text stops appearing (§ 4); the Instruct model card states no cutoff.
Reading a document For an excerpt , a dimension (policy stance, inflation risk, growth risk, uncertainty, forward guidance) and one of three wordings of the question, the reader is asked a question with two one-word answers, such as whether policy is more or less restrictive than neutral (Appendix § 7.§ 2). We read the next-token probabilities of the two answers instead of generating text. An answer’s probability sums its single-token forms, and the raw score is .
The output gate Two corrections precede any use of a score. Each reader’s score on a content-free input, the same prompt with the document replaced by “N/A”, is subtracted for the same reader, checkpoint, dimension and wording ((Zhao et al., 2021)). The calibrated scores of the three wordings must then agree in sign; if they do, the signal for dimension is , and if not, the gate abstains, sets it to zero and logs the three scores. The gate is there to flag readings a model cannot give consistently; § 5.1 and Appendix § 8 report how it changes accuracy.
Measuring memory
We measure memory three ways, each with its own null, before asking what memory is worth in a forecast.
Date-only probes A reader is given a date and nothing else, a two-answer form of the date-only recall queries of Gao et al. [19]. In the decision probe it is told that the FOMC changed its target on that date and asked whether it raised or lowered it (74 changes, 2000–2025). In the GDP probe it is asked whether first-release real GDP growth in a given quarter was above or below 2 percent at an annual rate (100 quarters, 2000–2024). A reader that has not seen the outcome can only guess, but a guess need not score an AUC of 0.5: answer priors can line up with eras in which one outcome was common, and a gated reader’s checkpoint changes every January. The null is therefore set by two other readers scored on the same items: the gated reader and a persistence rule that uses only what was public before the item’s year began.
Probe items cluster in time, so inference respects the clustering: -values come from rotating the labels against the scores in date order (a circular shift, which keeps the dependence in both series), and intervals for AUC differences from resampling calendar years. We also compare quarters with the same quarter number across years, tested by rotating the years of the outcomes against the scores, and quarters within the same year, which removes most between-era comparisons (a fourth quarter is dated by its release in the following January, so it is read by the next gated checkpoint).
The checkpoint ladder The gated reader mixes 26 checkpoints, and its 2024 sibling has seen more text in total, so the gated-against-ungated contrast differs in more than lookahead. The ladder separates the two. Each fixed checkpoint, ending in 2011, 2015, 2019 or 2024, reads every document and answers every probe. For each, an item is seen if its outcome was public by the checkpoint’s cutoff and unseen otherwise. Memory predicts that a checkpoint does better on what it could have seen than on what it could not; general capability, recency and checkpoint mixing do not predict that split within one checkpoint.
Membership Probes ask whether a model knows an outcome; membership scores ask whether it saw a text. For every excerpt and reader we compute Min-% Prob, the mean of the lowest percent of token log-probabilities of the excerpt read as plain text (Shi et al., 2024), and Min-%++, which standardises each token’s log-probability by the model’s own next-token distribution (Zhang et al., 2025). Membership scores are known to fail when members and non-members come from different periods [14], as ours do. For each fixed ChronoGPT checkpoint, documents released after its cutoff are certainly not members and documents released before it possibly are; for OLMo-2, whose pretraining corpus is indexed, we count exact spans of each excerpt in the corpus.
Results: what the readers remember
Memory-free readers do not score 0.5 Table 1 gives the GDP probe for every reader. The readers that cannot know an outcome are the right null, and they are not at chance: the gated ChronoGPT reader scores 0.48, the persistence rule 0.45, and checkpoints that cannot have seen most of their items score as high as 0.64 (ChronoGPT with a 2011 cutoff, which scores 0.60 on the 2012–2024 items it cannot know). AUCs up to about 0.64 therefore arise from priors on dates and eras, and a modern reader’s AUC is evidence of memory only where it exceeds these readers on the same items, with intervals that respect how outcomes cluster in years.
| Reader | AUC | against own gated reader | against ChronoGPT-2024 | AUC | rotation | |
| ChronoGPT, gated | 100 | 0.48 | n/a | −0.15 [−0.34, +0.02] | 0.48 | 0.680 |
| ChronoGPT, cutoff 2011 | 100 | 0.64 | +0.15 [−0.01, +0.32] | +0.00 [−0.11, +0.11] | 0.64 | 0.120 |
| ChronoGPT, cutoff 2015 | 100 | 0.62 | +0.14 [−0.00, +0.30] | −0.01 [−0.17, +0.15] | 0.58 | 0.040 |
| ChronoGPT, cutoff 2019 | 100 | 0.51 | +0.03 [−0.15, +0.24] | −0.12 [−0.29, +0.04] | 0.43 | 0.840 |
| ChronoGPT, cutoff 2024 | 100 | 0.64 | +0.15 [−0.03, +0.34] | n/a | 0.56 | 0.200 |
| DatedGPT, gated | 45 | 0.54 | n/a | −0.09 [−0.46, +0.19] | 0.56 | 0.333 |
| DatedGPT, cutoff 2016 | 100 | 0.59 | +0.12 [−0.06, +0.34] | −0.04 [−0.17, +0.09] | 0.57 | 0.120 |
| DatedGPT, cutoff 2020 | 100 | 0.36 | −0.23 [−0.44, +0.04] | −0.27 [−0.48, −0.07] | 0.40 | 1.000 |
| DatedGPT, cutoff 2024 | 100 | 0.44 | −0.06 [−0.27, +0.13] | −0.20 [−0.41, +0.02] | 0.45 | 0.960 |
| Qwen2.5-1.5B | 100 | 0.66 | +0.17 [+0.00, +0.33] | +0.02 [−0.14, +0.16] | 0.70 | 0.040 |
| Qwen2.5-3B | 100 | 0.76 | +0.28 [+0.13, +0.41] | +0.12 [+0.01, +0.24] | 0.77 | 0.040 |
| Qwen2.5-7B | 100 | 0.79 | +0.31 [+0.14, +0.47] | +0.16 [−0.02, +0.31] | 0.81 | 0.040 |
| Qwen2.5-14B | 100 | 0.79 | +0.31 [+0.15, +0.47] | +0.16 [+0.04, +0.28] | 0.79 | 0.040 |
| OLMo-2-7B | 100 | 0.78 | +0.30 [+0.15, +0.45] | +0.14 [+0.01, +0.27] | 0.78 | 0.040 |
| OLMo-2-13B | 100 | 0.75 | +0.27 [+0.11, +0.41] | +0.11 [−0.05, +0.26] | 0.73 | 0.040 |
| Mistral-7B | 100 | 0.83 | +0.35 [+0.20, +0.50] | +0.19 [+0.04, +0.32] | 0.82 | 0.040 |
| Persistence rule | 100 | 0.45 | −0.04 [−0.21, +0.14] | n/a | 0.44 | 0.760 |
Table 1. The GDP probe: was first-release growth in a quarter above 2 percent, given only the quarter? AUC of the raw label score. Differences against the reader’s own gated family (ChronoGPT or DatedGPT) and against ChronoGPT-2024, on the same items, with 95% intervals from resampling years. “Same quarter word”: pairs of quarters with the same quarter number; from rotating the years of the outcomes against the scores, whose floor is 1/25. DatedGPT’s gated reader covers items from 2014 on.
The modern readers exceed them Against the gated ChronoGPT reader on the same items, the intervals of six of the seven modern readers exclude zero after adjusting for seven comparisons. The sharper comparison is with ChronoGPT-2024, a 1.55-billion-parameter reader whose training also runs past every outcome: the modern readers, of 1.5 to 14 billion parameters, score +0.02 to +0.19 higher, the 95% intervals of four of the seven exclude zero (Qwen2.5-3B, Qwen2.5-14B, OLMo-2-7B and Mistral-7B), and after adjusting for seven comparisons none do. Qwen2.5-1.5B, the one reader of comparable size, scores +0.02 [−0.14, +0.16]. On pairs of quarters with the same quarter word across years, which removes any prior on the words “first” to “fourth”, the modern readers score 0.70 to 0.82; rotating the years of the outcomes against the scores, which keeps how outcomes cluster across years, puts every one of them at the smallest the 25 years allow (0.040), but also puts ChronoGPT-2015 there. Within a year, after each reader’s mean answer for each quarter word is removed, the modern readers score 0.53 to 0.62 (smallest 0.075); few years mix outcomes, so quarter-level memory is not resolvable (Appendix § 8).
The ladder cannot resolve memory of this size For the five checkpoints with both kinds of item, the seen-minus-unseen difference in AUC runs from −0.19 to +0.12, two are positive, and all five intervals include zero (Table 2, Figure 4 in Appendix § 8). The seen and unseen items come from different eras, so the cleaner contrast holds the items fixed and compares consecutive checkpoints on the outcomes released between their cutoffs: two of the five contrasts are positive, on 16 to 20 items each, with intervals from 0.61 to 1.08 wide (Table 8). The ladder cannot detect memory of the size the modern readers show over ChronoGPT-2024, so it neither confirms nor excludes memory in the small chronologically consistent models.
| Checkpoint | seen () | unseen () | seen minus unseen | seen (origins) | unseen (origins) |
| ChronoGPT, cutoff 2011 | 0.71 [0.54, 0.86] (47) | 0.59 [0.44, 0.73] (53) | +0.12 [−0.13, +0.34] | n/a (0) | +0.6 [−6.7, +10.2] (150) |
| ChronoGPT, cutoff 2015 | 0.60 [0.44, 0.78] (63) | 0.66 [0.52, 0.84] (37) | −0.06 [−0.33, +0.15] | −6.6 [−20.0, +4.1] (45) | +0.1 [−6.8, +10.2] (105) |
| ChronoGPT, cutoff 2019 | 0.51 [0.38, 0.66] (79) | 0.70 [0.53, 0.89] (21) | −0.19 [−0.46, +0.04] | −1.5 [−17.1, +11.1] (93) | +0.2 [−6.6, +10.4] (57) |
| ChronoGPT, cutoff 2024 | 0.64 [0.53, 0.76] (99) | n/a (1) | n/a | +3.4 [−2.9, +12.3] (147) | n/a (3) |
| DatedGPT, cutoff 2016 | 0.54 [0.39, 0.71] (67) | 0.66 [0.50, 0.86] (33) | −0.12 [−0.39, +0.11] | not run | not run |
| DatedGPT, cutoff 2020 | 0.39 [0.24, 0.54] (83) | 0.33 [0.00, 0.70] (17) | +0.06 [−0.31, +0.45] | not run | not run |
| DatedGPT, cutoff 2024 | 0.44 [0.32, 0.57] (99) | n/a (1) | n/a | not run | not run |
Table 2. The checkpoint ladder. “Seen”: probe items whose outcome was public by the checkpoint’s cutoff, and nowcast quarters whose first release was; “unseen”: the rest. Intervals from resampling years (probes) and quarters (nowcasts). The DatedGPT checkpoints answered the probes only.

Figure 4. The ChronoGPT ladder: each checkpoint on outcomes it could have seen (filled) and could not (open), GDP probe AUC (left) and GDP nowcast premium over the gated reader with 95% quarter-block intervals (right). Memory predicts filled above open; no difference is resolvable.
Table 8: Same items, consecutive checkpoints: GDP probe items whose outcome was released after the earlier checkpoint’s cutoff and by the later one’s. Memory predicts the later checkpoint does better. Intervals from resampling years.
| Earlier checkpoint | Later checkpoint | items | earlier AUC | later AUC | later minus earlier [95%] |
| ChronoGPT, cutoff 2011 | ChronoGPT, cutoff 2015 | 16 | 0.47 | 0.62 | +0.15 [−0.20, +0.41] |
| ChronoGPT, cutoff 2015 | ChronoGPT, cutoff 2019 | 16 | 0.57 | 0.30 | −0.27 [−0.83, +0.09] |
| ChronoGPT, cutoff 2019 | ChronoGPT, cutoff 2024 | 20 | 0.68 | 0.77 | +0.10 [−0.10, +0.51] |
| DatedGPT, cutoff 2016 | DatedGPT, cutoff 2020 | 16 | 0.51 | 0.18 | −0.33 [−1.00, +0.08] |
| DatedGPT, cutoff 2020 | DatedGPT, cutoff 2024 | 16 | 0.35 | 0.33 | −0.02 [−0.50, +0.33] |
Table 8.
The decision probe fails its null Readers that cannot know the outcome score well above 0.5 on the decision probe: the gated DatedGPT reader scores 0.79 on its 31 items, and ChronoGPT with a 2011 cutoff scores 0.65 on the 31 rate changes after its cutoff. We do not know which prior they exploit (the persistence rule, the direction of the last change before the year, scores only 0.54), so the probe cannot separate memory from such priors. The modern readers score 0.64 to 0.95. We report the probe in Appendix § 8 and draw no conclusion from it.
Membership Membership scores cannot be validated on the chronologically consistent checkpoints, because no one knows which documents released before a cutoff were in the training data. The comparison does show the era confound (Table 13): the mean log-probability ranks pre-cutoff documents as no more familiar than post-cutoff ones, usually less (AUC 0.16 to 0.49), because the Fed’s language changes over the decades. For OLMo-2 we count up to three novel 14-word spans of each excerpt in its indexed stage-1 pretraining corpus with infini-gram (Liu et al., 2024); a span is novel when no earlier FOMC document contains it, because statements repeat whole sentences. Of the 416 documents with a novel span, 68% of those released up to 2021 are found and 0% of those released from 2023; the last one found is the statement of 2 November 2022, a year before the model card’s cutoff. An absent span may have been seen in another form, and a statement quoted in the news may be found through the quotation, so the counts measure exposure to the text rather than to the document. Statements are almost all found (183/189 up to 2022) and minutes much less often (69/183), and Min-K% separates the two document types (AUC 0.83 for OLMo-2-7B). Within a year and a document type, where neither era nor type can help, Min-K% does not separate the documents found from those not found (AUC 0.54, permutation ). On these data, membership scores track era and document type rather than membership.
| Checkpoint | mean log-prob | Min-K% | Min-K%++ | mean log-prob | Min-K% | Min-K%++ | ||
| All documents | Two years either side of the cutoff | |||||||
| ChronoGPT, cutoff 2011 | 0.16 | 0.34 | 0.60 | 429 | 0.41 | 0.64 | 0.51 | 65 |
| ChronoGPT, cutoff 2015 | 0.21 | 0.37 | 0.51 | 429 | 0.44 | 0.60 | 0.51 | 64 |
| ChronoGPT, cutoff 2019 | 0.26 | 0.28 | 0.54 | 429 | 0.37 | 0.26 | 0.58 | 66 |
| ChronoGPT, cutoff 2024 | 0.49 | 0.52 | 0.64 | 429 | 0.76 | 0.77 | 0.72 | 50 |
| OLMo-2-7B, against corpus presence | 0.77 | 0.77 | 0.75 | |||||
| OLMo-2-13B, against corpus presence | 0.81 | 0.77 | 0.63 |
Table 13. Membership scores (). ChronoGPT rows: AUC of each score for documents released before the checkpoint’s cutoff (possible members) against documents released after it (certain non-members). OLMo rows: AUC against presence of a document’s novel text in OLMo-2’s indexed stage-1 corpus, over all documents with a novel span.
Measuring what memory is worth
The real-time nowcaster We price memory by putting each reader’s signal into a real-time nowcaster, which estimates growth in the current quarter before the official figure is published. At each origin the nowcaster takes one vintage of the eleven indicators. Indicators are released with different delays, so the latest months are missing for some series; this uneven end of the panel is the ragged edge. The nowcaster fills it with forecasts from an autoregression of order two, AR(2), on each indicator’s own monthly changes, averages to the quarter, and forms growth. It then reduces the target’s own block of indicators (real activity for GDP, prices for CPI) to its first three principal components, which we call block PCA, and regresses the target on these three factors with a ridge penalty. This ridge regression is the bridge, the usual name for a model that links monthly indicators to a quarterly target. It is estimated on the quarters from 2000 Q1 whose first release is out by the origin. A text model adds the five dimensions of the latest FOMC document published on or before the origin; training quarters take the document available at the same position in their own quarter. Every regressor is standardised by its training mean and standard deviation, so the penalty treats every reader alike. The fill and the indicator blocks were chosen among six specifications on a 2006–2011 hold-out; all six are reported on both windows (Appendix § 7.§ 5), and the order in which they were tried is stated there.
Baselines Text is only worth something if the hard-data model is competent, so we report two further benchmarks. The first is a mixed-frequency dynamic factor model in the style of Bańbura and Modugno [2], fit by expectation maximisation on the same vintages with the ragged edge left as missing data, which this estimator handles directly, and with factor number and order chosen on the hold-out. The second is the set of professional nowcasts practitioners use: the Survey of Professional Forecasters, GDPNow (Higgins, [28]) and the New York Fed Staff Nowcast (Bok et al., [3]), each used only where its value was public on the date it carries: GDPNow from July 2014, when it went online, and the New York Fed nowcast during its live publication, April 2016 to September 2021 and from September 2023. We also ask directly whether a text signal corrects the professional nowcasts, by regressing their first-release error on the signal out of sample.
The lookahead premium The lookahead premium of a leaky reader is the percentage reduction in nowcast RMSE of its text model against the gated reader’s text model, on the same origins. Three origins share each target quarter. The primary test is the Coroneo–Iacone test (Coroneo and
(Iacone, 2020), which compares the accuracy of two forecasts and is built for small samples with autocorrelated errors. We run it two-sided, with a weighted-periodogram estimate of the long-run variance, bandwidth for origins and a Student- reference with degrees of freedom. As checks, we compute the Harvey–Leybourne–Newbold test [25], a small-sample correction of the standard test of equal forecast accuracy, run with a horizon of three origins, and a 95% interval from resampling quarters. The gain of a text model over the model without text is a nested comparison, because the model without text is the text model with its text coefficients set to zero. We test it with the one-sided Clark–West test [8], which adjusts the loss difference for the noise the larger model adds by estimating coefficients that are zero under the null. Holm’s method, a step-down version of the Bonferroni correction, adjusts each family of tests for multiplicity: the premium tests within each target, the 26 text-against-no-text tests across both targets, and the 48 tests of text on top of the professional nowcasts. The minimum detectable effect is the premium the primary test would detect with 80% power, from a quarter-block bootstrap standard error of the mean loss difference and Student- quantiles with degrees of freedom. We report the premium leaving out each year in turn, and a placebo for the gated and ChronoGPT-2024 readers: the sequence of document signals is rotated against the calendar, which keeps the signal’s distribution and persistence and breaks its alignment with outcomes, legitimate information included.
Results: what memory is worth
A baseline without GDP skill We want to know what text adds to a competent nowcaster, and on GDP the hard-data models are not competent. Over all origins the bridge scores 1.913 and a running mean of past first releases 1.890 ($p = 0.81); without 2020–2021 the running mean scores 1.273 against 1.428 for the bridge () and 1.479 for the dynamic factor model. Any GDP premium is therefore measured on a base with no skill over the historical mean. On CPI the bridge is much more accurate than the running mean, 1.148 against 2.406 ().
Text against no text Table 3 puts every reader’s signal into the same nowcaster. Against the bridge without text, the gated reader’s signal changes GDP RMSE by −2.6%, the ChronoGPT-2024 signal by +0.9% (Clark–West ), and the best modern reader, Qwen2.5-14B, by +0.9% (). After Holm adjustment over the 26 text-against-no-text tests the smallest adjusted is 1.00. On CPI no reader’s signal improves the bridge.
Table 3: Real-time GDP nowcasts, 2012–2024, 150 origins over 50 quarters. RMSE against first release. Premium: percentage reduction in RMSE against the gated ChronoGPT reader’s text model, with a 95% interval from resampling quarters; the premium without 2021, without 2020, and its range when each year is left out in turn; and the premium with unguarded signals for both readers. CPI in Table 9.
| Reader | RMSE | premium, % [95%] | ex-2021 | ex-2020 | each year left out | unguarded premium [95%] |
| No text (factor bridge) | 1.913 | |||||
| ChronoGPT, gated | 1.962 | ref. | ref. | |||
| ChronoGPT, cutoff 2011 | 1.951 | +0.6 [−6.4, +9.7] | −4.1 | +4.9 | [−4.1, +4.9] | −0.7 [−6.0, +6.6] |
| ChronoGPT, cutoff 2015 | 1.977 | −0.8 [−7.1, +7.3] | −3.0 | +3.5 | [−3.0, +3.5] | −1.8 [−6.5, +5.3] |
| ChronoGPT, cutoff 2019 | 1.965 | −0.1 [−6.4, +8.2] | −3.2 | +3.7 | [−3.2, +3.7] | −1.5 [−6.7, +5.9] |
| ChronoGPT, cutoff 2024 | 1.895 | +3.4 [−2.7, +12.6] | −0.9 | +7.7 | [−0.9, +7.7] | +1.3 [−3.5, +8.2] |
| Qwen2.5-1.5B | 1.935 | +1.4 [−4.0, +7.6] | −2.4 | +2.9 | [−2.4, +2.9] | −0.3 [−4.5, +5.3] |
| Qwen2.5-3B | 1.979 | −0.9 [−10.1, +7.2] | −5.1 | +0.0 | [−5.1, +1.6] | +1.1 [−3.1, +6.7] |
| Qwen2.5-7B | 1.903 | +3.0 [−3.5, +11.1] | −2.2 | +5.3 | [−2.2, +5.3] | +0.2 [−4.7, +6.3] |
| Qwen2.5-14B | 1.895 | +3.4 [−1.8, +10.6] | −0.3 | +6.3 | [−0.3, +6.3] | +1.1 [−3.6, +8.2] |
| OLMo-2-7B | 1.963 | −0.1 [−5.6, +6.7] | −3.1 | +1.8 | [−3.1, +1.8] | −1.8 [−7.3, +5.9] |
| OLMo-2-13B | 1.902 | +3.1 [−3.4, +12.1] | −1.5 | +6.5 | [−1.5, +6.5] | −0.7 [−8.4, +7.1] |
| Mistral-7B | 1.914 | +2.4 [−3.1, +9.7] | −1.2 | +4.4 | [−1.2, +4.4] | −3.0 [−12.8, +5.5] |
| Dictionary | 1.907 | n/a | n/a | n/a |
Table 3.
Table 9: Real-time CPI nowcasts, as in Table 3.
| Reader | RMSE | premium, % [95%] | ex-2021 | ex-2020 | each year left out | unguarded premium [95%] |
| No text (factor bridge) | 1.148 | |||||
| ChronoGPT, gated | 1.161 | ref. | ref. | |||
| ChronoGPT, cutoff 2011 | 1.152 | +0.7 [−1.0, +2.6] | +1.2 | +0.6 | [+0.5, +1.2] | +0.1 [−1.8, +2.1] |
| ChronoGPT, cutoff 2015 | 1.157 | +0.3 [−1.8, +2.4] | +1.6 | +0.4 | [−0.2, +1.6] | +0.2 [−1.9, +2.1] |
| ChronoGPT, cutoff 2019 | 1.156 | +0.4 [−1.5, +2.3] | +1.3 | +0.4 | [+0.2, +1.3] | −0.1 [−2.7, +2.2] |
| ChronoGPT, cutoff 2024 | 1.162 | −0.1 [−2.5, +1.7] | −0.4 | −0.1 | [−0.4, +0.5] | −0.1 [−2.3, +1.7] |
| Qwen2.5-1.5B | 1.150 | +0.9 [−1.9, +3.5] | +1.6 | +0.9 | [+0.5, +1.6] | +1.9 [−0.6, +4.6] |
| Qwen2.5-3B | 1.172 | −1.0 [−3.7, +1.2] | +0.1 | −0.9 | [−2.3, +0.1] | +0.4 [−2.3, +2.5] |
| Qwen2.5-7B | 1.209 | −4.2 [−6.9, −1.6] | −3.1 | −4.1 | [−6.0, −3.1] | −0.4 [−3.5, +2.5] |
| Qwen2.5-14B | 1.164 | −0.3 [−2.9, +1.9] | +0.8 | −0.3 | [−1.3, +0.8] | −1.6 [−5.8, +2.3] |
| OLMo-2-7B | 1.184 | −2.0 [−5.2, +1.3] | −1.1 | −1.9 | [−2.4, −1.1] | −0.2 [−2.3, +1.9] |
| OLMo-2-13B | 1.173 | −1.0 [−3.4, +0.9] | −0.3 | −1.0 | [−2.0, −0.3] | −0.2 [−7.3, +1.4] |
| Mistral-7B | 1.205 | −3.8 [−6.3, −1.3] | −3.0 | −3.8 | [−4.8, −2.9] | −3.7 [−7.5, −1.0] |
| Dictionary | 1.172 | n/a | n/a |
Table 9.
The lookahead premium Against the gated reader, the GDP premiums of the eight leaky readers run from −0.9% to +3.4% (Figure 3); all eight intervals include zero and the smallest Holm-adjusted is 1.00. The premiums depend on two years. Without 2021 they run from −5.1% to −0.3%; without 2020, which contributes 2020 Q1 and Q4, from 0.0% to +7.7%. Every one of these lies below the minimum detectable effect of 8.8% to 13.1%. The 2021 swing is not a mark of memory: ChronoGPT with a 2011 cutoff, which cannot know any test-window outcome, also loses its premium without 2021 (−4.1%), which points to a poor 2021 for the gated reader rather than to knowledge of 2021 in the others (Figure 5, Appendix § 8). On CPI the premiums run from −4.2% to +0.9%, and two leaky readers are worse than the gated one with intervals that exclude zero: Qwen2.5-7B (−4.2%, [−6.9, −1.6], Coroneo–Iacone , Harvey–Leybourne–Newbold ) and Mistral-7B (−3.8%, and 0.013); after Holm adjustment within CPI the smallest is 0.24.

Figure 3. Lookahead premium of each leaky reader over the gated reader, all quarters (dot, 95% quarter-block interval) and without 2021 (bar).

Figure 5. (a) First-release GDP growth and three nowcasts at the first origin of each quarter. (b) Running sum of the gated reader’s squared error minus the ungated ChronoGPT-2024 reader’s: the gap rises before 2020, falls in 2020 and rises in 2021.
The output gate moves accuracy by about as much as the premiums. Against its unguarded signal (which also skips the calibration), the gated reader’s guarded signal changes GDP accuracy by −2.3%; with unguarded signals throughout, the GDP premiums run from −3.0% to +1.3%. The rotation placebo finds little: against the bridge, the real ChronoGPT-2024 signal beats 92% of its 64 rotations (a one-sided of about 0.08) and the real gated signal 22% of its own; measured against the real gated reader, rotated ChronoGPT-2024 signals, misaligned with every outcome, reach the observed premium in 8% of rotations.
nals, misaligned with every outcome, reach the observed premium in 8% of rotations.
Professional nowcasts No reader’s signal corrects the SPF, GDPNow or New York Fed nowcasts beyond what removing their mean error does: across the 48 tests of a text correction against a mean-error correction, the smallest Holm-adjusted is 0.99 (Table 12). Against GDPNow the mean-error correction alone improves the nowcast.
| Nowcast | Target | quarters | RMSE | bias-only | best text reader | ratio | Holm |
| GDPNow | GDP | 32 | 1.986 | 0.949 | Qwen2.5-3B | 0.975 | 1.00 |
| New York Fed | GDP | 19 | 1.945 | 1.031 | ChronoGPT, cutoff 2019 | 1.078 | 1.00 |
| SPF | CPI | 50 | 1.437 | 1.002 | Mistral-7B | 0.944 | 1.00 |
| SPF | GDP | 50 | 1.748 | 1.001 | ChronoGPT, cutoff 2015 | 0.996 | 1.00 |
Table 12. Text on top of the professional nowcasts. Bias-only: the professional nowcast plus its mean past error. Best text reader: the reader whose out-of-sample correction has the lowest RMSE; ratio to the professional RMSE. Holm : the Holm-adjusted Clark–West of that reader’s text correction against the mean-error correction, over all 48 text tests. RMSE is over the origins scored out of sample, so it differs slightly from Table 11, which weights each quarter once.
| Forecast | Origins | US GDP | US CPI |
| Survey of Professional Forecasters, median | mid-quarter, 50 quarters | 1.748 | 1.430 |
| GDPNow (Atlanta Fed), public from July 2014 | weekly, 40 quarters | 1.968 | n/a |
| New York Fed Staff Nowcast, live periods | weekly, 26 quarters | 1.752 | n/a |
| AR(), BIC | one per quarter | 2.365 | 2.094 |
| Naive last | one per quarter | 4.151 | 1.958 |
| Seasonal naive | one per quarter | 8.066 | 2.995 |
| Expanding mean | one per quarter | 1.900 | 2.399 |
| Real-time factor bridge (this paper) | month-end, 150 | 1.913 | 1.148 |
Table 11. Every forecast against first release, 2012–2024, pandemic quarters excluded, each on its own coverage. The univariate rules make one forecast per quarter, so the expanding mean differs slightly from the running mean scored at the 150 monthly origins in § 5.1.
Discussion
What the evidence supports The larger modern open models hold outcome memory that a date retrieves, beyond what memory-free readers achieve on the same items. Whether the small chronologically consistent models hold any is not resolvable with these probes. In a real-time nowcast, neither the memory nor the text shows up as resolvable skill. A probe can show that a model could use the future, but only a forecasting comparison can show that it does, and at macroeconomic sample sizes that comparison has little power.
Why the forecast comparison has little power With 150 origins over 50 quarters, the primary test detects a premium of about 8.8 to 13.1 percent of GDP nowcast RMSE with 80% power, larger than every premium we estimate with or without 2020 and 2021. The swings of 2020 and 2021 move the sample RMSE more than all other quarters together, which is why the premium changes sign depending on which of the two years is left out. Audits of language-model forecasters that rest on forecast accuracy alone should report the minimum detectable effect and the leave-one-period-out premium.
Beyond central-bank text The design needs only dated inputs, outcomes with known release dates, and one family of checkpoints with known cutoffs, as in forecasting benchmarks of questions that resolve on known dates or problems sorted by release date (Roberts et al., 2024).
Limitations The documents are one central bank’s, and the probes test two outcomes. The chronologically consistent families available are small, so the contrast between a chronologically consistent reader and a modern one confounds the cutoff with scale and training volume; the ladder within each family does not, but it measures small models. Training cutoffs are documented only for OLMo-2’s base model; for the other modern readers we bound them by release dates. Items after a reader’s cutoff are mislabelled as leaky, which biases its probe AUC toward chance, not away from it. The gate is conservative: a document of year is read by the checkpoint that ends in , so the gated reader misses up to a year of legitimate context. The nowcaster’s fill was chosen on a hold-out after two other fills had been seen on the test window; all six specifications are reported.
Use of AI assistants
The authors used AI assistants for editing text and for coding, under their direction, and take full responsibility for the content.
Replication details
Readers
Table 4 gives each reader’s size, the end of its training text and its role.
Prompts
Every document question uses the template below; the questions and probe prompts are listed in Table 5, read from the scoring code. Chat models receive the prompt as one user turn through their own chat template. The ChronoGPT-Instruct readers receive it in the authors’ instruction format, and the DatedGPT-Instruct readers through their chat template, encoded without a second beginning-of-sequence token.
| Dimension | Answers | Question |
| Stance | more / less | Compared with a neutral stance, is the monetary policy described in this text more or less restrictive? Is the Committee in this text setting policy that is more or less restrictive than neutral? Overall, does this text describe monetary policy that is more or less restrictive? |
| Inflation risk | high / low | Are the risks of inflation rising described in this text high or low? How large is the concern about inflation pressure in this text, high or low? Is the upside risk to inflation in this text high or low? |
| Growth risk | high / low | Are the downside risks to economic growth in this text high or low? How large is the concern about weak economic growth in this text, high or low? Is the risk of an economic slowdown in this text high or low? |
| Uncertainty | high / low | Is the economic uncertainty described in this text high or low? How uncertain is the outlook described in this text, high or low? Is the level of uncertainty expressed in this text high or low? |
| Forward guidance | higher / lower | Does the guidance about future policy in this text point to higher or lower interest rates ahead? Is the expected path of the policy rate in this text higher or lower? Does this text signal higher or lower interest rates in the coming months? |
| Decision probe | raise / lower | On <date>, the Federal Open Market Committee changed its target for the federal funds rate. Did it raise or lower the target? |
| GDP probe | above / below | According to the first official estimate, was US real GDP growth above or below 2 percent at an annual rate in <date>? |
Table 5. Every question asked of every reader. Three wordings per document dimension; one per probe. The answer pair is fixed per dimension.
Text from a Federal Open Market
Committee document:
"""
{excerpt}
"""
{question} Answer with one word:
{answer 1} or {answer 2}.
The content-free input used for calibration replaces the excerpt with “N/A”. The probe prompts end with the same instruction to answer in one word.
Corpus construction and cleaning
The corpus is built from the Federal Reserve’s own index of documents and release dates. Four corrections were applied before any result reported here.
Misdated minutes. Twenty-eight raw files held copies of minutes stored under the date of an intermeeting conference call, between three and sixty-six days before the minutes were published, which placed text before its public release for two test origins and thirteen training cells. Files with no entry in the Federal Reserve’s index are not used.
Missing minutes. Twenty-four sets of minutes from 2004–2006 were stored under non-standard file names. They are included, dated by their release.
Excerpt boundaries and dates. Twenty-six statements without a recorded vote run into the web page’s navigation text; their excerpts stop at the page footer. Eight statements carried a date one day before their release; statements are dated by their press-release URL. We added section markers for minutes whose outlook or policy discussion the standard markers miss; 6 sets of minutes still yield only one of the two sections.
Character encoding. Sixty-eight excerpts carried UTF-8 punctuation decoded as Latin-1. They are decoded as UTF-8.
Labels for the decision probe come from the Federal Reserve’s published table of target changes, 74 changes from 2000 to 2025, each dated by the statement that announced it. Every reader scored the corrected documents and probe items.
Determinism
Scores are deterministic up to floating-point noise. The ungated ChronoGPT-2024 reader was scored twice, on an Apple-silicon GPU and on a CUDA GPU, before the corpus cleaning; the 6,688 raw scores differ by at most and agree in sign in 100% of cases. Items changed by the cleaning were then scored again.
Nowcaster specifications
Six specifications were evaluated: three ways of filling the months not yet published (the mean growth of the last three months, no change, and an autoregression of order two on each indicator’s own monthly changes), each with all eleven indicators or with indicator blocks by target. The fills were run in the order listed, and once the autoregressive fill was added the selection rule (the single specification with the lowest mean hold-out RMSE across the two targets, on origins 2006–2011) was applied to all six. Four of the six had been seen on the test window before the rule was set. Table 6 reports all six on both windows and Table 10 the gated and ChronoGPT-2024 readers under each.
| Ragged-edge fill | Indicators | Hold-out 2006–2011 GDP | Hold-out 2006–2011 CPI | Test 2012–2024 GDP | Test 2012–2024 CPI |
| mean growth | all eleven | 1.831 | 2.545 | 2.376 | 1.675 |
| mean growth | by target | 1.704 | 2.303 | 2.419 | 1.327 |
| no change | all eleven | 1.896 | 2.375 | 1.921 | 1.577 |
| no change | by target | 1.751 | 2.089 | 1.933 | 1.655 |
| AR(2) | all eleven | 1.926 | 2.200 | 1.903 | 1.281 |
| AR(2) | by target | 1.798 | 1.772 | 1.913 | 1.148 |
Table 6. All six nowcaster specifications without text, on the hold-out window used for selection and on the test window. Starred: the selected specification.
Table 10: The gated and ChronoGPT-2024 readers under every specification. Premium: percentage reduction in RMSE of the ungated against the gated reader, with the Coroneo–Iacone . The GDP premium is positive in four of six specifications and significant in none; the CPI premium runs from −6.4% to +6.5% and is nominally significant in one.
| Fill | Blocks | US GDP factor | US GDP gated | US GDP ungated | US GDP premium () | US CPI factor | US CPI gated | US CPI ungated | US CPI premium () |
| mean growth | all eleven | 2.376 | 2.322 | 2.366 | −1.9 (0.58) | 1.675 | 1.646 | 1.719 | −4.5 (0.26) |
| mean growth | by target | 2.419 | 2.349 | 2.390 | −1.8 (0.54) | 1.327 | 1.334 | 1.356 | −1.6 (0.25) |
| no change | all eleven | 1.921 | 2.038 | 1.949 | +4.4 (0.13) | 1.577 | 1.724 | 1.612 | +6.5 (0.01) |
| no change | by target | 1.933 | 2.052 | 1.951 | +4.9 (0.15) | 1.655 | 1.673 | 1.672 | +0.1 (0.93) |
| AR(2) | all eleven | 1.903 | 1.943 | 1.899 | +2.3 (0.37) | 1.281 | 1.235 | 1.314 | −6.4 (0.10) |
| AR(2)* | by target | 1.913 | 1.962 | 1.895 | +3.4 (0.28) | 1.148 | 1.161 | 1.162 | −0.1 (0.92) |
Table 10.
Baselines
Professional nowcasts are used only where their value was public on its date. The dynamic factor model uses the same vintages, blocks and training quarters as the bridge; its number of factors and their order were chosen on the hold-out (two factors with two lags for GDP, two with one lag for CPI). For Table 12, the professional’s first-release error is regressed, out of sample on an expanding window, on the latest document’s signal, and the corrected nowcast is compared with the professional one by the Clark–West test and with a correction for the professional’s mean error alone.
Supplementary tables and figures
Tables and figures referenced from the main text.

Figure 6. Share of document-dimension readings set to zero because the three wordings of the question disagreed in sign. Against its unguarded signal, the gated reader’s guarded signal changes GDP accuracy by −2.3%; with unguarded signals throughout, the GDP premiums run from −3.0% to +1.3% instead of −0.9% to +3.4% (Table 3).
Table 7: GDP probe within years over all items (raw, and after removing each reader’s mean score per quarter word, with permutation tests within years), the GDP probe AUC on the 2012–2024 items alone, and the decision probe with its circular-shift . Before the quarter-word adjustment, the ChronoGPT checkpoints with 2015, 2019 and 2024 cutoffs score 0.70, 0.73 and 0.72 within years; after it, 0.52, 0.61 and 0.53. On the decision probe the gated DatedGPT reader scores 0.79 on its 31 items and ChronoGPT with a 2011 cutoff 0.65.
| Reader | Same year AUC | Same year | Same year, word-adjusted AUC | Same year, word-adjusted | 2012–2024 AUC | Decision probe AUC | Decision probe circular | |
| ChronoGPT, gated | 0.48 | 0.612 | 0.42 | 0.854 | 0.53 | 74 | 0.50 | 0.554 |
| ChronoGPT, cutoff 2011 | 0.58 | 0.206 | 0.47 | 0.678 | 0.60 | 74 | 0.65 | 0.041 |
| ChronoGPT, cutoff 2015 | 0.70 | 0.006 | 0.52 | 0.469 | 0.64 | 74 | 0.44 | 0.743 |
| ChronoGPT, cutoff 2019 | 0.73 | 0.002 | 0.61 | 0.107 | 0.48 | 74 | 0.54 | 0.297 |
| ChronoGPT, cutoff 2024 | 0.72 | 0.004 | 0.53 | 0.380 | 0.62 | 74 | 0.46 | 0.743 |
| DatedGPT, gated | 0.48 | 0.613 | 0.48 | 0.621 | 0.54 | 31 | 0.79 | 0.065 |
| DatedGPT, cutoff 2016 | 0.64 | 0.057 | 0.52 | 0.471 | 0.65 | 74 | 0.61 | 0.041 |
| DatedGPT, cutoff 2020 | 0.27 | 0.999 | 0.44 | 0.806 | 0.33 | 74 | 0.64 | 0.081 |
| DatedGPT, cutoff 2024 | 0.44 | 0.814 | 0.62 | 0.072 | 0.47 | 74 | 0.49 | 0.540 |
| Qwen2.5-1.5B | 0.48 | 0.595 | 0.62 | 0.075 | 0.56 | 74 | 0.64 | 0.243 |
| Qwen2.5-3B | 0.66 | 0.035 | 0.62 | 0.080 | 0.70 | 74 | 0.67 | 0.054 |
| Qwen2.5-7B | 0.58 | 0.175 | 0.59 | 0.145 | 0.71 | 74 | 0.70 | 0.013 |
| Qwen2.5-14B | 0.66 | 0.030 | 0.61 | 0.114 | 0.73 | 74 | 0.95 | 0.013 |
| OLMo-2-7B | 0.66 | 0.030 | 0.53 | 0.386 | 0.70 | 74 | 0.73 | 0.054 |
| OLMo-2-13B | 0.56 | 0.244 | 0.56 | 0.248 | 0.68 | 74 | 0.86 | 0.013 |
| Mistral-7B | 0.66 | 0.030 | 0.61 | 0.092 | 0.77 | 74 | 0.77 | 0.013 |
| Persistence rule | 0.44 | 0.897 | 0.41 | 0.961 | 0.47 | 74 | 0.54 | 0.392 |
Table 7.
References
- [2]Marta Bánbura and Michele Modugno. 2014. Maximum likelihood estimation of factor models on datasets with arbitrary pattern of missing data. Journal of Applied Econometrics, 29(1):133–160.
- [3]Brandyn Bok, Daniele Caratelli, Domenico Giannone, Argia M. Sbordone, and Andrea Tambalotti. 2018. Macroeconomic nowcasting and forecasting with big data. Annual Review of Economics, 10:615–643.
- [4]Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations (ICLR).arxiv.org/abs/2202.07646
- [5]Andrea Carriero, Davide Pettenuzzo, and Shubhranshu Shekhar. 2026. MACROCAST: A vintage-consistent time series foundation model for real-time macroeconomic forecasting. arXiv preprint arXiv:2606.28670.arxiv.org/abs/2606.28670
- [6]Laura Caspari, Alexander Trautsch, Michael Granz-itzer, and Steffen Herbold. 2025. Studying memorization of large language models using answers to Stack Overflow questions. Transactions on Machine Learning Research.
- [7]Jeffrey Cheng, Marc Marone, Orion Weller, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme. 2024. Dated data: Tracing knowledge cutoffs in large language models. In First Conference on Language Modeling (COLM).arxiv.org/abs/2403.12958
- [8]Todd E. Clark and Kenneth D. West. 2007. Approximately normal tests for equal predictive accuracy in nested models. Journal of Econometrics, 138(1):291–311.
- [11]Dean Croushore and Tom Stark. 2001. A real-time data set for macroeconomists. Journal of Econometrics, 105(1):111–130.
- [12]Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. 2022. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10:257–273.
- [13]Felix Drinkall, Eghbal Rahimikia, Janet Pierrehumbert, and Stefan Zohren. 2024. Time Machine GPT. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3281–3292. Association for Computational Linguistics.
- [14]Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. 2024. Do membership inference attacks work on large language models? In First Conference on Language Modeling (COLM).arxiv.org/abs/2402.07841
- [15]Alexander Eliseev and Sergei Seleznev. 2026. Fake date tests: Can we trust in-sample accuracy of LLMs in macroeconomic forecasting? arXiv preprint arXiv:2601.07992.arxiv.org/abs/2601.07992
- [16]Jon Ellingsen, Vegard H. Larsen, and Leif Anders Thorsrud. 2022. News media versus FRED-MD for macroeconomic forecasting. Journal of Applied Econometrics, 37(1):63–81.DOI
- [17]Miguel Faria-e Castro and Fernando Leibovici. 2024. Artificial intelligence and inflation forecasts. Federal Reserve Bank of St. Louis Review, 106(12):1–14.
- [18]Leonardo Gambacorta, Byeungchun Kwon, Taejin Park, Pietro Patelli, and Xingyun Sonya Zhu. 2026. CB-LMs: Language models for central banking. Journal of Financial Stability, 87:101585.DOI
- [19]Zhenyu Gao, Wenxi Jiang, and Yutong Yan. 2026. Detecting lookahead bias in LLM forecasts. arXiv preprint arXiv:2512.23847v2.arxiv.org/abs/2512.23847
- [20]Domenico Giannone, Lucrezia Reichlin, and David Small. 2008. Nowcasting: The real-time informational content of macroeconomic data. Journal of Monetary Economics, 55(4):665–676.DOI
- [21]Paul Glasserman and Caden Lin. 2024. Assessing look-ahead bias in stock return predictions generated by GPT sentiment analysis. The Journal of Financial Data Science, 6(1):25–42.
- [22]Shahriar Golchin and Mihai Surdeanu. 2024. Time travel in LLMs: Tracing data contamination in large language models. In The Twelfth International Conference on Learning Representations (ICLR).arxiv.org/abs/2308.08493
- [23]Stephen Hansen and Michael McMahon. 2016. Shocking language: Understanding the macroeconomic effects of central bank communication. Journal of International Economics, 99(S1):S114–S133.DOI
- [24]Stephen Hansen, Michael McMahon, and Andrea Prat. 2018. Transparency and deliberation within the FOMC: A computational linguistics approach. The Quarterly Journal of Economics, 133(2):801–870.
- [25]David Harvey, Stephen Leybourne, and Paul Newbold. 1997. Testing the equality of prediction mean squared errors. International Journal of Forecasting, 13(2):281–291.DOI
- [26]Songrun He, Linying Lv, Asaf Manela, and Jimmy Wu. 2025. Instruction tuning for chronologically consistent language models. arXiv preprint arXiv:2510.11677.arxiv.org/abs/2510.11677
- [27]Songrun He, Linying Lv, Asaf Manela, and Jimmy Wu. 2026. Chronologically consistent large language models. Journal of Financial Economics, 185:104374.
- [28]Patrick C. Higgins. 2014. GDPNow: A model for GDP “nowcasting”. Working Paper 2014-7, Federal Reserve Bank of Atlanta.DOI
- [29]Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825.arxiv.org/abs/2310.06825
- [30]Eleni Kalamara, Arthur Turrell, Chris Redl, George Kapetanios, and Sujit Kapadia. 2022. Making text count: Economic forecasting using newspaper text. Journal of Applied Econometrics, 37(5):896–919.
- [31]Edward S. Knotek, II and Saeed Zaman. 2017. Nowcasting U.S. headline and core inflation. Journal of Money, Credit and Banking, 49(5):931–968.
- [32]Evan F. Koenig, Sheila Dolmas, and Jeremy Piger. 2003. The use and abuse of real-time data in economic forecasting. The Review of Economics and Statistics, 85(3):618–628.