WorksComputer Science
Who Pays for the Knowledge Models Copy? Model Collapse as Lossy Transmission and Rogers' Paradox
Abstract
Generative models trained on the output of earlier models lose knowledge across generations, rare knowledge first. The remedies that work all bring in information from outside the loop, such as fresh human data, verifiers, or curators, and the theory of model collapse treats that supply as given. We argue that cultural evolution has a theory of the supply, and that it distinguishes two ways copying loses knowledge. In a fixed world, copies are lossy, as in Henrich's account of skill loss in small populations. In a changing world, even perfect copies go stale, which is Rogers' paradox. We write both as one chain with separately measurable parameters and test it on lineages of language models (word trigrams and a character transformer on two books, and OPT-125m on wikitext-2, on arXiv abstracts written after its pretraining, and on a world of 2,000 fictional facts that change at a set rate). In a fixed world, neutral drift on word frequencies predicts, with no free parameter, the rare-word coverage of every trigram lineage to within 0.03. In the fact world, the model's accuracy falls and its outdated answers rise with the rate of change; the chain calibrated on the static world predicts both for mid- and low-frequency facts to within 0.03 on average, with the rate of change set, not fitted. Models that inherit their weights keep more in a static world and carry more outdated facts in a changing one, as Rogers' theory says of individual memory. Finally we close the loop with the people who write the data. When they can ask the model instead of finding out, the model ends up exactly as reliable as finding out is costly, and in both regimes someone must keep paying to learn; we give the share in closed form, extend it to lossy copying, heterogeneous costs, aggregating models, and users who check, and show when checking is a stable way out.
Introduction
Training generative models on the output of earlier models degrades them. Across generations, the tails of the learned distribution thin and disappear, rare content is forgotten before common content, and the models’ output narrows [43, 1, 4, 23, 6]. Model collapse now has a substantial theory: it can be described as a change of scaling law concentrated in the tails [11], analysed exactly in regression [10], shown to arise from small proportions of synthetic data [12], bounded in rate [45, 42], and avoided when real data accumulate rather than being replaced [20, 30]. Its definitions and assumptions are themselves debated [41].
The remedies that work share one feature: each brings in information from outside the self-consuming loop. Fresh real data [1], retained real data [20], verifiers of correctness [16, 47], correction toward ground truth [21], and human curation [17] all import something the model does not already have. And the theory that supports them shares two assumptions. The outside information is treated as an exogenous archive, there to be drawn on in whatever quantity is needed. And the target distribution is fixed: what the model should know does not change while it is being copied.
Neither assumption holds for the systems that motivate the problem. The fresh text on which future models are trained is written by people, and people increasingly consult the models instead of finding things out for themselves; the supply of fresh data responds to the models it feeds. And the world that text describes keeps changing. Cultural evolution has studied both features for decades, and it separates two ways in which copying loses knowledge. In a fixed world, each copy is imperfect, and a population that learns only by copying loses its rarest skills unless enough people keep supplying good models (Henrich, 2004). In a world that changes, copies are also outdated: Rogers’ paradox [40] is that copying spreads because it is cheaper than learning, and spreads until the population is no better off than one that never copied, because someone must keep paying to learn what is true now. Both are answers to the question this paper asks of model collapse: who supplies the knowledge that a lineage of models copies, and what does it cost them?
Terms. A lineage is a sequence of models in which generation is trained on a corpus that mixes a fraction of fresh data, written by people about the world, with a fraction of synthetic data, sampled from generation . In a fixed world the source of fresh data stays the same; in a changing world it changes between generations, either because the text people write moves on (Section 6.1) or because facts change their value (Section 6.2).
Contributions. (i) We write lossy transmission and Rogers’ staleness as one chain whose parameters (what fresh data supply, how faithfully copies transmit, and how fast the world changes) can be measured separately (Section 4). (ii) In a fixed world, a zero-parameter neutral-drift baseline predicts the rare-word coverage of word-trigram lineages; the loss is Henrich’s, not Rogers’. A pretrained transformer deviates from it in a way that points to its pretrained weights as a reserve outside the chain (Section 5). (iii) In a world of facts that change at a set rate, we show Rogers’ paradox at the level of the model: accuracy falls and outdated answers rise with the rate of change, the chain calibrated on the static world predicts the changing one, and inheriting weights helps in a static world and carries stale facts in a changing one (Section 6). (iv) Closing the loop with the people who write the data, we derive the share of people who must keep learning, in a fixed and in a changing world, extend it to heterogeneous costs, aggregating models, and users who check answers, and show when checking is stable (Section 7). (v) We place synthetic-data filters among social learning strategies, and show that filters selecting plausible text act as conformist copying (Section 8).
Background: two ways copying loses knowledge
Lossy transmission. Henrich (2004) modelled a population that acquires a skill by copying the most skilled member of the previous generation. Each copy falls short on average, and only occasionally does a copier exceed its model; whether the skill is maintained or lost depends on how many people try, because more attempts make an improving copy more likely. When a population shrinks, complex skills are lost even though nothing in the environment has changed. In population genetics the same structure appears as neutral drift: a finite sample of the previous generation loses rare variants by chance, and migration from outside restores them [46, 15]. A lineage of models that samples its own training data is such a population, and fresh data are its migrants.
Rogers’ paradox. In Rogers’ model [40], the best behaviour in an environment changes with probability per generation. An individual learner pays a cost and acquires the currently correct behaviour; a social learner copies a random member of the previous generation at no cost, and is right only if its model was right and the environment has not changed since. Social learning spreads while rare, because its copies come mostly from learners, and stops spreading when social and individual learners do equally well. At that point, the population’s mean payoff equals that of a population of individual learners alone: copying has been adopted without making anyone better off. The two mechanisms differ in what goes wrong with a copy: under lossy transmission it is a worse version of something still true, under Rogers’ it is a faithful version of something no longer true.
Strategies. Three refinements matter here. A critical social learner copies, checks the copy against the world, and learns for itself if the copy fails; such learners raise the population’s payoff above the baseline, because the check imports information [14]. A conformist learner copies what is common rather than a random model; conformity is cheap, but it erodes variation, because rare variants are copied less often than they occur [5]. And a payoff-biased learner copies whoever does best, which concentrates the population on the current winner [5, 32]. The social learning strategies tournament found that successful strategies copy most of the time but copy well [39]. Training a model on another model’s output is social learning in the sense of Heyes [26]: the learner reconstructs competence from another’s behaviour through its own learning machinery.
Related work
Social dynamics of AI and collapse. Collins et al. [8] simulate Rogers’ paradox in a population that can also learn from an AI system, which itself updates by tracking the population’s mean. They find the paradox intact and show that it is broken when people can override the AI by relearning after a failed copy, a form of critical social learning; they also study how often the AI should update and how reliance can erode people’s own ability to learn. What they explicitly set aside is the part that defines model collapse, models trained on the output of other models, which we take as central: our models have a second copying channel, the share of model-written text, and our equilibria are in closed form (the critical asker of Section 7 is their override strategy). Baumann et al. [3] model people choosing between effort and a generative model whose benefit declines with use, and show that welfare can fall below its pre-AI level; their model does not use social-learning theory or the model-to-model loop. Guo et al. [22] and Ren et al. [38] read collapse through iterated learning, which explains which structures degrade and in what order. Peterson [36] argues that reliance on AI narrows public knowledge, and Brinkmann et al. [7] set out the programme of studying machines as participants in cultural evolution. None of these separates lossy transmission from staleness, or tests a chain’s quantitative predictions on trained language models.
Continual learning and distillation. A lineage that inherits its weights is a continual learner on a stream whose later batches are partly its own output. Continual learning studies how a network keeps old knowledge while learning new data [18, 31], and continual knowledge learning asks the same of the facts a language model holds when the world updates them [28]; temporal generalization studies how models trained on the past degrade on the future [33, 9]. Our inherited lineages show both sides of this trade-off in one system: the same weights that retain facts across generations also retain facts that have since changed (§ 6.3). Training a model on a model’s output is also distillation [27]; repeated self-distillation can improve a student [19] but progressively regularizes it toward a narrower function [35], which is the narrowing that collapse studies observe over generations of sampling.
Knowledge in language models. Language models learn facts in proportion to how often they are seen and struggle with long-tail facts [29]; synthetic worlds of fictional individuals have been used to study how such facts are stored and extracted [2]. Our fact world follows that design, adds change over generations, and measures not only whether a fact is known but whether an outdated value is given in its place.
Theory: one chain, two costs
Consider one unit of knowledge, such as a word type in a corpus or a fact in a model, and a lineage with fresh fraction . Generation holds the unit if the fresh part supplies it, with probability , or else if the synthetic part transmits it, with probability , and it is still true, with probability . The share of such units that the lineage holds then obeys
and the share of units for which the lineage holds a value that was true earlier but is false now obeys, to first order in ,
The three parameters have separate meanings. is what fresh data alone supply; it depends on and is concave in it, because a unit of frequency is absent from fresh tokens with probability . is the fidelity of the copying channel. is the rate at which the world changes. Eq. (1) contains both mechanisms of § 2:
Lossy transmission (, ). Knowledge is lost although nothing changes, , and only fresh supply () prevents total loss. For a corpus whose synthetic part is sampled from the previous corpus’s frequencies, is fixed by those frequencies: the lineage is a Wright–Fisher population with fresh data as migration, and its coverage can be predicted with no free parameter.
Rogers’ staleness (). Even perfect copies () lose value, , and the units lost are not missing but wrong: Eq. (2) predicts outdated answers that grow with and with the share of copied knowledge.
The first version of this paper used with , one rate fitted to fully synthetic training, and . As a reviewer pointed out, fresh data alone cover far less than of the rare words (measured below), and in a fixed world is sampling loss, not change; the good fit of that form at the final generation came from errors that cancel. Eq. (1) separates the terms, and the experiments test it in three ways that need no fitting to the condition predicted:
P1 In a fixed world, the coverage of a lineage’s corpus follows neutral drift with fresh data as migration (§ 5).
P2 In a changing world, and measured in the static world, together with the set rate , predict accuracy and outdated answers (§ 6.2).
P3 Knowledge kept outside the chain (pretrained weights restored at every generation) is neither lost by copying nor outdated by it; knowledge kept in inherited weights is copied with high fidelity, which helps when and carries outdated values when (§ 6.3).
In both regimes the plateau requires : someone must keep supplying fresh knowledge. Section 7 asks who.
A fixed world: lossy transmission
Setup. We train lineages of language models on two corpora from Project Gutenberg, Darwin’s On the Origin of Species and Melville’s Moby-Dick. Each corpus is split by sentence into a test set (10%), never used for training; a reference set (20%), used only by one filter (§ 8); and a pool (70%) from which the first corpus and all fresh data are drawn. The main model is a word trigram language model interpolating trigram, bigram, and unigram estimates, chosen because its knowledge can be counted exactly; we repeat the key conditions with a small character-level transformer and with a pretrained transformer language model. Generation 0 is trained on 30 000 tokens (words and punctuation marks) from the pool. Each later generation is trained on a new corpus of the same size, built from the previous model’s samples according to the condition, for 12 generations and 5 seeds. Fresh data are drawn anew at every generation: the pool is shuffled and whole sentences are taken until the token budget is met, so within a generation sentences are drawn without replacement, while across generations draws are independent and may repeat sentences of earlier corpora, including generation 0; seeds differ in the shuffles and in sampling (Appendix § 11).
Measures. Rare-word coverage: of the words outside the pool’s 1 000 most frequent, the share present in the training corpus, relative to generation 0. This is a property of the corpus, not of the model; model-level measures follow in § 6. Perplexity on the human test set, as a growth factor over generation 0. Variety: distinct three-word sequences per 1 000 words of the model’s own samples.
Prediction. The neutral-drift baseline replaces the trained model by its word frequencies: the synthetic part of each corpus is tokens drawn i.i.d. from the previous corpus’s word counts, and the fresh part is real pool sentences drawn as above. It has no free parameter. If the trigram lineage loses rare words faster than this, its model adds loss beyond sampling; if slower, something re-supplies them.
Results. Trained only on its own samples, the trigram model loses four in five of its rare words in twelve generations, and its perplexity on human text grows by a factor of 7.7 on Darwin and 4.1 on Melville (Table 1); a sample from generation 0 reads “i cannot see that a genus become fitted for most justly celebrated elder huber, but in all the large stock”, and one from generation 12 “struggle of the on that may inheritable detected i hybrids, kept.” Any fresh fraction halts the decline. Neutral drift predicts every trajectory, with no parameter: the largest error at generation 12 is 0.007, and at any generation 0.028, over all conditions and both corpora (Figure 1a,Figure 1b). The one-rate chain of the first version errs by up to 0.17 during the transient, and its accurate results rest on two errors that cancel: it assumes that 10%, 30%, and 50% fresh data alone supply 0.10, 0.30, and 0.50 of the rare words, whereas they supply 0.15, 0.40, and 0.60 on Darwin (0.13, 0.36, 0.57 on Melville; Table 1), and it underestimates what copying keeps. For a trigram model, then, collapse in a fixed world is the loss of rare variants by sampling, which fresh data replace as migration does: Henrich’s lossy transmission, not Rogers’ staleness. Keeping every past corpus avoids collapse, as Gerstgrasser et al. [20] found; in these terms the original corpus is a source of migrants that never leaves.
Figure 1. A fixed world is lossy transmission. Rare-word coverage of the training corpus relative to generation 0 (solid; mean of 5 seeds), the zero-parameter neutral-drift (Wright–Fisher) prediction (dashed), and the one-rate chain of the first version of this paper, its rate fitted to the all-synthetic condition (dotted). (a, b) Word trigram lineages. (c) OPT-125m on wikitext-2: the pretrained model loses more than drift at the first copy and less afterwards.
| condition | Darwin coverage | Darwin drift | Darwin fresh only | Darwin perplexity | Melville coverage | Melville drift | Melville fresh only | Melville perplexity |
| all synthetic | 0.20 | 0.20 | – | 7.7 | 0.18 | 0.18 | – | 4.1 |
| 10% fresh | 0.49 | 0.49 | 0.15 | 2.8 | 0.47 | 0.47 | 0.13 | 2.1 |
| 30% fresh | 0.75 | 0.75 | 0.40 | 1.6 | 0.73 | 0.73 | 0.36 | 1.4 |
| 50% fresh | 0.86 | 0.86 | 0.60 | 1.3 | 0.87 | 0.87 | 0.57 | 1.2 |
| accumulate all past data | 1.00 | – | – | 1.3 | 1.00 | – | – | 1.4 |
| filter: own likelihood | 0.24 | – | – | 7.9 | 0.21 | – | – | 4.6 |
| filter: reference likelihood | 0.02 | – | – | 11.2 | 0.01 | – | – | 4.9 |
| filter: n-gram match | 0.03 | – | – | 107.5 | 0.02 | – | – | 34.7 |
| filter: random half | 0.20 | – | – | 7.7 | 0.18 | – | – | 4.2 |
| filter: favour rare words | 0.56 | – | – | 12.3 | 0.44 | – | – | 7.3 |
Table 1. Word trigram lineages, generation 12, mean of 5 seeds. Coverage: rare-word coverage relative to generation 0. Drift: the zero-parameter neutral-drift prediction. Fresh only: coverage of the fresh part alone. Perplexity: growth factor on held-out human text.
A neural replication. To check that the pattern is not a property of count-based models, we repeated four conditions with a small character-level transformer (2 layers, width 128, context 128 characters) trained afresh each generation on 60 000 characters of Darwin’s text and sampled at temperature 1 (6 generations, 3 seeds; Figure 2). The qualitative picture is the same. Under fully synthetic training, error on held-out human text rises at every generation (1.77 more bits per character by generation 6), and rare-word coverage falls to 4% of its starting value; with 30% fresh data, both settle within two generations (coverage 0.41 of its starting value at generation 6, compared with 0.04; Welch , Cohen’s ). The loss is front-loaded: a character model sampled at temperature 1 misspells most rare words in its first generation, so its copying channel is far less faithful than word sampling. Filters are discussed in § 8. With three seeds per condition the rank-based tests cannot reach (Mann–Whitney – for every comparison); we report them for completeness.
Figure 2. Recursive training of a small character-level transformer (Darwin; 6 generations; mean ± s.d. over 3 seeds). (a) Bits per character on held-out human text. (b) Rare-word coverage of the training corpus, relative to generation 0. Fresh data stop the decline; keeping the most likely half of the samples accelerates it relative to keeping a random half.
A pretrained language model. We repeated the static experiment in the setting of Shumailov et al. [43]: OPT-125m (Zhang et al. [49]), fine-tuned afresh from its pretrained weights at every generation (2 epochs) on 4 000 blocks of 64 tokens from wikitext-2 (Merity et al. [34]), with each generation’s synthetic blocks sampled from the start token at temperature 1, so no human text enters through prompts (10 generations, 5 seeds per condition; Figure 3). Trained only on its own output, the lineage doubles its perplexity on human text (47 to 102) and keeps 0.59 of its rare words; with 10%, 30%, and 50% fresh blocks, perplexity levels off at 68, 56, and 52 and coverage at 0.69, 0.81, and 0.89 (30% fresh against fully synthetic at generation 10: coverage difference 0.23, Welch , Mann–Whitney , Cohen’s , bootstrap 95% CI [0.21, 0.24]). Here neutral drift does not suffice (Figure 1c). With 30% and 50% fresh data it predicts the plateau to within 0.01 and 0.04, but the lineage departs from it in both directions: it loses more at the first copy (0.75 against 0.78 retained), and, trained only on its own output, it keeps far more by generation 10 (0.59 against 0.33). Something outside the corpus re-supplies rare words. The obvious candidate is the pretrained weights, which every generation in this design starts from: in Henrich’s terms a source of innovation that does not pass through the copying chain. We test this directly in § 6.3.
Figure 3. Recursive fine-tuning of a pretrained language model (OPT-125m on wikitext-2; 10 generations; mean ± s.d. over 5 seeds). (a) Rare-word coverage of the training corpus, relative to generation 0, with the two-class chain of Appendix B (dashed). (b) Perplexity on held-out human text. (c) Distinct word trigrams per 1 000 words of the model’s own samples. Keeping the samples the model finds most likely collapses the lineage; keeping a random half does nothing.
What is new here. Three fixed-world findings confirm prior work: fresh data stop the decline (Alemohammad et al. [1]; Shumailov et al. [43]), accumulating data prevents collapse (Gerstgrasser et al. [20]), and likelihood-based selection speeds the loss of diversity (Alemohammad et al. [1]; Ferbach et al. [17]). What the cultural-evolution reading adds is the identification of the mechanism with a zero-parameter baseline, which turns “how much fresh data is needed” into a computation on word frequencies for count-based models, and a way to see when a model departs from it: the pretrained transformer departs in the direction of a reserve outside the chain.
A changing world: Rogers’ paradox
Rogers’ paradox requires a world that changes. We let it change in two ways: the text people write moves on, and facts change their value.
Text that moves on
Setup. The world at generation is a window of 3 000 consecutive sentences in the concatenated text of Darwin’s and Melville’s books, which slides forward by a fixed number of sentences per generation, wrapping around; new vocabulary therefore keeps arriving and old vocabulary keeps leaving. Fresh data are drawn from the current window, with replacement, and every tenth sentence of the window is held out. Each generation is trained on 30 000 tokens, a fraction fresh and the rest sampled from the previous model. Coverage is measured against the current window’s rare words, relative to a corpus of the same size drawn entirely fresh from the current window, and averaged over generations 6 to 12, after the initial transient (3 seeds).
Results. At every fresh fraction, coverage falls as the world changes faster (Figure 4a): with 30% fresh data, from 0.81 in a static world to 0.70 when the window is replaced every generation; with 10%, from 0.54 to 0.46. Equivalently, the fresh fraction needed to retain 80% of what a fully fresh corpus would cover rises from about 0.30 to about 0.45 (Figure 4b). The cause is measurable: the share of the current window’s rare words that were absent from the previous window rises from 0 to 0.09, 0.21, 0.40, and 0.56 as the window slides by 500 to 3 000 sentences, and the copied part of the corpus cannot contain them. But this is not yet Rogers’ mechanism. Neutral drift on the moving window, again with no parameter, matches the trigram lineage’s late coverage in all 15 combinations of speed and fresh fraction to within 0.009: new words are missing, not contradicted. Text that moves on adds vocabulary; it rarely makes what was copied false. The same holds for OPT-125m: on wikitext-2 windows of 6 000 blocks sliding by up to the whole window per generation, coverage of the current window’s rare words is the same whether the world stands still or is replaced (with 30% fresh data, 0.698 against 0.700, Welch ; 5 seeds), and perplexity on the current world rises only slightly (46.0 against 45.0).
Figure 4. Recursive training in a world that keeps changing (word trigram models; the world is a window of 3 000 sentences sliding through the concatenated corpora; mean of generations 6–12 and 3 seeds; bars: s.d. over seeds). (a) Coverage of the current world’s rare words, relative to a fully fresh corpus, against the share of the world replaced per generation. (b) The fresh fraction needed to keep 80% of fully fresh coverage rises from about 0.30 in a static world to about 0.45 when the world is replaced every generation.
Facts that change
Setup. To test Rogers’ mechanism we need copies that become false. We built a world of 2 000 fictional people (random pronounceable names, so that the pretrained model has no prior about them), each living in one of 50 cities whose names are single tokens. Human text at generation states the current truth in 16 paraphrase templates (“X lives in Y.”, “Y is home to X.”, “X took the train back to Y.”, . . .), with people mentioned at Zipf frequencies (exponent 1), so that the 200 most mentioned (head) appear hundreds of times per corpus, the next 800 (mid) a few times, and the last 1 000 (tail) about once. Between generations, each fact changes to another city with probability , which we set. Lineages are as before: OPT-125m, 4 000 blocks of 64 tokens per generation, a fraction fresh (rendered from the current truth) and the rest sampled from the previous model, 10 generations (Appendix A). We use two designs. In restart lineages every generation fine-tunes the pretrained weights, as above; in inherit lineages every generation continues from the previous generation’s weights. The grid is , , both designs, and a fully fresh lineage for each , with 3 seeds (105 lineages).
Measures. We ask the model “X lives in” and three other forward templates and average its next-token distribution over the 50 cities. Accuracy: the most probable city is the current one. Stale answers: the most probable city was this person’s city at an earlier generation and is not now. Both are properties of the model, measured on all 2 000 people, and are averaged over generations 6 to 10.
Results. In a static world () a restart lineage loses most of what a fully fresh lineage knows (with 30% fresh data, 0.19 against 0.55; Table 2), and it gives no stale answers: this is lossy transmission, now at the level of the model. As facts start to change, accuracy falls and stale answers rise in every combination of design and fresh fraction (Figure 5a,Figure 5b), while the fully fresh lineage stays near 0.53. The answers lost are not blanks: with 30% fresh data and , a restart lineage names an outdated city for 8% of people and an inherit lineage for 21% (Welch ), against at most 3% for the fully fresh lineage. This is Rogers’ mechanism: copies are faithful reports of a world that no longer exists, and they cost the lineage in proportion to how fast the world moves.
| 10% fresh | 30% fresh | 50% fresh | fully fresh | ||||
| restart | inherit | restart | inherit | restart | inherit | ||
| 0 | .11 [.00] | .36 [.00] | .19 [.00] | .65 [.00] | .32 [.00] | .81 [.00] | .55 |
| 0.05 | .10 [.02] | .29 [.05] | .20 [.02] | .55 [.05] | .31 [.01] | .74 [.04] | .53 |
| 0.1 | .08 [.04] | .24 [.09] | .18 [.03] | .50 [.08] | .29 [.02] | .68 [.07] | .53 |
| 0.2 | .06 [.06] | .16 [.15] | .15 [.05] | .40 [.14] | .27 [.04] | .59 [.12] | .52 |
| 0.4 | .04 [.10] | .09 [.20] | .12 [.08] | .28 [.21] | .23 [.07] | .46 [.19] | .53 |
Table 2. Fact world: accuracy on all 2 000 facts, generations 6–10, mean ± s.d. over 3 seeds; in brackets, the share of stale answers.
Figure 5. Rogers’ paradox at the level of the model (OPT-125m in a world of 2 000 fictional facts; generations 6–10; mean ± s.d. over 3 seeds). (a) Accuracy against the rate at which facts change; solid: inherit weights, dashed: restart from pretrained weights; dotted: fully fresh lineage. (b) Answers that name a city that was true earlier and is false now. (c) Restart lineages: Rogers’ chain calibrated on the static world () predicts accuracy and stale answers for with set (each point: one fresh fraction, frequency class, seed, and ; mid and tail facts).
Prediction. For restart lineages we calibrated and of Eq. (1) on the static lineage of each fresh fraction, frequency class, and seed, and then predicted every with set and nothing fitted (Eq. (1) for accuracy, Eq. (2) for stale answers; 72 cells for mid and tail facts). For mid-frequency facts the mean absolute error is 0.023 for accuracy (largest 0.060) and 0.015 for stale answers (largest 0.052); for tail facts, 0.009 (largest 0.026) and 0.025 (largest 0.066) (Figure 5c). For head facts the chain cannot be calibrated: their static accuracy barely declines over ten generations, so and are not separately identifiable, and predictions from such fits are off by up to 0.7. The chain describes inherit lineages poorly (§ 6.3).
Where knowledge is kept: pretrained, inherited, copied
The reviewers of the first version asked whether the insensitivity of the OPT lineage to a moving world was due to its pretrained weights, and suggested testing text written after pretraining and lineages that inherit their weights. We did both.
Text after the pretraining cut-off. We collected 20 700 abstracts of arXiv cs.CL papers, 450 per month from January 2023 to October 2026, all written after OPT’s pretraining data (Zhang et al., [49]), in submission order (77 904 blocks of 64 tokens). The world is a window of 6 000 blocks that either stands still or is replaced at every generation, so that by generation 10 the lineage is trained on text from more than three years later. Lineages restart or inherit as in the fact world, with 30% fresh data (10 generations, 4 seeds). Because coverage is a corpus measure, we measure the model directly: the negative log-likelihood (NLL) of the current window’s rare words (outside its 1 000 most frequent) where they occur in held-out human text, with the 1 000 most frequent words as a control, each reported as the excess over a fully fresh lineage on the same window and seed.
Results. The model, not only its corpus, loses rare words first: in late generations a restart lineage’s excess NLL is 0.48 nats per rare word against 0.21 per frequent word (difference 0.27, Welch , Cohen’s $d = 20; Figure 6). Inheriting the weights makes it worse, not better: 0.83 nats (difference to restart 0.35, Welch , bootstrap 95% CI [0.32, 0.37]), still rising by 0.07 nats per generation in every seed, while restart lineages have levelled off (0.007 per generation). So the pretrained weights are a reserve against lossy transmission, as the wikitext lineage suggested: a lineage that starts from them every generation keeps a floor, and one that carries its own weights forward erodes it, the forgetting that continual learning studies [18, 31]. But replacing the world at every generation, with text the pretrained model has never seen, still adds little: 0.04 nats for restart lineages () and for inherit lineages. The insensitivity to a moving world therefore does not come from the reserve being up to date; it comes from text drift itself, which, as for trigrams, adds what is new without making what was copied false.
Figure 6. OPT-125m on arXiv cs.CL abstracts written after its pretraining (2023–2026; 30% fresh; mean s.d. over 4 seeds). Excess negative log-likelihood of held-out human text over a fully fresh lineage on the same window: (a) rare words, (b) frequent words. Restart lineages level off; inherit lineages keep losing. Replacing the world every generation (dashed) adds little.
Inheritance in the fact world. Where facts do change, inherited weights behave as individual memory does in Rogers’ theory. In a static world they are an asset: inherit lineages accumulate facts across generations and, with 30% or 50% fresh data, exceed even the fully fresh lineage (0.65 and 0.81 against 0.55; Table 2), because each generation adds what it reads to what it already holds. As the world changes, the asset turns into stale knowledge: inherit lineages lose accuracy four times as fast as restart lineages (slope against per unit of with 30% fresh data, consistent across seeds), give two to four times as many stale answers, and their advantage over restart lineages shrinks from 0.46 at to 0.17 at . This is also why Eq. (1) fits them poorly: inherited weights are not a text-copying channel with a fixed fidelity, and fresh data overwrite old values in place. The three places where a lineage can keep knowledge therefore behave as § 4 predicts (P3): pretrained weights restored at every generation hold knowledge that copying neither loses nor outdates, but they hold nothing about a new world; inherited weights keep both what is still true and what no longer is; and text copied between generations loses knowledge in a fixed world and outdates it in a changing one.
Closing the loop: who supplies fresh data?
The analyses above, and the literature, treat fresh data as a resource outside the system. It is not. Fresh text is written by people, and people who can ask a model instead of finding something out will often do so. We now couple the two loops. The results of this section hold within the model stated here; they are derived and checked by simulation of the same model, and neither the equilibria nor the benefit of checking have been tested with trained models or real users.
Model. Knowledge consists of many independent facts, each of which changes to a new value with probability per generation, where a generation is one model release. A fraction of people learn each fact they need for themselves, at cost , and are always right; the rest ask the current model, at negligible cost, and are right with probability , the model’s accuracy at the time of use. What people write becomes the training corpus for the next model, together with a share of text the model writes directly. Learners write what they found; askers write what they were told. Text copied from the model keeps a fact with probability (lossy transmission; in the first version of this paper). People adjust their strategy toward whichever pays more. We assume probability matching: the model states a fact correctly with probability equal to the share of its corpus that states it correctly. We relax this, and the assumption of equal costs, below.
Proposition 1. Let be the fresh fraction of the corpus. The model’s accuracy at use obeys
If at equilibrium, then
where is mean payoff. An interior equilibrium exists only while . For , .
Proof. Askers earn and learners ; an interior equilibrium of payoff-driven strategy change requires . At the fixed point of the accuracy recursion, ; setting it to and solving for gives . Mean payoff is .
Corollary 1. In a fixed world () with lossy copying (), the fresh fraction at equilibrium is . Someone must keep paying to learn even when nothing changes.
The corollary answers the question of the title for both mechanisms: in a fixed world people pay to replace what copying loses, in a changing world also to replace what copying outdates, and the two costs add (Figure 8a). In both, the result is Rogers’ indifference condition: whatever the model’s architecture or data volume, its accuracy at use is set by the cost of the alternative it displaces.
Figure 8. Extensions of the ecosystem model (, , unless varied). (a) Fresh fraction at equilibrium against the rate of change, for copying fidelities (lines: Eq. (3); dots: dynamics); with it is positive even in a fixed world. (b) Heterogeneous costs: payoff gain from the model by a person’s cost of finding out. (c) A model that states the majority of its corpus keeps but needs more fresh data than one that matches probabilities. (d) Learners, askers, and checkers: mean payoff at equilibrium against the cost of checking (line: Proposition 2; dots: four starting mixtures).
Proposition 1 has three consequences for machine learning, within this model. First, at equilibrium the model is exactly as reliable as it is costly for people to find out for themselves. Second, the population is no better off than it would be without the model: this is Rogers’ paradox for the human-AI ecosystem (with equal costs; see below). Third, text written by models is paid for in human effort. The share of people who must keep learning for themselves rises as with the share of model-written text, and beyond the model is no longer worth asking at all.
Simulation. An agent-based simulation of the same model, with strategy change by payoff-proportional imitation and a small mutation rate, confirms the equilibrium (Figure 7). After a model is introduced, mean payoff first rises, while most people still learn and the model is accurate, and then falls back to as reliance spreads and the model’s corpus fills with copies (Figure 7a). The share of learners matches Eq. (3) across synthetic shares from 0 to 0.6, and the model’s accuracy at use stays at throughout (Figure 7b,Figure 7c). In the simulation each person writes about one random fact, so a fact receives any human-written text in a generation only with probability (0.63 for ), and the fresh fraction is ; hence (Appendix C).
Figure 7. The human-AI knowledge ecosystem, simulated (4 000 facts, 4 000 people, change rate , cost of finding out , checking cost 0.05). (a) Mean payoff after a model is introduced: when people simply ask, payoff rises and then returns to the no-model baseline ; when they check the answer and relearn if it is wrong, it stays well above. (b) Equilibrium share of people who learn for themselves, against the share of model-written text: simulation and Eq. (3) with the coverage correction derived in Appendix § 13. (c) The model’s accuracy when used stays at whatever .
Heterogeneous costs. If people differ in what learning costs them, those with cost below learn and the rest ask, and the indifference condition applies to the marginal learner: , where is the cost of the person indifferent between the two. People who learn are no better or worse off than without the model; people for whom learning is costlier than gain . With costs uniform on (mean 0.3) and , the model’s accuracy settles at 0.81, the marginal cost at 0.19, and mean payoff at 0.84 against 0.70 without the model (Figure 8b). So the strict paradox, no one better off, depends on equal costs; what survives heterogeneity is that the model’s reliability is pinned by the marginal learner, that the gains go to those for whom learning is most costly, and that more model-written text raises the marginal cost and lowers the model’s accuracy (0.84, 0.81, 0.76 at , 0.3, 0.6).
Aggregation. A model need not match probabilities. If it states the majority of what its corpus says about a fact, a fact that changes is answered wrongly until fresh statements of the new value form a majority, so the model is more stubborn. We simulated this per fact (each corpus holds nine statements per fact; Appendix C). The indifference result is unchanged, and , but the fresh fraction needed rises from 0.12 under probability matching (the formula gives 0.123) to 0.32 (Figure 8c). Aggregation does not break the paradox; it raises its price.
Checking, and whether it is stable. If people who ask the model also check its answer against the world, at a cost below that of finding out, and learn for themselves when it is wrong (critical social learning; Enquist et al., [14]), their payoff is , and what they write is correct.
Proposition 2. With learners, askers, and checkers, if and can be sustained when everyone checks, the population settles on askers and checkers only, at model accuracy and mean payoff , and this equilibrium is stable; if everyone checking cannot sustain , everyone checks. If , checkers do not outperform learners at , and the equilibrium of Proposition 1 is restored.
The proof is in Appendix C. With and the proposition gives , which matches the agent-based simulation (0.93, Figure 7a) and replicator dynamics from four different starting mixtures for every we tried (Figure 8d). Cheap checking is thus not only a way out of the paradox but a stable one, and its benefit is set by the ratio of checking to learning costs. The implication for the design of AI systems is direct: tools that make it cheap for users to verify an answer against a source are the mechanism that keeps the ecosystem’s knowledge, and the models trained on it, above the paradox.
Filters are social learning strategies
Most pipelines do not train on raw synthetic data; they filter it. In social-learning terms, a filter is a rule for choosing what to copy, and the theory distinguishes three kinds.
A conformist filter prefers what is common. Selecting synthetic samples by their likelihood under the generator, by the perplexity of a reference model, or by overlap with existing text are conformist in this sense: the criterion is resemblance, and rare content is by definition less resembled. Theory predicts that such filters erode variation faster than copying at random ([5]), and the evidence agrees. In image models, sampling biased toward quality preserves precision while accelerating the loss of diversity ([1]), and selection by typicality precipitates collapse, doing worse than random selection ([37]); iterated curation by any fixed score concentrates the model on the highest-scoring samples ([17]). Our text results are consistent (Table 1). For the trigram model, filtering by the generator’s own likelihood is indistinguishable from discarding at random; for the transformers, whose likelihood ranks whole lines more sharply, it is clearly worse than random and accelerates the loss of rare words. In the character transformer, keeping the most likely half of the sampled lines gives 5.18 against
3.93 bits per character at generation 6 for a random half (Welch , , bootstrap 95% CI of the difference [0.85, 1.52]; Figure 2); in the pretrained model it lowers rare-word coverage to 0.05 against 0.57 for a random half (Welch , Mann–Whitney , $d = 24) and the variety of its own samples to 267 distinct trigrams per 1 000 words against 940 (Figure 3). A random half is indistinguishable from no filter in every model. Filtering by a reference model’s likelihood, or by matching human n-grams, loses nearly every rare word within twelve generations, degrades perplexity on human text more than doing nothing, and sharply reduces the variety of the model’s own output. A filter that inverts the bias, favouring rare words, preserves more of them but degrades perplexity faster: it keeps rare words by keeping the errors around them.
A payoff-biased filter keeps what scores best on a task, such as outputs that a reward model rates highly or that pass a check. It concentrates the population on the current best and, like selection, reduces variation even when it improves average quality; iterated self-improvement narrows output diversity even when it preserves correctness ([6, 17]), and verifiers can raise a model only to the level of the verifier ([47]). We did not test payoff-biased filters ourselves.
A critical filter tests a copy against the world and replaces it when it fails. Verification against ground truth ([16]), correction toward an expert ([21]), and training on one’s own reasoning only when it reaches a verified answer ([48]) belong here. We have not yet tested a critical filter in our own lineages; the evidence for this class comes from the work cited. The distinction that matters is not how much a filter discards but where its criterion gets its information: from resemblance to what the system already has, or from the world.
Two results complicate the picture and are worth reconciling. Selecting synthetic text with a detector of machine-generated text can prevent collapse ([13]): such a detector penalises the over-represented features of model output, which makes it closer to an anti-conformist filter than to the likelihood filters above, and our rare-favouring filter shows the same partial benefit. And selecting synthetic data by n-gram similarity to human text had little effect in a setting that mixes synthetic with human data at the token level ([50]); in our fully recursive setting the same criterion, applied every generation, compounds.
Discussion
What the reading buys. The statistical theory of collapse is more detailed than Eq. (1) wherever both apply. What cultural evolution adds is a separation of two mechanisms that collapse analyses merge, each with its own signature: lossy transmission removes knowledge and is predicted by drift on frequencies; staleness replaces it with outdated values and scales with the rate of change. It also adds a frame in which the supply of fresh information is a variable, with a price that can be computed in both regimes, and a taxonomy in which the effect of a filter can be predicted from where its criterion gets its information.
Predictions. The analysis makes predictions that larger experiments could refute: in a fixed world, the coverage a lineage keeps should follow drift on its frequencies, with departures pointing to knowledge held outside the chain; in a changing world, models should give outdated answers in proportion to the rate of change of a domain and to their reliance on copied text, so that fast-changing domains degrade first as synthetic text accumulates; lineages that carry their weights forward should hold more stale facts than lineages that restart from fixed pretrained weights; likelihood- and typicality-based filters should accelerate the loss of rare knowledge at scale as they do in small models; and, in deployed systems, the share of users who verify answers should govern the long-run reliability of models trained on what users write.
One class or two. The appendix compares the one-rate chain of the first version with a chain of two classes of knowledge, fitted to fully synthetic training (Table 3). Two classes describe the all-synthetic trajectories better, yet predict the fresh-data plateaus of trigram lineages worse than one class. The drift baseline explains why: how easily a word is lost depends on how many copies of it a corpus holds, which fresh data change, so any chain with fixed per-class rates fitted to one mixture misstates the others. For trigram lineages we therefore rely on the drift baseline and report the chains only for comparison.
Table 3: One-class and two-class chains of the first version, fitted to fully synthetic training only. : the form preferred by the held-out generations of the all-synthetic condition. For trigram lineages the zero-parameter drift baseline has prediction errors of at most 0.007 (final) and 0.028 (any generation).
| setting | form | parameters | held-out error | prediction error final | prediction error any generation |
| trigram, Darwin | one | 0.115 | 0.032 | 0.107 | |
| trigram, Darwin | two* | , , | 0.029 | 0.081 | 0.091 |
| trigram, Melville | one | 0.111 | 0.029 | 0.120 | |
| trigram, Melville | two* | , , | 0.035 | 0.082 | 0.092 |
| character transformer | one | 0.049 | 0.072 | 0.230 | |
| character transformer | two* | , , | 0.002 | 0.037 | 0.054 |
| OPT-125m | one | 0.253 | 0.058 | 0.159 | |
| OPT-125m | two* | , , | 0.039 | 0.030 | 0.044 |
Table 3.
Limitations. The language models are small, the largest having 125 million parameters, chosen so that knowledge could be counted or probed exactly. Rare-word coverage is a corpus measure; we added model-level measures (rare-word likelihood, factual accuracy and stale answers) but only for OPT-125m. The fact world is synthetic and simple: one attribute per person, uniform changes, and facts stated in a fixed set of templates; real knowledge is correlated and changes unevenly. Several experiments use three or four seeds; their between-seed variation is small relative to the effects reported, but rank-based tests at these sizes have limited power. The ecosystem model treats facts as independent and strategies as population-level averages; we relaxed equal costs, probability matching, and the set of strategies, but not the assumption that the next model is trained on what people write, and we have not tested it with real users. Our filter experiments cover conformist filters only; payoff-biased and critical filters are supported here by prior work, not by our own lineages.
A wider correspondence. The identification is one instance of a broader parallel between the layers of modern training – pretraining, fine-tuning, adaptation in context, and training on other models’ output – and the layers of biological adaptation – evolution, individual learning, and social learning [44, 5, 25]. The parallel is not one of mechanism; it is that both are stacks of adaptive processes with different spreads and costs that must divide the work of keeping up with a changing world, and results that depend only on rates, costs, and channels transfer between them. The pretrained weights restored at every generation play the part of a slow layer that holds what the fast layers keep losing, and the inherited weights the part of individual memory that keeps what has since become false.
Conclusion
Read through cultural evolution, model collapse is two old problems met at industrial scale: copies that lose what they copy, and copies of a world that has moved on. Cultural evolution’s answer was never to stop copying: it was to keep learners in the population and to check copies against the world. Taking that answer seriously changes the question machine learning asks about collapse. The question is not only how much fresh data a model needs, but who produces it, what it costs them, and whether the systems built on their work leave them any reason to keep producing it.
Code and data. Code for every experiment and figure, and the per-seed results, are available at https://github.com/drnamlabs-gif/rogers-model-collapse.
Experimental details
Word trigram models. Sentences of each book (Project Gutenberg; lower-cased, split into sentences at full stops, question and exclamation marks, tokenised into words and punctuation marks, and kept if 4 to 100 tokens long) are divided at random into a test set (10%), a reference set (20%, used only by the reference-likelihood filter), and a pool (70%). Each model interpolates trigram, bigram, and unigram estimates with weights 0.6, 0.3, and 0.1; the unigram is add-0.01 smoothed. Sentences are sampled word by word from the interpolated distribution until the end token; samples shorter than four tokens are discarded. Each generation is trained on 30 000 tokens, for 12 generations and 5 seeds. Rare words are the pool’s words outside its 1 000 most frequent. Filters sample 60 000 tokens and keep sentences, in order of their score, up to 30 000 tokens. Fresh data. At every generation the pool is shuffled with the seed’s random generator and whole sentences are taken in order until the token budget is met. Draws are therefore without replacement within a generation and independent across generations: a fresh sentence may have appeared in an earlier corpus, including generation 0. Repeated draws do not inflate coverage relative to the fully fresh reference, which is drawn the same way. In the moving world, fresh sentences are drawn with replacement from the current window. Neutral drift. The synthetic part is drawn as a multinomial sample of tokens from the previous corpus’s token counts; the fresh part is drawn exactly as for the trained lineages.
Character transformer. A 2-layer transformer encoder with causal masking (width 128, 4 heads, feed-forward width 512, dropout 0.1, context 128 characters, alphabet of 35 characters), trained from scratch each generation with AdamW (learning rate , batch 32, 1 000 steps) on 60 000 characters of Darwin’s text; samples at temperature 1; 6 generations, 3 seeds. Filters sample twice the corpus and keep half of the lines, by likelihood under the current model or at random.
OPT-125m. The pretrained facebook/opt-125m is fine-tuned afresh from its pretrained weights at every generation, for 2 epochs with AdamW (learning rate , batch 32, bfloat16 autocast), on 4 000 blocks of 64 tokens. The human pool is the wikitext-2 (raw) training split with headings removed, cut into consecutive 64-token blocks; perplexity is measured on the first 600 blocks of the test split. Synthetic blocks are sampled from the start token with pure sampling (temperature 1, no top- or nucleus truncation), exactly 64 new tokens each. Fresh blocks are drawn without replacement within a generation and independently across generations. The likelihood filter samples 8 000 blocks and keeps the 4 000 with the lowest mean per-token negative log-likelihood under the current model; the random filter keeps a random 4 000. Rare words are the pool’s words outside its 1 000 most frequent; variety is measured on 1 000 blocks sampled from each generation’s model. 10 generations, 5 seeds.
arXiv lineages. 20 700 abstracts of arXiv cs.CL papers (the first 450 listed per month, January 2023 to October 2026), in submission order, cut into 77 904 blocks of 64 tokens. The world at generation is the window of 6 000 blocks starting at times the speed (0 or 6000); every tenth block is held out (600 for evaluation) and fresh blocks are drawn without replacement from the rest. Training is as for OPT-125m above; inherit lineages continue from the previous generation’s weights with a new optimizer. Rare and frequent words are defined on the current window’s training blocks. A word’s NLL is the sum of the NLL of its tokens in held-out human blocks (the first word of each block is skipped, as it may be cut). Excess NLL is the difference to a lineage with fully fresh data on the same window and seed. 10 generations, 4 seeds.
Fact world. 2 000 fictional names of two parts, each of two or three syllables from a fixed set; 50 city names that are single tokens for OPT’s tokenizer. Human text renders people drawn with probability proportional to (rank ) with one of 16 templates. Probes: “X lives in”, “X moved to”, “X is based in”, “X resides in”; the model’s next-token probabilities are renormalised over the 50 cities and averaged over probes. Training as for OPT-125m but for 3 epochs at learning rate , chosen on a fully fresh generation 0 so that head, mid, and tail facts span high, middle, and low accuracy (0.99, 0.65, 0.31). Changes: each fact moves to a uniformly chosen other city with probability between generations. Coverage of a fact in a corpus (reported in the code) is whether some sentence states the person with the current city. Calibration of (1): least squares on the static lineage’s accuracy trajectory for each fresh fraction, frequency class, and seed, , starting from the observed generation-0 accuracy. 10 generations, 3 seeds; one lineage takes about 3 minutes on one consumer GPU.
Statistics. The unit of analysis is the seed (one lineage). We report Welch’s -test, the Mann–Whitney test, Cohen’s with pooled standard deviation, and a percentile bootstrap 95% confidence interval of the difference in means (10 000 resamples).
One-rate and two-class chains
These are the chains of the first version of this paper, kept for comparison (§ 9). Each form is fitted by least squares on log coverage to the mean relative coverage of the all-synthetic condition, and applied unchanged to the fresh-data conditions with , using . The two-class form is . To choose between forms without using any fresh-data run, each is also fitted to the first half of the all-synthetic generations and scored on the second half (held-out error, maximum absolute error). The prediction errors are the largest absolute differences between predicted and observed coverage over the fresh-data conditions, at the final generation and over all generations.
Derivations for the ecosystem model
Coverage correction in the simulation. In the agent-based simulation each of people writes about one fact drawn uniformly from . A fact receives at least one human-written statement in a generation with probability ; a fact that receives none keeps the previous model’s value. Of the facts that do receive one, a share enter the corpus from people, and the last writer is a learner with probability . The fresh fraction per fact is therefore , and Proposition 1 gives . For , .
Proof of Proposition 2. Payoffs are , , and . Checkers and learners both write the truth, so the fresh fraction is , and accuracy follows Proposition 1’s recursion, increasing in . Askers and checkers earn the same when , that is at . At , , which is positive for (then for ), so learners cannot invade; mean payoff is . Along the asker-checker edge, decreases in , and increases with the share of checkers, so a perturbation toward more checkers raises and makes checking pay less, and vice versa: the equilibrium is stable. If even cannot reach , that is if , checkers always earn more than askers, everyone checks, and mean payoff is . If , then at checkers earn no more than learners and the population returns to the learner-asker equilibrium of Proposition 1.
Numerical checks. Mean-field dynamics of and the strategy shares, with replicator updates (step 0.5) and mutation , run for 6 000 generations, match of Proposition 1 to three decimals for and , and of Proposition 2 to within 0.001 from four starting mixtures for from 0 to 0.25. Heterogeneous costs use 61 cost groups uniform on , each with its own learner share. The aggregation check simulates 4 000 facts, each with nine statements per corpus; after a change all statements of the fact are false; the model states either a random statement (probability matching) or the majority.
References
- [1]Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Shiahkoohi, and Richard G. Baraniuk. Self-consuming generative models go MAD. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2307.01850
- [2]Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.1, knowledge storage and extraction. arXiv preprint arXiv:2309.14316, 2023.
- [3]Fabian Baumann, Erol Akçay, and Joshua B. Plotkin. Generative artificial intelligence reduces social welfare through model collapse. arXiv preprint arXiv:2604.21853, 2026.arxiv.org/abs/2604.21853
- [4]Quentin Bertrand, Avishek Joey Bose, Alexandre Duplessis, Marco Jiralerspong, and Gauthier Gidel. On the stability of iterative retraining of generative models on their own data. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2310.00429
- [5]Robert Boyd and Peter J. Richerson. Culture and the Evolutionary Process. University of Chicago Press, Chicago, 1985.
- [6]Martin Briesch, Dominik Sobania, and Franz Rothlauf. Large language models suffer from their own output: An analysis of the self-consuming training loop. arXiv preprint arXiv:2311.16822, 2023.arxiv.org/abs/2311.16822
- [7]Levin Brinkmann, Fabian Baumann, Jean-François Bonnefon, et al. Machine culture. Nature Human Behaviour, 7:1855–1868, 2023.
- [8]Katherine M. Collins, Umang Bhatt, and Ilia Sucholutsky. Revisiting Rogers’ paradox in the context of human–AI interaction. Philosophical Transactions of the Royal Society A, 384(2320):20250149, 2026.
- [9]Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10:257–273, 2022.
- [10]Elvis Dohmatob, Yunzhen Feng, and Julia Kempe. Model collapse demystified: The case of regression. In Advances in Neural Information Processing Systems (NeurIPS), 2024a.arxiv.org/abs/2402.07712
- [11]Elvis Dohmatob, Yunzhen Feng, Pu Yang, François Charton, and Julia Kempe. A tale of tails: Model collapse as a change of scaling laws. In International Conference on Machine Learning (ICML), 2024b.arxiv.org/abs/2402.07043
- [12]Elvis Dohmatob, Yunzhen Feng, Arjun Subramonian, and Julia Kempe. Strong model collapse. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2410.04840
- [13]George Drayson, Emine Yilmaz, and Vasileios Lampos. Machine-generated text detection prevents language model collapse. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025.
- [15]Warren J. Ewens. Mathematical Population Genetics I: Theoretical Introduction. Springer, New York, 2nd edition, 2004.DOI
- [16]Yunzhen Feng, Elvis Dohmatob, Pu Yang, François Charton, and Julia Kempe. Beyond model collapse: Scaling up with synthesized data requires verification. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2406.07515
- [17]Damien Ferbach, Quentin Bertrand, Avishek Joey Bose, and Gauthier Gidel. Self-consuming generative models with curated data probably optimize human preferences. In Advances in Neural Information Processing Systems (NeurIPS), 2024.arxiv.org/abs/2407.09499
- [18]Robert M. French. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4):128–135, 1999.
- [19]Tommaso Furlanello, Zachary C. Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born again neural networks. In Proceedings of the 35th International Conference on Machine Learning (ICML), pp. 1607–1616, 2018.arxiv.org/abs/1805.04770
- [20]Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey, Rafael Rafailov, Henry Sleight, John Hughes, Tomasz Korbak, Rajashree Agrawal, Dhruv Pai, Andrey Gromov, et al. Is model collapse inevitable? Breaking the curse of recursion by accumulating real and synthetic data. In Conference on Language Modeling (COLM), 2024.
- [21]Nate Gillman, Michael Freeman, Daksh Aggarwal, Chia-Hong Hsu, Calvin Luo, Yonglong Tian, and Chen Sun. Self-correcting self-consuming loops for generative model training. In International Conference on Machine Learning (ICML), 2024.
- [22]Dongxin Guo, Jikun Wu, and Siu Ming Yiu. Model collapse as cultural evolution. arXiv preprint arXiv:2605.23054, 2026.arxiv.org/abs/2605.23054
- [23]Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, and Chloé Clavel. The curious decline of linguistic diversity: Training language models on synthetic text. In Findings of the Association for Computational Linguistics: NAACL 2024, 2024.arxiv.org/abs/2311.09807
- [25]Joseph Henrich. The Secret of Our Success. Princeton University Press, Princeton, 2015.DOI
- [27]Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.arxiv.org/abs/1503.02531
- [28]Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Stanley Jungkyu Choi, and Minjoon Seo. Towards continual knowledge learning of language models. In International Conference on Learning Representations (ICLR), 2022.
- [29]Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. Large language models struggle to learn long-tail knowledge. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp. 15696–15707, 2023.
- [30]Joshua Kazdan, Rylan Schaeffer, Apratim Dey, Matthias Gerstgrasser, Rafael Rafailov, David L. Donoho, and Sanmi Koyejo. Collapse or thrive: Perils and promises of synthetic data in a self-generating world. In International Conference on Machine Learning (ICML), volume 267 of PMLR, pp. 29469–29494, 2025.
- [31]James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017.DOI
- [33]Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, et al. Mind the gap: Assessing temporal generalization in neural language models. In Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021.
- [34]Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations (ICLR), 2017.arxiv.org/abs/1609.07843
- [35]Hossein Mobahi, Mehrdad Farajtabar, and Peter L. Bartlett. Self-distillation amplifies regularization in Hilbert space. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020.arxiv.org/abs/2002.05715
- [36]Andrew J. Peterson. AI and the problem of knowledge collapse. arXiv preprint arXiv:2404.03502, 2024.DOI
- [37]Xiaoyu Qiao, Xiao Du, Wei Liu, Jie Zhang, Peng Mai, Min Zhang, and Yan Pang. When sample selection bias precipitates model collapse. In International Conference on Machine Learning (ICML), 2026.arxiv.org/abs/2606.13732
- [38]Yi Ren, Shangmin Guo, Linlu Qiu, Bailin Wang, and Danica J. Sutherland. Bias amplification in language model evolution: An iterated learning perspective. In Advances in Neural Information Processing Systems (NeurIPS), 2024.arxiv.org/abs/2404.04286
- [39]Luke Rendell, Robert Boyd, Daniel Cownden, Magnus Enquist, Kimmo Eriksson, Marcus W. Feldman, Laurel Fogarty, Stefano Ghirlanda, Timothy Lillicrap, and Kevin N. Laland. Why copy others? insights from the social learning strategies tournament. Science, 328(5975):208–213, 2010.DOI
- [40]Alan R. Rogers. Does biology constrain culture? American Anthropologist, 90(4):819–831, 1988.DOI
- [41]Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, and Sanmi Koyejo. Position: Multiple definitions & unrealistic assumptions of model collapse distract from real world threats. In International Conference on Machine Learning (ICML), Position Track, 2026.
- [42]Mohamed El Amine Seddik, Suei-Wen Chen, Soufiane Hayou, Pierre Youssef, and Merouane Debbah. How bad is training on synthetic data? a statistical analysis of language model collapse. In Conference on Language Modeling (COLM), 2024.
- [43]Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. AI models collapse when trained on recursively generated data. Nature, 631:755–759, 2024.DOI
- [44]David W. Stephens. Change, regularity, and value in the evolution of animal learning. Behavioral Ecology, 2(1):77–89, 1991.DOI
- [45]Ananda Theertha Suresh, Andrew Thangaraj, and Aditya Nanda Kishore Khandavally. Rate of model collapse in recursive training. In International Conference on Artificial Intelligence and Statistics (AISTATS), volume 258 of PMLR, pp. 1396–1404, 2025.arxiv.org/abs/2412.17646
- [46]Sewall Wright. Evolution in Mendelian populations. Genetics, 16(2):97–159, 1931.
- [47]Bingji Yi, Qiyuan Liu, Yuwei Cheng, and Haifeng Xu. Escaping model collapse via synthetic data verification: Near-term improvements and long-term convergence. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2510.16657
- [48]Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. STaR: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022), 2022.arxiv.org/abs/2203.14465
- [49]Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. OPT: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022.arxiv.org/abs/2205.01068
- [50]Xuekai Zhu, Daixuan Cheng, Hengli Li, Kaiyan Zhang, Ermo Hua, Xingtai Lv, Ning Ding, Zhouhan Lin, Zilong Zheng, and Bowen Zhou. How to synthesize text data without model collapse? In International Conference on Machine Learning (ICML), 2025.arxiv.org/abs/2412.14689







