WorksComputer Science
Meta-memorization and memorization scaling laws in transformers
Abstract
Larger transformers can memorize more data, with a capacity (in bits) that scales roughly linearly with the number of parameters. Counterintuitively, larger models also memorize faster, requiring fewer repetitions of a sequence during training before it is memorized. We reproduce both memorization scaling laws in transformers trained on random binary sequences, showing that rapid memorization can arise even when data contain no semantic or compositional structure. Rapid memorization is due to a meta-memorization effect, where models trained to memorize a large set of random sequences learn to memorize novel sequences more quickly. The rate at which models memorize grows with training time as a power-law with a fixed exponent but an increasing upper cutoff across model scales. Meta-memorization enables even one-layer transformers to few-shot memorize a 100-bit sequence. This effect is partly explained by the formation of structured attention heads that convey position-invariant sequence information to MLPs where memorized sequences are stored. Meta-memorization is recapitulated in a simplified model of a dense associative memory where retrieval is constrained by an attentional bottleneck. Our results suggest transformers, through pre-training, acquire reusable mechanisms that enable highly efficient storage and retrieval of memories.
Introduction
Significant attention has been paid to the ability of transformers to generalize with limited data, but transformers are also remarkable memorizers [1, 2]. Large language models (LLMs) can reproduce training passages verbatim when prompted with sufficient context and this can be exploited to extract training data [3, 4, 5]. For example, production language models verbatim recall much of Harry Potter and the Sorcerer’s Stone when prompted with the first few tokens of the first chapter [6, 7].
One influential approach to quantify memorization is based on extractability. Given a prefix of a training example, one can ask whether the model reproduces its exact continuation, and measure how this probability changes with context length, dataset duplication, and scale [4, 8, 9, 10]. A complementary approach measures memorization dynamically during training, asking how many presentations are necessary before examples become exactly reproducible [11, 12].
Several robust features have emerged from these measurements. Multiple analyses of pretrained transformers suggest that MLP layers store associations, consistent with the idea that MLP blocks function as key-value memories [13, 14, 15, 16, 17, 18]. Memorization increases with an LLM’s parameter count and with the number of times a sequence is duplicated in the training corpus, often in simple log-linear relationships [4, 19]. More broadly, recent work shows that transformers’ storage capacity (measured in bits, after controlling for generalization) scales linearly, with a few bits stored per parameter [18, 20, 21]. Training proceeds through regimes in which the model first fills this capacity before shifting toward more reusable structure [22, 23].
Counterintuitively, memorization does not just scale in capacity but also in rate; that is, larger models require fewer presentations of a sentence before it
is memorized [4, 12]. Having more parameters can explain greater capacity, but this does not explain why incremental gradient updates should become systematically more effective at creating a memory. Resolving this puzzle in language models is difficult as memorization rate is inherently tied to the model’s improving ability to exploit the structure of language [24].
Contributions. In this work, we study how transformers memorize random binary sequences that have no internal statistical structure. We reproduce several empirical results in the literature, in particular that both transformers’ memorization capacity and rate increase with size [4, 12], derive a scaling law in the memorization rate as a function of model size, and show that large models can achieve one-shot memorization. Training models to memorize random sequences reduces the number of presentations required to memorize any arbitrary new sequence, an effect we call meta-memorization (in connection with meta-learning). We show that transformers meta-memorize through a functional division of labor, where attention heads convey position-invariant, linearly separable information about sequence identity and the MLPs store dataset-specific memories. Finally, we instantiate our theory of the distinct functional roles of attention and MLPs in a conceptual associative memory model, where attention heads convey information through a constrained bottleneck to the associative memory. The code for reproducing experiments and analysis can be found in the following repository.
Setup
Transformers of different sizes are trained autoregressively on a dataset of randomly generated binary sequences (Figure 1a). The dataset is constructed with a Zipfian rank-frequency distribution (Figure 1b). Specifically, we sample random binary sequences of length and assign each sequence a rank . Each bit in each sequence is sampled i.i.d. from a Bernoulli distribution () (Figure 1a). During training, a sequence of rank is sampled from a Zipfian distribution (Figure 1b). Given the first bits, the model predicts the -th bit under a binary cross-entropy objective.

Figure 1. Experimental setup. We train autoregressive transformers on random binary sequences of length . (a) Four example sequences. (b) Sequences are assigned ranks and sampled from a Zipfian distribution, producing repetition counts spanning orders of magnitude. For illustrative purposes we only plot up to rank . Model size is varied through the number of heads and model dimension .
This design serves three purposes. First, since the sequences are unstructured, it inherently controls for any unintended memorization due to generalization. Second, it mirrors a salient, domain-general feature of natural data that is known to influence emergent transformer behaviors in other contexts [25]. Third, it allows us to ask whether the rate at which a novel (or a less frequent) sequence is memorized depends on the model’s prior experience memorizing other more common sequences.
We train decoder-only transformers with learnable positional embeddings and causal masking. We consider two classes of models: (1) GPT-2 style transformers with depth 12 [26, 27], and (2) one-layer models with one layer of attention followed by a three-layer MLP. In both cases, model size is varied by varying the number of heads () and the per-head width (), which in turn fixes the model dimension (). The 12-layer models are trained for iterations, whereas the 1-layer models are trained for iterations since they converge more quickly. All models are trained with an Adam optimizer (learning rate and batch size 256) [28].
For each sequence, we compute the average next-token prediction accuracy by feeding the model the sequence prefix and comparing the predicted next token to the ground truth. Accuracy is averaged across all positions. A sequence is considered “memorized” when the model achieves 90% average per-token accuracy. For a given model and a sequence of rank , we measured the training iteration at which the model first reached this threshold, and the total number of sequence presentations the model had received up to iteration . The use of both measures allows us to disentangle the effects of training iterations from sequence frequency.
Results
Memorization scaling laws and meta-memorization
For both 12-layer and 1-layer models, the average loss and accuracy across models with different numbers of heads and model dimensions collapse onto a single curve that depends on architecture solely through the total parameter count (Figure 2a, Figure S1). As expected, larger models generally show lower train loss and higher recall of the most frequent sequences. The dependence of loss and accuracy on parameter count was also recapitulated in separate experiments with uniformly-drawn (rather than Zipfian) sequences (Figure S2).

Figure 2. Memorization scaling laws and meta-memorization for 12-layer and 1-layer architectures. (a) For both architectures, final train loss decreases primarily with total parameter count of models even though models vary in both number of heads and model dimension . The 12-layer and 1-layer models are trained for 200,000 and 80,000 iterations, respectively, with a fixed learning rate. Note that the largest models failed to converge with this learning rate and are not shown in the plot. (b) The maximum rank of memorized sequences ( the memorization capacity in bits) shown at 60,000, 120,000 and 190,000 iterations for the 12-layer models and 20,000, 50,000 and 79,000 iterations for the 1-layer model. The exponent of a power law fit is for both architectures and during training, thus showing that memorization capacity scales linearly with parameter count . (c) Larger models memorize sequences faster. Moreover, models memorize data from a second dataset faster after a dataset switch (red dashed line). Due to the rapid learning in 1-layer models, here we show results only for iterations and to better illustrate the effect.

(a)

(b)
Figure S1. Recall accuracy scales with parameter count. Average next-bit accuracy on the top 10000 sequences increases with total parameters across architectures that vary in and .


Figure S2. Train loss and recall accuracy versus total parameter count for 12-layer GPTs trained on binary sequences, sampled uniformly with replacement. Models are trained on sequences with context length 100 over training iterations. We vary the number of attention heads and the number of hidden features per head . Train loss decreases and recall accuracy increases with total parameter count, mostly independently of architecture.
Memorization capacity scaling law. We define memorization capacity () as the total number of bits memorized by the model after convergence. Because models, to a good approximation, memorize sequences sequentially by rank, the maximum memorized rank (which is easily measured) tracks the number of memorized sequences. In all capacity estimates below we therefore use as a proxy for the number of memorized sequences. Since each sequence is 100 bits, we have bits. We measure vs at three different points during training as the loss continues to decline gradually at the end of training (Figure S3). For both model classes, scales with parameter count as (Figure 2b). We find that with a pre-factor and for 1-layer and 12-layer models, respectively. In other words, memorization capacity scales linearly with parameter count, consistent with previous measurements. Our measurement leads to an estimate of bits per parameter.

Figure S3. Training loss dynamics across model scales for 12-layer models trained on Zipfian distributed sequences. Larger models generally reach lower loss, even though the largest models show increased loss compared to mid-size models since we did not explicitly tune our optimization hyperparameters.
Meta-memorization. By measuring the training iteration at which a sequence with rank is memorized, we find that larger models memorize sequences (of a given rank) earlier in training (Figure S4). That is, larger models appear to memorize sequences faster. Re-plotting this data using number of presentations after which a sequence of rank is memorized over all models shows that generally decreases with rank (Figure S4). Under Zipfian sampling, high-rank sequences are seen less frequently and therefore tend to cross the threshold later during training. If they nevertheless require fewer presentations to be memorized, then the effective amount of memory written per presentation must have increased over the course of training. We term this effect meta-memorization, where memorizing a set of sequences allows a model to memorize new sequences more efficiently. We suggest the term meta-memorization to evoke the connection to meta-learning, where training on a task distribution improves model performance on other similar tasks. Our finding replicates [12]’s finding that larger models memorize linguistic data faster, but importantly, we show that this memorization rate scaling law holds even when the training data lack any statistical structure.

Figure S4. Meta-memorization in 12-layer and 1-layer models. (a) Saturation iteration vs. sequence rank for 12-layer and 1-layer models. We define a sequence’s saturation iteration as the iteration the model “memorized” it (i.e. attains average accuracy above 90%). We observe that sequences of higher rank are learned later in time, mainly because they are presented less frequently due to the Zipfian data distribution. However, by taking a vertical slice through our figures, we see that larger models memorize faster, since for any given sequence rank, larger models generally require fewer iterations to memorize it. By taking a horizontal slice through our figures, we see that at any given time, larger models will generally have learned more sequences. (b) Appearances at saturation vs. sequence rank for different model sizes. We define appearances at saturation as the number of presentations the model takes to “memorize” a sequence (i.e. learned to 90% accuracy). We observe that higher-ranked sequences require fewer presentations to be memorized. Interestingly, we observe that the very largest 1-layer models require as few as 2 presentations to memorize a sequence (bottom right).
Data switch experiments. To test this directly, we performed an experiment where the training set of sequences was switched after iterations to a fresh set of random sequences sampled from the Zipfian distribution. Accuracy immediately returned to chance after the switch, but recovered substantially faster than it had at the start of training (Figure 2c). The speedup was visible for sequences across all ranks (see Figure S5 for 12-layer models and Figure S6 for 1-layer models).

Figure S5. Meta-memorization in data-switching experiments for 12-layer models across sequences of different ranks. We switched the training set of sequences after iterations to a fresh set of random sequences drawn from the same Zipfian distribution. For sequences across different ranks, accuracy immediately returned to chance after the switch, but recovered faster relative to the start of training (See Figure 2c).

Figure S6. Meta-memorization in data-switching experiments for 1-layer models across sequences of different ranks. We switched the training set of sequences after iterations to a fresh set of random sequences drawn from the same Zipfian distribution. For sequences across different ranks, accuracy immediately returned to chance after the switch, but recovered faster relative to the start of training (See Figure 2c). Due to the very rapid learning in 1-layer models, here we show results only for iterations and to better illustrate the effect.
Memorization rate scaling law. Next, we estimate how memorization rate evolves during training. Suppose a sequence of rank is sampled with probability
and first crosses the memorization threshold at iteration . Let denote the effective memorization rate at training time in units of bits per iteration. The extent to which sequence is memorized (in bits) at is , where is the rank of the sequence that was presented at iteration . A sequence of bits is memorized when . Let . If , we have
Suppose the sequences are drawn according to a rank-frequency distribution , and that (or equivalently ) for some . Empirically, we find (Figure S4). Under the ansatz for some , we obtain . Combining (1) and these scaling relationships, we get
is estimated by fitting vs to a power law (recall
for the Zipfian rank-frequency distribution). Remarkably, we find that a fixed and fits the data from the 12-layer and 1-layer models, respectively, across all model scales (Figure 3 panels a and b). Model sizes are primarily differentiated by the rank at which stops decreasing (Figure 3 panels a and b). In other words, for all models (with an almost unvarying ), but the range of for which the scaling law applies is larger for larger models. Using Eq. (2), we estimate across the 12-layer and 1-layer model classes (the estimate has a wide range as is close to 1).

Figure 3. Meta-memorization scaling laws. Across models, we find that fewer presentations are required to memorize a rarer sequence (of higher rank) (Figure S4). Since higher-rank sequences tend to appear later on during training, this suggests that models become more efficient at memorizing over the course of training. In panels a) and b), the number of presentations () vs rank () data is first fit to a power-law for each model. We plot the rescaled number of presentations (i.e., after dividing by for each model) vs rank for all models. Across model sizes, 12-layer and 1-layer models display power laws with exponents and , respectively. The exponent is common to all model sizes within a model class, but the scaling extends to higher ranks for larger models. In panels c) and d) we show appearances at saturation vs. sequence rank in data-switching experiments for the largest 1-layer models. Appearances at saturation is the number of presentations at which a sequence is memorized (i.e. learned to 90% accuracy). To avoid visual clutter, we bin the x-axis logarithmically and show the geometric mean of sequence ranks and appearances at saturation within each bin. Error bands show the minimum number of appearances among ranks that fall in each logarithmic bin. We show these pre-switch and post-switch data and observe that after the switch, sequences of equal rank require fewer presentations to be learned. (d) Furthermore, the largest model at parameters was able to learn some sequences in one-shot after the switch (the error band reaches 1 appearance after the switch).
Thus, the memorization rate accelerates during training. In the extreme case, the largest 1-layer models are able to memorize a novel sequence after a single presentation of the sequence (Figure 3 panel d).
The division of labor between attention heads and feedforward blocks
The one-layer architecture allows for a deeper analysis of the differential roles of attention and feed-forward components in memorization, while still capturing the essential phenomenology. We perform a series of experiments motivated by previous work on factual recall that suggests facts are localized in feedforward blocks (MLPs) [13, 14, 17].
Component swapping. First, we conducted a component-swapping experiment, building on the data switch experiments discussed previously. Recall that the data switch experiments consist of models trained for iterations, and subsequently trained for another iterations on a newly generated set of sequences. We saved model weights at two timepoints: immediately before the switch () and at the end of training (). If the MLP stores sequence-specific associations, then patching in the weights of an MLP saved before a dataset switch should rescue recall accuracy on the pre-switch set of sequences even when paired with post-switch attention weights.
As expected, the final model () achieved high accuracy on the new set of sequences (Figure 4a) but has

Figure 4. Division of labor among feedforward blocks and attention heads. Top: We train models where half-way through training, we resample all training sequences. (a) Accuracy of models after training on second sequences as a function of model size and sequence ranks. (b) Accuracy of models on first sequences as a function of model size and sequence ranks. (c) Accuracy of component-swapped models on first sequences, using MLPs at the end of the first period. Bottom: Linear decodability of 1-layer representations. (d) Linear decodability of 1-layer attention representations of novel sequences unseen during training as a function of training iteration. (e) Linear decodability of MLP representations of novel sequences unseen during training as a function of training iteration. (f) Linear decodability of attention representations predicts accuracy of probe experiments.
catastrophically forgotten the original set (Figure 4b). We then constructed a hybrid model combining the attention weights from with the MLP weights from , denoted . We evaluated this hybrid on the original set without any additional training. If sequence-specific information were stored primarily in attention weights, then substituting the post-switch attention should eliminate recall of the original sequences. Conversely, if sequence-specific associations reside in the MLP, the pre-switch MLP weights should enable above-chance performance on the original set, even when paired with attention weights associated with a model that has catastrophically forgotten the original set. Figure 4c shows that the hybrid model indeed achieves above-chance accuracy on the original set.
Linear decodability. Next, we hypothesize that attention outputs preserve information about sequence content by developing linearly separable representations of the sequence, and that the MLP transforms this information into predictions about the next bit. To quantify how explicitly sequence identity is represented in different layers, every 1000 iterations, we presented 1000 novel sequences to the model and extracted activation vectors from both attention outputs and MLP outputs at 25 evenly spaced positions across sequences of length (positions 0, 4, 8, . . . , 96). This yielded a classification dataset where the task is to predict sequence identity from activations at a given position. We trained separate logistic classifiers for attention and MLP representations at regular intervals throughout model training.
We observed that attention representations rapidly developed highly linearly separable representations of sequence identity: linear classifier accuracy exceeded 80% early in training and continued to improve (Figure 4d). While MLP classifier accuracy initially increased, it plateaued well below 50% accuracy and decreased over the course of training (Figure 4e). We hypothesized that this decline is because the MLP must focus on predicting the next bit rather than maintaining sequence identity information.
Probe fork experiments. To further show that linearly separable attention representations benefit memorization, we train 1-layer models where every 50 iterations during training, we initialize a new model with the same weights as the current model, freeze the attention weights and train only the MLP weights. We then “fine-tune” this forked model exclusively on a held-out set of probe sequences (never seen during training) which are presented uniformly at random. We fine-tune for up to 1,000 iterations, stopping early if the model obtains 90% accuracy.
quences (never seen during training) which are presented uniformly at random. We fine-tune for up to 1,000 iterations, stopping early if the model obtains 90% accuracy. For each 1-layer model, we plot the test accuracy of the linear classifier at various times during training against the logit-transformed fine-tuning accuracy and find that more linearly separable attention representations predict better fine-tuning accuracy (Figure 4f). Figure 4f also shows that larger models attain higher accuracy during fine-tuning. The fine-tuning accuracy of 1-layer models also increases throughout training, with the effect most apparent for larger models (Figure S7).

Figure S7. Fine-tuning accuracy of 1-layer transformers on novel unseen sequences increases as a function of training. Larger models achieve higher fine-tuning accuracy.
Homogenized attention. We verified that models with attention weights set to zero throughout training (such that attention heads provide no position-specific contextual information) failed to learn the training sequences and, correspondingly, forks showed little to no improvement in fine-tuning accuracy regardless of training duration (Figure S8).

Figure S8. 1-layer models with attention weights set to zero throughout training and probing fail to learn novel sequences.
Attention maps. Finally, we asked if attention heads acquire particular structure that allows them to better represent sequence identity. To do this, we visualize attention maps for a trained one-layer model with 16 heads (64 features per head) in Figure 5. Here we compute the attention paid at each query position to the prior positions for Sequence 1 (the highest rank and thus most commonly presented sequence). Sample attention maps for Sequence

Figure 5. Learned attention maps in a trained one-layer transformer. Each panel shows one of 16 heads with features per head, evaluated on the most frequent sequence. Rows are query positions and columns are causal key positions. Vertical bands indicate that heads specialize to fixed positions or position subsets, providing reusable context features to the MLP at each position. See Figure S9 and Figure S10 for attention maps for a different sequence and for a single head.
10 are presented in Figure S9 and the fifth head’s attention maps in Figure S10. Across heads and sequences, we observe prominent vertical bars in the attention maps, indicating that each attention map specializes to read from a particular set of positions. The vertical stripe patterns can be intuitively understood: each head conveys the same information to the MLP regardless of the length of the current subsequence. This allows the MLP to learn a position-independent decoder of sequence identity.

Figure S9. Learned attention maps in a trained one-layer transformer for Sequence 10 (16 heads, 64 features per head). Each panel shows attention maps of one head, evaluated on the that sequence. Rows are query positions and columns are causal key positions.

Figure S10. Attention patterns for Sequences 1 to 10 (Head 5 in a model with 64 features, 16 heads). Attention maps for the fifth head in a trained one-layer transformer across multiple sequences (16 heads, 64 features per head). Each panel shows the attention maps for this head, evaluated on the 10 lowest-ranked sequences. Rows are query positions and columns are causal key positions. We observe vertical bands at similar positions across multiple random sequences.
A theory of meta-memorization
Put together, these results support a functional division between attention and MLP layers in transformers such that attention heads convey sequence information to MLPs where memories are stored. This picture is consistent with prior work, notably [22], who show that attention circuits develop before MLPs begin to store factual information. Note that their task involves memorization of random synthetic facts expressed in natural-language which retains generalizable linguistic structure, making it challenging to identify what is learned by the attention heads. Our results show that attention heads learn to construct position-invariant, linearly separable representations of sequence identity and the MLP transforms these representations into dataset-specific features optimized for next-token prediction.
Conceptual picture. Based on our results and prior work, we propose a theory of meta-memorization where the attention layers and feedforward blocks in a transformer specialize towards performing specific computational roles over the course of training (Figure 6a). Specifically, we propose the attention heads convey position-invariant information about sequence identity to a key-value memory, which stores sequences encountered during training. During evaluation, the key-value memory uses sequence information conveyed by attention heads to first retrieve sequence identity (or index, denoted ). A downstream decoder then uses sequence index and the current position in the sequence to predict the th bit. The amount of sequence information that can be conveyed to the key-value memory is limited, since the number of attention heads and the number of features conveyed by each head are finite. The attentional bottleneck is necessary to encourage attention heads to learn an efficient coding scheme. Importantly, this bottleneck explains why larger models learn faster, since the amount of information transmitted from attention heads to MLPs is greater in larger models. We now express the core ingredients of this theory in a simplified mathematical model.

Figure 6. A phenomenology-driven theory of meta-memorization. (a) In this model, the attention heads present position-invariant information about the sequence to a key-value memory through a finite attentional bottleneck. The key-value memory uses this sequence information to first retrieve sequence index () and a downstream decoder uses sequence index and the current position in the sequence to predict the th bit. (b) The DAM model recapitulates the finding that switching to new random sequences reveals meta-memorization. Here, and are trained continuously under the next-bit objective. Accuracy drops when the sequence set is replaced but recovers faster than during initial training. We show results only for iterations and to better illustrate the effect. (c) Here, and are trained using an InfoNCE-style retrieval objective. The model shows post-switch accuracy comparable to pre-switch accuracy even when and are frozen after the switch. In both cases, transfer strengthens with the number of heads . Note that the noisy accuracy curves (due to finite-batch effects) are smoothed by computing a running average. This explains why the accuracy does not appear to return to random chance in panel b after the data switch, since the first point depicted is a running average of the first 100 steps’ accuracies after the switch. Here, we use , , , batch size = 50.
A minimal model of meta-memorization. In this model, a set of heads learn to convey information to a dense associative memory (DAM) for next-token prediction [29]. We consider a dataset with Bernoulli-binary sequences of length , . is much smaller than , thus creating an attentional bottleneck. The sequence is drawn with probability . We denote as the state of the -th bit in the -th sequence. The DAM has a capacity to store at most binary sequences of length (where ). During training, the DAM stores the keys to these sequences as -dimensional vectors , where for . is a learnable matrix. The state of the DAM at a particular time is summarized by the set of sequences stored in it, . The DAM is structured as a queue. Given an -bit sequence during training (which could be a sequence already in ), the DAM is updated by adding as the last element. If the queue has already stored elements, the first element is removed when is added.
During retrieval, the model is given a partial cue (i.e. the first bits in the sequence) and has to predict the -th bit. To do this, the model computes the -dimensional vector , where is an matrix such that and whose rows sum to 1. Thus, one can view as an attention map specific to position and as the query to the DAM. Note that, as in a transformer, the attention maps in general depend on the length of the sequence unmasked thus far. Given the cue and inverse temperature , we define the probability of retrieving sequence from the DAM as
The probability that the model assigns to the th bit being is given by where is the set of sequences in the DAM that have at the -th position.
Intuitively, the model first computes a posterior distribution over sequences in the DAM based on their match with the observed partial sequence , and predicts the next bit by computing the posterior predictive distribution using only sequences that have a bit in the -th location. Thus, we posit that the MLP layers in the transformer organize during training to implement this two-step inferential procedure.
Our minimal model has two parameters which are trained by jointly optimizing the loss at step for sequence :
where is the binary cross-entropy loss of predicting the th bit of sequence given , and the current state of the DAM . In our simulations, we compute the loss over a batch of 50 sequences and add these sequences together to the DAM after the update.
Meta-memorization in the DAM model. The DAM model recapitulates the meta-memorization effect. To show this,
we performed the data-switch experiment where the training set of sequences is switched halfway to a new set of random sequences, and continued training the matrices and . The sequences are drawn from a uniform distribution. In Figure 6b, we show that accuracy immediately dropped to chance after the switch, but, like Figure 2c, recovered faster than it had at the start of training. The post-switch speedup also increased with scale: models with more heads showed steeper recovery curves.
Next, we also tested a variant of our DAM where the model uses an InfoNCE objective to learn and that maximize the probability of retrieving the correct sequence given the cue (i.e. partial sequence) , instead of the posterior predictive probability [30]. We re-ran our experiment where we switched the training set of sequences halfway through training to a fresh set of random sequences and froze the matrices and after the switch. The sequences are drawn from a uniform distribution. In Figure 6c, we show that even though accuracy dropped after the switch, it remained comparable to the pre-switch accuracy. Thus, the attention heads learn an efficient, dataset-invariant coding scheme for conveying information to the DAM.
The frontier of the scaling law. To understand the extent to which architectural details matter beyond a simple parameter count, we trained 1-layer models with a grid search over number of attention heads , width per head , and MLP width , for iterations (Figure 7), allowing us to trade off attention and the MLP parameters for a given parameter budget. For a fixed model dimension , we observe that performance degrades substantially when , suggesting the MLP is the bottleneck. When , additional width yields only marginal improvements, suggesting the MLP is no longer the limiting factor. The elbow at (for a fixed ) indicates how parameters should be allocated to attention heads and MLPs to minimize loss given a fixed parameter budget.
Figure 7. Train loss across a wider parameter sweep across 3 hyperparameters: number of attention heads , width per head , and width of MLP for iterations. Each color represents models with a fixed model dimension but different . The loss of models with is plotted with cross marks. The elbow of the loss for a fixed begins approximately at , suggesting an optimal allocation of parameters to attention heads and MLPs.
Conclusion
By systematically quantifying memorization in models of different architectures and scales, we find that memorization capacity scales primarily with parameter count, recapitulating key scaling laws observed in models trained on language and synthetic data [4, 12, 21, 31], and obtain a scaling law of memorization rate with parameter count. The faster memorization of larger models reflects a meta-memorization effect, where the model’s memorization of sequences in general accelerates after memorizing other, unrelated, sequences. While the memorization rate follows a common scaling law with training iterations , the upper cutoff of the scaling law increases with model size.
Mechanistic experiments in one-layer transformers suggest a division of labor between attention and feedforward blocks. MLPs store sequence-specific associations, whereas attention heads learn reusable, position-invariant representations of partial contexts that make new sequences easier to write and retrieve. We develop a minimal model of this division of labor using a dense associative memory with an attentional bottleneck. Together, these results suggest that transformers acquire general-purpose attentional mechanisms that make each incremental gradient update more effective at writing memories and offer a mechanistic account of why scale accelerates memorization.
Our definition of memorization relies on a fixed per-sequence accuracy threshold and on rank-based summaries of the memorized set. Although these choices reveal robust trends across model scale, the precise capacity and rate estimates may depend on the threshold, sequence length, optimizer, learning rate, sampling distribution, and architecture.
One limitation of our work is our focus on memorization of random binary sequences. LLMs in realistic settings need to balance memorization of arbitrary information with acquisition of reusable linguistic structure. A natural direction for future work is to extend the present setting to data distributions with controlled amounts of structure, such as Markov chains or latent-variable sequence families. Such experiments could determine whether meta-memorization is strengthened or weakened when memorization competes with reusable abstraction. Finally, a complete theory should derive the rate exponent and its scale-dependent cutoff from architectural features such as the number of heads, feature dimension, memory capacity, and context length.
Acknowledgments
We thank members of the Reddy lab for insightful comments. GR was partially supported by a joint research agreement between NTT Research Inc. and Princeton University. The simulations presented in this article were performed on computational resources managed and supported by Princeton Research Computing, a consortium of groups including the Princeton Institute for Computational Science and Engineering (PICSciE) and Research Computing at Princeton University.
References
Additional figures
References
- [1]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, Advances in neural information processing systems 30, (2017).arxiv.org/abs/1706.03762
- [2]A. K. Lampinen, S. C. Chan, A. K. Singh, and M. Shanahan, The broader spectrum of in-context learning, arXiv preprint arXiv:2412.03782 (2024).arxiv.org/abs/2412.03782
- [3]N. Carlini, C. Liu, Ú. Erlingsson, J. Kos, and D. Song, The secret sharer: Evaluating and testing unintended memorization in neural networks, in 28th USENIX security symposium (USENIX security 19) (2019) pp. 267–284.arxiv.org/abs/1802.08232
- [4]N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang, Quantifying memorization across neural language models, in The Eleventh International Conference on Learning Representations (2022).arxiv.org/abs/2202.07646
- [5]M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee, Scalable extraction of training data from (production) language models, arXiv preprint arXiv:2311.17035 (2023).arxiv.org/abs/2311.17035
- [6]A. Ahmed, A. F. Cooper, S. Koyejo, and P. Liang, Extracting books from production language models, arXiv preprint arXiv:2601.02671 (2026).arxiv.org/abs/2601.02671
- [7]A. F. Cooper, A. Gokaslan, A. Ahmed, A. B. Cyphert, C. De Sa, M. A. Lemley, D. E. Ho, and P. Liang, Extracting memorized pieces of (copyrighted) books from open-weight language models, arXiv preprint arXiv:2505.12546 (2025).arxiv.org/abs/2505.12546
- [8]S. Biderman, U. Prashanth, L. Sutawika, H. Schoelkopf, Q. Anthony, S. Purohit, and E. Raff, Emergent and predictable memorization in large language models, Advances in Neural Information Processing Systems 36, 28072 (2023).arxiv.org/abs/2304.11158
- [9]J. Hayes, M. Swanberg, H. Chaudhari, I. Yona, I. Shumailov, M. Nasr, C. A. Choquette-Choo, K. Lee, and A. F. Cooper, Measuring memorization in language models via probabilistic extraction, in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (2025) pp. 9266–9291.
- [10]N. Kandpal, E. Wallace, and C. Raffel, Deduplicating training data mitigates privacy risks in language models, in International Conference on Machine Learning (PMLR, 2022) pp. 10697–10707.arxiv.org/abs/2202.06539
- [11]J. Huang, D. Yang, and C. Potts, Demystifying verbatim memorization in large language models, in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (2024) pp. 10711–10732.
- [12]K. Tirumala, A. Markosyan, L. Zettlemoyer, and A. Aghajanyan, Memorization without overfitting: Analyzing the training dynamics of large language models, Advances in Neural Information Processing Systems 35, 38274 (2022).arxiv.org/abs/2205.10770
- [13]Y. Dong, L. Noci, M. Khodak, and M. Li, Attention retrieves, mlp memorizes: Disentangling trainable components in the transformer, arXiv e-prints , arXiv (2025).
- [14]M. Geva, R. Schuster, J. Berant, and O. Levy, Transformer feed-forward layers are key-value memories, in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021) pp. 5484–5495.arxiv.org/abs/2012.14913
- [15]M. Geva, A. Caciularu, K. Wang, and Y. Goldberg, Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space, in Proceedings of the 2022 conference on empirical methods in natural language processing (2022) pp. 30–45.arxiv.org/abs/2203.14680
- [16]M. Geva, J. Bastings, K. Filippova, and A. Globerson, Dissecting recall of factual associations in auto-regressive language models, in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (2023) pp. 12216–12235.arxiv.org/abs/2304.14767
- [17]K. Meng, D. Bau, A. Andonian, and Y. Belinkov, Locating and editing factual associations in gpt, Advances in neural information processing systems 35, 17359 (2022).arxiv.org/abs/2202.05262
- [18]E. Nichani, J. D. Lee, and A. Bietti, Understanding factual recall in transformers via associative memories, arXiv preprint arXiv:2412.06538 (2024).arxiv.org/abs/2412.06538
- [19]B. Chen, N. Han, and Y. Miyao, A comparative analysis of llm memorization at statistical and internal levels: Cross-model commonalities and model-specific signatures, arXiv preprint arXiv:2603.21658 (2026).arxiv.org/abs/2603.21658
- [20]Z. Allen-Zhu and Y. Li, Physics of language models: Part 3.3, knowledge capacity scaling laws, arXiv preprint arXiv:2404.05405 (2024).
- [21]J. X. Morris, C. Sitawarin, C. Guo, N. Kokhlikyan, G. E. Suh, A. M. Rush, K. Chaudhuri, and S. Mahloujifar, How much do language models memorize?, arXiv preprint arXiv:2505.24832 (2025).arxiv.org/abs/2505.24832
- [22]N. Zucchet, J. Bornschein, S. Chan, A. Lampinen, R. Pascanu, and S. De, How do language models learn facts? dynamics, curricula and hallucinations, arXiv preprint arXiv:2503.21676 (2025).arxiv.org/abs/2503.21676
- [23]N. Zucchet, F. d’Angelo, A. K. Lampinen, and S. C. Chan, The emergence of sparse attention: impact of data distribution and benefits of repetition, arXiv preprint arXiv:2505.17863 (2025).arxiv.org/abs/2505.17863
- [24]M. Panwar, G. Weiss, N. Goyal, and A. Bosselut, For better or for worse, transformers seek patterns for memorization, in The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025).DOI
- [25]S. Chan, A. Santoro, A. Lampinen, J. Wang, A. Singh, P. Richemond, J. McClelland, and F. Hill, Data distributional properties drive emergent in-context learning in transformers, Advances in neural information processing systems 35, 18878 (2022).arxiv.org/abs/2205.05055
- [26]A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al., Language models are unsupervised multitask learners, OpenAI blog 1, 9 (2019).
- [27]A. Karpathy, nanoGPT, https://github.com/karpathy/nanoGPT (2022).
- [28]D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).arxiv.org/abs/1412.6980
- [29]H. Ramsauer, B. Sch¨afl, J. Lehner, P. Seidl, M. Wiedrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlovic, G. K. Sandve, et al., Hopfield networks is all you need, arXiv preprint arXiv:2008.02217 (2020).arxiv.org/abs/2008.02217
- [30]A. v. d. Oord, Y. Li, and O. Vinyals, Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.03748 (2018).arxiv.org/abs/1807.03748
- [31]B. Li, Incompressible knowledge probes: Estimating black-box llm parameter counts via factual capacity, arXiv preprint arXiv:2604.24827 (2026).arxiv.org/abs/2604.24827
