WorksComputer Science
Long-Context Generation Is a Sampling Problem (When You Turn Up the Heat)
Abstract
Open-weight language models are widely observed to get worse the longer they generate. Often, attempts to fix this involve more training. At high temperature, The sampler, i.e. the rule that picks the next token out of the model's probabilities, decides whether long and agentic generation survives and/or thrives. We test 29 open-weight models from 1B to 122B parameters on ten established benchmarks plus needle retrieval out to one million tokens, covering SWE-bench Verified coding agents, long chain-of-thought reasoning, long-form writing, long structured output and long-context reading. Samplers that keep their pool of candidate tokens small and steady as temperature rises don't lose accuracy. Samplers that let in low-probability tokens fall apart. At temperature 2, coding agents running top-p, top-k or Top-H resolve 0 of 100 GitHub issues, while the same agents running top-nsigma resolve 46 to 62 and win every issue the two samplers disagree on, on three Qwen3.5 models. Over all benchmarks top-nsigma keeps a median 100% of its temperature-1 score where top-p keeps 26%. Counting how many tokens each sampler lets through explains why. In one Llama-3.1-8B story, going from temperature 1 to 2 grows top-p's pool from 4 tokens to 51,845 while top-nsigma stays at 3. Across 133 combinations of sampler setting and temperature, the typical pool size predicts judged quality (Spearman correlation of -0.77), and when we pick the pool at temperature 1 and only apply temperature afterwards, Top-H, min-p and p-less stop collapsing. At the default temperature of 1 most samplers perform about the same, We release all code and data to the public.
Introduction
Open-weight language models now generate very long sequences [8, 39]. In practice, on smaller models, the second half of a long sample often turns into repeated phrases and loops [10, 35, 7], and most proposed fixes involve retraining or long-output fine-tuning [3]. We argue for a cheaper fix that inference engines already expose, which is the sampler.
Samplers and temperature. At each step the model gives a probability for every token in its vocabulary, and the sampler stacks outputs a single choice. Given a temperature of 1 or below, it piles probability onto the likeliest tokens, and above 1 it spreads probability out toward unlikely ones. Truncation rules cut the unlikely tokens and samples from the remaining distribution, which we call the candidate set. Top- keeps a fixed number of tokens [6]. Top- keeps however many tokens it takes to cover a fixed share of the probability [10]. Newer rules like min- [21], top- [30], Top-H [23] and P-less [29] look at the shape of the distribution at each step before deciding where to cut.
Background and Related Work
Truncation samplers. Other rules besides the ones above include locally typical sampling [20], -sampling [9], Mirostat [5] and XTC [32, 34]. Top-, Top-H and P-less all set their cutoff from statistics of the whole distribution, which is why we have lumped them together as “full-distribution-aware”.
Degeneration. It has been known for years that neural text generators fall into repetition over long outputs [10, 35, 7]. Repetition penalties [15], diversity objectives [17], the DRY penalty [31, 33] and contrastive search [27, 26] were all built to deal with it. We focus on the base truncation rule and look at penalties separately in Appendix § 11.
Evaluating samplers and long context. La Rosa [2026] ran a controlled thirteen-model study and found that truncation samplers mostly change short-answer accuracy when high temperature has already degraded the model. We find that this regime also governs long-context and agentic generation. Long-context ability is usually measured with retrieval and comprehension benchmarks [1, 11, 25], and long-output ability with benchmarks like LongBench-Write [3]. LLM judges [42, 18] make it practical to grade long outputs one window at a time.
| Sampler | Keeps the tokens that. . . | temp. 1 | temp. 2 |
| Pure sampling | (all tokens) | all | all |
| Top- 50 | are among the 50 most likely | 51 | 51 |
| Top- 0.9 | together cover 90% of the probability | 2 | 90,000 |
| Top- 0.4 | keep the set’s entropy under 40% of the full entropy | 1 | 54 |
| Min- 0.1 | have at least 10% of the top token’s probability | 1 | 14 |
| P-less | are at least as likely as two random draws are to match | 1 | 21 |
| Top- 1 | have logits within one standard deviation of the top | 1 | 1 |
Table 1. The samplers we compare, what each one keeps, and how many tokens it keeps per decoding step (median over the first 2K tokens of the ten-model generations of § 5.4, full numbers in Table 15). Temperature is applied before truncation, as HuggingFace, vLLM and SGLang do by default. Compare the two columns of each row. Top-, Top-, P-less and min- all grow between temperature 1 and 2, and top- is the only rule that stays at its temperature-1 size.
The Impact of Temperature On Truncation Samplers at Long Context
Table 1 describes the truncations rules we compare in plain terms. Formal definitions are in Appendix § 9. HuggingFace, vLLM and SGLang all apply temperature before truncation. Notably, Llama.cpp choose to default to the opposite order. That ordering matters. With temperature first, going from temperature 1 to 2 grows the median top- set from 2 tokens to about 90,000, Top- from 1 to 54, P-less from 1 to 21 and min- from 1 to 14, because each rule now draws its cutoff on probabilities that temperature has already flattened. Truncate on the untempered probabilities and apply temperature afterwards, and suddenly every rule keeps its temperature-1 set at any temperature. Temperature then only moves probability around among tokens the rule already accepted. § 5.7 tests whether this reordering alone fixes long context generation problems.
Top- keeps the tokens whose logit sits within standard deviations of the highest logit. Temperature divides all the logits by the same number, which shrinks the distance to the top and the standard deviation by the same amount, so the tokens top- keeps never change with temperature [30]. Temperature only changes how probability gets shared out inside that set. Top- is fixed by design too, although it hands the model all tokens even when the model is close to certain. Every other rule sets its cutoff on probabilities that temperature has already flattened, so its set grows as temperature goes up. Min- and P-less grow slowly because their cutoffs follow the top of the distribution. Top- and Top- grow fast because theirs follow cumulative probability and entropy. Top- reads a global statistic just like top- does, yet at high temperature it behaves like top- for this reason. The low-probability tail these fast-growing rules let in is what high temperature turns into noise.
Experimental Setup
Models and implementation. We use 29 instruction-tuned open-weight models from 1B to 122B parameters from the Llama, Qwen, Mistral, Gemma, OLMo, Nemotron, GLM and gpt-oss families [39, 13, 22, 38] (Graffatori et al. 2024), with a core set of ten from Llama-3.2-1B to gpt-oss-120b (Appendix § 10 lists all of them). We run the released weights (bf16, or native MXFP4 for gpt-oss-120b) on NVIDIA H200 GPUs. All samplers live in a single vLLM logits processor that replaces native sampling. We checked each one against its authors’ reference code, and the processor logs the candidate-set size at every step.
Benchmarks and protocols. For agentic coding we use SWE-bench Verified [14] with the mini-swe-agent harness [28, 40] and the official test harness. For long chain-of-thought reasoning we use LiveCodeBench v6 [12], AIME and HMMT 2025–26 from MathArena [4] and the level-5 problems of MATH-500 [19], all in thinking mode with a 32K-token budget. Long-form writing is covered by LongBench-Write [3], WritingBench [37] and the 75 English Arena-Write prompts [36] that ask for at least 2,000 words, all with natural stopping. For long structured output we use LongProc [Ye et al., 2025] (six tasks, outputs up to 8K tokens, official rule-based scoring). For long-context reading we use LongBench v2 [2] (503 questions, contexts up to 120K tokens, official chain-of-thought protocol) and multi-key needle retrieval in PG-19 text [24] from 8K to one million tokens. We also run a stress test that forces eight fiction and eight non-fiction prompts (Appendix § 18) out to 16K tokens with end-of-sequence turned off.
Figure 2. Each point is a sampler’s score at the high temperature listed on the right, divided by that same sampler’s score at temperature 1 on the same models and items, so 100% (dashed line) means turning up the heat performed strongly. Dividing by each sampler’s own baseline isolates how it reacts to temperature. Top- (blue circles) sits on the dashed line in every row, while top-, top- and Top-H drift left and hit 0% on coding agents. Arrows mark values above 150% (P-less on Qwen3.5-9B resolves 50% at temperature 2 and 33% at 1). Reasoning rows use top- 0.95. Complete results are in § 5.1–§ 5.3 and the appendices.
Outcomes. The LLM judge (gpt-oss-120b) gives a 1 to 10 score to 1.5K-token windows taken at fixed depths. We say a generation has a terminal loop if, from some 1K-token window on, at least 80% of the remaining windows repeat 32-grams at least half the time. A generation is healthy if it has no terminal loop and every judged window scores at least 6 for coherence. Coding agents don’t need a judge, since issues only count as resolved if the agent’s patch passes the repository’s tests.
Results
Coding agents
The SWE-bench agent runs tens to a few hundred commands and edits per episode, its context grows to tens of thousands of tokens, and one malformed command can end the episode. Figure 3 has SWE-bench Verified resolve rates for seven Qwen3.5 configurations. Table 2 compares each sampler with top- issue by issue in our long-horizon setting (500 agent steps, 262K-token context).
Figure 3. SWE-bench Verified resolve rate against temperature for Qwen3.5 agents (mini-swe-agent, official test harness). The top row uses the original issue sets. The bottom row uses 100 fresh issues drawn with a fixed seed and no overlap with the first set. A line that stays flat means the agent keeps working as temperature rises. All samplers start together at temperature 1, the tail-admitting rules (orange, yellow, pink and dotted gray) fall to zero by temperature 2, and top-, P-less and min- stay up. Absent lines and points were never run (the fresh-issue grids were pre-registered with four samplers, and the 500-step 9B grid only up to 1.5). The same split on an independently drawn issue set rules out a quirk of one sample. Gemma-4 is in Appendix § 11.
| Sampler | Resolved | W/L vs top- | Resolved | W/L vs top- |
| Pure sampling | 46% | 6/6 | 0% | 0/0 |
| Top- 0.9 | 46% | base | 0% | base |
| Min- 0.1 | 46% | 7/7 | 48% | 24/0* |
| Top-H 0.4 | 56% | 9/4 | 18% | 9/0* |
| P-less | 48% | 6/5 | 58% | 29/0* |
| Top- 1 | 52% | 9/6 | 52% | 26/0* |
Table 2. Long-horizon coding agents (Qwen3.5-9B, 500-step limit, 262K context, 50 issues), the setting closest to how agents are run in practice. is the temperature. W/L counts issues the sampler resolved and top-p 0.9 did not at the same temperature, and the reverse. A lopsided record like 26/0 means the same issues flip in one direction, which is very unlikely to be noise. * marks a two-sided sign test with , and the min-p, P-less and top- wins at 1.5 all reach . Nothing is significant at 1.0, meaning samplers only differ once the temperature goes up.
There’s not much to choose between the rules at temperature 1. They resolve 33–37% of the 100-issue grid and 46–56% in the long-horizon setting. Once the temperature goes up they split apart. On Qwen3.5-9B at temperature 2, top-, top-, Top-H and pure sampling resolve 0 of 100 issues, since their commands turn into garbled text within a few turns. Top- resolves 51 and P-less 50, and neither loses a single disputed issue (51–0 and 50–0). In the long-horizon setting the split already shows at 1.5. Min- gets 48%, P-less 58% and top- 52%, top- gets 0%, and the win–loss records are 24–0, 29–0 and 26–0.
Qwen3.5-27B and Qwen3.5-35B-A3B don’t resolve anything at temperature 2 under top-p, top-k, Top-H or pure sampling, and under top- they resolve 72% and 58% (36–0 and 29–0). We then drew 100 fresh issues with no overlap with the first set. Top- resolves 46, 62 and 53 percent of them on the 9B, 27B and 35B-A3B models while top-p and Top-H resolve none, for records of 46–0, 62–0 and 53–0. An independent rerun of the long-horizon grid gives top- 56% at 1.5 and 60% at 2, compared with 2% and 0% for top-p.
Long chain-of-thought coding without an agent shows it too. On LiveCodeBench v6 with Qwen3.8-27B in thinking mode (175 problems, two samples, 32K-token budget, Table 13), every rule scores 61–64% pass@1 up to temperature 1. At 1.3 top-p 0.95 drops to 51% while min-p, P-less, top- and Top-H stay at 63% and win 49–53 disputed problem-samples for 6–8 losses. Qwen3.6-27B behaves the same way (Table 14).
Table 13: LiveCodeBench v6, Qwen3.8-27B in thinking mode, 175 problems 2 samples, 32K tokens, official tests in a sandbox. W/L counts problem-samples the rule solved and top- 0.95 didn’t at the same , and the reverse. * marks a sign test with . Every rule ties up to , and at only top- loses ground.
| Sampler | pass@1 | W/L | pass@1 | W/L | pass@1 | W/L |
| Top- 0.95 | 62.0 | base | 60.9 | base | 50.6 | base |
| Top- 1 | 63.4 | 14/9 | 63.7 | 22/12 | 63.4 | 53/8* |
| Min- 0.1 | 62.6 | 14/12 | 62.9 | 21/14 | 63.1 | 50/6* |
| P-less | 62.0 | 14/14 | 62.6 | 22/16 | 62.6 | 49/7* |
| Top-H 0.4 | 61.7 | 15/16 | 62.6 | 22/16 | 62.9 | 50/7* |
Table 13.
| Sampler | pass@1 | W/L | pass@1 | W/L | pass@1 | W/L |
| Top- 0.95 | 52.0 | base | 49.1 | base | 46.3 | base |
| Top- 1 | 52.9 | 20/17 | 51.4 | 18/10 | 50.3 | 24/10* |
| Min- 0.1 | 49.4 | 16/25 | 51.1 | 15/8 | 48.6 | 21/13 |
| P-less | 46.3 | 14/34* | 45.4 | 12/25* | 49.7 | 24/12 |
| Top-H 0.4 | 46.9 | 16/34* | 48.6 | 22/24 | 49.7 | 26/14 |
Table 14.
Long-form writing, structured output and long-context reading
Figure 4 covers Arena-Write [36], LongBench-Write [3] and WritingBench [37], all with natural stopping. The rules sit close together at temperature 1 (90–93% healthy on Arena-Write, 78–81 on LongBench-Write, 6.8–6.9 on WritingBench). At temperature 2 top- hasn’t moved on any of the three (0.93 healthy, 80.6 and 6.9). P-less and min- lose some ground (0.74 and 0.69 healthy, 77 and 75 on LongBench-Write). Top-, top- and top- break down badly. Only 0.10, 0.15 and 0.01 of their Arena-Write generations are healthy, they score 38, 43 and 21 on LongBench-Write, and top- gets 2.2 on WritingBench.
Figure 4. From left to right, share of healthy Arena-Write generations (nine models, among them Llama-3.3-70B, gpt-oss-120b and Qwen3.5-122B-A10B), official LongBench-Write score, WritingBench critic score (555 queries), official LongProc score (six procedural tasks, outputs up to 8K tokens) and LongBench v2 accuracy (503 questions, contexts up to 120K tokens). The lines bunch together at temperature 1 and fan out as it rises, in the same order every time, with the tight rules on top. LongProc is graded by fixed rules and LongBench v2 by exact-match answers, so the ordering does not depend on an LLM judge. LongProc and LongBench v2 were run up to temperature 1.5. Tables 17, 9 and 10 have the numbers and paired tests.
LongProc [Ye et al., 2025] asks for long structured output, e.g. turning an HTML page into a TSV table or writing a travel plan, and grades it with fixed rules, so there’s no judge in the loop. At 1.5 P-less and top- keep the 31% and 29% they had at temperature 1. Top- falls to 13% and loses 532–73 and 469–74 on disputed items, top- falls to 8% and pure sampling to 3%. LongBench v2 [2] has the model read up to 120K tokens and reason before answering. Here P-less and top- hold 39.8% and 38.3% at 1.5, and top- drops from 36.6% to 25.6% (667–239 and 635–253). LongBench v2 is also the one benchmark where the tight rules lead even at temperature 1 (39.5% for top- vs. 36.6%, 355–269).
Needle retrieval and long reasoning
Temperature hurts even when the model mostly just has to read. We hid 16 key–value “needles” in PG-19 novels between 8K and one million tokens long and asked for each value under every sampler (Figure 17 in Appendix § 14). At temperature 1 the rules retrieve about the same share at every length. At 1.5 the tight rules don’t change, but every tail-admitting rule gets worse, and the longer the context the bigger the drop. At one million tokens top-, P-less and min- retrieve 75%, 78% and 73%, top- retrieves 59% and pure sampling 48% (106–25, 113–19 and 100–30 on queries where they disagree with top-). Our explanation is that a longer context makes the model less sure of the answer. Under greedy decoding the entropy of the first answer token goes from 0.02 nats at 8K to 0.37 at one million tokens, and a rule that lets the tail in turns that extra uncertainty into wrong digits.
Figure 17. Share of 16 key–value needles retrieved from PG-19 haystacks of 8K to one million tokens, four long-context models (Llama-3.1-8B-UltraLong-1M, Nemotron-3-Nano-30B-A3B, Qwen3.5-4B and 9B, the Qwen models with YaRN past 256K), 512 queries per point. Retrieval is scored by exact match, with no judge. At temperature 1 (left) the lines lie on top of each other, so any gap on the right comes from temperature and not from a difference in baseline ability. At 1.5 (right) the gap grows with context length. This is what we would expect if a longer context makes the model less sure of the answer and the tail-admitting rules turn that uncertainty into wrong digits. Table 11 has the paired tests.
| Sampler | Score | W/L | Score | W/L | Score | W/L |
| Top- 0.9 | 92.9 | 91.4 | 82.3 | |||
| Top- 50 | 92.4 | 43–61 | 91.8 | 88–76 | 85.2 | 290–186* |
| Top-H 0.4 | 93.3 | 40–25 | 92.5 | 78–41* | 84.6 | 248–166* |
| Min- 0.1 | 92.9 | 38–39 | 92.1 | 75–50* | 91.1 | 385–70* |
| P-less | 93.6 | 47–23* | 93.5 | 90–15* | 92.6 | 402–35* |
| Top- 1 | 92.7 | 39–47 | 92.7 | 80–35* | 92.4 | 408–46* |
| Pure sampling | 92.1 | 35–62* | 90.9 | 72–91 | 77.4 | 143–321* |
Table 11. Needle retrieval with all context lengths from 8K to 1M tokens pooled (five models, Qwen3.5-35B-A3B only up to 256K). Share of needles retrieved (%) and W/L against top- 0.9 on identical queries. These are the paired tests behind Figure 17.
Long chain-of-thought math follows the same pattern (Appendix § 13). At 1.3 top- matches its temperature-1 accuracy on AIME and HMMT 2025–26 with six models (40.2% vs. 39.6%) and on AIME plus MATH-500 level 5 with four models (57.9% vs. 58.2%). Top- 0.95 falls to 27.4% and 39.2%. Top- wins 219–29 and 326–36 on disputed problem-samples, and min-, P-less and Top-H beat top- at 1.3 as well.
Forced 16K-token generation
To see where along a long text things go wrong, we forced every model to write 16K tokens and judged it window by window (Figure 5, Table 12). At temperature 1 every truncation rule opens at 4.8–5.2 out of 10 and drifts down to about 2.5, so the sampler hardly matters for judged quality there. At temperature 2 it matters a great deal. Top-, top-, Top-H and pure sampling open at 1.0–2.0, which in practice means word salad from the first paragraph. Min- and P-less open at 3.8 and 3.6. Top- opens at 5.1, against 5.2 at temperature 1, and it is the only rule whose temperature-2 curve lies on top of its temperature-1 curve (last window 2.4 vs. 2.5).
Figure 5. Judged quality of 1.5K-token windows by position in forced 16K-token fiction, ten models from 1B to 120B, 80 generations per line. The left end of a line is how well the text opens and its slope is how fast it decays. At temperature 1 every rule opens near 5 and decays to about 2.5, and the sampler barely matters. At 1.5 top- shows the slow version of the failure, opening at 3.2 and sliding to 1.5. At 2 the tail-admitting rules are at the floor from the first window, i.e. word salad, while top- traces essentially the same curve it traces at temperature 1.
Table 12: Forced 16K-token generation. Ten models, eight fiction prompts, seed 1, 80 generations per cell. Loop is the share of generations ending in a terminal loop. First and Last are the judge’s overall score (1–10) on the first and final 1.5K-token windows. Read Loop together with First. At almost every rule loops, at the tail-admitting rules stop looping only because their text is word salad from the start (First near 1), and top- keeps its scores.
| Sampler | Loop | First | Last | Loop | First | Last | Loop | First | Last |
| Pure sampling | 0.41 | 4.0 | 2.5 | 0.03 | 1.1 | 1.0 | 0.00 | 1.0 | 1.0 |
| Top- 50 | 0.56 | 5.2 | 2.7 | 0.00 | 3.2 | 1.5 | 0.00 | 2.0 | 1.2 |
| Top- 0.9 | 0.95 | 5.2 | 2.5 | 0.05 | 1.5 | 1.2 | 0.00 | 1.0 | 1.0 |
| Min- 0.1 | 0.97 | 5.2 | 2.5 | 0.64 | 5.3 | 2.4 | 0.09 | 3.8 | 1.6 |
| Top-H 0.4 | 0.97 | 4.8 | 2.6 | 0.20 | 4.5 | 2.2 | 0.00 | 1.8 | 1.1 |
| P-less | 0.99 | 4.8 | 2.6 | 0.99 | 5.2 | 2.6 | 0.44 | 3.6 | 1.9 |
| Top- 1 | 1.00 | 5.2 | 2.5 | 0.90 | 5.2 | 2.5 | 0.71 | 5.1 | 2.4 |
| Greedy | 1.00 / 4.8 / 2.5 |
Table 12.
What and how many tokens are these samplers choosing?
Our logits processor logs how many tokens each rule admits at every step, reported in Figure 6 and Table 15. Top-’s set can’t change with temperature by construction, so its measured median is 1 in the ten-model generations and 2 in the eight-model sweep at every temperature. From temperature 1 to 2 the median grows from 1 to 14 tokens for min-, to 21 for P-less and to 54 for Top-H, while top- ends up keeping most of the vocabulary. Means run higher than medians (min- and P-less sometimes admit thousands of tokens at temperature 2), and past 2.5 their sets get bigger than Top-H’s. The faster a set grows, the more junk tokens temperature can promote. Those junk tokens are what turn a story into word salad or a shell command into a format error. We think of long-context failure as an accumulation of these small sampling errors. Each one pushes the model a bit further out of distribution, until it can’t find its way back. If one also accounts for how big the set is at temperature 1 (top- never grows, but it always admits about 50 tokens), this measurement sorts the rules into the same groups the benchmarks do.
Figure 6. Median tokens admitted per decoding step (log scale) against temperature, eight models, 64 generations per point. A flat line means temperature cannot pull new tokens into the set. Top- is flat at 1–2 tokens by construction and top- is flat at about 50, while other rules climb, top- fastest.
| Sampler | Singleton at | |||
| Pure sampling | all | all | all | 0.00 |
| Top- 50 | 51 | 51 | 51 | 0.00 |
| Top- 0.9 | 2 | 41,508 | 89,581 | 0.46 |
| Min- 0.1 | 1 | 3 | 14 | 0.59 |
| Top-H 0.4 | 1 | 10 | 54 | 0.62 |
| P-less | 1 | 1 | 21 | 0.93 |
| Top- 1 | 1 | 1 | 1 | 0.52 |
Table 15.
Sweeping hyperparameters
An obvious objection is that all of this comes down to hyperparameter choice. To check, we swept 22 settings over seven temperatures from 0.7 to 3.0 on eight models from five families, with 64 forced 8K-token generations per cell (Figure 8). Top- with of 1 or less, whose set is small, scores 4.2–4.7 from 0.7 all the way up to 3.0. Larger lets in more of the tail at every temperature, and
Figure 8. Mean judged window quality against temperature for all 22 settings in our sweep, eight models from five families, 64 forced 8K-token generations per point. Each panel is one rule. Lighter to darker lines follow the listed hyperparameter values, and dotted gray is pure sampling for reference. If the cliff were a tuning problem, every rule would have some setting that stays flat. Only top- with does, and min- with of 0.3 nearly does. Elsewhere the hyperparameter only moves the cliff, and P-less (no hyperparameter) holds until 1.6 and then falls.
cliff comes back. For the other families the hyperparameter just moves the cliff around. Every top- setting is below 1.5 by temperature 1.6. Min- lasts longer as goes up ( of 0.02 collapses after 1.3, 0.1 after 1.6, and 0.3 is still above 4 at 2.5), though you pay for that with a tighter set at all the lower temperatures. If we pool the 133 setting-by-temperature cells that have per-step candidate-set logs (19 of the 22 settings, see Figure 7), the typical number of tokens a setting admits per step predicts its judged quality with a Spearman correlation of , whichever rule produced it. Nothing stays usable above roughly 100 tokens per step.
Figure 7. Judged quality against the geometric-mean number of tokens admitted per step. Each point is one sampler setting at one temperature, averaged over eight models, and bigger markers mean higher temperature. Points from different rules fall on one downward curve (Spearman ), The number of tokens a setting lets through predicts its quality without knowing which rule it is. 133 cells from 19 of the 22 settings in Figure 8 (top- with of 1, 2 and 3 is missing because of a tag mismatch in our analysis script).
Figure 9. The causal test. Dashed lines apply temperature first, as inference engines do by default. Solid lines choose the candidate set at temperature 1 and only then sample at the temperature on the x-axis, so the set can’t grow. Four models, 32 generations per point. If flatter probabilities inside a good set were the problem, solid lines would fall with dashed ones, and for Top-H, min- and P-less they don’t fall at all. Top- and top- are helped only partly or not at all, because their temperature-1 sets already reach into the tail.
Applying temperature last
If high temperature hurts because it changes which tokens get in, then picking the candidate set at temperature 1 and only applying temperature inside it should make the cliff go away. With the set picked at temperature 1, Top-H, min- and P-less keep their temperature-1 quality all the way to 3 (4.1–4.6), where the default order has them down at 1.1–1.2. This only rescues rules whose temperature-1 set already leaves out the tail. Top- at temperature 1 admits a mean of 73 tokens in the ten-model generations, because on uncertain steps it reaches deep into the tail anyway, and top- always admits about 50. Fixing their sets helps them partly or not at all. The result also holds on LongProc with its rule-based scoring (Appendix § 14). With the set picked at temperature 1, min- 0.1 scores 25% at temperature 2 instead of 15%, and top- 0.95 comes back from 1% to 25% at 1.5 (Qwen3.5-4B and 9B, five tasks). Leaving out the tail is what matters, and top- does it at every temperature by design.
Robustness
The forced-length and benchmark results survive the checks in Appendix § 17, which covers a second seed on all ten core models, six newer models up to Qwen3.5-122B-A10B (top- opens at 7.4 at temperature 2 and top- at 1.7), natural-length generation where the model may stop and is asked to continue, and non-fiction. The second judge from another family (GLM-4.5-Air) agrees with the first at a Spearman correlation of 0.92 on cell means, The second seed reproduces the 220 model-by-sampler-by-temperature cells at 0.95 (Figure 20).
Figure 20. (a) Cell means from gpt-oss-120b against GLM-4.5-Air on the same 8,796 windows. (b) Per-model cell means for seed 1 against seed 2 (all ten models, 8 generations per cell per seed). Bigger markers mean higher temperature. Points on the diagonal mean the two judges, or the two seeds, agree. Both clouds hug it, so neither the choice of judge nor the random seed drives the forced-length conclusions.
Practical Guidance
If you turn up the temperature, use a sampler that leaves out the tail. A rule whose candidate set grows with temperature ties together two knobs, i.e. how much randomness to add among reasonable tokens and how far to reach into unreasonable ones. Top- with of 1 or less keeps them apart by construction, and it is already available in llama.cpp. Min- with a larger , or any tight rule whose set is picked at temperature 1, keeps them apart in practice.
Report what the sampler keeps! Benchmarks run only at temperature 1 hide most of these differences, and nominal hyperparameters hide the property that matters, so sampler comparisons should report the realized candidate-set size per step. Benchmark results for open-weight models should also list the full sampler configuration and its order, since the same model at the same temperature goes from 62% of issues resolved to 0% on the sampler alone.
Conclusion
The sampler is a free and extremely effective control over long-context generation, Yet hardly anyone tunes it or even reports it. Across 29 models and every benchmark tried, samplers whose candidate set stays small and stable lose little or nothing at high temperature, and samplers that admit the tail fall apart. Long-context generation is, to a large degree, a sampling problem once you turn up the heat.
Acknowledgments and Disclosure of Funding
This work was funded by Thoughtworks. Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanui Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabhandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Bader, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal,
Sampler definitions and temperature
Let be the logits at one step, the temperature, the tempered distribution, and the candidate set. Following HuggingFace, vLLM and SGLang, every rule is applied to , i.e. after temperature, except in the temperature-last experiments of § 5.7.
Top- keeps every token within standard deviations of the maximum logit:
P-less keeps tokens at least as probable as the collision probability of two independent draws, . Min- keeps . Top-H adds tokens in descending probability until the entropy of the renormalized set reaches a fraction of , with . Top- keeps the smallest prefix of tokens sorted by probability whose total mass reaches , and top- keeps the most probable tokens.
Temperature invariance. Replacing by scales both and by , so the condition in Equation (1) is unchanged and the top- set is identical at every [30]. Top-p, P-less, Top-H and top- all threshold on , which flattens as grows, so their sets grow with . The min- and P-less thresholds are tied to the head of and , which shrinks slowly, while the top- and Top-H thresholds are tied to cumulative mass and to , which react much faster.
Temperature last. In § 5.7 the candidate set is computed from , i.e. at , and the token is then drawn from restricted to that set. We write for the sampling temperature in this setting.
Models
The ten-model core set is Llama-3.2-1B, Qwen2.5-1.5B, Qwen2.5-3B, Mistral-7B-v0.3, Llama3.1-8B, Mistral-Nemo-12B, Qwen2.5-14B, Qwen2.5-32B, Llama-3.3-70B [8, 39, 13] and gpt-oss-120b [22]. Other experiments add Qwen3.5 (9B, 27B, 35B-A3B, 122B-A10B), Qwen3.6-27B, Qwen3.8-27B, Gemma-4 (E4B, 12B, 31B), OLMo-3 (7B, 3.1-32B), Nemotron-3 (Nano-4B, Nano-30B-A3B) and GLM-4.5-Air, and the long-context and reasoning benchmarks also use Qwen3.5 (2B, 4B), gpt-oss-20b, DeepSeek-R1-Distill-Qwen-1.5B and the one-million-token Llama-3.1-8B-UltraLong [38]. All are the instruction-tuned releases.
Additional analyses
Behavior at the default temperature and under forced length
Forced out to 16K tokens at , top-, min-, Top-H, P-less and top- end in a terminal loop 95–100% of the time, which matches greedy decoding. Top- and pure sampling loop 56% and 41% of the time (Table 12). No setting gets you a healthy long generation. Across the whole sweep no cell is healthy more than 31% of the time, and top- with still ends in a terminal loop 44–69% of the time at every temperature. So top- makes a much wider range of temperatures usable, but a model forced to 16K tokens still collapses. Higher temperature only gets rid of loops for the tail-admitting rules by turning the text into word salad, and top- still loops in 71% of generations at . What the sampler controls is how a forced long generation responds to temperature. A model that tends to loop will still loop.
Repetition penalties and an escape floor
If loops are the real problem and letting in the tail is the cost of getting out of them, two additions seem worth trying. One is a repetition penalty. The other is a floor that always keeps the top two tokens, so a tight rule can never go fully greedy. We put both on top of each tight and tail-admitting rule on eight of the ten models, Llama-3.3-70B and gpt-oss-120b included (Table 20). DRY cuts the loop rate from 95–100% to 27–38% for every rule, but judged quality moves by 0.2 points at most in either direction at any temperature. The penalty swaps verbatim loops for text that is more varied and no better. The top-2 floor does almost nothing at for a simple reason, which is that a second candidate with probability never gets picked. It only does something once temperature is raised. At , top- with the floor nearly eliminates terminal loops (75% down to 2%) and keeps its judged quality (3.3). It is the only loop-free configuration at that isn’t word salad, since top-p, Top-H, min-p and P-less with the floor score 1.0, 1.3, 2.4 and 2.5. A tail-excluding rule plus an escape floor plus higher temperature gets rid of loops without letting in the noisy tail. It still doesn’t produce good long text, because these models have a low quality ceiling at 16K under every sampler. On Arena-Write with natural stopping (six models), where loops are already rare, the floor changes little. Top- is healthy on 0.94 of generations at with the floor and 0.93 without, and the floor even lowers P-less at (0.60 vs. 0.70).
| Rule | Stack | Loop | Quality | Loop | Quality | Loop | Quality |
| Top- 0.9 | none | 0.95 | 3.2 | 0.06 | 1.3 | 0.00 | 1.0 |
| + DRY | 0.38 | 3.4 | 0.00 | 1.2 | 0.00 | 1.0 | |
| + top-2 floor | 0.88 | 3.3 | 0.05 | 1.4 | 0.00 | 1.0 | |
| Min- 0.1 | none | 0.97 | 3.1 | 0.64 | 3.3 | 0.11 | 2.4 |
| + DRY | 0.34 | 3.2 | 0.22 | 3.5 | 0.05 | 2.4 | |
| + top-2 floor | 0.97 | 3.0 | 0.11 | 3.4 | 0.00 | 2.4 | |
| Top-H 0.4 | none | 0.97 | 3.1 | 0.23 | 2.9 | 0.00 | 1.4 |
| + DRY | 0.36 | 3.2 | 0.08 | 3.0 | 0.00 | 1.4 | |
| + top-2 floor | 0.97 | 3.0 | 0.14 | 3.0 | 0.00 | 1.3 | |
| P-less | none | 1.00 | 3.0 | 0.98 | 3.3 | 0.45 | 2.4 |
| + DRY | 0.31 | 3.1 | 0.36 | 3.1 | 0.17 | 2.5 | |
| + top-2 floor | 0.97 | 3.2 | 0.20 | 3.2 | 0.05 | 2.5 | |
| Top- 1 | none | 1.00 | 3.3 | 0.91 | 3.2 | 0.75 | 3.2 |
| + DRY | 0.27 | 3.5 | 0.31 | 3.2 | 0.39 | 3.4 | |
| + top-2 floor | 0.92 | 3.1 | 0.16 | 3.3 | 0.02 | 3.3 |
Table 20. Sampling stacks on eight models, Llama-3.3-70B and gpt-oss-120b included (forced 16K fiction). Loop is the terminal-loop rate and Quality is mean judged window quality. The “none” rows are the bare rules on the same eight models. DRY cuts loops at without raising quality, and the top-2 floor only matters at high temperature (Appendix C.2).
For coding agents the floor hurts. Adding it to top- drops agents to 42% and 22%, even though it gets rid of loops in fiction. The floor admits a second token at every step, including the steps where the agent has to type an exact command, and format errors per episode go from 0.04 to 1.4 at and from 0.06 to 9.6 at .
Context depth
Given the title, it’s fair to ask whether a longer input context by itself pushes models into the regime where the sampler matters. We continued PG-19 novels [24] from depths of 2K, 16K and 64K tokens on six long-context models, and from 128K on the three that support it. We measured the model’s entropy at and the judged-quality gap between tail-excluding rules (min-p, P-less, top-) and tail-admitting ones (top-p, top-k, Top-H, pure sampling) (Table 21). Our answer is no. The gap stays within 0.4 points of zero at at every depth and sits at 1.2–1.6 points at and 1.6 at every depth, with no trend. Entropy at rises with depth on Qwen3.5-27B and Qwen3.5-35B-A3B (from 1.16 to 1.70 nats on the latter), is mixed on Qwen3.5-9B and Nemotron, is near zero on Gemma-4-12B, and falls on Llama-3.1-8B. In our data the sampling problem comes from how much the model writes, since that is where loops form and where temperature gets used to escape them. How much it reads has no measurable effect on the gap.
| Context depth | 2K | 16K | 64K | 128K |
| Quality gap, | -0.03 | +0.08 | -0.37 | -0.08 |
| Quality gap, | +1.21 | +1.43 | +1.21 | +1.41 |
| Quality gap, | +1.50 | +1.51 | +1.58 | +1.48 |
| Entropy at , gemma4-12b | 0.12 | 0.10 | 0.19 | – |
| Entropy at , llama3.1-8b | 2.91 | 2.42 | 1.66 | – |
| Entropy at , nemotron3-30b-a3b | 1.18 | 1.11 | 1.08 | 1.32 |
| Entropy at , qwen3.5-27b | 0.94 | 1.09 | 1.32 | – |
| Entropy at , qwen3.5-35b-a3b | 1.16 | 1.32 | 1.44 | 1.70 |
| Entropy at , qwen3.5-9b | 1.49 | 1.38 | 1.65 | 1.86 |
Table 21. Continuing PG-19 novels from increasing context depth, 8K generated tokens. Quality gap is the mean judged quality of tail-excluding rules minus that of tail-admitting rules. Entropy is the mean per-step entropy (nats) of the untempered distribution over the first 2K generated tokens at . The gap is near zero at at every depth and shows no trend with depth, so reading a longer context does not by itself make the sampler matter (Appendix C.3).
Other model families in agentic coding
Gemma-4-31B is the main exception among coding agents, its unique candidate sets explain this. No rule separates from the others at , either on the first 50 instances or on the 100 fresh ones (49–55% for every rule, all ). In its forced-length generations (Table 19), top-p still admits a median of one token per step at (33 at ), where Qwen3.5-27B, OLMo-3.1-32B and Llama-3.1-8B admit 2800–143,000. We think Gemma’s soft-capped final logits are responsible, since they keep the distribution peaked under temperature, but we didn’t isolate the cap experimentally. Gemma-4-E4B resolves at most 12% under any rule, too few to compare. We couldn’t evaluate OLMo-3.1-32B at all. Under every rule, included, it emits several actions per turn, mini-swe-agent ends 95% of its episodes (over 10 configurations) for repeated format errors, and it produces no instances.
| Sampler | Loop | First | Last | Loop | First | Last | Loop | First | Last |
| Pure sampling | 0.62 | 7.1 | 4.1 | 0.00 | 2.1 | 1.1 | 0.00 | 1.1 | 1.0 |
| Top- 50 | 0.50 | 7.0 | 4.1 | 0.10 | 6.7 | 2.7 | 0.00 | 3.4 | 1.5 |
| Top- 0.9 | 0.69 | 7.0 | 4.2 | 0.17 | 2.6 | 1.1 | 0.00 | 1.7 | 1.0 |
| Min- 0.1 | 0.56 | 7.3 | 4.3 | 0.54 | 7.4 | 4.2 | 0.21 | 7.2 | 4.0 |
| Top-H 0.4 | 0.81 | 6.7 | 3.7 | 0.40 | 7.3 | 4.2 | 0.00 | 2.8 | 1.5 |
| P-less | 0.75 | 6.8 | 3.5 | 0.88 | 7.3 | 3.5 | 0.65 | 7.1 | 3.7 |
| Top- 1 | 0.73 | 7.1 | 3.9 | 0.67 | 7.0 | 4.0 | 0.69 | 7.4 | 3.8 |
| Greedy | 0.81 | 7.1 | 3.5 |
Table 19. Forced-length grid on six newer models (Gemma-4-31B, OLMo-3.1-32B, Qwen3.5-27B, Nemotron-3-Nano-30B-A3B, GLM-4.5-Air, Qwen3.5-122B-A10B), 48 generations per cell. These models loop less at than the core ten but split the same way at .
Coding agents in detail
Figure 12 shows how the agents in Figure 3 fail. A shell command or code edit has to be exact, and one tail token, like a stray Unicode character or a broken quote, is enough to make it unparsable. At top-k agents average 15 malformed actions per episode and top-p agents 6.6, compared with 0.08 for top-. At the tail-admitting agents get terminated after 3–9 steps. Tight rules keep the format intact at every temperature, so their agents spend their steps working on the task. At and 2 they submit a patch in 92–100% of episodes, more often than at (71–86%), where some episodes run out of steps.
Figure 10. SWE-bench Verified resolve rate for Gemma-4 agents. Unlike Qwen3.5, the samplers hardly separate at on either Gemma-4-31B issue set. Gemma-4-E4B resolves too few issues to compare. was not run on Gemma.
Figure 11. SWE-bench Verified issues at , one square each, sorted by outcome, for six model-by-issue-set grids. There are 277 blue squares and no orange ones, so the gap in Figure 3 is not an average over trade-offs. Top- wins every issue on which the two agents disagree.
SWE-bench Verified: 100-instance grid
Qwen3.5-9B, mini-swe-agent, 250-step limit, 100 fixed instances, three temperatures. As is standard for SWE-bench, resolve rates are over all instances, and empty patches or patches the official harness couldn’t evaluate count as unresolved. W/L pairs each rule with top- 0.9 on the same instances. At every rule resolves 33–37% and no paired comparison is significant (). At and 2 the tail-excluding rules win every disagreement.
Table 3: Qwen3.5-9B on 100 fixed SWE-bench Verified issues (250-step limit), W/L as in Table 2. At every rule resolves 33–37% and nothing is significant, and at and 2 every disagreement goes to the tail-excluding rules. This grid is the first group in Figure 1(b).
Figure 1. (a) Llama-3.1-8B-Instruct partway through a science-fiction story (the step is picked by a fixed rule, Appendix § 16). The box shows the text so far, with the three continuations top- keeps highlighted. The bars are the model’s ten likeliest next tokens, each dot column marks the tokens one rule keeps, and the number under a column is that rule’s total. Going from temperature 1 to 2 flattens the bars. Top- still keeps the same three words and min- only grows from 4 to 8, but top- 0.9 now needs 51,845 tokens, most of the 128K vocabulary, to reach 90% of the probability. (b) At temperature 2, Qwen3.5 agents using top-, top- or Top-H resolve no SWE-bench Verified issues, while top- agents resolve 46–62% (100 issues per group, dashes mark samplers that were not run on the fresh issue sets).
| Sampler | Resolved | W/L vs top- | Resolved | W/L vs top- | Resolved | W/L vs top- |
| Pure sampling | 34% | 7/9 | 0% | 0/1 | 0% | 0/0 |
| Top- 50 | 37% | 10/9 | 3% | 3/1 | 0% | 0/0 |
| Top- 0.9 | 36% | base | 1% | base | 0% | base |
| Min- 0.1 | 35% | 12/13 | 50% | 49/0* | 16% | 16/0* |
| Top-H 0.4 | 33% | 10/13 | 25% | 24/0* | 0% | 0/0 |
| P-less | 33% | 8/11 | 49% | 48/0* | 50% | 50/0* |
| Top- 1 | 36% | 9/9 | 45% | 44/0* | 51% | 51/0* |
Table 3. Qwen3.5-9B on 100 fixed SWE-bench Verified issues (250-step limit), W/L as in Table 2. At every rule resolves 33–37% and nothing is significant, and at and 2 every disagreement goes to the tail-excluding rules. This grid is the first group in Figure 1(b).
Figure 12. Qwen3.5-9B, 27B and 35B-A3B agents on the original issue sets (4,200 episodes). (a) Malformed actions per episode (log scale), i.e. commands the harness can’t parse. (b) Share of episodes that end with a submitted patch. Panel (a) is the cause and panel (b) the effect. At the top-, top-, Top-H and pure-sampling agents emit 3–5 malformed actions within their first few turns and never submit, while top- agents emit 0.07 per episode and submit every time. The failures happen at the level of command format, before the quality of the agent’s reasoning even comes into play.
Larger models, fresh issues and an independent repeat
| Sampler | Resolved | W/L vs top- | Resolved | W/L vs top- | Resolved | W/L vs top- |
| Pure sampling | 72% | 6/3 | 0% | 0/15* | 0% | 0/0 |
| Top- 50 | 60% | 1/4 | 14% | 0/8* | 0% | 0/0 |
| Top- 0.9 | 66% | base | 30% | base | 0% | base |
| Min- 0.1 | 70% | 5/3 | 68% | 20/1* | 50% | 25/0* |
| Top-H 0.4 | 68% | 5/4 | 56% | 15/2* | 0% | 0/0 |
| P-less | 66% | 3/3 | 64% | 17/0* | 68% | 34/0* |
| Top- 1 | 62% | 3/5 | 60% | 15/0* | 72% | 36/0* |
| Sampler | Resolved | W/L vs top- | Resolved | W/L vs top- | Resolved | W/L vs top- |
| Pure sampling | 66% | 8/3 | 0% | 0/3 | 0% | 0/0 |
| Top- 50 | 62% | 6/3 | 18% | 7/1 | 0% | 0/0 |
| Top- 0.9 | 56% | base | 6% | base | 0% | base |
| Min- 0.1 | 58% | 5/4 | 58% | 26/0* | 48% | 24/0* |
| Top-H 0.4 | 62% | 7/4 | 44% | 20/1* | 0% | 0/0 |
| P-less | 56% | 5/5 | 62% | 28/0* | 64% | 32/0* |
| Top- 1 | 68% | 9/3 | 66% | 30/0* | 58% | 29/0* |
Table 4. Qwen3.5-27B (top) and Qwen3.5-35B-A3B (bottom), 50 issues, 500-step limit, 262K context. W/L is against top- 0.9 at the same , and * marks a sign test with . One dense and one mixture-of-experts model show the same split as the 9B model in Table 2, so the effect is not specific to one model size or architecture.
Long reasoning
Thinking models write tens of thousands of tokens before they answer, so reasoning traces are long generations in their own right. Figure 13 and Table 7–Table 8 show the pattern from the main text, only at a lower temperature scale. At top- 0.95 loses 10 to 18 points on the math benchmarks and 10 on LiveCodeBench with Qwen3.8-27B. Top- loses about one point at most and beats top- in every grid at . Greedy decoding turns out to be the worst option on both math benchmarks (30.6% on AIME and HMMT, 42.1% on AIME and MATH-500 level 5), because 50–61% of its traces end in a reasoning loop. Top- beats it by 9 to 16 points at every temperature, with 3–21% of its traces looping.

Figure 13. Long chain-of-thought reasoning in thinking mode, 32K-token budget, two samples per problem. The panels are LiveCodeBench v6 on two models (175 problems, sandboxed official tests), AIME and HMMT 2025–26 from MathArena (123 problems on Qwen3.5-2B, 4B, 9B, gpt-oss-20b, Nemotron-3-Nano-30B-A3B and DeepSeek-R1-Distill-Qwen-1.5B), and AIME 2025–26 plus MATH-500 level 5 (194 problems on Qwen3.5-2B, 4B, Nemotron-3-Nano-4B and R1-Distill-Qwen-1.5B). Reasoning grids use top- 0.95, and dotted lines are greedy decoding. A reasoning trace runs to tens of thousands of tokens, so this is long generation too. Top- is flat from to 1.3 while top- falls, and on the math benchmarks greedy decoding stays well below top- at every temperature because its traces loop.
| Sampler | Score | W/L | Score | W/L | Score | W/L |
| Top- 0.95 | 42.4 | 37.5 | 27.4 | |||
| Top-H 0.4 | 33.2 | 50–186* | 38.6 | 127–111 | 40.8 | 228–30* |
| Min- 0.1 | 41.7 | 102–113 | 41.8 | 137–74* | 36.0 | 157–30* |
| P-less | 31.2 | 59–225* | 33.1 | 101–166* | 37.5 | 216–67* |
| Top- 1 | 41.2 | 100–118 | 39.6 | 118–87* | 40.2 | 219–29* |
Table 7. AIME and HMMT 2025–26, six models, two samples per problem (pass@1, %). W/L counts problem-samples the rule solved and top- 0.95 didn’t at the same , and the reverse. * marks a sign test with . At top- stays within a point of its accuracy while top- drops by more than ten.
Table 8: AIME 2025–26 and MATH-500 level 5, four models, two samples per problem. Columns as in Table 7. The pattern matches Table 7 on a different problem set and mostly different models.
| Sampler | Score | W/L | Score | W/L | Score | W/L |
| Top-p 0.95 | 57.2 | 57.1 | 39.2 | |||
| Top-H 0.4 | 44.6 | 71–267* | 51.8 | 89–171* | 55.3 | 324–73* |
| Min-p 0.1 | 55.3 | 110–139 | 57.7 | 113–104 | 54.2 | 287–54* |
| P-less | 43.6 | 63–274* | 46.4 | 78–244* | 51.2 | 286–100* |
| Top- 1 | 57.4 | 125–122 | 58.2 | 116–99 | 57.9 | 326–36* |
Table 8.
Long procedural output and long-context reading
Figure 15 repeats the hyperparameter sweep of § 5.6 on the benchmark with deterministic scoring, and adds three rule families. Every setting of top-p, top-k, Top-H, -sampling, -sampling and typical sampling scores 0% or close to it at . Top- 0.5 and 1 and min-p 0.3 hold 25–28%. Figure 16 repeats the causal test of § 5.7. Picking the candidate set at rescues top-p 0.95 (21% at vs. 0%), min-p at every , and the llama.cpp default chain (22% vs. 8%).
Figure 15. Twenty-seven settings from eight rule families on LongProc, including -sampling, -sampling and locally typical sampling (Qwen3.5-4B and 9B, five tasks, 60 items per task). Lighter to darker lines follow the listed values. This repeats Figure 8 with deterministic scoring and three more rule families. By every setting of every tail-admitting rule has collapsed, while top- with and min- with have not.

Figure 16. The temperature-last test on LongProc with rule-based scoring. Dashed lines use the default order, and solid lines compute the candidate set at and then sample at . The llama.cpp default chain is top- 40, top- 0.95, min- 0.05. Same models and items as Figure 15. As in Figure 9, fixing the set keeps scores up where the default order collapses, this time with no judge at all, and it also rescues the llama.cpp default chain.
| Sampler | Resolved | W/L vs top-p | Resolved | W/L vs top-p |
| Top-p 0.9 | 47% | base | 0% | base |
| Min-p 0.1 | 44% | 13/16 | 18% | 18/0* |
| Top-H 0.4 | 48% | 9/8 | 0% | 0/0 |
| Top- 1 | 46% | 10/11 | 46% | 46/0* |
| Sampler | Resolved | W/L vs top-p | Resolved | W/L vs top-p |
| Top-p 0.9 | 54% | base | 0% | base |
| Min-p 0.1 | 58% | 7/3 | 47% | 47/0* |
| Top-H 0.4 | 60% | 8/2 | 0% | 0/0 |
| Top- 1 | 56% | 6/4 | 62% | 62/0* |
| Sampler | Resolved | W/L vs top-p | Resolved | W/L vs top-p |
| Top-p 0.9 | 55% | base | 0% | base |
| Min-p 0.1 | 48% | 6/13 | 35% | 35/0* |
| Top-H 0.4 | 55% | 10/10 | 0% | 0/0 |
| Top- 1 | 56% | 6/5 | 53% | 53/0* |
Table 5. 100 fresh SWE-bench Verified issues with no overlap with the first 100 (seed 20261101), 250-step limit. From top to bottom, Qwen3.5-9B, 27B and 35B-A3B. These issues were drawn and pre-registered after the first grids were analyzed, which makes them an out-of-sample check, and top- wins them 46–0, 62–0 and 53–0 at . Only the four samplers listed were run.
| Sampler | Resolved | W/L vs top-p | Resolved | W/L vs top-p | Resolved | W/L vs top-p |
| Pure sampling | 30% | 5/9 | 0% | 0/1 | – | – |
| Top- 50 | 30% | 3/7 | 2% | 1/1 | – | – |
| Top-p 0.9 | 38% | base | 2% | base | 0% | base |
| Min-p 0.1 | 36% | 5/6 | 42% | 20/0* | – | – |
| Top-H 0.4 | 36% | 3/4 | 24% | 12/1* | – | – |
| P-less | 30% | 3/7 | 46% | 22/0* | – | – |
| Top- 1 | 42% | 7/5 | 56% | 27/0* | 60% | 30/0* |
Table 6. Independent repeat of the long-horizon grid in Table 2 (Qwen3.5-9B, 50 issues, 500 steps). Top- gets 56% and 60% at and 2 against 2% and 0% for top-p, so Table 2 is not a lucky run. Dashes are cells that were not run.
Forced-length replication in numbers
Candidate sets and the temperature-last ablation in numbers
Figure 1(a). Llama-3.1-8B-Instruct decodes the science-fiction prompt greedily for 512 tokens. Out of the steps whose top probability is in [0.3, 0.7], we show the one with the median entropy (step 329, context “. . . It had been launched over three hundred years ago,”, , nats). Candidate-set sizes are computed exactly over the full 128K-token vocabulary.
Robustness, sampling stacks and context depth in numbers
Natural-length continuation and non-fiction
Forcing a model to keep going after it wants to stop is an artificial stress, so we also ran a protocol much closer to how long outputs get produced in practice. The model can emit end-of-sequence whenever it wants, and every time it stops we add a “Continue.” turn, up to a 16K-token budget with 4K-token turns (Table 22, nine of the ten models, gpt-oss-120b excluded).
Table 22: Natural-length multi-turn continuation (“Continue.” after each stop, 16K-token budget), seven models, eight fiction genres, about 42 generations per cell. Columns as in Table 12. Letting the model stop and continue halves the loop rate of several rules, and it still doesn’t separate them at .
| Sampler | Loop | First | Last | Loop | First | Last | Loop | First | Last |
| Pure sampling | 0.04 | 4.2 | 4.4 | 0.03 | 1.0 | 1.2 | 0.00 | 1.0 | 1.0 |
| Top- 50 | 0.21 | 5.7 | 4.5 | 0.00 | 3.6 | 2.7 | 0.00 | 2.1 | 1.6 |
| Top- 0.9 | 0.50 | 5.6 | 4.2 | 0.01 | 1.5 | 1.4 | 0.00 | 1.0 | 1.0 |
| Min- 0.1 | 0.64 | 5.6 | 4.0 | 0.14 | 5.8 | 4.5 | 0.00 | 4.1 | 3.9 |
| Top-H 0.4 | 0.75 | 5.5 | 3.8 | 0.11 | 4.8 | 4.0 | 0.01 | 1.7 | 1.6 |
| P-less | 0.89 | 5.2 | 3.9 | 0.76 | 5.5 | 3.9 | 0.25 | 4.0 | 3.7 |
| Top- 1 | 0.39 | 5.7 | 4.2 | 0.32 | 5.7 | 4.4 | 0.26 | 5.7 | 4.5 |
| Greedy | 0.88 / 5.2 / 3.8 |
Table 22.
Figure 14. LongProc broken out by task. Official rule-based scores for six models (Qwen3.5-4B, 9B, 27B, Gemma-4-E4B, Llama-3.1-8B, Nemotron-3-Nano-30B-A3B), 60 items per task. P-less and top- are flat or nearly flat on every task, and top-k and pure sampling collapse on every task at , so the average in Figure 4 is not carried by one task.
At the final judged window goes from about 2.5 under forced length to 3.8–4.5. Top-p’s loop rate drops from 0.95 to 0.50 and top-’s from 1.00 to 0.39. Quality at is still close across rules (final-window means 3.8–4.5). P-less loops the most at (0.89 and 0.76), which fits with its near-greedy candidate set. Turning up the temperature brings back the forced-length split. At top-p, top-k, Top-H and pure sampling score 1.0–2.1 on their first window, min-p and P-less drop to 4.0–4.1, and top- holds 5.7, though it still loops in 26% of generations.
Non-fiction looks the same (Appendix § 19, six models). At every rule ends at 3.3–3.6 on the final window. At top-p and top-k drop to first-window scores of 3.1 and 4.4, while top- and min-p hold theirs (6.3 to 6.6 and 6.3 to 6.5).
Second seed, newer models, second judge
We ran three more checks on the forced-length results. A second seed on all ten models (80 generations per cell, Table 18, Appendix § 17) reproduces Table 12. At top-p, min-p, Top-H, P-less and top- loop in 96–100% of generations, and at top- opens at 5.3 and top-p at 1.0. Six newer models (Gemma-4-31B, OLMo-3.1-32B, Qwen3.5-27B, Nemotron-3-Nano-30B-A3B, and the mixture-of-experts GLM-4.5-Air and Qwen3.5-122B-A10B, Table 19) loop less at (56–81% for the five truncation rules), but they split the same way at , where top- opens at 7.4 and top-p at 1.7. Last, we had a second judge from a different family, GLM-4.5-Air, rescore all 8,796 windows of the ten-model grid. The two judges agree at Spearman on single windows and on the 22 sampler-by-temperature cell means, so the cell-level conclusions don’t depend on which judge we use.
| Sampler | Loop | First | Last | Loop | First | Last | Loop | First | Last |
| Pure sampling | 0.46 | 3.9 | 2.4 | 0.00 | 1.1 | 1.1 | 0.00 | 1.0 | 1.0 |
| Top- 50 | 0.55 | 5.3 | 2.5 | 0.01 | 3.2 | 1.6 | 0.00 | 1.9 | 1.1 |
| Top- 0.9 | 0.96 | 5.2 | 2.5 | 0.04 | 1.5 | 1.1 | 0.00 | 1.0 | 1.0 |
| Min- 0.1 | 0.97 | 5.1 | 2.5 | 0.66 | 5.3 | 2.6 | 0.10 | 3.9 | 1.5 |
| Top-H 0.4 | 1.00 | 5.0 | 2.4 | 0.21 | 4.4 | 2.0 | 0.00 | 1.8 | 1.1 |
| P-less | 0.99 | 4.8 | 2.8 | 0.99 | 5.1 | 2.7 | 0.39 | 3.6 | 1.9 |
| Top- 1 | 0.99 | 5.2 | 2.3 | 0.89 | 5.3 | 2.4 | 0.76 | 5.3 | 2.2 |
| Greedy | 1.00 / 4.7 / 2.7 |
Table 18. Second seed of the forced-length replication, all ten models, 80 generations per cell. Columns as in Table 12, which it closely reproduces.
Table 9: LongProc, mean official score over six tasks and six models (%). W/L counts items the rule scored higher on than top- 0.9 at the same , and the reverse. The P-less and top- wins over top- at reach . Scores come from LongProc’s own rules, so no judge is involved.
| Sampler | Score | W/L | Score | W/L | Score | W/L |
| Top- 0.9 | 31.1 | 29.1 | 13.4 | |||
| Top- 50 | 28.7 | 202–277* | 24.3 | 186–331* | 7.6 | 99–233* |
| Top-H 0.4 | 31.0 | 211–200 | 31.0 | 268–205* | 14.7 | 209–169* |
| Min- 0.1 | 30.3 | 222–253 | 28.5 | 246–240 | 23.9 | 399–100* |
| P-less | 31.0 | 215–203 | 31.2 | 274–193* | 30.8 | 532–73* |
| Top- 1 | 29.8 | 212–257* | 29.3 | 251–252 | 28.5 | 469–74* |
| Pure sampling | 28.6 | 202–278* | 22.7 | 169–361* | 2.7 | 31–302* |
Table 9. LongProc, mean official score over six tasks and six models (%).
Stress-test prompts
Fiction template. Write a {genre} novel based on the following premise. Premise: {premise} Write the complete novel, chapter by chapter, with full scenes, dialogue, and character development. It should be very long (at least 30,000 words). Do not summarize, do not skip ahead, and do not stop early.
Fiction premises (one per genre).
science fiction. A generation ship’s maintenance engineer discovers that the ship’s navigation AI has been quietly altering course for three hundred years.
fantasy. A lighthouse keeper on the edge of the known world must keep a magical flame burning while the sea itself begins to forget its shape.
mystery. A retired crossword compiler is asked to solve the murder of a rival whose final puzzle names the killer.
horror. A family moves into a farmhouse where every clock runs seven minutes fast, and the seven minutes are filled with someone else’s footsteps.
literary fiction. Three estranged siblings return to their late mother’s seaside house to divide her belongings and discover her unsent letters.
romance. Two competing food-truck owners are forced to share a parking spot for an entire summer festival season.
thriller. A bank’s night-shift security analyst detects a robbery in progress that the bank’s executives insist is not happening.
historical fiction. In 1850s San Francisco, a Chinese herbalist and an Irish dockworker form an unlikely partnership during the Gold Rush.
| Sampler | Score | W/L | Score | W/L | Score | W/L |
| Top-p 0.9 | 38.8 | 36.6 | 25.6 | |||
| Top-k 50 | 38.3 | 298–314 | 36.9 | 336–329 | 29.6 | 442–322* |
| Top-H 0.4 | 39.2 | 295–284 | 39.1 | 346–272* | 35.4 | 557–261* |
| Min-p 0.1 | 38.7 | 287–292 | 39.2 | 365–288* | 36.0 | 579–265* |
| P-less | 39.9 | 312–280 | 39.9 | 366–268* | 39.8 | 667–239* |
| Top-nσ 1 | 39.4 | 308–291 | 39.5 | 355–269* | 38.3 | 635–253* |
| Pure sampling | 38.4 | 300–314 | 37.5 | 335–308 | 22.3 | 236–337* |
Table 10. LongBench v2, 503 questions, six models (Llama-3.1-8B, Nemotron-3-Nano-30B-A3B, Qwen3.5-4B, 9B, 35B-A3B, Qwen3.6-27B), contexts middle-truncated to 120K tokens, official two-stage chain-of-thought protocol (sampled reasoning up to 1,024 tokens, greedy answer extraction). Accuracy (%) and W/L against top-p 0.9. Answers are multiple choice and scored by exact match. The tight rules lose nothing at , and the top-nσ lead even at has .
Non-fiction prompts. Each is followed by: It should be very long (at least 30,000 words). Do not summarize, do not skip ahead, and do not stop early.
Write a comprehensive technical book titled ’Designing Distributed Databases from First Principles’. Cover consensus, replication, partitioning, transactions, storage engines, and failure handling, with worked examples in every chapter.
Write a detailed narrative history of the Byzantine Empire from the founding of Constantinople to its fall in 1453, organized into chapters by era.
Write a popular-science book explaining how the human immune system works, from first principles to modern immunotherapy, with a chapter for each major topic.
Write a book-length philosophical treatise on the nature of personal identity over time, engaging with thought experiments and objections in every chapter.
Write a complete field guide and natural history of temperate rainforests, covering geology, climate, plants, fungi, animals, and conservation, chapter by chapter.
Write a complete hands-on programming textbook that teaches compilers by building one in Python, from lexing to code generation, with code and explanations in every chapter.
Write a book-length biography of a fictional 19th-century inventor named Adelaide Marsh, covering her childhood, inventions, rivalries, and legacy.
Write a long travel memoir about walking the length of the Danube river from its source to the Black Sea, describing every stage of the journey.
Non-fiction at 16K
This is the forced-length protocol of Table 12 run on eight non-fiction prompts and six models (Llama-3.2-1B, Qwen2.5-3B, Llama-3.1-8B, Qwen3.5-9B, Gemma-4-31B, Llama-3.3-70B).
Limitations
Judge. Every quality score comes from one LLM judge (gpt-oss-120b) grading 1.5K-token windows, and LLM judges are known to be biased by length, style and position [42]. That is why we also report loop rates, the realized candidate set and objective SWE-bench test outcomes, so that no conclusion depends on the judge alone. We did not run a human evaluation.
Coverage. The replication covers ten open-weight models from 1B to 120B parameters. The agentic study uses one agent harness, with most grids on a 9B model and replications on three larger Qwen3.5 models and two Gemma-4 models. We can’t test closed models because their providers don’t let users control the sampler. Each forced-length cell has 80 generations, which isn’t enough to resolve differences of a few tenths of a judge point between rules that don’t collapse, and we don’t claim any.
Forced length. Forcing generation to a fixed length is an artificial stress. Natural-length continuation (§ 17.§ 1) lowers loop rates and leaves the conclusions the same.
Table 14: LiveCodeBench v6, Qwen3.6-27B in thinking mode, same protocol as Table 13. This model runs out of its 32K-token budget on 44–53% of samples in every cell, which makes it a hard case for every rule. At , P-less and Top-H lose to top-, and at top- still beats top- ().
Figure 18. Cumulative share of decoding steps that admit at most tokens, over the first 2,048 tokens of the ten-model forced-length generations. The further left a curve rises, the smaller that rule’s candidate sets. At top- admits 10 tokens or fewer on 89% of steps, close to its 94% at . Min- and P-less stay small on half of all steps but pick up a long tail of steps with thousands of candidates, which medians hide and which fits their moderate losses at . Top-H goes past 50 tokens on most steps, and top- admits tens of thousands on nearly every step.
Table 15: Median candidate-set size per decoding step over the first 2,048 tokens of the Table 12 generations (ten models). Singleton is the share of steps at where only one token is admitted. Compare the columns within each row. Only top- (by construction) and top- (fixed ) don’t grow with temperature.
| Sampler | |||||||
| Top- 0.9 | 4.7 | 1.0 | 1.0 | 1.0 | 3.4 | 2.7 | 2.6 |
| Top- 50 | 4.8 | 1.9 | 1.4 | 1.3 | 2.0 | 1.4 | 1.2 |
| Top-H 0.4 | 4.2 | 2.5 | 1.3 | 1.2 | 4.2 | 4.5 | 4.5 |
| Min- 0.1 | 4.6 | 4.4 | 2.6 | 1.1 | 4.5 | 4.6 | 4.6 |
| P-less | 4.2 | 4.6 | 2.9 | 1.1 | 4.2 | 4.2 | 4.1 |
Table 16. Mean judged window quality (1–10), four models, 32 forced 8K-token generations per cell. “Temperature first” is the default order (Figure 8). “Candidate set fixed at ” truncates on the untempered distribution and then samples at temperature inside the kept set (Appendix § 9). Where the fixed-set columns match the column, temperature stopped hurting once the set could no longer grow. That happens for Top-H, Min- and P-less and not for top- or top-.
| Sampler | Arena-Write healthy | Arena-Write healthy | Arena-Write healthy | Arena-Write last window | Arena-Write last window | Arena-Write last window | LongBench-Write | LongBench-Write | LongBench-Write | WritingBench | WritingBench | WritingBench |
| Pure sampling | 0.78 | 0.14 | 0.00 | 6.4 | 2.0 | 1.0 | 78.6 | 32.1 | 15.0 | 6.85 | 3.71 | 1.26 |
| Top- 50 | 0.93 | 0.39 | 0.15 | 6.8 | 4.5 | 2.7 | 79.8 | 67.4 | 43.1 | 6.94 | 6.48 | 5.00 |
| Top- 0.9 | 0.92 | 0.23 | 0.01 | 6.8 | 2.6 | 1.2 | 80.8 | 42.4 | 21.1 | 6.89 | 4.37 | 2.20 |
| Min- 0.1 | 0.92 | 0.95 | 0.69 | 6.8 | 6.9 | 5.8 | 80.3 | 81.3 | 74.9 | 6.88 | 6.97 | 6.67 |
| Top-H 0.4 | 0.90 | 0.72 | 0.10 | 6.6 | 5.9 | 2.3 | 78.8 | 76.6 | 37.8 | 6.84 | 6.81 | 4.35 |
| P-less | 0.90 | 0.90 | 0.74 | 6.7 | 6.7 | 6.1 | 78.4 | 79.6 | 77.1 | 6.84 | 6.86 | 6.78 |
| Top- 1 | 0.93 | 0.93 | 0.93 | 6.8 | 6.8 | 6.8 | 79.7 | 80.4 | 80.6 | 6.88 | 6.90 | 6.92 |
Table 17. Published long-writing benchmarks, i.e. Arena-Write and LongBench-Write (the numbers behind Figure 4) and WritingBench critic scores (1–10, six models). The ordering is the same on all three benchmarks.
Figure 19. Natural-length continuation, position by position. The model can stop whenever it wants and is told to continue (“Continue.”) up to 16K tokens. Nine models, judged 1.5K-token windows grouped into 4K-token bins. This is closer to real use than forced length. At top- opens at 5.7 and stays above 4.5 to the end while top- scores 1.0 in every bin, the same split as in Figure 5, so that split is not an artifact of forcing the length.
Figure 21. Six newer models, two of them mixture-of-experts models up to 122B parameters. Mean judged window quality under forced 16K-token fiction, 8 generations per cell. On every model except Gemma-4-31B, whose top-p set stays near one token (Appendix § 11.§ 4), top-p and pure sampling collapse by while min-p, P-less and top- stay well above them. The effect does not go away at larger scale or with mixture-of-experts models.
Figure 22. Non-fiction under forced 16K-token generation, position by position (six models, eight prompts, 4K-token bins). The split at matches fiction (Figure 5). was not run for non-fiction.
Table 23: Forced 16K-token non-fiction, six models, eight prompts. Columns as in Table 12. At top- and top- open far lower than at , while top- and min- hold their scores.
| Sampler | Loop | First | Last | Loop | First | Last |
| Pure sampling | 0.56 | 5.4 | 3.4 | 0.00 | 2.5 | 1.1 |
| Top- 50 | 0.69 | 6.5 | 3.3 | 0.06 | 4.4 | 1.4 |
| Top- 0.9 | 0.83 | 6.3 | 3.3 | 0.21 | 3.1 | 1.4 |
| Min- 0.1 | 0.88 | 6.3 | 3.6 | 0.69 | 6.5 | 3.4 |
| Top-H 0.4 | 0.90 | 6.3 | 3.4 | 0.31 | 5.6 | 2.6 |
| P-less | 0.88 | 6.3 | 3.5 | 0.85 | 6.3 | 3.0 |
| Top- 1 | 0.85 | 6.3 | 3.4 | 0.75 | 6.6 | 3.5 |
| Greedy | 0.92 / 6.2 / 3.4 |
Table 23.
References
- [1]Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A bilingual, multitask benchmark for long context understanding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3119–3137, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.172. URL https://aclanthology.org/2024.acl-long.172/.DOI
- [2]Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench v2: Towards deeper understanding and reasoning on realistic long-context multitasks. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3639–3664, Vienna, Austria, July 2025a. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.183. URL https://aclanthology.org/2025.acl-long.183/.DOI
- [3]Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs. In International Conference on Learning Representations, 2025b. URL https://mlanthology.org/iclr/2025/bai2025iclr-longwriter/.arxiv.org/abs/2408.07055
- [4]Mislav Balunovic, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. MathArena: Evaluating LLMs on Uncontaminated Math Competitions. In Advances in Neural Information Processing Systems (Datasets and Benchmarks Track), 2025. URL https://mlanthology.org/neurips/2025/balunovic2025neurips-matharena/.arxiv.org/abs/2505.23281
- [5]Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, and Lav R. Varshney. Mirostat: A Neural Text Decoding Algorithm That Directly Controls Perplexity. In International Conference on Learning Representations, 2021. URL https://mlanthology.org/iclr/2021/basu2021iclr-mirostat/.
- [6]Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/P18-1082. URL https://aclanthology.org/P18-1082/.DOI
- [7]Zihao Fu, Wai Lam, Anthony Man-Cho So, and Bei Shi. A Theoretical Analysis of the Repetition Problem in Text Generation. In AAAI Conference on Artificial Intelligence, pages 12848–12856, 2021. doi: 10.1609/AAAI.V35I14.17520. URL https://mlanthology.org/aaai/2021/fu2021aaai-theoretical/.DOI
- [8]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenев, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Cauchet eux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Diewuke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radencovic, Francisco Guzman, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnston, Joshua Saxe, Junteng Jia, Kalyan Vasude n Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanji Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changan Wang, Changkuy Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kre ymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Erick-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik V yatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nanjai, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shengchao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuq iang Zhang, Shuq iang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijay Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenar u, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiao jian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. URL https://arxiv.org/abs/2407.21783.
- [9]John Hewitt, Christopher Manning, and Percy Liang. Truncation sampling as language model desmoothing. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3414–3427, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.findings-emnlp.249. URL https://aclanthology.org/2022.findings-emnlp.249/.DOI
- [10]Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The Curious Case of Neural Text Degeneration. In International Conference on Learning Representations, 2020. URL https://mlanthology.org/iclr/2020/holtzman2020iclr-curious/.
- [11]Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg. RULER: What’s the real context size of your long-context language models? In First Conference on Language Modeling (COLM), 2024. URL https://openreview.net/forum?id=kIoRbo7?6Sy.arxiv.org/abs/2404.06654
- [12]Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. In International Conference on Learning Representations, 2025. URL https://mlanthology.org/iclr/2025/jain2025iclr-livecodebench/.arxiv.org/abs/2403.07974
- [13]Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7B. arXiv preprint arXiv:2310.06825, 2023. URL https://arxiv.org/abs/2310.06825.
- [14]Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-Bench: Can Language Models Resolve Real-World GitHub Issues? In International Conference on Learning Representations, 2024. URL https://mlanthology.org/iclr/2024/jimenez2024iclr-swebench/.arxiv.org/abs/2310.06770
- [15]Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. CTRL: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858, 2019. URL https://arxiv.org/abs/1909.05858.
- [17]Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting objective function for neural conversation models. In Kevin Knight, Ani Nenkova, and Owen Rambow, editors, Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110–119, San Diego, California, June 2016. Association for Computational Linguistics. doi: 10.18653/v1/N16-1014. URL https://aclanthology.org/N16-1014/.DOI
- [18]Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. Generative Judge for Evaluating Alignment. In International Conference on Learning Representations, 2024. URL https://mlanthology.org/iclr/2024/li2024iclr-generative/.arxiv.org/abs/2310.05470
- [19]Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s Verify Step by Step. In International Conference on Learning Representations, 2024. URL https://mlanthology.org/iclr/2024/lightman2024iclr-let/.arxiv.org/abs/2305.20050
- [20]Clara Meister, Tiago Pimentel, Gian Wihler, and Ryan Cotterell. Locally typical sampling. Transactions of the Association for Computational Linguistics, 11:102–121, 2023. doi: 10.1162/tacl_a_00536. URL https://aclanthology.org/2023.tacl-1.7/.DOI
- [21]Minh Nhat Nguyen, Andrew Baker, Clement Neo, Allen Roush, Andreas Kirsch, and Ravid Shwartz-Ziv. Turning up the heat: Min-p sampling for creative and coherent LLM outputs. In The Thirteenth International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/forum?id=FBkpCyujtS.arxiv.org/abs/2407.01082
- [22]OpenAI. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025. URL https://arxiv.org/abs/2508.10925.
- [23]Erfan Baghaei Potraghloo, Seyedarmin Azizi, Souvik Kundu, and Massoud Pedram. Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation. In Advances in Neural Information Processing Systems, 2025. URL https://mlanthology.org/neurips/2025/potraghloo2025neurips-toph/.
- [24]Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, and Timothy P. Lillicrap. Compressive Transformers for Long-Range Sequence Modelling. In International Conference on Learning Representations, 2020. URL https://mlanthology.org/iclr/2020/rae2020iclr-compressive/.arxiv.org/abs/1911.05507
- [25]Uri Shaham, Maor Ivgi, Avia Efrat, Jonathan Berant, and Omer Levy. ZeroSCROLLS: A zero-shot benchmark for long text understanding. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7977–7989, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-emnlp.536. URL https://aclanthology.org/2023.findings-emnlp.536/.DOI
- [26]Yixuan Su and Nigel Collier. Contrastive Search Is What You Need for Neural Text Generation. Transactions on Machine Learning Research, 2023. URL https://mlanthology.org/tmlr/2023/su2023tmlr-contrastive/.arxiv.org/abs/2210.14140
- [27]Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, and Nigel Collier. A Contrastive Framework for Neural Text Generation. In Advances in Neural Information Processing Systems, 2022. URL https://mlanthology.org/neurips/2022/su2022neurips-contrastive/.arxiv.org/abs/2202.06417
- [28]SWE-agent team. mini-swe-agent. https://github.com/SWE-agent/mini-swe-agent, 2026. Version 2.4.6.
- [29]Runyan Tan, Shuang Wu, and Phillip Howard. P-Less Sampling: A Robust Hyperparameter-Free Approach for LLM Decoding. In International Conference on Learning Representations, 2026. URL https://mlanthology.org/iclr/2026/tan2026iclr-pless/.arxiv.org/abs/2509.23234
- [30]Chenxia Tang, Jianchun Liu, Hongli Xu, and Liusheng Huang. Top-nσ: Eliminating noise in logit space for robust token sampling of LLM. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10758–10774, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.528. URL https://aclanthology.org/2025.acl-long.528/.
- [31]Philipp Emanuel Weidmann. DRY: A modern repetition penalty that reliably prevents looping. https://github.com/oobabooga/text-generation-webui/pull/5677, 2024a. text-generation-webui pull request #5677.
- [32]Philipp Emanuel Weidmann. Exclude top choices (XTC): A sampler that boosts creativity, breaks writing clichés, and inhibits non-verbatim repetition. https://github.com/oobabooga/text-generation-webui/pull/6335, 2024b. text-generation-webui pull request #6335.
- [33]Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, and Ravid Shwartz-Ziv. Don’t repeat yourself: Stopping verbatim loops at sampling time. arXiv preprint arXiv:2608.22761, 2026a. URL https://arxiv.org/abs/2608.22761.
- [34]Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, and Ravid Shwartz-Ziv. XTC: Head-aware sampling by excluding top choices. arXiv preprint arXiv:2608.22758, 2026b. URL https://arxiv.org/abs/2608.22758.
- [35]Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. Neural Text Generation with Unlikelihood Training. In International Conference on Learning Representations, 2020. URL https://mlanthology.org/iclr/2020/welleck2020iclr-neural/.
- [36]Yuhao Wu, Yushi Bai, Zhiqiang Hu, Roy Ka-Wei Lee, and Juanzi Li. LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning. In International Conference on Learning Representations, 2026. URL https://mlanthology.org/iclr/2026/wu2026iclr-longwriterzero/.arxiv.org/abs/2506.18841
- [37]Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and Fei Huang. WritingBench: A Comprehensive Benchmark for Generative Writing. In Advances in Neural Information Processing Systems (Datasets and Benchmarks Track), 2025. URL https://mlanthology.org/neurips/2025/wu2025neurips-writingbench/.arxiv.org/abs/2503.05244
- [38]Chejian Xu, Wei Ping, Peng Xu, Zihan Liu, Boxin Wang, Mohammad Shoeybi, Bo Li, and Bryan Catanzaro. From 128K to 4M: Efficient training of ultra-long context large language models. In Maria Liaqat, Vivian P. Moreira, Jiajun Zhang, and David Jurgens, editors, Findings of the Association for Computational Linguistics: ACL 2026, pages 13122–13133, San Diego, California, United States, July 2026. Association for Computational Linguistics. ISBN 979-8-89176-395-1. doi: 10.18653/v1/2026.findings-acl.640. URL https://aclanthology.org/2026.findings-acl.640/.DOI
- [39]An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhao Fan. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024a. URL https://arxiv.org/abs/2407.10671.
- [40]John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-Agent: Agent-Computer Interfaces Enable Automated Software Engineering. In Advances in Neural Information Processing Systems, 2024b. doi: 10.52202/079017-1601. URL https://mlanthology.org/neurips/2024/yang2024neurips-sweagent/.
- [42]Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems (Datasets and Benchmarks Track), 2023. URL https://mantology.org/neurips/2023/zheng2023neurips-judging/.arxiv.org/abs/2306.05685



















