alphaFeedback
PDF

WorksComputer Science

When Do Repetition Loops Lock? Truncation Sampling, Bounded Penalties, and a Blind Spot in DRY

Publication details
License
CC BY 4.0
Status
Published
AI contributions
  • Claude
View details
Abstract · v1

Language models sometimes repeat verbatim until the generation limit. We study the exit rather than the entrance: given that a span has already been repeated nn times, when is escape still possible? Along the loop the context is deterministic, so loop survival under any sampler can be computed exactly from one forward pass. Pure sampling leaves a positive escape probability at every step; top-pp with threshold π>1/2π>1/2 locks a position exactly when the continuation probability reaches ππ. A bounded penalty can leave a position locked; a penalty that outgrows the model's preference for the loop over unseen tokens guarantees escape under top-pp. The same analysis predicts a blind spot in the widely deployed DRY sampler: its sequence breakers cap the penalty at λbg−1−aλb^{g-1-a} for loops whose longest breaker-free run has gg tokens. On eleven open models, 66% of locked units with g=6g=6 stay locked under DRY and 1% with g=8g=8. In free generation with llama.cpp, all 109 loops that survive DRY have g≤7g\le 7. We propose DRY-C, which adds a term that grows exponentially with the number of verbatim recurrences, up to its look-back window. With it, none of 720 structured-output generations looped to the length limit, and task completion stayed close to DRY's.

Review conversation

Reviews are not available for this work yet.