WorksComputer Science
Internalizing Chain-of-thought into Looped Transformers
Abstract
Chain-of-thought (CoT) enables constant-bit-size transformers to perform general-purpose computation through generated tokens (Li and Wang, 2025a), whereas looped transformers carry out the same computation by recurrence in hidden states (latent thought). We construct a family of constant-bit-size looped transformers that interpolates between CoT and latent thought. For a prescribed loop count and a fixed integer satisfying , our transformer simulates a time-, space- Turing machine using a context window of and generated tokens, reducing both resources by a factor of relative to the corresponding unlooped queue-machine simulation. The construction also holds with softmax attention at constant bit-size, under the finite-precision kernel specified in this paper.