betaFeedback

WorksComputer Science

Energy-Based Reasoners: Your Recurrent Reasoner Should Be an Energy-Based Model

Abstract

Looping is a powerful inductive bias for reasoning. By repeatedly iterating a recurrent block, looped models can scale their effective depth and enable multi-step reasoning in latent space. However, existing looped models are not inherently guided by an explicit measure of progress, making it difficult to determine when to stop looping and how to improve upon the current state. To enable this, we introduce Energy-Based Reasoners (EBRs), which learn an energy function over latent reasoning states and formulate looped updates as gradient descent on this energy. The energy scores latent states, while its gradient guides refinement and provides a signal for stopping. To train EBRs effectively, we further introduce exploration-based training to discover additional modes and to better shape the energy landscape over a broader set of states. Across four algorithmic reasoning benchmarks, EBRs match or outperform prior recurrent reasoning models, achieving 99.9% accuracy on Sudoku-Extreme while using 345×\times less inference compute than the next-best model and maintaining comparable training compute. On all multi-solution benchmarks, EBRs achieve state-of-the-art coverage of distinct valid solutions while maintaining near-100% single-sample accuracy. These results suggest that energy-based models are a promising approach for strong reasoning performance.

energy-based-reasoners.github.io github.com/wwillchen/ebr

Introduction

Generalization requires the ability to compute answers that cannot be retrieved. For a model, this means using what it has learned to construct solutions to unfamiliar problems [1]. For difficult reasoning tasks, this computation may involve a sequence of dependent steps, each building on the results of the last [2]. Over the past few years, chain-of-thought has been widely used to support such multi-step reasoning in large language models. The model uses additional tokens to express intermediate reasoning steps before producing its final answer [3]. In this setting, test-time scaling means generating more tokens to achieve better solutions, for example, generating longer reasoning traces, revising predictions, and searching over candidate solutions [4, 5, 6].

A complementary approach carries out these reasoning steps through iterative updates to an internal latent state [7, 8]. Looped architectures organize these updates through recurrence [9, 10]. They repeatedly apply the same layers to a state that carries information from one reasoning step to the next. Reusing these layers allows the model to increase its computational depth without increasing its number of parameters [2, 10]. Recent works on looped reasoning models, including HRM and TRM, demonstrate the effectiveness of this approach on challenging algorithmic reasoning tasks, achieving strong performance with small models and sometimes outperforming much larger language models [11, 12, 13].

However, moving reasoning into latent space raises the question of how we can actually guide updates to facilitate better search. Chain-of-thought solves this problem by exposing intermediate steps as token sequences that can be evaluated to decide which reasoning paths to pursue, how much computation to allocate to them, and which solution to select [4, 6, 14, 15]. However, latent states do not directly expose a comparable explicit trace, and existing looped reasoners typically rely on separate mechanisms to evaluate predictions and determine stopping. For example, HRM and TRM use learned heads to discern between correct and incorrect states and decide when to halt [11, 12, 16], while FPRM uses convergence of the recurrent state as a stopping criterion [17]. Other approaches introduce stochasticity to explore alternative latent trajectories and use additional criteria to select among the resulting predictions [16, 18, 19]. In each case, the evaluation is used only to stop or select, while the update itself is not guided by any explicit assessment of the current state. This motivates a broader question: can we learn a unifying objective so that evaluation guides the reasoning process, decides when to stop, and verifies which candidates are best?

Figure 1. Reasoning as energy minimization. Energy-Based Reasoners (EBRs) solve a puzzle xx by gradient descent on a learned energy Eθ(z;x)E_\theta(z; x) over latent reasoning states zz. The surface shows this energy for one Sudoku puzzle on the plane through three states of a descent trajectory.

We argue that energy-based models offer a natural way to connect these capabilities. Prior work has demonstrated the potential of energy-based models for assessing uncertainty through learned energy scores [20], generating diverse samples via Langevin dynamics [21], and enabling iterative refinement through energy minimization [22, 23, 24]. Building on these ideas, we introduce Energy-Based Reasoners (EBRs), which formulate reasoning as optimization over a learned energy landscape. Given a problem xx, an EBR learns an energy function Eθ(z;x)E_\theta(z; x) over latent reasoning states zz, assigning lower energy to states that produce better predictions. At inference time, reasoning proceeds by iteratively refining the latent state through gradient descent on this energy function. Importantly, the same learned energy also determines which predictions to select and when to stop refining them. Among multiple candidate solutions, an EBR selects the lowest-energy prediction without requiring a separate verifier. Meanwhile, the norm of the energy gradient provides a convergence signal, allowing each trajectory to stop once further refinement is no longer needed rather than relying on a fixed iteration budget. In this way, EBRs unify iterative refinement, candidate evaluation, and adaptive stopping within a single learned objective.

In order to generate diverse solutions, EBR must learn an energy landscape that captures multiple solution modes. Recent generative reasoning approaches, including Flow Reasoning Models and Looped Flows [25, 26], demonstrate the potential of generative modeling for recurrent reasoning. However, conventional generative frameworks, such as diffusion models, can suffer from exposure bias due to mismatches between training and inference. Explorative Modeling (XM) [27] offers an alternative by exploring model-generated candidates and matching them to suitable targets. This is particularly natural for energy-based reasoning, where supervising different trajectories provides learning signals across the latent states visited during descent, progressively shaping the energy landscape so that descent can better isolate modes. We therefore introduce state-based Explorative Modeling (SXM), extending XM to persistent energy-descent trajectories. By matching trajectories to solutions and supervising their intermediate states, SXM encourages the learned energy to guide refinement across a broader range of states and better realize the generative capacity of EBRs.

Across four challenging algorithmic reasoning benchmarks, Sudoku-Extreme, Maze-Hard, N-Queens, and Graph Coloring, EBRs match or outperform prior looped reasoning models. On Sudoku-Extreme, EBRs achieve 99.9% accuracy while using 345× less inference compute than the next-best model, without requiring an external verifier. On multi-solution benchmarks, EBRs achieve high coverage of distinct valid solutions while maintaining near-perfect solution accuracy, demonstrating the ability of the learned energy landscape to support diverse reasoning outcomes. Together, these results establish energy minimization as an effective framework for accurate, efficient, and flexible iterative reasoning.

Our contributions are threefold,

  • We introduce Energy-Based Reasoners (EBRs), which unify iterative refinement, candidate evaluation, and adaptive stopping through a single learned energy function.

  • We extend Explorative Modeling (XM) to latent reasoning trajectories through state-based Explorative Modeling (SXM), enabling the discovery of diverse solutions.

  • We demonstrate improvements in reasoning accuracy, inference efficiency, and solution diversity across four algorithmic reasoning benchmarks.

Related Work

Recurrent and recursive reasoners. Looped architectures perform step-by-step computation by repeatedly applying the same block of layers to a latent state that holds the model’s current guess. Each pass refines this guess, adding effective depth without additional parameters [8, 9, 10]. A recent line of work, HRM and TRM, has shown that small looped models can solve hard algorithmic reasoning problems much better than most large language models. They train the model to repeatedly refine a hierarchy of two latents. Intuitively, the low-level latent acts as a scratchpad that is updated several times in quick succession, and the high-level latent, which holds the model’s solution, is updated from the scratchpad [11, 12]. However, both models are deterministic and can only generate one reasoning trajectory in inference, limiting their ability to explore alternative solutions.

Stochastic recursive reasoning. Several approaches introduce stochasticity into recursive reasoning to explore different trajectories and candidate solutions. PTRM injects noise into a TRM-style recurrence and selects among the resulting candidates using the model’s existing halting head [16]. GRAM instead approaches diversity from a generative modeling perspective, treating the reasoning trajectory as a latent variable trained with amortized variational inference so that independent samples recover different solutions [19]. Flow Reasoning Models and Looped Flows formulate recurrent refinement through learned denoising dynamics [25, 26]. While these approaches demonstrate the potential of stochastic reasoning, stepwise generative training can introduce exposure bias. This motivates exploration-based supervision, where model-generated trajectories are matched to suitable targets. Our state-based Explorative Modeling (SXM) extends this idea to energy descent, providing supervision across different trajectories and intermediate states to better shape the learned energy landscape.

Attractors and fixed-point reasoning. Recent studies have observed that recursive reasoning models can converge to fixed points. Specifically, previous work has shown that HRM can converge to both correct and incorrect fixed points, sometimes getting stuck at incorrect solutions [28]. EqR builds on this observation by learning task-conditioned attractors and using damping and stochasticity to improve convergence toward valid solutions [18]. FPRM shows that fixed-point convergence can also be used to adapt computation prior to problem difficulty. It introduces architectural modifications and damping to stabilize long recurrences, allowing a single latent state to be refined until convergence [17]. Like FPRM, EBRs use a single latent state without hierarchy and rely on convergence to determine when to stop. However, rather than learning an unconstrained recurrent update, EBR defines its dynamics through gradient descent on a learned energy function. This imposes structure on the update vector field and stationary points correspond to vanishing gradients. EBR further introduces convergence loss to encourage correct predictions to approach those stationary points.

Algorithm 1 EBR training

Input: instance $x$; solution $y$
Parameters: segments $S$, noise scale $\sigma$, threshold $\delta$
1: $z \sim \mathcal{N}(0,I)$
2: for $s = 1,\ldots,S$ do
3:   sample steps $n$ and step size $\alpha$
4:   for $t = 1,\ldots,n$ do
5:     $\xi \sim \mathcal{N}(0,I)$
6:     $\tilde{z} \leftarrow \operatorname{RMS}(z) + \sigma \xi$
7:     $z \leftarrow \Pi_{\mathcal{M}}(\tilde{z} - \alpha \nabla_{\tilde{z}} E_\theta(\tilde{z};x))$
8:   end for
9:   update $\theta$ with $\nabla_\theta \mathcal{L}$ ▷ {xref}`eq. (3)`
10:  if $r_n < \delta$ then break ▷ {xref}`eq. (4)`
11:  $z \leftarrow \operatorname{sg}(z)$
12: end for
Algorithm 2 EBR inference

Input: instance $x$
Parameters: segments $S$, steps $n$, step size $\alpha$, noise scale $\sigma$, threshold $\tau$
1: $z \sim \mathcal{N}(0,I)$
2: for $s = 1,\ldots,S$ do
3:   for $t = 1,\ldots,n$ do
4:     $\xi \sim \mathcal{N}(0,I)$
5:     $\tilde{z} \leftarrow \operatorname{RMS}(z) + \sigma \xi$
6:     $z \leftarrow \Pi_{\mathcal{M}}(\tilde{z} - \alpha \nabla_{\tilde{z}} E_\theta(\tilde{z};x))$
7:   end for
8:   if $\lVert\nabla_{\mathcal{M}}E_\theta(z;x)\rVert < \tau$ then break
9: end for
10: return $\hat{y}_\theta(z)$

Energy-based reasoning. Energy-Based Models (EBMs) provide a natural formulation of fixed-point reasoning that also supports candidate verification. EBMs learn a scalar function that assigns an energy to a candidate prediction given an input. The function is trained so that better predictions receive lower energy [29]. Gradient descent on this energy gives an iterative update whose fixed points are exactly the stationary points of the energy, and the energy value itself acts as a learned score for comparing candidate solutions. Prior approaches differ in what the energy scores and how it is parameterized. IREM and IRED apply this idea to algorithmic reasoning, with IREM directly optimizing a learned energy over candidate solutions [22] and IRED incorporating diffusion-based training and a sequence of annealed energy landscapes [23]. Energy-Based Transformers similarly refine predictions by minimizing a learned energy [24, 30], while Energy Transformers instead adopt an associative-memory formulation, defining an energy over internal token representations and iteratively refining these representations through energy minimization [31]. Unlike IREM, IRED, and Energy-Based Transformers, EBRs use energy minimization to update a latent reasoning state rather than directly refining candidate predictions. And unlike Energy Transformers, EBRs do not use an associative-memory formulation; instead, they directly parameterize their scalar energy with a small transformer trained end-to-end. Closer to recursive reasoning, ERM uses energies derived from known task constraints to select among solutions generated by existing reasoners [32]. Although ERM achieves strong results, its energy functions are designed around each task’s rules rather than learned end-to-end, limiting the generality and scalability of the approach. In contrast, EBRs achieve stronger performance without relying on hard-coded task features.

Explorative modeling. Learning an energy landscape that supports multiple valid solutions requires exploring and training on different reasoning trajectories. Previous work has explored training with multiple predictions: Multiple Choice Learning (MCL) and Best of Many (BoM) train using the prediction or sample that best matches a target [33, 34], while Implicit Maximum Likelihood Estimation (IMLE) matches each training example to its closest generated sample [35, 36, 37]. Explorative Modeling (XM) generalizes this matching principle to both forward and reverse directions, applying it to end-to-end generation as well as the training of existing generative models [27]. Forward XM selects the prediction that best matches a target, while reverse XM selects the closest target to a generated prediction. EBRs extend this bidirectional selection to recurrent reasoning through state-based XM (SXM), selecting which latent trajectory to supervise and which valid solution best matches its prediction. By training on the states visited along different trajectories, SXM expands coverage and shapes an energy landscape that better guides reasoning toward valid and diverse solutions.

Energy-Based Reasoning Model

Background: Looped reasoning models. We consider algorithmic reasoning problems (x,y)(x,y), where xx is a problem instance and yy is a solution. A recurrent reasoner predicts yy by iteratively refining a latent state ztz_t through a learned transition fθf_\theta,

zt+1=fθ(zt,x),t=0,…,T−1,(1)z_{t+1}=f_\theta(z_t,x), \qquad t=0,\ldots,T-1, \tag*{(1)}

and decoding the final state zTz_T. To train long recurrences, HRM [11] and TRM [12] use deep supervision, which divides the recurrence into segments, supervises their predictions, and detaches the state between segments. EBRs adopt this segment-wise training procedure, illustrated in Figure 2.

Figure 2. Segment-wise training. Each segment consists of nn descent steps. The first n−Hn-H steps are computed without retaining a backpropagation graph (gray), while gradients propagate through the final HH steps (blue), whose states are supervised by L\mathcal{L} (eq. (3)). After each segment, the loss is propagated and converged trajectories stop; otherwise, the detached endpoint initializes the next segment.

Energy-based reasoners. Rather than learning the transition fθf_\theta directly, an EBR learns a scalar energy Eθ(z;x)E_\theta(z; x) and updates its latent state through gradient descent,

zt+1=zt−α∇zEθ(zt;x),(2)z_{t+1} = z_t - \alpha\nabla_z E_\theta(z_t; x), \tag*{(2)}

where α\alpha is the step size. The prediction y^θ(zt)\hat{y}_\theta(z_t) is decoded from the current state ztz_t. In practice, we use a variant of projected Langevin dynamics which injects noise to encourage exploration and projects latent states onto a sphere to stabilize descent and keep trajectories bounded. See algorithm 2 for the exact update. We denote this projection by ΠM\Pi_{\mathcal{M}} and the gradient tangent to the resulting manifold M\mathcal{M} by ∇MEθ\nabla_{\mathcal{M}} E_\theta.

Training Through Energy Descent

Segment-wise supervision. An EBR is trained on trajectories generated by its own energy using deep supervision (Figure 2 and algorithm 1). Each segment Z=(z0,…,zn)Z = (z_0, \ldots, z_n) contains nn descent steps. The initial state z0z_0 is sampled from noise or obtained by detaching the endpoint of the previous segment. To bound memory usage, we use truncated backpropagation through time. This retains the gradients only through the final HH steps of each segment. The preceding n−Hn-H steps are computed without gradient tracking. Since each update depends on the energy gradient, differentiating through the dynamics requires second-order derivatives of EθE_\theta.

Let ℓθ(zt,y)\ell_\theta(z_t, y) denote the loss at state ztz_t, where ℓθ(zt,y)\ell_\theta(z_t, y) defines the stablemax cross-entropy loss following prior works [12, 18, 38]. The final HH states are supervised with,

L(Z,y)=∑t=n−H+1nwtℓθ(zt,y)+λLconv(Z,y),(3)\mathcal{L}(Z, y) = \sum_{t=n-H+1}^{n} w_t \ell_\theta(z_t, y) + \lambda\mathcal{L}_{\mathrm{conv}}(Z, y), \tag*{(3)}

where the weights wtw_t increase toward the endpoint and λ\lambda controls the convergence loss.

Convergence supervision. Convergence alone does not imply correctness, as recurrent dynamics can settle at fixed points that produce incorrect predictions [18, 28]. We therefore introduce a convergence loss that encourages stationarity to coincide with prediction correctness. We measure convergence using the average relative change in latent states over the final HH steps of each segment,

rn=1H∑t=n−H+1n∥zt−zt−1∥∥zt−1∥.(4)r_n = \frac{1}{H} \sum_{t=n-H+1}^{n} \frac{\lVert z_t - z_{t-1} \rVert}{\lVert z_{t-1} \rVert}. \tag*{(4)}

We then convert this residual into a convergence score pn=σ((δ−rn)/κ)p_n = \sigma((\delta-r_n)/\kappa), where δ\delta is the convergence threshold and κ>0\kappa> 0 controls the sharpness of the score. We supervise this score with the correctness of the final prediction,

Lconv(Z,y)=BCE⁡(pn,1[y^θ(zn)=y]).(5)\mathcal{L}_{\mathrm{conv}}(Z,y) = \operatorname{BCE}(p_n, \mathbb{1}[\hat{y}_{\theta}(z_n)=y]). \tag*{(5)}

This encourages trajectories producing correct predictions to converge, while discouraging incorrect predictions from becoming stationary. During training, trajectories halt when rn<δr_n < \delta or when the maximum number of segments is reached. Unlike a learned halt head, our convergence criterion is directly derived from the dynamics and requires no additional trainable parameters.

Randomized descent. To keep the learned energy from relying on a single descent schedule, each training segment samples its number of descent steps nn and its step size α\alpha, which stays fixed within the segment (algorithm 1). Evaluation uses a fixed number of steps and step size. The energy must therefore support refinement across depths and step lengths rather than along one particular trajectory. We include the training details in Section A.3.

State-Based Explorative Modeling

Training on a single trajectory only supervises the energy along a limited region of the latent space, leaving other regions of the energy landscape unexplored. Moreover, when a problem admits multiple valid solutions, supervising every trajectory against the same target can discourage alternative correct predictions. To address these limitations, we extend Explorative Modeling (XM) [27] from independent predictions to reasoning trajectories based on energy minimization. Our state-based variant (SXM) maintains KK parallel descent trajectories per input across training segments. At each segment, all KK trajectories are advanced, but only the trajectory best matched to a valid solution is supervised. Importantly, unselected trajectories continue evolving and remain eligible for selection in later segments. Over SS segments, SXM therefore explores up to K×SK \times S states while supervising only one trajectory per segment. This allows training to progressively shape the energy landscape along different descent paths, while accommodating distinct modes that correspond to different solutions.1

More concretely, let zn(k)z_n^{(k)} denote the endpoint of the kk-th trajectory at a given training segment. We consider two directions for matching trajectories to targets. Forward SXM (F-SXM) selects the lowest-loss trajectory for a given target yy, whereas reverse SXM (R-SXM) selects the best-matching target from the available valid solutions Y(x)\mathcal{Y}(x) for a given trajectory,

kF∗(y)=arg⁡min⁡1≤k≤K ℓθ(zn(k),y),yR∗(k)=arg⁡min⁡y′∈Y(x) ℓθ(zn(k),y′).(6)k_{\mathrm{F}}^{*}(y) = \underset{1 \leq k \leq K}{\arg\min}\ \ell_{\theta}(z_n^{(k)},y), \qquad y_{\mathrm{R}}^{*}(k) = \underset{y' \in\mathcal{Y}(x)}{\arg\min}\ \ell_{\theta}(z_n^{(k)},y'). \tag*{(6)}

When both the trajectory and target are free to vary, we select the pair with the lowest prediction loss. In practice, we apply the two directions sequentially, i.e., forward SXM selects the trajectory, and reverse SXM then selects its target (section A.4). The set Y(x)\mathcal{Y}(x) also does not need to include all valid solutions. We study the effect of training on incomplete solution sets in section E. The matching procedure is illustrated in Figure 3.

Figure 3. State-based XM. KK candidate trajectories are rolled out without gradients, and the trajectory–target pair with the lowest prediction loss is selected. The selected trajectory is replayed, with gradients propagated through its final HH steps (Figure 2).

Inference and Prediction Selection

At inference time, an EBR initializes its latent state from Gaussian noise and refines it through energy descent. We introduce an early stopping criterion that terminates when the tangent gradient norm ∥∇MEθ(z;x)∥\lVert\nabla_M E_\theta(z;x)\rVert falls below a threshold τ\tau, or after SS segments of nn descent steps. In practice, both the gradient norm and the residual work as early stopping criteria, as shown in Figure 6. The inference procedure is summarized in algorithm 2.

Figure 6. Correct solutions converge to low-energy stationary states. Energy (a), tangent gradient norm (b), and relative stepwise residual (c) throughout descent on Sudoku-Extreme, grouped by whether the final prediction is correct. Solved trajectories reach lower energies with small gradients and residuals, while incorrect trajectories continue to exhibit larger gradients and residuals.

The energy function also provides a criterion for selecting among candidates generated by independent trajectories. As shown in Figure 6, the learned energy distinguishes solved from unsolved boards. An EBR therefore selects the lowest-energy endpoint and returns its decoded prediction.

Experiments

We evaluate EBRs on four algorithmic reasoning benchmarks to investigate three questions: (1) Can energy-based recurrence achieve strong reasoning performance and generalization from limited training data? (2) Can the learned energy landscape provide reliable signals for adaptive stopping and selecting among candidate solutions? (3) Can stochastic energy descent recover diverse valid solutions, and how does state-based explorative modeling (SXM) improve this capability?

To answer these questions, we benchmark Sudoku-Extreme and Maze-Hard [11, 12], evaluating reasoning accuracy and generalization capabilities of EBRs. We then investigate whether convergence of energy descent can guide adaptive stopping and whether learned energy values can reliably select among independently generated candidate solutions. Finally, we benchmark N-Queens and Graph Coloring [19], where multiple valid solutions exist, to examine the diversity of solutions recovered through stochastic descent and how SXM helps the learned energy landscape better capture these solutions. Our results are presented using mean ± standard deviation over three training seeds.

Reasoning Performance and Test-Time Scaling

Setup. We first evaluate whether energy minimization supports accurate reasoning and generalization, and whether continued iteration improves the model’s predictions. We consider Sudoku-Extreme and Maze-Hard [11]. Sudoku-Extreme consists of 9×99 \times9 puzzles with unique solutions, with 1,000 training puzzles augmented through validity-preserving transformations and 422,786 test puzzles. We report exact accuracy which requires all 81 cells to be correct. Maze-Hard consists of 30×3030 \times30 mazes, with 1,000 training and 1,000 test instances, where the task is to find a specific shortest path between a designated start and goal.

Test-time scaling with more iterations and one sample

(a) More iterations, one sample

Test-time scaling with more samples

(b) More samples

Figure 4. Test-time scaling on Sudoku-Extreme. Exact accuracy as we increase (a) the number of iterations with one sample and (b) the number of samples. EBRs select the lowest-energy sample. GRAM and TRM are from Baek et al. [19]. EqR is evaluated from its released checkpoint.

Training compute

(a) Training compute

Inference compute

(b) Inference compute

Figure 5. Training and inference efficiency on Sudoku-Extreme. (a) Single-sample exact accuracy at 16 iterations versus training compute. (b) Exact accuracy versus inference FLOPs per puzzle. We compare EBRs with and without convergence-based stopping against GRAM, EqR and TRM, using N=8N=8 samples for EBRs, N=20N=20 and 50 for GRAM, N=128N=128 for EqR, and N=1N=1 for TRM. GRAM and TRM are from Baek et al. [19], and EqR is its reported accuracy [18]. Convergence-based stopping allows EBRs to maintain high accuracy with 10.8× less inference computation compared to EBRs with fixed iterations.

Benchmark performance. Table 1 compares EBRs against existing recursive reasoning models on both benchmarks. All baseline results in Table 1 are taken from the original papers. With a single sample, EBRs achieve 99.4% on Sudoku-Extreme and 87.4% on Maze-Hard, outperforming every compared method with only 1.8M parameters. EBRs also show stronger multi-sample accuracy, achieving 99.9% and 88.3% with significantly fewer samples than previous state-of-the-art methods. Figure 4b highlights this sample efficiency: EBRs approach peak accuracy with only a few samples, whereas competing methods improve more gradually. On Sudoku-Extreme, EBRs surpass EqR with just three samples, achieving 99.85% accuracy compared with EqR’s 99.8% using 128 samples (Figure 4b). This sample efficiency translates into substantially lower inference FLOPs despite the additional cost of computing energy gradients at each recurrent step (Figure 5b). EBRs also achieve higher accuracy at every evaluated training-compute budget (Figure 5a).

Method# ParamsSudoku-Extreme (↑)Maze-Hard (↑)
Single sample (%)
TRM [12]5–7M87.485.3
EqR [18]5–7M93.0–
FPRM [17]7M94.287.0
GRAM [19]10M86.6±2.2–
Looped Flows [26]5–7M97.9±0.486.7±1.1
EBR (Ours)1.8M99.4±0.187.4±1.1
Multiple samples (%)
PTRM [16]5–7M98.8 (100)86.7 (100)
EqR [18]5–7M99.8 (128)–
GRAM [19]10M96.1±0.9 (20)–
Looped Flows [26]5–7M99.3±0.2 (5)86.9±1.5 (5)
EBR (Ours)1.8M99.9±0.1 (5)88.3±0.3 (5)

Table 1. Accuracy on Sudoku-Extreme and Maze-Hard. EBRs achieve strong single-sample accuracy on both benchmarks while using the fewest parameters. For multiple samples, the number of samples, NN, is shown in parentheses. EBRs select the lowest-energy solution, while baseline results are taken from their papers and follow their respective evaluation protocols.

Effect of continued refinement. EBRs also scale well with additional descent steps. Figure 4a shows that accuracy on Sudoku-Extreme steadily improves with increasing refinement depth, consistently outperforming the baselines at matched iteration budgets. Despite being trained with only four iterations, EBRs continue to improve their predictions through further descent on the same learned energy landscape, without requiring additional parameters, although the gains diminish with more iterations.

Adaptive Computation and Solution Selection

Adaptive stopping. We next investigate whether convergence provides a reliable signal for stopping early. The descent curves in Figure 6 show that correct predictions approach stationary points, while incorrect predictions tend to maintain larger gradients (Figure 6b) and continue changing (Figure 6c). This suggests that convergence can provide a useful signal for determining when further iterations are needed. Based on this observation, EBRs stop each trajectory when the projected energy-gradient norm falls below a threshold τ\tau. With τ=0.2\tau= 0.2, EBRs maintain 99.9% exact accuracy on Sudoku-Extreme while using 10.8×10.8\times fewer inference FLOPs than a fixed-iteration budget (Figure 5b and table C.2). At comparable accuracy, EBRs require up to 345×345\times fewer FLOPs than EqR.

Energy-based solution selection. While small gradient norms or residuals can indicate convergence, they do not guarantee correctness, as trajectories may also settle at spurious minima. The learned energy provides an additional signal, since solved trajectories end at lower energy than incorrect ones (Figure 6a). Selecting the lowest-energy endpoint among independent candidates therefore picks the correct answer more often than selecting the most converged one, and comes close to oracle selection (Figure 7).

Figure 7. Energy-based selection closely approaches oracle accuracy. Sudoku-Extreme exact accuracy when selecting one of NN independently generated samples using the lowest energy, lowest gradient norm, or convergence criterion. Oracle uses test label.

Multi-Solution Reasoning

Setup. We evaluate the ability of EBRs to generate multiple valid solutions on the N-Queens and Graph Coloring benchmarks of GRAM [19]. N-Queens provides partial boards with 5–7 queens removed in the 8×88 \times8 setting and 7–9 queens removed in the 10×1010 \times10 setting. Graph Coloring uses three-colorable Erdős–Rényi graphs with eight or ten vertices. For each input, we draw 20 independent samples and measure their validity and their coverage of distinct valid solutions (Table 2).

Method# ParamsNNN-Queens 8 × 8 Acc.↑N-Queens 8 × 8 Cov@20↑N-Queens 10 × 10 Acc.↑N-Queens 10 × 10 Cov@20↑Graph Coloring 8-vertex Conflict↓Graph Coloring 8-vertex Cov@20↑Graph Coloring 10-vertex Conflict↓Graph Coloring 10-vertex Cov@20↑
Looped TF7M–68.1±7.423.6±1.950.0±7.66.2±3.2136.0±16.120.5±1.5157.3±9.07.2±0.7
HRM27M–78.7±2.926.7±1.337.8±0.74.7±0.1109.7±1.521.8±0.3164.3±21.68.9±1.7
TRM7M–66.8±5.622.6±0.817.5±11.22.0±1.3109.3±3.122.3±0.6170.7±17.96.8±0.3
GRAM10M199.5±0.3–89.7±2.7–8.3±2.5–31.3±15.0–
GRAM10M20100.0±0.089.7±1.7100.0±0.057.5±3.42.7±2.185.8±0.53.3±1.551.3±2.8
EBR (Ours)7M1100±0–99.51±0.5–0±0–0±0–
EBR (Ours)7M20100±098.1±0.299.99±0.0192.0±0.90±088.6±0.10±054.5±0.5

Table 2. Evaluation on N-Queens and Graph Coloring. N=1N=1 is a single sample. For N=20N=20, EBRs keep the lowest-energy of 20 samples, and the baselines use majority vote. Cov@20 is the fraction of distinct valid solutions found with 20 samples per input. Conflict is the number of violated edges summed over the test graphs. Baseline results are from Baek et al. [19].2

Validity and diversity. Across all four benchmarks, EBRs achieve near-perfect validity while recovering more diverse solutions than prior methods. With a single sample, EBRs reach 100% and 99.51% accuracy on 8×88 \times8 and 10×1010 \times10 N-Queens, respectively, and zero conflicts on both Graph Coloring tasks, outperforming GRAM even with 20 samples on the latter. With 20 samples, EBRs maintain near-perfect validity while achieving higher coverage of distinct valid solutions across all2. four benchmarks. The improvement is particularly significant on 10 × 10 N-Queens, where coverage increases from 57.5% for GRAM to 92.0% for EBRs.

Ablations and Analysis

Energy-based recurrence. To isolate the role of energy descent, we replace EBR’s gradient-based updates with a directly learned recurrent transition (Table 3). This reduces single-sample accuracy on Sudoku-Extreme from 98.96% to 60.05%, demonstrating the importance of energy-based recurrence. We note that removing the energy head does not reproduce TRM’s [12, 16] settings as it uses a smaller model (1.7M versus 5–7M parameters) and retains EBR’s training setup, including a single latent state, self-attention, noise and normalization at each update, and convergence-based halting (section A.1). By changing only the recurrence mechanism, this comparison isolates the contribution of energy descent within EBR’s framework.

VariantN=1↑N=5 ↑
EBR (K=1K=1)98.9699.79
− energy head60.0576.59
− randomized α\alpha97.5999.39
− convergence loss45.3263.12
convergence loss → q-head92.4295.56
+ SXM (K=2K=2)99.3699.90
+ SXM (K=3K=3)99.1799.78
+ SXM (K=4K=4)99.2799.70

Table 3. Sudoku-Extreme training ablations. Each row changes one component of the EBR (K=1K=1); the bottom rows add SXM with KK candidates. q-head selects smallest last step.

Randomizing the step size exposes the model to a broader range of descent trajectories and latent states, improving generalization. Fixing α\alpha reduces single-sample accuracy from 98.96% to 97.59%. Convergence supervision is quite important for preventing overfitting in the absence of another halting mechanism like a learned halting head (q-head). Removing it drops accuracy to 45.32%. Replacing the convergence loss with q-head, as in HRM and TRM [11, 12], partially recovers the accuracy drop.

State-based explorative modeling. We also include other training ablations and show that training with convergence loss performs better than training with a q-head loss. Training with SXM (K=2K=2) improves single-sample accuracy, and larger KK brings no further gain. However, accuracy with five samples remains comparable, as energy-based selection already recovers correct solutions for nearly all instances. In addition to Sudoku, we also examine the contributions of forward and reverse SXM on N-Queens and Graph Coloring (Table 4). Each improves both validity and coverage, and we see that combining both improves accuracy and coverage on both benchmarks.

VariantNQ 8×8 Acc.↑NQ 8×8 Cov.↑GC 8-vertex Conflict↓GC 8-vertex Cov.↑
EBR (no XM)65.4075.91883.4
+ F-SXM (K=2K=2)72.1887.5386.1
+ R-SXM10097.9088.5

Table 4. SXM ablations on 8×8 N-Queens and 8-vertex Graph Coloring (N=1N=1). Each row adds to the one above, and the last row is the model of Table 2.

Limitations and Future Work

While EBRs achieve strong performance, energy-based recurrence introduces additional computational overhead. Training through energy descent requires second-order derivatives, making each update more expensive than a directly learned recurrent transition. SXM further increases training cost by rolling out KK candidate trajectories per training segment. We measured this overhead by comparing training FLOPs to previous models in section B, and on Sudoku-Extreme EBRs are more accurate than the baselines at every training budget (Figure 5a). To address these computational requirements, we optimized our implementation for computing second-order gradients, which required additional engineering such as custom GPU kernels (which are provided in our codebase). Nevertheless, we demonstrate that the accuracy gains justify the additional compute and obtain better results using less inference compute (Figure 5b). Hence, developing more compute efficient energy-based models is an important direction for future work.

Training energy-based models also introduces additional hyperparameters, such as the step size, the noise scale and the number of descent steps (section A.3 and table A.2). Unlike prior energy-based models, which are often sensitive to these choices, we found them broadly transferable across benchmarks. Still, these additional hyperparameters add further complexity in configuring and training EBRs.

We also note that SXM, and XM in general, tends to benefit from larger training sets. On Maze-Hard, SXM brought no gain, so we train it with K=1K = 1 (table A.2). Maze-Hard provides only 1,000 training mazes without augmentation, whereas Sudoku-Extreme augments its 1,000 training puzzles, resulting in roughly one million training boards. With little data, the additional trajectories explored by SXM have few distinct examples to cover, and we believe this limits both SXM and the overall gains on Maze-Hard (section D).

Our experiments are also currently limited to structured algorithmic reasoning tasks, leaving open whether energy-based recurrence can scale to larger models and more general reasoning problems. With the recent renewed interest in looped architectures, an important direction is exploring whether learned energies can guide recurrent computation in tasks such as reasoning problems for language modeling or planning in world models.

Conclusion

We introduced Energy-Based Reasoners (EBRs), reasoning models that reason by gradient descent on a learned energy function. By formulating reasoning as energy minimization, EBRs learn a single function to guide refinement, halting, and selection among candidate solutions. We further introduced state-based exploratory modeling (SXM), which trains the energy on diverse reasoning trajectories to support multiple valid solutions. Across four algorithmic reasoning benchmarks, EBRs match or outperform prior looped models. With only 1.8M parameters, EBRs achieve 99.4% single-sample accuracy on Sudoku-Extreme and 87.4% on Maze-Hard. On Sudoku-Extreme, EBRs further achieve 99.9% accuracy while using 345×345\times less inference compute than the next-best model. On N-Queens and Graph Coloring, EBRs maintain near-perfect single-sample accuracy while recovering more distinct valid solutions than prior methods, raising coverage on 10×1010 \times10 N-Queens from 57.5% to 92.0%. These results suggest that learning an energy landscape provides a promising alternative to directly learning recurrent state transitions for algorithmic reasoning.

Acknowledgement

Thanks to Purvish Jajal for ideas regarding XM and reasoning models. A.G. is supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 21-46756. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This research used the Delta and DeltaAI advanced computing and data resources, which are supported by the National Science Foundation (award OAC 2320345 and award OAC 2005572) and the State of Illinois. Delta and DeltaAI are joint efforts of the University of Illinois Urbana-Champaign and its National Center for Supercomputing Applications. We also acknowledge the Duke Compute Cluster (DCC), maintained by Duke Research Computing, for providing computational resources used in this work.

Model and Training Details

Energy Network

The energy EθE_\theta reuses the recurrent Transformer block of TRM and EqR [12, 18] as its backbone and adds a lightweight per-token head that reduces the block’s output to a scalar. Table A.1 summarizes the key differences between the two models.

TRMEBR
Block outputthe next latent statea scalar energy (eq. (7))
State updatereplaced by the block outputa step along −∇zEθ-\nabla_z E_\theta
Latent statehierarchical, zHz_H and zLz_La single zz
Initial stateone fixed vector for every inputfresh draw from a normal
Haltinglearned Q headresidual of the descent (eq. (4))

Table A.1. Key mechanism differences between TRM and EBRs.

Backbone. The backbone consists of two Transformer blocks, each with four-head self-attention and a SwiGLU MLP [40], followed by residual connections and RMSNorm [41]. We use self-attention across all tasks, whereas TRM replaces attention with a token-axis MLP on Sudoku-Extreme. The hidden dimension dd varies by task (table A.2).

As in TRM and HRM [11], problem tokens are embedded, scaled by d\sqrt{d}, and added to the latent state, so the backbone computes h=fθ(z+emb⁡(x))h = f_\theta(z + \operatorname{emb}(x)). We use two-dimensional rotary positional embeddings (RoPE) [42], assigning half of each attention head’s channels to row positions and half to column positions. Sudoku, Maze, and N-Queens use their board coordinates, while Graph Coloring is treated as a one-dimensional sequence.

The key difference between EBRs and prior works lies in how the backbone is used. TRM directly predicts the next latent state, whereas an EBR uses the backbone to define a scalar energy and updates the state along −∇zEθ-\nabla_z E_\theta. Each descent step therefore requires a forward and backward pass through the backbone, and training through these steps involves second-order derivatives.

Following TRM, linear layers are initialized from a truncated normal distribution and use no bias, except the hidden layer of the energy head (eq. (7)). We additionally learn a separate attention temperature for each head, parameterized in log space.

Energy head. The energy has the same form on every task,

Eθ(z;x)=∑i=1Nϕ(ei),ei=w⊤SiLU⁡(Whi+b),h=fθ(z+emb⁡(x)),(7)E_\theta(z; x) = \sum_{i=1}^{N} \phi(e_i), \qquad e_i = w^\top\operatorname{SiLU}(W h_i + b), \qquad h = f_\theta(z + \operatorname{emb}(x)), \tag*{(7)}

where z∈RN×dz \in\mathbb{R}^{N \times d} holds one latent vector per input token, the head (W,b,w)(W, b, w) is shared by all tokens and adds d2+2dd^2 + 2d parameters, and ϕ\phi is the identity except on Maze-Hard, where it is a softplus. For example, the head adds 66,048 parameters at d=256d = 256 (Sudoku-Extreme and Maze-Hard) and 263,168 at d=512d = 512 (N-Queens and Graph Coloring), about 3.7% of the model in both cases. Although the energy is a sum of per-token terms, each hih_i incorporates information from other tokens through self-attention. Thus, interactions and constraints between tokens are captured by the backbone, and the energy gradient couples their updates during descent.

We find that the energy head can be adapted to different tasks through the choice of ϕ\phi. We use the identity function for all tasks except Maze-Hard, where we apply a softplus. With the identity, individual token contributions can be negative, and potentially favors solutions that contain more tokens of a particular type. This is less problematic for Sudoku, N-Queens, and Graph Coloring, where valid solutions have a fixed number of assigned positions. In Maze-Hard, however, valid paths can differ in length. We observe that the identity head learns negative contributions for path cells, allowing longer detours to receive lower energy than shorter paths. Using softplus constrains token energies to be nonnegative, and we observe that the model assigns higher contributions to path cells than to open cells, thereby discouraging unnecessary detours.

In addition to the energy network and token embeddings, an EBR uses a linear readout to decode predictions from each token’s latent state. The full model has 1.8M parameters at d=256d = 256 and 7.1M at d=512d = 512.

Data and Settings

Data. We follow the original token layouts used by HRM for Sudoku-Extreme and Maze-Hard, and by GRAM for N-Queens and Graph Coloring. Sudoku-Extreme contains 1,000 training puzzles, each augmented with 1,000 validity-preserving transformations, while Maze-Hard contains 1,000 training mazes without augmentation.

For N-Queens and Graph Coloring, we use GRAM’s datasets [19], preserving the original training and test splits. N-Queens instances are generated by enumerating all solutions for N=8N = 8 and N=10N = 10, then removing k∈{5,6,7}k \in\{5, 6, 7\} and k∈{7,8,9}k \in\{7, 8, 9\} queens, respectively. Distinct inputs are split 85:15 into training and test sets, yielding 4,376/772 examples for N=8N = 8 and 36,907/6,513 for N=10N = 10. Graph Coloring consists of three-colorable graphs with 8 or 10 vertices, with all valid colorings enumerated up to color permutation. The corresponding test sets contain 255 and 192 graphs, respectively. For reverse SXM, each training input is paired with its valid solutions.

SudokuMazeN-Queens 8×8N-Queens 10×10Graph Coloring 8-vertexGraph Coloring 10-vertex
Model
Hidden size256256512512512512
Parameters1.8M1.8M7.1M7.1M7.1M7.1M
Per-cell energylinearsoftpluslinearlinearlinearlinear
Descent
Noise scale σ\sigma0.4→0.050.40.30.30.30.3
Training segments SS463344
Evaluation segments64163344
Exploration
Forward SXM candidates KK212222
Reverse SXM––✓✓✓✓
Start from a noisy solution––✓✓✓✓
Halting
Loss weight λ\lambda0.750.50.250.50.250.25
Threshold δ\delta0.250.750.250.250.250.25
Optimization
Batch size76864768768768768
Warmup steps1005001,0001,0001,0001,000
Training steps60k50k30k45k70k70k

Table A.2. Per-task settings. Settings shared by all tasks are given in the text.

Optimization. Hidden weight matrices are trained with Muon [43] (five Newton–Schulz iterations, momentum 0.95, Nesterov). The learning rate of each matrix is scaled by 0.2 times the square root of its larger dimension, which matches the update size of AdamW [44]. Embeddings, the readout, the last layer of the energy head and all vector parameters are trained with AdamW [45] (β1=0.9\beta_1 = 0.9, β2=0.95\beta_2 = 0.95). We use a base learning rate of 10−410^{-4} with linear warmup followed by a constant schedule, weight decay of 0.1. All evaluations use an exponential moving average of model weights with decay 0.999.

Compute. All experiments are trained on a single NVIDIA H200 GPU. Table A.3 reports the training time for one seed, including intermediate evaluations.

SudokuMazeN-Queens 8×8N-Queens 10×10Graph Coloring 8-vertexGraph Coloring 10-vertex
Training steps60k50k30k45k70k70k
Wall time (hours)334858

Table A.3. Training time of one seed on a single NVIDIA H200 GPU, including training evaluations.

Training Through Energy Descent

Training follows algorithm 1 (§ 3.1). This section gives the settings of its randomized descent and the remaining choices that stabilize energy descent and encourage the learned energy to support refinement beyond the trajectories encountered during training.

Randomized number of descent steps. During training, we uniformly sample 15–21 descent steps per segment, while evaluation uses a fixed 21 steps. For the first segment after a trajectory reset, we instead sample 7–15 steps, exposing the model to earlier stages of descent.

Randomized step size. We sample the step size independently for each trajectory and segment as α∼U(200,250)\alpha\sim U(200, 250), keeping it fixed within the segment. At evaluation, we use α=250\alpha= 250. Removing this randomization reduces single-sample accuracy on Sudoku-Extreme (Table 3).

Noise and projection. We inject Gaussian noise before each descent step to encourage exploration, with task-specific scales given in table A.2. On Sudoku-Extreme, the noise scale follows a cosine schedule across segments, decaying from 0.4 to 0.05; evaluation uses the same schedule. Other tasks use a constant noise scale. To stabilize the dynamics, we project latent states to unit RMS after each update using ΠM\Pi_M, corresponding to token-wise normalization onto a sphere. Convergence residuals are computed on normalized states.

Truncated backpropagation and second-order gradients. To reduce training cost, we backpropagate only through the final H=7H = 7 descent steps of each segment and compute earlier steps without retaining a training computation graph. Since each descent update depends on ∇zEθ\nabla_z E_\theta, differentiating through these updates requires computing a second-order derivative. The final state of each segment is detached before initializing the next, preventing gradients from propagating across segments.

Training objective. We supervise the final H=7H = 7 states of each segment with weights wtw_t (eq. (3)) that increase linearly toward later steps. The convergence loss (eq. (5)) computes the residual rnr_n of eq. (4) over the last 2 of the HH updates, with κ=0.05\kappa= 0.05 and task-specific δ\delta and λ\lambda given in table A.2. Following HRM, with probability 0.1 we delay halting until a randomly sampled number of segments between 2 and SS, encouraging training on longer trajectories.

Initialization during training. While inference always starts from pure Gaussian noise, training on N-Queens and Graph Coloring also uses noisy solution embeddings, similar to the denoising supervision of IRED [23]. We randomly vary the amount of solution information in the initial state, ranging from pure noise to states that decode exactly to the target solution,

z0=(I−P)ϵ+1−a2 Pϵ+a s(y),ϵ∼N(0,I),a∼U(0,1),(8)z_0 = (I - P)\epsilon+ \sqrt{1-a^2}\,P\epsilon+ a\,s(y), \qquad\epsilon\sim\mathcal{N}(0,I), \qquad a \sim U(0,1), \tag*{(8)}

where PP projects each token’s state onto the directions that the readout decodes, and s(y)s(y) is a state that decodes to the target solution yy. Only the decodable part of the noise is mixed with the solution, so a=0a = 0 recovers the inference initialization and a=1a = 1 decodes exactly to yy. This encourages the learned energy to guide partially correct states toward valid solutions.

Robustness of energy training. Training through energy descent backpropagates through second-order derivatives of the energy, and this can cause gradient instability during training. We found gradient clipping to be helpful in addressing this. Not all benchmarks needed it, but we found gradient clipping particularly helpful on Maze-Hard.

State-Based Explorative Modeling

Forward SXM rolls out KK candidates for each training row (table A.2). The candidates are ranked by the cross-entropy of their endpoints against the stored solution of that row, the best one is replayed with gradients, and all KK endpoints are carried into the next segment. On N-Queens and Graph Coloring, reverse SXM then regrades the replayed trajectory. Each of its last three graded states, which include the endpoint, is compared with the valid solutions of its input that appear in the same batch, and is graded against the one with the lowest cross-entropy. In practice, more of the graded states, up to all HH, can be regraded in the same way, but we only tried the last three. The solution chosen for the endpoint is also the target of Lconv\mathcal{L}_{\mathrm{conv}}.

Because reverse SXM grades each trajectory against its closest valid solution, the trajectories of one input can all be pulled toward the same solution, causing collapse. Following the reverse XM of Gladstone et al. [27], we avoid this by not reusing solutions. Once a solution has been assigned to one trajectory, the other trajectories in the training batch are graded against the closest solution that has not been assigned yet. After 75% of the input’s solutions have been assigned, all of them become available again.

Training Compute Compared to GRAM

Table B.1 compares the training FLOPs and single-sample results of EBRs and GRAM [19]. We measure the forward and backward computation of each training step, including second-order gradients through the descent steps of EBRs and the KK candidate rollouts used by SXM. With PyTorch’s FLOP counter, we count matrix multiplications and attention products at two FLOPs per multiply-add, excluding elementwise operations, normalization, and optimizer updates. For EBRs, we use the configurations and training lengths in table A.2, averaging per-step costs over the randomized descent lengths. For GRAM, we use its training implementation and derive the total training steps from its dataset sizes, batch sizes, and number of epochs. Intermediate evaluations are excluded from both counts.

Training stepsTotal EFLOPsNN=1 result
TaskEBRGRAMEBRGRAMEBRGRAM
Sudoku-Extreme60k65.1k1.922.2699.41±0.1286.6±2.2
Maze-Hard50k–1.16–87.37±1.1–
N-Queens 8×830k47.0k2.451.08100±099.5±0.3
N-Queens 10×1045k140.8k5.824.7499.51±0.4589.7±2.7
Graph Coloring, 8-vertex70k45.5k2.460.670±08.3±2.5
Graph Coloring, 10-vertex70k87.6k3.981.770±031.3±15.0

Table B.1. Training compute and single-sample results of EBRs and GRAM. Total training FLOPs of the full run and the NN=1 result, as in Table 1 and Table 2. GRAM reports no Maze-Hard results.

On Sudoku-Extreme, EBRs use 15% fewer training FLOPs than GRAM. On N-Queens and Graph Coloring, EBRs require 1.2–3.7× more total training FLOPs, primarily due to SXM candidate rollouts. In exchange, EBRs outperform GRAM on every shared task, with zero conflicts on both Graph Coloring tasks, and their coverage improvements are reported in Table 2.

Evaluating EqR. Evaluating EqR at its full inference budget is expensive. Their inference configuration for their main results uses 128 samples and 64 iterations, so each Sudoku-Extreme puzzle costs about 210 TFLOPs, and the full test set of 422,786 puzzles would take 89 EFLOPs, or around 120 GPU-hours on an NVIDIA H200. We therefore compare against EqR’s paper results in Table 1 and Figure 5b: 93.0% with one sample and 99.8% with 128 samples [18]. For the sample-scaling curve in Figure 4b, we run their released code with EqR’s own evaluation on Sudoku-Lite, the first 2,048 test puzzles. EqR’s points in Figure 4b are therefore lower than its full-test results: 99.2% with 128 samples and 87.9% with one sample. However, the sample comparison in the same figure compares EBRs against EqR’s reported accuracy of 99.8%.

Evaluation Details

Protocols. On Sudoku-Extreme, we evaluate all 422,786 test puzzles using eight independent samples, each run for up to 64 segments. On Maze-Hard, we evaluate 1,000 test mazes with five samples of 16 segments each. A maze is considered solved when its predicted path exactly matches the labeled solution; we additionally report any-shortest-path accuracy in section D. Pass@1Pass@1 denotes the exact accuracy of the first sample, while Pass@nPass@n selects the lowest-energy candidate among the first nn samples. Results are averaged over three training seeds. The ablations in Table 3, Table 4 and E.1 use one seed.

N-Queens and Graph Coloring. We use GRAM’s test splits [19]. For each test input, we generate 20 independent samples, stopping each trajectory when its tangent gradient norm ∥∇θEθ∥\lVert\nabla_{\theta}E_{\theta}\rVert falls below τ=0.2\tau= 0.2, or when it reaches the maximum number of segments specified in table A.2. In Table 2, NN=1 evaluates the first sample, while NN=20 selects the EBR’s lowest-energy sample. For N-Queens, a prediction is valid if it places nn non-attacking queens while preserving the given queens. For Graph Coloring, we report the total number of conflicting edges across test graphs. Ground-truth solutions are used only for evaluation and never for prediction selection.

Coverage. We generate 20 samples for each unique test input xx. Let S(x)S(x) denote its solution set and y^(1),…,y^(20)\hat{y}^{(1)},\ldots,\hat{y}^{(20)} its decoded samples, and let F(x)=S(x)∩{y^(1),…,y^(20)}F(x)=S(x)\cap\{\hat{y}^{(1)},\ldots,\hat{y}^{(20)}\} be the distinct solutions among the samples. We define

Cov⁡@20=∑x∣F(x)∣∑xmin⁡(∣S(x)∣,20)(9)\operatorname{Cov}@20 = \frac{\sum_x |F(x)|}{\sum_x \min(|S(x)|,20)} \tag*{(9)}

where the sums run over the unique test inputs. Since 20 samples can find at most 20 solutions, we cap the denominator at 20 so that Cov@20 can reach 100%. On N-Queens, S(x)S(x) contains the completions that keep the given queens. On Graph Coloring, it contains the proper 3-colorings of xx up to color permutation. We use the same definition on both tasks.

Differences from GRAM’s code. GRAM’s code [19] computes coverage differently on the two tasks, and neither matches eq. (9) on inputs with more than 20 solutions. On Graph Coloring, it divides by ∑x∣S(x)∣\sum_x |S(x)| without the cap. A graph with more than 20 colorings can then never be fully covered, so no method can exceed 99.6% coverage on 8-vertex graphs or 63.9% on 10-vertex graphs. On N-Queens, it caps the denominator at 20 but replaces S(x)S(x) by the first 20 solutions, so finding a solution outside these 20 earns no credit. On N-Queens, the two definitions agree on inputs with at most 20 solutions, which covers all 8×88\times8 inputs and 6,494 of the 6,513 10×1010\times10 inputs. On the remaining 19 inputs, GRAM’s definition lowers our 10×1010\times10 coverage from 92.0±0.9%92.0\pm0.9\% to 89.3±1.0%89.3\pm1.0\%. The GRAM results in Table 2 are taken from their paper and use their definitions. To match them, the table reports our Graph Coloring coverage without the cap (88.6% and 54.5%). With the cap, it is 89.0±0.1%89.0\pm0.1\% and 85.3±0.8%85.3\pm0.8\%.

Inference compute. Table C.1 reports the accuracy and per-iteration inference cost underlying Figure 5b following the evaluation format of GRAM’s Table 12 [19]. 3 FLOPs are counted per puzzle with PyTorch’s FLOP counter. One iteration is one supervision step, and for an EBR it is one segment of 21 descent steps, each a forward and a backward pass of the energy. With early stopping at ∥∇MEθ∥<0.2\lVert\nabla_M E_\theta\rVert<0.2, EBRs preserve the accuracy of the 16- and 64-iteration configurations while reducing their inference costs to 0.060±0.0020.060\pm0.002 and 0.076±0.0080.076\pm0.008 TFLOPs per puzzle, respectively. The five-sample cost is 0.380±0.0390.380\pm0.039 TFLOPs per puzzle.

ModelTFLOPs / iter8163264128320
Looped TF0.025650.0±2.550.0\pm2.561.7±0.961.7\pm0.964.5±0.964.5\pm0.9–65.8±1.065.8\pm1.066.1±1.066.1\pm1.0
HRM0.013850.4±1.150.4\pm1.156.8±1.256.8\pm1.260.2±1.360.2\pm1.3–63.8±1.563.8\pm1.565.4±1.665.4\pm1.6
TRM0.025679.3±1.279.3\pm1.283.6±1.383.6\pm1.386.6±1.486.6\pm1.4–90.4±1.590.4\pm1.592.1±1.492.1\pm1.4
GRAM, N=1N=10.026580.2±2.480.2\pm2.486.6±2.286.6\pm2.290.7±1.890.7\pm1.8–––
GRAM, N=5N=50.132589.8±2.089.8\pm2.093.6±1.493.6\pm1.495.3±1.195.3\pm1.1–––
GRAM, N=10N=100.265092.4±1.692.4\pm1.695.1±1.195.1\pm1.196.3±0.896.3\pm0.8–––
GRAM, N=20N=200.530094.1±1.394.1\pm1.396.1±0.996.1\pm0.996.9±0.796.9\pm0.7–––
GRAM, N=50N=501.325095.4±1.095.4\pm1.096.8±0.796.8\pm0.797.4±0.697.4\pm0.6–––
EBR (Ours), N=1N=10.012992.04±0.4392.04\pm0.4396.83±0.3596.83\pm0.3598.74±0.2198.74\pm0.2199.41±0.1299.41\pm0.12––
EBR (Ours), N=3N=30.038798.21±0.3598.21\pm0.3599.37±0.2099.37\pm0.2099.73±0.1299.73\pm0.1299.85±0.0799.85\pm0.07––
EBR (Ours), N=5N=50.064499.15±0.2699.15\pm0.2699.69±0.1499.69\pm0.1499.84±0.0999.84\pm0.0999.90±0.0699.90\pm0.06––
EBR (Ours), N=8N=80.103199.57±0.1899.57\pm0.1899.81±0.1099.81\pm0.1099.89±0.0799.89\pm0.0799.92±0.0599.92\pm0.05––

Table C.1. Accuracy (%) on Sudoku-Extreme as a function of recursion iterations, with the per-iteration inference cost. Baseline and GRAM rows are copied from GRAM’s Table 12 [19], where GRAM selects among its NN samples by majority vote. The EBR evaluates all 422,786 test puzzles, and selects the lowest-energy of its first NN samples. – marks iteration counts that were not reported.

Stopping threshold. Table C.2 examines the trade-off between accuracy and inference computation as the stopping threshold τ\tau varies on Sudoku-Extreme. Increasing τ\tau from 0.1 to 1 reduces the average number of iterations from 7.6 to 3.5 while maintaining 99.4% single-sample accuracy. At τ=0.2\tau= 0.2, EBRs preserve both Pass@1 and Pass@8 accuracy while using 10.8×\times fewer inference FLOPs than the fixed 64-iteration budget. More aggressive stopping reduces accuracy. At τ=2\tau= 2, Pass@1 drops to 98.8% while Pass@8 remains near 99.9%, whereas τ=5\tau= 5 causes a substantial degradation in both.

τ\tauAvg. iterationsPass@1Pass@8TFLOPs / puzzleSaving
No stopping6499.41±0.1299.41\pm0.1299.92±0.0599.92\pm0.056.601.0×\times
0.17.6±1.47.6\pm1.499.41±0.1299.41\pm0.1299.92±0.0599.92\pm0.050.828.0×\times
0.25.6±0.65.6\pm0.699.41±0.1299.41\pm0.1299.92±0.0599.92\pm0.050.6110.8×\times
0.53.6±0.13.6\pm0.199.42±0.1299.42\pm0.1299.86±0.0799.86\pm0.070.3917.1×\times
13.5±0.13.5\pm0.199.41±0.1299.41\pm0.1299.88±0.0799.88\pm0.070.3817.4×\times
23.5±0.13.5\pm0.198.81±0.1798.81\pm0.1799.89±0.0699.89\pm0.060.3817.6×\times
52.1±0.72.1\pm0.774.22±14.4974.22\pm14.4995.66±5.9395.66\pm5.930.2229.7×\times

Table C.2. Effect of the stopping threshold τ\tau on Sudoku-Extreme. Iterations are averaged over samples, with a cap of 64. Pass@8 keeps the lowest-energy of eight samples. TFLOPs are per puzzle for eight samples and include the cost of the gradient check, and the saving is relative to running all 64 iterations.

Maze-Hard: Limitations and Shortest-Path Accuracy

Limited gains. The improvements of EBRs on Maze-Hard are modest compared with Sudoku-Extreme. One possible explanation is the limited training data since the benchmark provides only 1,000 training mazes, which may be insufficient to learn the search behavior required by this task. However, we do not vary the training-set size, so whether additional data would improve performance remains an open question.

Exact match versus shortest paths. The original Maze-Hard evaluation requires predictions to match a single labeled shortest path, even when multiple shortest paths exist. The dataset labels can be reproduced exactly using breadth-first search with the direction priority D<L<R<UD < L < R < U for both training and test annotations. Consequently, an equally short alternative path is counted as incorrect (fig. D.1). To distinguish failures in path finding from differences in path selection, we additionally report any-shortest-path accuracy, which accepts any valid start-to-goal path whose length equals the shortest-path distance.

Using the same checkpoints and trajectories as Table 1, EBRs achieve 94.07% single-sample shortest-path accuracy and 94.77% with lowest-energy selection among five samples (table D.1).

Figure D.1. Different shortest paths for the same Maze-Hard instance. (a) The benchmark’s labeled path and (b) an alternative both reach the goal (G) from the start (S) in 111 steps. (c) Path-specific edges are colored, and shared edges are gray. Exact match accepts only (a), whereas shortest-path accuracy accepts both.

Relation to prior work. EqR [18] introduces the revised Maze-Unique benchmark, which is also used by FRM [25], making their reported maze results not directly comparable to ours. We retain the original benchmark, while reporting any-shortest-path accuracy to better reflect whether the predicted routes solve the underlying maze.

InferenceLabeled-path exact matchAny shortest path
EBR, Pass@187.37±1.1087.37\pm1.1094.07±0.9194.07\pm0.91
EBR, Pass@588.27±0.3288.27\pm0.3294.77±0.5994.77\pm0.59

Table D.1. Exact-match and shortest-path accuracy (%) on Maze-Hard.

Reverse SXM with Partial Solution Sets

Reverse SXM matches each trajectory to the closest valid solution in Y(x)\mathcal{Y}(x) (eq. (6)). Each training batch contains every solution of its inputs, so Y(x)\mathcal{Y}(x) holds all valid solutions of xx in the training set (section A.4). To investigate whether reverse SXM requires access to all valid solutions, we restrict each training example to at most kk targets, retaining its original solution and randomly selecting up to k−1k-1 others. Evaluations are unaffected.

Figure E.1 and table E.1 show that a partial solution set already helps achieve strong accuracy and coverage. On 8×88 \times8 N-Queens, accuracy and coverage both increase with kk: from 72.18% accuracy and 87.5% coverage without reverse SXM (k=1k = 1) to 90.73% and 93.5% at k=2k = 2, 98.55% and 97.7% at k=8k = 8, and 100% and 97.9% with all solutions. On 8-vertex Graph Coloring, coverage stays within about two points across kk (87.7–89.8%), with comparable accuracy (98.0–100%). Reverse SXM therefore does not require access to every valid solution and performs well with partial sets. We did not tune hyperparameters separately for each kk, so it remains unclear whether other values of kk could outperform the full solution set.

Figure E.1. Reverse SXM with partial solution sets. We restrict reverse SXM to at most kk valid targets per training example and measure their accuracy and coverage.

N-Queens 8×8Graph Coloring 8-vertex
kkAcc.↑\uparrowCov@20↑\uparrowAcc.↑\uparrowCov@20↑\uparrow
172.1887.5––
290.7393.598.0488.5
494.3595.299.6189.6
898.5597.799.6189.8
1699.5297.810087.7
All10097.910088.5

Table E.1. Values of fig. E.1 (NN=1, in %).

References

  1. [1]François Chollet. On the measure of intelligence. arXiv preprint arXiv:1911.01547, 2019.arxiv.org/abs/1911.01547
  2. [2]Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J Reddi. Reasoning with latent thoughts: On the power of looped transformers. arXiv preprint arXiv:2502.17416, 2025.arxiv.org/abs/2502.17416
  3. [3]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022.arxiv.org/abs/2201.11903
  4. [4]Charlie Snell, Jaehoon Lee, Kelvin Xu, and Anil Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024.arxiv.org/abs/2408.03314
  5. [5]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=1PL1NIMrW.arxiv.org/abs/2203.11171
  6. [6]Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models, 2023.arxiv.org/abs/2305.10601
  7. [7]Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024.arxiv.org/abs/2412.06769
  8. [8]Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach. arXiv preprint arXiv:2502.05171, 2025.arxiv.org/abs/2502.05171
  9. [9]Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. arXiv preprint arXiv:1807.03819, 2018.arxiv.org/abs/1807.03819
  10. [10]Angeliki Giannou, Shashank Rajput, Jy-yong Sohn, Kangwook Lee, Jason D Lee, and Dimitris Papailiopoulos. Looped transformers as programmable computers. In International Conference on Machine Learning, pages 11398–11442. PMLR, 2023.arxiv.org/abs/2301.13196
  11. [11]Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, and Yasin Abbasi Yadkori. Hierarchical reasoning model. arXiv preprint arXiv:2506.21734, 2025.arxiv.org/abs/2506.21734
  12. [12]Alexia Jolicoeur-Martineau. Less is more: Recursive reasoning with tiny networks. arXiv preprint arXiv:2510.04871, 2025.
  13. [13]Zitian Gao, Lynx Chen, Yihao Xiao, He Xing, Ran Tao, Haoming Luo, Joey Zhou, and Bryan Dai. Universal reasoning model, 2025. URL https://arxiv.org/abs/2512.14693.
  14. [14]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.arxiv.org/abs/2110.14168
  15. [15]Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Ian Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023.arxiv.org/abs/2305.20050
  16. [16]Amin Sghaier, Ali Parviz, and Alexia Jolicoeur-Martineau. Probabilistic tiny recursive model, 2026. URL https://arxiv.org/abs/2605.19943.
  17. [17]Sajad Movahedi, Vera Milovanović, Shlomo Libo Feigin, Alexander Theus, Thomas Hofmann, Valentina Boeva, T. Konstantin Rusch, and Antonio Orvieto. Fixed-point reasoners: Stable and adaptive deep looped transformers. arXiv preprint arXiv:2606.18206, 2026.arxiv.org/abs/2606.18206
  18. [18]Benhao Huang, Zhengyang Geng, and Zico Kolter. Equilibrium reasoners: Learning attractors enables scalable reasoning. arXiv preprint arXiv:2605.21488, 2026.arxiv.org/abs/2605.21488
  19. [19]Junyeob Baek, Mingyu Jo, Minsu Kim, Mengye Ren, Yoshua Bengio, and Sungjin Ahn. Generative recursive reasoning. arXiv preprint arXiv:2605.19376, 2026.arxiv.org/abs/2605.19376
  20. [20]Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribution detection. In Advances in Neural Information Processing Systems, volume 33, pages 21464–21475, 2020.arxiv.org/abs/2010.03759
  21. [21]Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1903.08689
  22. [22]Yilun Du, Shuang Li, Joshua Tenenbaum, and Igor Mordatch. Learning iterative reasoning through energy based minimization. In International Conference on Machine Learning, pages 5570–5582. PMLR, 2022.arxiv.org/abs/2206.15448
  23. [23]Yilun Du, Jiayuan Mao, and Joshua B Tenenbaum. Learning iterative reasoning through energy diffusion. arXiv preprint arXiv:2406.11179, 2024.arxiv.org/abs/2406.11179
  24. [24]Alexi Gladstone, Ganesh Nanduru, Md Mofijul Islam, Peixuan Han, Hyeonjeong Ha, Aman Chadha, Yilun Du, Heng Ji, Jundong Li, and Tariq Iqbal. Energy-based transformers are scalable learners and thinkers. arXiv preprint arXiv:2507.02092, 2025.arxiv.org/abs/2507.02092
  25. [25]Alec Helbling, Andrey Bryutkin, Mauro Martino, Duen Horng Chau, Nima Dehmamy, and Hendrik Strobelt. Flow reasoning models: Turning flows into efficient recurrent reasoners, 2026. URL https://arxiv.org/abs/2606.29150.
  26. [26]Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom, Nicholas M. Boffi, İsmail İlkan Ceylan, and Jinwoo Kim. Thinking with looped flows, 2026. URL https://arxiv.org/abs/2609.11801.
  27. [27]Alexi Gladstone, Heng Ji, and Yilun Du. Explorative modeling: Unlocking a third pretraining axis and end-to-end generation, 2026. URL https://arxiv.org/abs/2607.27372.
  28. [28]Zirui Ren and Ziming Liu. Are your reasoning models reasoning or guessing? a mechanistic analysis of hierarchical reasoning models, 2026. URL https://arxiv.org/abs/2601.10679.
  29. [29]Yann LeCun, Sumit Chopra, Raia Hadsell, M. Ranzato, Fujie Huang, et al. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
  30. [30]Travis Davies, Yiqi Huang, Alexi Gladstone, Yunxin Liu, Xiang Chen, Heng Ji, Huxian Liu, and Luhui Hu. Ebt-policy: Energy unlocks emergent physical reasoning capabilities. arXiv preprint arXiv:2510.27545, 2025.arxiv.org/abs/2510.27545
  31. [31]Benjamin Hoover, Yuchen Liang, Bao Pham, Rameswar Panda, Hendrik Strobel t, Duen Horng Chau, Mohammed Zaki, and Dmitry Krotov. Energy transformer. Advances in Neural Information Processing Systems, 36, 2024.arxiv.org/abs/2302.07253
  32. [32]Yifei Zhao and Ying Tang. Energy-guided recursive model, 2026. URL https://arxiv.org/abs/2607.10128.
  33. [33]Abner Guzman-Rivera, Dhruv Batra, and Pushmeet Kohli. Multiple choice learning: Learning to produce multiple structured outputs. In Advances in Neural Information Processing Systems, volume 25, 2012. URL https://papers.nips.cc/paper_files/paper/2012/hash/cf bce4c1d7c425abf21d6b6f2ab6eb6e-Abstract.html.
  34. [34]Apratim Bhattacharyya, Bernt Schiele, and Mario Fritz. Accurate and diverse sampling of sequences based on a “best of many” sample objective. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 8485–8493, 2018.arxiv.org/abs/1806.07772
  35. [35]Ke Li and Jitendra Malik. Implicit maximum likelihood estimation. arXiv preprint arXiv:1809.09087, 2018. URL https://arxiv.org/abs/1809.09087.
  36. [36]Mehran Aghabozorgi, Shichong Peng, and Ke Li. Adaptive imle for few-shot pretraining-free generative modelling, 2023.
  37. [37]Shichong Peng, Seyed Alireza Moazzenipourasil, and Ke Li. Chimle: Conditional hierarchical imle for multimodal conditional image synthesis. Advances in Neural Information Processing Systems, 35:280–296, 2022.arxiv.org/abs/2211.14286
  38. [38]Lucas Prieto, Melih Barsbey, Pedro A. M. Mediano, and Tolga Birdal. Grokking at the edge of numerical stability, 2025. URL https://arxiv.org/abs/2501.04697.
  39. [39]Lei Yang. Discrete distribution networks. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=xNslIzleIG.arxiv.org/abs/2401.00036
  40. [40]Noam Shazeer. Glu variants improve transformer, 2020.arxiv.org/abs/2002.05202
  41. [41]Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1910.07467
  42. [42]Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024.arxiv.org/abs/2104.09864
  43. [43]Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks. Cited on, page 10, 2024.
  44. [44]Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, et al. Muon is scalable for llm training. arXiv preprint arXiv:2502.16982, 2025.arxiv.org/abs/2502.16982
  45. [45]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.arxiv.org/abs/1711.05101

Paper details

vlnc:W_pwug72s4
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents