betaFeedback

WorksComputer Science

JEPA as a Transferable Optimizer: Learning and Understanding Black-Box Optimization from Reward-Agnostic Pretraining

Abstract

Black-box optimization evaluates candidates through simulations, rollouts, or experiments, often without gradients. Learned optimizers are usually judged by whether they outperform existing methods. We ask a complementary question: what information does a learned optimizer use from its search history? We introduce JEPA-PS (JEPA Policy Search), based on a Joint-Embedding Predictive Architecture. It is pretrained on random rollouts using parameter vectors and scalar returns. Its latent-prediction objective predicts the next point in an optimization trajectory rather than its reward or gradient, allowing us to examine what was tried and how well it performed. At test time, JEPA-PS reads a short evaluation history and proposes a candidate in one forward pass, without task descriptions, reward shaping, or test-time gradients. We evaluate it on 15 control environments against 12 optimizers, including evolutionary methods, Bayesian optimization, CMA-ES, and an LLM optimizer. JEPA-PS is competitive with tuned classical baselines and outperforms the LLM optimizer by more than 2x on Swimmer at 71x lower per-step cost. It also transfers zero-shot to environments excluded from pretraining. Null-model comparisons and decoder-training ablations indicate that its latent representation carries information beyond simple history statistics; every trained decoder variant performs worse than the default frozen representation. Our main finding is that the history JEPA-PS uses depends on the task. On smooth landscapes it uses the full trajectory, whereas on rugged landscapes the best-seen point carries nearly all the signal. This memory requirement has a weak positive correlation with fitness-distance correlation (FDC; r=0.112, n=7), while an independent reward-ordering sensitivity probe does not corroborate the relationship (r=-0.024, n=7). Code: https://github.com/mdebnath22/jepa-props.

Introduction

Optimization is used everywhere, often without a clear view of what drives its success: a robot’s gait is evaluated through a rollout, a protein through an assay, and a hyperparameter configuration through a training run [1, 2]. Every black-box optimizer implicitly answers a question that is rarely asked: how much of its search history is needed to choose the next query? Hand-designed methods answer it by construction. CEM refits a Gaussian to the elite set and discards the rest of the trajectory [3], CMA-ES compresses history into a covariance matrix and evolution paths [4], and Bayesian optimization retains every observation in a surrogate model [1, 5]. Large-scale benchmarks find that no single choice dominates [6], which suggests that the right amount of memory depends on the problem, yet in these methods the designer, not the problem, fixes it.

Learned optimizers [7, 8, 9, 10, 11], including LLM-based optimizers that read search histories as text [12, 13, 14, 15, 16, 17, 18], are usually evaluated by whether they beat their baselines. That establishes effectiveness but not how the history is used to pick the next candidate. The distinction matters most for in-context optimizers, which read a history and generate a candidate with no prescribed rule for using it. Their decisions let us ask which parts of the history they rely on and whether that reliance varies across problems.

We study this with JEPA-PS (JEPA Policy Search), a small in-context optimizer built on a joint-embedding predictive architecture [19, 20]. It is pretrained on parameter vectors and scalar returns from random rollouts with a loss that predicts the latent representation of the next trajectory point rather than the reward. At test time it reads a short history and proposes the next candidate in a single forward pass. Because its only input is that history, we can subset, reorder, permute, or corrupt it and observe how the proposal changes. JEPA-PS is therefore used here as an instrument for measuring memory use, not as a new optimizer to adopt.

What we find.

  1. History use depends on the problem. On CartPole the full trajectory beats the best-seen point by 22% (mean best reward 321.4→410.9321.4 \to410.9) and permuting reward order costs 37%; on MountainCar neither ablation has a measurable effect. We decompose this into four separable dependencies: the trajectory beyond its best point, parameter-space geometry, reward ordering rather than magnitude, and coarse temporal direction. Different problems switch each on or off to different degrees (Section 7).

  1. The pattern is associated with landscape ruggedness. History dependence is associated with fitness–distance correlation [21] at 0.738 across seven environments, and an independent reward-ordering probe at 0.832 across five. These are small-sample associations, so we pair them with an explicit sensitivity analysis and treat them as preliminary (§ 8).

  1. Transfer is real but limited. A pre-registered, six-part investigation finds genuine, well-powered transfer across unseen CartPole physics variants, a mechanism probe that does not corroborate that context is the source, and no practical advantage over classical search started cold (§ 9).

  1. JEPA-PS beats LLM optimizers, not classical search. It is consistently cheaper per proposal than LLM optimizers (about 77× on Swimmer, where it also scores more than twice as high) and usually competitive or better (§ 5). In a further comparison it beats ProPS and ProPS-Redundant while trailing plain CEM, and a restart-wrapped LLM result near the environment ceiling is attributable chiefly to the restart mechanism rather than reasoning (§ 6).

  1. No advantage over CMA-ES anywhere we tested. A six-part battery finds a narrow representational signal but no case in which JEPA-PS’s optimization performance exceeds a properly run CMA-ES (Appendix § 18).

What we do not claim. JEPA-PS is not a general-purpose optimizer. It does not consistently outperform restart CMA-ES, run faster per step than classical rules, or transfer across structurally different environments. The transfer we document is not shown to be practically useful, since in a matched-budget comparison JEPA-PS performs worse than cold-started classical search. We also do not propose an optimizer that exploits the fitness–distance/memory association; the association itself is the contribution.

Contributions.

  1. We develop a controlled ablation for measuring how much of its history a learned optimizer uses, and show that with the architecture fixed, history dependence varies across problems and reflects reward ordering, parameter geometry, broad temporal direction, and the trajectory after its best point.

  1. Across two mechanistically distinct interventions, we find an association between history dependence and fitness–distance correlation, accounting explicitly for sample size, leverage, and potential confounds.

  1. Ablations of the decoder and comparisons with latent-space CEM show that pretraining mainly provides reusable geometry rather than a proposal mechanism.

Related Work

Learned and LLM-based optimizers. One line of work trains networks directly as gradient-free or RL optimizers [7, 8, 9]; [10] scale this across thousands of tasks, and [11] frames the area as amortized optimization. A second line prompts an LLM with a running list of solutions and scores to optimize prompts, designs, programs, and policies [12, 13, 14, 22, 23, 15, 16]. [17] frame this explicitly as Bayesian optimization, using the LLM for candidate sampling and surrogate ranking; we adapt this two-role structure as a baseline (Appendix § 14). [18] instead fine-tune an open-weight LLM on BO trajectories directly – a closer architectural relative of JEPA-PS than pure prompting, which we do not reproduce. [24] independently find classical HPO beats pure-LLM optimizers on an LM-tuning task, consistent with our own comparisons (§ 6). [25] amortize black-box optimization across tasks, and [26]’s rank-based, ordinal archives parallel our finding that JEPA-PS is invariant to monotone reward transformations.

Joint-embedding predictive architectures. JEPA [19] predicts a target view’s embedding from a context view rather than reconstructing inputs; I-JEPA, V-JEPA, and related latent-prediction objectives instantiate it across modalities [20, 27, 28, 29], as do latent world models for control [30, 31]. To our knowledge, JEPA-PS is the first use of a JEPA-style latent-prediction objective to propose optimization parameters directly, rather than to represent states, images, or trajectories for downstream control.

Landscape analysis, population search, and restarts. Fitness–distance correlation [21] and random-walk autocorrelation [32] characterize search difficulty from cheap random samples; we use them as optimizer-independent descriptors and ask whether they predict a learned optimizer’s history use, a connection we are not aware of having been drawn. We adopt a standard island/swarm construction [33, 34, 35] as an off-the-shelf stabilizer, not as a contribution. Domain randomization across physics variants of one environment [36, 37] is structurally what our physics-variant transfer test (§ 9) does. Restarting on stagnation and running parallel instances, standard devices most notably in restart CMA-ES [38], are inference-time interventions we apply to JEPA-PS, explicitly testing rather than assuming whether their benefit is attributable to pretraining or to the restart mechanism alone (§ 9; Appendix § 18).

JEPA-PS as an Instrument

Setup. We optimize θ∈Rd\theta\in\mathbb{R}^{d} of a policy πθ\pi_{\theta}, θ∗=arg⁡max⁡θJ(θ)\theta^{*} = \arg\max_{\theta} J(\theta), J(θ)=Eτ∼πθ[∑tγtrt]J(\theta) = \mathbb{E}_{\tau\sim\pi_{\theta}}\left[\sum_{t} \gamma^{t}r_{t}\right], with no gradients or structural knowledge of JJ; one evaluation is a full rollout. Classical optimizers carry explicit state (a distribution, covariance, population); JEPA-PS is instead a stateless in-context function reading history Ht={(θi,ri)}i≤t\mathcal{H}_{t} = \{(\theta_{i},r_{i})\}_{i \le t} from a sliding buffer of capacity LL and emitting

θ^t+1=fϕ(Ht,r∗),r∗=pct⁡80({ri}i≤t),(1)\hat{\theta}_{t+1}=f_{\phi}(\mathcal{H}_{t},r^{*}), \qquad r^{*}=\operatorname{pct}_{80}(\{r_{i}\}_{i\le t}), \tag*{(1)}

where r∗r^{*} is a target-return conditioning signal. This isolates the two inputs we probe: the history, a set we can subset, reorder, and permute, and the reward encoding, which we can rank, rescale, permute, or corrupt. Figure 1 overviews the resulting architecture.

JEPA Optimizer: Architecture Overview

Figure 1. Overview of the JEPA optimizer.

Pretraining. We collect (θ,J(θ))(\theta,J(\theta)) pairs from random θ∼N(0,I)\theta\sim\mathcal{N}(0,I) on source environments spanning balance, locomotion, and control; the corpus holds only scalar returns and raw parameter vectors (no task descriptions, observations, or reward shaping), returns percentile-normalized. Windows of length Lctx∈[4,20]L_{\mathrm{ctx}} \in[4,20] slide over the stored pairs. A Transformer context encoder gϕg_{\phi} [39] maps a window of mean-normalized (θˉi,ri)(\bar{\theta}_{i},r_{i}) tokens to a latent sx∈Rdms_{x} \in\mathbb{R}^{d_{m}}; a predictor pψp_{\psi} maps (sx,r∗)(s_{x},r^{*}) to s^y\hat{s}_{y}; a decoder hξh_{\xi} maps latents to parameters, with Gaussian noise ϵ∼N(0,σI2I)\epsilon\sim\mathcal{N}(0,\sigma^{2}_{I}I) added at inference for exploration. Encoder and predictor train against a stop-gradient target from an EMA copy gϕ−g_{\phi}^{-} applied to a window including the next pair:

LJEPA=∥s^y−sg⁡(sy)∥22,sy=gϕ−({(θˉi+,ri+)}).(2)\mathcal{L}_{\mathrm{JEPA}}=\left\lVert\hat{s}_{y}-\operatorname{sg}(s_{y})\right\rVert_{2}^{2}, \qquad s_{y}=g_{\phi}^{-}\left(\{(\bar{\theta}_{i}^{+},r_{i}^{+})\}\right). \tag*{(2)}

In plain terms, the model predicts what the next latent summary will look like, not the reward itself. Two properties matter for what follows. (i) The loss never predicts reward: shuffling the θ\theta–rr correspondence during pretraining leaves downstream performance unchanged (Appendix § 22), yet reward information is used at test time (Section 7), by a mechanism the objective never supervised. (ii) The decoder receives no gradient: hξh_{\xi} does not appear in Eq. (2) and is a fixed random projection in the default configuration (measured gradient norms: encoder 0.248, predictor 0.453, decoder 0.000; Appendix § 23), so the default system is latent-guided.

Inference and population wrapper. Algorithm 1 is the single-agent loop. Since history can collapse onto one region on harder tasks, we optionally wrap KK agents with independent warm-starts, migration of the global best

Algorithm 1 JEPA in-context policy optimization (single agent)

Require: pretrained gϕ,pψ,hξg_{\phi}, p_{\psi}, h_{\xi}; environment EE; budget TT; capacity LL; warm-start length LwarmL_{\mathrm{warm}}

  1. H←∅\mathcal{H} \gets\varnothing; warm-start with LwarmL_{\mathrm{warm}} random rollouts

  2. for t=Lwarm+1,…,Tt = L_{\mathrm{warm}}+1, \ldots, T do

  3. μ,σ←mean⁡,std⁡\mu, \sigma\gets\operatorname{mean},\operatorname{std} of {θi}\{\theta_i\} over H\mathcal{H}; r∗←pct⁡80({ri})r^{*} \gets\operatorname{pct}_{80}(\{r_i\})

  4. s^y←pψ(gϕ({((θi−μ)/σ,ri)}),r∗)\hat{s}_y \gets p_{\psi}\left(g_{\phi}(\{((\theta_i-\mu)/\sigma, r_i)\}), r^{*}\right)

  5. θ^t←hξ(s^y)⋅σ+μ+ϵ\hat{\theta}_t \gets h_{\xi}(\hat{s}_y) \cdot\sigma+ \mu+ \epsilon, ϵ∼N(0,σn2I)\epsilon\sim N(0,\sigma_n^2 I)

  6. rt←J(θ^t)r_t \gets J(\hat{\theta}_t); H←H∪{(θ^t,rt)}\mathcal{H} \gets\mathcal{H} \cup\{(\hat{\theta}_t,r_t)\}

  7. end for

  8. return arg max⁡(θ,r)∈Hr\operatorname*{arg\,max}_{(\theta,r)\in\mathcal{H}} r

every 30 iterations, and isolation phases suspending migration for 6 steps every 60 iterations [34, 33] – Swarm-JEPA. Section 9 separately introduces a restart-on-stagnation wrapper, reinitializing one agent’s history rather than running several in parallel. Every method here, learned or classical, is charged the same total rollouts (fixed-total accounting; Appendix M).

Experimental Setup

Environments and baselines. We use 19 control environments [40, 41]: balance (CartPole, InvertedPendulum, InvertedDoublePendulum), locomotion (Swimmer, Hopper, Walker2d, Reacher), continuous/discrete control (MountainCarContinuous, MountainCar), discrete navigation (FrozenLake, CliffWalking), and six custom/externally-sourced tasks (Maze, Nim, Navigation, Pong, BallLanding), linear policies πθ(s)=Ws+b\pi_{\theta}(s) = Ws + b, d∈[3,108]d \in[3, 108] [2]. Eight were added to broaden coverage beyond continuous control; Appendix § 13 gives provenance, confirms dimensionality against source. We compare against 21 optimizers: classical (CEM, CMA-ES, PSO, BO, OpenAI-ES, (μ+λ)(\mu+\lambda)-ES, Tabu, random), strong modern baselines (TuRBO, SMAC, TPE, IPOP-/BIPOP-CMA-ES, L-BFGS-B), policy-search/amortized methods (ARS, CEM-RL, CERL, PES, PABBO), an OPRO-style LLM optimizer (Qwen2.5-7B, [12]), and an LLAMBO adaptation [17] (Appendix § 14). Full grids: Appendix § 16, Appendix § 19.

Protocol. Every method receives T=400T=400 total evaluations, nep=10n_{\mathrm{ep}}=10 rollouts each, 3–5 seeds, on the 11 original environments; we report mean best reward ±\pm std. The eight extended environments (Appendix § 13) use a timestep-matched budget (T=2,000T=2{,}000–20,00020{,}000, nep=5n_{\mathrm{ep}}=5, n=10n=10 seeds) with classical/policy-search baselines only. Swarm-JEPA uses K=3K=3, σn=0.3\sigma_n=0.3. Section 7’s ablations use the reduced-budget protocol of Appendix § 21; § 9 uses its own pre-registered split. A standardized 10-seed comparison is in Appendix § 28.

Quantifying memory and landscape. We summarize each environment’s trajectory use with two scalars derived from inference-time ablations: history dependence, the fractional gain from using the full history rather than the best-seen point, and ordering sensitivity, the fractional loss when history rewards are randomly permuted:

HD=Rfull−Rbest−only∣Rfull∣,OrdSens=Rorig−Rperm∣Rorig∣.(3)\mathrm{HD}=\frac{R_{\mathrm{full}}-R_{\mathrm{best-only}}}{\lvert R_{\mathrm{full}}\rvert}, \qquad\mathrm{OrdSens}=\frac{R_{\mathrm{orig}}-R_{\mathrm{perm}}}{\lvert R_{\mathrm{orig}}\rvert}. \tag*{(3)}

Both are zero when the optimizer ignores the ablated information. The landscape descriptor is the fitness–distance correlation, FDC=corr⁡(J(θ),−∥θ−θ∗∥)\mathrm{FDC}=\operatorname{corr}(J(\theta),-\lVert\theta-\theta^*\rVert) [21]; since θ∗\theta^* is unknown we use the best probe sample as proxy, computed from 150–300 random evaluations, and treat FDC as an ordinal smoothness descriptor (sample-size sensitivity in Appendix § 20). Five further descriptors (ACL, basin size, smoothness, neutrality, information content, dispersion) are defined in Appendix § 20.

JEPA-PS as an Optimizer

Section 7 examines what information JEPA-PS uses, and Section 9 tests what its pretraining transfers.

Competitive with classical search, but not consistently better. Swarm-JEPA is competitive with tuned classical search on several tasks, but it does not consistently outperform it. Swarm-JEPA (K=8K=8) reaches 416.1 on CartPole, 293.1 on Swimmer, and 1039.7 on Hopper. These scores exceed CEM’s respective scores of 360.9, 250.8, and 620.2 (Appendix § 16, Table 7). IPOP-CMA-ES and ARS solve CartPole and reach approximately 352 on Swimmer, while TuRBO reaches 358.0 on Swimmer. On Hopper, TuRBO, BIPOP-CMA-ES, and TPE finish within approximately 40 points of Swarm-JEPA (Table 13). On InvertedPendulum, JEPA-PS reaches only 27.9, compared with IPOP-CMA-ES’s 890.0. Diagnostics in Appendix § 31 do not indicate an encoder or decoder failure. A 300-sample probe finds a best reward of only approximately 58, about 15 times below IPOP-CMA-ES’s score. These results are consistent with difficulty finding the high-reward basin, rather than simply failing to use a trajectory once that basin is found. The evaluated JEPA-PS configuration has no restart mechanism; Section 9 tests whether adding one helps. LLM comparisons appear in Section 6, and the dedicated CMA-ES battery appears in Appendix § 18.

MethodCartPoleSwimmerHopperInvPend.MtnCar(C)
Random Search55.5±16.9120.5±38.219.2±3.688.5±27.624.9±19.1
CEM360.9±95.3250.8±59.0620.2±201.7436.4−5.0
PSO322.5±96.5263.4±40.0463.3±240.8296.82.3
IPOP-CMA-ES500.0±0.0352.2±3.3659.9±361.4890.0±220.0−0.0±0.0
ARS500.0±0.0352.1±2.5789.1±203.250.9±8.5−0.0±0.0
LLM optimizer (OPRO)186.0±218.0140.3±93.31026.7±0.8356.0±455.4−0.1±0.0
LLAMBO (Appendix C)190.1±219.4————
Swarm-JEPA (K=3K=3)421.6±156.9273.8±32.1133.3±67.827.4±28.8−4.9±6.2
Swarm-JEPA (K=8K=8)416.1±28.7293.1±28.71039.7±33.527.9±29.9−5.1±6.3
Latent-CEM393.7±45.2304.5±38.5684.0±340.827.7±28.6−1.2±1.4

Table 7. Mean best reward (mean ± std, 5 seeds) at T=400T=400 evaluations. Bold: best in column among rows shown. The full grid (17 methods, BallLanding) is Table 8; stronger modern baselines are in Table 13 (e.g. TuRBO reaches 1047.4 on Hopper); the eight extended environments are in Table 5.

MethodCartPoleSwimmerHopperMtnCar(C)
Swarm-JEPA (K=8K=8, n=10n=10)416.1293.11039.7−5.1-5.1
Swarm-JEPA (single agent, n=10n=10)410.9±178.2410.9 \pm178.290.6±54.190.6 \pm54.1136.0±67.4136.0 \pm67.4102.3
IPOP-CMA-ES500.0±0.0500.0 \pm0.0352.2±3.3352.2 \pm3.3659.9±361.4659.9 \pm361.4−0.0±0.0-0.0 \pm0.0
BIPOP-CMA-ES500.0±0.0500.0 \pm0.0353.2±1.7353.2 \pm1.7999.4±31.8999.4 \pm31.833.0±46.833.0 \pm46.8
TuRBO500.0±0.0500.0 \pm0.0358.0±3.3358.0 \pm3.31047.4±29.51047.4 \pm29.533.0±46.733.0 \pm46.7
TPE (Optuna)500.0±0.0500.0 \pm0.0293.5±120.8293.5 \pm120.81035.5±13.41035.5 \pm13.4−0.1±0.1-0.1 \pm0.1
CMA-ES / PSO / TPE / IPOP500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
CEM484.2±31.7484.2 \pm31.7250.8620.2−5.0-5.0
Random Search55.5±16.955.5 \pm16.9258.1±29.6258.1 \pm29.619.2±3.619.2 \pm3.624.9±19.124.9 \pm19.1
L-BFGS-B (FD)119.5±101.5119.5 \pm101.575.3±90.975.3 \pm90.985.8±28.685.8 \pm28.6−34.2±30.9-34.2 \pm30.9

Table 13. Strong-baseline grid (n=10n=10 seeds unless noted, T=400T=400, runner rollout). All modern methods outperform Swarm-JEPA on Swimmer.

Eight additional environments. Table 5 in Appendix § 13 reports ARS, CEM, CMA-ES, and PES results across eight additional domains. Pong, InvDbIPendulum, MountainCar(D), and Navigation show variation across methods, while FrozenLake and Nim yield weak performance.

On CliffWalking, all four methods converge to the same cliff-fall outcome. The result has zero variance across ten seeds. The diagnostics in Appendix § 13 attribute this behavior to the landscape under linear policies rather than an implementation error. Random policies concentrate in two outcome classes: an immediate cliff fall or full-length wandering. Almost no intermediate outcomes occur, and every evaluated method converges to the cliff-fall outcome.

Reward-relevant structure in the latent space. The latent space contains reward-relevant information, but its advantage over a sampling control depends on the protocol. In the ten-seed comparison in Table 23, a null-proposer sampling method reaches 302.1 on CartPole and 151.9 on Swimmer. JEPA-PS reaches 410.9 and 293.1, respectively. However, a more closely matched Swimmer comparison, using shared warm starts and noise seeding, finds that the null model matches or exceeds JEPA-PS (Appendix § 23). Separately, latent norm correlates with reward at 0.507, compared with 0.015 for a random projection. This correlation does not, by itself, show that the learned proposer uses that structure better than a matched sampling control.

JEPA-PS versus LLM-Based Optimizers

We compare JEPA-PS with several LLM-based optimization systems: the OPRO-style and LLAMBO baselines in Table 7, an independent five-seed comparison with two LLM agents, and restart-wrapped LLM evaluations on nominal and perturbed CartPole. These experiments use different implementations and search configurations. We interpret their scores within their respective protocols.

Single-call LLM optimization and LLAMBO. Swarm-JEPA performs better than the single-call LLM optimizer on CartPole and Swimmer, but not on every task. Swarm-JEPA reaches 416.1 on CartPole and 293.1 on Swimmer, compared with 186.0 and 140.3 for the LLM optimizer. The methods are comparable on Hopper (1039.7 versus 1026.7), while the LLM performs better on InvertedPendulum (356.0 versus 27.9). JEPA-PS makes no language-model call at search time and requires no text parsing. Its measured proposal latency is 2.61 ms, compared with approximately 200 ms for the 7B LLM, a roughly 77-fold difference (Table 9). Classical updates are cheaper still: CEM takes 0.024 ms and CMA-ES takes 0.053 ms in the measured configurations. JEPA-PS uses a fixed-size latent representation, and its pretraining takes a few GPU-minutes (Appendix D). LLAMBO [17] adds an LLM-generated candidate pool and a surrogate-ranking step on the same backend (Appendix C). Many ranking calls fall back to unranked defaults after parsing failures. This experiment therefore measures LLAMBO with an unreliable ranking stage, not its performance under consistently successful ranking.

An independent comparison with LLM agents. Both JEPA variants have higher mean scores than the two LLM agents, but plain CEM performs best. A separate five-seed CartPole experiment compares JEPA-PS and Swarm-JEPA with two Qwen2.5:7B agents, ProPS and the retrieval-augmented ProPS-Redundant, alongside CEM and random search at a matched budget (Table 1; per-seed and per-task breakdowns, including the MountainCar robustness sweep, are in Appendix O).

MethodMean ± std
CEM Baseline500.00±0.00500.00 \pm0.00
JEPA (ours)365.79±182.71365.79 \pm182.71
Swarm-JEPA (ours)223.73±114.07223.73 \pm114.07
ProPS-Redundant (Qwen2.5:7B)170.53±74.25170.53 \pm74.25
ProPS (Qwen2.5:7B)103.01±43.47103.01 \pm43.47
Random Search60.52±14.4960.52 \pm14.49

Table 1. Independent five-seed CartPole comparison at a matched budget. Scores report mean best reward ± standard deviation.

Restart-wrapped LLM optimization on nominal CartPole. The restart-wrapped LLM nearly solves nominal CartPole, but this score includes random fallback after parsing failures. Restarting after 40 consecutive non-improving evaluations raises the score to 499.44±1.68499.44 \pm1.68 over ten seeds. The parse-failure rate is 0.387, or 38.7% (Table 2). The score therefore measures the complete system: valid LLM proposals, random-noise fallback, and the restart wrapper. The per-evaluation trace contains long stretches at reward 500 interrupted by drops to approximately 9, recurring on roughly the restart timescale.

What the comparisons explain. The weaker LLM-agent scores cannot be attributed solely to complete generation failures. ProPS and ProPS-Redundant report a 0.0%-complete-failure rate across their five-seed runs. However, complete failure and per-call parsing failure are different measures. Across the evaluated OPRO-style, LLAMBO, and ProPS configurations, LLM-based search does not consistently match JEPA-PS on CartPole or Swimmer. Restarting substantially improves the complete LLM system on CartPole. The present experiments do not isolate how much of this gain comes from valid LLM proposals, fallback sampling, or restarting. JEPA-PS is trained on continuous (θ,r)(\theta, r) trajectory structure, whereas the general LLM must infer a search direction from serialized examples without the same task-specific objective. This account is compatible with [24] and the modest LLAMBO gains observed here, but it remains an interpretation rather than an experimentally established mechanism.

Held-out CartPole physics variants. The restart-wrapped LLM maintains high scores on the held-out physics variants in this experiment. We evaluate ten seeds per variant with a budget of 400 evaluations and a restart after 40 non-improving evaluations. Table 2 includes the three training variants and four held-out variants from Appendix G. These held-outs include both single-factor and combined perturbations. They are distinct from the four transfer-test variants in Section 9. Mean best reward ranges from 480.92 to 500.00 on the training variants and from 474.06 to 500.00 on the held-out variants. All four held-out variants have zero parsing failures, while force_x0.5 has a failure rate of 0.045. Nominal CartPole and gravity_x2 have higher failure rates of 0.387 and 0.569, respectively. These results show no obvious held-out performance deficit for the complete restart-wrapped system on this perturbation family. They do not separate the contribution of the LLM’s proposal rule from that of the wrapper.

VariantSplitMean ±\pm stdParse-failure rate
nominalTRAIN499.44±1.68499.44\pm1.680.387
gravity_x2TRAIN500.00±0.00500.00\pm0.000.569
force_x0.5TRAIN480.92±51.04480.92\pm51.040.045
gravity_x1.5+length_x1.3HELD-OUT474.06±57.59474.06\pm57.590.000
masspole_x2HELD-OUT487.12±38.24487.12\pm38.240.000
masscart_x0.5+force_x1.5HELD-OUT500.00±0.00500.00\pm0.000.000
length_x0.5+force_x2HELD-OUT500.00±0.00500.00\pm0.000.000

Table 2. Restart-wrapped LLM optimization on training and held-out CartPole physics variants: ten seeds, budget 400, and restart after 40 non-improving evaluations. Scores report mean best reward ±\pm standard deviation. Parse-failure rates are fractions of calls; failed calls use random-noise fallback.

What Does the Optimizer Use?

We now examine JEPA-PS through controlled interventions. These experiments change the history presented to the model while leaving the true environment evaluations unchanged. The protocols use different budgets and seed configurations, so comparisons are made within each experiment.

History content matters more than buffer capacity in the tested setting. Changing what the model remembers matters more than increasing the buffer size in the low-budget experiment. Sweeping buffer capacity LL from 4 to 400 leaves best reward unchanged under the fixed low-budget protocol of Table 18. This result remains to be tested at longer budgets, such as T=400T=400. In the history-content comparison reported in Table 18, full history reaches 410.9, best-only reaches 325.5, and recent-8 reaches 333.0. CEM reaches 360.9 in the baseline comparison, although that comparison changes the search rule as well as its memory.

ConditionFullBest-onlyRecent-8Oldest-8CEM
Mean best410.9325.5333.0317.6360.9

Table 18. History, order, and reward-transformation ablations on CartPole (and Swimmer/MountainCar where noted). The full trajectory outperforms every truncated or corrupted variant; the optimizer is sensitive to coarse temporal direction and reward rank but not to fine-grained order or reward magnitude. Memory conditions vs. CEM (n=5n=5 seeds, 200 iterations, K=8K=8 for subset conditions)

Several parts of the history affect performance. JEPA-PS uses more than the best point in its history. In Table 3A, performance falls when the history is reduced to its best point (−22.3%), parameter dimensions are permuted (−19.8%), reward values are shuffled (−17.0%), or temporal order is reversed (−13.6%). These results show sensitivity to trajectory content, coordinate identity, parameter–reward associations, and some changes in ordering. Randomly shuffling temporal order has no measured cost in the three-seed experiment, whereas reversing it does. This difference does not uniquely identify a temporal-direction mechanism. The pretraining-time shuffle in Appendix K is a separate intervention and also has no measured performance effect.

InterventionMean bestΔ\DeltaRewardsCartPoleSwimmerMtnCar(C)
Control (full history)410.9—
Best-only (single point)319.2−22.3%
Remove best-seen point†285.1−30.6%
Permute parameter dims.329.7−19.8%
Shuffle reward values341.1−17.0%
Reverse temporal order354.7−13.6%
Shuffle temporal order†410.9+0.0%
Original337.9263.4−28.41
Ranked338.3262.9−28.39
Monotone (log)337.7263.1−28.42
Permuted212.8245.7−28.40
OrdSens0.3700.068−0.000

Table 3. Inference-time history interventions. Panel A uses CartPole with five seeds and 200 iterations; † denotes three seeds. Panel B uses the reduced-budget protocol in Appendix J. The panels use different protocols, so their reward-permutation effects are not directly interchangeable.

Reward ordering matters more than the tested reward scales. The tested changes in reward scale have little effect, but shuffling rewards can hurt performance. Original rewards, rank-transformed rewards, and a monotone logarithmic transformation produce nearly identical scores in Table 3B. This is consistent with reliance on relative reward ordering, as in rank-based archives [26]. It does not establish invariance to every monotone transformation. Permuting rewards reduces performance by approximately 37% on CartPole and 6.8% on Swimmer, while producing essentially no change on MountainCar. Section 8 examines this variation.

History rewards matter more than the target-return input. Removing rewards from the history hurts performance; changing the target-return input does not. In Eq. (1), the predictor receives r∗=pct⁡80({ri}i≤t)r^{*}=\operatorname{pct}_{80}(\{r_i\}_{i\le t}) as a target-return input. This value is separate from the per-step rewards stored in the history. Setting r∗r^{*} to zero, a random value, or the worst Table 4: Fitness–distance correlation (FDC), history dependence (HD), and reward-ordering sensitivity (OrdSens). The reported Pearson correlation between HD and FDC is 0.738 across seven environments, with a bootstrap 95% confidence interval excluding zero. The original OrdSens correlation uses five environments, excluding InvertedPendulum and Hopper. The all-seven OrdSens calculation includes their displayed zero entries. The all-seven OrdSens and Spearman correlations are computed from the rounded entries shown.

EnvironmentFDCHDOrdSens
CartPole0.6610.2180.370
InvPendulum0.5480.0500.000
Swimmer0.4820.0440.068
Hopper0.439−0.0060.000
Reacher0.4010.0740.003
Walker2d0.3260.0000.000
MountainCar(C)0.1530.004−0.000
Pearson rr with FDC, all 70.7380.66
Pearson rr with FDC, 5-env. subset—0.832
Spearman ρ\rho with FDC, all 70.5710.571

Table 4. Fitness–distance correlation (FDC), history dependence (HD), and reward-ordering sensitivity (OrdSens).

observed reward produces no measured change in final CartPole performance. Percentiles from 50 to 99 also produce identical results (Appendix § 22). In contrast, zeroing the history rewards reduces performance by 34.8%. Changing r∗r^* does alter the proposal distribution. On Swimmer, the reported shifts are 5–9 standard deviations, with MMD values of 1.6–1.8. The model therefore responds to this input, but the response provides no measured final-reward benefit in these experiments.

Pretraining provides a useful latent geometry. Classical search can use the pretrained latent space without relying on the learned proposal rule. Latent-CEM matches Swarm-JEPA on CartPole and Swimmer in the reported comparison (Table 7). It also improves over raw-space CEM by 41.6% in the three-seed experiment of Appendix § 27 and by 21% in a separate five-seed experiment. Reconstruction fine-tuning of hξh_{\xi} reduces CartPole reward from 410.9 to 293.7 (−28.5%), and end-to-end retraining reduces it to 269.0 (−34.5%; Appendix § 23). Across ten random linear decoders applied to the same pretrained latents, the between-decoder standard deviation is 26.7, compared with within-decoder seed variation of 178.2. Performance is relatively insensitive to the particular fixed linear readout in this experiment. These findings favor a representational contribution from pretraining over a uniquely effective decoder-specific proposal mechanism. They do not imply that all decoder changes are harmless: the fine-tuning interventions reduce performance substantially. The learned proposal direction is also nearly orthogonal to six classical update directions, with cosine similarities of 0.003–0.015 (Appendix § 29). This directional difference does not, by itself, establish a performance advantage.

Does Landscape Structure Predict the Memory Requirement?

Section 7 shows that the benefit of history varies across environments. We now ask whether that benefit is associated with an independently measured property of the search landscape, and how much evidence supports the association.

History helps by different amounts across environments. History helps most on CartPole and has little measured benefit on several other tasks. Table 4 reports history dependence HD and ordering sensitivity OrdSens from Eq. (3), together with fitness–distance correlation FDC, for seven environments with JEPA checkpoints. History dependence ranges from 0.218 on CartPole to approximately zero on MountainCar, Walker2d, and Hopper. In the cross-environment history-dependence protocol of Table 15, CartPole reaches 410.9 with full history and 321.4 with best-only history. This best-only score is separate from the score of 319.2 in the five-seed, 200-iteration intervention experiment of Table 3A and the score of 325.5 in the history-content comparison of Table 18. On MountainCar, the full-history and best-only scores in the HD evaluation are nearly identical: −28.41 and −28.51.

EnvFDCFullBest-onlyHDMetric pairrrnn
CartPole0.661410.87321.430.218
Swimmer0.482263.43251.730.044HD vs. FDC0.7387
MountainCar(C)0.153−28.41−28.510.004HD vs. FDC | ACL−0.4877
HD vs. FDC | Basin0.5517
OrdSens vs. FDC0.8325

Table 15. History dependence by memory condition (mean best reward, 5 seeds) and correlations with landscape descriptors (bootstrap 95% CI, 1,000 resamples).

History dependence is associated with FDC. Higher FDC is associated with a larger history benefit in this small sample. Across seven environments, the Pearson correlation between HD and FDC is 0.738 (Table 4, Figure 2). The correlation between OrdSens and FDC is 0.832 in the five-environment analysis. These probes capture different aspects of the history, but they are not independent replications. They use overlapping environments and checkpoints, and both associations are strongly influenced by CartPole. On synthetic NK landscapes, FDC decreases monotonically from 0.389 to 0.028 as epistasis increases (Appendix I). This supports its use as a landscape descriptor in that controlled setting. It does not independently establish a link between FDC and memory.

Figure 2. Landscape structure and measured history benefits. (A) History dependence versus fitness–distance correlation across seven environments. (B) Reward-ordering sensitivity versus FDC. The tested monotone reward transformations have little effect, while reward permutation has an environment-dependent effect. (C) Memory-condition performance normalized to the full-history baseline.

Limits of the evidence. The Spearman correlation between HD and FDC is 0.571, below the two-tailed significance threshold at n=7n=7. Removing CartPole reduces Pearson correlation to approximately 0.50 and Spearman correlation to approximately 0.31. The OrdSens correlation of 0.832 applies only to the five-environment subset; the all-seven calculation using the displayed entries gives approximately 0.66. Conditioning on ACL changes the partial correlation to −0.487-0.487. With only seven environments and substantial FDC–ACL collinearity, this sign change does not support a stable conditional interpretation (Appendix I). History dependence is computed from best-over-history reward. A near-zero value may mean that extra history is unnecessary, but it may also mean that neither condition improves beyond initialization. This distinction matters on floor-limited tasks such as MountainCar and in basin-discovery failures such as InvertedPendulum. The record-statistics analysis in Appendix N finds more warm-up-determined best scores than its null model predicts, with p<10−4p < 10^{-4} in each environment tested. Initialization dominance is therefore a substantive confound for best-over-history metrics. Section 9 also measures post-warm-up gain to address it.

Interpretation. The results suggest a link between landscape structure and history use, but they do not establish a general rule. CartPole has both the highest FDC and the largest measured history benefit. Lower-FDC environments show less benefit. However, the experiments do not fully separate a sufficient best point from a failure to make progress. The checkpoints are also environment-specific, although they share an architecture and training recipe (Appendix A). The comparison therefore concerns that shared recipe, not a single weight vector used across tasks.

Does Pretraining Transfer, or Memorize?

Every checkpoint in Sections 5–8 is pretrained and evaluated within a single environment. These results alone do not establish transfer beyond the pretraining dynamics. The tests separate three questions: whether pretrained weights help on unseen dynamics, whether that help depends on the supplied context, and whether the resulting optimizer competes with classical search.

Part 1: transfer to held-out physics variants. Pretraining helps on CartPole physics variants that were not used in training. We pretrain an encoder and predictor on nominal CartPole, gravity ×2\times2, and force ×0.5\times0.5. The control uses the same decoder but a reinitialized encoder and predictor. We evaluate both models on these training variants and on two held-outs: gravity ×0.5\times0.5 and length ×2\times2. Across 100 paired seeds, the pretrained model exceeds the control on every training variant (p<10−5p < 10^{-5}) and both held-outs (p<10−8p < 10^{-8}). The held-out differences are 104–110 reward points, compared with a mean training difference of approximately 83 points. This is within-environment physics transfer, similar to domain randomization [36, 37], not cross-task generalization. Parts 4 and 6 compare optimization performance with classical search. Appendix G provides the separate CMA-ES battery.

Part 2: larger, joint physics perturbations. The pretraining advantage persists when two physical constants change together. Training changes one constant at a time, using multipliers of 2.0 or 0.5. We test two held-out variants with joint changes and unseen multipliers: gravity×3.0\times3.0 plus length×1.5\times1.5 (EXTREME-A), and gravity×0.33\times0.33 plus force×2.0\times2.0 (EXTREME-B). Without retraining, the pretrained model exceeds the untrained control on both variants over 100 seeds. The difference is +50.6+50.6 on EXTREME-A (p=2.0×10−5p=2.0\times10^{-5}), approximately 61% of Part 1’s mean training effect. It is +90.2+90.2 on EXTREME-B (p=2.9×10−8p=2.9\times10^{-8}), approximately 108%.

Part 3: matched versus mismatched context. The context-swap test does not detect a reliable benefit from matched warm-up rewards. Parts 1 and 2 show that pretrained weights help. They do not show whether that help comes from adapting to the current variant through the history. To test this, we keep the warm-up parameter draws fixed but replace their rewards with rewards from a different physics variant. If variant-specific warm-up information helps, matched histories should perform better than mismatched histories. A run’s best reward can be determined entirely during warm-up, which can hide differences in later search. We therefore compare post-warm-up gain. Across 100 seeds, the matched-minus-mismatched difference is +3.4+3.4 on gravity×0.5\times0.5 (p=0.85p=0.85; 72 informative seeds) and +24.0+24.0 on length×2\times2 (p=0.37p=0.37; 87 informative seeds).

Part 4: classical search on the two Part 1 held-outs. On both Part 1 held-outs, ARS and CEM outperform unwrapped JEPA-PS. We run ARS and CEM from scratch at T=400T=400 on gravity×0.5\times0.5 and length×2\times2. Both reach 500.0±0.0500.0\pm0.0 across 20 seeds on each variant. Unwrapped JEPA-PS reaches 437.5±139.2437.5\pm139.2 and 319.9±156.4319.9\pm156.4, respectively. This comparison shows that transfer relative to an untrained control does not provide an advantage over classical search here. Part 6 reports the broader four-variant comparison, adding EXTREME-A and EXTREME-B. Appendix § 18 compares against CMA-ES.

Part 5: selecting an inference-time intervention. Inference-time interventions can greatly improve success, but that gain alone does not show a benefit from pretraining. We evaluate 17 interventions on three validation variants that are separate from the test set: force×1.5\times1.5, gravity×1.5\times1.5, and length×0.6\times0.6. The interventions include longer warm-ups, noise schedules, restart-on-stagnation, multi-start search, selection variants, and combinations. Several configurations raise pooled success, defined as reward ≥490\ge490, from 0.38 to as high as 0.99. A wrapper could also improve an untrained network, so we run the corresponding untrained controls. We use the pretrained-versus-untrained comparison when selecting the final configuration. Restarting after 40 non-improving evaluations is carried forward to Part 6. Full per-configuration numbers and the pre-registered attribution rule are in Appendix § 17.

Part 6: final evaluation on four held-out variants. Restarting helps both models, and the pretrained model remains ahead of the untrained control. The final evaluation uses 100 seeds on four held-out variants: gravity×0.5\times0.5, length×2\times2, EXTREME-A, and EXTREME-B. Restart-after-40 raises pooled pretrained success from 0.47 to 0.84. It raises pooled untrained-control success from 0.14 to 0.66. Thus, the wrapper contributes substantially, while pretrained weights retain an advantage under the same wrapper.

Wrapped JEPA-PS matches the classical success rate on two variants, but not on all four. Its success rate is 1.00 on gravity×0.5\times0.5 and EXTREME-B, 0.81 on length×2\times2, and 0.56 on EXTREME-A. The four-variant ARS/CEM comparison gives a success rate of 1.00 for both classical methods on every variant. The restart intervention narrows the overall gap but does not close it.

Part 7: broader training curricula. Neither tested curriculum expansion produces a reliable improvement. First, we broaden pretraining by sampling gravity, force, and length log-uniformly over [0.7,1.5][0.7, 1.5], excluding test multipliers. The broader curriculum is directionally ahead on all three Part 5 validation variants. However, the pooled success-rate gain of +0.06+0.06 falls below the 0.10 criterion, and no individual variant shows a significant improvement. Second, we pretrain on a mixture of CartPole and synthetic NK landscapes with K=0,2,4,8K=0, 2, 4, 8. Zero-shot evaluation on Rastrigin gives −49.5±8.1-49.5\pm8.1 over 20 seeds. This does not show a detectable improvement over the narrow-curriculum or BBOB-only results, both approximately −53.0-53.0 (Appendix § 30). All remain well below CEM’s −16.2-16.2.

Summary of the transfer investigation. Pretraining transfers across CartPole dynamics, but it does not establish a practical advantage over classical search. The pretrained model exceeds its untrained control on unseen physics variants, including large joint perturbations. However, the limited context-swap test does not identify variant-specific warm-up conditioning as the cause. The best tested intervention also does not consistently match ARS or CEM, and the curriculum expansions provide no reliable additional gain. Appendix § 18 extends the investigation with a direct six-part comparison against CMA-ES.

Discussion

Memory benefits depend on the problem. The same architecture and training recipe can show very different benefits from history across tasks. History interventions have a large effect on CartPole and little measured effect on several other environments. This suggests that useful memory depends on the search problem, not only on the optimizer’s design. The association with FDC offers one possible explanation. However, the small sample, CartPole’s influence, and warm-up-dominated runs limit that conclusion. Little measured history benefit on a difficult task does not prove that the best point is sufficient. It may instead mean that neither condition makes progress. The results therefore motivate landscape-aware studies of memory, rather than a single prescription for how much history every optimizer should retain.

What pretraining provides. The strongest evidence is for a useful representation, not a uniquely effective learned proposal rule. Latent-CEM uses the pretrained geometry without relying on the learned decoder and matches Swarm-JEPA in the reported comparisons. The decoder experiments support the same interpretation. The physics-transfer results further show that the benefit of pretrained weights is not confined to the exact training dynamics. The practical and mechanistic limits remain clear. The context-swap probe does not establish how variant-specific history supports transfer. Restarting improves both pretrained and untrained systems, and JEPA-PS still does not consistently match ARS or CEM across the transfer test set. The direct CMA-ES battery also finds a representational signal without a performance advantage at the evaluated budgets (Appendix § 18).

Conclusion

We asked what a learned optimizer actually uses of its history, and built a small, reward-agnostic in-context model so the question could be answered by ablation rather than speculation. The resulting model is capable but not dominant; its main value is diagnostic. History use ranges from a 22% gain on CartPole to no measurable gain on MountainCar, the model reads reward ordering, parameter geometry, and coarse temporal direction rather than magnitude, and this variation associates with landscape FDC under two mechanistically different probes – though sample size and floor effects (§ 8) limit the claim today. A six-part, preregistered investigation compared transfer with memorization and reported a full, unreconciled picture. Transfer across physics variants is real, well-powered, and robust to perturbations beyond the training data. However, a direct probe of the proposed mechanism found no supporting evidence, and even the best intervention did not match classical search started from scratch. Compared with LLM-based optimizers, JEPA-PS is consistently less expensive and usually competitive or better. Still, plain CEM outperforms every learned method. A dedicated six-part comparison with CMA-ES reveals there is a genuine but narrow representational signal, with no optimization or transfer advantage at any tested budget or condition. Based on this evidence, optimization memory appears to be shaped by the search problem rather than being a fixed property of the optimizer.

Implementation Details

The model uses dm=128d_m = 128, a 4-head Transformer context encoder, and 3-layer MLP predictor and decoder, trained with Adam [42] at learning rate 10−310^{-3} and EMA decay τ=0.99\tau= 0.99. We pretrain two primary checkpoints, CartPole (d=8d=8) and Swimmer (d=16d=16), on 20,000 trajectories for 250 epochs, together with task-specific checkpoints for Hopper, InvertedPendulum, Reacher, and Walker2d used in the landscape analysis. Checkpoints are environment-specific but share one architecture and pretraining recipe; the history-dependence comparison of § 8 is therefore across checkpoints trained with an identical recipe, not across a single weight vector. Inference noise is σn=0.3\sigma_n = 0.3. The predictor conditions on r∗r^* by taking it directly as a scalar input alongside sxs_x; the target return is the 80th percentile of observed returns by default, a choice to which performance is invariant (Table 20). Swarm-JEPA defaults to K=3K=3, Δmig=30\Delta_{\mathrm{mig}}=30, Δiso=60\Delta_{\mathrm{iso}}=60, Tiso=6T_{\mathrm{iso}}=6. The § 9 checkpoints are pretrained with the same architecture and optimizer settings, on data generated by local perturbation around the running best (matching the main pretraining recipe’s trajectory-generation procedure), with an auxiliary decoder-reconstruction loss (weight 0.1) and a working early-stopping check (cosine similarity of context-encoder embeddings across independent histories, aborting if it exceeds 0.95) that we verified catches representation collapse before it completes a full run, having observed collapse in earlier, unrelated pretraining attempts on higher-dimensional locomotion environments not reported in the main text.

Extended Environment Suite

We add eight environments drawn from or matched to an external reference implementation of a related black-box policy-search benchmark, to broaden coverage beyond continuous control. For each we confirmed the parameter dimension against the source implementation’s own configuration rather than assuming a convention; the custom environments (Nim, Pong) were vendored directly from that source.

Nim. The source implementation’s action space allows an action value corresponding to removing zero sticks, which is treated as an immediately invalid move; this is a property of the environment as sourced, not an artifact we introduced, but it means roughly one third of any linear policy’s action space is a guaranteed loss regardless of the optimizer, and the uniformly low scores in Table 5 should be read with this in mind.

EnvironmentARSCEMCMA-ESPES
Pong3.00±0.003.00\pm0.002.28±0.902.28\pm0.902.28±0.892.28\pm0.893.00±0.003.00\pm0.00
InvertedDoublePendulum1949.3±1750.81949.3\pm1750.8189.4±64.4189.4\pm64.4250.1±92.4250.1\pm92.4217.8±18.3217.8\pm18.3
MountainCar (discrete)−168.0±34.1-168.0\pm34.1−199.5±1.4-199.5\pm1.4−200.0±0.0-200.0\pm0.0−176.3±22.0-176.3\pm22.0
Navigation3414.0±68.83414.0\pm68.82746.6±1028.52746.6\pm1028.52522.8±1306.32522.8\pm1306.33111.3±310.53111.3\pm310.5
FrozenLake0.74±0.130.74\pm0.130.56±0.200.56\pm0.200.68±0.130.68\pm0.130.74±0.090.74\pm0.09
Nim†0.44±0.200.44\pm0.200.28±0.470.28\pm0.470.28±0.470.28\pm0.470.44±0.200.44\pm0.20
CliffWalking‡−100.0±0.0-100.0\pm0.0−100.0±0.0-100.0\pm0.0−100.0±0.0-100.0\pm0.0−100.0±0.0-100.0\pm0.0

Table 5. Classical/policy-search baselines on the eight extended environments (mean best reward ±\pm std, n=10n=10 seeds, timestep-matched budget; see text for TT and population size per environment). CliffWalking’s zero-variance result is a diagnosed landscape property, not an error (see text).

CliffWalking. We directly diagnosed the zero-variance result by sampling ten independent random policies at a larger-than-typical weight scale and confirmed genuine, substantial variation in outcome (rewards of either −100-100, a single cliff-fall, or −10,000-10,000, full-length wandering with no resolution, across seeds), ruling out an implementation bug in the policy-to-action mapping. Every classical optimizer we tested nonetheless converges to the −100-100 attractor across all seeds: the two outcomes are far enough apart, and the region between them flat enough, that none of ARS, CEM, CMA-ES, or PES finds a path to the better-wandering or any superior policy from a zero-mean initialization. We report this as a genuine, diagnosed landscape property.

LLAMBO Baseline

We adapt LLAMBO [17], reusing this project’s existing OPRO-style prompt construction and response parsing for candidate generation (ensuring an identical backend, model, and prompt framing to the existing LLM-optimizer baseline) and adding a second, genuinely distinct role not present in the OPRO baseline: before any candidate is evaluated on the real environment, the LLM is shown the full candidate pool and the history and asked to rank candidates by predicted promise, acting as a natural-language surrogate model. We use Qwen2.5-7B via a local Ollama server, matching the backend of the existing LLM-optimizer comparison.

On CartPole, n=3n=3 seeds, 100 iterations (reduced from T=400T=400 given LLM call cost, matching how the existing OPRO baseline is also run at reduced scale for the same reason), with a candidate pool of 5 per round, LLAMBO reaches 190.1±\pm219.4, against 186.0±\pm218.0 for single-shot OPRO prompting under comparable conditions – statistically indistinguishable at this sample size. We note a specific limitation that likely attenuates any advantage the ranking step might otherwise provide: across the three seeds, a substantial fraction of ranking-step LLM responses could not be parsed into a valid ranking and fell back to the pool’s original (unranked) order; lowering the ranking call’s sampling temperature reduced but did not eliminate this. We therefore do not read this comparison as a clean negative result for LLAMBO’s surrogate-ranking mechanism, only as evidence that our current adaptation does not yet realize its intended benefit, and flag a more robust ranking-response parser as necessary before drawing a stronger conclusion.

Pretraining Cost

We benchmark pretraining cost as a function of corpus size on a single CPU core (AMD EPYC, no GPU), covering rollout collection and 50 epochs of training (Table 6).

Corpus size NNCollectPer-epoch50-epoch totalPer-rollout
1,0003.5 s1.2 s1.1 min3.5 ms
5,00018.6 s4.5 s4.1 min3.7 ms
10,00036.6 s8.4 s7.6 min3.7 ms
20,00074.6 s17.4 s15.7 min3.7 ms

Table 6. Pretraining cost by corpus size, CartPole, single CPU core.

On an A100 GPU (estimated 15-fold speedup over the CPU-only figures above), pretraining 5,000 rollouts for 50 epochs takes about 16 s per environment, or under 2 GPU-minutes across all five pretraining environments. Since inference requires no GPU (per-step latency 2.61 ms, Table 9), the total cost of ownership at the main evaluation budget is approximately 2 GPU-minutes of one-time, amortised pretraining cost plus under 2 seconds of inference per run, in contrast to LLM-guided search, which incurs a model call at every proposal.

MethodTime/stepRelative
CEM update (numpy)0.024 ms109-fold faster
CMA-ES update (numpy)0.053 ms49-fold faster
JEPA propose (CPU)2.607 ms—
LLM 7B (Qwen2.5)∼200 ms77-fold slower
LLM 14B∼550 ms211-fold slower

Table 9. Per-step wall-clock time (1,000 repetitions, CPU, excluding the environment rollout).

Complete Baseline Grid, Cost, and Leave-One-Out Status

Table 7 is the main-text headline grid (restricted to five environments); Table 8 is the complete grid behind it, Table 9 gives per-step cost, and Figure 3 the performance comparison.

MethodCartPoleSwimmerHopperInvPend.MtnCar(C)
Random Search55.5±16.9120.5±38.219.2±3.688.5±27.624.9±19.1
Tabu Search145.2±98.4220.1±52.3246.0±122.2272.3±364.817.8±35.6
(μ+λ\mu+\lambda)-ES180.5±112.3215.7±48.9141.7±53.3406.4±327.3−0.0±0.0
OpenAI-ES36.7±28.1198.5±55.3141.5±112.055.7±42.0−20.1±39.9
BayesOpt274.0±154.7182.3±72.157.0±42.393.6−13.6
PSO322.5±96.5263.4±40.0463.3±240.8296.82.3
CEM360.9±95.3250.8±59.0620.2±201.7436.4−5.0
IPOP-CMA-ES500.0±0.0352.2±3.3659.9±361.4890.0±220.0−0.0±0.0
LLM optimizer (OPRO)186.0±218.0140.3±93.31026.7±0.8356.0±455.4−0.1±0.0
LLAMBO (Appendix C)190.1±219.4————
ARS [2]500.0±0.0352.1±2.5789.1±203.250.9±8.5−0.0±0.0
CEM-RL [43]485.6±28.8148.1±81.6424.7±255.9443.1±455.9−0.0±0.0
CERL [44]500.0±0.0227.0±63.3552.4±370.867.7±28.326.1±32.4
PES [37]500.0±0.0303.6±25.5766.0±334.2105.7±27.024.2±18.9
PABBO [25]407.8±112.9279.6±92.6474.9±270.892.5±19.6−0.0±0.0
Swarm-JEPA (K=3K=3)_421.6±156.9_273.8±32.1133.3±67.827.4±28.8−4.9±6.2
Swarm-JEPA (K=8K=8)416.1±28.7293.1±28.7_1039.7±33.5_27.9±29.9−5.1±6.3
Latent-CEM393.7±45.2_304.5±38.5_684.0±340.827.7±28.6−1.2±1.4

Table 8. Full baseline grid: mean best reward (mean ± std, 5 seeds), T=400T=400 total evaluations. Bold: best per environment. <u>Underlined</u>: best JEPA-family result. “—”: not run.

Figure 3. Complete performance comparison across all optimizers (mean best reward ± std, 5 seeds); JEPA-family methods are outlined. Latent-CEM (Appendix P) adds a further gain over raw-space CEM on Swimmer.

Wrapper-Attribution and Final Test-Set Results

This appendix gives the full numbers behind § 9, Parts 5–6.

Validation-stage wrapper sweep and attribution (Part 5). Table 10 reports pooled results (mean over the three validation variants: force×\times1.5, gravity×\times1.5, length×\times0.6; n=30n=30 seeds per variant) for the un-wrapped baseline and the four wrapper configurations that passed our pre-registered attribution rule, alongside the untrained control (S: pretrained encoder/predictor, same decoder; U: reinitialized encoder/predictor, same decoder) and the broader-dynamics-curriculum checkpoint (B, Part 7).

ConfigurationS succ.U succ.B succ.S-vs-UB-vs-S
Baseline (no wrapper)0.380.160.44PASS (+0.22, 3/3)FAIL (+0.06, 0/3)
Longer warm-up (32)0.900.330.89PASS (+0.56, 3/3)FAIL (+0.01, 0/3)
Restart on stagnation (40)0.940.720.98PASS (+0.22, 2/3)FAIL (+0.03, 0/3)
Multi-start (K=8K=8)0.990.800.98PASS (+0.19, 3/3)FAIL (−0.01-0.01, 0/3)
Combined (restart+anneal+warm-up)0.990.831.00FAIL (ceiling eff., +0.16, 1/3)FAIL (+0.01, 0/3)

Table 10. Validation-stage pooled success rate (reward ≥490\ge490) across three variants, n=30n=30 seeds each. S: Part 1 pretrained checkpoint. U: untrained-encoder/predictor control, identical decoder. B: broader-dynamics-curriculum checkpoint (Part 7). PASS/FAIL is the pre-registered attribution rule (pooled gap ≥0.10\ge0.10 and paired significance on ≥2/3\ge2/3 variants).

Test-stage confirmation (Part 6). Using restart-on-stagnation (40), the single configuration offering the best combination of strong pretraining attribution and strong absolute performance, Table 11 reports the final n=100n=100 test-set result on all four held-out variants (the two Part 1 held-outs and the two Part 2 extremes), for both the pretrained (S) and untrained (U) checkpoints, alongside the un-wrapped baseline for reference.

VariantS baselineS restart40U baselineU restart40
Gravity×\times0.5420.9±153.8420.9\pm153.8 (0.74)500.0±0.0500.0\pm0.0 (1.00)263.7±166.5263.7\pm166.5 (0.19)492.0±23.2492.0\pm23.2 (0.87)
Length×$2\times\$2 | 316.5±160.2316.5\pm160.2 (0.27) | 483.5±38.5483.5\pm38.5 (0.81) | 155.9±100.5155.9\pm100.5 (0.03) | 420.5±94.4420.5\pm94.4 (0.50)
EXTREME-A178.9±141.0178.9\pm141.0 (0.07)414.7±114.6414.7\pm114.6 (0.56)101.6±86.0101.6\pm86.0 (0.02)330.6±130.7330.6\pm130.7 (0.28)
EXTREME-B430.5±155.7430.5\pm155.7 (0.81)500.0±0.0500.0\pm0.0 (1.00)308.4±176.4308.4\pm176.4 (0.30)500.0±0.0500.0\pm0.0 (1.00)
Pooled336.7 (0.47)474.5 (0.84)207.4 (0.14)435.8 (0.66)

Table 11. Test-set results, n=100n=100 seeds per variant. Baseline and restart-on-stagnation (40), for both the pretrained checkpoint (S) and the untrained control (U). “succ.” is the fraction of seeds reaching reward ≥490\ge490. The from-scratch ARS/CEM reference (Part 4) reaches 1.00 success on every seed on gravity×\times0.5 and length×\times2; it was not separately re-run on the two extreme variants.

Each of the eight restart-vs-baseline comparisons within a single checkpoint (S or U) is individually significant (p<10−4p<10^{-4} in every case, paired Wilcoxon against that checkpoint’s own baseline on the same variant). We did not additionally compute a formal paired S-vs-U significance test specifically at the test stage; the S-vs-U attribution claim rests on Part 5’s validation-stage test, which was pre-registered and passed for this configuration, and the test-stage pooled success gap (0.84 vs. 0.66) is consistent in direction and approximate magnitude with that validation-stage finding.

JEPA-PS versus CMA-ES: A Six-Part Investigation

Section 9’s pre-registered investigation establishes that JEPA-PS’s pretraining transfers in a representational sense, but that even with its strongest inference-time intervention, it does not reliably match classical search started from scratch. Here we go further and report six additional, independently designed tests against real CMA-ES – never a hand-rolled summary-statistic proxy standing in for it. These six tests separate two distinct questions: a representational question (does JEPA’s latent carry information that CMA-ES’s own internal state does not?) and a performance question (does JEPA-PS ever actually beat CMA-ES at an optimization task, under any budget or condition we tried?). As elsewhere in this paper, we report all six regardless of outcome.

Exp1 — does the latent encode information CMA-ES’s own state does not? We probe whether JEPA’s context latent sxs_x linearly predicts four order-flavored targets (last_reward_quantile, reward_trend_slope, argmax_quartile, improves_next, a forward-looking binary target) better than a probe given CMA-ES’s own full internal state (mean, covariance, σ\sigma, evolution paths) at the matching iteration. On the pretrained (S) checkpoint, sxs_x passes on two of four targets: last_reward_quantile (Rsx2=0.173R^2_{s_x} = 0.173 vs. Rcma2=0.109R^2_{\mathrm{cma}} = 0.109, p=0.000329p = 0.000329) and reward_trend_slope (Rsx2=0.237R^2_{s_x} = 0.237 vs. Rcma2=0.130R^2_{\mathrm{cma}} = 0.130, p=8.17×10−6p = 8.17 \times10^{-6}). The untrained (U) checkpoint also passes on last_reward_quantile, with an even higher Rsx2=0.284R^2_{s_x} = 0.284 than S — consistent with this particular order-information being an architectural property of the context encoder rather than something pretraining specifically adds — but reward_trend_slope is not significant on U (p=0.156p = 0.156), so pretraining does contribute there. Neither checkpoint shows a clear winner on argmax_quartile or improves_next. A methods footnote: the full CMA-ES state probe scored below a simpler Gaussian-summary baseline in this comparison, almost certainly a feature-scaling artifact (the state’s raw covariance entries, σ\sigma, and evolution paths live on very different numeric scales, which a plain linear probe handles poorly), not evidence that the fuller state carries less signal than the summary.

Exp2 — does the model’s search loop causally depend on history order? We ablate order causally rather than correlationally: the pretrained model’s real search loop is run with its visible history shuffled or reversed before each proposal, and we measure the resulting change in search quality (area under the best-so-far curve). The result is a null, in both a nominal and a harder variant: shuffling does not significantly hurt pretrained JEPA’s search (normal: AUCS=316.6±171.1\mathrm{AUC}_S = 316.6 \pm171.1 vs. AUCS,shuffled=289.4±179.0\mathrm{AUC}_{S,\mathrm{shuffled}} = 289.4 \pm179.0, mean drop ≈27\approx27, p=0.21p = 0.21; the untrained checkpoint shows ≈\approxzero drop, as architecturally expected). We report this as an underpowered null at n=20n=20 seeds, not a clean negative — the effect size sits in the plausible direction but the sample cannot distinguish “no real effect” from “a real but small one.”

Exp3/Exp3b — adaptation speed after a mid-search distribution shift. We test whether JEPA-S recovers faster than CMA-ES after a physics shift mid-search (gravity ×3\times3+length ×1.5\times1.5 at step 200 for Exp3; a force-direction reversal at step 400 for Exp3b, designed to guarantee the shift “bites”). CMA-ES decisively wins both times. Exp3: JEPA-S vs. CMA-ES post-shift AUC difference = −334.2-334.2 (p=0.0001318p = 0.0001318); vs. restart-CMA-ES, −328.3-328.3 (p=3.815×10−6p = 3.815 \times10^{-6}). JEPA-S does beat JEPA-U (post-shift AUC difference +108.1+108.1, p=6.294×10−5p = 6.294 \times10^{-5}, confirming pretraining helps *some), but neither comes close to CMA-ES: post-shift AUC 493.7±10.7493.7\pm10.7 for CMA-ES against 159.4±122.0159.4\pm122.0 for JEPA-S, with a recovery rate of 1.00 within 4.3 evaluations for CMA-ES against 0.20 within 161 evaluations for JEPA-S. Exp3b reproduces the pattern under a different, guaranteed-to-bite shift: JEPA-S vs. CMA-ES post-shift AUC difference =−160.6=-160.6 (p=0.0003223p=0.0003223).

Exp4 — does pretraining amortize across repeated re-optimization? Across five CartPole physics variants, we measure evaluations-to-threshold (first reaching reward ≥475\ge475) for JEPA-S versus CMA-ES from scratch, the direct test of whether a pretrained optimizer’s upfront cost pays for itself across repeated tasks. Amortization is negative: mean saving per task is −299.46-299.46 evaluations — JEPA-S needs more evaluations per task than CMA-ES from scratch, on every single variant tested (JEPA-S success rate 0.0–0.4 vs. CMA-ES’s 1.00; CMA-ES typically converges in 10–40 evaluations against JEPA-S’s near-total budget 362–401 mean evaluations-to-threshold).

Exp5 — few-shot quality: does pretraining beat a cold start at tiny budgets? We test the narrower hypothesis that JEPA’s pretrained prior gives a better zero/few-shot proposal than CMA-ES’s uninformed cold start, sweeping budgets K∈{1,3,5,10,15,20,30,50}K\in\{1,3,5,10,15,20,30,50\} across four CartPole variants (nominal, gravity×\times0.5, length×\times2, gravity×\times3+length×\times1.5), n=20n=20 seeds. The hypothesis is falsified: CMA-ES wins or ties at every single KK across all four variants, with no crossover anywhere in the tested range (e.g. nominal, K=1K=1: JEPA-S =37.4±47.5=37.4\pm47.5 vs. CMA-ES =47.4±42.7=47.4\pm42.7, difference −10.0-10.0, p=0.17p=0.17; by K=50K=50: JEPA-S =306.2±180.4=306.2\pm180.4 vs. CMA-ES =493.8±17.7=493.8\pm17.7, difference −187.6-187.6, p=0.0008p=0.0008, i.e. the gap widens with budget). A methodological caveat: K∈{1,3,5}K\in\{1,3,5\} falls inside JEPA’s warm-up window, where both JEPA-S and JEPA-U fall back to unconditioned random sampling, so this range is not yet testing the pretrained prior at all. This caveat does not rescue the hypothesis, however — past warm-up, at the budgets where JEPA’s decoder-conditioned proposal is genuinely in play (K=10K=10–50), CMA-ES still wins or ties everywhere, and the gap grows rather than shrinks.

Held-out task family, with a warm-started control. Finally, we close a specific gap in the comparisons above: none of them pit JEPA-S against a CMA-ES baseline that gets an equivalently amortized head start. We evaluate on four variants held out entirely from pretraining (gravity×\times1.5+length×\times1.3, masspole×\times2, masscart×\times0.5+force×\times1.5, length×\times0.5+force×\times2), n=20n=20 seeds, budgets K∈{10,25,50,100,250}K\in\{10,25,50,100,250\}, against two CMA-ES controls: CMA-ES-cold (zero-mean start, as in every comparison above) and CMA-ES-warm (mean seeded from a one-time 200-evaluation solve on the nominal variant, its cost reported once and amortized across all four held-out tasks rather than charged per-task — the same accounting logic as pretraining cost). Table 12 reports the full grid. JEPA-S beats both controls in 0 of 20 variant×\timesbudget cells. CMA-ES-cold alone already saturates at 500.0±0.0500.0\pm0.0 on most variants by K=50K=50–100 while JEPA-S remains at 236–328; CMA-ES-warm reaches 500.0 almost immediately, by K=25K=25 on every variant.

VariantKKJEPA-SJEPA-UCMA-ES-coldCMA-ES-warm
gravity×1.5+\times1.5+length×1.3\times1.310190.3±159.7190.3 \pm159.7139.9±135.7139.9 \pm135.7187.0±161.4187.0 \pm161.4481.5±50.4481.5 \pm50.4
gravity×1.5+\times1.5+length×1.3\times1.325212.6±161.7212.6 \pm161.7140.5±135.3140.5 \pm135.3313.3±138.0313.3 \pm138.0500.0±0.0500.0 \pm0.0
gravity×1.5+\times1.5+length×1.3\times1.350236.3±173.3236.3 \pm173.3140.6±135.2140.6 \pm135.2466.4±71.3466.4 \pm71.3500.0±0.0500.0 \pm0.0
gravity×1.5+\times1.5+length×1.3\times1.3100245.2±173.9245.2 \pm173.9141.1±134.8141.1 \pm134.8500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
gravity×1.5+\times1.5+length×1.3\times1.3250260.1±177.0260.1 \pm177.0142.7±133.8142.7 \pm133.8500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
masspole×2\times210227.5±176.7227.5 \pm176.7171.8±151.4171.8 \pm151.4288.0±178.3288.0 \pm178.3500.0±0.0500.0 \pm0.0
masspole×2\times225260.3±174.4260.3 \pm174.4173.2±150.4173.2 \pm150.4444.6±95.8444.6 \pm95.8500.0±0.0500.0 \pm0.0
masspole×2\times250267.5±173.7267.5 \pm173.7173.7±150.0173.7 \pm150.0497.9±6.4497.9 \pm6.4500.0±0.0500.0 \pm0.0
masspole×2\times2100270.8±171.6270.8 \pm171.6174.4±149.6174.4 \pm149.6500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
masspole×2\times2250281.0±174.2281.0 \pm174.2175.8±148.3175.8 \pm148.3500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
masscart×0.5+\times0.5+force×1.5\times1.510301.0±188.9301.0 \pm188.9—352.6±171.7352.6 \pm171.7500.0±0.0500.0 \pm0.0
masscart×0.5+\times0.5+force×1.5\times1.525308.9±185.0308.9 \pm185.0—457.4±97.7457.4 \pm97.7500.0±0.0500.0 \pm0.0
masscart×0.5+\times0.5+force×1.5\times1.550315.8±184.1315.8 \pm184.1—500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
masscart×0.5+\times0.5+force×1.5\times1.5100322.8±181.5322.8 \pm181.5—500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
masscart×0.5+\times0.5+force×1.5\times1.5250328.1±181.7328.1 \pm181.7—500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
length×0.5+\times0.5+force×2\times210306.5±195.1306.5 \pm195.1—338.1±181.2338.1 \pm181.2498.3±7.2498.3 \pm7.2
length×0.5+\times0.5+force×2\times225309.1±192.6309.1 \pm192.6—430.3±115.5430.3 \pm115.5500.0±0.0500.0 \pm0.0
length×0.5+\times0.5+force×2\times250312.2±189.2312.2 \pm189.2—500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
length×0.5+\times0.5+force×2\times2100316.0±186.8316.0 \pm186.8—500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0
length×0.5+\times0.5+force×2\times2250317.6±184.7317.6 \pm184.7—500.0±0.0500.0 \pm0.0500.0±0.0500.0 \pm0.0

Table 12. JEPA-S vs. CMA-ES-cold and CMA-ES-warm on four held-out CartPole physics variants, never seen during pretraining (n=20n=20 seeds). CMA-ES-warm’s mean is seeded from a one-time 200-evaluation solve on the nominal variant, amortized across all four tasks. JEPA-S beats both controls in 0 of 20 cells.

JEPA-S does reliably beat JEPA-U (e.g. gravity×1.5+\times1.5+length×1.3\times1.3 at K=250K=250: 260.1 vs. 142.7, p<0.01p < 0.01 throughout), confirming pretraining again captures something relative to random initialization — but that something is far short of matching CMA-ES’s online adaptation, warm-started or not.

Synthesis. Across these six parts, a single, consistent body of evidence emerges. JEPA-PS’s latent does carry a genuine, reproducible representational signal: Exp1 shows that sxs_x linearly predicts two of four order-flavored targets better than CMA-ES’s own full internal state does, a real result that survives its own methods caveat. But that signal never translates into a performance advantage. Across every performance-oriented test we ran – stationary search at any budget from 1 to 400 evaluations (Exp5, held-out family), adaptation after a mid-search distribution shift (Exp3/Exp3b), amortized re-optimization across repeated tasks (Exp4), and a properly warm-started transfer control on a genuinely held-out task family (the held-out-family table above) – JEPA-PS shows no evidence of an optimization-performance or transfer advantage over correctly run CMA-ES, whether cold-started or warm-started. This is the same conclusion § 9 reaches from its own pre-registered investigation, arrived at here independently, across six further and differently designed tests. The paper’s defensible claim is therefore the representational one, not an optimization-superiority one, and we report it that way throughout.

Strong Baselines: Grid and Tuning Protocol

We tune each strong baseline within standard ranges. TuRBO uses an initial trust-region length of 0.8 and success/failure tolerances (3, max⁡(4,d/2)\max(4, d/2)) [45]. SMAC uses a random-forest surrogate with expected-improvement acquisition [46]. TPE uses the usual default budgets [47]. IPOP- and BIPOP-CMA-ES use population-doubling restarts with σ0=0.5\sigma_0=0.5 and initial population λ=4+⌈3ln⁡d⌉\lambda= 4 + \lceil3 \ln d \rceil [38, 48]. L-BFGS-B uses a finite-difference step of 0.05 with five random restarts. All methods receive the same fixed total budget and episode count. IPOP-CMA-ES’s advantage on Swimmer is expected: its restart schedule is designed for multimodal landscapes [38], consistent with benchmarks showing CMA-ES variants dominating at these budgets [6]. L-BFGS-B attains only 119.5±\pm101.5 on CartPole, so finite-difference gradients struggle on this stochastic objective even though the landscape is globally smooth. Figure 4 summarizes per-step cost and model footprint alongside this grid.

Per-step cost and model footprint

Figure 4. Per-step cost and model footprint. (A) Wall-clock time per proposal on a log scale. A JEPA forward pass is 77 times faster than a 7B-parameter LLM call, and 49–109 times slower than a classical covariance or elite-mean update. (B) Model size against multi-billion-parameter language models.

Landscape Descriptors, FDC Estimator, and Correlations

Additional landscape descriptors. Beyond FDC, Table 17 reports six descriptors, each estimated from the same 150–300-sample random-probe budget together with n=10n=10 random walks of length 50 per environment:

EnvironmentDomainFDC ↑\uparrowACL↑\uparrowSmooth↓\downarrowBasin↑\uparrowHDOrdSensNRICDM
CartPolebalance0.66189.840.60.2180.3700.5802.2571.327
InvPendulumbalance0.548414.400.00.0500.0000.6921.5170.864
Swimmerlocomotion0.482312.840.370.0440.0680.4242.7701.220
Hopperlocomotion0.439532.570.0−0.0060.0000.9820.1822.695
CliffWalkingdiscrete0.42421.440.0—————
Reachercontrol0.401513.760.00.0740.0030.2202.9411.808
InvDblPendulumbalance0.382345.370.0——0.4722.8581.588
Walker2dlocomotion0.326347.140.00.0000.0000.2542.8994.566
FrozenLakediscrete0.3131—0.0——0.9760.1772.038
Nimcustom0.24810.150.0——0.7181.7943.999
MountainCar(C)control0.153124.510.10.004−0.0000.4762.1300.734
Navigationcustom0.1301104.850.1——0.1302.7831.510
BallLandingcustom0.03749.630.0——0.0462.3541.159
Mazecustom−0.09510.001.000†——0.9740.2842.038
MountainCar(D)discrete—1—1.000†——0.9960.0431.312

Table 17. Landscape descriptors for the 15 originally-profiled environments, sorted by FDC. †: basin values of 1.0 are threshold artifacts. OrdSens is — for environments without a JEPA checkpoint. NR, IC, and DM are computed from random walks. The eight extended environments of Appendix B have not yet been profiled on these descriptors. Figure 5 visualizes this ranking.

  • Autocorrelation length (ACL) is the lag ℓ\ell at which the autocorrelation of reward along a random walk first drops below 1/e1/e [32].

  • Basin is the fraction of random-search samples whose reward exceeds 90% of the best reward found during probing, a coarse proxy for the size of the near-optimal attraction region.

  • Smooth is the mean absolute reward change per unit parameter-space distance along a random walk (lower is smoother).

  • Neutrality rate (NR) is the fraction of random-walk steps for which the reward change is smaller than 1% of the observed reward range [49].

  • Information content (IC) is the Shannon entropy of the sequence of {−1,0,+1}\{-1, 0, +1\} reward-change signs along a random walk [49].

  • Dispersion (DM) is the standard deviation of pairwise parameter-space distances among the top-decile of random-search samples by reward.

FDC protocol and sensitivity. For each environment we sample NN parameter vectors θi∼N(0,Id)\theta_i \sim\mathcal{N}(0, I_d), evaluate each over nep=5n_{\mathrm{ep}}=5 episodes, and take the proxy optimum as θ∗=arg⁡max⁡iJ(θi)\theta^* = \arg\max_i J(\theta_i); FDC is then corr⁡(J(θi),−∥θi−θ∗∥)\operatorname{corr}(J(\theta_i), -\lVert\theta_i-\theta^*\rVert) over all NN pairs. The canonical per-environment values in Table 17 use the full 250-epoch protocol of Appendix § 12; the sensitivity analysis uses a smaller, independently repeated resampling protocol. Table 16 (left) sweeps NN from 50 to 1,000 with 10 repetitions per value on CartPole. Every confidence interval excludes zero, so CartPole’s FDC is reliably positive at every sample size. The point estimate declines as NN grows because larger samples uncover a better proxy optimum, which sits further from the average random sample; this means any future use of FDC as a decision threshold would need to be defined at a fixed probe size rather than treated as a single stable number. The ordering across environments on which our analysis depends is preserved at every sample size tested. The seven-environment suite adds Reacher and Walker2d to the original six; per-environment FDC values match Table 17.

NNFDC meanFDC std95% CICI excl. 0?QuantityValuenn
500.5040.135[0.242, 0.659]yesrr(HD, FDC), uncond.0.7387
1000.4310.093[0.266, 0.562]yespartial | ACL−0.4877
2000.3810.084[0.274, 0.552]yespartial | Basin0.5517
3000.3630.082[0.260, 0.533]yesrr(OrdSens, FDC)0.8325
5000.3340.093[0.245, 0.519]yes
10000.2990.084[0.204, 0.461]yes

Table 16. FDC estimator robustness on CartPole across sample sizes (10 repetitions per NN, 95% bootstrap CI), and the partial-correlation summary (right).

FDC under controlled ruggedness. We validate FDC on synthetic NK landscapes [50], where ruggedness is set by a tunable epistasis parameter KK (Table 14); FDC decreases monotonically as KK increases. This validates FDC as a ruggedness descriptor on a landscape family with known ground truth, independent of any control-task confound, and these same KK levels are reused as the ruggedness curriculum in § 9, Part 7.

KKFDCClassification
00.389smooth (additive)
20.236mildly rugged
40.113intermediate
80.036rugged
120.028maximally rugged

Table 14. FDC on NK landscapes across ruggedness levels KK (network size 16, 5 instances per KK).

Interpretation of the partial correlations. The sign reversal under partial correlation (from 0.738 unconditional to −0.487 conditioning on ACL) most plausibly reflects collinearity: FDC and ACL are themselves correlated across our seven environments, since both increase with landscape smoothness, so with seven points conditioning on one absorbs most of the shared variance and leaves an unstable, sign-flipped residual for the other. We treat the partial correlations as evidence that FDC and ACL are not separable at this sample size, read the unconditional correlation as a preliminary association with landscape smoothness broadly construed, and draw no conclusion from the sign of the partial effects. We report Pearson correlation for comparability with the bootstrap CI; the Spearman correlation on the same seven environments is ρ=0.571\rho=0.571, below the two-tailed significance threshold at n=7n=7.

History, Order, and Reward-Transformation Ablations

The history, order, reward-transformation, and inference-time perturbation ablations are consolidated in Table 18. Read the inference-time perturbations against the pretraining-time shuffle of Table 19 (Appendix § 22), where the same manipulation has no effect: the two act at different stages of the pipeline. The best-only condition appears with slightly different values in three places (321.4 in Table 15, 325.5 in the memory-condition block, 319.2 in the inference-time block), and the same reward-shuffle intervention costs 17.0% in the inference-time block but the reward-permutation intervention costs 37% in the reward-transformation block; these differences track the two different protocols reported alongside each table (the main T=400T=400 protocol versus the reduced-budget, 200-iteration protocol used for the sweeps in this appendix) rather than a single consistent number, and we report each figure with the protocol it was measured under rather than reconciling them to one value.

Condition2k/50ep20k/250ep
Original (control)214.03207.30
Rewards shuffled236.30208.07
Params shuffled210.27208.00
Both shuffled219.13207.63

Table 19. Pretraining reward/parameter shuffle (CartPole, 5 seeds).

Landscape ruggedness across environments

Figure 5. Landscape ruggedness across the 15 originally-profiled environments (green: smoother; red: more rugged). CartPole is the most exploitable landscape; Maze, Navigation, and MountainCar the least.

Reward-Agnostic Pretraining and the Target Return

Table 19 shows that shuffling rewards, parameters, or both during pretraining leaves downstream performance unchanged, consistent with the latent-prediction objective of Eq. (2) carrying no reward-prediction term.

Table 20 isolates the two reward channels at test time and shows that the target-return percentile ρ\rho has no measurable effect over the range 50 to 99 at either of two budgets. Table 21 repeats the corruption on Swimmer and quantifies the shift in the proposal distribution. The identical means and standard deviations across conditions in Table 20 (410.9, 178.2) coincide with the single-agent baseline to three significant figures, which is consistent with r∗r^{*} genuinely not affecting the proposal on this landscape, but is also the pattern we would see if the reported best-over-history outcome were unaffected because it is already fixed during the random warm-up phase; the two explanations are not distinguished by this table alone.

Decoder Gradient Flow, Null Model, and Random Decoders

The decoder receives exactly zero gradient, since LJEPA\mathcal{L}_{\mathrm{JEPA}} compares two latent vectors and never routes gradient through hξh_{\xi}, which is invoked only inside the no-gradient path used at inference (Table 22, bottom). We freeze the encoder and predictor and fine-tune hξh_{\xi} for 50 epochs with Lrecon=∥hξ(s^y)−θt+1∥22\mathcal{L}_{\mathrm{recon}} = \lVert h_{\xi}(\hat{s}_{y})-\theta_{t+1}\rVert_{2}^{2}; the reconstruction loss decreases as expected, but downstream performance falls from 410.9 to 293.7. Both trained variants underperform the frozen decoder by a substantial margin, which is inconsistent with a mean-plus-noise account of JEPA-PS (a decoder trained to approximate the relevant statistic should do at least as well as an untrained one) and consistent with the encoder and predictor arranging a latent geometry in which a fixed random projection already lands near useful parameters.

Null model: two protocols, two directions of result. Table 23 compares JEPA-PS with a proposer sampling N(μh,σh)\mathcal{N}(\mu_h,\sigma_h) from the history statistics alone under our main evaluation protocol, where JEPA-PS leads on both environments. We separately ran a matched-protocol head-to-head on Swimmer, with identical warm-starts and an identical proposal-noise seed shared across both conditions (n=10n=10): under this stricter design, the null proposer reached 170.5±\pm77.3 against 133.3±\pm73.3 for JEPA-PS, reversing the direction of the comparison. A constant-latent control (the same frozen decoder fed a single fixed input regardless of history) scored 137.8±\pm70.1, statistically indistinguishable from JEPA-PS. We report both results rather than retain only the first: under our main protocol the latent appears to add value beyond history statistics; under the more tightly matched protocol it does not, on Swimmer specifically.

Table 23. JEPA-PS vs. a null statistics-only proposer under two protocols. Top: main protocol (n=10n=10 seeds, T=400T=400). Bottom: matched-protocol head-to-head with shared warm-starts and proposal-noise seed (n=10n=10, Swimmer only), which reverses the direction of the comparison.

Method (main protocol)CartPoleSwimmer
Null N(μh,σh)\mathcal{N}(\mu_h,\sigma_h)302.1151.9
JEPA-PS410.9293.1
Method (matched protocol, Swimmer only)Mean ±\pm std
Null N(μh,0.3σh)\mathcal{N}(\mu_h,0.3\sigma_h)170.5±77.3170.5\pm77.3
Constant-latent control137.8±70.1137.8\pm70.1
JEPA-PS133.3±73.3133.3\pm73.3

Ten independent random decoders. We draw ten independent random linear decoders and run Latent-CEM. Nine of the ten draws give 410.9±\pm178.2, identical to the default decoder to three significant figures; one draw reaches the environment ceiling. Across all ten the mean is 419.8 with a between-decoder standard deviation of 26.7, small relative to the within-decoder seed variance of 178.2, so performance under this ablation is governed by the encoder–predictor latent geometry rather than by the specific decoder draw; we note this finding concerns the decoder’s identity specifically and does not resolve the matched-protocol null-model reversal above, which concerns the latent’s content.

Population Wrapper Ablations

Figure 6 ablates the three swarm terms, sweeps the island count, and sweeps the exploration noise; Table 24 lists Swarm-JEPA’s sensitivity to migration frequency and exploration noise, and compares fixed-total against fixed-per-island budget accounting. On the easy task the social term alone suffices; on the harder task all three contribute. Island count improves the mean and reduces variance with diminishing returns, and σn=0.3\sigma_n = 0.3 is best on both tasks. Separately, we found that this migration mechanism does not supply genuine cross-island diversity at any population size we swept (Appendix T), which bears on InvertedPendulum’s failure specifically. Note that this population wrapper is architecturally distinct from the restart-on-stagnation wrapper of Section 9: Swarm-JEPA runs KK agents in parallel with periodic migration, while restart-on-stagnation reinitializes a single agent’s history after a fixed number of non-improving steps; the two are not interchangeable and we did not test Swarm-JEPA under the Section 9 attribution protocol.

Figure 6. Population-wrapper ablations. (A) Swarm-term ablation on CartPole and Swimmer. (B) Island-count sweep on Swimmer: the mean rises and variance falls, then widens, with diminishing returns. (C) Exploration-noise sweep, with σn=0.3\sigma_n=0.3 best on both tasks. (D) Candidate-selection rule and Latent-CEM comparison on CartPole.

Table 24. Swarm-JEPA budget accounting and hyperparameter sensitivity (CartPole). Top: fixed-total vs. fixed-per-island budget accounting (n=5n=5 seeds). Bottom: sensitivity to migration frequency and exploration noise (K=3K=3, n=3n=3 seeds, 400 iterations).

KKModeIters/islandTotal evalsMean bestStd
1either400400410.9178.2
3fixed-total133400421.6156.9
3fixed-per-island4001200421.6156.9
8fixed-total50400416.128.7
8fixed-per-island4003200500.00.0
ConfigurationMean bestStd
Migration frequency
Every 10 steps413.1123.6
Every 20 steps417.8150.3
Every 30 steps (default)421.6156.9
Every 60 steps421.6156.9
No migration421.6156.9
Single agent (K=1K=1)410.9178.2
Exploration noise σn\sigma_n
0.1423.2163.7
0.3 (default)421.6156.9
0.5420.8139.3
1.0399.1152.0

Budget accounting. At K=3K=3 the fixed-total and fixed-per-island schemes give identical results (421.6), so the gain of K=3K=3 over K=1K=1 (410.9) is attributable to population structure rather than to additional evaluations. At K=8K=8, fixed-total accounting reproduces the headline result of Table 7 (416.1), while fixed-per-island accounting, which grants eight times the actual evaluation budget, reaches the environment ceiling; the gap is therefore attributable to the extra budget itself.

Robustness and Sample Efficiency

We stress-test Swarm-JEPA under restricted budgets, larger populations, reduced pretraining data, and corrupted reward signals. Unless stated otherwise, experiments use CartPole; Swimmer and Hopper replication follows (Tables 26–27).

A note on CartPole variance. CartPole’s mean best reward varies across tables (roughly 337, 410.9, 416.1, 421.6, 459.0, and 480–492). Budget and protocol differences account for the largest swings: 337 under a reduced budget (200 iterations, 5 episodes per evaluation, Table 18), 410.9 (single-agent) under the main T=400T=400 protocol, and 480–492 in the 10-seed, high-trajectory-count data-efficiency experiment (Table 25). Under the identical T=400T=400 protocol, 410.9 is single-agent JEPA-PS, 421.6 is Swarm-JEPA (K=3K=3), and 416.1 is Swarm-JEPA (K=8K=8); these are different population sizes sharing a fixed total budget. The environment’s difficulty is bimodal, with most seeds reaching the ceiling of 500 and a minority failing to, so standard deviations of 150–200 are a property of the task rather than of any one method. We separately found, using an exact distribution-free test based on record statistics, that on several environments the probability a run’s reported best is determined entirely by its random warm-up phase (i.e. the optimizer never subsequently improves on it) is far higher than chance would predict if post-warmup proposals were simply as good as continued random search (p<10−4p < 10^{-4} on every environment tested); this does not change the numbers reported here but means comparisons on best-over-history specifically should be interpreted with this floor in mind.

Budget scaling. Swarm-JEPA rises from 283.8 at ten evaluations to 404.9 at fifty and saturates near 410.9 from one hundred onward. CEM matches JEPA’s 100-evaluation performance at that budget and reaches 477.1 by the full budget: JEPA-PS converges faster early, but CEM catches up sooner and plateaus higher on this landscape. We additionally tested, on a timestep-matched locomotion suite (HalfCheetah, Swimmer, Hopper, Ant, Walker2d), whether JEPA-PS’s pretrained prior confers a low-budget sample-efficiency advantage over ARS, CEM, and a CMA-ES-like baseline; it does not, trailing all three from the smallest budget tested onward, so we do not claim a sample-efficiency advantage.

Population size. On Swimmer under a fixed total budget, performance increases monotonically from 263.4 at one island to 305.2 at 32 islands. The standard deviation contracts from 28.71 at eight islands to 19.30 at sixteen and widens again to 34.60 at thirty-two. Figure 7 visualizes the resulting Swimmer and Hopper policies qualitatively.

Policy visualization on Swimmer and Hopper

Figure 7. Policy visualization on Swimmer and Hopper. Weight matrices (left) evolve from unstructured initialization to action-specific structure; rendered frames (right) show behavior across an episode. A random Swimmer policy flails without net forward motion while an optimized policy produces coordinated undulation; a random Hopper policy collapses immediately while an optimized policy sustains hopping.

Data efficiency. Every checkpoint trained on 300 or more trajectories reaches at least 481, approaching the full 20,000-trajectory model (492.3); even 100 trajectories yield 480.2. This near-saturation reflects CartPole’s smoothness; data efficiency on a rougher landscape remains open.

Noisy, sparse, and delayed rewards. On CartPole, degradation under Gaussian reward noise is non-monotone (427.2 at 50% noise vs. 270.0 at 5%), within one standard deviation and not significant, so we do not interpret the ordering; the robust observation is the absence of monotone collapse. Feedback only every kk evaluations costs 35% between k=1k=1 and k=5k=5 and then plateaus. A short delay costs 21%, while longer delays recover and exceed the undelayed baseline (487.8 at 50 steps). The replication on Swimmer and Hopper is less benign: on Hopper, 10% noise, 50% noise, and sparse feedback all yield 127.6±69.9, close to the single-agent result (133–136), and on Swimmer 10% noise costs about half the baseline. Robustness to corrupted rewards is therefore environment-dependent.

Non-stationary objectives. When the reward transformation is switched partway through a run (sign flip, rescaling, or additive offset), performance collapses to the random-policy floor: the history encodes the original objective. Detecting distribution shift and warm-restarting is a natural extension; Appendix G’s Exp. 3/3b show that, where CMA-ES can adapt online to such a shift, JEPA-PS’s recovery is far slower and less complete.

LLM Optimizer Comparison

The LLM optimizer solves the easy CartPole seed under every prompt configuration and separates only on the harder seeds, where behavioral hints help (Table 28; summarized visually in Figure 8). On MountainCar every configuration remains near the random floor regardless of prompt content or history length (Table 29), consistent with that task’s low FDC. On Swimmer it trails the population wrapper at 77 times higher per-step cost. On Hopper, the two are comparable (Table 7). § 6 gives the main-text treatment of this comparison, including two further, independently run LLM-agent baselines and a restart-wrapper control.

LLM optimizer comparison

Figure 8. (A) CartPole across three seeds: hints help on the hard seed and not on the easy one. (B) MountainCar: all prompt variants remain near the random floor. (C) Swimmer: the LLM optimizer trails the population wrapper. (D) History-length sweep on MountainCar.

CartPole (best reward, 3 seeds)
ModeSeed 0Seed 1Seed 2
Numbers only5002336
+ task description5003054
+ behavioral hints5002969
Swarm-JEPA (K=8K=8)500500248
Swimmer (3 seeds)
MethodS0S1S2Mean±\pmStd
LLM (Qwen2.5-7B)240.1124.756.0140.3±\pm93.3
Swarm-JEPA (K=8K=8)———293.1±\pm28.7

Table 28. LLM optimizer vs. Swarm-JEPA on CartPole and Swimmer.

By seed
ModeS0S1S2Mean
Numbers only−0.21-0.21−0.02-0.02−0.00-0.00−0.08-0.08
+ description−0.10-0.10−0.22-0.22−0.33-0.33−0.22-0.22
+ hints−0.04-0.04−0.04-0.04−0.02-0.02−0.03-0.03
Swarm-JEPA (K=3K=3)−1.5-1.5−6.2-6.2−6.9-6.9−4.9-4.9
By prompt variant (seed 0)
ModeOrig.Var. 1Var. 2Var. 3
Numbers only−0.09-0.09−0.15-0.15−0.23-0.23−0.40-0.40
+ description−0.40-0.40−0.07-0.07−0.16-0.16−0.40-0.40
By history length (seed 0)
Max historyL=1L=1L=20L=20L=40L=40L=80L=80
Best reward−0.28-0.28−0.03-0.03−0.40-0.40−0.25-0.25
Mean reward−41.12-41.12−1.12-1.12−3.30-3.30−4.14-4.14

Table 29. MountainCar: LLM optimizer robustness across seeds, prompt variants, and history length. All configurations remain near the random floor (FDC=0.153).

Latent-Space Search and Candidate Selection

Running CEM’s elite-resampling update inside the learned latent space rather than in raw parameter space improves mean best reward by 9.1% on CartPole and 41.6% on Swimmer (Table 30), indicating that the latent space is a more tractable search manifold than the raw parameters. Replacing cosine-similarity candidate selection with best-of-nn or random selection changes nothing on CartPole, so the explicit selection rule contributes little beyond the decoder’s proposal.

MethodCartPoleSwimmer
CEM (raw parameter space)360.86±95.3360.86 \pm95.3215.00±42.1215.00 \pm42.1
Latent-CEM (JEPA latent)393.71±45.2393.71 \pm45.2304.48±38.5304.48 \pm38.5

Table 30. Latent-space CEM vs. raw-space CEM (mean best reward, 3 seeds).

Standardized Cross-Method Comparison

To provide a single point of comparison unaffected by the protocol differences discussed in Appendix § 25, Table 31 fixes T=400T=400 evaluations, n=10n=10 seeds, and 10 episodes per evaluation for every method on CartPole. CEM, CMA-ES, and PSO reuse the tuned n=10n=10 figures from Table 13; Random Search has no separate tuned variant, so its untuned n=5n=5 result from Table 7 is reported. The median of 500.0 against a JEPA mean of 410.9 reflects the bimodal difficulty of the environment: most seeds solve it outright and a minority do not.

MethodMeanStdMedian
Swarm-JEPA (single agent)410.9178.2500.0
CEM484.231.7500.0
CMA-ES500.00.0500.0
PSO500.00.0500.0
Random500.00.0500.0

Table 31. Standardized CartPole comparison: T=400T=400, n=10n=10 seeds, 10 episodes per evaluation.

Additional Mechanistic Observations

For completeness we record secondary observations that are consistent with, but not central to, the paper’s claims. Proposal directions are close to orthogonal to finite-difference gradient ascent, CEM’s elite-mean update, PSO, expected improvement, upper confidence bound, the natural gradient, and a random direction (Figure 9; all cosine similarities lie between 0.003 and 0.015). The exploration radius contracts over the course of a run with no programmed schedule (correlation of −0.590-0.590 with iteration count). The latent trajectory concentrates on a near-one-dimensional manifold and separates into clusters aligned with the early, middle, and late phases of optimization. None of these is load-bearing for § 8. A separate analysis searching for signatures of chaotic dynamics in optimizer trajectories on NK landscapes (largest Lyapunov exponent, FFT power spectra) found no relationship supporting that explanation for the landscape-dependent effects reported in this paper and is not pursued further.

Figure 9. Secondary mechanistic observations. (A) Cosine similarity between JEPA proposals and classical update directions, all near zero. (B) Exploration radius contracting over a run. (C) History-length robustness. (D) Leave-one-out transfer.

A Non-Control Black-Box Setting: BBOB

To probe generality beyond control tasks, we run the JEPA optimizer unmodified, without additional pretraining, on five standard BBOB synthetic functions (d=8d=8) against CEM (Table 32). The optimizer is competitive only on the unimodal Sphere function and is clearly outperformed by CEM on Ellipsoid, Rosenbrock, and Rastrigin; the Schwefel result is unreliable owing to a domain mismatch between the function’s native range and the scale seen in pretraining. BBOB functions have structural properties (Rosenbrock’s curved narrow valley, Rastrigin’s regular periodic multimodality) quite unlike the funnel-shaped control landscapes used for pretraining. This establishes a boundary condition: the model as pretrained here is a control-task optimizer, not a general-purpose black-box optimizer.

FunctionFDCJEPAJEPA stdJEPA (BBOB-pretrained)CEM meanCEM std
Sphere (f1)0.861−2.12-2.120.91−1.19-1.19−1.32-1.321.01
Ellipsoid (f2)0.250−119190-11919031871−60630-60630−3415-34152340
Rosenbrock (f8)0.649−524.3-524.3112.1−159.1-159.1−152.2-152.2110.2
Rastrigin (f15)0.215−53.0-53.010.7−53.0-53.0−16.2-16.25.7
Schwefel (f20)0.109−10.2-10.25.3−1959.4-1959.42057821

Table 32. JEPA optimizer vs. CEM on BBOB synthetic functions (d=8d=8, n=5n=5 seeds, 400 evaluations; higher is better as scored), with FDC measured directly on each function (300 random samples) and results for a checkpoint pretrained directly on BBOB rollouts (all 5 functions mixed, no control-task data).

Failure severity tracks landscape smoothness, and pretraining on BBOB trajectories recovers most of the gap. Table 32 also reports FDC measured directly on each BBOB function: Sphere is by far the smoothest, followed by Rosenbrock, with Ellipsoid, Rastrigin, and Schwefel all rugged. This ordering tracks the severity of JEPA-PS’s failure: the optimizer is competitive only on Sphere, the one function with high FDC. We then pretrain a checkpoint directly on BBOB rollouts (no control-task data), sampled across all five functions, and re-evaluate. Measured as a reduction in the deficit to CEM, the BBOB-pretrained model closes most of the gap on Ellipsoid (deficit from 115,775 to 57,215, a 51% reduction) and on Rosenbrock (from 372.1 to 6.9, a 98% reduction); Rastrigin is essentially unchanged and Schwefel worsens numerically, consistent with Schwefel’s low FDC making it hard for any fixed-budget in-context method. The transfer failure thus reflects a mismatch between pretraining distribution and test landscape rather than an architectural ceiling – and is itself a further instance of the same pattern as § 9: pretrained structure transfers within a family of related landscapes and does not transfer across structurally different ones without re-exposure. § 9, Part 7 extends this specific finding: a curriculum deliberately broadened to include NK landscapes spanning smooth to rugged, not just BBOB functions, still fails to transfer to Rastrigin (−49.5±8.1-49.5 \pm8.1, statistically indistinguishable from both numbers in this table), so the failure is not simply fixed by adding more landscape diversity of the kind we tried.

InvertedPendulum Diagnostic Battery

We applied the diagnostic battery used to investigate failure modes on other environments to the InvertedPendulum checkpoint, motivated by the gap discussed in § 5. Across 20 seeds, JEPA-PS’s proposals moved consistently toward a good-solution region estimated from independent random probing (mean distance to that region: 1.81 postwarmup vs. 2.65 during warmup), rather than away from it as in the divergence pathology characterized for other environments. The context encoder’s representation diversity across 40 distinct histories and the decoder’s weight-matrix conditioning were both unremarkable relative to other checkpoints. A 300-sample random probe of InvertedPendulum’s parameter space locates a best reward of only around 58, some 15-fold below IPOP-CMA-ES’s eventual 890.0. IPOP-CMA-ES reaches this region through its population-doubling restart schedule [38]; JEPA-PS’s single-agent form has no analogous restart or basin-hopping mechanism, and the Swarm-JEPA population wrapper does not supply one either, since its migration step synchronizes islands before any could independently locate a narrow basin (confirmed separately via an island-count sweep that found performance flat across K=1K=1 through K=16K=16, consistent with the migration schedule equalizing islands’ histories faster than diversity can accumulate). We conclude the gap reflects the interaction between IPOP-CMA-ES’s restart mechanism and a narrow high-reward basin that a fixed random-probe budget does not reliably locate, rather than a defect in the proposal mechanism. This is consistent with, and anticipates, § 9’s later finding that a different restart mechanism (restart-on-stagnation, introduced there) substantially improves JEPA-PS’s reach on CartPole physics variants: restarts matter for this architecture generally, not only on InvertedPendulum. This concerns whether the optimizer can discover a hard-to-reach basin, which is a question about basin discovery, not about how much of an already-discovered trajectory the optimizer exploits; it also means that HD and OrdSens on this environment are measured on an optimizer that is not making progress, a caveat we carry forward into § 8.

TransformationCartPoleSwimmerMountainCar(C)
Original337.85263.43−28.41-28.41
Ranked338.31262.87−28.39-28.39
Monotonic (log)337.73263.11−28.42-28.42
Permuted212.84245.67−28.40-28.40
Ordering sensitivity (Eq. (3))0.3700.068−0.000-0.000
Magnitude sensitivity−0.001-0.0010.002−0.001-0.001

Table 33. Reward transformation (mean best reward, 5 seeds; monotone transforms leave performance unchanged)

ConditionMean bestΔ\Delta
Full history410.87—
Remove best285.10−30.6%-30.6\%
Remove early half300.90−26.8%-26.8\%
Best-only321.43−21.8%-21.8\%
Recent-8309.23−24.7%-24.7\%
Random-8320.94−21.9%-21.9\%

Table 34. Forgetting sweep (3 seeds)

ConditionMean bestΔ\Delta
Forward (control)410.87—
Reversed355.40−13.5%-13.5\%
Shuffled410.90+0.0%+0.0\%

Table 35. Order sweep (3 seeds)

LL48163264400
Mean best54.554.554.554.554.554.5

Table 36. Context-length sweep (fixed low budget)

ConditionMean bestStdDrop vs. control
Control (full history)410.9178.2—
Shuffle reward values341.1197.0−17.0%-17.0\%
Permute parameter dimensions329.7188.9−19.8%-19.8\%
Reverse temporal order354.7185.4−13.6%-13.6\%
Best-only (single point)319.2157.6−22.3%-22.3\%

Table 37. Inference-time history perturbations (n=5n=5 seeds, 200 iterations; control is the unperturbed full history)

Table 20: Isolating the two reward channels at test time (CartPole, n=5n=5 seeds, T=400T=400; top), and the target-return percentile sweep at a small-scale (2k/50ep) protocol and the main protocol (bottom). Corrupting r∗r^* has no measurable effect at any percentile from 50 to 99, at either budget; corrupting the reward values inside the history does.

ConditionMeanStdΔ\Delta vs. normal
Normal (r∗=80r^*=80th percentile)410.9178.2—
r∗=0r^*=0410.9178.20.0%
r∗=r^*= random in range410.9178.20.0%
r∗=min⁡(r)r^*=\min(r) (worst)410.9178.20.0%
History reward values zeroed268.0—−34.8%-34.8\%
ρ\rho506070
809095
99
CartPole (small-scale)338.9339.1338.7
Swimmer (small-scale)178.5178.6178.7
CartPole mean (main protocol)410.9410.9410.9
CartPole std (main protocol)178.2178.2178.2

Table 20.

ρ\rho50607080909599
CartPole (small-scale)338.9339.1338.7338.5338.6338.9339.2
Swimmer (small-scale)178.5178.6178.7178.5178.6178.7178.7
CartPole mean (main protocol)410.9410.9410.9410.9410.9410.9410.9
CartPole std (main protocol)178.2178.2178.2178.2178.2178.2178.2

Table 20.

Table 21: Effect of r∗r^* conditioning (n=5n=5 seeds, T=400T=400). Disabling r∗r^* has no measurable effect on final reward on either landscape (top); yet varying r∗r^* measurably shifts the proposal distribution, as quantified by maximum mean discrepancy (MMD, bottom).

r∗r^* conditionCartPole (FDC=0.661)Swimmer (FDC=0.482)
Normal (80th/99th pct.)410.9272.1
Zero / worst / 20th pct.408.5–412.7268.4–276.8
Δ\Delta vs. normal≈0%\approx0\%≈0%\approx0\%

Table 21.

r∗r^* percentile pair (Swimmer, n=5n=5 seeds)MMD
20% vs. 50%1.762
20% vs. 99%1.829
80% vs. 99%1.633

Table 21.

Table 22: Effect of decoder training on downstream reward (CartPole, n=5n=5 seeds), and mean gradient norm reaching each component during a training step.

Decoder variantRewardΔ\Delta vs. frozen
Frozen random (default)410.9—
Reconstruction fine-tuned293.7−28.5%-28.5\%
End-to-end joint retraining269.0−34.5%-34.5\%

Table 22.

ComponentMean gradient normRole
Context encoder gϕg_\phi0.2479encodes the history
Predictor pψp_\psi0.4533predicts the target latent
Decoder hξh_\xi0.0000maps the latent to θ^\hat{\theta}

Table 22.

Budget scaling (Swarm-JEPA saturates near 100 evaluations; CEM overtakes it by 200)
Method102050100200400
Swarm-JEPA283.8328.1404.9410.9410.9410.9
CEM224.1278.3278.3413.9447.6483.1
Gaussian reward noise (reward range ≈410\approx410; degradation is non-monotone and not statistically significant)
Noise level0%5%10%20%50%
Mean best410.9270.0239.3287.8427.2
Pretraining data efficiency (both mean reward and variance improve smoothly with corpus size)
nn trajectories1003005001,0003,0005,000
20,000
Mean best480.2481.1483.3487.6488.1490.1
Std29.643.527.624.522.118.5
15.4
Sparse feedback
Feedback every kk steps151020
Mean best410.9268.0268.0268.0
Delayed feedback (a short delay costs performance; longer delays recover and exceed it)
Delay dd (steps)05102050
Mean best410.9324.1452.9435.2487.8

Table 25. Full CartPole robustness grid: evaluation-budget scaling, Gaussian reward noise, pretraining data efficiency, sparse feedback, and delayed feedback (n=5n=5 seeds unless noted).

ConditionMeanStd
Baseline (noise 0%)293.128.7
Noise 10%150.625.7
Noise 20%156.218.5
Noise 50%190.625.4
Sparse k=5k=593.962.0
Sparse k=10k=1093.962.0
Sparse k=20k=2097.267.0
Delay d=5d=596.859.4
Delay d=10d=10100.456.0
Delay d=20d=20103.056.0
Delay d=50d=50274.989.3

Table 26. Robustness on Swimmer-v4 (n=5n=5 seeds, T=400T=400). A 50-step delay substantially improves performance relative to the short-delay and sparse conditions, though it remains slightly below the no-perturbation baseline.

ConditionMeanStd
Baseline (noise 0%)1039.733.5
Noise 10%127.669.9
Noise 50%127.669.9
Sparse k=5k=5127.669.9
Delay d=50d=50345.3328.9

Table 27. Robustness on Hopper-v4 (n=5n=5 seeds, T=400T=400).

References

  1. [1]Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 25, 2012.arxiv.org/abs/1206.2944
  2. [2]Horia Mania, Aurelia Guy, and Benjamin Recht. Simple random search of static linear policies is competitive for reinforcement learning. Advances in Neural Information Processing Systems, 31, 2018.
  3. [3]Pieter-Tjerk de Boer, Dirk P. Kroese, Shie Mannor, and Reuven Y. Rubinstein. A tutorial on the cross-entropy method. In Annals of Operations Research, volume 134, pages 19–67. Springer, 2005.DOI
  4. [4]Nikolaus Hansen. The cma evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772, 2016.arxiv.org/abs/1604.00772
  5. [5]Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):148–175, 2015.DOI
  6. [6]Jeremy Rapin and Olivier Teytaud. Open source evolutionary structured optimization. GitHub repository, Nevergrad: A gradient-free optimization platform, 2018.
  7. [7]Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. Advances in Neural Information Processing Systems, 29, 2016.arxiv.org/abs/1606.04474
  8. [8]Ke Li and Jitendra Malik. Learning to optimize. arXiv preprint arXiv:1606.01885, 2016.arxiv.org/abs/1606.01885
  9. [9]Yutian Chen, Matthew W. Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Timothy P. Lillicrap, Matt Botvinick, and Nando de Freitas. Learning to learn without gradient descent by gradient descent. In International Conference on Machine Learning, pages 748–756, 2017.arxiv.org/abs/1611.03824
  10. [10]Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, and Sascha Boehl-Dickstein. Velo: Training versatile learned optimizers by scaling up. arXiv preprint arXiv:2211.09760, 2022.arxiv.org/abs/2211.09760
  11. [11]Brandon Amos. Tutorial on amortized optimization. Foundations and Trends in Machine Learning, 16(5):592–732, 2023.
  12. [12]Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In International Conference on Learning Representations, 2024.arxiv.org/abs/2309.03409
  13. [13]Junru Song, Yang Yang, Huan Xiao, Wei Peng, Wen Yao, and Feifei Wang. Laser: Towards diversified and generalized robot design with large language models. In International Conference on Learning Representations, 2025.
  14. [14]Kevin Qiu, Władysław Pałucki, Krzysztof Cie biera, Paweł Fijałkowski, Marek Cygan, and Łukasz Kuciński. Robomorph: Evolving robot morphology using large language models. arXiv preprint arXiv:2407.08626, 2024.
  15. [15]Robert Tjarko Lange, Yingtao Tian, and Yujin Tang. Large language models as evolution strategies. arXiv preprint arXiv:2402.18381, 2024.arxiv.org/abs/2402.18381
  16. [16]Yifan Zhou, Sachin Grover, Mohamed El Mistiri, Kamalesh Kalirathinam, Pratyush Kerhalkar, Swaroop Mishra, Neelesh Kumar, Sanket Gaurav, Oya Aran, and Heni Ben Amor. Prompted policy search: Reinforcement learning through linguistic and numerical reasoning in llms. In Advances in Neural Information Processing Systems, 2025. arXiv:2511.21928.
  17. [17]Tennison Liu, Nicolás Astorga, Nabeel Seedat, and Mihaela van der Schaar. Large language models to enhance bayesian optimization. In International Conference on Learning Representations, 2024.
  18. [18]Jamison Meindl, Yunsheng Tian, Tony Cui, Veronika Thost, Zhang-Wei Hong, Jie Chen, Wojciech Matusik, and Mina Konakovic Lukovic. Gptopt: Towards efficient llm-based black-box optimization. arXiv preprint arXiv:2510.25404, 2025.
  19. [19]Yann LeCun. A path towards autonomous machine intelligence version 0.9.2. Open Review, 2022.
  20. [20]Mahmoud Assran, Quentin Duval, Ishaan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15619–15629, 2023.
  21. [21]Terry Jones and Stephanie Forrest. Fitness distance correlation as a measure of problem difficulty for genetic algorithms. Proceedings of the Sixth International Conference on Genetic Algorithms, pages 184–192, 1995.
  22. [22]Elliot Meyerson, Mark J. Nelson, Herbie Bradley, Adam Gaier, Arash Moradi, Amy K. Hoover, and Joel Lehman. Language model crossover: Variation through few-shot prompting. In ACM Transactions on Evolutionary Learning and Optimization, 2024.
  23. [23]Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Paw an Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. Mathematical discoveries from program search with large language models. Nature, 625:468–475, 2024.
  24. [24]Fabio Ferreira, Lucca Wobbe, Arjun Krishnakumar, Frank Hutter, and Arber Zela. Can llms beat classical hyperparameter optimization algorithms? a study on autoresearch. arXiv preprint arXiv:2603.24647, 2026.
  25. [25]Xinyu Zhang, Daolang Huang, Samuel Kaski, and Julien Martinelli. Pabbo: Preferential amortized black-box optimization. In International Conference on Learning Representations, 2025.arxiv.org/abs/2503.00924
  26. [26]Edouard R. Dufour and Pascal Fua. Generative refinement for low-budget black-box optimization. arXiv preprint arXiv:2607.00691, 2026. Introduces the SPARROW method, used here as the source for the rank-based, ordinal-archive comparison.arxiv.org/abs/2607.00691
  27. [27]Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mahmoud Assran, and Nicolas Ballas. Revisiting feature prediction for learning visual representations from video. arXiv preprint arXiv:2404.08471, 2024.
  28. [28]Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. data2vec: A general framework for self-supervised learning in speech, vision and language. arXiv preprint arXiv:2202.03555, 2022.
  29. [29]Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent: A new approach to self-supervised learning. Advances in Neural Information Processing Systems, 33, 2020.
  30. [30]David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. Advances in Neural Information Processing Systems, 31, 2018.arxiv.org/abs/1809.01999
  31. [31]Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023.arxiv.org/abs/2301.04104
  32. [32]Edward Weinberger. Correlated and uncorrelated fitness landscapes and how to tell the difference. Biological Cybernetics, 63(5):325–336, 1990.DOI
  33. [33]James Kennedy and Russell Eberhart. Particle swarm optimization. In Proceedings of the IEEE International Conference on Neural Networks, volume 4, pages 1942–1948, 1995.
  34. [34]Darrell Whitley, Soraya Rana, and Robert B. Heckendorn. The island model genetic algorithm: On separability, population size and convergence. In Journal of Computing and Information Technology, volume 7, pages 33–47, 1999.
  35. [35]Marco Tomassini. Spatially structured evolutionary algorithms: Artificial evolution in space and time. Natural Computing Series, Springer, 2005.DOI
  36. [36]Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 23–30, 2017.arxiv.org/abs/1703.06907
  37. [37]Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. In IEEE International Conference on Robotics and Automation, pages 3803–3810, 2018.
  38. [38]Anne Auger and Nikolaus Hansen. A restart cma evolution strategy with increasing population size. IEEE Congress on Evolutionary Computation, 2:1769–1776, 2005.DOI
  39. [39]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.arxiv.org/abs/1706.03762
  40. [40]Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016.arxiv.org/abs/1606.01540
  41. [41]Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033, 2012.DOI
  42. [42]Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.arxiv.org/abs/1412.6980
  43. [43]Aloïs Pourchot and Olivier Sigaud. Cem-rl: Combining evolutionary and gradient-based methods for policy search. International Conference on Learning Representations, 2019. arXiv:1810.01222.arxiv.org/abs/1810.01222
  44. [44]Shauharda Khadka, Sombed Majumdar, Tarek Nassar, Zach Dwell, Evren Turner, Santiago Miret, Yinyin Liu, and Kagan Turner. Collaborative evolutionary reinforcement learning. In International Conference on Machine Learning, pages 3341–3350, 2019.arxiv.org/abs/1905.00976
  45. [45]David Eriksson, Michael Pearce, Jacob Gardner, Ryan D. Turner, and Matthias Poloczek. Scalable global optimization via local bayesian optimization. Advances in Neural Information Processing Systems, 32, 2019.arxiv.org/abs/1910.01739
  46. [46]Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In International Conference on Learning and Intelligent Optimization, pages 507–523, 2011.DOI
  47. [47]James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems, volume 24, 2011.
  48. [48]Nikolaus Hansen, Steffen Finck, Raymond Ros, and Anne Auger. Real-parameter black-box optimization benchmarking (bbob): Noiseless functions definitions. INRIA Research Report RR-6829, 2009.
  49. [49]Vesselin K. Vassilev, Terence C. Fogarty, and Julian F. Miller. Fitness landscapes and search. Advances in Evolutionary Computing: Theory and Applications, pages 31–54, 2003.
  50. [50]Stuart A. Kauffman and Simon Levin. Towards a general theory of adaptive walks on rugged landscapes. Journal of Theoretical Biology, 128(1):11–45, 1987.pubmed.ncbi.nlm.nih.gov/3431131

Paper details

vlnc:W_pz87rxka
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents