WorksComputer Science
Particle Filtering with Generative Trajectory Models for State Estimation and Control
Abstract
Generative models can support both state estimation and control in robotics. For control, a common approach is to use the model as a policy that directly samples actions conditioned on observation histories. However, models that jointly represent states, actions, and observations can also predict how the robot state and its environment evolve. We investigate whether these predictions can improve state estimation and control through a change in inference procedure, without modifying the pretrained model. To this end, we use particle filtering to maintain competing hypotheses about state histories. A single pretrained diffusion model generates these hypotheses along with their predicted observations and associated action proposals. As new measurements arrive, particles are reweighted according to how well their predicted observations match the measurements. While particle filtering conventionally uses these weights only for state estimation, we also use them to guide action selection, favoring proposals associated with hypotheses that better explain the observations. Simulation experiments in partially observed navigation and vision-based manipulation demonstrate improvements in both state estimation and task performance compared to action generation without filtering, using the same pretrained model and no additional test-time information.
Introduction
Diffusion- and flow-based generative models have become powerful tools for robot learning and control. They have been used as policies that directly generate actions conditioned on observations, as world models that predict the evolution of the environment, and, more recently, as unified world-action models that jointly model states, observations, and actions over time. Such joint trajectory models can support a variety of conditional generation tasks, depending on which variables are provided as context and which are generated. Their usefulness therefore depends not only on the distributions they learn, but also on how those distributions are used at inference time.
For control, a natural approach is to directly generate actions conditioned on the available observation-action history. In principle, if this procedure produces an optimal policy, no alternative inference strategy with access to the same information can achieve better expected task performance. In practice, however, learned policies are generally imperfect due to limited training data, finite model capacity, and distribution shifts between training and deployment. Moreover, a joint generative model may capture useful relationships between states, observations, and actions that are not fully
exploited by direct action generation. This creates an opportunity to improve control through additional inference-time computation, without modifying the learned model. This question is particularly relevant under partial observability, where successful control depends on inferring the underlying state from observations accumulated over time.
In this work, we explore particle filtering as an alternative inference-time approach to state estimation and control with pretrained world-action models (Figure 1). Under partial observability, several different state histories may be consistent with the observations available so far. Maintaining a diverse set of plausible histories allows later observations to help resolve this ambiguity. While traditional particle filters maintain a distribution over possible states of the environment, our approach also associates each state hypothesis with candidate future actions, generated jointly by the same model. As a result, evidence about the underlying state can directly influence action selection, allowing particle filtering to serve both state estimation and control within a unified framework. Importantly, this approach requires no additional filtering-specific training or separately learned dynamics, observation, or policy models. We compare particle filtering against direct action generation using the same pretrained model.

Figure 1. Two inference procedures using the same pretrained model . (a) Direct action generation. (b) Particle filtering maintained weighted state histories and generates joint state-action-observation proposals. Weights guide action selection. Agreement between predicted and measured observations updates the belief for the next step. Stacked blocks denote per-particle computations.
We investigate the benefits and limitations of this approach through experiments in partially observed navigation and visuomotor robotic manipulation. Navigation provides a controlled setting in which we systematically study the effects of training data, distribution shift, and different particle filtering configurations. We further evaluate the approach on manipulation tasks with high-dimensional RGB observations and substantially different action spaces. Our results demonstrate that inference-time particle filtering can improve state estimation and control without additional model training, while highlighting the conditions under which these improvements can be expected.
Related Work
a) Generative policies, trajectory models, and world-action models: Diffusion models have emerged as expressive models for both robot policies and trajectory-level decision making. Diffusion Policy represents multimodal visuomotor action distributions through conditional denoising [1], while Diffuser and Decision Diffuser formulate planning and control as conditional trajectory generation [2, 3]. More recently, world-action models jointly represent world evolution and executable robot actions within a unified generative model. Unified World Models, for example, couples video and action diffusion such that a single model can serve as a policy, forward or inverse dynamics model, or observation generator [4]. Diffusion Forcing provides a complementary mechanism for flexible conditioning over arbitrary subsets of sequential variables [5]. We build on these flexible conditioning capabilities by using a pretrained joint state-action-observation model as the primitive in an inference-time particle filtering procedure.
b) Particle filtering with learned models: Prior learned particle-filter approaches replace hand-specified filtering models with learnable dynamics and observation components embedded in a differentiable particle-filter structure, allowing the resulting estimator to be optimized end-to-end [6, 7]. More recently, DiffPF uses a conditional diffusion model to generate filtering-posterior particles [8]. These methods use learned models to improve estimation, but their particles are used to represent state hypotheses rather than candidate decisions. In our setting, a pretrained generative model jointly produces states, observations, and actions, so each particle simultaneously represents a state hypothesis and an associated control rollout. Moreover, our generative model is trained independently of the filtering procedure and without filtering-specific training or learned components.
c) Particle beliefs for control: Particle beliefs have also been used for decision making under partial observability. Classical online POMDP methods represent uncertainty with particles and couple this belief representation to online planning, typically using repeated forward simulation and tree search to evaluate candidate actions [9, 10]. These approaches demonstrate the value of particle-based beliefs for control, but generally require a separate test-time planning procedure on top of the filter. In our setting, each particle instead carries an action rollout generated jointly with its state and observation hypotheses. Observation-based particle weights can therefore be used directly for action selection, without a separate belief-space planning or search procedure.
d) Particle methods for diffusion inference: Particle-based methods have also been used within diffusion inference to improve sampling or steer generation toward task-specific objectives. Such approaches maintain and resample candidate diffusion samples according to guidance or reward potentials [11, 12]. This use of particles is orthogonal to ours: their particles exist within a single generative inference process, whereas our particles persist across environment timesteps as hypotheses over latent state and associated future trajectories, and are reweighted using newly observed sensor data.
Preliminaries
We consider filtering and control under partial observability, with state , action , observation , and available history . Filtering estimates .
a) Particle filtering: A particle filter represents this posterior by weighted state trajectories,
with . In a bootstrap filter [13], each trajectory is extended using the dynamics model,
The arriving observation reweights particles by their likelihood , favoring hypotheses that better explain the measurement.
b) Diffusion forcing: Diffusion models generate samples by starting from noise and iteratively denoising toward structured outputs. Diffusion forcing [5] extends this procedure to sequences by sampling a separate noise level for each token during training. At inference, known tokens can be held at zero or low noise levels, while queried tokens are denoised fully starting from pure noise. This enables the same model to condition on different subsets of sequential variables and modalities [14].
Method
We combine particle filtering with a learned trajectory diffusion model. The central difference from a bootstrap particle filter is the proposal distribution: instead of advancing each particle with a one-step dynamics kernel, we use a learned generative model to propose future states, observations, and actions jointly. The resulting particles are therefore both state-estimation hypotheses and candidate control rollouts.
Flexible Conditional Trajectory Sampler
Our method exposes the diffusion model through a flexible conditioning interface over state, action, and observation tokens. Each model query fixes a subset of trajectory variables and samples the remaining variables jointly. Let denote query index sets to be generated, and let denote conditioning index sets supplied to the model. We write the resulting conditional sampler as
This construction uses the diffusion model as a flexible proposal distribution . By changing which variables are conditioned on and which are queried, the same model can act as a state-transition proposal, an observation predictor, a future-action generator, or a joint trajectory model.
Particle Filter
We now describe how this flexible sampling interface enables particle filtering for both state estimation and decision making at inference time. Each iteration uses the new observation to reweight competing state-history hypotheses, then generates an action proposal conditioned on each history. The updated weights determine how these proposals influence the executed action.
Particle State
1) Particle State: At the current timestep , each particle carries a hypothesized state history and a corresponding weight . In the first iteration, particle weights are initialized uniformly and the algorithm proceeds directly to action proposal.
State and Observation Prediction
2) State and Observation Prediction: Before incorporating the observation at the current time , we predict the current state and observation for each particle, conditioned only on its state history hypothesis, past observations and actions , and the action commanded at the preceding step :
Observation-Consistent Weight Updates
3) Observation-Consistent Weight Updates: We update each particle’s weight according to how well its prediction agrees with the new observation , favoring histories that better explain the observed sensor stream. Let denote this nonnegative observation score, which need not be normalized. It may be an explicit sensor likelihood or another model-based compatibility score. We may also avoid sensor-specific design by using an observation-consistency score directly comparing the realized observation to the previously predicted observation . Our implementation compares predicted and measured observations using cosine similarity in a learned latent observation space, as described in Section VI. The particle weights are then updated as
Resampling is applied using the updated weights if the effective sample size becomes small.
Action Proposal
4) Action Proposal: We sample current and future states and future actions for each particle, sharing the commanded action and observation history across particles:
where is the planning horizon. The newly sampled current state incorporates the current measurement . It replaces the prediction used for scoring and is appended to the particle’s state history.
Action Aggregation
5) Action Aggregation: We use the particle weights to choose the next action from the proposals, evaluating two action-selection rules. Best-particle action selection executes the proposal associated with the highest-weight particle. Weighted action averaging instead combines the action proposals using the weighted average
The selected action is then executed and the algorithm proceeds to the next iteration.
When Can Particle Filtering Help?
Particle filtering and direct action generation both operate using the same observation-action history. If direct action generation already realizes an optimal policy with exact sampling, filtering cannot improve expected task performance. Its potential benefit lies in exploiting learned relationships among states, observations, and actions that an imperfect direct policy may underuse. Maintaining and reweighting explicit state hypotheses could expose this structure to action selection. We hypothesize that this improves control when three conditions hold:
State-action alignment. The generative model must capture meaningful relationships between states and actions, such that more accurate or task-relevant state hypotheses tend to produce higher-quality action proposals.
Observation-state alignment. The generative model must capture meaningful relationships between states and observations, such that compatibility between predicted and realized observations provides useful evidence about the quality of the corresponding state hypotheses.
Sufficient particle coverage and diversity. The generative model must assign sufficient probability to high-quality state-action hypotheses, and the particle population must retain enough diversity to represent these hypotheses until observations can distinguish between them.
These conditions allow observation-based particle weighting to favor state hypotheses associated with better control proposals. In Section VII, we test whether each of the conditions holds in practice through a series of experiments.
Implementation
Latent Observation World-Action Model
Raw sensor observations, such as RGB images, are high-dimensional and may contain substantial task-irrelevant variation. This can make defining an observation-consistency metric directly in the raw observation space challenging. We therefore jointly learn an observation encoder and a diffusion model over states, actions, and latent observations,
The resulting latent observations provide a compact representation for both trajectory modeling and the observation-consistency score .
Directly backpropagating the latent-observation prediction loss into the parameters of may encourage a degenerate representation that is easy to predict but uninformative. We therefore train the encoder only through the prediction of task variables. Let
denote states, actions and latent observations corrupted by the diffusion forward process using diffusion noise levels which are sampled independently for each timestep and each input modality. Let denote the denoising network, conditioned on the diffusion noise level of each token (we suppress this conditioning below for readability) and compute
where gradients are allowed to flow through to . We then compute the observation prediction using detached latent observations,
where denotes the stop-gradient operator. Both passes share model parameters and use the same corrupted training example, differing only in the gradient flow to the observation encoder.
Using the clean-data-prediction parameterization for notational simplicity, the training objective is
Thus, the observation encoder is optimized to retain information useful for predicting task variables, while the diffusion model is simultaneously trained to model the resulting latent observation sequence.
Latent Cosine Similarity Observation-Consistency Metric
Our choice of observation-consistency metric used to compare predicted and realized observations in the particle filter uses cosine similarity of latent observations:
where is a temperature scaling hyperparameter.
Experiments
We evaluate our approach in two domains: goal-directed navigation in partially observed 2D environments and visuomotor robotic manipulation. We primarily use navigation as a controlled testbed, where synthetic data generation and access to ground-truth state enable systematic studies of model quality, observation consistency, and particle filtering behavior. We then evaluate our method on robotic manipulation tasks with high-dimensional RGB observations and substantially different action spaces, demonstrating its applicability beyond the navigation setting.
For each task and training-dataset size, all inference variants use the same trained trajectory model and observation encoder, with identical parameters. The baseline denoted Diffusion Policy performs direct action generation using this model without particle filtering. At inference, conditioning histories are restricted to a rolling window that fits the model’s context length. Navigation executes one action per update, while manipulation executes eight actions open-loop. For chunked execution, state and observation prediction conditions on the full previously commanded action chunk, and the observation scores over that chunk are multiplied in a single weight update.
Experimental Setup (Navigation)
Our navigation task requires collision-free traversal of an unknown 2D environment towards a goal specified in the frame of the robot’s initial pose. At each step, the robot receives a 2D LiDAR observation consisting of distance measurements and executes a control action, specified in the robot’s body frame. To ensure that LiDAR-based localization becomes essential, we add zero-mean Gaussian noise to executed actions with a standard deviation of 40% of the commanded step length. Heading is not controlled by the policy, instead evolving according to an exogenous random walk process. At each timestep, the state is the 4D vector , combining the goal’s position in the agent’s current body frame with its heading relative to the trajectory’s initial pose. This representation is invariant to global translation and rotation, while the sine-cosine encoding avoids angular discontinuities.
Dataset
Dataset: We generate a synthetic dataset containing 1000 examples with randomly placed circular and rectangular obstacles and randomly selected start and goal locations. Each demonstration trajectory is created by first planning the shortest collision-free path between the start and goal. Then, the pure-pursuit controller [15] is used to simulate tracking of the planned reference under noisy action execution, and heading is assigned according to a correlated random walk.
Model & Training
Model & Training: We implement the observation encoder using circular convolutions, producing one or more latent observation tokens per timestep. The denoising network is implemented as a diffusion transformer [16] which receives noised latent observation, state, and action tokens. A block-triangular attention mask is used to implement temporal causal masking. Goal conditioning is achieved through in-context conditioning in which the first timestep’s state is set to the relative goal position. Since the goal is provided only through the initial state, the navigation Diffusion Policy baseline retains its sampled state history to carry goal information forward as the context window advances.
Evaluation Metrics and Protocol
Evaluation Metrics and Protocol: We measure task-relevant performance using the deviation between the goal position and the position at termination. State-estimation metrics use weighted mean particle positions, independently of action selection. We report ADE (mean absolute position error) and translational RPE (mean relative-translation error over consecutive intervals of distance 0.1). Unless otherwise stated, navigation experiments use 200 held-out evaluation scenarios. Each experiment is repeated eight times using
| Method | Retained histories | Accum. weights | Final dist ↓ | ADE ↓ | RPE ↓ |
| Diffusion Policy | 1 | – | 0.898 | 0.494 | 0.116 |
| One-step selection | 1 | No | 0.528 [−44%,−38%] | 0.253 [−51%,−47%] | 0.067 [−43%,−40%] |
| Multi-history sel. | 32 | No | 0.589 [−37%,−32%] | 0.282 [−46%,−40%] | 0.092 [−24%,−15%] |
| Full PF | 32 | Yes | 0.426 [−56%,−49%] | 0.195 [−62%,−59%] | 0.061 [−49%,−45%] |
Table I. Full particle filtering outperforms naïve action selection under the pose-oracle observation model.
different random seeds. Tables report mean metrics and paired 95% bootstrap confidence intervals for percentage changes in mean performance relative to Diffusion Policy. We average repeated runs within each scenario and resample scenarios, using the same sampled scenarios for each method and its baseline. When applicable, 32 candidates or particles are used, unless noted otherwise. Unless otherwise stated, navigation particle-filter experiments use best-particle action selection.
Comparison To Other Candidate-Selection Methods
We begin by investigating how particle filtering compares to other methods that use selection from among several candidates.
Ground-truth pose oracle likelihood
To evaluate state-action alignment independently of observation model accuracy, we first use a privileged pose oracle score that assigns higher likelihood to particles whose predicted poses agree more closely with ground truth:
Here, refers to the position error (positions share the initial-pose frame), and each particle’s heading error is defined analogously. The scales and are hyperparameters.
Baselines
Our first baseline, one-step selection, maintains a single state trajectory history. At each step, it samples and executes an action conditioned on that history and the latest observation. It then compares candidate extensions of the history, selecting the one that most accurately predicts the resulting ground-truth pose. The selected history is retained for the next step.
Multi-history selection instead maintains distinct state histories. At each update, it selects the action proposal associated with the highest current pose-oracle score. Compared to full particle filtering, selection uses only the latest prediction error, without accumulating scores over time.
Results (Table I)
We find that both baselines are able to leverage ground-truth pose-informed state selection to achieve improved task performance and state estimation accuracy compared to diffusion policy, indicating that the model has indeed learned significant state-action alignment, providing evidence for Condition 1. However, simple one-step selection that does not persist diverse histories and multi-history selection that does not accumulate weights across timesteps both perform significantly worse than full particle filtering.
Observation-Consistency Observation Model
In the previous section, we have demonstrated that particle filtering provides a principled way of incorporating scores assigned to multiple candidates. However, in practice, access to an oracle may not be available. We therefore investigate the effectiveness of the proposed cosine similarity observation-consistency score (eq. 14) as an observation model.
To distinguish the benefits of particle filtering from those of generating and combining multiple proposals, we evaluate two additional baselines. The ensemble baseline uses the same pretrained model to generate proposals and uniformly averages their state estimates and proposed actions, providing a comparison with similar proposal-generation cost. Each ensemble member retains its own sampled state history across planning iterations. Particle filtering with shuffled weights randomly permutes the observation-model scores among particles at each update, before applying them to the accumulated weights. This disrupts the correspondence between each particle hypothesis and its observation-consistency score.
Known-map observation model
To establish that cosine-similarity observation-consistency is a suitable score, we must first verify that the latent observation representation learned by the model supports meaningful comparison via cosine similarity. To this end, we consider a known-map observation model
where denotes the result of computing a LiDAR observation by raycasting from the pose in the known map. As shown in Table II, this known-map observation model enables a substantial improvement across both task performance and state estimation metrics, demonstrating that the latent observation representation is indeed suitable for use with the proposed cosine-similarity-based observation model.
| Method | Final dist ↓ | ADE ↓ | RPE ↓ |
| Baselines: | |||
| Diffusion Policy | 0.898 | 0.4941 | 0.1155 |
| Ensemble | 0.940 [−1%,+10%] | 0.4516 [−16%,−2%] | 0.0658 [−44%,−40%] |
| PF w/ shuffled weights | 0.874 [−6%,+10%] | 0.4383 [−15%,−8%] | 0.0703 [−40%,−37%] |
| PF (weighted mean) | 0.885 [−7%,+4%] | 0.4196 [−22%,−9%] | 0.0653 [−44%,−41%] |
| With privileged information: | |||
| PF w/ known map | 0.633 [−33%,−26%] | 0.3143 [−40%,−33%] | 0.0677 [−43%,−38%] |
| PF w/ pose oracle | 0.426 [−56%,−49%] | 0.1947 [−62%,−59%] | 0.0609 [−49%,−45%] |
| Particle Filter | 0.812 [−13%,−6%] | 0.4087 [−20%,−14%] | 0.0697 [−41%,−37%] |
Table II. Observation-consistency observation model enables improved task performance and absolute state estimation.
Predicted observation consistency
We further verify whether the model has learned to predict latent observations that are consistent and responsive to each particle’s state history. As shown in Table II, using the cosine observation model (eq. 14) to compare each particle’s predicted latent observation with the latent representation of the measured observation significantly outperforms the Diffusion Policy baseline, supporting Condition 2. The ensemble and shuffled-weight baselines also improve state estimation, but neither yields a clear improvement in task performance. In particular, the comparison with shuffled scores indicates that observation-consistent weighting helps identify state-action hypotheses that lead to better control. In Figure 2, we show an example of particle filtering using the predicted-observation consistency model, alongside the Diffusion Policy baseline.

Figure 2. Closed-loop navigation with Diffusion Policy (top) and particle filtering (bottom). Magenta: estimated path; black: executed path; green/red crosses: goal/start. Particle brightness indicates weight. Diffusion Policy stops at , particle filter stops at .
Action Selection Rule
Table II additionally compares best-particle action selection with weighted action averaging, using the same particle-filtering procedure and observation-consistency score. Weighted action averaging achieves similar ADE and lower RPE than best-particle selection, yet only best-particle selection yields a clear improvement in final goal distance over Diffusion Policy. These results suggest that particle filtering’s estimation benefits are robust to the choice between these two rules, while using the resulting belief effectively for control requires an appropriate action-selection rule.
Particle Count Scaling
Increasing the particle count tests whether the model supplies sufficient hypothesis coverage (Condition 3). Figure 3 shows improvements in task performance and state estimation under all three observation models. Predicted-observation weighting exhibits diminishing returns beyond 16 particles, while the pose oracle continues to improve, supporting both coverage and state-action alignment. Known-map weighting achieves intermediate task performance and ADE. The remaining gap to the pose oracle may reflect both
Figure 3. Particle-count scaling with pose-oracle (orange), known-map (green), and predicted-observation (blue) weighting.
| Method | 1,000 | 10,000 | 100,000 |
| Diffusion Policy | 0.898 | 0.454 | 0.369 |
| Particle Filter | 0.812 [−13%,−6%] | 0.415 [−13%,−4%] | 0.395 [+4%,+11%] |
| PF w/ GT oracle | 0.426 [−56%,−49%] | 0.237 [−52%,−44%] | 0.231 [−42%,−33%] |
Table 3. FINAL DISTANCE ACHIEVED BY NAVIGATION MODELS TRAINED ON DATASETS OF INCREASING SIZE.
| Method | Final dist ↓ | ADE ↓ | RPE ↓ |
| Diffusion Policy | 0.863 | 0.4772 | 0.0515 |
| Particle Filter | 0.792 [−11%,−5%] | 0.4501 [−8%,−4%] | 0.0444 [−15%,−13%] |
| PF w/ pose oracle | 0.490 [−48%,−38%] | 0.2750 [−46%,−39%] | 0.0364 [−31%,−27%] |
Table 4. ON OOD SCENARIOS, PARTICLE FILTERING HELPS EVEN USING THE MODEL TRAINED ON 100,000 SCENARIOS.
ambiguity in the observations and limitations of the learned representation and cosine similarity score.
Dataset Scaling
We view particle filtering as a way to exploit learned state-action-observation relationships that direct action generation may not fully utilize, particularly when training data is limited. We therefore test increasing training data from 1,000 to 100,000 demonstrations, which substantially improves direct action generation (Table III). Observation-consistency filtering helps at 1,000 and 10,000 demonstrations but performs slightly worse at 100,000. Pose-oracle filtering retains a substantial advantage throughout, indicating that the model continues to produce state-action hypotheses supporting better control, and showing that particle filtering can remain an effective way of incorporating additional known information.
We additionally evaluate the 100,000-demonstration model on 100 out-of-distribution scenarios with eight random seeds (Table IV). While training scenes contain multiple randomly placed obstacles and randomly selected endpoints, OOD scenes contain a single rectangular or circular obstacle (dimensions and endpoints remain randomized). Filtering improves both task performance and state estimation under this distribution shift, demonstrating that additional inference can help even where direct action generation performs better in-distribution.
| Method | Action selection | Num. particles | Persistent state? | Wrist-Camera-Only Transport SR (%) ↑ | State RMSE ↓ | Wrist-Camera-Only ToolHang SR (%) ↑ | State RMSE ↓ |
| Diffusion Policy (DP) | – | 1 | 88.4 | 0.282 | 22.8 | 0.042 | |
| DP w/ persistent state | – | 1 | ✓ | 85.0 [−7.2,+0.2] | 0.324 [+7%,+24%] | 23.4 [−4.2,+5.4] | 0.050 [+7%,+28%] |
| DP ensemble | uniform mean | 32 | 88.6 [−5.4,+5.0] | 0.216 [−29%,−19%] | 5.4 [−22.0,−13.2] | 0.032 [−37%,−12%] | |
| DP ensemble w/ persistent state | uniform mean | 32 | ✓ | 88.8 [−4.8,+5.0] | 0.224 [−28%,−13%] | 8.8 [−19.2,−8.8] | 0.034 [−38%,−2%] |
| PF w/ weighted action averaging | weighted mean | 32 | ✓ | 92.0 [−0.2,+7.2] | 0.187 [−41%,−28%] | 11.2 [−16.0,−7.0] | 0.031 [−38%,−15%] |
| PF w/ best-particle action selection | best particle | 32 | ✓ | 92.0 [+0.4,+7.0] | 0.194 [−38%,−25%] | 28.6 [+1.0,+10.6] | 0.033 [−30%,−15%] |
Table V. Particle filtering improves success rate (SR) and state estimates on challenging wrist-camera-only manipulation tasks.
Extension to Visuomotor Manipulation
Datasets
We consider two datasets based on the Transport and ToolHang tasks from Robomimic [17]. States concatenate object positions and 6D rotations: three object poses for Transport (27 dimensions) and two for ToolHang (18 dimensions). The original dataset includes two available camera views per manipulator: one fixed to the end effector (wrist camera) and another at a static external location (shoulder camera). Since standard diffusion policies already perform strongly when all cameras are available, we instead remove the static cameras, keeping only the wrist camera. As illustrated in Figure 4, this makes the tasks substantially harder, since the shoulder camera shows task-relevant state in almost every individual frame, while the wrist camera sees only changing and partial views of the scene.

Figure 4. Camera observations from a validation set episode of the ToolHang task. All of our manipulation experiments withhold access to the shoulder camera, so the policy must infer actions and state solely from the wrist camera’s view.
Model & Training
The observation encoder uses the same convolutional image encoder architecture as Diffusion Policy [1]. A block-causal transformer is used for the denoising network, just like for the navigation task.
Baselines
For manipulation, we evaluate both stateless and persistent variants of Diffusion Policy and the ensemble baseline. Stateless variants regenerate all state tokens within the current context window at each planning iteration, including those corresponding to past observations, conditioned on the observation-action history. Persistent variants retain previously sampled states and feed them back as conditioning at subsequent iterations. This comparison is possible because manipulation has no explicit goal condition to preserve, while in navigation, persistent states carry goal information forward after the initial goal token leaves the context window. The persistent ensemble retains a separate state history for each member and uses uniform observation scores and uniform action averaging. As in navigation, we compare best-particle action selection with weighted action averaging for particle filtering.
Results (Table V)
We evaluate each method on 50 scenarios with 10 different random seeds, totaling 500 closed-loop simulations per task. State estimates are computed as the componentwise weighted mean, and RMSE is computed across state components and timesteps. State estimation errors are reported only for successful rollouts to reduce confounding from differences in task progress and failure modes. We again report paired 95% confidence intervals for changes from Diffusion Policy.
Using the same trained model, particle filtering with best-particle action selection improves success and reduces state RMSE relative to Diffusion Policy on both tasks. These results extend the benefits of inference-time filtering to RGB-based manipulation under restricted visual observations.
Retaining sampled state histories does not automatically provide useful memory for control. Single-particle persistence offers no clear success benefit, suggesting that carrying forward an uncorrected hypothesis can be insufficient under partial observability. Maintaining multiple histories with uniform averaging likewise provides no reliable control improvement, even though it lowers state-estimation error. This highlights the value of maintaining competing hypotheses and using observation evidence to determine their influence on control.
ToolHang highlights a potential limitation of averaging action proposals under ambiguity. Proposals associated with different plausible behaviors may each be reasonable individually, yet yield ineffective actions when averaged. Best-particle selection preserves an individual proposed action sequence and achieves better task performance despite slightly higher reported state RMSE. The stronger dependence on the choice of action selection rule in ToolHang (as well as in our earlier navigation experiments) suggests that the consequences of action aggregation vary across tasks.
Conclusion
We presented an inference-time approach that uses a joint generative model of states, actions, and observations for both particle filtering and control. Each particle maintains a state-history hypothesis and an associated action proposal, while agreement between predicted and measured observations updates its weight. These weights guide action selection, allowing observation evidence to influence control through persistent hypotheses. The procedure uses the same trained model as direct action generation, without additional filtering-specific training or separately learned dynamics, observation, or policy models.
Experiments in partially observed navigation and manipulation demonstrate that this approach can improve both state estimation and task performance. Controlled studies demonstrate the importance of useful state-action hypotheses, observation-consistent weighting, particle coverage, and the choice of action-selection rule. In navigation, filtering improves control with limited training data and remains useful even with a large synthetic training dataset when obstacle layouts differ from those seen during training. For manipulation, where collecting demonstrations is costly, filtering improves control without requiring more training data.
Particle filtering also provides a natural way to incorporate information available at deployment, such as known maps or explicit sensor models, by using it to assess generated hypotheses without retraining the model. More broadly, it provides a principled framework for using additional inference-time computation with pretrained joint trajectory models.
References
- [1]C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in Proceedings of Robotics: Science and Systems (RSS), 2023.
- [2]M. Janner, Y. Du, J. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” in Proceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 162. PMLR, 2022, pp. 9902–9915.arxiv.org/abs/2205.09991
- [3]A. Ajay, Y. Du, A. Gupta, J. B. Tenenbaum, T. S. Jaakkola, and P. Agrawal, “Is conditional generative modeling all you need for decision making?” in The Eleventh International Conference on Learning Representations, 2023.arxiv.org/abs/2211.15657
- [4]C. Zhu, R. Yu, S. Feng, B. Burchfiel, P. Shah, and A. Gupta, “Unified world models: Coupling video and action diffusion for pretraining on large robotic datasets,” in Proceedings of Robotics: Science and Systems, 2025.
- [5]B. Chen, D. Martí Monsó, Y. Du, M. Simchowitz, R. Tedrake, and V. Sitzmann, “Diffusion forcing: Next-token prediction meets full-sequence diffusion,” Advances in Neural Information Processing Systems, vol. 37, pp. 24 081–24 125, 2024.arxiv.org/abs/2407.01392
- [6]R. Jonchowskij, D. Rastogi, and O. Brock, “Differentiable particle filters: End-to-end learning with algorithmic priors,” in Proceedings of Robotics: Science and Systems, Pittsburgh, Pennsylvania, Jun. 2018.
- [7]P. Karkus, D. Hsu, and W. S. Lee, “Particle filter networks with application to visual localization,” in Proceedings of the 2nd Conference on Robot Learning, ser. Proceedings of Machine Learning Research, vol. 87. PMLR, 2018, pp. 169–178.arxiv.org/abs/1805.08975
- [8]Z. Wan and L. Zhao, “DiffPF: Differentiable particle filtering with generative sampling via conditional diffusion models,” IEEE Robotics and Automation Letters, vol. 11, no. 3, pp. 3166–3173, 2026.arxiv.org/abs/2507.15716
- [9]D. Silver and J. Veness, “Monte-carlo planning in large pomdps,” in Advances in Neural Information Processing Systems, vol. 23, 2010, pp. 2164–2172.
- [10]Z. N. Sunberg and M. J. Kochenderfer, “Online algorithms for pomdps with continuous state, action, and observation spaces,” in Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, 2018, pp. 259–263.arxiv.org/abs/1709.06196
- [11]M. Du and S. Song, “DynaGuide: Steering diffusion policies with active dynamic guidance,” in Advances in Neural Information Processing Systems, vol. 38, 2025.arxiv.org/abs/2506.13922
- [12]R. Singhal, Z. Horvitz, R. Teehan, M. Ren, Z. Yu, K. McKeown, and R. Ranganath, “A general framework for inference-time scaling and steering of diffusion models,” in Forty-second International Conference on Machine Learning, 2025.arxiv.org/abs/2501.06848
- [13]N. J. Gordon, D. J. Salmond, and A. F. Smith, “Novel approach to nonlinear/non-gaussian bayesian state estimation,” in IEE Proceedings F (Radar and Signal Processing), vol. 140, no. 2. IET, 1993, pp. 107–113.DOI
- [14]K. Song, B. Chen, M. Simchowitz, Y. Du, R. Tedrake, and V. Sitzmann, “History-guided video diffusion,” in Forty-second International Conference on Machine Learning, 2025.arxiv.org/abs/2502.06764
- [15]R. C. Coulter, “Implementation of the pure pursuit path tracking algorithm,” The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, Tech. Rep. CMU-RI-TR-92-01, Jan. 1992.
- [16]W. Peebles and S. Xie, “Scalable diffusion models with transformers,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023, pp. 4172–4182.arxiv.org/abs/2212.09748
- [17]A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y. Zhu, and R. Martín-Martín, “What matters in learning from offline human demonstrations for robot manipulation,” in Conference on Robot Learning (CoRL), 2021.arxiv.org/abs/2108.03298
