betaFeedback

WorksComputer Science

DroneDiffusion: Uncertainty-Guided Diffusion Path Planner for UAV Navigation

Abstract

We present DroneDiffusion, a perception-aware diffusion planner for local 3-D UAV trajectory generation that explicitly models and avoids failure zones caused by lens flare, overexposure, and reflective surfaces. A vision-language perceptual safety estimator produces a pixel-aligned reliability map that highlights regions of degraded sensing. A denoising diffusion probabilistic model (DDPM) is conditioned on this map together with the onboard RGB image, goal point, and initial velocity to generate egocentric 3-D trajectories. To enforce obstacle avoidance at inference time without retraining, we apply training-free CHOMP guidance over a 3-D Euclidean signed distance field. The model is trained entirely in simulation and deployed zero-shot on a real drone. Reliability conditioning raises the mean perceptual reliability along generated trajectories to 0.71, compared with 0.63 for a geometric baseline, while guidance halves the collision rate of the learned prior from 14.7% to 7.8%. Perceptual safety conditioning and geometric guidance are therefore complementary: treating sensor reliability as a first-class signal reduces incursions into unreliable regions while maintaining competitive navigation performance.

Introduction

Autonomous UAV navigation in cluttered indoor environments exposes onboard sensors to perceptual hazards that geometric planners routinely ignore: overexposed corridors, lens flare, transparent walls, and specular surfaces. A planner unaware of such failures may route a drone confidently through a region where the sensor stream cannot be trusted.

Diffusion models have recently emerged as expressive trajectory generators for navigation and manipulation, offering flexible multimodal conditioning and strong generalization from data [1], [2], [3]. Yet existing diffusion planners condition only on geometric and semantic cues such as RGB images, goal specifications, or occupancy maps. They provide no mechanism to reason about input reliability when the image is saturated or the depth measurements are invalid. Vision-language tools such as CLIPSeg [4] and AnyTraverse [5] can detect such failure zones, but have not been integrated into trajectory-level planning.

We present a diffusion planner for 3-D UAV trajectory generation that treats a perceptual safety map as a first-class conditioning signal alongside the RGB image, goal point, and initial velocity. A CLIPSeg-based estimator produces a pixel-aligned reliability map that penalizes overexposure, lens flare, and transparent surfaces. A DDPM denoiser generates egocentric 4-DoF paths (x,y,z,ψ)(x,y,z,\psi) of 24 waypoints at

10 Hz conditioned on this map, naturally steering away from unreliable regions. Training-free CHOMP guidance on a 3-D ESDF enforces geometric obstacle avoidance at inference time. The model trains in simulation and transfers to real drone flights without fine-tuning.

The main contributions are:

  • A perceptual safety map formulation that encodes sensor failure zones as a spatial reliability prior for diffusion trajectory planning.

  • A conditional DDPM planner for 3-D UAV navigation jointly conditioned on visual context and perceptual reliability, with sim-to-real transfer validation.

  • A training-free guidance module enforcing obstacle avoidance and goal reaching without retraining.

UAV Trajectory Planning

Local trajectory planning for UAVs has been addressed through gradient-based optimization [6], sampling-based methods [7], and Model Predictive Control [8]. These approaches generate smooth, collision-free trajectories from explicit geometric representations such as a signed distance field, but each query returns a single solution rather than a distribution over feasible behaviors.

Diffusion Models for Robot Navigation

Diffusion models have demonstrated strong performance as trajectory generators for robot navigation. Diffuser [2] shows that diffusion policies can plan over long horizons from offline data. NoMaD [3] extends this to goal-conditioned exploration using a masked diffusion policy over image observations. DTG [9] applies diffusion to mapless outdoor navigation with traversability-aware losses. For UAVs specifically, CGD [10] incorporates constraint guidance into a diffusion planner for obstacle avoidance, while related guidance schemes have been applied to other platforms, such as autonomous racing [11].

More recently, aerial diffusion planners have been extended along further conditioning axes, including swarm path generation with generative impedance control [12], an agentic reasoning layer above the planner [13], and goals inferred from a detected person for search and rescue [14]. Across all of this work, the visual input itself is treated as trustworthy. Our work shares the guidance-based inference paradigm of these methods but introduces perceptual reliability as a conditioning modality, which none of them address.

Traversability and Safety-Aware Navigation

Traversability estimation has long been studied as a precondition for safe robot motion in unstructured environments. Recent work has shown that coupling traversability directly into the diffusion generation process, rather than treating it as a post-hoc filter, yields more robust and consistent trajectories. EmbodiedDiffusion [15] demonstrates that safety awareness through traversability conditioning plays a vital role in navigation: jointly generating traversability maps and trajectories from a single RGB image achieves strong sim-to-real transfer across heterogeneous platforms. Our work extends this direction in two respects. Rather than encoding terrain-level traversability, we model sensor-level perceptual failures as the source of risk. Additionally, trajectory generation and guidance operate in full 3-D space, addressing the geometric limitations of image-space planning for aerial platforms.

Perceptual Failure Detection

Detecting regions of degraded sensing has been studied in the context of robust perception pipelines. Vision-language models enable open-vocabulary scene understanding [16], and CLIPSeg [4] extends this to dense pixel-level segmentation from text prompts. AnyTraverse [5] leverages CLIPSeg to produce prompt-driven traversability maps for ground robots, covering concepts such as glass surfaces and overexposed regions. These tools have not previously been used to condition trajectory-level planning. We repurpose the AnyTraverse framework specifically to identify sensor failure zones and encode them as a spatial prior for a diffusion-based 3-D planner.

Methods

We propose a novel diffusion-based local UAV path-planning pipeline that accounts for both perceptual safety and geometric obstacles (Figure 2). It consists of three blocks: a perceptual safety estimator that predicts reliable regions from onboard observations; a conditional diffusion planner that generates trajectories based on the current UAV state, observation, and estimated safety map; and a cost-guidance module that enforces collision avoidance.

DroneDiffusion pipeline

Figure 2. DroneDiffusion pipeline. Left: the corrupted trajectory xtx_t is denoised over nn reverse-diffusion steps by a conditional 1-D U-Net, conditioned on the RGB observation, the AnyTraverse reliability map, the initial speed and the goal point, and corrected at each step by training-free obstacle and goal guidance. Right: scene and reliability tokens enter the U-Net bottleneck through two sequential cross-attention blocks, which keeps appearance and perceptual-reliability reasoning separate.

The model generates an egocentric path of H=24H = 24 states sampled at 10 Hz:

x={(xi,yi,zi,sin⁡ψi,cos⁡ψi)}i=1H,(1)x = \left\{(x_i, y_i, z_i, \sin\psi_i, \cos\psi_i)\right\}_{i=1}^{H}, \tag*{(1)}

where HH is the planning horizon, (xi,yi,zi)(x_i,y_i,z_i) are the metric coordinates of the ii-th waypoint in the egocentric frame, and ψi\psi_i is its heading. Each state therefore has four degrees of freedom, (x,y,z,ψ)(x,y,z,\psi), stored as five values because the heading is encoded as the unit vector (sin⁡ψi,cos⁡ψi)(\sin\psi_i,\cos\psi_i) instead of a raw angle, which avoids angle wrapping and preserves the smoothness of the training data.

Perceptual Safety

Certain scene conditions degrade the RGB and depth streams: overexposure and lens flare saturate the image and corrupt depth returns, while transparent surfaces stay invisible to the depth sensor and specular ones produce phantom geometry from reflected scene content. Because the onboard camera is the primary sensing modality for the UAV, such degradation propagates directly into planning. A planner that reads such observations at face value may confidently route the UAV through regions where the sensor data cannot be trusted. Since these failures are visible in the image itself, we exploit the ability of diffusion models to handle multimodal conditioning and feed the planner a pixel-aligned reliability map alongside the RGB observation.

This map is produced using AnyTraverse [5]. However, it is not used to highlight traversable zones but rather potential perceptual failure zones. Built on CLIPSeg, the pipeline pools per-prompt responses with weights ωi\omega_i; a weight of 0 marks a concept as explicitly safe. The prompts for failure modes are listed in Table I.

PromptWeight ωi\omega_i
Transparent and specular geometry
a window1.0
a glass surface1.0
a transparent surface1.0
a transparent obstacle1.0
a mirror1.0
Glare and reflectivity
strong light glare0.9
a specular highlight0.9
a bright reflection0.9
highly reflective surfaces0.9
a glossy surface0.8
Negative anchors
a white wall0.0

Table I. Perceptual uncertainty prompts and pooling weights ω\omega used in AnyTraverse.

Since the prompts describe failure modes rather than traversable terrain, the raw AnyTraverse output is a risk map that is high where a prompt matches and zero elsewhere. Inverting it gives the pixel-aligned reliability map MM:

M=1−AnyTraverse⁡(I;ω),M∈[0,1]h×w,(2)M = 1 - \operatorname{AnyTraverse}(I;\omega), \quad M \in[0,1]^{h \times w}, \tag*{(2)}

where II is the input RGB image, ω\omega is the vector of concept weights from Table I, and hh and ww are the image height and width in pixels. Each entry of MM scores how reliable the corresponding pixel is, with 1 denoting fully reliable sensing and 0 a perceptual failure zone. The resulting heatmap then steers trajectory generation toward reliable zones (Figure 1).

Perception failures in indoor UAV flight

Figure 1. Perception failures in indoor UAV flight. Left: RGB observation with severe lens flare. Middle: corresponding depth map with missing and phantom geometry in the corrupted region. Right: our safety map flags the unreliable zone (red), guiding trajectory generation (magenta) away from it.

Diffusion Planning

Diffusion models [17] are a family of generative models that iteratively generate samples from Gaussian noise. The pipeline consists of two stages: forward and reverse.

The forward process is a Markov chain that injects Gaussian noise at each step according to a predefined schedule {βt}t=1T\{\beta_t\}_{t=1}^{T}:

q(xt∣xt−1)=N(xt;1−βtxt−1,βtI),(3)q(x_t \mid x_{t-1}) = \mathcal{N}\left(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t\mathbf{I}\right), \tag*{(3)}

which admits a closed-form marginal conditioned on x0x_0:

q(xt∣x0)=N(xt;αˉtx0,(1−αˉt)I),(4)q(x_t \mid x_0) = \mathcal{N}\left(x_t; \sqrt{\bar{\alpha}_t}x_0, (1-\bar{\alpha}_t)\mathbf{I}\right), \tag*{(4)}

with αt=1−βt\alpha_t = 1-\beta_t and αˉt=∏s=1tαs\bar{\alpha}_t = \prod_{s=1}^{t}\alpha_s. This allows sampling xtx_t directly from x0x_0 as:

xt=αˉtx0+1−αˉt ε,ε∼N(0,I).(5)x_t = \sqrt{\bar{\alpha}_t}x_0 + \sqrt{1-\bar{\alpha}_t}\,\varepsilon,\quad\varepsilon\sim\mathcal{N}(0,\mathbf{I}). \tag*{(5)}

The reverse process inverts (5) step by step. However, without a clean sample it cannot be computed directly. Instead, a network εθ(xt,t)\varepsilon_\theta(x_t,t) is trained to approximate the noise ε\varepsilon injected into xtx_t at noise level tt, by minimizing:

Ldiff=Et,xt,ε[∥ε−εθ(xt,t)∥22].(6)\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_{t,x_t,\varepsilon}\left[\left\|\varepsilon-\varepsilon_\theta(x_t,t)\right\|_2^2\right]. \tag*{(6)}

Given this prediction, a single reverse step is performed as:

xt−1=1αt(xt−βt1−αˉtεθ(xt,t))+σtz,(7)x_{t-1} = \frac{1}{\sqrt{\alpha_t}}\left(x_t-\frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}}\varepsilon_\theta(x_t,t)\right)+\sigma_t z, \tag*{(7)}

where z∼N(0,I)z \sim\mathcal{N}(0,\mathbf{I}) if t>1t>1, else z=0z=\mathbf{0}. Generation starts from xT∼N(0,I)x_T \sim\mathcal{N}(0,\mathbf{I}) and applies (7) iteratively for t=T,T−1,…,1t=T,T-1,\ldots,1.

The formulation above describes unconditional generation, in which samples are drawn from the learned data distribution pθ(x0)p_\theta(x_0) without any external control. For path planning, generated samples must be consistent with the current task and environment. This is achieved by conditioning the model on an additional signal cc, yielding εθ(xt,t,c)\varepsilon_\theta(x_t,t,c). The resulting architecture is shown in Figure 2, and the conditioning signals used for planning are:

  • Initial speed for start dynamics;

  • Goal point for goal pursuit;

  • RGB image for global context;

  • Safety map for highlighting unreliable regions.

The denoiser is implemented as a 1-D U-Net with three resolution levels, chosen for its low inference latency given the short planning horizon. Scalar and vector conditions (initial speed, goal) are injected via Feature-wise Linear Modulation (FiLM) [18] in every residual block (in stages of downsampling, bottleneck, and upsampling), while spatially structured inputs are injected via two sequential cross-attention blocks at the U-Net bottleneck: frozen CLIP ViT-B/16 [16] patch tokens from the RGB image provide scene-level semantic context, and patch tokens from a lightweight safety-map encoder provide the perceptual reliability signal (Figure 2). Using two independent cross-attention blocks allows the model to separately reason about scene appearance and perceptual safety without conflating the two signals.

To further shape the generated trajectory, several auxiliary losses are applied to the estimated clean sample x^0\hat{x}_0, reconstructed from the noisy sample and predicted noise. The reconstruction loss Lrec\mathcal{L}_{\mathrm{rec}} is the mean squared error between x^0\hat{x}_0 and the ground-truth trajectory and reinforces the ability of the network to recover the clean sample. The smoothness loss Lsm\mathcal{L}_{\mathrm{sm}} suppresses jerky motion through the squared second finite difference of the waypoint sequence, which acts as an acceleration proxy. The origin-anchor loss Lor\mathcal{L}_{\mathrm{or}} keeps the trajectory egocentric by pulling the first state to the UAV-frame origin [0,0,0,0,1][0,0,0,0,1]. The speed-consistency loss Lsp\mathcal{L}_{\mathrm{sp}} matches the finite-difference displacement between the first two waypoints to the observed initial velocity v0v_0 scaled by the sampling interval. The yaw-geometry loss Lyaw\mathcal{L}_{\mathrm{yaw}} keeps the predicted (sin⁡ψ,cos⁡ψ)(\sin\psi,\cos\psi) pair on the unit circle and aligns the heading with both the ground-truth yaw and the direction of motion along moving segments.

The perceptual-safety loss Lreliab\mathcal{L}_{\mathrm{reliab}} couples the trajectory to the reliability map MM. Each predicted waypoint is projected into the image plane through the camera intrinsics, and its reliability si∈[0,1]s_i \in[0,1] is sampled from MM as the mean over a perspective-aware n×nn \times n kernel matching the projected UAV footprint at the waypoint depth. Reliability is treated as a soft constraint, so each in-frame waypoint receives the penalty:

ρ(si)=τsoftplus⁡(sthr−siτ)(8)\rho(s_i)=\tau\operatorname{softplus}\left(\frac{s_{\mathrm{thr}}-s_i}{\tau}\right) \tag*{(8)}

where sthrs_{\mathrm{thr}} is the reliability threshold and τ\tau is the temperature. Waypoints above sthrs_{\mathrm{thr}} incur a vanishing penalty, while those below it are penalized almost linearly in the deficit. The full term averages this penalty over the in-frame waypoints, adds a conditional tail contribution over the least reliable fraction so that a few dangerous states are not masked by an otherwise safe path, and includes a smooth field-of-view penalty ρfov\rho_{\mathrm{fov}} that pulls back projections leaving the frame or falling behind the camera:

Lreliab=ρˉ+wtρtail1+wt+wfovρfov.(9)\mathcal{L}_{\mathrm{reliab}}=\frac{\bar{\rho}+w_t\rho_{\mathrm{tail}}}{1+w_t}+w_{\mathrm{fov}}\rho_{\mathrm{fov}}. \tag*{(9)}

Combined with the main diffusion loss Ldiff\mathcal{L}_{\mathrm{diff}} (6), the complete training objective is a weighted sum of the losses. The auxiliary terms are applied only to the samples drawn from the lower 30% of the noise schedule, where the clean estimate is accurate enough to supervise trajectory geometry.

Guidance

The denoiser provides a trajectory prior but does not guarantee collision avoidance in unseen scenes. Collision constraints are introduced through training-free CHOMP guidance [19] on the estimated clean manifold sample (ECM) [20]:

x^0=xt−1−αˉt ϵθ(xt,t,c)αˉt.(10)\hat{x}_0=\frac{x_t-\sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t,t,c)}{\sqrt{\bar{\alpha}_t}}. \tag*{(10)}

Its position channels define the trajectory P=[p1,…,pH]T\mathbf{P}=[p_1,\ldots,p_H]^{\mathsf{T}}, where pi∈R3p_i\in\mathbb{R}^3 is the ii-th waypoint. Guidance corrects positions only; the heading channels remain as produced by the denoiser. Collision geometry is represented by a 3-D Euclidean Signed Distance Field (ESDF) [21] ϕ(p)\phi(p) derived from the current depth point cloud. The field is positive in free space, zero at an obstacle surface, and negative inside an obstacle. With the UAV radius and safety margin represented by rclrr_{\mathrm{clr}}, the signed clearance is:

di=ϕ(pi)−rclr.(11)d_i=\phi(p_i)-r_{\mathrm{clr}}. \tag*{(11)}

The CHOMP obstacle potential is:

c(di)={0,di≥η,(di−η)22η,0<di<η,−di+η2,di≤0,Cobs(P)=∑i=1Hc(di),(12)c(d_i)= \begin{cases} 0, & d_i\ge\eta,\\ \frac{(d_i-\eta)^2}{2\eta}, & 0<d_i<\eta,\\ -d_i+\frac{\eta}{2}, & d_i\le0, \end{cases} \qquad C_{\mathrm{obs}}(P)=\sum_{i=1}^{H}c(d_i), \tag*{(12)}

where η\eta is the obstacle influence distance. The cost is zero beyond η\eta, quadratic in the near-obstacle free-space region, and linear inside an obstacle. The complete objective also includes smoothness, speed, and goal terms:

C(P)=sobsCobs(P)+λsCsmooth(P)+λvCspd(P)+λgCgoal(P).(13)C(P)=s_{\mathrm{obs}}C_{\mathrm{obs}}(P)+\lambda_s C_{\mathrm{smooth}}(P) +\lambda_v C_{\mathrm{spd}}(P)+\lambda_g C_{\mathrm{goal}}(P). \tag*{(13)}

Guidance

Directional weighting of the metric-space gradient G=∇PC(P)G=\nabla_P C(P) is defined by the anisotropy vector A=[ax,ay,az]A=[a_x,a_y,a_z]:

G~=G⊙A.(14)\widetilde{G}=G\odot A. \tag*{(14)}

Since guidance acts on positions alone, AA is three-dimensional, and its coefficients independently determine the relative contribution of forward, lateral, and vertical corrections. Independent waypoint gradients may produce sharp or alternating corrections. Temporal coupling is represented by:

Λ=ρ0I+ρ2D2TD2+ρ3D3TD3,(15)\Lambda=\rho_0 I+\rho_2D_2^{\mathsf{T}}D_2+\rho_3D_3^{\mathsf{T}}D_3, \tag*{(15)}

where D2D_2 and D3D_3 are the second- and third-order finite-difference operators. The inverse metric gives the smoothed gradient G~=Λ−1G~\widetilde{G}=\Lambda^{-1}\widetilde{G}, coupling each correction to neighboring waypoints. At least the first waypoint remains unchanged to preserve the trajectory origin. For warm-started generation, more initial waypoints can be fixed to maintain temporal consistency with the previous plan. If a correction exceeds a predefined metric limit, all waypoint corrections are reduced by the same factor. This common scaling bounds each guidance step while preserving the relative shape of the smoothed trajectory. The resulting path P∗P^* defines the guided clean estimate x^0∗\hat{x}_0^*, and the next reverse state retains its effective noise:

ϵ~t=xt−αˉtx^0∗1−αˉt,xt′=αˉt′x^0∗+1−αˉt′ϵ~t.(16)\widetilde{\epsilon}_t=\frac{x_t-\sqrt{\bar{\alpha}_t}\hat{x}_0^*}{\sqrt{1-\bar{\alpha}_t}}, \qquad x_{t'}=\sqrt{\bar{\alpha}_{t'}}\hat{x}_0^*+\sqrt{1-\bar{\alpha}_{t'}}\widetilde{\epsilon}_t. \tag*{(16)}

The goal term λgCgoal\lambda_g C_{\mathrm{goal}} in (13) provides goal attraction. It is defined as:

Cgoal(P)=∥pH−g∥2,(17)C_{\mathrm{goal}}(P)=\lVert p_H-g\rVert^2, \tag*{(17)}

which penalizes the distance between the trajectory endpoint pHp_H and the local goal gg. Its gradient acts only on the terminal waypoint, but the inverse metric Λ−1\Lambda^{-1}, derived from (15), distributes the correction over the whole trajectory, so the path bends toward the goal coherently instead of forming a sharp terminal kink. Because the obstacle weight sobss_{\mathrm{obs}} dominates λg\lambda_g, collision avoidance retains priority over goal attraction.

Evaluation

Dataset and Training

The diffusion planner was trained on synthetic warehouse data generated in NVIDIA Isaac Sim [22] with Pegasus [23]. The dataset comprises ∼5,000\sim5{,}000 RGB-D observation frames across eight warehouse scenes, expanded to ∼20,000\sim20{,}000 trajectory samples. Each sample contains an RGB observation, a pixel-aligned perceptual reliability map from AnyTraverse inference, an initial velocity, a goal point, and an egocentric trajectory of HH states, where each state is defined by [x,y,z,sin⁡ψ,cos⁡ψ][x,y,z,\sin\psi,\cos\psi]. The training-validation split was approximately 5:1. It was performed at the environment level, so that frames from the same or a similar point of view never appeared in both training and validation. The trajectory and velocity normalization ranges were estimated from the training split only.

The denoiser is the 1-D U-Net described above, conditioned on frozen CLIP ViT-B/16 tokens and a lightweight safety-map embedding. It was trained with AdamW under FP16 mixed precision for 100 epochs at a learning rate of 10−410^{-4} and batch size 128. The forward process used 1000 diffusion steps with a squared-cosine schedule and epsilon prediction. The reliability condition was dropped with probability 0.10 so that the planner remains usable when no safety map is available.

Hardware and Sensing

Training and inference ran on an NVIDIA RTX 4070 Super (12 GB). Onboard sensing used an Intel RealSense D455 providing 640×480640 \times480 RGB-D, from which the depth stream feeds the ESDF. The ground server and UAV operated within the same ROS 2 domain.

Inference and Guidance Configuration

The sampling, refinement, and geometry settings are summarized in Table II. The obstacle coefficient remains constant during refinement; no timestep-dependent strength schedule is used.

ParameterValueParameterValue
Cold reverse steps500Refinement start tt300
Refinement steps11ESDF voxel size0.05m
UAV radius0.25mSafety margin0.10m
CHOMP η\eta0.7mObstacle coefficient0.75
Maximum correction0.08mAnisotropy AA[0.8, 1.0, 1.0]
ρ0,ρ2,ρ3\rho_0,\rho_2,\rho_31.0, 1.0, 0.2Goal weight λg\lambda_g0.25
Smoothness λs\lambda_s1.0Speed λv\lambda_v0.5

Table II. Diffusion and geometric-guidance configuration.

The local ESDF covers x∈[−0.1,5.0]x \in[-0.1,5.0], y∈[−3.0,3.0]y \in[-3.0,3.0], and z∈[−1.5,2.0]z \in[-1.5,2.0] m in the egocentric FLU frame.

Evaluation Protocol and Metrics

Evaluation observations came from a controlled drone flight in which trajectories were planned online by the proposed method from the onboard RGB-D stream; the reported statistics were obtained from four additional seeds replayed offline on these observations. While planning uses only the partial onboard view, collisions are verified against a precise 3-D scan of the entire arena that serves as ground truth. Each generated trajectory is transformed from the local FLU frame into the global map frame using the corresponding camera pose, sampled every 0.02m, and queried against this scan within the camera field of view; a sample is colliding when its nearest-map distance is below the collision-body radius in Table II.

The following planner variants are evaluated on identical observations, goals, and random seeds (for diffusion):

  • Full (Ours): warm-started generation with ESDF guidance and safety-map conditioning;

  • Without guidance: the same planner with geometric guidance disabled;

  • Without guidance and reliability: guidance disabled and the safety map replaced by an all-reliable map, so perceptual conditioning is also removed;

  • Artificial Potential Field (APF) [24]: a geometric non-diffusion baseline evaluated with the same collision model.

Collision rate is the fraction of generated trajectories containing at least one colliding interpolated sample, and minimum clearance is the smallest nearest-map distance along a trajectory. Goal error is the Euclidean distance between the final waypoint and the specified local goal. Reliability is evaluated by projecting waypoints onto the reliability map: the mean and the lower-tail (worst-case) reliability are reported, and the unsafe-waypoint rate is the fraction of projected waypoints below a reliability threshold (0.2). Jerk is measured from finite differences of the waypoint sequence, so lower values indicate smoother motion.

Quantitative Results

Table III reports the ablation results over five diffusion seeds. The learned planner provides a useful prior for safe and perceptually reliable trajectory generation. Adding geometric guidance reduces the collision rate from 14.71% to 7.77%, while leaving perceptual reliability approximately unchanged. This improvement comes at the cost of increased goal error and higher jerk, despite including goal and smoothness terms in the guidance objective. The APF baseline produces shorter and smoother geometric paths but does not account for perceptual reliability. Overall, DroneDiffusion improves perceptual safety relative to the geometric baseline while remaining competitive in collision avoidance.

MetricFull (Ours)w/o Guid.w/o Guid. and Reliab.APF
Collision (%) ↓\downarrow7.770 ±\pm 1.15014.710 ±\pm 2.27013.320 ±\pm 1.3206.650 ±\pm 0.000
Min. clearance (m) ↑\uparrow0.618 ±\pm 0.0130.602 ±\pm 0.0140.610 ±\pm 0.0160.613 ±\pm 0.000
Unsafe WP (%) ↓\downarrow12.890 ±\pm 1.55014.590 ±\pm 1.78016.590 ±\pm 1.43020.520 ±\pm 0.000
Mean reliability ↑\uparrow0.714 ±\pm 0.0160.730 ±\pm 0.0130.689 ±\pm 0.0150.626 ±\pm 0.000
Tail reliability ↑\uparrow0.522 ±\pm 0.0330.538 ±\pm 0.0050.518 ±\pm 0.0180.390 ±\pm 0.000
Goal error (m) ↓\downarrow0.723 ±\pm 0.0120.610 ±\pm 0.0110.666 ±\pm 0.0140.601 ±\pm 0.000
Path length (m) ↓\downarrow3.352 ±\pm 0.0143.042 ±\pm 0.0053.043 ±\pm 0.0052.687 ±\pm 0.000
Jerk ↓\downarrow0.0690 ±\pm 0.00080.0150 ±\pm 0.00020.0150 ±\pm 0.00020.0050 ±\pm 0.0000

Table III. Trajectory-generation results on real drone observations, averaged over five seeds (mean and std across seeds, APF is deterministic). Best per row in bold.

Runtime Evaluation

Table IV shows that AnyTraverse dominates runtime; the complete-pipeline mean corresponds to a trajectory-generation rate of approximately 5.51 Hz. Reliability reasoning is the main latency bottleneck.

StageMeanMin.Max.Share
ESDF18.3816.8421.1710.13%
AnyTraverse122.14117.13125.2367.35%
SceneEncoder5.235.065.782.88%
Generation + guidance (11 steps)35.6133.2838.3519.63%
Full pipeline (warm start)181.36173.87186.21100%

Table IV. Runtime breakdown on an NVIDIA RTX 4070 Super (12 GB).

Time dimension is in milliseconds.

Conclusion and Future Work

We presented a perception-aware diffusion planner for safe UAV trajectory generation in indoor environments. The key idea is to condition a lightweight 1-D U-Net denoiser jointly on visual scene context and a dense perceptual reliability map and to couple it with training-free ESDF-based cost guidance for obstacle avoidance. This design lets the planner reason simultaneously about what is geometrically free and what is perceptually trustworthy, a distinction that classical occupancy-based planners do not make.

Ablation experiments confirm that both components contribute. Safety-map conditioning internalizes general safety into the prior, so that even with the map removed at inference the planner stays more perceptually reliable than a deterministic geometric baseline. CHOMP-style guidance roughly halves the collision rate on unseen obstacle configurations, at the cost of a moderate decrease in trajectory quality. More broadly, the results suggest that explicitly reasoning about sensor reliability, rather than treating all visual input as uniformly trustworthy, is a promising direction at the intersection of diffusion-based planning and perception-aware robotics.

Several limitations remain. The training data is entirely synthetic and limited to warehouse scenes, and real-world evaluation was conducted only in a controlled environment; deployment in uncontrolled, visually diverse settings remains untested. End-to-end latency is dominated not by the diffusion model but by the AnyTraverse reliability estimator, which accounts for about two-thirds of the pipeline latency. Cold-start generation is also expensive: it relies on a 500-step reverse chain, which is reduced to a guided-refinement schedule to keep replanning tractable. Finally, the reliability signal is static per frame and does not model dynamic obstacles or rapidly changing illumination.

Future work targets three directions: (i) attacking latency at its true bottleneck by distilling or replacing the AnyTraverse estimator, together with fast sampling via consistency distillation or flow matching to bring end-to-end latency below 100 ms; (ii) replacing the static reliability map with direct per-frame uncertainty estimation; and (iii) extending the goal-conditioned local planner toward long-horizon operation, including goal-free exploration in which the planner navigates and discovers unmapped environments without a specified target.

Acknowledgment

Research reported in this publication was financially supported by the RSF grant No. 24-41-02039.

References

  1. [1]C. Chi et al., “Diffusion policy: Visuomotor policy learning via action diffusion,” Int. J. Robot. Res., vol. 44, no. 10–11, pp. 1684–1704, Sep. 2025.
  2. [2]M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” in Proc. Int. Conf. on Machine Learning (ICML), Jul. 17–23, 2022, pp. 9902–9915.arxiv.org/abs/2205.09991
  3. [3]A. Sridhar, D. Shah, C. Glosson, and S. Levine, “NoMaD: Goal masked diffusion policies for navigation and exploration,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), May 13–17, 2024, pp. 63–70.arxiv.org/abs/2310.07896
  4. [4]T. Lüddecke and A. Ecker, “Image segmentation using text and image prompts,” in Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Jun. 19–24, 2022, pp. 7076–7086.arxiv.org/abs/2112.10003
  5. [5]S. Sahu, A. Singh, K. Nambiar, S. Saripalli, and P. B. Sujit, “Any-Traverse: An off-road traversability framework with VLM and human operator in the loop,” 2025, arXiv:2506.16826.arxiv.org/abs/2506.16826
  6. [6]B. Zhou, F. Gao, L. Wang, C. Liu, and S. Shen, “Robust and efficient quadrotor trajectory generation for fast autonomous flight,” IEEE Robot. Autom. Lett., vol. 4, no. 4, pp. 3529–3536, Oct. 2019.arxiv.org/abs/1907.01531
  7. [7]S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning,” Int. J. Robot. Res., vol. 30, no. 7, pp. 846–894, Jun. 2011.arxiv.org/abs/1105.1186
  8. [8]M. Kamel, T. Stastny, K. Alexis, and R. Siegwart, “Model predictive control for trajectory tracking of unmanned aerial vehicles using robot operating system,” in Robot Operating System (ROS): The Complete Reference (Volume 2), A. Koubaa, Ed. Cham, Switzerland: Springer, 2017, pp. 3–39.DOI
  9. [9]J. Liang, A. Payandeh, D. Song, X. Xiao, and D. Manocha, “DTG: Diffusion-based trajectory generation for mapless global navigation,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Oct. 14–18, 2024, pp. 5340–5347.arxiv.org/abs/2403.09900
  10. [10]K. Kondo et al., “CGD: Constraint-guided diffusion policies for UAV trajectory planning,” 2024, arXiv:2405.01758.arxiv.org/abs/2405.01758
  11. [11]H. Ma, S. Bodmer, A. Carron, M. Zeilinger, and M. Muehlebach, “Constraint-aware diffusion guidance for robotics: Real-time obstacle avoidance for autonomous racing,” in Proc. Conf. on Robot Learning (CoRL), Sep. 27–30, 2025, pp. 1756–1776.arxiv.org/abs/2505.13131
  12. [12]F. Batool, Y. Yaqoot, M. A. Mustafa, R. A. Khan, A. Fedoseev, and D. Tsetserukou, “ImpedanceDiffusion: Diffusion-based global path planning for UAV swarm navigation with generative impedance control,” in Proc. IEEE Int. Conf. on Unmanned Aircraft Systems (ICUAS), Jun. 15–18, 2026, pp. 537–544.
  13. [13]F. Batool, M. A. Mustafa, F. Mehboob, V. Serpiva, and D. Tsetserukou, “AgenticDiffusion: Agentic diffusion-based path planning for vision-based UAV navigation,” 2026, arXiv:2606.04111.
  14. [14]F. Batool et al., “HumanDiffusion: A vision-based diffusion trajectory planner with human-conditioned goals for search and rescue UAV,” in Proc. ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI), Mar. 16–19, 2026, pp. 1023–1027.
  15. [15]I. Zhura et al., “EmbodiedDiffusion: End-to-end traversability-guided visual diffusion for heterogeneous robot navigation,” 2025, arXiv:2512.02851.DOI
  16. [16]A. Radford et al., “Learning transferable visual models from natural language supervision,” in Proc. Int. Conf. on Machine Learning (ICML), Jul. 18–24, 2021, pp. 8748–8763.arxiv.org/abs/2103.00020
  17. [17]J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), Dec. 6–12, 2020, pp. 6840–6851.DOI
  18. [18]E. Perez, F. Strub, H. de Vries, V. Dumoulin, and A. Courville, “FiLM: Visual reasoning with a general conditioning layer,” in Proc. AAAI Conf. on Artificial Intelligence (AAAI), Feb. 2–7, 2018, pp. 3942–3951.
  19. [19]N. Ratliff, M. Zucker, J. A. Bagnell, and S. Srinivasa, “CHOMP: Gradient optimization techniques for efficient motion planning,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), May 12–17, 2009, pp. 489–494.DOI
  20. [20]Y. Wang et al., “Optimizing diffusion models for joint trajectory prediction and controllable generation,” in Proc. European Conf. on Computer Vision (ECCV), Sep. 29–Oct. 4, 2024, pp. 324–341.arxiv.org/abs/2408.00766
  21. [21]H. Oleynikova, Z. Taylor, M. Fehr, R. Siegwart, and J. Nieto, “Voxblox: Incremental 3D Euclidean signed distance fields for onboard MAV planning,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Sep. 24–28, 2017, pp. 1366–1373.DOI
  22. [22]NVIDIA Corporation, “NVIDIA Isaac Sim,” (2023). [Online]. Available: https://developer.nvidia.com/isaac-sim, accessed: Jul. 21, 2026.
  23. [23]M. Jacinto et al., “Pegasus simulator: An Isaac Sim framework for multiple aerial vehicles simulation,” in Proc. IEEE Int. Conf. on Unmanned Aircraft Systems (ICUAS), Jun. 4–7, 2024, pp. 917–922.DOI
  24. [24]O. Khatib, “Real-time obstacle avoidance for manipulators and mobile robots,” Int. J. Robot. Res., vol. 5, no. 1, pp. 90–98, 1986.

Paper details

vlnc:W_vcrfctsg
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents