betaFeedback

WorksComputer Science

CrossLiDAR: Controllable Multi-Sensor LiDAR Sequence Generation

Abstract

High-fidelity LiDAR simulation remains a bottleneck for scalable development and validation of autonomous systems. Existing generators can synthesize realistic scans, but they are typically constrained to a single sensor configuration, offer limited control over structure and geometry, and drift in long-horizon 4D generation. We present CrossLiDAR, a unified transformer-based framework capable of generating controllable and spatio-temporally consistent multi-sensor LiDAR sequences. Our approach leverages a rectified flow objective with a Multi-Modal Diffusion Transformer (MM-DiT) backbone, within a compact latent space of range images. To support multiple LiDAR sensors, an Inter-sensor Module enables interaction across sensors, while a ControlNet-style branch provides structural conditioning (e.g., BEV priors or 3D bounding boxes). For modeling the temporal consistency, CrossLiDAR generates short NN-frame sequences and extends them autoregressively to improve long-horizon coherence. Our model significantly outperforms state-of-the-art baselines, achieving an average improvement of 29.56% in JSD and 15.36% in MMD evaluated on nuScenes and the Waymo Dataset. CrossLiDAR bridges the gap to modern autonomous-driving simulation by enabling calibration-consistent multi-sensor LiDAR rollouts over long horizons for controllable, scenario-based testing.

Introduction

Autonomous systems have become fundamental in modern robotics and industrial innovation. These systems, ranging from self-driving vehicles [47, 66] to surveying drones [42, 69], often rely heavily on LiDAR to navigate complex 3D environments. By providing high-fidelity geometric data, LiDAR enables precise perception; however, it is constrained by the high cost of the sensors and logistical complexity of real-world data collection. Furthermore, capturing critical “corner cases” [71], such as high-impact collisions or extreme meteorological conditions, remains inherently dangerous and rare in physical datasets [55].

Traditional simulation toolkits [7, 20, 30, 33] require labor-intensive manual scene modeling or existing real-world scans, limiting their scalability. Therefore, research has shifted toward generative models to automate this process, despite the inherent challenges of LiDAR data: high sparsity, non-uniformity, and intrinsic geometric structure. Early generators built on Generative Adversarial Networks (GANs) [3, 12] and Variational Autoencoders (VAEs) [17] often produced “fuzzy” geometries and detail loss.

While score-based models like LiDARGen [74] improved fidelity, they suffered from slow sampling speed and poor long-range performance. More recently, Ultra-LiDAR [61] utilized VQ-VAEs [51] in the Bird’s-Eye-View (BEV) to enhance realism. However, processing sparse BEV representations often results in significant computational waste on empty voxels, leading to inefficient generation. Alternatively, RangeLDM [13] and LiDMs [41] leverage Latent Diffusion Models (LDMs) [43] in the range-view to improve speed. However, these frameworks are primarily restricted to fixed sensor architectures and input dimensions, preventing cross-sensor scalability. Furthermore, they lack fine-grained structural guidance (e.g., ControlNet [70]), rendering them impractical for real-world usage, where side LiDAR proves to be more affordable. More recently, LiDARCrafter [22] extended LiDAR generation to 4D scene modeling by employing a tri-branch diffusion framework that generates temporally ordered 3D LiDAR sequences, with time serving as the fourth dimension. While it achieves impressive temporal consistency, the model remains limited to single-LiDAR sensor configurations. In modern autonomous driving architectures, however, configurations with multiple LiDAR sensors have become the industry standard [49]. This shift is driven by the move toward cost-effective sensor redundancy, where multiple low-profile units augment or replace a single high-performance rooftop scanner to eliminate blind spots. Unlike single-sensor setups, these distributed arrays introduce non-trivial geometric disparities and complex overlapping fields of view. Existing 4D models typically lack the architectural mechanisms to perform cross-sensor fusion.

To address these limitations, we introduce CrossLiDAR, a rectified flow model [26] that departs from conventional U-Net-based architectures [44] in favor of a transformer-based design: the Multi-Modal Diffusion Transformer (MM-DiT) [9], see Figure 1. By leveraging the superior modeling capacity and flexibility of transformers [52], CrossLiDAR captures the intricate geometric dependencies of LiDAR sequences with significantly higher fidelity than previous methods. Central to our framework is a specialized Inter-sensor Module designed to decouple the generation process from specific hardware constraints. This module enables our model to handle multiple LiDAR sensors in parallel, modeling their interactions and ensuring geometric consistency, without the need to design separate models for each sensor configuration. This is particularly effective for the increasingly common integration of low-cost side sensors. To provide more fine-grained geometric consistency, we incorporate a ControlNet-style branch, which allows for precise scene manipulation through various priors, including 3D bounding boxes and BEV maps [40]. Finally, to achieve high-fidelity temporal LiDAR generation, CrossLiDAR introduces a LiDAR Sequence Module. Unlike existing methods that focus solely on single-frame generation, our model performs multi-frame generation (e.g., 5-frame sequence). These short-term sequences can then be iteratively extended through an autoregressive mechanism, enabling the synthesis of long-horizon [34], temporally stable LiDAR sequences. To the best of our knowledge, CrossLiDAR serves as the first truly controllable and multi-sensor pipeline for LiDAR simulation.

Figure 1. CrossLiDAR jointly generates synchronized LiDAR sequences for multi-sensor setups. Top: given multiple sensors mounted around the ego vehicle, CrossLiDAR synthesizes geometrically aligned per-sensor views (front/rear/left/right) that can be fused into a consistent ego-frame point cloud, while supporting structural conditions such as 3D bounding boxes. Bottom: our pipeline operates in a VAE-compressed range-image latent space; a rectified-flow module integrates an Inter-Sensor Module for sensor-aware generation and a ControlNet branch for structural conditioning using BEV maps with projected 3D bounding boxes, yielding controllable multi-sensor LiDAR sequences.

We evaluate CrossLiDAR on two large-scale datasets: Waymo Open Dataset [49], which provides the multi-sensor data from peripheral side LiDARs, and nuScenes [4]. Compared to existing baselines, in the image-based setup, CrossLiDAR achieves an average improvement of 29.56% in JSD and 15.36% in MMD, with particularly superior performance in the multi-sensor configurations.

Our main contributions are as follows:

  • We introduce CrossLiDAR, a unified diffusion transformer that enables multi-sensor LiDAR generation via an Inter-sensor Module, and supports controllable synthesis from geometric cues through a ControlNet branch.

  • We present a 4D generation strategy that jointly synthesizes short multi-frame sequences with a LiDAR Sequence Module and extends them autoregressively, reducing temporal drift and improving long-horizon consistency.

  • We show state-of-the-art (SOTA) performance on Waymo (multi-sensor) and nuScenes (single-sensor) for both single-frame generation and LiDAR sequences.

Related Work and Background

Multisensor LiDAR Generation. Generative models for autonomous driving assume a single LiDAR sensor, whereas real vehicles often deploy a heterogeneous suite (e.g., top-mounted long-range and multiple side-mounted sensors) to capture overlapping perspectives. Multi-sensor data generation has been extensively explored in the camera domain, where recent works target the consistent synthesis of multi-camera surround-view [11, 21, 54, 64]. In contrast, to our knowledge, no prior generative model explicitly leverages multiple LiDAR sensors simultaneously. A key technical challenge is that multi-LiDAR point clouds must be carefully aligned: sensors at different positions yield point clouds in distinct coordinate frames. Unless corrected, naively merging multi-LiDAR data leads to distributional artifacts and geometric inconsistencies [13, 59]. Existing LiDAR generators, including LiDARGen [74], RangeLDM [13], and LiDARCrafter [22], operate on a single aggregated point cloud per scene without explicitly addressing sensor-specific geometries. Recent range- and layout-based methods broaden LiDAR generation beyond unconditional scan synthesis. SPIRAL [73] jointly generates range-view depth, reflectance, and semantic maps, while La La LiDAR [27] introduces scene-graph and layout conditioning for controllable object-level LiDAR scene generation. Veila [28] generates panoramic LiDAR from a monocular RGB image, and DynamicCity [1] models dynamic scenes through 4D occupancy generation. However, these methods do not explicitly model synchronized, calibrated multi-LiDAR sensors. Furthermore, other works using the Waymo Open Dataset [49], one of the few available multi-sensor datasets, often restrict themselves to only the front-facing sensor [8, 60, 63, 72]. As a result, there is a need for dedicated architectures employing inter-sensor modules, which maintain the unique geometric properties of individual sensors while ensuring global spatial consistency across the entire sensor suite.

LiDAR Generation Architectures. Beyond sensor configuration, the architectural foundations of LiDAR generative models are rapidly evolving. Previous latent diffusion models (LDMs), such as RangeLDM [13], LiDM [41], and LiDARCrafter [22], rely on conventional 2D convolutional U-Net backbones. These architectures are often constrained by local receptive fields, which can struggle to capture the global context and long-range dependencies inherent in sparse 3D data. To address these structural bottlenecks, MM-DiT [9] architectures offer a more scalable solution. By leveraging self-attention, transformer-based approaches capture spatial and temporal dependencies more effectively than traditional U-Net-based frameworks.

LiDAR Sequence Generation. Point cloud prediction [31, 56, 57, 58] has emerged as a fundamental self-supervised task in autonomous driving, utilizing historical sequences to forecast future spatial states. Early methods focused on estimating scene flow between consecutive frames to project future point cloud sequences [25, 68]. However, these flow-based approaches incur substantial computational costs, limiting their practical applicability in real-time systems. Consequently, recent studies have shifted toward direct prediction from raw point cloud data [29, 37, 56, 57] to improve efficiency. Further advancements include 4D-Occ [16], which formulates the task as geometric occupancy prediction, and ViDAR [65], which leverages historical images to predict future LiDAR scenes. Despite these developments, existing prediction-based frameworks generally require multiple historical frames as input, still struggle with long-horizon LiDAR forecasting, and provide limited controllability for dynamic scene generation. LiDARCrafter [22] addressed long-horizon generation through an autoregressive diffusion framework, but relies on ego-motion-based extrapolation, a strong prior that restricts the diversity of generated sequences. More recently, U4D [62] introduced uncertainty-aware 4D LiDAR generation by allocating generation capacity to challenging regions and improving spatio-temporal coherence. In contrast, CrossLiDAR focuses on controllable multi-sensor LiDAR generation, combining long-horizon temporal consistency with calibration-aware cross-sensor alignment through a ControlNet-based conditioning branch.

CrossLiDAR

Problem Formulation

We consider SS LiDAR sensors mounted at distinct viewpoints around the vehicle. At each time step kk, each sensor s∈{1,…,S}s \in\{1,\ldots,S\} generates a point cloud Pk(s)⊂R3P_k^{(s)} \subset\mathbb{R}^3, which is projected into a range image Xk(s)∈RH×W×CX_k^{(s)} \in\mathbb{R}^{H \times W \times C}, where HH and WW denote the vertical and horizontal resolutions, respectively, and CC represents the feature channels (e.g., range, intensity, and elongation). To create a unified input for our model, we stack these images along the sensor dimension to form a multi-sensor tensor Xk∈RS×H×W×CX_k \in\mathbb{R}^{S \times H \times W \times C}.

Since our objective is to generate sequences of multi-sensor LiDAR data, we denote the ground-truth and generated spatio-temporal tensors as X∈RK×S×H×W×CX \in\mathbb{R}^{K \times S \times H \times W \times C} and X^∈RK×S×H×W×C\hat{X} \in\mathbb{R}^{K \times S \times H \times W \times C}, respectively, obtained by stacking {Xk}k=1K\{X_k\}_{k=1}^{K} (and {X^k}k=1K\{\hat{X}_k\}_{k=1}^{K}) along the temporal dimension, such that the kk-th slice corresponds to the multi-sensor tensor at time step kk. To ensure the generated range images correspond to valid 3D structures, a pixel at (u,v)(u,v) with range value rr (depth) is mapped to Cartesian coordinates p=[α,β,γ]Tp = [\alpha,\beta,\gamma]^\mathsf{T} using the following spherical-to-Cartesian transformation: α=r⋅cos⁡(θ)⋅cos⁡(ϕ)\alpha= r \cdot\cos(\theta) \cdot\cos(\phi), β=−r⋅cos⁡(θ)⋅sin⁡(ϕ)\beta= -r \cdot\cos(\theta) \cdot\sin(\phi), and γ=r⋅sin⁡(θ)\gamma= r \cdot\sin(\theta), where ϕ\phi and θ\theta denote the horizontal and vertical angles, respectively. These angles are derived from the pixel coordinates (u,v)(u,v) based on the field-of-view (FOV) and resolution of each sensor.

Our framework consists of two components: a LiDAR compression module and a generative backbone. For compression, we utilize a VAE that operates purely on the spatial dimensions. The encoder EζE_\zeta processes each sensor frame independently (without temporal or cross-sensor coupling) and maps Xk(s)X_k^{(s)} to a compact latent representation zk(s)z_k^{(s)}. The decoder DηD_\eta reconstructs the corresponding range image from the latent representation. The generation process is modeled using an MM-DiT architecture [9], which models joint spatio-temporal dependencies across sensors and frames. To maintain long-term temporal consistency and mitigate cumulative drift, we employ an autoregressive strategy by adding noise to the first NN-conditional frame.

The Variational Autoencoder

To mitigate the computational cost of modeling high-dimensional range-image sequences, we employ a VAE that compresses each input range image into a lower-dimensional latent space [43]. In our setting, the input tensor is X∈RK×S×H×W×CX \in\mathbb{R}^{K \times S \times H \times W \times C}. Differently from [13, 41], we apply a linear normalization to the raw range values, mapping the input to a symmetric interval [−1,1][-1,1]. We treat each (k,s)(k,s) pair (frame kk and sensor view ss) as an independent sample by reshaping (K,S)(K,S) into the batch dimension, i.e., B=B⋅K⋅SB = B \cdot K \cdot S, and encode x∈RH×W×Cx \in\mathbb{R}^{H \times W \times C}.

Overview of CrossLiDAR Framework

Figure 2. Overview of CrossLiDAR Framework. Our framework utilizes a latent rectified-flow [26] formulation to generate synchronized multi-sensor LiDAR sequences. (Left) A frozen VAE encoder first maps range images into a compact latent space ztz_t. (Right) These latents are processed by an MM-DiT [9] backbone, which incorporates two different attention mechanisms: an Inter-sensor Module for cross-sensor alignment and a LiDAR Sequence Module for temporal consistency. The generation is guided by a dedicated ControlNet [70] branch for BEV and 3D layout control. Finally, a frozen VAE decoder transforms the denoised latents back into high-fidelity temporal point clouds.

The encoder/decoder (EζE_\zeta and DηD_\eta) follow the design of [13]: the encoder maps each input range image to a latent feature map z=Eζ(x)∈Rc×h×wz = E_\zeta(x) \in\mathbb{R}^{c \times h \times w}, where the spatial dimensions are downsampled by a factor ff, i.e., (h,w)=(H/f,W/f)(h, w) = (H/f, W/f). The decoder reconstructs x^=Dη(z)\hat{x} = D_\eta(z). We train our VAE by maximizing the Evidence Lower Bound (ELBO) [17]:

LELBO(ζ,η)=Eqζ(z∣x)[log⁡pη(x∣z)]−λ1DKL(qζ(z∣x) ∥ p(z)).(1)\mathcal{L}_{\mathrm{ELBO}}(\zeta,\eta) = \mathbb{E}_{q_\zeta(z\mid x)}\left[\log p_\eta(x\mid z)\right] - \lambda_1 D_{\mathrm{KL}}\left(q_\zeta(z\mid x)\,\|\,p(z)\right). \tag*{(1)}

where xx denotes a single range image drawn from XX, and p(z)=N(0,I)p(z) = \mathcal{N}(0,I). The first term evaluates the reconstruction likelihood of the decoder DηD_\eta from the latent zz. In our implementation, this corresponds to an L1L_1 pixel-wise reconstruction loss [13]. However, as a key departure from [13], we omit the adversarial discriminator and perceptual loss.

Rectified Flow for LiDAR Generation

After VAE pretraining, we freeze its encoder/decoder, and model the empirical latent distribution induced by the frozen encoder using a flow-based generative model built on a Transformer backbone, MM-DiT [9]. The Transformer processes spatio-temporal latent inputs via specialized temporal-attention and inter-sensor-attention blocks.

Latent Rectified-Flow Formulation. Given the multi-sensor range image sequence X\mathcal{X} and EζE_\zeta as the pretrained VAE encoder, for the data we compute the latent z1=Eζ(X)∼p1(z)z_1 = E_\zeta(\mathcal{X}) \sim p_1(z), and draw an independent noise latent z0∼p0(z)=N(0,I)z_0 \sim p_0(z) = \mathcal{N}(0,I). For t∈[0,1]t \in[0,1], we form the rectified (linear) bridge zt=(1−t)z0+tz1z_t = (1-t)z_0 + tz_1. Following [9], the timestep tt is sampled according to a Logit-Normal distribution to prioritize the mid-range of the transport path. The instantaneous ground-truth velocity is constant along the bridge:

ddtzt=z1−z0.\frac{d}{dt}z_t = z_1 - z_0.

Training. We train a time-dependent vector field vθ(zt,t,c)v_\theta(z_t,t,c) (parameterized by the Transformer) to regress this displacement by minimizing the rectified flow loss:

LRF=Et,z0,z1∥vθ(zt,t,c)−(z1−z0)∥22.\mathcal{L}_{\mathrm{RF}} = \mathbb{E}_{t,z_0,z_1}\left\|v_\theta(z_t,t,c) - (z_1-z_0)\right\|_2^2.

where cc denotes conditioning information (sensor identifiers, textual prompts, bounding-box embeddings, etc.). Under this objective, the optimal vector field is the conditional expectation:

v⋆(zt,t)=E[z1−z0∣zt],v^\star(z_t,t) = \mathbb{E}\left[z_1-z_0 \mid z_t\right],

which provides the probabilistic interpretation of the regression target.

Inference. At inference, we generate latents by integrating the learned ODE:

dzdt=vθ(z,t,c),\frac{dz}{dt} = v_\theta(z,t,c),

from t=0t = 0 to t=1t = 1. Because the rectified flow objective encourages near-linear transport paths, accurate samples are typically obtainable with a small number of integration steps using a numerical solver (e.g., Euler or midpoint method). Once the latent z(1)z(1) is obtained, it is passed through the frozen VAE decoder DηD_\eta to reconstruct the multi-sensor range image sequence X^\hat{\mathcal{X}}. While the model operates on range-image latents, the generated range images are back-projected into 3D points using the corresponding LiDAR sensor geometry. The reconstructed points are then transformed into the ego-vehicle coordinate frame using calibrated sensor extrinsics and concatenated to form a unified multi-LiDAR point cloud (Appendix 8.1).

The Architecture Modules

MM-DiT Pre-Attention Stage. The generative backbone is implemented as a stack of MM-DiT blocks operating on VAE-compressed latents within the rectified-flow framework. Each block processes the latent tokens ztz_t along with auxiliary concatenated modality tokens Φ={ϕ1(m1)∥⋯∥ϕD(mD)}\Phi= \{\phi_1(m_1) \parallel\cdots\parallel\phi_D(m_D)\}. Following the canonical MM-DiT design, each block begins with a pre-attention stage utilizing Adaptive Layer Normalization (AdaLN). The tokens are then processed through a spatio-temporal attention, where 3D rotary positional embeddings (3D RoPE) [48] are applied to queries and keys. A dedicated inter-sensor attention is inserted at this stage to facilitate information exchange across multiple sensors at the same timestep.

The LiDAR Sequence Module. After reshaping the latent tokens as z~t∈R(B⋅K)×N×c′\tilde{z}_t \in\mathbb{R}^{(B\cdot K)\times N\times c'} and the modality tokens Φ~tseq∈R(B⋅K)×N×c′\tilde{\Phi}^{seq}_t \in\mathbb{R}^{(B\cdot K)\times N\times c'}, with N≡S⋅h⋅wN \equiv S\cdot h\cdot w, spatiotemporal dependencies are modeled via a multi-head joint self-attention mechanism. Let HH be the number of heads and dhd_h the per-head dimension so that c′=Hdhc' = H d_h. We first compute separate linear projections for z~t\tilde{z}_t and Φ~t\tilde{\Phi}_t, using two distinct sets of learnable weights:

Qz=z~tWQ′,Kz=z~tWK′,Vz=z~tWV′QΦ=Φ~tseqWQ′′,KΦ=Φ~tseqWK′′,VΦ=Φ~tseqWV′′\begin{aligned} Q_z &= \tilde{z}_t W'_Q,\quad K_z = \tilde{z}_t W'_K,\quad V_z = \tilde{z}_t W'_V \\ Q_\Phi&= \tilde{\Phi}^{seq}_t W''_Q,\quad K_\Phi= \tilde{\Phi}^{seq}_t W''_K,\quad V_\Phi= \tilde{\Phi}^{seq}_t W''_V \end{aligned}

where WQ,K,V′∈Rc′×c′W'_{Q,K,V} \in\mathbb{R}^{c'\times c'} and WQ,K,V′′∈Rc′×c′W''_{Q,K,V} \in\mathbb{R}^{c'\times c'}. We then apply 3D RoPE to the query and key of the latent. Next, the tokens are concatenated to form joint representations QQ, KK, V∈R(BK)×2N×c′V \in\mathbb{R}^{(BK)\times2N\times c'}:

Q=[Q~z∥Q~Φ],K=[K~z∥K~Φ],V=[Vz∥VΦ].Q = [\tilde{Q}_z \parallel\tilde{Q}_\Phi],\quad K = [\tilde{K}_z \parallel\tilde{K}_\Phi],\quad V = [V_z \parallel V_\Phi].

To maintain temporal consistency, we enforce causal masking along the temporal axis. For the joint sequence length 2N2N, we define a mapping σ(i)\sigma(i) that returns the temporal frame ss for any index i∈{0,…,2N−1}i \in\{0,\ldots,2N-1\} such that σ(i)=si mod N\sigma(i) = s_i \bmod N. We define the causal mask Mcausal∈R2N×2N\mathcal{M}^{\mathrm{causal}} \in\mathbb{R}^{2N\times2N} by 00 if sj≤sis_j \leq s_i, while −∞-\infty if sj>sis_j > s_i. This configuration allows for bidirectional interaction between the LiDAR latent and the modality token at any given time step, while preventing leakage from future frames. Scaled dot-product attention with the causal mask (per head) A(h)A^{(h)} is performed. The resulting joint output O=Concat⁡(A(1)V(1),…,A(H)V(H))WOO = \operatorname{Concat}(A^{(1)}V^{(1)},\ldots,A^{(H)}V^{(H)})W_O is split back into the latent and modality tensors [Oz∥OΦ]=O∈R(BK)×2N×c′[O_z \parallel O_\Phi] = O \in\mathbb{R}^{(BK)\times2N\times c'}, where Oz,OΦ∈R(BK)×N×c′O_z,O_\Phi\in\mathbb{R}^{(BK)\times N\times c'} are passed to their respective residual blocks and point-wise feed-forward networks, obtaining z~t′\tilde{z}'_t and Φ~t′\tilde{\Phi}'_t.

The Inter-sensor Module. To resolve geometric disparities between the various LiDAR sensors positioned around the vehicle, the Inter-sensor module treats each sensor as a structured viewpoint and models their spatial interactions through joint attention rather than independent per-sensor generation. For this module, we employ a similar architectural configuration to the LiDAR Sequence Module, which consists of projected bounding box embedders to provide explicit spatial grounding for the different views. To compute the attention between sensors, we reshape the latent output as z~t′∈R(B⋅S)×M×c′\tilde{z}'_t \in\mathbb{R}^{(B\cdot S)\times M\times c'} and the modality tokens as Φ~tinter∈R(B⋅S)×M×c′\tilde{\Phi}^{inter}_t \in\mathbb{R}^{(B\cdot S)\times M\times c'}, where M=K⋅h⋅wM = K\cdot h\cdot w. By shifting the sensor count KK into the sequence dimension, the joint attention mechanism captures dependencies across different viewpoints within the same temporal frame.

The attention process follows the same mathematical formulation as described in the LiDAR Sequence Module, utilizing separate linear projections WQ,K,V′W'_{Q,K,V} and WQ,K,V′′W''_{Q,K,V}. However, the 3D RoPE is applied across the sensor index k∈{0,…,K−1}k \in\{0,\ldots,K-1\} and spatial coordinates (u,v)(u,v), rather than the temporal axis, while the causal mask is not adopted.

LiDAR ControlNet. To enable precise BEV-based structural control over the synthesis process, we incorporate a ControlNet-style branch [70] conditioned on BEV maps containing projected 3D bounding boxes (Appendix Appendix § 8). Following [70], we create a trainable copy of the MM-DiT backbone blocks while keeping the original generative weights frozen.

The ControlNet branch adopts an internal structure identical to the primary modules, including the factorized attention schemes of the LiDAR Sequence and Inter-sensor modules. For each block in this branch, we first fuse the structural control signals (3D bounding boxes and BEV maps), with the current noisy latent ztz_t. Specifically, the control signals are processed by a lightweight encoder to produce the embedding Φctrl\Phi^{ctrl}, which is summed with ztz_t to form a unified input for the ControlNet blocks. Therefore, the control signals are directly grounded in the current denoising state, allowing for more accurate structural guidance. The output of the nn-th ControlNet block, Block⁡ctrl(n)\operatorname{Block}^{(n)}_{\mathrm{ctrl}}, integrates into the corresponding backbone, Block⁡backbone(l)\operatorname{Block}^{(l)}_{\mathrm{backbone}}, via a zero-initialized residual connection.

Autoregressive Generation

Naïve autoregressive LiDAR generation iteratively conditions on a model’s own predictions, leading to exposure bias and compounding errors over long horizons [6, 15, 53]. Small inaccuracies in early frames are amplified during rollout, causing drift and degraded generation quality. To ameliorate this, we add a small amount of noise to the ground-truth conditioning frame during training. More precisely, rather than using clean conditioning frames during training we fix the noise level of initial frames to t∈[0.7,1]t \in[0.7,1], encouraging robustness to exposure bias during autoregression.

Experimental Results

Experimental Settings

Datasets. The nuScenes dataset [4] is designed for multi-modal perception in complex urban environments. We utilize 32-beam LiDAR sweeps. Each LiDAR sweep (sam ple) is initially represented as a range image with a resolution of 32×102432 \times1024 pixels (totaling 32,768 pixels), a vertical Field of View (FOV) spanning (+10∘+10^\circ, −30∘-30^\circ), and a depth range between 1 and 45 meters. The Waymo Open Dataset [49] utilizes a sensor suite consisting of five LiDAR sensors. This configuration includes a primary high-performance mid-range LiDAR (Top) for long-range 360∘360^\circ coverage, supplemented by four short-range side LiDARs (Front, Left, Right, and Rear) positioned around the vehicle’s perimeter. These side sensors are specifically designed to capture the immediate surroundings with high precision, providing a unique perspective that is less susceptible to the top-down occlusions common in roof-only configurations. Further details on both datasets are provided in Appendix § 7.

Baselines. We compare CrossLiDAR against a comprehensive set of baselines categorized by their underlying representation. Range-based methods include LiDARGen [74], R2DM [35], RangeLDM [13], LiDM [41], and LiDARCrafter [22]. Voxel-based methods include the OpenDWM models [36, 46], and we additionally compare against the occupancy-centric UniScene [19]. Details of these methods under our setting are discussed in Appendix § 11.

Evaluation Metrics. Following [22], we evaluate the quality of the generated LiDAR data across three complementary representations: (1) Range-image fidelity using Fréchet Range Distance (FRD) and Maximum Mean Discrepancy (MMD); (2) Point-cloud geometry using Fréchet

Point Distance (FPD) and MMD; and (3) BEV distribution using Jensen–Shannon Divergence (JSD) and MMD. To evaluate temporal dynamics, we follow [22] by measuring frame-to-frame transformation accuracy and sequence smoothness using Temporal Transform Consistency Error (TTCE) and Chamfer Temporal Consistency (CTC). We additionally adapt three metrics from the video synthesis literature to the LiDAR domain to provide a more comprehensive assessment of (4) Temporal coherence: Blind Video Temporal Consistency (BVC) [2], Fréchet Video Distance (FVD) [50], and Fréchet Video Motion Distance (FVMD) [24]. Finally, we evaluate rollout quality by analyzing the accumulated point-cloud sequences. MMD values are reported in 10−410^{-4} and JSD values in 10−210^{-2}. In all tables, the best result is highlighted in bold and the second-best in <u>underline</u>. We evaluate a fixed, calibrated four-LiDAR configuration, using sensor-specific range-image geometry and extrinsic calibration for ego-frame alignment. More details are provided in Appendix § 9.

Image Generation

We evaluate CrossLiDAR on single-frame LiDAR generation across multi- and single-sensor benchmarks. Additional details and ablation studies are in Appendix § 8 and Appendix § 10.

Waymo. Table 1 reports scene-level fidelity on Waymo, our primary benchmark for the multi-sensor design, where CrossLiDAR outperforms all baselines. In the range-image domain, it improves FRD by 93.0% over the second-best method. Point-cloud fidelity follows the same trend, with a 71.7% FPD reduction relative to LiDM. On layout, CrossLiDAR reduces JSD by 25.4% over the next-best baseline. These results indicate better scene-geometry preservation, further supported by our Ablation Study (Appendix § 10.2) showing that the combined LiDAR Sequence and Inter-sensor Modules reduce FRD by 88.8% over naive stacking.

MethodVenueRangePointsBEV
FRD↓MMD↓FPD↓MMD↓JSD↓MMD↓
Voxel
OpenDWM[36]CVPR’25——7.5552.2316.252.29
OpenDWM-DiT[46]CVPR’25——8.41105.4010.851.80
Range
LiDM[41]CVPR’242.29148.834.6765.7619.341.00
RangeLDM[13]ECCV’2410.97979.33122.381236.5922.331.63
R2DM[35]ICRA’2421.632283.9040.56228.0427.541.97
CrossLiDAR—0.1611.891.326.908.09<u>1.05</u>

Table 1. Evaluations of Scene-level Fidelity on Waymo dataset for multi-sensor LiDAR generation.

nuScenes. Table 2 reports results on the single-sensor nuScenes benchmark, where CrossLiDAR achieves SOTA across all three representation domains. The largest improvement is in the point-cloud domain, where our model achieves a 57.1% reduction in FPD compared with OpenDWM-DiT. BEV alignment is also improved, with CrossLiDAR BEV MMD representing a 35.7% reduction over LiDARCrafter. Our method also improves rangeimage fidelity, reducing FRD by 6.6% compared with LiDARCrafter. These results show the proposed framework remains effective even without the Inter-sensor Module.

MethodVenueRangePointsBEV
FRD↓MMD↓FPD↓MMD↓JSD↓MMD↓
Voxel
UniScene[19]CVPR’25————31.5513.61
OpenDWM[36]CVPR’25——0.955.4320.175.61
OpenDWM-DiT[46]CVPR’25——0.490.5019.905.73
Range
LiDM[41]CVPR’24495.540.182.061.835.860.73
RangeLDM[13]ECCV’24——14.22184.745.471.92
R2DM[35]ICRA’24243.351.40——3.510.71
LiDARCrafter[22]AAAI’26194.30<u>0.08</u>——3.110.42
CrossLiDAR—<u>181.41</u>0.090.210.933.080.27

Table 2. Evaluations of Scene-level Fidelity on nuScenes dataset for LiDAR generation.

4D LiDAR Generation

We evaluate the temporal coherence of CrossLiDAR on multi- and single-sensor LiDAR sequence generation.

Waymo. Table 3 reports 4D generation metrics on Waymo. Since no prior method supports multi-sensor 4D LiDAR generation, we evaluate CrossLiDAR on Waymo’s fixed four-SIDE-LiDAR configuration at a rollout horizon of t=5t=5 to assess temporal coherence and cross-sensor consistency. The model remains highly stable across all four CTC steps and achieves the best performance across all rollout-level metrics. Qualitative results in Figure 3 further show that CrossLiDAR better preserves foreground structure, temporal alignment, and cross-sensor geometric consistency than range-based baselines. Additional multi-sensor rollouts, temporal consistency, and controllability examples are provided in Appendix Figures 12 and 13.

MethodVenueTTCE↓CTC↓Rollout↓
121234CDJSDMMD
Voxel
OpenDWM[36] (tt=5)CVPR’25990.17990.740.370.520.650.682.810.530.57
OpenDWM-DiT[46] (tt=5)CVPR’251025.781026.430.390.590.760.810.780.130.34
Range-Temporal
LiDM[41] (tt=3)CVPR’241444.741444.480.200.24——0.530.521.81
RangeLDM[13] (tt=3)ECCV’241686.141686.890.240.27——0.840.648.75
CrossLiDAR (tt=5)—1016.741017.380.060.100.120.140.070.100.03

Table 3. 4D LiDAR generation on Waymo using video metrics including: Temporal Consistency and Rollout-level metrics.

Qualitative results on nuScenes and Waymo

Figure 3. Qualitative Results. The top row illustrates temporal sequence generation on nuScenes, where CrossLiDAR maintains strict temporal consistency and 3D bounding box alignment. The bottom row demonstrates multi-sensor Waymo generation, where a single unified scene is reconstructed by synchronizing outputs from four distinct generated sensors. Compared to baselines, CrossLiDAR produces scan patterns with improved scene-level geometric fidelity, effectively handling the complexities of multi-sensor generation.

nuScenes. Table 4 reports temporal generation results on the single-sensor nuScenes benchmark. CrossLiDAR achieves the best TTCE scores and outperforms prior methods on longer-horizon CTC metrics, with the performance gap increasing as sequence length increases. CrossLiDAR achieves a 15.4% reduction in TTCE at step 4 compared with LiDARCrafter, indicating better adherence to physically plausible motion trajectories. Although OpenDWM-DiT achieves a lower CTC-1 error, its error accumulates rapidly over longer horizons. In contrast, CrossLiDAR reduces CTC-4 by 36.6% relative to OpenDWM-DiT. Rollout quality also improves, with a 39.7% and 65.8% reduction in CD compared with OpenDWM-DiT and OpenDWM, respectively. These results demonstrate our proposed framework effectively mitigates the geometric drift commonly associated with long-horizon 4D LiDAR generation.

MethodVenueTTCE ↓CTC ↓Rollout ↓
341234CDJSDMMD
Voxel
UniScene[19]CVPR’252.743.690.901.843.643.90———
OpenDWM[36]CVPR’252.683.651.022.023.375.051.200.179.99
OpenDWM-DiT[46]CVPR’252.713.660.891.793.064.640.680.178.80
Range-Temporal
LiDARCrafter[22]AAAI’262.653.561.122.383.024.81———
CrossLiDAR—2.403.011.421.712.102.940.410.140.03

Table 4. 4D LiDAR generation on nuScenes using video metrics including: Temporal Consistency and Rollout-level metrics. The temporal horizon is fixed to t=5t = 5 for all models.

Autoregressive 4D LiDAR Generation

We assess long-horizon stability by extending CrossLiDAR autoregressively beyond its training window, in both multi- and single-sensor settings.

Waymo. Table 5 reports autoregressive generation on Waymo, where four side LiDARs make cross-sensor consistency especially hard to maintain as the rollout horizon grows. Extending the generated window to t=12t=12, CrossLiDAR achieves the best temporal consistency, reducing TTCE by 13.0% over the strongest baseline and reducing CTC by an average of 87.8% across all four evaluation steps. At the rollout level, CrossLiDAR further reduces JSD by 67.6% and MMD by 85.7% compared with the strongest baseline. These results show the proposed framework effectively preserves cross-sensor geometric consistency and temporal coherence during long autoregressive rollouts.

MethodVenueTTCE ↓ 3TTCE ↓ 4CTC ↓ 1CTC ↓ 2CTC ↓ 3CTC ↓ 4Rollout ↓ CDRollout ↓ JSDRollout ↓ MMDMotion BVC↑Motion FVD↓
OpenDWM [36] (t=12t=12)CVPR’25471.34471.400.420.550.660.730.110.380.17——
OpenDWM-DiT [46] (t=12t=12)CVPR’25424.65424.862.072.632.813.170.070.370.14——
CrossLiDAR—369.63369.740.050.070.080.090.070.120.020.7957.42

Table 5. Auto-regressive generation on Waymo, including Temporal Consistency, Rollout-level and Motion-level metrics.

nuScenes. Table 6 evaluates 16-frame rollouts on nuScenes, where CrossLiDAR shows substantially improved long-horizon stability over previous methods. While LiDARCrafter’s CTC increases by 329.5% across the evaluated steps, CrossLiDAR grows by only 34.9%, consistent with our noise-augmented training strategy for imperfect autoregressive conditioning. CrossLiDAR further reduces TTCE by 66.7% and 64.0% at steps 3 and 4, respectively, over the second-best method, suggesting the proposed flow-matching transformer better preserves physically plausible ego-motion. The gap between TTCE and CTC suggests dynamic objects remain harder to model than ego-motion. At the distributional level, CrossLiDAR reduces FVD by 89.2% over OpenDWM-DiT, bringing the generated sequences substantially closer to the real LiDAR distribution in appearance and dynamics. CrossLiDAR reduces FVMD by 26.0% while improving BVC by 104.7%, demonstrating more realistic motion and stronger temporal stability. These results show CrossLiDAR maintains high quality over long horizons, a key capability for scalable autonomy simulation.

MethodVenueTTCE ↓ 3TTCE ↓ 4CTC ↓ 1CTC ↓ 2CTC ↓ 3CTC ↓ 4Rollout ↓ CDRollout ↓ JSDRollout ↓ MMDMotion BVC↑Motion FVD↓Motion FVMD↓
UniScene [19] (t=5t=5)CVPR’252.743.690.901.843.643.90——————
OpenDWM [36] (t=16t=16)CVPR’252.643.160.821.622.834.670.970.460.270.223923.114147.61
OpenDWM-DiT [46] (t=16t=16)CVPR’251.231.390.681.422.423.830.830.240.130.432835.682261.98
LiDARCrafter [22] (t=5t=5)AAAI’262.653.561.122.383.024.81——————
CrossLiDAR (t=16t=16)—0.410.501.091.221.361.470.470.220.260.88307.071672.90

Table 6. Autoregressive generation on nuScenes, including Temporal Consistency, Rollout-level and Motion-level metrics.

Conclusions and Limitations

We present CrossLiDAR, a unified rectified-flow transformer for controllable multi-sensor LiDAR generation in a compact range-image latent space. It comprises three key components: an Inter-sensor Module that models geometric dependencies across heterogeneous LiDAR suites, a ControlNet-style conditioning branch for fine-grained structural control via BEV priors and 3D bounding boxes, and a noise-augmented autoregressive strategy that mitigates cumulative drift over long horizons. On Waymo and nuScenes, CrossLiDAR achieves strong performance in single-frame fidelity, temporal consistency, and long-horizon rollout quality, with particularly strong gains in the multi-sensor setting unsupported by previous methods.

Current limitations include the lack of quantitative evaluation of foreground fidelity and bounding-box adherence. Our evaluation focuses on Waymo’s fixed, calibrated four-LiDAR configuration, motivating further study of sensor conditioning, cross-sensor overlap, and adaptability to different sensor counts, types, layouts, and calibrations. The LiDAR Sequence and Inter-sensor Modules are evaluated jointly, leaving their individual contributions unquantified. Evaluation against additional recent 4D LiDAR generation methods remains limited. Downstream benefits for 3D detection, semantic segmentation, occupancy prediction, and planning remain unevaluated. Future work will address richer dynamic object modeling and integration with downstream planning and validation pipelines, while further reducing the sim-to-real gap.

Appendix

Datasets

We evaluate our approach using two large-scale autonomous driving datasets: nuScenes [4] and the Waymo Open Dataset [49]. Each provides multi-modal sensor data together with annotations. In this section, we summarize their overall statistics, sensor configurations, image resolutions, and available labels.

nuScenes

The nuScenes dataset [4] provides diverse multimodal sensor data collected using two identically instrumented vehicles for autonomous driving research. The data were recorded with a surround-view sensor suite comprising one spinning 32-beam LiDAR sensor, six RGB cameras, and five automotive radar sensors. Together, these sensors provide complementary observations with full 360° coverage around the ego vehicle (specifications in Table 7). The LiDAR sensor captures point clouds at 20 Hz, with a 360° horizontal field of view and a vertical field of view from −30° to 10°, producing up to 1.4 million points per second with an effective range of up to 70 m. In addition, the cameras operate at 12 Hz and the radar sensors at 13 Hz.

SensorVFOVHFOVResolutionRange (m)Intensity
Velodyne HDL-32E[+10°; −30°]360°(32, 1080)[2.0; 100.0][0; 225]

Table 7. LiDAR Sensor Specifications for the nuScenes Dataset. We report the vertical field of view (VFOV), horizontal coverage (HFOV), range-image resolution (rows×columns), valid range (m), and intensity range.

nuScenes provides a large corpus of annotated 3D bounding boxes for urban driving scenes captured under diverse traffic, lighting, and weather conditions. The full dataset contains 1,000 scenes, each 20s long, selected from 84 driving logs collected in Boston and Singapore. Although the sensor streams are temporally dense, 3D box annotations are provided only for keyframes sampled at 2 Hz. In total, the dataset includes approximately 40,000 annotated keyframes, about 400,000 LiDAR sweeps, 1.4M camera images, 1.4M radar sweeps, and 1.4M annotated 3D bounding boxes across 23 classes. Annotations are defined in the global coordinate frame and are distributed with calibrated sensor poses, enabling projection into the ego or sensor coordinate frames when needed for model training and evaluation. Beyond object boxes, nuScenes also provides rich scene-level metadata, including location, weather, time of day, and object attributes.

Waymo

The Waymo Open Dataset [49] provides diverse multi-modal sensor data collected from a fleet of self-driving vehicles. Each vehicle is equipped with five high-resolution LiDAR sensors: one mounted on the roof (TOP) and four located on the SIDEs: front (F), rear (R), side-left (SL), and side-right (SR). All LiDAR data is provided as range images; detailed specifications3 are listed in Table 8.

SensorVFOVHFOVResolutionRange (m)Intensity
TOP[−17.6°, +2.4°]360°64 × 2650[1.2, 75.0][7 × 10^{-6}, 4 × 10^{6}]
F/SL/SR/R[−90°, +30°]—200 × 600[5 × 10^{-3}, 20.0][5 × 10^{-6}, 40.5]

Table 8. LiDAR Sensor Specifications on Waymo Open Dataset. We report vertical field of view (VFOV), horizontal coverage (HFOV), range-image resolution, valid range (m), and intensity range for TOP and SIDE sensors.

Figures 4 and 5 show typical range images and corresponding point clouds from the TOP sensor, covering 360° horizontally, up to 75 m range, and 64 vertical beams; and from the four SIDE sensors, each with a larger vertical FOV, 200 beams, and a maximum range of 20m. The precise horizontal field of view (FOV) for each SIDE sensor is not specified in the official documentation. We therefore estimate each sensor’s effective FOV as the proportion of valid columns (i.e., columns with at least one valid measurement) in the range image, out of 600 columns. Assuming a total sensor FOV of X∘X^\circ, the effective FOV is computed as #valid columns600×X∘\frac{\#\text{valid columns}}{600} \times X^\circ.

In all experiments, we use only the four side LiDAR sensors to emulate a multi-sensor input scenario.

Waymo provides a large corpus of high-quality annotated 3D LiDAR bounding boxes. We use annotations for three semantic classes: VEHICLE, PEDESTRIAN, and CYCLIST. Since boxes are labeled in the vehicle (ego) frame, we project them into the LiDAR frame by inverting the extrinsic calibration matrix. As SIDE LiDAR sensors capture only short-range points, we retain boxes fully contained within each sensor’s valid range.

The resulting dataset contains approximately 1.45M LiDAR box annotations across over 1000 scenes from geographically distinct regions. Each scene is 20 seconds at 10Hz, totaling 200 frames per scene. For input construction, each scene is divided into overlapping clips of 17 frames with a stride of 5 frames, so consecutive clips share up to 12 frames.

Statistics for our training set using only SIDE LiDAR sensors are provided in Table 9, 10 and Figure 6.

TrainValTest
Vehicle941.6K227K
Pedestrian499.1K121.4K
Cyclist11.8K3.4K
Total1451.6K352K

Table 9. Statistics on annotations.

Per-frame annotations

Figure 6. Per-frame annotations.

Point Cloud Densification. The SIDE LiDAR range images in Waymo are sparse and contain many invalid pixels. Following [18], we apply a fast CPU-based depthcompletion algorithm to fill invalid pixels with estimated depths. The method first performs nearest-neighbor dilation using a diamond-shaped kernel, and then refines the result via smoothing steps that encourage locally planar surfaces and suppress outliers with bilateral filtering. As

Figure 4. Waymo range-image examples. Top: TOP LiDAR range image (64×265064 \times2650), with 360∘360^\circ horizontal coverage. Bottom: four SIDE LiDAR range images (200×600200 \times600 each), with higher vertical resolution and partial-view coverage.

Examples of Waymo point-cloud coverage

Figure 5. Examples of Waymo point-cloud coverage. Left: Point cloud from each SIDE LiDAR sensor, illustrating the partial horizontal coverage provided by each individual sensor. Right: Point cloud obtained by fusing the four SIDE LiDAR sensors, shown in comparison with the TOP LiDAR. The SIDE sensors are color-coded by position: front (yellow), left (pink), rear (blue), and right (green).

TrainValTest
Scenes798202150
Frames158K40K30K
Clips21K5K5K

Table 10. Size of the data splits.

shown in Figure 7, densification substantially improves the visual quality of the resulting depth maps and point clouds.

Densification on the Waymo SIDE LiDAR range images

Figure 7. Densification on the Waymo SIDE LiDAR range images. The top row shows the original sparse range images from Waymo, while the bottom row shows the densified results. The densification algorithm [18] fills in missing pixels and significantly enhances the visual quality and completeness of the range images.

Implementation Details

Data Pre-processing & Post-processing. For point cloud generation, each LiDAR scan is optionally projected to a dense 2D range image using spherical projection: Given a 3D point (x,y,z)(x, y, z), the corresponding spherical coordinates are computed as:

d=x2+y2+z2,θ=arcsin⁡(zd),ϕ=arctan⁡2(−y,x).(2)\begin{aligned} d &= \sqrt{x^2 + y^2 + z^2}, \\ \theta&= \arcsin\left(\frac{z}{d}\right), \\ \phi&= \arctan2(-y, x). \tag*{(2)} \end{aligned}

where dd is the measured range, θ\theta is the elevation angle, and ϕ\phi is the azimuth angle. For the Waymo dataset, range images are provided directly, and this step is skipped. To address sparsity in the raw range images, we use the ip_basic implementation4 for efficient depth completion.

Each valid range measurement dd is normalized to [−1,1][-1, 1] via linear scaling:

dnorm=ddmax⁡×2−1,(3)d^{\mathrm{norm}} = \frac{d}{d_{\max}} \times2 - 1, \tag*{(3)}

where dmax⁡d_{\max} is the maximum valid range for the sensor. For generation, we apply the inverse mapping.

Predicted range images are projected back to 3D space using the inverse of the spherical mapping:

x=d⋅cos⁡θ⋅cos⁡ϕ,y=−d⋅cos⁡θ⋅sin⁡ϕ,z=d⋅sin⁡θ,(4)x = d \cdot\cos\theta\cdot\cos\phi,\quad y = -d \cdot\cos\theta\cdot\sin\phi,\quad z = d \cdot\sin\theta, \tag*{(4)}

where dd is the depth value at the pixel, and θ\theta, ϕ\phi are the vertical and horizontal angles for that pixel. Points with range dd outside the post-processing interval are filtered out: [0.5,20.0][0.5, 20.0] for Waymo SIDE LiDAR, and [2.0,95.0][2.0, 95.0] for nuScenes.

For BEV controllable generation, we project 3D bounding boxes and other semantic elements from the ego-vehicle frame onto the BEV plane. Given a bounding box with corners {biego}\{b_i^{\mathrm{ego}}\} in the ego frame, we apply a fixed ego-to-BEV transformation. Boxes and other features are then rasterized onto a discrete 2D grid, where the grid bounds and cell size are chosen according to the dataset. We also incorporate BEV roadmap layers (e.g., road, lanes), which are available from both Waymo and nuScenes.

Training Procedures. CrossLiDAR follows a three-stage latent generative training protocol.

VAE Pretraining. We first pre-train the VAE using a reconstruction objective on range images, encoding each LiDAR sensor independently into a compact latent representation. After convergence, the VAE weights are frozen.

Latent Rectified Flow Training. We then train a Rectified Flow model to learn the dynamics in the latent space. For sequence modeling, latent representations from all time steps (TT) and sensors (SS) are stacked along the batch dimension (BB), enabling joint spatio-temporal learning directly in latent space.

ControlNet Training. Finally, we freeze the base Rectified Flow model and train a ControlNet branch [70] for conditional generation. Control blocks are inserted into the backbone at a fixed interval (one every three backbone blocks). The model is conditioned on structural inputs, including BEV maps with projected 3D bounding boxes, enabling controllable scene generation while preserving the pretrained generative prior.

Training Hyper-parameters. The VAE is trained for 50 epochs using the AdamW optimizer with a base learning rate of 4.5×10−64.5 \times10^{-6} and a batch size of 16. We employ an ℓ1\ell_1 reconstruction objective with a KL regularization weight of 1×10−61 \times10^{-6}. The encoder—decoder architecture uses a latent dimensionality of 4 channels, a base channel width of 64 with multipliers [1,2,2,4][1, 2, 2, 4], and two residual blocks per resolution level with SiLU activations. Circular padding is adopted to respect the periodic structure of range images, and no attention layers are used in the VAE. Mixed-precision (FP16) training is used throughout.

The rectified flow model is trained with an ℓ2\ell_2 loss and a log-normal noise schedule for 500000 iterations using a batch size of 16. We use AdamW optimizer with learning rate 1×10−41 \times10^{-4} for nuScenes, 8×10−58 \times10^{-5} for Waymo, β1=0.9\beta_1 = 0.9, β2=0.999\beta_2 = 0.999, ϵ=10−8\epsilon= 10^{-8}. The learning rate is linearly warmed up over the first 0.01% of steps and then held constant. Gradients are clipped to a maximum norm of 0.1. Mixed-precision (FP16) training is used throughout, with FSDP and zero3 sharding for memory efficiency.

During diffusion and ControlNet training, we set the MM-DiT patch size to 1. Multiple conditioning modalities are used, including text, images, frame pose, and BEV projections of 3D bounding boxes for explicit scene-level guidance. To enhance robustness and generalization, condition dropout is applied independently to each condition with probability 0.15; for short clips (fewer than 5 frames), all conditions are jointly dropped with probability 0.15. BEV maps are embedded into the VAE latent space using a spatial encoder consisting of 5 layers of 2D convolutional blocks with residual connections.

For ControlNet, 30% of the dataset is set aside for training, as reusing the diffusion model data for this stage leads to minimal improvements.

Sampling and Inference. For generation, we employ rectified flow with a linear transport map for the diffusion process and use the Euler–Maruyama solver for sampling. We run 30 sampling steps with a classifier-free guidance scale of 3.0. Additional sampling hyperparameters are set as needed.

Evaluation

We evaluate LiDAR sequence generation using complementary metrics that capture: (1) Per-scene fidelity, (2) Temporal rollout realism, (3) Temporal consistency across frames, and (4) Video-level quality aggregated over sequences. Unless stated otherwise, all metrics are computed on Waymo using the four SIDE LiDAR sensors.

Scene-level Metrics

Scene Definition. We define a scene at time tt as the fused multi-sensor point cloud obtained by combining the LiDAR point clouds. Specifically, for each sensor s∈{1,…,S}s \in\{1,\ldots,S\} (with S=4S=4 in Waymo and with S=1S=1 in nuScenes), we project points from the sensor frame into the ego-vehicle frame using the provided extrinsics:

psego=Tego←sps,s∈{1,…,S},(5)p_s^{\mathrm{ego}} = T_{\mathrm{ego}\leftarrow s}p_s,\qquad s\in\{1,\ldots,S\}, \tag*{(5)}

Furthermore, we concatenate the transformed points to form the fused scene point cloud Ptego=⋃s=1Spt,segoP_t^{\mathrm{ego}}=\bigcup_{s=1}^{S}p_{t,s}^{\mathrm{ego}}.

Sampling Protocol. For statistical reliability, we evaluate a total of 10000 sensor frames (i.e., per-sensor range images) in all scene-level metrics. Concretely, for models that generate single frames, we generate 10000 scenes; each scene comprises independently generated range images (one per sensor) that are re-projected and fused in the ego frame. For models that generate sequences, we generate 2000 sequences of length T=5T=5 (10000 scenes total), and evaluate each time step as an individual scene.

Distributional Fidelity Metrics. To quantify scene-level fidelity, we adopt standard distributional similarity metrics widely used in LiDAR generative modeling [74]. For each metric, we compute features for real samples and generated samples, fit Gaussians to their distributions, and measure the divergence between the two distributions.

Fréchet Range Distance (FRD). FRD [74] measures the discrepancy between feature distributions of real and generated range images for each sensor. Following prior work, we extract features using a pretrained RangeNet-53 [32] segmentation network.5 Let {fir}i=1N\{f_i^r\}_{i=1}^{N} and {fig}i=1N\{f_i^g\}_{i=1}^{N} denote features from real and generated samples, with empirical means (μr,μg)(\mu_r,\mu_g) and covariances (Σr,Σg)(\Sigma_r,\Sigma_g). FRD is computed as the Fréchet distance between two Gaussians:

FRD=∥μr−μg∥22+Tr⁡(Σr+Σg−2(ΣrΣg)12).(6)\mathrm{FRD}=\lVert\mu_r-\mu_g\rVert_2^2+\operatorname{Tr}\left(\Sigma_r+\Sigma_g-2(\Sigma_r\Sigma_g)^{\frac{1}{2}}\right). \tag*{(6)}

Fréchet Point Volume Distance (FPD). To assess realism in 3D, we re-project generated range images into point clouds, and extract their features using a pretrained Sparse Point-Voxel Convolution Network [39].6 We then compute the Fréchet distance in the same form as above, using point-cloud feature statistics.

Jensen-Shannon Divergence (JSD). To compare spatial occupancy patterns, we voxelize each fused ego-frame scene into a BEV occupancy histogram over an H×WH \times W grid. Let PP and QQ denote the marginal BEV occupancy distributions obtained by averaging normalized histograms over generated and real scenes. We compute:

JSD⁡(P∥Q)=12KL⁡(P∥M)+12KL⁡(Q∥M),(7)\operatorname{JSD}(P\|Q)=\frac{1}{2}\operatorname{KL}(P\|M)+\frac{1}{2}\operatorname{KL}(Q\|M), \tag*{(7)}

where M=12(P+Q)M=\frac{1}{2}(P+Q).

Maximum Mean Discrepancy (MMD). We additionally compute the maximum mean discrepancy between the BEV occupancy distributions. Given samples {xi}i=1N∼P\{x_i\}_{i=1}^{N}\sim P (real) and {yj}j=1N∼Q\{y_j\}_{j=1}^{N}\sim Q (generated), MMD with kernel k(⋅,⋅)k(\cdot,\cdot) is:

MMD⁡2(P,Q)=1N(N−1)∑i≠i′k(xi,xi′)+1N(N−1)∑j≠j′k(yj,yj′)−2N2∑i,jk(xi,yj).(8)\operatorname{MMD}^{2}(P,Q)=\frac{1}{N(N-1)}\sum_{i\ne i'}k(x_i,x_{i'})+\frac{1}{N(N-1)}\sum_{j\ne j'}k(y_j,y_{j'})-\frac{2}{N^2}\sum_{i,j}k(x_i,y_j). \tag*{(8)}

Unless noted otherwise, we use an RBF kernel on the flattened BEV histograms.

For each fused scene PtegoP_t^{\mathrm{ego}}, we compute a BEV occupancy grid by discretizing the plane into uniform cells and marking a cell occupied if it contains at least one point. The resulting binary occupancy map is normalized into a histogram and used to form the marginal distributions for JSD and MMD.

Temporal Consistency Metrics

Temporal Transformation Consistency Error (TTCE)

To measure video generation performance, we follow [22] and inherit the temporal consistency metrics TTCE and CTC, which quantify how consistently generated sequences preserve temporal motion by comparing frame-to-frame rigid transformations with ground truth. Using Iterative Closest Point (ICP), we compute the estimated transformations Ttpred=[Rtpred∣ttpred]T_t^{\mathrm{pred}} = [R_t^{\mathrm{pred}} \mid t_t^{\mathrm{pred}}] between consecutive frames and compare them to the true transformations Ttgt=[Rtgt∣ttgt]T_t^{\mathrm{gt}} = [R_t^{\mathrm{gt}} \mid t_t^{\mathrm{gt}}]:

TTCErot=1T−1∑t=1T−1∥Rtpred(Rtgt)⊤−I∥F(9)\mathrm{TTCE}_{\mathrm{rot}} = \frac{1}{T-1} \sum_{t=1}^{T-1} \left\| R_t^{\mathrm{pred}} (R_t^{\mathrm{gt}})^\top- I \right\|_F \tag*{(9)}
TTCEtrans=1T−1∑t=1T−1∥ttpred−ttgt∥22(10)\mathrm{TTCE}_{\mathrm{trans}} = \frac{1}{T-1} \sum_{t=1}^{T-1} \left\| t_t^{\mathrm{pred}} - t_t^{\mathrm{gt}} \right\|_2^2 \tag*{(10)}

Smaller TTCE values indicate that the predicted motion closely matches the ground truth, reflecting smoother and more realistic temporal dynamics.

Chamfer Temporal Consistency (CTC) measures the frame-to-frame geometric smoothness of generated point clouds. Consecutive frames are aligned using the ground-truth transformations:

P~t+1=(Rtgt)−1(Pt+1−ttgt)(11)\widetilde{P}_{t+1} = (R_t^{\mathrm{gt}})^{-1} \left(P_{t+1} - t_t^{\mathrm{gt}}\right) \tag*{(11)}

The Chamfer Distance between PtP_t and the aligned frame P~t+1\widetilde{P}_{t+1} is:

CD(Pt,P~t+1)=1∣Pt∣∑x∈Ptmin⁡y∈P~t+1∥x−y∥22+1∣P~t+1∣∑y∈P~t+1min⁡x∈Pt∥y−x∥22(12)\begin{aligned} \mathrm{CD}(P_t, \widetilde{P}_{t+1}) ={}& \frac{1}{|P_t|} \sum_{x \in P_t} \min_{y \in\widetilde{P}_{t+1}} \|x-y\|_2^2 \\ &+ \frac{1}{|\widetilde{P}_{t+1}|} \sum_{y \in\widetilde{P}_{t+1}} \min_{x \in P_t} \|y-x\|_2^2 \tag*{(12)} \end{aligned}

The overall CTC score is the average Chamfer Distance over all consecutive frame pairs:

CTC=1T−1∑t=1T−1CD(Pt,P~t+1)(13)\mathrm{CTC} = \frac{1}{T-1} \sum_{t=1}^{T-1} \mathrm{CD}(P_t, \widetilde{P}_{t+1}) \tag*{(13)}

Lower CTC values correspond to smoother temporal transitions and higher frame-to-frame coherence in the generated sequence.

Rollout-level Metrics

To assess temporal consistency and scene fidelity across a sequence altogether, we evaluate the quality of LiDAR rollouts, i.e. the accumulation of generated LiDAR frames over time.

Global Alignment and Accumulation. Following the KITTI-360 protocol [23], at each time step tt we transform each frame’s generated point cloud Ptego\mathcal{P}_t^{\mathrm{ego}} to the global coordinate frame using the ground-truth ego-pose TtT_t:

P^tglobal={Ttpt,sego:pt,sego∈Ptego}.(14)\widehat{\mathcal{P}}_t^{\mathrm{global}} = \left\{ T_t p_{t,s}^{\mathrm{ego}} : p_{t,s}^{\mathrm{ego}} \in\mathcal{P}_t^{\mathrm{ego}} \right\}. \tag*{(14)}

A rollout for a sequence of length TT is then defined as the set union of the aligned point clouds:

Pgenrollout=⋃t=1TP^tglobal.(15)\mathcal{P}_{\mathrm{gen}}^{\mathrm{rollout}} = \bigcup_{t=1}^{T} \widehat{\mathcal{P}}_t^{\mathrm{global}}. \tag*{(15)}

We compare rollout-level statistics using both generated and ground-truth sequences Pgenrollout\mathcal{P}_{\mathrm{gen}}^{\mathrm{rollout}} and Prealrollout\mathcal{P}_{\mathrm{real}}^{\mathrm{rollout}}. This procedure captures static scene structure (e.g., buildings, road geometry) as well as the spatiotemporal evolution of dynamic objects (e.g., vehicles, pedestrians) (Figure 8).

Figure 8. Rollout examples of temporal consistency on nuScenes and Waymo. For each dataset, we visualize a scene composed of multiple point clouds acquired at different timestamps. The consistent alignment of the colored regions across time indicates that the reconstruction preserves temporal consistency. The highlighted objects include a vehicle (orange), a pedestrian (blue), vegetation (green), and background structures (purple).

We compute the Jensen-Shannon divergence (JSD) and maximum mean discrepancy (MMD) as described in § 9.1, but using BEV histograms aggregated over the combined rollout point cloud.

We further quantify geometric fidelity between the ground-truth and generated rollout point clouds using the symmetric Chamfer Distance (CD):

CD(Preal,Pgen)=1∣Preal∣∑p∈Prealmin⁡p^∈Pgen∥p−p^∥22+1∣Pgen∣∑p^∈Pgenmin⁡p∈Preal∥p^−p∥22(16)\mathrm{CD}(\mathcal{P}_{\mathrm{real}}, \mathcal{P}_{\mathrm{gen}}) = \frac{1}{|\mathcal{P}_{\mathrm{real}}|} \sum_{p \in\mathcal{P}_{\mathrm{real}}} \min_{\hat{p} \in\mathcal{P}_{\mathrm{gen}}} \|p-\hat{p}\|_2^2 + \frac{1}{|\mathcal{P}_{\mathrm{gen}}|} \sum_{\hat{p} \in\mathcal{P}_{\mathrm{gen}}} \min_{p \in\mathcal{P}_{\mathrm{real}}} \|\hat{p}-p\|_2^2 \tag*{(16)}

where Prealrollout\mathcal{P}_{\mathrm{real}}^{\mathrm{rollout}} and Pgenrollout\mathcal{P}_{\mathrm{gen}}^{\mathrm{rollout}} denote the accumulated ground-truth and model-generated rollout point clouds, respectively.

These rollout-level metrics provide a holistic measure of both structural realism and temporal coherence in generative LiDAR modeling.

Video Evaluation Metrics

The temporal consistency metrics (TTCE and CTC) discussed above primarily assess rigid geometric alignment between consecutive frames. However, these metrics do not adequately capture the quality of dynamic scenes in which objects undergo non-rigid motion or deformation. To address this limitation, we adapt established video quality metrics from the computer vision literature to the LiDAR domain.

We evaluate video generation quality through three metrics organized across two complementary dimensions:

Temporal Consistency. To quantify temporal stability without relying on rigid transformations, we employ an optical-flow warping error EwE_w and a relative Blind Video temporal Consistency index (BVC) [67]. We estimate dense optical flow [10] between consecutive frames, warp the previous frame toward the current frame using the estimated flow field, and compute a masked photometric discrepancy on non-occluded pixels. We aggregate both short-range errors (between consecutive frames) and long-range errors (relative to the first frame) to obtain EwE_w. The BVC index is defined as BVC=Ewgt/Ewpred\mathrm{BVC} = E_w^{\mathrm{gt}}/E_w^{\mathrm{pred}}, providing a normalized measure where BVC>1\mathrm{BVC} > 1 indicates superior temporal smoothness compared to ground truth, BVC=1\mathrm{BVC} = 1 indicates matching stability, and BVC<1\mathrm{BVC} < 1 indicates increased flicker or temporal instability.

Video-level Realism and Motion. At the distributional level, Fréchet Video Distance (FVD) [50] compares real and generated videos in the feature space of an I3D network [5] using 16-frame clips at 224×224224 \times224 resolution. Complementing FVD’s appearance-focused evaluation, Fréchet Video Motion Distance (FVMD) [24] compares distributions of tracked keypoint trajectories, providing a motion-centric assessment of temporal dynamics.

Ablation Studies

We conduct ablations on the VAE design choice, the diffusion backbone, and other key designs for their validation. Unless specified, all ablation experiments are conducted on the Waymo Open Dataset [49], for multi-sensor LiDAR scene generation (t=1t = 1) setting.

VAE

Our framework adopts the Latent Diffusion [43] paradigm, employing a VAE for compressing high-dimensional range images into a compact latent space. We base our VAE design on recent state-of-the-art architectures, LiDM [41] and RangeLDM [13], and systematically study the impact of key modeling choices.

All ablations are performed with a fixed compression ratio f=64f = 64, a batch size of 16, and 50 training epochs, following the default settings for each method’s official implementation. Following [19, 36], we adopt an image-based VAE (2D) instead of a temporal 3D architecture, as the goal here is efficient per-frame compression of range images rather than joint spatiotemporal modeling. This architecture also enables more efficient training and inference, similar to designs in [19, 36].

Table 11 presents quantitative VAE reconstruction results for different configurations on Waymo SIDE LiDAR. We observe the following: (1) The use of Circular Convolution [45] in RangeLDM leads to improved latent representations relative to LiDM across all metrics (cf. rows 3–6 vs 1–2), confirming the benefit of circularity for sensors with 360∘360^{\circ} coverage. (2) Incorporating adversarial loss into VAE training is challenging, with performance highly sensitive to the activation schedule; improper tuning can significantly degrade results (cf. rows 1–2 and 3–4). (3) VAE performance deteriorates with aggressive input resizing, likely due to increased information loss (cf. rows 3–4 vs rows 5–6). Nevertheless, this resizing may be necessary for compatibility with U-Net-based diffusion backbones, which constrain input resolution and highlight a limitation of the U-Net design. (4) For SIDE LiDAR on Waymo (range 0–20 m), log compression (Equation (17)) provides little benefit, since the dynamic range is limited and linear normalization (Equation (3)) already preserves the range scale adequately without introducing an additional nonlinear transformation (cf. rows 6–7 and Figure 9).

Waymo range value distributions for TOP and SIDE LiDAR, before and after logarithmic transformation

Figure 9. Impact of Logarithmic Transformation on LiDAR range distributions. Left: For TOP LiDAR, the log transform significantly reduces the skewness of high dynamic range values. Right: For SIDE LiDAR on Waymo, where the range values are less skewed and cover a shorter distance, the effect of the log transform is minimal.

Diffusion Backbone: DiT vs U-Net

For LiDAR generation in the Latent Diffusion framework [43], the choice of diffusion backbone is critical. Range images, representing 360∘360^\circ LiDAR sweeps as anisotropic 2D panoramas, exhibit a pronounced width–height asymmetry: the width spans the full angular resolution, whereas the limited number of sensor beams constrains the height. This results in unique long-range horizontal dependencies that differ fundamentally from those in natural images. In this section, we ablate U-Net and Diffusion Transformer (DiT) [38] backbones, comparing their effectiveness in modeling these dependencies and their parameter efficiency.

While U-Net architectures remain a popular backbone that preserves local structures, they are inherently limited in modeling the extreme aspect ratios and horizontal context of range images. In contrast, DiT leverages global self-attention to capture long-range dependencies directly and offers flexible input resolution via dynamic padding, therefore avoiding the destructive resizing often required by the U-Net. As observed in § 10.1 and Figure 10, maintaining native resolution is crucial, as resizing artifacts can significantly degrade downstream point cloud reconstruction.

Effects of range image resizing on generated point clouds

Figure 10. Effects of range image resizing on generated point clouds. Resizing the range image from (200, 600) to (224, 608) introduces noise and geometric distortions in the resulting point cloud, noticeably affecting object shapes (e.g., trees, cars). Best viewed when zoomed in.

Table 12 reports single-frame LiDAR generation results for state-of-the-art methods LiDM [41] and RangeLDM [13] across multiple backbones. LiDM demonstrates strong performance on Waymo SIDE LiDAR generation, notably outperforming RangeLDM (cf. rows 1 vs. 3), with further gains when employing DiT. We find that the default hyperparameters of RangeLDM require more tuning (e.g., learning rates), particularly for side LiDARs with smaller dynamic ranges and higher cross-sensor correlation demands. DiT-based models outperform U-Nets across most settings (cf. rows 1 vs. 2 and 3 vs. 4), while using approximately 50% fewer parameters and reducing GPU memory consumption to about two-thirds of that required by U-Net. These results highlight the parameter-efficiency advantages of DiT over U-Nets, validating the benefits of transformer-based backbones for LiDAR range image generation.

MethodInput SizeBackboneRange FRD ↓Range MMD ↓Points FPVD ↓Points MMD ↓BEV JSD ↓BEV MMD ↓
LiDM†^{\dagger}[41]208×640208 \times640U-Net2.29148.834.6765.760.191.00
200×600200 \times600DiT0.4577.912.52113.330.151.00
RangeLDM†^{\dagger}[13]224×608224 \times608U-Net10.97979.33122.381236.590.221.63
200×600200 \times600DiT9.07938.8361.47374.810.191.16
CrossLiDAR†^{\dagger} (t=5t=5)200×600200 \times600DiT + Naive-Stacking1.6655.6714.82156.550.201.62
MM-DiT + Naive-Stacking1.4351.9414.48139.310.161.51
MM-DiT + LiDAR-Sequence + Inter-Sensor0.1611.891.326.900.081.05

Table 12. Ablations on Diffusion Backbone on Waymo SIDE LiDAR. Configuration †\dagger: compression rate f=64f = 64, range densification, filling invalid points and without discriminator. The best result is highlighted in bold and the second-best in _underline_.

Finally, we perform two ablation studies on the proposed CrossLiDAR architecture. First, we compare DiT and MM-DiT under the same Naive Stacking strategy, where temporal and multi-sensor latent representations are simply concatenated before being processed by the backbone. This isolates the contribution of the backbone architecture itself. Second, we compare this Naive Stacking baseline against the proposed LiDAR Sequence and Inter-sensor modules while keeping the MM-DiT backbone fixed, therefore isolating the contribution of explicit temporal and cross-sensor modeling. Table 12 shows that replacing DiT with MM-DiT under the same Naive Stacking setting improves the results by an average of 10.1% across all evaluation metrics. In contrast, replacing Naive Stacking with the proposed LiDAR Sequence and Inter-sensor modules yields substantially larger gains, reducing FRD by 88.8%, point-cloud FPD by 90.9%, and BEV JSD by 50.0% relative to the MM-DiT baseline. These results demonstrate that the improvements reported in the main paper are largely driven by the combined proposed temporal and cross-sensor modules rather than by the MM-DiT backbone alone.

Baselines

We compare CrossLiDAR with a broad suite of recent methods for LiDAR generation. Table 13 summarizes the key characteristics of each baseline, including problem setting, network architecture, generative paradigm, datasets used for training, and the availability of code and pretrained checkpoints at the time of submission. In contrast to prior work, CrossLiDAR is specifically designed for the challenging multi-sensor LiDAR sequence generation setting and supports a wide range of input conditions.

VenueModalityLiDAR SensorsLiDAR FramesSupported ConditionsDiffusion ModelDatasetCodeCheckpoint
LiDAR Diffusion[41]CVPR’24LiDAR11image,semantic mapU-NetKYesK
RangeLDM[13]ECCV’24LiDAR11inpaint,downsampleU-NetK,NYesK,N
R2DM[35]ICRA’24LiDAR11—U-NetKYesK
LiDARDM[75]ICRA’25LiDAR1multimapU-NetK,WYesK,W
UniScene[19]CVPR’25LiDAR &Camera1multiBEV→semantic occupancySparse U-NetNYesN
OpenDWM[36]CVPR’25LiDAR1multiBEV,map,textU-Net/DiTNYesN
LiDARCrafter[22]AAAI’26LiDAR1multitext→layout→object→trajectoryU-NetNYes—
CrossLiDAR—LiDARmultimultiBEV,object,image,textMM-DiTN,W——

Table 13. Summary of existing methods for LiDAR generation. Overview of compared methods, indicating sensor setup, generative model, available modalities, supported input conditions, training datasets (K: KITTI-360 [23], N: nuScenes [4], W: Waymo [49]), and code/checkpoint availability.

Many existing methods [13, 35, 41, 74] are designed for single-sensor LiDAR generation. To ensure fair comparison, we adapt these methods for multi-sensor LiDAR generation, as detailed in the following sections, which also specify the training hyperparameters we used on Waymo. Certain methods could not be included in our Waymo experiments due to design complexity or prohibitive data requirements. For example, UniScene [19] requires semantic occupancy annotations, which are not readily available at scale for Waymo. Similarly, LiDARDM [75], a rendering-based technique that relies on Neural Kernel Surface Reconstruction [14] and ray-casting, is not directly comparable to range-image or voxel-based LiDAR generation.

R2DM

Adaptation for Waymo. We adapt R2DM [35] for the Waymo Dataset with four SIDE LiDAR sensors: front, left, rear, and right. The range image resolution is set to 200×600200 \times600 with a single depth channel, and depth values span between [1,20][1, 20] meters. Ray angles are initialized for a vertical FOV from 30∘30^\circ (top) to −90∘-90^\circ (bottom). For the SIDE LiDAR sensors, we use the effective horizontal FOV estimated from valid range-image columns, following § 7.2. To handle multiple sensors, we stack their views into a [B,S,C,H,W][B, S, C, H, W] tensor (S=4S = 4 sensors) and reshape to [B×S,C,H,W][B \times S, C, H, W] for efficient batch processing as in Figure 11.

Naive sensor stacking for multi-sensor LiDAR generation

Figure 11. Naive sensor stacking for multi-sensor LiDAR generation. Each sensor range image at time step tt is independently encoded by the VAE. The diffusion model operates on the stacked multi-sensor latents, enabling generation (or prediction) of joint sensor, and can be extended to sequences by stacking along the temporal dimension.

Training Configuration. Following the upstream implementation7, we train with AdamW (betas 0.9/0.99, weight decay 0.0, ϵ=10−8\epsilon=10^{-8}), a base learning rate of 4.5×10−64.5 \times10^{-6} scaled to 3.6×10−53.6 \times10^{-5}. The learning rate follows a cosine annealing schedule with a 10k-step warmup, and training proceeds for 300k optimization steps. Training uses gradient clipping with a maximum norm of 1.0 and an exponential moving average (EMA, β=0.995\beta= 0.995) with updates every 10 steps. The code is run with Python 3.12 and PyTorch version 2.9.0.

LiDM

Adaptation for Waymo. We adapt LiDM [41] to Waymo SIDE LiDAR range images by using depth-only inputs. The range images are resized to 208×640208 \times640 to match the down-sampling constraints of its U-Net backbone. Depth values are clipped to [1,20][1, 20] for compatibility with the LiDM baseline and normalized using the logarithmic mapping from the original work:

dnorm=log⁡2(d+1)log⁡2(dmax⁡+1)×2−1(17)d^{\mathrm{norm}} = \frac{\log_{2}(d+1)}{\log_{2}(d_{\max}+1)} \times2 - 1 \tag*{(17)}

To accommodate multiple sensors, we train a single 2D VAE shared across sensors while encoding each sensor independently (i.e., without cross-sensor interaction). We use a compression setting (f=64f=64) with curve-wise factor fc=4f_c=4 and patch-wise factor fp=4f_p=4, producing latents of size 16×52×4016 \times52 \times40 per sensor. For diffusion training, we stack latents across sensors and time: per-sensor latents are arranged as [B,S,16,52,40][B, S, 16, 52, 40] and reshaped to [B,S×16,52,40][B, S \times16, 52, 40]; for sequences, we additionally fold the temporal dimension TT into the channel axis, yielding [B,T×S×16,52,40][B, T \times S \times16, 52, 40], as shown in Figure 11.

Training configuration. Unless stated otherwise, we follow the official implementation8. We retain the adversarial loss in the VAE objective. The VAE (encoder/decoder and discriminator) is trained with Adam (betas 0.5/0.9) and learning rate 4.5×10−64.5 \times10^{-6}; we train the diffusion model with learning rate 1×10−61 \times10^{-6}. We train for 300k optimization steps with a batch size of 8, and use EMA with decay β=0.995\beta=0.995 updated every 10 steps. Experiments are run with Python 3.12 and PyTorch version 2.9.0.

RangeLDM

Adaptation for Waymo. We adapt RangeLDM [13] to Waymo SIDE LiDAR range images following the same multi-sensor stacking strategy as for LiDM (Figure 11). We use the default RangeLDM compression setting with patch factor fp=8f_p=8, corresponding to an overall compression ratio f=64f=64.

Training configuration. We follow the official implementation9 and train for 500k optimization steps (batch size 8 per GPU; Python 3.12; PyTorch 2.9.0). In preliminary experiments, we observed training instability when generating Waymo SIDE LiDAR data: although the VAE achieves strong reconstruction performance (Table 11 and Table 12), diffusion training may collapse mid-training, leading to a sharp degradation in sample quality without obvious numerical failures. We attribute this sensitivity primarily to the learning rate. While the original work uses 10−410^{-4}, we reduce the learning rate to 10−610^{-6} for Waymo; with the original setting, training does not reliably progress.

MethodCompressionInput SizeTransformVariantR-FSVD ↓R-FPVD ↓CD (10−310^{-3}) ↓EMD (10−310^{-3}) ↓
LiDM[41]fc=4f_c = 4, fp=4†f_p = 4^{\dagger}208×640208 \times640LogNorm—426.71191.74505.98102.68
fc=4f_c = 4, fp=4†f_p = 4^{\dagger}208×640208 \times640LogNormw/o LDescL_{\mathrm{Desc}}1.830.322.8116.25
RangeLDM[13]fp=8†f_p = 8^{\dagger}224×608224 \times608LogNorm—3.940.313.4323.96
fp=8†f_p = 8^{\dagger}224×608224 \times608LogNormw/o LDescL_{\mathrm{Desc}}1.260.190.987.39
fp=8†f_p = 8^{\dagger}200×600200 \times600LogNorm—3.550.271.2713.11
fp=8†f_p = 8^{\dagger}200×600200 \times600LogNormw/o LDescL_{\mathrm{Desc}}1.440.150.494.40
CrossLiDAR VAEfp=8†f_p = 8^{\dagger}200×600200 \times600Linearw/o LDescL_{\mathrm{Desc}}0.410.130.484.37

Table 11. VAE ablation results on Waymo SIDE LiDAR. Reconstruction quality is compared across VAE backbones and normalization strategies. Configuration †\dagger: compression rate f=64f = 64, range densification, and filling invalid points. LDescL_{\mathrm{Desc}} denotes the discriminator (adversarial) loss. Performance is evaluated using perceptual metrics: R-FSVD, R-FPVD and statistical metrics: Chamfer Distance (CD), Earth Mover’s Distance (EMD), following [41]. The best result is highlighted in bold and the second-best in _underline_.

OpenDWM

Adaptation for Waymo. We adapt the LiDAR generation components provided in the OpenDWM codebase10 to the Waymo SIDE LiDAR setting. Specifically, we use its LiDAR VAE tokenizer together with the temporal DiT diffusion model. Because the released configuration is designed for sequence generation from a single LIDAR_TOP stream, we replace the original single-sensor setup with the four SIDE sensors (front, left, rear, right) and apply the same naive multi-sensor stacking strategy used for LiDM and RangeLDM (Figure 11). Each sensor is encoded independently using a shared VAE. The resulting latent tensors are then stacked along the sensor and temporal dimensions before being processed by the temporal diffusion transformer. We otherwise retain the point-cloud voxelization and latent parameterization from the official release, which voxelizes the scene within [−50,50]×[−50,50]×[−3,5][-50,50] \times[-50,50] \times[-3,5] m volume with voxel sizes (0.15625,0.15625,0.125)(0.15625, 0.15625, 0.125) and uses a 32-channel latent representation.

Training configuration. We follow the public OpenDWM LiDAR configurations as closely as possible. The released VAE configuration uses AdamW with learning rate 10−410^{-4} and betas (0.9,0.95)(0.9, 0.95). The temporal DiT configuration uses AdamW with learning rate 5×10−55 \times10^{-5}, betas (0.9,0.99)(0.9, 0.99), and weight decay 10−410^{-4}, together with cosine annealing (Tmax⁡=100000T_{\max}=100000, ηmin⁡=10−5\eta_{\min}=10^{-5}). The official temporal configuration enables gradient checkpointing, uses batch size 1, a FlowMatch Euler scheduler with 1000 training timesteps, and conditions on three reference frames. We note that the released OpenDWM LiDAR models are primarily configured for LiDAR_TOP data and layout-conditioned generation. As a result, the Waymo SIDE LiDAR setting requires non-trivial adaptation and is not a native training configuration in the original release.

Waymo Qualitative Results

In this section, we present qualitative results on the Waymo dataset. We highlight the impact of our architectural modules on the generation quality, demonstrating improvements in cross-sensor alignment, temporal consistency, and controllable scene synthesis.

Inter-sensor Alignment. The Inter-sensor Module provides clear benefits by enabling joint modeling of all LiDAR sensors within the same temporal frame. Instead of processing each sensor independently, the module allows the network to learn direct interactions across viewpoints through attention. As shown in Figure 12, the module enables coherent multi-view integration, leading to improved alignment and more accurate scene understanding compared to the independent sensor generation of the competitors.

Figure 12. Cross-Sensor Alignment on Waymo. Qualitative comparison between our method (the last two rows) and competing approaches (the first row). While competitors generate each sensor stream independently, leading to misalignment and geometric inconsistencies across viewpoints, our Inter-Sensor Module enables coherent multi-view generation with consistent object geometry and spatial alignment across all LiDAR sensors.

Temporal Consistency. One of the key capabilities of our approach, enabled by the LiDAR Sequence Module, is the modeling of long-range spatio-temporal dependencies within each sensor stream. By jointly attending over spatial tokens across multiple time steps under a causal constraint, the module captures motion patterns and temporal context while preventing information leakage from future frames. This design enforces temporal coherence during generation and improves reasoning over evolving 3D structures. As shown in Figure 13, the LiDAR Sequence Module yields smoother predictions, more stable object representations over time, and enhanced dynamic scene understanding.

Temporal consistency and controllable generation on Waymo

Figure 13. Temporal Consistency and Controllable Generation on Waymo. Qualitative results demonstrating both temporal coherence and controllable generation. The first column shows BEV and 3D-box conditioning signals, while the remaining columns show generated LiDAR frames at different timesteps. Across consecutive frames, CrossLiDAR preserves stable object geometry and motion consistency while adhering to structural control signals for object placement and scene layout.

Controllable LiDAR Generation. By conditioning the model on 3D bounding boxes and BEV maps through the ControlNet-style branch, we enable precise object-level control during generation. These structural cues guide the generation, allowing the model to accurately place objects according to the specified layout. As shown in Figure 13, this mechanism reliably regulates object presence and spatial arrangement, resulting in strong alignment between the input structural constraints and the generated LiDAR scenes.

References

  1. [1]Hengwei Bian, Lingdong Kong, Haozhe Xie, Liang Pan, Yu Qiao, and Ziwei Liu. Dynamicity: Large-scale 4d occupancy generation from dynamic scenes. In International Conference on Learning Representations (ICLR), 2025. 3arxiv.org/abs/2410.18084
  2. [2]Nicolas Bonneel, James Tompkin, Kalyan Sunkavalli, Deqing Sun, Sylvain Paris, and Hanspeter Pfister. Blind video temporal consistency. ACM Trans. Graph., 34(6), 2015. 6DOI
  3. [3]Lucas Caccia, Herke Van Hoof, Aaron Courville, and Joelle Pineau. Deep generative modeling of LiDAR data. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5034–5040. IEEE, 2019. 1arxiv.org/abs/1812.01180
  4. [4]Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuScenes: A multi-modal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 2, 5, 12, 20
  5. [5]João Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4724–4733, 2017. 17
  6. [6]Boyuan Chen, Diego Martí Monsó, Yulin Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081–24125, 2024. 5arxiv.org/abs/2407.01392
  7. [7]Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, pages 1–16, 2017. 1arxiv.org/abs/1711.03938
  8. [8]Gamaleldin Elsayed, Aravindh Mahendran, Sjoerd Van Steenkiste, Klaus Greff, Michael C Mozer, and Thomas Kipf. Savi++: Towards end-to-end object-centric learning from real-world videos. Advances in Neural Information Processing Systems, 35:28940–28954, 2022. 3
  9. [9]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Proceedings of the 41st International Conference on Machine Learning, pages 12606–12633, 2024. 2, 3, 4
  10. [10]Xiaonan Fang and Song-Hai Zhang. Single-video temporal consistency enhancement with rolling guidance. In Computational Visual Media: 12th International Conference, CVM 2024, Wellington, New Zealand, April 10–12, 2024, Proceedings, Part II, page 109–130, Berlin, Heidelberg, 2024. Springer-Verlag. 17
  11. [11]Yongjie Fu, Anmol Jain, Xu Chen, Zhaobin Mo, and Xuan Di. Drivinggen: Real-world video generation for vision language model based autonomous driving. In 2024 IEEE International Automated Vehicle Validation Conference (IAVVC), pages 1–6. IEEE, 2024. 2
  12. [12]Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020. 1
  13. [13]Qianjiang Hu, Zhimin Zhang, and Wei Hu. RangeLDM: Fast realistic LiDAR point cloud generation. In European Conference on Computer Vision, pages 115–135. Springer, 2024. 2, 3, 4, 6, 17, 18, 19, 20, 21
  14. [14]Jiahui Huang, Zan Gojcic, Matan Atzmon, Or Litany, Sanja Fidler, and Francis Williams. Neural kernel surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4369–4379, 2023. 18arxiv.org/abs/2305.19590
  15. [15]Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. In Advances in Neural Information Processing Systems, pages 167283–167308. Curran Associates, Inc., 2025. 5
  16. [16]Tarasha Khurana, Peiyun Hu, David Held, and Deva Ramanan. Point cloud forecasting as a proxy for 4d occupancy forecasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1116–1124, 2023. 3
  17. [17]Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 1, 4arxiv.org/abs/1312.6114
  18. [18]Jason Ku, Ali Harakeh, and Steven L Waslander. In defense of classical image processing: Fast depth completion on the cpu. In 2018 15th conference on computer and robot vision (CRV), pages 16–22. Ieee, 2018. 12, 14
  19. [19]Bohan Li, Jiazhe Guo, Hongsi Liu, Yingshuang Zou, Yikang Ding, Xiwu Chen, Hu Zhu, Feiyang Tan, Chi Zhang, Tiancai Wang, et al. Uniscene: Unified occupancy-centric driving scene generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 11971–11981, 2025. 6, 7, 8, 17, 18, 20
  20. [20]Chenqi Li, Yuan Ren, and Bingbing Liu. Pcgen: Point cloud generator for LiDAR simulation. arXiv preprint arXiv:2210.08738, 2022. 1
  21. [21]Xiaofan Li, Yifu Zhang, and Xiaoqing Ye. Drivingdiffusion: layout-guided multi-view driving scenarios video generation with latent diffusion model. In European Conference on Computer Vision, pages 469–485. Springer, 2024. 2
  22. [22]Alan Liang, Youquan Liu, Yu Yang, Dongyue Lu, Linfeng Li, Lingdong Kong, Huaici Zhao, and Wei Tsang Ooi. Lidar-crafter: Dynamic 4d world modeling from lidar sequences. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 18406–18414, 2026. 2, 3, 6, 7, 8, 16, 20
  23. [23]Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2023. 16, 20
  24. [24]Jiahe Liu, Youran Qu, Qi Yan, Xiaohui Zeng, Lele Wang, and Renjie Liao. Erect video motion distance: A metric for evaluating motion consistency in videos. arXiv preprint arXiv:2407.16124, 2024. 6, 17
  25. [25]Xingyu Liu, Charles R Qi, and Leonidas J Guibas. Flownet3d: Learning scene flow in 3D point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 529–537, 2019. 3
  26. [26]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. 2, 4
  27. [27]Youquan Liu, Lingdong Kong, Weidong Yang, Xin Li, Alan Liang, Runnan Chen, Ben Fei, and Tongliang Liu. La la lidar: Large-scale layout generation from point cloud data. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7377–7385, 2026. 2
  28. [28]Youquan Liu, Lingdong Kong, Weidong Yang, Ao Liang, Jianxiong Gao, Yang Wu, Xiang Xu, Xin Li, Linfeng Li, Runnan Chen, et al. Veila: Panoramic lidar generation from a monocular rgb image. In 2026 IEEE International Conference on Robotics and Automation (ICRA), pages 10426–10433. IEEE, 2026. 2arxiv.org/abs/2508.03690
  29. [29]Zhen Luo, Junyi Ma, Zijie Zhou, and Guangming Xiong. Pcpnet: An efficient semantic-enhanced transformer network for point cloud prediction. IEEE Robotics and Automation Letters, 8(7):4267–4274, 2023. 3arxiv.org/abs/2304.07773
  30. [30]Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyu an Zeng, Mikita Sazanovich, Shuhan Tan, Bin Yang, Wei-Chiu Ma, and Raquel Urtasun. LidarSim: Realistic LiDAR simulation by leveraging the real world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11167–11176, 2020. 1arxiv.org/abs/2006.09348
  31. [31]Benedikt Mersch, Xieyuanli Chen, Jens Behley, and Cyrill Stachniss. Self-supervised point cloud prediction using 3D spatio-temporal convolutional networks. In Conference on Robot Learning, pages 1444–1454. PMLR, 2022. 3arxiv.org/abs/2110.04076
  32. [32]Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss. Rangenet++: Fast and accurate LiDAR semantic segmentation. In 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 4213–4220. IEEE, 2019. 15DOI
  33. [33]Javier Montalvo, Juan Ignacio Bravo Pérez-Villar, Álvaro García-Martín, Pablo Carballeira, and Jesús Bescós. SPIN: Spacecraft imagery for navigation. arXiv preprint arXiv:2406.07500, 2024. 1arxiv.org/abs/2406.07500
  34. [34]Arian Mousakhan, Sudhanshu Mittal, Silvio Galesso, Karim Farid, and Thomas Brox. Orbis: Overcoming challenges of long-horizon prediction in driving world models. arXiv preprint arXiv:2507.13162, 2025. 2arxiv.org/abs/2507.13162
  35. [35]Kazuto Nakashima and Ryo Kurazume. LiDAR data synthesis with denoising diffusion probabilistic models. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14724–14731. IEEE, 2024. 6, 18, 20arxiv.org/abs/2309.09256
  36. [36]Jingcheng Ni, Yuxin Guo, Xichen Liu, Rui Chen, Lewei Lu, and Zehuan Wu. MaskGWM: A generalizable driving world model with video mask reconstruction. arXiv preprint arXiv:2502.11663, 2025. 6, 7, 8, 17, 20
  37. [37]Kaustab Pal, Aditya Sharma, Avinash Sharma, and K Madhava Krishna. ATPPNet: Attention based temporal point cloud prediction network. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 11140–11146. IEEE, 2024. 3arxiv.org/abs/2401.17399
  38. [38]William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023. 18arxiv.org/abs/2212.09748
  39. [39]Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017. 15arxiv.org/abs/1612.00593
  40. [40]Hui Qian, Mingchen Wang, Maotao Zhu, and Hai Wang. A review of multi-sensor fusion in autonomous driving. Sensors, 25(19):6033, 2025. 2DOI
  41. [41]Haoxi Ran, Vitor Guizilini, and Yue Wang. Towards realistic scene generation with LiDAR diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14738–14748, 2024. 2, 3, 6, 17, 18, 19, 20arxiv.org/abs/2404.00815
  42. [42]Jonathan P Resop, Laura Lehmann, and W Cully Hession. Drone laser scanning for modeling riverscape topography and vegetation: Comparison with traditional aerial LiDAR. Drones, 3(2):35, 2019. 1DOI
  43. [43]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 3, 17, 18arxiv.org/abs/2112.10752
  44. [44]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015. 2arxiv.org/abs/1505.04597
  45. [45]Stefan Schubert, Peer Neubert, Johannes Pöschmann, and Peter Protzel. Circular convolutional neural networks for panoramic images and laser data. In 2019 IEEE intelligent vehicles symposium (IV), pages 653–660. IEEE, 2019. 17DOI
  46. [46]SenseTime-FVG. OpenDWM: Open driving world models, 2025. https://github.com/SenseTime-FVG/OpenDWM. 6, 7, 8
  47. [47]Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-rcnn: Point-voxel feature set abstraction for 3D object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10529–10538, 2020. 1arxiv.org/abs/1912.13192
  48. [48]Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024. 5arxiv.org/abs/2104.09864
  49. [49]Pei Sun, Henrik Kretzschmar, Xerxes Dotwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 2, 3, 6, 12, 17, 20arxiv.org/abs/1912.04838
  50. [50]Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018. 6, 17arxiv.org/abs/1812.01717
  51. [51]Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Advances in Neural Information Processing Systems (NeurIPS), 2017.arxiv.org/abs/1711.00937
  52. [52]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.arxiv.org/abs/1706.03762
  53. [53]Arun Venkatraman, Martial Hebert, and J. Bagnell. Improving multi-step prediction of learned time series models. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1), 2015.DOI
  54. [54]Qitai Wang, Lue Fan, Yuqi Wang, Yuntao Chen, and Zhaoxiang Zhang. Generative view synthesis on free driving trajectory. In International Conference on Learning Representations, pages 58640–58662, 2025.arxiv.org/abs/2410.18079
  55. [55]Ziyu Wang, Jing Ma, and Edmund MK Lai. A survey of scenario generation for automated vehicle testing and validation. Future Internet, 16(12):480, 2024.DOI
  56. [56]Xinshuo Weng, Jianren Wang, Sergey Levine, Kris Kitani, and Nicholas Rhinehart. Inverting the pose forecasting pipeline with spf2: Sequential pointcloud forecasting for sequential pose forecasting. In Conference on robot learning, pages 11–20. PMLR, 2021.arxiv.org/abs/2003.08376
  57. [57]Xinshuo Weng, Junyu Nan, Kuan-Hui Lee, Rowan McAllister, Adrien Gaidon, Nicholas Rhinehart, and Kris M Kitani. S2net: Stochastic sequential pointcloud forecasting. In European Conference on Computer Vision, pages 549–564. Springer, 2022.DOI
  58. [58]Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhonny Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting. arXiv preprint arXiv:2301.00493, 2023.
  59. [59]Jin Wu and Chengxi Zhang. Multi-LiDAR registration: A joint sensor-centric optimization approach. IEEE Sensors Letters, 10(2):1–4, 2025.DOI
  60. [60]Ziyi Wu, Yulia Rubanova, Rishabh Kabra, Drew A Hudson, Igor Gilitschenski, Yusuf Aytar, Sjoerd Van Steenkiste, Kelsey R Allen, and Thomas Kipf. Neural assets: 3D-aware multi-object scene synthesis with image diffusion models. Advances in Neural Information Processing Systems, 37:76289–76318, 2024.
  61. [61]Yuwen Yang, Wei-Chiu Ma, Jingkang Wang, and Raquel Urtasun. Learning compact representations for lidar completion and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1074–1083, 2023.arxiv.org/abs/2311.01448
  62. [62]Xiang Xu, Alan Liang, Youquan Liu, Linfeng Li, Lingdong Kong, Ziwei Liu, and Qingshan Liu. U4d: Uncertainty-aware 4d world modeling from lidar sequences. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10027–10039, 2026.arxiv.org/abs/2512.02982
  63. [63]Yinghao Xu, Menglei Chai, Zifan Shi, Sida Peng, Ivan Skorokhodov, Aliaksandr Siarohin, Ceyuan Yang, Yujun Shen, Hsin-Ying Lee, Bolei Zhou, et al. Disconscene: Spatially disentangled generative radiance fields for controllable 3D-aware scene synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4402–4412, 2023.arxiv.org/abs/2212.11984
  64. [64]Yunzhi Yan, Zhen Xu, Haotong Lin, Haian Jin, Haoyu Guo, Yida Wang, Kun Zhan, Xianpeng Lang, Hujun Bao, Xiaowei Zhou, and Sida Peng. StreetCrafter: Street view synthesis with controllable video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.arxiv.org/abs/2412.13188
  65. [65]Zetong Yang, Li Chen, Yanan Sun, and Hongyang Li. Visual point cloud forecasting enables scalable autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14673–14684, 2024.arxiv.org/abs/2312.17655
  66. [66]Tianwei Yin, Xingyi Zhou, and Philipp Krähenbühl. Center-based 3D object detection and tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11784–11793, 2021.arxiv.org/abs/2006.11275
  67. [67]Zhiyu Yin, Kehai Chen, Xuefeng Bai, Ruili Jiang, Juntao Li, Hongdong Li, Jin Liu, Yang Xiang, Jun Yu, and Min Zhang. ASurvey: Spatiotemporal consistency in video generation. arXiv preprint arXiv:2502.17863, 2025.arxiv.org/abs/2502.17863
  68. [68]Mingliang Zhai, Xuezhi Xiang, Ning Lv, and Xiangdong Kong. Optical flow and scene flow estimation: A survey. Pattern Recognition, 114:107861, 2021.DOI
  69. [69]Ji Zhang, Sanjiv Singh, et al. LOAM: LiDAR odometry and mapping in real-time. In Robotics: Science and systems, pages 1–9. Berkeley, CA, 2014.DOI
  70. [70]Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023.arxiv.org/abs/2302.05543
  71. [71]Jian Zhao, Wenxu Li, Bing Zhu, Peixing Zhang, Zhaozheng Hu, and Jie Meng. Corner case dataset for autonomous vehicle testing based on naturalistic driving data. Smart Cities, 8(4):129, 2025.
  72. [72]Yunsong Zhou, Michael Simon, Zhenghao Peng, Sicheng Mo, Hongzi Zhu, Minyi Guo, and Bolei Zhou. Simgen: Simulator-conditioned driving scene generation. Advances in Neural Information Processing Systems, 37:48838–48874, 2024.arxiv.org/abs/2406.09386
  73. [73]Dekai Zhu, Yixuan Hu, Youquan Liu, Dongyue Lu, Lingdong Kong, and Slobodan Ilic. Spiral: Semantic-aware progressive lidar scene generation and understanding. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.arxiv.org/abs/2505.22643
  74. [74]Vlas Zyrianov, Xiye Zhu, and Shenlong Wang. Learning to generate realistic LiDAR point clouds. In European Conference on Computer Vision, pages 17–35. Springer, 2022.arxiv.org/abs/2209.03954
  75. [75]Vlas Zyrianov, Henry Che, Zhijian Liu, and Shenlong Wang. Lidardm: Generative LiDAR simulation in a generated world. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 6055–6062. IEEE, 2025.arxiv.org/abs/2404.02903

Paper details

vlnc:W_fy3x3254
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents