WorksComputer Science
CrossLiDAR: Controllable Multi-Sensor LiDAR Sequence Generation
Abstract
High-fidelity LiDAR simulation remains a bottleneck for scalable development and validation of autonomous systems. Existing generators can synthesize realistic scans, but they are typically constrained to a single sensor configuration, offer limited control over structure and geometry, and drift in long-horizon 4D generation. We present CrossLiDAR, a unified transformer-based framework capable of generating controllable and spatio-temporally consistent multi-sensor LiDAR sequences. Our approach leverages a rectified flow objective with a Multi-Modal Diffusion Transformer (MM-DiT) backbone, within a compact latent space of range images. To support multiple LiDAR sensors, an Inter-sensor Module enables interaction across sensors, while a ControlNet-style branch provides structural conditioning (e.g., BEV priors or 3D bounding boxes). For modeling the temporal consistency, CrossLiDAR generates short -frame sequences and extends them autoregressively to improve long-horizon coherence. Our model significantly outperforms state-of-the-art baselines, achieving an average improvement of 29.56% in JSD and 15.36% in MMD evaluated on nuScenes and the Waymo Dataset. CrossLiDAR bridges the gap to modern autonomous-driving simulation by enabling calibration-consistent multi-sensor LiDAR rollouts over long horizons for controllable, scenario-based testing.
Introduction
Autonomous systems have become fundamental in modern robotics and industrial innovation. These systems, ranging from self-driving vehicles [47, 66] to surveying drones [42, 69], often rely heavily on LiDAR to navigate complex 3D environments. By providing high-fidelity geometric data, LiDAR enables precise perception; however, it is constrained by the high cost of the sensors and logistical complexity of real-world data collection. Furthermore, capturing critical “corner cases” [71], such as high-impact collisions or extreme meteorological conditions, remains inherently dangerous and rare in physical datasets [55].
Traditional simulation toolkits [7, 20, 30, 33] require labor-intensive manual scene modeling or existing real-world scans, limiting their scalability. Therefore, research has shifted toward generative models to automate this process, despite the inherent challenges of LiDAR data: high sparsity, non-uniformity, and intrinsic geometric structure. Early generators built on Generative Adversarial Networks (GANs) [3, 12] and Variational Autoencoders (VAEs) [17] often produced “fuzzy” geometries and detail loss.
While score-based models like LiDARGen [74] improved fidelity, they suffered from slow sampling speed and poor long-range performance. More recently, Ultra-LiDAR [61] utilized VQ-VAEs [51] in the Bird’s-Eye-View (BEV) to enhance realism. However, processing sparse BEV representations often results in significant computational waste on empty voxels, leading to inefficient generation. Alternatively, RangeLDM [13] and LiDMs [41] leverage Latent Diffusion Models (LDMs) [43] in the range-view to improve speed. However, these frameworks are primarily restricted to fixed sensor architectures and input dimensions, preventing cross-sensor scalability. Furthermore, they lack fine-grained structural guidance (e.g., ControlNet [70]), rendering them impractical for real-world usage, where side LiDAR proves to be more affordable. More recently, LiDARCrafter [22] extended LiDAR generation to 4D scene modeling by employing a tri-branch diffusion framework that generates temporally ordered 3D LiDAR sequences, with time serving as the fourth dimension. While it achieves impressive temporal consistency, the model remains limited to single-LiDAR sensor configurations. In modern autonomous driving architectures, however, configurations with multiple LiDAR sensors have become the industry standard [49]. This shift is driven by the move toward cost-effective sensor redundancy, where multiple low-profile units augment or replace a single high-performance rooftop scanner to eliminate blind spots. Unlike single-sensor setups, these distributed arrays introduce non-trivial geometric disparities and complex overlapping fields of view. Existing 4D models typically lack the architectural mechanisms to perform cross-sensor fusion.
To address these limitations, we introduce CrossLiDAR, a rectified flow model [26] that departs from conventional U-Net-based architectures [44] in favor of a transformer-based design: the Multi-Modal Diffusion Transformer (MM-DiT) [9], see Figure 1. By leveraging the superior modeling capacity and flexibility of transformers [52], CrossLiDAR captures the intricate geometric dependencies of LiDAR sequences with significantly higher fidelity than previous methods. Central to our framework is a specialized Inter-sensor Module designed to decouple the generation process from specific hardware constraints. This module enables our model to handle multiple LiDAR sensors in parallel, modeling their interactions and ensuring geometric consistency, without the need to design separate models for each sensor configuration. This is particularly effective for the increasingly common integration of low-cost side sensors. To provide more fine-grained geometric consistency, we incorporate a ControlNet-style branch, which allows for precise scene manipulation through various priors, including 3D bounding boxes and BEV maps [40]. Finally, to achieve high-fidelity temporal LiDAR generation, CrossLiDAR introduces a LiDAR Sequence Module. Unlike existing methods that focus solely on single-frame generation, our model performs multi-frame generation (e.g., 5-frame sequence). These short-term sequences can then be iteratively extended through an autoregressive mechanism, enabling the synthesis of long-horizon [34], temporally stable LiDAR sequences. To the best of our knowledge, CrossLiDAR serves as the first truly controllable and multi-sensor pipeline for LiDAR simulation.
Figure 1. CrossLiDAR jointly generates synchronized LiDAR sequences for multi-sensor setups. Top: given multiple sensors mounted around the ego vehicle, CrossLiDAR synthesizes geometrically aligned per-sensor views (front/rear/left/right) that can be fused into a consistent ego-frame point cloud, while supporting structural conditions such as 3D bounding boxes. Bottom: our pipeline operates in a VAE-compressed range-image latent space; a rectified-flow module integrates an Inter-Sensor Module for sensor-aware generation and a ControlNet branch for structural conditioning using BEV maps with projected 3D bounding boxes, yielding controllable multi-sensor LiDAR sequences.
We evaluate CrossLiDAR on two large-scale datasets: Waymo Open Dataset [49], which provides the multi-sensor data from peripheral side LiDARs, and nuScenes [4]. Compared to existing baselines, in the image-based setup, CrossLiDAR achieves an average improvement of 29.56% in JSD and 15.36% in MMD, with particularly superior performance in the multi-sensor configurations.
Our main contributions are as follows:
We introduce CrossLiDAR, a unified diffusion transformer that enables multi-sensor LiDAR generation via an Inter-sensor Module, and supports controllable synthesis from geometric cues through a ControlNet branch.
We present a 4D generation strategy that jointly synthesizes short multi-frame sequences with a LiDAR Sequence Module and extends them autoregressively, reducing temporal drift and improving long-horizon consistency.
We show state-of-the-art (SOTA) performance on Waymo (multi-sensor) and nuScenes (single-sensor) for both single-frame generation and LiDAR sequences.
Related Work and Background
Multisensor LiDAR Generation. Generative models for autonomous driving assume a single LiDAR sensor, whereas real vehicles often deploy a heterogeneous suite (e.g., top-mounted long-range and multiple side-mounted sensors) to capture overlapping perspectives. Multi-sensor data generation has been extensively explored in the camera domain, where recent works target the consistent synthesis of multi-camera surround-view [11, 21, 54, 64]. In contrast, to our knowledge, no prior generative model explicitly leverages multiple LiDAR sensors simultaneously. A key technical challenge is that multi-LiDAR point clouds must be carefully aligned: sensors at different positions yield point clouds in distinct coordinate frames. Unless corrected, naively merging multi-LiDAR data leads to distributional artifacts and geometric inconsistencies [13, 59]. Existing LiDAR generators, including LiDARGen [74], RangeLDM [13], and LiDARCrafter [22], operate on a single aggregated point cloud per scene without explicitly addressing sensor-specific geometries. Recent range- and layout-based methods broaden LiDAR generation beyond unconditional scan synthesis. SPIRAL [73] jointly generates range-view depth, reflectance, and semantic maps, while La La LiDAR [27] introduces scene-graph and layout conditioning for controllable object-level LiDAR scene generation. Veila [28] generates panoramic LiDAR from a monocular RGB image, and DynamicCity [1] models dynamic scenes through 4D occupancy generation. However, these methods do not explicitly model synchronized, calibrated multi-LiDAR sensors. Furthermore, other works using the Waymo Open Dataset [49], one of the few available multi-sensor datasets, often restrict themselves to only the front-facing sensor [8, 60, 63, 72]. As a result, there is a need for dedicated architectures employing inter-sensor modules, which maintain the unique geometric properties of individual sensors while ensuring global spatial consistency across the entire sensor suite.
LiDAR Generation Architectures. Beyond sensor configuration, the architectural foundations of LiDAR generative models are rapidly evolving. Previous latent diffusion models (LDMs), such as RangeLDM [13], LiDM [41], and LiDARCrafter [22], rely on conventional 2D convolutional U-Net backbones. These architectures are often constrained by local receptive fields, which can struggle to capture the global context and long-range dependencies inherent in sparse 3D data. To address these structural bottlenecks, MM-DiT [9] architectures offer a more scalable solution. By leveraging self-attention, transformer-based approaches capture spatial and temporal dependencies more effectively than traditional U-Net-based frameworks.
LiDAR Sequence Generation. Point cloud prediction [31, 56, 57, 58] has emerged as a fundamental self-supervised task in autonomous driving, utilizing historical sequences to forecast future spatial states. Early methods focused on estimating scene flow between consecutive frames to project future point cloud sequences [25, 68]. However, these flow-based approaches incur substantial computational costs, limiting their practical applicability in real-time systems. Consequently, recent studies have shifted toward direct prediction from raw point cloud data [29, 37, 56, 57] to improve efficiency. Further advancements include 4D-Occ [16], which formulates the task as geometric occupancy prediction, and ViDAR [65], which leverages historical images to predict future LiDAR scenes. Despite these developments, existing prediction-based frameworks generally require multiple historical frames as input, still struggle with long-horizon LiDAR forecasting, and provide limited controllability for dynamic scene generation. LiDARCrafter [22] addressed long-horizon generation through an autoregressive diffusion framework, but relies on ego-motion-based extrapolation, a strong prior that restricts the diversity of generated sequences. More recently, U4D [62] introduced uncertainty-aware 4D LiDAR generation by allocating generation capacity to challenging regions and improving spatio-temporal coherence. In contrast, CrossLiDAR focuses on controllable multi-sensor LiDAR generation, combining long-horizon temporal consistency with calibration-aware cross-sensor alignment through a ControlNet-based conditioning branch.
CrossLiDAR
Problem Formulation
We consider LiDAR sensors mounted at distinct viewpoints around the vehicle. At each time step , each sensor generates a point cloud , which is projected into a range image , where and denote the vertical and horizontal resolutions, respectively, and represents the feature channels (e.g., range, intensity, and elongation). To create a unified input for our model, we stack these images along the sensor dimension to form a multi-sensor tensor .
Since our objective is to generate sequences of multi-sensor LiDAR data, we denote the ground-truth and generated spatio-temporal tensors as and , respectively, obtained by stacking (and ) along the temporal dimension, such that the -th slice corresponds to the multi-sensor tensor at time step . To ensure the generated range images correspond to valid 3D structures, a pixel at with range value (depth) is mapped to Cartesian coordinates using the following spherical-to-Cartesian transformation: , , and , where and denote the horizontal and vertical angles, respectively. These angles are derived from the pixel coordinates based on the field-of-view (FOV) and resolution of each sensor.
Our framework consists of two components: a LiDAR compression module and a generative backbone. For compression, we utilize a VAE that operates purely on the spatial dimensions. The encoder processes each sensor frame independently (without temporal or cross-sensor coupling) and maps to a compact latent representation . The decoder reconstructs the corresponding range image from the latent representation. The generation process is modeled using an MM-DiT architecture [9], which models joint spatio-temporal dependencies across sensors and frames. To maintain long-term temporal consistency and mitigate cumulative drift, we employ an autoregressive strategy by adding noise to the first -conditional frame.
The Variational Autoencoder
To mitigate the computational cost of modeling high-dimensional range-image sequences, we employ a VAE that compresses each input range image into a lower-dimensional latent space [43]. In our setting, the input tensor is . Differently from [13, 41], we apply a linear normalization to the raw range values, mapping the input to a symmetric interval . We treat each pair (frame and sensor view ) as an independent sample by reshaping into the batch dimension, i.e., , and encode .

Figure 2. Overview of CrossLiDAR Framework. Our framework utilizes a latent rectified-flow [26] formulation to generate synchronized multi-sensor LiDAR sequences. (Left) A frozen VAE encoder first maps range images into a compact latent space . (Right) These latents are processed by an MM-DiT [9] backbone, which incorporates two different attention mechanisms: an Inter-sensor Module for cross-sensor alignment and a LiDAR Sequence Module for temporal consistency. The generation is guided by a dedicated ControlNet [70] branch for BEV and 3D layout control. Finally, a frozen VAE decoder transforms the denoised latents back into high-fidelity temporal point clouds.
The encoder/decoder ( and ) follow the design of [13]: the encoder maps each input range image to a latent feature map , where the spatial dimensions are downsampled by a factor , i.e., . The decoder reconstructs . We train our VAE by maximizing the Evidence Lower Bound (ELBO) [17]:
where denotes a single range image drawn from , and . The first term evaluates the reconstruction likelihood of the decoder from the latent . In our implementation, this corresponds to an pixel-wise reconstruction loss [13]. However, as a key departure from [13], we omit the adversarial discriminator and perceptual loss.
Rectified Flow for LiDAR Generation
After VAE pretraining, we freeze its encoder/decoder, and model the empirical latent distribution induced by the frozen encoder using a flow-based generative model built on a Transformer backbone, MM-DiT [9]. The Transformer processes spatio-temporal latent inputs via specialized temporal-attention and inter-sensor-attention blocks.
Latent Rectified-Flow Formulation. Given the multi-sensor range image sequence and as the pretrained VAE encoder, for the data we compute the latent , and draw an independent noise latent . For , we form the rectified (linear) bridge . Following [9], the timestep is sampled according to a Logit-Normal distribution to prioritize the mid-range of the transport path. The instantaneous ground-truth velocity is constant along the bridge:
Training. We train a time-dependent vector field (parameterized by the Transformer) to regress this displacement by minimizing the rectified flow loss:
where denotes conditioning information (sensor identifiers, textual prompts, bounding-box embeddings, etc.). Under this objective, the optimal vector field is the conditional expectation:
which provides the probabilistic interpretation of the regression target.
Inference. At inference, we generate latents by integrating the learned ODE:
from to . Because the rectified flow objective encourages near-linear transport paths, accurate samples are typically obtainable with a small number of integration steps using a numerical solver (e.g., Euler or midpoint method). Once the latent is obtained, it is passed through the frozen VAE decoder to reconstruct the multi-sensor range image sequence . While the model operates on range-image latents, the generated range images are back-projected into 3D points using the corresponding LiDAR sensor geometry. The reconstructed points are then transformed into the ego-vehicle coordinate frame using calibrated sensor extrinsics and concatenated to form a unified multi-LiDAR point cloud (Appendix 8.1).
The Architecture Modules
MM-DiT Pre-Attention Stage. The generative backbone is implemented as a stack of MM-DiT blocks operating on VAE-compressed latents within the rectified-flow framework. Each block processes the latent tokens along with auxiliary concatenated modality tokens . Following the canonical MM-DiT design, each block begins with a pre-attention stage utilizing Adaptive Layer Normalization (AdaLN). The tokens are then processed through a spatio-temporal attention, where 3D rotary positional embeddings (3D RoPE) [48] are applied to queries and keys. A dedicated inter-sensor attention is inserted at this stage to facilitate information exchange across multiple sensors at the same timestep.
The LiDAR Sequence Module. After reshaping the latent tokens as and the modality tokens , with , spatiotemporal dependencies are modeled via a multi-head joint self-attention mechanism. Let be the number of heads and the per-head dimension so that . We first compute separate linear projections for and , using two distinct sets of learnable weights:
where and . We then apply 3D RoPE to the query and key of the latent. Next, the tokens are concatenated to form joint representations , , :
To maintain temporal consistency, we enforce causal masking along the temporal axis. For the joint sequence length , we define a mapping that returns the temporal frame for any index such that . We define the causal mask by if , while if . This configuration allows for bidirectional interaction between the LiDAR latent and the modality token at any given time step, while preventing leakage from future frames. Scaled dot-product attention with the causal mask (per head) is performed. The resulting joint output is split back into the latent and modality tensors , where are passed to their respective residual blocks and point-wise feed-forward networks, obtaining and .
The Inter-sensor Module. To resolve geometric disparities between the various LiDAR sensors positioned around the vehicle, the Inter-sensor module treats each sensor as a structured viewpoint and models their spatial interactions through joint attention rather than independent per-sensor generation. For this module, we employ a similar architectural configuration to the LiDAR Sequence Module, which consists of projected bounding box embedders to provide explicit spatial grounding for the different views. To compute the attention between sensors, we reshape the latent output as and the modality tokens as , where . By shifting the sensor count into the sequence dimension, the joint attention mechanism captures dependencies across different viewpoints within the same temporal frame.
The attention process follows the same mathematical formulation as described in the LiDAR Sequence Module, utilizing separate linear projections and . However, the 3D RoPE is applied across the sensor index and spatial coordinates , rather than the temporal axis, while the causal mask is not adopted.
LiDAR ControlNet. To enable precise BEV-based structural control over the synthesis process, we incorporate a ControlNet-style branch [70] conditioned on BEV maps containing projected 3D bounding boxes (Appendix Appendix § 8). Following [70], we create a trainable copy of the MM-DiT backbone blocks while keeping the original generative weights frozen.
The ControlNet branch adopts an internal structure identical to the primary modules, including the factorized attention schemes of the LiDAR Sequence and Inter-sensor modules. For each block in this branch, we first fuse the structural control signals (3D bounding boxes and BEV maps), with the current noisy latent . Specifically, the control signals are processed by a lightweight encoder to produce the embedding , which is summed with to form a unified input for the ControlNet blocks. Therefore, the control signals are directly grounded in the current denoising state, allowing for more accurate structural guidance. The output of the -th ControlNet block, , integrates into the corresponding backbone, , via a zero-initialized residual connection.
Autoregressive Generation
Naïve autoregressive LiDAR generation iteratively conditions on a model’s own predictions, leading to exposure bias and compounding errors over long horizons [6, 15, 53]. Small inaccuracies in early frames are amplified during rollout, causing drift and degraded generation quality. To ameliorate this, we add a small amount of noise to the ground-truth conditioning frame during training. More precisely, rather than using clean conditioning frames during training we fix the noise level of initial frames to , encouraging robustness to exposure bias during autoregression.
Experimental Results
Experimental Settings
Datasets. The nuScenes dataset [4] is designed for multi-modal perception in complex urban environments. We utilize 32-beam LiDAR sweeps. Each LiDAR sweep (sam ple) is initially represented as a range image with a resolution of pixels (totaling 32,768 pixels), a vertical Field of View (FOV) spanning (, ), and a depth range between 1 and 45 meters. The Waymo Open Dataset [49] utilizes a sensor suite consisting of five LiDAR sensors. This configuration includes a primary high-performance mid-range LiDAR (Top) for long-range coverage, supplemented by four short-range side LiDARs (Front, Left, Right, and Rear) positioned around the vehicle’s perimeter. These side sensors are specifically designed to capture the immediate surroundings with high precision, providing a unique perspective that is less susceptible to the top-down occlusions common in roof-only configurations. Further details on both datasets are provided in Appendix § 7.
Baselines. We compare CrossLiDAR against a comprehensive set of baselines categorized by their underlying representation. Range-based methods include LiDARGen [74], R2DM [35], RangeLDM [13], LiDM [41], and LiDARCrafter [22]. Voxel-based methods include the OpenDWM models [36, 46], and we additionally compare against the occupancy-centric UniScene [19]. Details of these methods under our setting are discussed in Appendix § 11.
Evaluation Metrics. Following [22], we evaluate the quality of the generated LiDAR data across three complementary representations: (1) Range-image fidelity using Fréchet Range Distance (FRD) and Maximum Mean Discrepancy (MMD); (2) Point-cloud geometry using Fréchet
Point Distance (FPD) and MMD; and (3) BEV distribution using Jensen–Shannon Divergence (JSD) and MMD. To evaluate temporal dynamics, we follow [22] by measuring frame-to-frame transformation accuracy and sequence smoothness using Temporal Transform Consistency Error (TTCE) and Chamfer Temporal Consistency (CTC). We additionally adapt three metrics from the video synthesis literature to the LiDAR domain to provide a more comprehensive assessment of (4) Temporal coherence: Blind Video Temporal Consistency (BVC) [2], Fréchet Video Distance (FVD) [50], and Fréchet Video Motion Distance (FVMD) [24]. Finally, we evaluate rollout quality by analyzing the accumulated point-cloud sequences. MMD values are reported in and JSD values in . In all tables, the best result is highlighted in bold and the second-best in <u>underline</u>. We evaluate a fixed, calibrated four-LiDAR configuration, using sensor-specific range-image geometry and extrinsic calibration for ego-frame alignment. More details are provided in Appendix § 9.
Image Generation
We evaluate CrossLiDAR on single-frame LiDAR generation across multi- and single-sensor benchmarks. Additional details and ablation studies are in Appendix § 8 and Appendix § 10.
Waymo. Table 1 reports scene-level fidelity on Waymo, our primary benchmark for the multi-sensor design, where CrossLiDAR outperforms all baselines. In the range-image domain, it improves FRD by 93.0% over the second-best method. Point-cloud fidelity follows the same trend, with a 71.7% FPD reduction relative to LiDM. On layout, CrossLiDAR reduces JSD by 25.4% over the next-best baseline. These results indicate better scene-geometry preservation, further supported by our Ablation Study (Appendix § 10.2) showing that the combined LiDAR Sequence and Inter-sensor Modules reduce FRD by 88.8% over naive stacking.
| Method | Venue | Range | Points | BEV | |||
| FRD↓ | MMD↓ | FPD↓ | MMD↓ | JSD↓ | MMD↓ | ||
| Voxel | |||||||
| OpenDWM[36] | CVPR’25 | — | — | 7.55 | 52.23 | 16.25 | 2.29 |
| OpenDWM-DiT[46] | CVPR’25 | — | — | 8.41 | 105.40 | 10.85 | 1.80 |
| Range | |||||||
| LiDM[41] | CVPR’24 | 2.29 | 148.83 | 4.67 | 65.76 | 19.34 | 1.00 |
| RangeLDM[13] | ECCV’24 | 10.97 | 979.33 | 122.38 | 1236.59 | 22.33 | 1.63 |
| R2DM[35] | ICRA’24 | 21.63 | 2283.90 | 40.56 | 228.04 | 27.54 | 1.97 |
| CrossLiDAR | — | 0.16 | 11.89 | 1.32 | 6.90 | 8.09 | <u>1.05</u> |
Table 1. Evaluations of Scene-level Fidelity on Waymo dataset for multi-sensor LiDAR generation.
nuScenes. Table 2 reports results on the single-sensor nuScenes benchmark, where CrossLiDAR achieves SOTA across all three representation domains. The largest improvement is in the point-cloud domain, where our model achieves a 57.1% reduction in FPD compared with OpenDWM-DiT. BEV alignment is also improved, with CrossLiDAR BEV MMD representing a 35.7% reduction over LiDARCrafter. Our method also improves rangeimage fidelity, reducing FRD by 6.6% compared with LiDARCrafter. These results show the proposed framework remains effective even without the Inter-sensor Module.
| Method | Venue | Range | Points | BEV | |||
| FRD↓ | MMD↓ | FPD↓ | MMD↓ | JSD↓ | MMD↓ | ||
| Voxel | |||||||
| UniScene[19] | CVPR’25 | — | — | — | — | 31.55 | 13.61 |
| OpenDWM[36] | CVPR’25 | — | — | 0.95 | 5.43 | 20.17 | 5.61 |
| OpenDWM-DiT[46] | CVPR’25 | — | — | 0.49 | 0.50 | 19.90 | 5.73 |
| Range | |||||||
| LiDM[41] | CVPR’24 | 495.54 | 0.18 | 2.06 | 1.83 | 5.86 | 0.73 |
| RangeLDM[13] | ECCV’24 | — | — | 14.22 | 184.74 | 5.47 | 1.92 |
| R2DM[35] | ICRA’24 | 243.35 | 1.40 | — | — | 3.51 | 0.71 |
| LiDARCrafter[22] | AAAI’26 | 194.30 | <u>0.08</u> | — | — | 3.11 | 0.42 |
| CrossLiDAR | — | <u>181.41</u> | 0.09 | 0.21 | 0.93 | 3.08 | 0.27 |
Table 2. Evaluations of Scene-level Fidelity on nuScenes dataset for LiDAR generation.
4D LiDAR Generation
We evaluate the temporal coherence of CrossLiDAR on multi- and single-sensor LiDAR sequence generation.
Waymo. Table 3 reports 4D generation metrics on Waymo. Since no prior method supports multi-sensor 4D LiDAR generation, we evaluate CrossLiDAR on Waymo’s fixed four-SIDE-LiDAR configuration at a rollout horizon of to assess temporal coherence and cross-sensor consistency. The model remains highly stable across all four CTC steps and achieves the best performance across all rollout-level metrics. Qualitative results in Figure 3 further show that CrossLiDAR better preserves foreground structure, temporal alignment, and cross-sensor geometric consistency than range-based baselines. Additional multi-sensor rollouts, temporal consistency, and controllability examples are provided in Appendix Figures 12 and 13.
| Method | Venue | TTCE↓ | CTC↓ | Rollout↓ | ||||||
| 1 | 2 | 1 | 2 | 3 | 4 | CD | JSD | MMD | ||
| Voxel | ||||||||||
| OpenDWM[36] (=5) | CVPR’25 | 990.17 | 990.74 | 0.37 | 0.52 | 0.65 | 0.68 | 2.81 | 0.53 | 0.57 |
| OpenDWM-DiT[46] (=5) | CVPR’25 | 1025.78 | 1026.43 | 0.39 | 0.59 | 0.76 | 0.81 | 0.78 | 0.13 | 0.34 |
| Range-Temporal | ||||||||||
| LiDM[41] (=3) | CVPR’24 | 1444.74 | 1444.48 | 0.20 | 0.24 | — | — | 0.53 | 0.52 | 1.81 |
| RangeLDM[13] (=3) | ECCV’24 | 1686.14 | 1686.89 | 0.24 | 0.27 | — | — | 0.84 | 0.64 | 8.75 |
| CrossLiDAR (=5) | — | 1016.74 | 1017.38 | 0.06 | 0.10 | 0.12 | 0.14 | 0.07 | 0.10 | 0.03 |
Table 3. 4D LiDAR generation on Waymo using video metrics including: Temporal Consistency and Rollout-level metrics.

Figure 3. Qualitative Results. The top row illustrates temporal sequence generation on nuScenes, where CrossLiDAR maintains strict temporal consistency and 3D bounding box alignment. The bottom row demonstrates multi-sensor Waymo generation, where a single unified scene is reconstructed by synchronizing outputs from four distinct generated sensors. Compared to baselines, CrossLiDAR produces scan patterns with improved scene-level geometric fidelity, effectively handling the complexities of multi-sensor generation.
nuScenes. Table 4 reports temporal generation results on the single-sensor nuScenes benchmark. CrossLiDAR achieves the best TTCE scores and outperforms prior methods on longer-horizon CTC metrics, with the performance gap increasing as sequence length increases. CrossLiDAR achieves a 15.4% reduction in TTCE at step 4 compared with LiDARCrafter, indicating better adherence to physically plausible motion trajectories. Although OpenDWM-DiT achieves a lower CTC-1 error, its error accumulates rapidly over longer horizons. In contrast, CrossLiDAR reduces CTC-4 by 36.6% relative to OpenDWM-DiT. Rollout quality also improves, with a 39.7% and 65.8% reduction in CD compared with OpenDWM-DiT and OpenDWM, respectively. These results demonstrate our proposed framework effectively mitigates the geometric drift commonly associated with long-horizon 4D LiDAR generation.
| Method | Venue | TTCE ↓ | CTC ↓ | Rollout ↓ | ||||||
| 3 | 4 | 1 | 2 | 3 | 4 | CD | JSD | MMD | ||
| Voxel | ||||||||||
| UniScene[19] | CVPR’25 | 2.74 | 3.69 | 0.90 | 1.84 | 3.64 | 3.90 | — | — | — |
| OpenDWM[36] | CVPR’25 | 2.68 | 3.65 | 1.02 | 2.02 | 3.37 | 5.05 | 1.20 | 0.17 | 9.99 |
| OpenDWM-DiT[46] | CVPR’25 | 2.71 | 3.66 | 0.89 | 1.79 | 3.06 | 4.64 | 0.68 | 0.17 | 8.80 |
| Range-Temporal | ||||||||||
| LiDARCrafter[22] | AAAI’26 | 2.65 | 3.56 | 1.12 | 2.38 | 3.02 | 4.81 | — | — | — |
| CrossLiDAR | — | 2.40 | 3.01 | 1.42 | 1.71 | 2.10 | 2.94 | 0.41 | 0.14 | 0.03 |
Table 4. 4D LiDAR generation on nuScenes using video metrics including: Temporal Consistency and Rollout-level metrics. The temporal horizon is fixed to for all models.
Autoregressive 4D LiDAR Generation
We assess long-horizon stability by extending CrossLiDAR autoregressively beyond its training window, in both multi- and single-sensor settings.
Waymo. Table 5 reports autoregressive generation on Waymo, where four side LiDARs make cross-sensor consistency especially hard to maintain as the rollout horizon grows. Extending the generated window to , CrossLiDAR achieves the best temporal consistency, reducing TTCE by 13.0% over the strongest baseline and reducing CTC by an average of 87.8% across all four evaluation steps. At the rollout level, CrossLiDAR further reduces JSD by 67.6% and MMD by 85.7% compared with the strongest baseline. These results show the proposed framework effectively preserves cross-sensor geometric consistency and temporal coherence during long autoregressive rollouts.
| Method | Venue | TTCE ↓ 3 | TTCE ↓ 4 | CTC ↓ 1 | CTC ↓ 2 | CTC ↓ 3 | CTC ↓ 4 | Rollout ↓ CD | Rollout ↓ JSD | Rollout ↓ MMD | Motion BVC↑ | Motion FVD↓ |
| OpenDWM [36] () | CVPR’25 | 471.34 | 471.40 | 0.42 | 0.55 | 0.66 | 0.73 | 0.11 | 0.38 | 0.17 | — | — |
| OpenDWM-DiT [46] () | CVPR’25 | 424.65 | 424.86 | 2.07 | 2.63 | 2.81 | 3.17 | 0.07 | 0.37 | 0.14 | — | — |
| CrossLiDAR | — | 369.63 | 369.74 | 0.05 | 0.07 | 0.08 | 0.09 | 0.07 | 0.12 | 0.02 | 0.79 | 57.42 |
Table 5. Auto-regressive generation on Waymo, including Temporal Consistency, Rollout-level and Motion-level metrics.
nuScenes. Table 6 evaluates 16-frame rollouts on nuScenes, where CrossLiDAR shows substantially improved long-horizon stability over previous methods. While LiDARCrafter’s CTC increases by 329.5% across the evaluated steps, CrossLiDAR grows by only 34.9%, consistent with our noise-augmented training strategy for imperfect autoregressive conditioning. CrossLiDAR further reduces TTCE by 66.7% and 64.0% at steps 3 and 4, respectively, over the second-best method, suggesting the proposed flow-matching transformer better preserves physically plausible ego-motion. The gap between TTCE and CTC suggests dynamic objects remain harder to model than ego-motion. At the distributional level, CrossLiDAR reduces FVD by 89.2% over OpenDWM-DiT, bringing the generated sequences substantially closer to the real LiDAR distribution in appearance and dynamics. CrossLiDAR reduces FVMD by 26.0% while improving BVC by 104.7%, demonstrating more realistic motion and stronger temporal stability. These results show CrossLiDAR maintains high quality over long horizons, a key capability for scalable autonomy simulation.
| Method | Venue | TTCE ↓ 3 | TTCE ↓ 4 | CTC ↓ 1 | CTC ↓ 2 | CTC ↓ 3 | CTC ↓ 4 | Rollout ↓ CD | Rollout ↓ JSD | Rollout ↓ MMD | Motion BVC↑ | Motion FVD↓ | Motion FVMD↓ |
| UniScene [19] () | CVPR’25 | 2.74 | 3.69 | 0.90 | 1.84 | 3.64 | 3.90 | — | — | — | — | — | — |
| OpenDWM [36] () | CVPR’25 | 2.64 | 3.16 | 0.82 | 1.62 | 2.83 | 4.67 | 0.97 | 0.46 | 0.27 | 0.22 | 3923.11 | 4147.61 |
| OpenDWM-DiT [46] () | CVPR’25 | 1.23 | 1.39 | 0.68 | 1.42 | 2.42 | 3.83 | 0.83 | 0.24 | 0.13 | 0.43 | 2835.68 | 2261.98 |
| LiDARCrafter [22] () | AAAI’26 | 2.65 | 3.56 | 1.12 | 2.38 | 3.02 | 4.81 | — | — | — | — | — | — |
| CrossLiDAR () | — | 0.41 | 0.50 | 1.09 | 1.22 | 1.36 | 1.47 | 0.47 | 0.22 | 0.26 | 0.88 | 307.07 | 1672.90 |
Table 6. Autoregressive generation on nuScenes, including Temporal Consistency, Rollout-level and Motion-level metrics.
Conclusions and Limitations
We present CrossLiDAR, a unified rectified-flow transformer for controllable multi-sensor LiDAR generation in a compact range-image latent space. It comprises three key components: an Inter-sensor Module that models geometric dependencies across heterogeneous LiDAR suites, a ControlNet-style conditioning branch for fine-grained structural control via BEV priors and 3D bounding boxes, and a noise-augmented autoregressive strategy that mitigates cumulative drift over long horizons. On Waymo and nuScenes, CrossLiDAR achieves strong performance in single-frame fidelity, temporal consistency, and long-horizon rollout quality, with particularly strong gains in the multi-sensor setting unsupported by previous methods.
Current limitations include the lack of quantitative evaluation of foreground fidelity and bounding-box adherence. Our evaluation focuses on Waymo’s fixed, calibrated four-LiDAR configuration, motivating further study of sensor conditioning, cross-sensor overlap, and adaptability to different sensor counts, types, layouts, and calibrations. The LiDAR Sequence and Inter-sensor Modules are evaluated jointly, leaving their individual contributions unquantified. Evaluation against additional recent 4D LiDAR generation methods remains limited. Downstream benefits for 3D detection, semantic segmentation, occupancy prediction, and planning remain unevaluated. Future work will address richer dynamic object modeling and integration with downstream planning and validation pipelines, while further reducing the sim-to-real gap.
Appendix
Datasets
We evaluate our approach using two large-scale autonomous driving datasets: nuScenes [4] and the Waymo Open Dataset [49]. Each provides multi-modal sensor data together with annotations. In this section, we summarize their overall statistics, sensor configurations, image resolutions, and available labels.
nuScenes
The nuScenes dataset [4] provides diverse multimodal sensor data collected using two identically instrumented vehicles for autonomous driving research. The data were recorded with a surround-view sensor suite comprising one spinning 32-beam LiDAR sensor, six RGB cameras, and five automotive radar sensors. Together, these sensors provide complementary observations with full 360° coverage around the ego vehicle (specifications in Table 7). The LiDAR sensor captures point clouds at 20 Hz, with a 360° horizontal field of view and a vertical field of view from −30° to 10°, producing up to 1.4 million points per second with an effective range of up to 70 m. In addition, the cameras operate at 12 Hz and the radar sensors at 13 Hz.
| Sensor | VFOV | HFOV | Resolution | Range (m) | Intensity |
| Velodyne HDL-32E | [+10°; −30°] | 360° | (32, 1080) | [2.0; 100.0] | [0; 225] |
Table 7. LiDAR Sensor Specifications for the nuScenes Dataset. We report the vertical field of view (VFOV), horizontal coverage (HFOV), range-image resolution (rows×columns), valid range (m), and intensity range.
nuScenes provides a large corpus of annotated 3D bounding boxes for urban driving scenes captured under diverse traffic, lighting, and weather conditions. The full dataset contains 1,000 scenes, each 20s long, selected from 84 driving logs collected in Boston and Singapore. Although the sensor streams are temporally dense, 3D box annotations are provided only for keyframes sampled at 2 Hz. In total, the dataset includes approximately 40,000 annotated keyframes, about 400,000 LiDAR sweeps, 1.4M camera images, 1.4M radar sweeps, and 1.4M annotated 3D bounding boxes across 23 classes. Annotations are defined in the global coordinate frame and are distributed with calibrated sensor poses, enabling projection into the ego or sensor coordinate frames when needed for model training and evaluation. Beyond object boxes, nuScenes also provides rich scene-level metadata, including location, weather, time of day, and object attributes.
Waymo
The Waymo Open Dataset [49] provides diverse multi-modal sensor data collected from a fleet of self-driving vehicles. Each vehicle is equipped with five high-resolution LiDAR sensors: one mounted on the roof (TOP) and four located on the SIDEs: front (F), rear (R), side-left (SL), and side-right (SR). All LiDAR data is provided as range images; detailed specifications3 are listed in Table 8.
| Sensor | VFOV | HFOV | Resolution | Range (m) | Intensity |
| TOP | [−17.6°, +2.4°] | 360° | 64 × 2650 | [1.2, 75.0] | [7 × 10^{-6}, 4 × 10^{6}] |
| F/SL/SR/R | [−90°, +30°] | — | 200 × 600 | [5 × 10^{-3}, 20.0] | [5 × 10^{-6}, 40.5] |
Table 8. LiDAR Sensor Specifications on Waymo Open Dataset. We report vertical field of view (VFOV), horizontal coverage (HFOV), range-image resolution, valid range (m), and intensity range for TOP and SIDE sensors.
Figures 4 and 5 show typical range images and corresponding point clouds from the TOP sensor, covering 360° horizontally, up to 75 m range, and 64 vertical beams; and from the four SIDE sensors, each with a larger vertical FOV, 200 beams, and a maximum range of 20m. The precise horizontal field of view (FOV) for each SIDE sensor is not specified in the official documentation. We therefore estimate each sensor’s effective FOV as the proportion of valid columns (i.e., columns with at least one valid measurement) in the range image, out of 600 columns. Assuming a total sensor FOV of , the effective FOV is computed as .
In all experiments, we use only the four side LiDAR sensors to emulate a multi-sensor input scenario.
Waymo provides a large corpus of high-quality annotated 3D LiDAR bounding boxes. We use annotations for three semantic classes: VEHICLE, PEDESTRIAN, and CYCLIST. Since boxes are labeled in the vehicle (ego) frame, we project them into the LiDAR frame by inverting the extrinsic calibration matrix. As SIDE LiDAR sensors capture only short-range points, we retain boxes fully contained within each sensor’s valid range.
The resulting dataset contains approximately 1.45M LiDAR box annotations across over 1000 scenes from geographically distinct regions. Each scene is 20 seconds at 10Hz, totaling 200 frames per scene. For input construction, each scene is divided into overlapping clips of 17 frames with a stride of 5 frames, so consecutive clips share up to 12 frames.
Statistics for our training set using only SIDE LiDAR sensors are provided in Table 9, 10 and Figure 6.
| TrainVal | Test | |
| Vehicle | 941.6K | 227K |
| Pedestrian | 499.1K | 121.4K |
| Cyclist | 11.8K | 3.4K |
| Total | 1451.6K | 352K |
Table 9. Statistics on annotations.

Figure 6. Per-frame annotations.
Point Cloud Densification. The SIDE LiDAR range images in Waymo are sparse and contain many invalid pixels. Following [18], we apply a fast CPU-based depthcompletion algorithm to fill invalid pixels with estimated depths. The method first performs nearest-neighbor dilation using a diamond-shaped kernel, and then refines the result via smoothing steps that encourage locally planar surfaces and suppress outliers with bilateral filtering. As
Figure 4. Waymo range-image examples. Top: TOP LiDAR range image (), with horizontal coverage. Bottom: four SIDE LiDAR range images ( each), with higher vertical resolution and partial-view coverage.

Figure 5. Examples of Waymo point-cloud coverage. Left: Point cloud from each SIDE LiDAR sensor, illustrating the partial horizontal coverage provided by each individual sensor. Right: Point cloud obtained by fusing the four SIDE LiDAR sensors, shown in comparison with the TOP LiDAR. The SIDE sensors are color-coded by position: front (yellow), left (pink), rear (blue), and right (green).
| Train | Val | Test | |
| Scenes | 798 | 202 | 150 |
| Frames | 158K | 40K | 30K |
| Clips | 21K | 5K | 5K |
Table 10. Size of the data splits.
shown in Figure 7, densification substantially improves the visual quality of the resulting depth maps and point clouds.

Figure 7. Densification on the Waymo SIDE LiDAR range images. The top row shows the original sparse range images from Waymo, while the bottom row shows the densified results. The densification algorithm [18] fills in missing pixels and significantly enhances the visual quality and completeness of the range images.
Implementation Details
Data Pre-processing & Post-processing. For point cloud generation, each LiDAR scan is optionally projected to a dense 2D range image using spherical projection: Given a 3D point , the corresponding spherical coordinates are computed as:
where is the measured range, is the elevation angle, and is the azimuth angle. For the Waymo dataset, range images are provided directly, and this step is skipped. To address sparsity in the raw range images, we use the ip_basic implementation4 for efficient depth completion.
Each valid range measurement is normalized to via linear scaling:
where is the maximum valid range for the sensor. For generation, we apply the inverse mapping.
Predicted range images are projected back to 3D space using the inverse of the spherical mapping:
where is the depth value at the pixel, and , are the vertical and horizontal angles for that pixel. Points with range outside the post-processing interval are filtered out: for Waymo SIDE LiDAR, and for nuScenes.
For BEV controllable generation, we project 3D bounding boxes and other semantic elements from the ego-vehicle frame onto the BEV plane. Given a bounding box with corners in the ego frame, we apply a fixed ego-to-BEV transformation. Boxes and other features are then rasterized onto a discrete 2D grid, where the grid bounds and cell size are chosen according to the dataset. We also incorporate BEV roadmap layers (e.g., road, lanes), which are available from both Waymo and nuScenes.
Training Procedures. CrossLiDAR follows a three-stage latent generative training protocol.
VAE Pretraining. We first pre-train the VAE using a reconstruction objective on range images, encoding each LiDAR sensor independently into a compact latent representation. After convergence, the VAE weights are frozen.
Latent Rectified Flow Training. We then train a Rectified Flow model to learn the dynamics in the latent space. For sequence modeling, latent representations from all time steps () and sensors () are stacked along the batch dimension (), enabling joint spatio-temporal learning directly in latent space.
ControlNet Training. Finally, we freeze the base Rectified Flow model and train a ControlNet branch [70] for conditional generation. Control blocks are inserted into the backbone at a fixed interval (one every three backbone blocks). The model is conditioned on structural inputs, including BEV maps with projected 3D bounding boxes, enabling controllable scene generation while preserving the pretrained generative prior.
Training Hyper-parameters. The VAE is trained for 50 epochs using the AdamW optimizer with a base learning rate of and a batch size of 16. We employ an reconstruction objective with a KL regularization weight of . The encoder—decoder architecture uses a latent dimensionality of 4 channels, a base channel width of 64 with multipliers , and two residual blocks per resolution level with SiLU activations. Circular padding is adopted to respect the periodic structure of range images, and no attention layers are used in the VAE. Mixed-precision (FP16) training is used throughout.
The rectified flow model is trained with an loss and a log-normal noise schedule for 500000 iterations using a batch size of 16. We use AdamW optimizer with learning rate for nuScenes, for Waymo, , , . The learning rate is linearly warmed up over the first 0.01% of steps and then held constant. Gradients are clipped to a maximum norm of 0.1. Mixed-precision (FP16) training is used throughout, with FSDP and zero3 sharding for memory efficiency.
During diffusion and ControlNet training, we set the MM-DiT patch size to 1. Multiple conditioning modalities are used, including text, images, frame pose, and BEV projections of 3D bounding boxes for explicit scene-level guidance. To enhance robustness and generalization, condition dropout is applied independently to each condition with probability 0.15; for short clips (fewer than 5 frames), all conditions are jointly dropped with probability 0.15. BEV maps are embedded into the VAE latent space using a spatial encoder consisting of 5 layers of 2D convolutional blocks with residual connections.
For ControlNet, 30% of the dataset is set aside for training, as reusing the diffusion model data for this stage leads to minimal improvements.
Sampling and Inference. For generation, we employ rectified flow with a linear transport map for the diffusion process and use the Euler–Maruyama solver for sampling. We run 30 sampling steps with a classifier-free guidance scale of 3.0. Additional sampling hyperparameters are set as needed.
Evaluation
We evaluate LiDAR sequence generation using complementary metrics that capture: (1) Per-scene fidelity, (2) Temporal rollout realism, (3) Temporal consistency across frames, and (4) Video-level quality aggregated over sequences. Unless stated otherwise, all metrics are computed on Waymo using the four SIDE LiDAR sensors.
Scene-level Metrics
Scene Definition. We define a scene at time as the fused multi-sensor point cloud obtained by combining the LiDAR point clouds. Specifically, for each sensor (with in Waymo and with in nuScenes), we project points from the sensor frame into the ego-vehicle frame using the provided extrinsics:
Furthermore, we concatenate the transformed points to form the fused scene point cloud .
Sampling Protocol. For statistical reliability, we evaluate a total of 10000 sensor frames (i.e., per-sensor range images) in all scene-level metrics. Concretely, for models that generate single frames, we generate 10000 scenes; each scene comprises independently generated range images (one per sensor) that are re-projected and fused in the ego frame. For models that generate sequences, we generate 2000 sequences of length (10000 scenes total), and evaluate each time step as an individual scene.
Distributional Fidelity Metrics. To quantify scene-level fidelity, we adopt standard distributional similarity metrics widely used in LiDAR generative modeling [74]. For each metric, we compute features for real samples and generated samples, fit Gaussians to their distributions, and measure the divergence between the two distributions.
Fréchet Range Distance (FRD). FRD [74] measures the discrepancy between feature distributions of real and generated range images for each sensor. Following prior work, we extract features using a pretrained RangeNet-53 [32] segmentation network.5 Let and denote features from real and generated samples, with empirical means and covariances . FRD is computed as the Fréchet distance between two Gaussians:
Fréchet Point Volume Distance (FPD). To assess realism in 3D, we re-project generated range images into point clouds, and extract their features using a pretrained Sparse Point-Voxel Convolution Network [39].6 We then compute the Fréchet distance in the same form as above, using point-cloud feature statistics.
Jensen-Shannon Divergence (JSD). To compare spatial occupancy patterns, we voxelize each fused ego-frame scene into a BEV occupancy histogram over an grid. Let and denote the marginal BEV occupancy distributions obtained by averaging normalized histograms over generated and real scenes. We compute:
where .
Maximum Mean Discrepancy (MMD). We additionally compute the maximum mean discrepancy between the BEV occupancy distributions. Given samples (real) and (generated), MMD with kernel is:
Unless noted otherwise, we use an RBF kernel on the flattened BEV histograms.
For each fused scene , we compute a BEV occupancy grid by discretizing the plane into uniform cells and marking a cell occupied if it contains at least one point. The resulting binary occupancy map is normalized into a histogram and used to form the marginal distributions for JSD and MMD.
Temporal Consistency Metrics
Temporal Transformation Consistency Error (TTCE)
To measure video generation performance, we follow [22] and inherit the temporal consistency metrics TTCE and CTC, which quantify how consistently generated sequences preserve temporal motion by comparing frame-to-frame rigid transformations with ground truth. Using Iterative Closest Point (ICP), we compute the estimated transformations between consecutive frames and compare them to the true transformations :
Smaller TTCE values indicate that the predicted motion closely matches the ground truth, reflecting smoother and more realistic temporal dynamics.
Chamfer Temporal Consistency (CTC) measures the frame-to-frame geometric smoothness of generated point clouds. Consecutive frames are aligned using the ground-truth transformations:
The Chamfer Distance between and the aligned frame is:
The overall CTC score is the average Chamfer Distance over all consecutive frame pairs:
Lower CTC values correspond to smoother temporal transitions and higher frame-to-frame coherence in the generated sequence.
Rollout-level Metrics
To assess temporal consistency and scene fidelity across a sequence altogether, we evaluate the quality of LiDAR rollouts, i.e. the accumulation of generated LiDAR frames over time.
Global Alignment and Accumulation. Following the KITTI-360 protocol [23], at each time step we transform each frame’s generated point cloud to the global coordinate frame using the ground-truth ego-pose :
A rollout for a sequence of length is then defined as the set union of the aligned point clouds:
We compare rollout-level statistics using both generated and ground-truth sequences and . This procedure captures static scene structure (e.g., buildings, road geometry) as well as the spatiotemporal evolution of dynamic objects (e.g., vehicles, pedestrians) (Figure 8).
Figure 8. Rollout examples of temporal consistency on nuScenes and Waymo. For each dataset, we visualize a scene composed of multiple point clouds acquired at different timestamps. The consistent alignment of the colored regions across time indicates that the reconstruction preserves temporal consistency. The highlighted objects include a vehicle (orange), a pedestrian (blue), vegetation (green), and background structures (purple).
We compute the Jensen-Shannon divergence (JSD) and maximum mean discrepancy (MMD) as described in § 9.1, but using BEV histograms aggregated over the combined rollout point cloud.
We further quantify geometric fidelity between the ground-truth and generated rollout point clouds using the symmetric Chamfer Distance (CD):
where and denote the accumulated ground-truth and model-generated rollout point clouds, respectively.
These rollout-level metrics provide a holistic measure of both structural realism and temporal coherence in generative LiDAR modeling.
Video Evaluation Metrics
The temporal consistency metrics (TTCE and CTC) discussed above primarily assess rigid geometric alignment between consecutive frames. However, these metrics do not adequately capture the quality of dynamic scenes in which objects undergo non-rigid motion or deformation. To address this limitation, we adapt established video quality metrics from the computer vision literature to the LiDAR domain.
We evaluate video generation quality through three metrics organized across two complementary dimensions:
Temporal Consistency. To quantify temporal stability without relying on rigid transformations, we employ an optical-flow warping error and a relative Blind Video temporal Consistency index (BVC) [67]. We estimate dense optical flow [10] between consecutive frames, warp the previous frame toward the current frame using the estimated flow field, and compute a masked photometric discrepancy on non-occluded pixels. We aggregate both short-range errors (between consecutive frames) and long-range errors (relative to the first frame) to obtain . The BVC index is defined as , providing a normalized measure where indicates superior temporal smoothness compared to ground truth, indicates matching stability, and indicates increased flicker or temporal instability.
Video-level Realism and Motion. At the distributional level, Fréchet Video Distance (FVD) [50] compares real and generated videos in the feature space of an I3D network [5] using 16-frame clips at resolution. Complementing FVD’s appearance-focused evaluation, Fréchet Video Motion Distance (FVMD) [24] compares distributions of tracked keypoint trajectories, providing a motion-centric assessment of temporal dynamics.
Ablation Studies
We conduct ablations on the VAE design choice, the diffusion backbone, and other key designs for their validation. Unless specified, all ablation experiments are conducted on the Waymo Open Dataset [49], for multi-sensor LiDAR scene generation () setting.
VAE
Our framework adopts the Latent Diffusion [43] paradigm, employing a VAE for compressing high-dimensional range images into a compact latent space. We base our VAE design on recent state-of-the-art architectures, LiDM [41] and RangeLDM [13], and systematically study the impact of key modeling choices.
All ablations are performed with a fixed compression ratio , a batch size of 16, and 50 training epochs, following the default settings for each method’s official implementation. Following [19, 36], we adopt an image-based VAE (2D) instead of a temporal 3D architecture, as the goal here is efficient per-frame compression of range images rather than joint spatiotemporal modeling. This architecture also enables more efficient training and inference, similar to designs in [19, 36].
Table 11 presents quantitative VAE reconstruction results for different configurations on Waymo SIDE LiDAR. We observe the following: (1) The use of Circular Convolution [45] in RangeLDM leads to improved latent representations relative to LiDM across all metrics (cf. rows 3–6 vs 1–2), confirming the benefit of circularity for sensors with coverage. (2) Incorporating adversarial loss into VAE training is challenging, with performance highly sensitive to the activation schedule; improper tuning can significantly degrade results (cf. rows 1–2 and 3–4). (3) VAE performance deteriorates with aggressive input resizing, likely due to increased information loss (cf. rows 3–4 vs rows 5–6). Nevertheless, this resizing may be necessary for compatibility with U-Net-based diffusion backbones, which constrain input resolution and highlight a limitation of the U-Net design. (4) For SIDE LiDAR on Waymo (range 0–20 m), log compression (Equation (17)) provides little benefit, since the dynamic range is limited and linear normalization (Equation (3)) already preserves the range scale adequately without introducing an additional nonlinear transformation (cf. rows 6–7 and Figure 9).

Figure 9. Impact of Logarithmic Transformation on LiDAR range distributions. Left: For TOP LiDAR, the log transform significantly reduces the skewness of high dynamic range values. Right: For SIDE LiDAR on Waymo, where the range values are less skewed and cover a shorter distance, the effect of the log transform is minimal.
Diffusion Backbone: DiT vs U-Net
For LiDAR generation in the Latent Diffusion framework [43], the choice of diffusion backbone is critical. Range images, representing LiDAR sweeps as anisotropic 2D panoramas, exhibit a pronounced width–height asymmetry: the width spans the full angular resolution, whereas the limited number of sensor beams constrains the height. This results in unique long-range horizontal dependencies that differ fundamentally from those in natural images. In this section, we ablate U-Net and Diffusion Transformer (DiT) [38] backbones, comparing their effectiveness in modeling these dependencies and their parameter efficiency.
While U-Net architectures remain a popular backbone that preserves local structures, they are inherently limited in modeling the extreme aspect ratios and horizontal context of range images. In contrast, DiT leverages global self-attention to capture long-range dependencies directly and offers flexible input resolution via dynamic padding, therefore avoiding the destructive resizing often required by the U-Net. As observed in § 10.1 and Figure 10, maintaining native resolution is crucial, as resizing artifacts can significantly degrade downstream point cloud reconstruction.

Figure 10. Effects of range image resizing on generated point clouds. Resizing the range image from (200, 600) to (224, 608) introduces noise and geometric distortions in the resulting point cloud, noticeably affecting object shapes (e.g., trees, cars). Best viewed when zoomed in.
Table 12 reports single-frame LiDAR generation results for state-of-the-art methods LiDM [41] and RangeLDM [13] across multiple backbones. LiDM demonstrates strong performance on Waymo SIDE LiDAR generation, notably outperforming RangeLDM (cf. rows 1 vs. 3), with further gains when employing DiT. We find that the default hyperparameters of RangeLDM require more tuning (e.g., learning rates), particularly for side LiDARs with smaller dynamic ranges and higher cross-sensor correlation demands. DiT-based models outperform U-Nets across most settings (cf. rows 1 vs. 2 and 3 vs. 4), while using approximately 50% fewer parameters and reducing GPU memory consumption to about two-thirds of that required by U-Net. These results highlight the parameter-efficiency advantages of DiT over U-Nets, validating the benefits of transformer-based backbones for LiDAR range image generation.
| Method | Input Size | Backbone | Range FRD ↓ | Range MMD ↓ | Points FPVD ↓ | Points MMD ↓ | BEV JSD ↓ | BEV MMD ↓ |
| LiDM[41] | U-Net | 2.29 | 148.83 | 4.67 | 65.76 | 0.19 | 1.00 | |
| DiT | 0.45 | 77.91 | 2.52 | 113.33 | 0.15 | 1.00 | ||
| RangeLDM[13] | U-Net | 10.97 | 979.33 | 122.38 | 1236.59 | 0.22 | 1.63 | |
| DiT | 9.07 | 938.83 | 61.47 | 374.81 | 0.19 | 1.16 | ||
| CrossLiDAR () | DiT + Naive-Stacking | 1.66 | 55.67 | 14.82 | 156.55 | 0.20 | 1.62 | |
| MM-DiT + Naive-Stacking | 1.43 | 51.94 | 14.48 | 139.31 | 0.16 | 1.51 | ||
| MM-DiT + LiDAR-Sequence + Inter-Sensor | 0.16 | 11.89 | 1.32 | 6.90 | 0.08 | 1.05 |
Table 12. Ablations on Diffusion Backbone on Waymo SIDE LiDAR. Configuration : compression rate , range densification, filling invalid points and without discriminator. The best result is highlighted in bold and the second-best in _underline_.
Finally, we perform two ablation studies on the proposed CrossLiDAR architecture. First, we compare DiT and MM-DiT under the same Naive Stacking strategy, where temporal and multi-sensor latent representations are simply concatenated before being processed by the backbone. This isolates the contribution of the backbone architecture itself. Second, we compare this Naive Stacking baseline against the proposed LiDAR Sequence and Inter-sensor modules while keeping the MM-DiT backbone fixed, therefore isolating the contribution of explicit temporal and cross-sensor modeling. Table 12 shows that replacing DiT with MM-DiT under the same Naive Stacking setting improves the results by an average of 10.1% across all evaluation metrics. In contrast, replacing Naive Stacking with the proposed LiDAR Sequence and Inter-sensor modules yields substantially larger gains, reducing FRD by 88.8%, point-cloud FPD by 90.9%, and BEV JSD by 50.0% relative to the MM-DiT baseline. These results demonstrate that the improvements reported in the main paper are largely driven by the combined proposed temporal and cross-sensor modules rather than by the MM-DiT backbone alone.
Baselines
We compare CrossLiDAR with a broad suite of recent methods for LiDAR generation. Table 13 summarizes the key characteristics of each baseline, including problem setting, network architecture, generative paradigm, datasets used for training, and the availability of code and pretrained checkpoints at the time of submission. In contrast to prior work, CrossLiDAR is specifically designed for the challenging multi-sensor LiDAR sequence generation setting and supports a wide range of input conditions.
| Venue | Modality | LiDAR Sensors | LiDAR Frames | Supported Conditions | Diffusion Model | Dataset | Code | Checkpoint | |
| LiDAR Diffusion[41] | CVPR’24 | LiDAR | 1 | 1 | image,semantic map | U-Net | K | Yes | K |
| RangeLDM[13] | ECCV’24 | LiDAR | 1 | 1 | inpaint,downsample | U-Net | K,N | Yes | K,N |
| R2DM[35] | ICRA’24 | LiDAR | 1 | 1 | — | U-Net | K | Yes | K |
| LiDARDM[75] | ICRA’25 | LiDAR | 1 | multi | map | U-Net | K,W | Yes | K,W |
| UniScene[19] | CVPR’25 | LiDAR &Camera | 1 | multi | BEV→semantic occupancy | Sparse U-Net | N | Yes | N |
| OpenDWM[36] | CVPR’25 | LiDAR | 1 | multi | BEV,map,text | U-Net/DiT | N | Yes | N |
| LiDARCrafter[22] | AAAI’26 | LiDAR | 1 | multi | text→layout→object→trajectory | U-Net | N | Yes | — |
| CrossLiDAR | — | LiDAR | multi | multi | BEV,object,image,text | MM-DiT | N,W | — | — |
Table 13. Summary of existing methods for LiDAR generation. Overview of compared methods, indicating sensor setup, generative model, available modalities, supported input conditions, training datasets (K: KITTI-360 [23], N: nuScenes [4], W: Waymo [49]), and code/checkpoint availability.
Many existing methods [13, 35, 41, 74] are designed for single-sensor LiDAR generation. To ensure fair comparison, we adapt these methods for multi-sensor LiDAR generation, as detailed in the following sections, which also specify the training hyperparameters we used on Waymo. Certain methods could not be included in our Waymo experiments due to design complexity or prohibitive data requirements. For example, UniScene [19] requires semantic occupancy annotations, which are not readily available at scale for Waymo. Similarly, LiDARDM [75], a rendering-based technique that relies on Neural Kernel Surface Reconstruction [14] and ray-casting, is not directly comparable to range-image or voxel-based LiDAR generation.
R2DM
Adaptation for Waymo. We adapt R2DM [35] for the Waymo Dataset with four SIDE LiDAR sensors: front, left, rear, and right. The range image resolution is set to with a single depth channel, and depth values span between meters. Ray angles are initialized for a vertical FOV from (top) to (bottom). For the SIDE LiDAR sensors, we use the effective horizontal FOV estimated from valid range-image columns, following § 7.2. To handle multiple sensors, we stack their views into a tensor ( sensors) and reshape to for efficient batch processing as in Figure 11.

Figure 11. Naive sensor stacking for multi-sensor LiDAR generation. Each sensor range image at time step is independently encoded by the VAE. The diffusion model operates on the stacked multi-sensor latents, enabling generation (or prediction) of joint sensor, and can be extended to sequences by stacking along the temporal dimension.
Training Configuration. Following the upstream implementation7, we train with AdamW (betas 0.9/0.99, weight decay 0.0, ), a base learning rate of scaled to . The learning rate follows a cosine annealing schedule with a 10k-step warmup, and training proceeds for 300k optimization steps. Training uses gradient clipping with a maximum norm of 1.0 and an exponential moving average (EMA, ) with updates every 10 steps. The code is run with Python 3.12 and PyTorch version 2.9.0.
LiDM
Adaptation for Waymo. We adapt LiDM [41] to Waymo SIDE LiDAR range images by using depth-only inputs. The range images are resized to to match the down-sampling constraints of its U-Net backbone. Depth values are clipped to for compatibility with the LiDM baseline and normalized using the logarithmic mapping from the original work:
To accommodate multiple sensors, we train a single 2D VAE shared across sensors while encoding each sensor independently (i.e., without cross-sensor interaction). We use a compression setting () with curve-wise factor and patch-wise factor , producing latents of size per sensor. For diffusion training, we stack latents across sensors and time: per-sensor latents are arranged as and reshaped to ; for sequences, we additionally fold the temporal dimension into the channel axis, yielding , as shown in Figure 11.
Training configuration. Unless stated otherwise, we follow the official implementation8. We retain the adversarial loss in the VAE objective. The VAE (encoder/decoder and discriminator) is trained with Adam (betas 0.5/0.9) and learning rate ; we train the diffusion model with learning rate . We train for 300k optimization steps with a batch size of 8, and use EMA with decay updated every 10 steps. Experiments are run with Python 3.12 and PyTorch version 2.9.0.
RangeLDM
Adaptation for Waymo. We adapt RangeLDM [13] to Waymo SIDE LiDAR range images following the same multi-sensor stacking strategy as for LiDM (Figure 11). We use the default RangeLDM compression setting with patch factor , corresponding to an overall compression ratio .
Training configuration. We follow the official implementation9 and train for 500k optimization steps (batch size 8 per GPU; Python 3.12; PyTorch 2.9.0). In preliminary experiments, we observed training instability when generating Waymo SIDE LiDAR data: although the VAE achieves strong reconstruction performance (Table 11 and Table 12), diffusion training may collapse mid-training, leading to a sharp degradation in sample quality without obvious numerical failures. We attribute this sensitivity primarily to the learning rate. While the original work uses , we reduce the learning rate to for Waymo; with the original setting, training does not reliably progress.
| Method | Compression | Input Size | Transform | Variant | R-FSVD ↓ | R-FPVD ↓ | CD () ↓ | EMD () ↓ |
| LiDM[41] | , | LogNorm | — | 426.71 | 191.74 | 505.98 | 102.68 | |
| , | LogNorm | w/o | 1.83 | 0.32 | 2.81 | 16.25 | ||
| RangeLDM[13] | LogNorm | — | 3.94 | 0.31 | 3.43 | 23.96 | ||
| LogNorm | w/o | 1.26 | 0.19 | 0.98 | 7.39 | |||
| LogNorm | — | 3.55 | 0.27 | 1.27 | 13.11 | |||
| LogNorm | w/o | 1.44 | 0.15 | 0.49 | 4.40 | |||
| CrossLiDAR VAE | Linear | w/o | 0.41 | 0.13 | 0.48 | 4.37 |
Table 11. VAE ablation results on Waymo SIDE LiDAR. Reconstruction quality is compared across VAE backbones and normalization strategies. Configuration : compression rate , range densification, and filling invalid points. denotes the discriminator (adversarial) loss. Performance is evaluated using perceptual metrics: R-FSVD, R-FPVD and statistical metrics: Chamfer Distance (CD), Earth Mover’s Distance (EMD), following [41]. The best result is highlighted in bold and the second-best in _underline_.
OpenDWM
Adaptation for Waymo. We adapt the LiDAR generation components provided in the OpenDWM codebase10 to the Waymo SIDE LiDAR setting. Specifically, we use its LiDAR VAE tokenizer together with the temporal DiT diffusion model. Because the released configuration is designed for sequence generation from a single LIDAR_TOP stream, we replace the original single-sensor setup with the four SIDE sensors (front, left, rear, right) and apply the same naive multi-sensor stacking strategy used for LiDM and RangeLDM (Figure 11). Each sensor is encoded independently using a shared VAE. The resulting latent tensors are then stacked along the sensor and temporal dimensions before being processed by the temporal diffusion transformer. We otherwise retain the point-cloud voxelization and latent parameterization from the official release, which voxelizes the scene within m volume with voxel sizes and uses a 32-channel latent representation.
Training configuration. We follow the public OpenDWM LiDAR configurations as closely as possible. The released VAE configuration uses AdamW with learning rate and betas . The temporal DiT configuration uses AdamW with learning rate , betas , and weight decay , together with cosine annealing (, ). The official temporal configuration enables gradient checkpointing, uses batch size 1, a FlowMatch Euler scheduler with 1000 training timesteps, and conditions on three reference frames. We note that the released OpenDWM LiDAR models are primarily configured for LiDAR_TOP data and layout-conditioned generation. As a result, the Waymo SIDE LiDAR setting requires non-trivial adaptation and is not a native training configuration in the original release.
Waymo Qualitative Results
In this section, we present qualitative results on the Waymo dataset. We highlight the impact of our architectural modules on the generation quality, demonstrating improvements in cross-sensor alignment, temporal consistency, and controllable scene synthesis.
Inter-sensor Alignment. The Inter-sensor Module provides clear benefits by enabling joint modeling of all LiDAR sensors within the same temporal frame. Instead of processing each sensor independently, the module allows the network to learn direct interactions across viewpoints through attention. As shown in Figure 12, the module enables coherent multi-view integration, leading to improved alignment and more accurate scene understanding compared to the independent sensor generation of the competitors.
Figure 12. Cross-Sensor Alignment on Waymo. Qualitative comparison between our method (the last two rows) and competing approaches (the first row). While competitors generate each sensor stream independently, leading to misalignment and geometric inconsistencies across viewpoints, our Inter-Sensor Module enables coherent multi-view generation with consistent object geometry and spatial alignment across all LiDAR sensors.
Temporal Consistency. One of the key capabilities of our approach, enabled by the LiDAR Sequence Module, is the modeling of long-range spatio-temporal dependencies within each sensor stream. By jointly attending over spatial tokens across multiple time steps under a causal constraint, the module captures motion patterns and temporal context while preventing information leakage from future frames. This design enforces temporal coherence during generation and improves reasoning over evolving 3D structures. As shown in Figure 13, the LiDAR Sequence Module yields smoother predictions, more stable object representations over time, and enhanced dynamic scene understanding.

Figure 13. Temporal Consistency and Controllable Generation on Waymo. Qualitative results demonstrating both temporal coherence and controllable generation. The first column shows BEV and 3D-box conditioning signals, while the remaining columns show generated LiDAR frames at different timesteps. Across consecutive frames, CrossLiDAR preserves stable object geometry and motion consistency while adhering to structural control signals for object placement and scene layout.
Controllable LiDAR Generation. By conditioning the model on 3D bounding boxes and BEV maps through the ControlNet-style branch, we enable precise object-level control during generation. These structural cues guide the generation, allowing the model to accurately place objects according to the specified layout. As shown in Figure 13, this mechanism reliably regulates object presence and spatial arrangement, resulting in strong alignment between the input structural constraints and the generated LiDAR scenes.
References
- [1]Hengwei Bian, Lingdong Kong, Haozhe Xie, Liang Pan, Yu Qiao, and Ziwei Liu. Dynamicity: Large-scale 4d occupancy generation from dynamic scenes. In International Conference on Learning Representations (ICLR), 2025. 3arxiv.org/abs/2410.18084
- [2]Nicolas Bonneel, James Tompkin, Kalyan Sunkavalli, Deqing Sun, Sylvain Paris, and Hanspeter Pfister. Blind video temporal consistency. ACM Trans. Graph., 34(6), 2015. 6DOI
- [3]Lucas Caccia, Herke Van Hoof, Aaron Courville, and Joelle Pineau. Deep generative modeling of LiDAR data. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5034–5040. IEEE, 2019. 1arxiv.org/abs/1812.01180
- [4]Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuScenes: A multi-modal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 2, 5, 12, 20
- [5]João Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4724–4733, 2017. 17
- [6]Boyuan Chen, Diego Martí Monsó, Yulin Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081–24125, 2024. 5arxiv.org/abs/2407.01392
- [7]Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, pages 1–16, 2017. 1arxiv.org/abs/1711.03938
- [8]Gamaleldin Elsayed, Aravindh Mahendran, Sjoerd Van Steenkiste, Klaus Greff, Michael C Mozer, and Thomas Kipf. Savi++: Towards end-to-end object-centric learning from real-world videos. Advances in Neural Information Processing Systems, 35:28940–28954, 2022. 3
- [9]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Proceedings of the 41st International Conference on Machine Learning, pages 12606–12633, 2024. 2, 3, 4
- [10]Xiaonan Fang and Song-Hai Zhang. Single-video temporal consistency enhancement with rolling guidance. In Computational Visual Media: 12th International Conference, CVM 2024, Wellington, New Zealand, April 10–12, 2024, Proceedings, Part II, page 109–130, Berlin, Heidelberg, 2024. Springer-Verlag. 17
- [11]Yongjie Fu, Anmol Jain, Xu Chen, Zhaobin Mo, and Xuan Di. Drivinggen: Real-world video generation for vision language model based autonomous driving. In 2024 IEEE International Automated Vehicle Validation Conference (IAVVC), pages 1–6. IEEE, 2024. 2
- [12]Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020. 1
- [13]Qianjiang Hu, Zhimin Zhang, and Wei Hu. RangeLDM: Fast realistic LiDAR point cloud generation. In European Conference on Computer Vision, pages 115–135. Springer, 2024. 2, 3, 4, 6, 17, 18, 19, 20, 21
- [14]Jiahui Huang, Zan Gojcic, Matan Atzmon, Or Litany, Sanja Fidler, and Francis Williams. Neural kernel surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4369–4379, 2023. 18arxiv.org/abs/2305.19590
- [15]Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. In Advances in Neural Information Processing Systems, pages 167283–167308. Curran Associates, Inc., 2025. 5
- [16]Tarasha Khurana, Peiyun Hu, David Held, and Deva Ramanan. Point cloud forecasting as a proxy for 4d occupancy forecasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1116–1124, 2023. 3
- [17]Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 1, 4arxiv.org/abs/1312.6114
- [18]Jason Ku, Ali Harakeh, and Steven L Waslander. In defense of classical image processing: Fast depth completion on the cpu. In 2018 15th conference on computer and robot vision (CRV), pages 16–22. Ieee, 2018. 12, 14
- [19]Bohan Li, Jiazhe Guo, Hongsi Liu, Yingshuang Zou, Yikang Ding, Xiwu Chen, Hu Zhu, Feiyang Tan, Chi Zhang, Tiancai Wang, et al. Uniscene: Unified occupancy-centric driving scene generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 11971–11981, 2025. 6, 7, 8, 17, 18, 20
- [20]Chenqi Li, Yuan Ren, and Bingbing Liu. Pcgen: Point cloud generator for LiDAR simulation. arXiv preprint arXiv:2210.08738, 2022. 1
- [21]Xiaofan Li, Yifu Zhang, and Xiaoqing Ye. Drivingdiffusion: layout-guided multi-view driving scenarios video generation with latent diffusion model. In European Conference on Computer Vision, pages 469–485. Springer, 2024. 2
- [22]Alan Liang, Youquan Liu, Yu Yang, Dongyue Lu, Linfeng Li, Lingdong Kong, Huaici Zhao, and Wei Tsang Ooi. Lidar-crafter: Dynamic 4d world modeling from lidar sequences. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 18406–18414, 2026. 2, 3, 6, 7, 8, 16, 20
- [23]Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2023. 16, 20
- [24]Jiahe Liu, Youran Qu, Qi Yan, Xiaohui Zeng, Lele Wang, and Renjie Liao. Erect video motion distance: A metric for evaluating motion consistency in videos. arXiv preprint arXiv:2407.16124, 2024. 6, 17
- [25]Xingyu Liu, Charles R Qi, and Leonidas J Guibas. Flownet3d: Learning scene flow in 3D point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 529–537, 2019. 3
- [26]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. 2, 4
- [27]Youquan Liu, Lingdong Kong, Weidong Yang, Xin Li, Alan Liang, Runnan Chen, Ben Fei, and Tongliang Liu. La la lidar: Large-scale layout generation from point cloud data. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7377–7385, 2026. 2
- [28]Youquan Liu, Lingdong Kong, Weidong Yang, Ao Liang, Jianxiong Gao, Yang Wu, Xiang Xu, Xin Li, Linfeng Li, Runnan Chen, et al. Veila: Panoramic lidar generation from a monocular rgb image. In 2026 IEEE International Conference on Robotics and Automation (ICRA), pages 10426–10433. IEEE, 2026. 2arxiv.org/abs/2508.03690
- [29]Zhen Luo, Junyi Ma, Zijie Zhou, and Guangming Xiong. Pcpnet: An efficient semantic-enhanced transformer network for point cloud prediction. IEEE Robotics and Automation Letters, 8(7):4267–4274, 2023. 3arxiv.org/abs/2304.07773
- [30]Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyu an Zeng, Mikita Sazanovich, Shuhan Tan, Bin Yang, Wei-Chiu Ma, and Raquel Urtasun. LidarSim: Realistic LiDAR simulation by leveraging the real world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11167–11176, 2020. 1arxiv.org/abs/2006.09348
- [31]Benedikt Mersch, Xieyuanli Chen, Jens Behley, and Cyrill Stachniss. Self-supervised point cloud prediction using 3D spatio-temporal convolutional networks. In Conference on Robot Learning, pages 1444–1454. PMLR, 2022. 3arxiv.org/abs/2110.04076
- [32]Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss. Rangenet++: Fast and accurate LiDAR semantic segmentation. In 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 4213–4220. IEEE, 2019. 15DOI
- [33]Javier Montalvo, Juan Ignacio Bravo Pérez-Villar, Álvaro García-Martín, Pablo Carballeira, and Jesús Bescós. SPIN: Spacecraft imagery for navigation. arXiv preprint arXiv:2406.07500, 2024. 1arxiv.org/abs/2406.07500
- [34]Arian Mousakhan, Sudhanshu Mittal, Silvio Galesso, Karim Farid, and Thomas Brox. Orbis: Overcoming challenges of long-horizon prediction in driving world models. arXiv preprint arXiv:2507.13162, 2025. 2arxiv.org/abs/2507.13162
- [35]Kazuto Nakashima and Ryo Kurazume. LiDAR data synthesis with denoising diffusion probabilistic models. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14724–14731. IEEE, 2024. 6, 18, 20arxiv.org/abs/2309.09256
- [36]Jingcheng Ni, Yuxin Guo, Xichen Liu, Rui Chen, Lewei Lu, and Zehuan Wu. MaskGWM: A generalizable driving world model with video mask reconstruction. arXiv preprint arXiv:2502.11663, 2025. 6, 7, 8, 17, 20
- [37]Kaustab Pal, Aditya Sharma, Avinash Sharma, and K Madhava Krishna. ATPPNet: Attention based temporal point cloud prediction network. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 11140–11146. IEEE, 2024. 3arxiv.org/abs/2401.17399
- [38]William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023. 18arxiv.org/abs/2212.09748
- [39]Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017. 15arxiv.org/abs/1612.00593
- [40]Hui Qian, Mingchen Wang, Maotao Zhu, and Hai Wang. A review of multi-sensor fusion in autonomous driving. Sensors, 25(19):6033, 2025. 2DOI
- [41]Haoxi Ran, Vitor Guizilini, and Yue Wang. Towards realistic scene generation with LiDAR diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14738–14748, 2024. 2, 3, 6, 17, 18, 19, 20arxiv.org/abs/2404.00815
- [42]Jonathan P Resop, Laura Lehmann, and W Cully Hession. Drone laser scanning for modeling riverscape topography and vegetation: Comparison with traditional aerial LiDAR. Drones, 3(2):35, 2019. 1DOI
- [43]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 3, 17, 18arxiv.org/abs/2112.10752
- [44]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015. 2arxiv.org/abs/1505.04597
- [45]Stefan Schubert, Peer Neubert, Johannes Pöschmann, and Peter Protzel. Circular convolutional neural networks for panoramic images and laser data. In 2019 IEEE intelligent vehicles symposium (IV), pages 653–660. IEEE, 2019. 17DOI
- [46]SenseTime-FVG. OpenDWM: Open driving world models, 2025. https://github.com/SenseTime-FVG/OpenDWM. 6, 7, 8
- [47]Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-rcnn: Point-voxel feature set abstraction for 3D object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10529–10538, 2020. 1arxiv.org/abs/1912.13192
- [48]Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024. 5arxiv.org/abs/2104.09864
- [49]Pei Sun, Henrik Kretzschmar, Xerxes Dotwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 2, 3, 6, 12, 17, 20arxiv.org/abs/1912.04838
- [50]Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018. 6, 17arxiv.org/abs/1812.01717
- [51]Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Advances in Neural Information Processing Systems (NeurIPS), 2017.arxiv.org/abs/1711.00937
- [52]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.arxiv.org/abs/1706.03762
- [53]Arun Venkatraman, Martial Hebert, and J. Bagnell. Improving multi-step prediction of learned time series models. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1), 2015.DOI
- [54]Qitai Wang, Lue Fan, Yuqi Wang, Yuntao Chen, and Zhaoxiang Zhang. Generative view synthesis on free driving trajectory. In International Conference on Learning Representations, pages 58640–58662, 2025.arxiv.org/abs/2410.18079
- [55]Ziyu Wang, Jing Ma, and Edmund MK Lai. A survey of scenario generation for automated vehicle testing and validation. Future Internet, 16(12):480, 2024.DOI
- [56]Xinshuo Weng, Jianren Wang, Sergey Levine, Kris Kitani, and Nicholas Rhinehart. Inverting the pose forecasting pipeline with spf2: Sequential pointcloud forecasting for sequential pose forecasting. In Conference on robot learning, pages 11–20. PMLR, 2021.arxiv.org/abs/2003.08376
- [57]Xinshuo Weng, Junyu Nan, Kuan-Hui Lee, Rowan McAllister, Adrien Gaidon, Nicholas Rhinehart, and Kris M Kitani. S2net: Stochastic sequential pointcloud forecasting. In European Conference on Computer Vision, pages 549–564. Springer, 2022.DOI
- [58]Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhonny Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting. arXiv preprint arXiv:2301.00493, 2023.
- [59]Jin Wu and Chengxi Zhang. Multi-LiDAR registration: A joint sensor-centric optimization approach. IEEE Sensors Letters, 10(2):1–4, 2025.DOI
- [60]Ziyi Wu, Yulia Rubanova, Rishabh Kabra, Drew A Hudson, Igor Gilitschenski, Yusuf Aytar, Sjoerd Van Steenkiste, Kelsey R Allen, and Thomas Kipf. Neural assets: 3D-aware multi-object scene synthesis with image diffusion models. Advances in Neural Information Processing Systems, 37:76289–76318, 2024.
- [61]Yuwen Yang, Wei-Chiu Ma, Jingkang Wang, and Raquel Urtasun. Learning compact representations for lidar completion and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1074–1083, 2023.arxiv.org/abs/2311.01448
- [62]Xiang Xu, Alan Liang, Youquan Liu, Linfeng Li, Lingdong Kong, Ziwei Liu, and Qingshan Liu. U4d: Uncertainty-aware 4d world modeling from lidar sequences. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10027–10039, 2026.arxiv.org/abs/2512.02982
- [63]Yinghao Xu, Menglei Chai, Zifan Shi, Sida Peng, Ivan Skorokhodov, Aliaksandr Siarohin, Ceyuan Yang, Yujun Shen, Hsin-Ying Lee, Bolei Zhou, et al. Disconscene: Spatially disentangled generative radiance fields for controllable 3D-aware scene synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4402–4412, 2023.arxiv.org/abs/2212.11984
- [64]Yunzhi Yan, Zhen Xu, Haotong Lin, Haian Jin, Haoyu Guo, Yida Wang, Kun Zhan, Xianpeng Lang, Hujun Bao, Xiaowei Zhou, and Sida Peng. StreetCrafter: Street view synthesis with controllable video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.arxiv.org/abs/2412.13188
- [65]Zetong Yang, Li Chen, Yanan Sun, and Hongyang Li. Visual point cloud forecasting enables scalable autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14673–14684, 2024.arxiv.org/abs/2312.17655
- [66]Tianwei Yin, Xingyi Zhou, and Philipp Krähenbühl. Center-based 3D object detection and tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11784–11793, 2021.arxiv.org/abs/2006.11275
- [67]Zhiyu Yin, Kehai Chen, Xuefeng Bai, Ruili Jiang, Juntao Li, Hongdong Li, Jin Liu, Yang Xiang, Jun Yu, and Min Zhang. ASurvey: Spatiotemporal consistency in video generation. arXiv preprint arXiv:2502.17863, 2025.arxiv.org/abs/2502.17863
- [68]Mingliang Zhai, Xuezhi Xiang, Ning Lv, and Xiangdong Kong. Optical flow and scene flow estimation: A survey. Pattern Recognition, 114:107861, 2021.DOI
- [69]Ji Zhang, Sanjiv Singh, et al. LOAM: LiDAR odometry and mapping in real-time. In Robotics: Science and systems, pages 1–9. Berkeley, CA, 2014.DOI
- [70]Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023.arxiv.org/abs/2302.05543
- [71]Jian Zhao, Wenxu Li, Bing Zhu, Peixing Zhang, Zhaozheng Hu, and Jie Meng. Corner case dataset for autonomous vehicle testing based on naturalistic driving data. Smart Cities, 8(4):129, 2025.
- [72]Yunsong Zhou, Michael Simon, Zhenghao Peng, Sicheng Mo, Hongzi Zhu, Minyi Guo, and Bolei Zhou. Simgen: Simulator-conditioned driving scene generation. Advances in Neural Information Processing Systems, 37:48838–48874, 2024.arxiv.org/abs/2406.09386
- [73]Dekai Zhu, Yixuan Hu, Youquan Liu, Dongyue Lu, Lingdong Kong, and Slobodan Ilic. Spiral: Semantic-aware progressive lidar scene generation and understanding. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.arxiv.org/abs/2505.22643
- [74]Vlas Zyrianov, Xiye Zhu, and Shenlong Wang. Learning to generate realistic LiDAR point clouds. In European Conference on Computer Vision, pages 17–35. Springer, 2022.arxiv.org/abs/2209.03954
- [75]Vlas Zyrianov, Henry Che, Zhijian Liu, and Shenlong Wang. Lidardm: Generative LiDAR simulation in a generated world. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 6055–6062. IEEE, 2025.arxiv.org/abs/2404.02903



