WorksComputer Science
The Reality Gap in Automated Driving Systems Testing: Landscape, Challenges, and Research Directions
Abstract
Simulation is essential for testing automated driving systems (ADS), but discrepancies between simulation and reality can undermine testing conclusions. It remains unclear which discrepancies the literature addresses and what evidence supports correspondence claims. We present a systematic mapping of 235 primary studies that measure or mitigate the reality gap in ADS testing, selected from 4,195 publications. We characterise objectives, targets, approaches, experimental setups, and evaluation evidence. Our classification distinguishes simulated and real-world execution across six ADS test components, separately for each approach and its evaluation reference. This distinguishes real-world information used to construct an approach from evidence used to evaluate its claims. Of the included studies, 76% address mitigation, primarily targeting scenario and actor realism and perception, while 94% report validation or measurement procedures. Only 61 studies report metrics from multiple families covering sensor/perception correspondence, ADS behaviour, vehicle dynamics, or safety outcomes; this coverage does not establish causal links between discrepancies and test verdicts. The synthesis highlights that real-world content and physical execution must be interpreted against the claim evaluated. The findings motivate claim-specific evaluation and component-level reporting to establish which discrepancies affect testing conclusions and when simulation evidence supports conclusions about real-world ADS behaviour.
ACM Reference Format: Stefano Carlo Lambertenghi, Oliver Weißl, Johannes Betz, and Andrea Stocco. 2026. The Reality Gap in Automated Driving Systems Testing: Landscape, Challenges, and Research Directions. ACM Trans. Softw. Eng. Methodol. 1, 1, Article 1 (January 2026), 50 pages. https://doi.org/XXXXXXX.XXXXXXX
Introduction
Advanced driver assistance systems (ADAS) and automated driving systems (ADS) operate in an open and largely unconstrained environment [288]. Their behaviour depends on learned perception components, planning and control software, and physical actuators. On-road testing alone cannot establish that such a system is acceptably safe. Demonstrating a fatality rate comparable to human drivers with statistical confidence would require hundreds of millions of kilometres of testing [117]. Software changes can require renewed evidence, while on-road testing exposes road
Authors’ addresses: Stefano Carlo Lambertenghi, [email protected], Technical University of Munich; fortiss GmbH, Boltzmannstraße 3, Garching, Bavaria, Germany, 85748; Oliver Weißl, Technical University of Munich; NorCom GmbH & Co. KGaA, Boltzmannstraße 3, Garching, Bavaria, Germany, 85748; Johannes Betz, Technical University of Munich, Boltzmannstraße 15, Garching, Bavaria, Germany, 85748; Andrea Stocco, Technical University of Munich; fortiss GmbH, Boltzmannstraße 3, Garching, Bavaria, Germany, 85748.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
© 2026 Association for Computing Machinery.
Manuscript submitted to ACM users to the failures that the campaign aims to find. Simulation makes rare and hazardous situations reproducible, less expensive to test, and executable in parallel. It is therefore widely used in ADS research and scenario-based safety assessment [27, 111, 127, 299].
Simulation introduces a different problem. A simulated test relies on models of the road, traffic, sensors, and vehicle dynamics, each approximating reality. The difference between a simulated execution and the corresponding real-world execution is the simulation-to-reality gap, or reality gap. This gap can undermine a test verdict in two ways. A simulator that is more benign than reality can create false confidence by failing to expose failures that occur in the real system. Conversely, it can create false alarms by reporting failures that do not reproduce in reality, directing engineering effort towards simulator artefacts rather than defects of the system under test. These concerns are supported by empirical observations. Offline and online evaluations can disagree on which inputs cause failures [42, 91, 92]. Failures found in simulation may transfer only partially to the corresponding physical platform [132, 231]. Different simulators can produce different results for the same ADAS and differ from reality [24, 26, 224]. Simulator non-determinism can introduce further differences [7].
Existing approaches include physics-based sensor models, calibrated vehicle dynamics, scenario reconstruction from driving logs and crash reports, learned and generative content synthesis, hardware-, vehicle-, and mixed-reality-in-the-loop testing, domain randomisation, and policy transfer [22, 38, 39, 63, 97, 98, 103, 161]. Across software engineering, robotics, computer vision, and vehicle engineering, researchers use partially overlapping or inconsistent terms such as reality gap, fidelity, realism, sim2real, and validity. These terms can concern the faithfulness of a sensor signal, the plausibility of a traffic scenario, the accuracy of a dynamics model, or the transferability of a control policy.
Existing secondary studies cover different parts of this problem. Surveys address ADS testing techniques [244, 298], scenario specification and generation [299], testing of learned components [196, 291], and simulation platforms and capabilities [118, 201]. The closest work to our scope surveys sim2real and digital twins in autonomous driving [103], primarily organising the field around technologies for transfer and virtual representation. These studies do not jointly characterise the reality gap as a testing problem, the real and simulated execution used by individual studies, and the evaluation references supporting their claims.
This systematic mapping examines which parts of an ADS or its environment are targeted by reality-gap research, how studies measure or mitigate discrepancies, and how real and simulated elements are combined in experimental setups. Following established guidelines for systematic reviews [125, 185], we analyse 235 primary studies selected from 4,195 publications. We characterise their research context, proposed approaches, evaluation, and experimental components. We distinguish sim2real approaches, which make simulation-derived artefacts applicable to reality, from real2sim approaches, which use real-world information to construct or calibrate simulated content. We also classify execution separately for the proposed approach and its evaluation reference across six components: scenario, actors, perception, localization, actuation, and ECU/control. This separation makes it possible to distinguish how real-world information contributes to an approach from how its claims are evaluated.
Our results show that most studies address mitigation, with research concentrating on scenario and actor realism and perception. Sim2real and real2sim approaches occur at similar overall rates, and experimental setups combine simulation, recorded observations, and physical equipment in diverse ways. These combinations must be interpreted against the research target and evaluation claim. Reducing a discrepancy or including real-world content does not by itself establish correspondence of ADS behaviour or agreement between simulated and physical test verdicts.
This paper makes the following contributions:
a systematic mapping of 235 primary studies on the simulation-to-reality gap in ADAS and ADS testing;
a characterisation of existing approaches, distinguishing sim2real and real2sim strategies;
a component-level taxonomy of real and simulated execution across scenario, actors, perception, localization, actuation, and ECU/control, applied separately to the proposed approach and its evaluation reference;
a characterisation of reported evaluation techniques, references, and metrics used to measure the gap or validate mitigation;
a synthesis relating research targets, mitigation strategies, execution configurations, and evaluation evidence, motivating research on how discrepancies affect ADS behaviour and testing conclusions. We provide a replication package containing the search results, extracted data, and analysis scripts [192].
Background
Automated Driving Systems
SAE J3016 distinguishes six levels of driving automation, from no automation to full automation [220]. Although ADAS and ADS have different responsibilities under this classification, our mapping concerns their testing configurations. We therefore use ADS broadly to include the driving automation systems evaluated in the literature, including systems formally classified as ADAS.
ADS commonly comprise sensing, perception, localization, planning, and control functions [288]. In modular architectures, perception identifies objects and road structures from sensor observations, localization estimates vehicle position and state, planning determines trajectories or manoeuvres, and control produces steering, braking, and acceleration commands. End-to-end architectures replace parts or all of this pipeline with learned mappings from observations to driving decisions or commands [25, 37]. Other approaches incorporate vision-language models [77] or learn policies through reinforcement learning [124]. Regardless of architecture, closed-loop behaviour depends on interactions between the ADS, vehicle, and environment.
Simulation and Reality in ADS Development and Testing
Simulation and synthetic data support component development, training, and evaluation as well as system-level testing. The distinction between simulated and real execution therefore extends beyond the use of a driving simulator.
Synthetic and real-world data. Synthetic sensor observations, vehicle states, and annotations can complement physical measurements, including recorded datasets such as KITTI [80] and the Waymo Open Dataset [236]. These sources can also be combined: recordings can be replayed, driving logs can support scenario reconstruction, and physical measurements can calibrate models. Processing recorded real-world observations differs from conducting a new physical execution. The observations originate from reality, but the ADS may process them offline or within an otherwise simulated setup.
Sensor simulation and learned generation. Conventional sensor simulation models how a sensor observes a virtual environment. Discrepancies can arise from both the environment representation and the sensing model. Learned approaches can instead generate or transform observations, for example through image translation [22, 98], diffusion models, or neural rendering of reconstructed scenes [161]. An observation remains synthetic even when its appearance or underlying representation is learned from real-world measurements.
Scenario and closed-loop simulation. A scenario describes the road and environmental conditions, participating actors, and their behaviour over time. Scenarios can be specified manually, generated from parameterised spaces, or reconstructed from logs and reports [299]. Driving simulators such as CARLA [49], SVL/LGSVL [198], and BeamNG [73] provide environments for executing them. In closed-loop testing, ADS actions affect vehicle motion and subsequent observations. This feedback distinguishes closed-loop execution from processing a fixed sequence of recorded inputs.
Physical and mixed execution. In-the-loop configurations combine simulation with different forms of system execution. In model-in-the-loop (MiL) testing, the driving logic and environment are computational models. In software-in-the-loop (SiL) testing, ADS software interacts with a simulated environment. In hardware-in-the-loop (HiL) testing, physical hardware, such as an ECU, interacts with simulated inputs or environments [38, 63]. In vehicle-in-the-loop (ViL) testing, a physical vehicle interacts with a partly or fully virtual environment. Mixed- and augmented-reality approaches similarly combine physical execution with virtual actors or stimuli [97]. These labels do not uniquely specify which components are simulated or real.
Scaled platforms, such as DonkeyCar [128] and F1TENTH [174], enable physical closed-loop experiments using real sensors, computing hardware, and actuators at reduced size and cost. Their physical execution does not by itself establish transfer to full-size vehicles. Digital twins provide another connection between domains by associating a physical system or environment with a corresponding virtual representation [103]; their implementations differ in the information exchanged and its direction.
The Simulation-to-Reality Gap
The simulation-to-reality gap, or reality gap, is the difference between an artifact, behavior, or observation obtained using a simulated or synthetic representation and its corresponding real-world counterpart. It can concern sensor observations, environmental and actor models, or system properties such as dynamics, timing, and actuation (Figure 1 illustrates discrepancies at the sensor and scene level). For testing, these discrepancies can affect individual verdicts and the assessment of test adequacy.

Figure 1. Four visual discrepancies between a real camera recording and a comparable simulation: (1) image appearance, (2) vehicle model, (3) road markings, and (4) road-user representation. Boxes highlight the differences; their impact on ADS behaviour and test verdicts requires separate investigation.
Simulation and reality are therefore not necessarily properties of an entire experiment. A physical ECU can process simulated observations, and a real vehicle can interact with virtual traffic. To describe these combinations, we distinguish six components:
Scenario: static road characteristics and environmental conditions;
Actors: other vehicles, pedestrians, cyclists, and their behaviour;
Perception: sensing and the observations provided to the ADS;
Localization: estimation of the ego vehicle’s position and state;
Actuation: steering, braking, throttle, and vehicle dynamics; and
ECU/control: the execution platform and driving software.
This decomposition describes the experimental setup, rather than the internal architecture of the ADS. It applies to both modular and end-to-end systems. Planning and decision-making remain part of the system under test rather than separate execution components. The operational classification is defined in § 4.
Sim2real and real2sim. We distinguish two directions in addressing the gap. Sim2real approaches make simulation-derived models, data, behaviours, or systems applicable to reality. Examples include improving simulator fidelity and adapting policies trained on synthetic data. Real2sim approaches use real-world information to construct, calibrate, or reconstruct simulated content. The directions can coexist: a study may calibrate a simulator using physical measurements and then evaluate whether its outcomes transfer to reality.
Related notions. Fidelity describes how accurately a model represents selected properties of its reference system. Realism more broadly concerns resemblance to real observations, scenarios, or behaviours. Domain shift concerns differences between data distributions, particularly those used for training and deployment. Robustness concerns stability under changes or perturbations and does not itself establish correspondence with reality. Simulator non-determinism, or flakiness, causes repeated simulated executions to produce different outcomes [7]; it can complicate gap measurement but does not itself constitute the gap.
We use simulation-to-reality gap as an umbrella term for discrepancies between simulated or synthetic representations and their real-world counterparts. The related notions retain the distinct meanings defined above.
Research Questions
This survey investigates how research on ADS testing and validation addresses the gap between simulation and real-world execution. We analyse the selected studies from five perspectives: context, proposed approaches, evaluation, and experimental components. The research questions for each perspective are presented below and summarised in Table 1.
| Category | Research Question | What it captures |
| Context | What is the primary objective of the study? | Whether the study measures the gap, mitigates it, or does both. |
| Which aspect(s) of the simulation–reality gap are addressed? | The aspect of the simulation–reality gap addressed by the study, including combinations of aspects. | |
| What is the direction of the simulation–reality relationship? | Whether the study addresses the gap from simulation to reality, from reality to simulation, or in both directions. | |
| Approach | Which research target does the study’s simulation–reality gap concern? | The specific capability, component, or phenomenon considered in relation to the simulation–reality gap. |
| How is the proposed simulation–reality mitigation implemented? | How the gap is mitigated, from the specific technique to its broader family. | |
| What are the execution levels of the proposed approach? | The execution level of each component and functional group across simulation and reality. | |
| Evaluation | How is the simulation–reality relationship evaluated? | How the study measures the gap or validates its mitigation, including cases with no explicit evaluation. |
| What are the execution levels of the evaluation reference? | The same execution-level classification as , applied to the evaluation reference. | |
| Which metrics are used to quantify the simulation–reality gap or the effectiveness of the proposed approach? | The quantities the study measures, and whether any quantitative measurement is reported at all. | |
| Experimental components | Which simulation platforms and reference datasets are employed? | The simulators and reference datasets used by the study. |
| Which hardware platforms are used? | The physical equipment involved, by its role in the experimental setup. | |
| Which system under test is evaluated? | The driving system or component being exercised, and the specific technology implementing it. | |
| How are evaluation scenarios or test cases generated? | The origin of the tested scenarios, ranging from manually specified scenarios to scenarios reconstructed from real driving. | |
| Which testing artefacts are publicly available? | Whether code, data, both, or nothing is released. |
Table 1. Research questions and what each one captures from a study.
Context
We distinguish the study objective, the aspect of the gap addressed, and the direction between simulation and reality. RQ identifies whether a study measures the simulation–reality gap, mitigates it, or does both, because measurement and mitigation studies make different claims. RQ identifies which aspect of the simulation–reality gap each study addresses. The gap may originate from different parts of an ADS testing setup, such as the representation of the driving scenario, the behaviour of traffic participants, the sensing process, localization, or vehicle actuation. A study may also address several of these aspects together or consider the system as a whole. RQ records direction separately because sim2real and real2sim, as defined in § 2, are independent of whether a study measures or mitigates the gap.
Approach
We characterise each approach by its research target, mitigation technique, and execution across simulation and reality.
identifies the specific research target considered in relation to the simulation–reality gap. While identifies the broader aspect of the gap, captures the particular capability, component, or phenomenon investigated by the authors. For example, studies addressing a perception gap may focus on camera imagery, LiDAR sensing, object detection, or combinations of sensing modalities. characterises how studies that aim to mitigate the gap implement their approach. Different studies may address similar research targets using substantially different techniques. We describe how the approach reduces the gap, from the specific technique to its broader family. characterises how the proposed approach is executed across simulation and reality. We decompose the experimental setup into the six execution components introduced in § 2: scenario, actors, perception, localization, actuation, and ECU/control. This captures experiments in which different components operate at different levels of simulation and reality. We record the execution level separately from the execution source, such as a live vehicle or a recorded dataset, because the same level can be reached in different ways. We also group the six components into three functional groups to simplify presentation: scene (scenario and actors), sensing (perception and localization), and control (actuation and ECU/control). Each component and functional group is characterised according to how it is executed across simulation and reality. The execution-level classification and its definitions are provided in § 4.
Evaluation
We characterise how each study evaluates the simulation–reality relationship, including the reference used for comparison and the metrics used to quantify differences. What the evidence establishes depends on the study objective. For “measure only” studies, we examine how the simulation–reality gap is measured. For “mitigate only” studies, we examine how the proposed mitigation is validated. Measure and mitigate studies contribute to both analyses.
records how each study evaluates the simulation–reality relationship, including cases with no explicit evaluation. An evaluation reference may be real for some components and simulated for others, just like the proposed approach. classifies this reference using the same components and execution categories. We compare scene with scene, sensing with sensing, and control with control, and inspect the individual components within each group. We do not assume that a reference with more physical components provides stronger evidence. identifies the metrics used to quantify the simulation–reality gap or evaluate the effectiveness of the proposed mitigation. These metrics quantify the simulation-to-reality gap or the effectiveness of the proposed mitigation. They include measures of perception performance, trajectory behaviour, and sensor similarity.
Experimental Components
These approaches are implemented and evaluated using different simulators, hardware, systems under test, and test cases. Together, these define the experimental setting from which the evidence is obtained. We record the simulation platforms and reference datasets (), hardware (), system under test (), origin of evaluation scenarios or test cases (), and publicly available testing artefacts ().
Methodology
We follow the systematic mapping guidelines of Kitchenham and Charters [125]. The methodology covers the identification and selection of primary studies, data extraction, and data analysis. We first establish the review procedure using a pilot set, refine it through a preliminary selection phase, and then apply the resulting procedure to the full corpus.
Identification of Research
Pilot Study
Before the systematic search, we selected an initial set of 10 studies, listed in Table 2, to establish and validate the review procedure. The purpose of the pilot is to identify missing or ambiguous categories rather than to provide a representative sample of the literature. We therefore selected studies already known to the authors that covered different objectives, directions, gap aspects, approaches, and execution levels. The set covers all three objectives and directions. It spans perception, ego behaviour [231], scenario and traffic realism [32], actuation [173], and combinations of aspects [5]. Its approaches include sensor physics modelling [159, 255], generative image techniques [133], data-driven simulator construction [5], learned traffic behaviour [32], and real-equipment integration [173, 214].
| Study | Objective | Direction | Gap aspect |
| VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles [5] | Mitigate | Real2sim | Perception, Behaviour |
| The Effect of Spray Water on an Automotive LIDAR Sensor: A Real-Time Simulation Study [255] | Mitigate | Real2sim | Perception |
| Synthetic to Real Gap Estimation of Autonomous Driving Datasets using Feature Embedding [71] | Measure | Sim2real | Perception |
| Sim-on-Wheels: Physical World in the Loop Simulation for Self-Driving [214] | Mitigate | Both | Perception |
| Reinforcement Learning with Human Feedback for Realistic Traffic Simulation [32] | Mitigate | Sim2real | Scenario |
| Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analysing Point Clouds at Target-Level [108] | Measure | Sim2real | Perception |
| Mind the Gap! A Study on the Transferability of Virtual Versus Physical-World Testing of Autonomous Driving Systems [231] | Measure | Sim2real | Behaviour |
| LiDARsim: Realistic LiDAR simulation by leveraging the real world [159] | Mitigate | Real2sim | Perception |
| Assessing Quality Metrics for Neural Reality Gap Input Mitigation in Autonomous Driving Testing [133] | Measure and Mitigate | Sim2real | Perception |
| AD-VILS: Implementation and Reliability Validation of Vehicle-in-the-Loop Simulation Platform for Evaluating Autonomous Driving Systems [173] | Measure and Mitigate | Both | Actuation |
Table 2. The 10 pilot studies used to calibrate the review protocol and the coded values that motivated their selection.
The spread of experimental setups was the primary selection criterion because the execution-level classification defined in § 4 must capture different combinations of simulation and reality. VISTA 2.0 [5] combines sensor observations synthesised from real driving data with simulated vehicle behaviour and deployment on a full-size vehicle. The spray study [255] calibrates simulated LiDAR and spray models using real measurements. The feature-embedding study [71] compares real and synthetic image datasets without a vehicle or control loop. Sim-on-Wheels [214] combines a real vehicle, sensors, and control with virtual traffic participants. TrafficRLHF [32] remains within simulation but uses human preferences and real driving data to improve traffic behaviour. The LiDAR domain-shift study [108] compares real and simulated sensing while controlling scenario and actor motion, whereas Stocco et al. [231] compare a small-scale physical vehicle with its digital twin. LiDARsim [159] reconstructs scenes and actors from real captures to generate synthetic LiDAR data. The neural reality-gap study [133] uses image translation and compares ADS behaviour across real, simulated, and translated inputs. Finally, AD-VILS [173] combines physical actuation, localization, and ECU/control with simulated traffic, actors, and perception inputs.
Together, these studies cover dataset-only comparisons, fully simulated experiments, simulation calibrated or reconstructed from real data, mixed real and simulated execution, and comparisons with physical vehicles. This diversity exposed the need to classify each execution component separately rather than treating an entire study as simulated or real. Three authors of this paper independently analysed the pilot studies using a preliminary version of the classification scheme. The resulting classifications were compared to identify ambiguities, inconsistencies, and missing categories. The research questions, extraction fields, and coding rules were then refined until consensus was reached. This produced the initial review protocol, which was further refined during the preliminary phase described in § 4.4.
Database Selection and Search Query
The literature search was conducted using Scopus [61], a bibliographic database of peer-reviewed scientific literature. Scopus was selected for its coverage of engineering and computer science and its support for Boolean searches over titles, abstracts, and keywords. The search query was developed iteratively using terminology identified during the pilot study. It was refined until all 10 pilot studies were retrieved while limiting the number of irrelevant results. The query combines three groups of search terms:
Automated driving systems. Terms describing ADAS and ADS, such as autonomous vehicle, self-driving, and ADAS.
Testing and validation. Terms related to software testing and validation, including test⁎, quality, and validat⁎.
Simulation-reality relationship. Terms describing the reality gap and related concepts, including sim2real, real2sim, reality gap, realism, and in-the-loop testing approaches.
Terms within each group are connected using the OR operator, while the three groups are combined using the AND operator. The search is restricted to titles, abstracts, and keywords.
Additional Selection Constraints
We apply the following constraints:
SC1 (Language). Only publications written in English.
SC2 (Publication year). Publications from 2018 onward, to focus on recent research.
SC3 (Peer review). Only publications from a publisher or venue with a documented, verifiable peer-review process.
The language and publication year constraints were applied through the Scopus search interface [61] as part of the query shown below. The peer-review constraint was applied during post-processing of the exported Scopus records.
SC3 applies a reproducible classification rule to Scopus publisher and source-title metadata. Retained venues include IEEE and ACM publications; peer-reviewed journals from MDPI, Elsevier, Wiley, SAGE, Taylor & Francis, Springer/Nature, and SAE; IFAC-PapersOnLine; ASME IMECE; SPIE proceedings; ASCE transportation conferences; Springer Proceedings in Advanced Robotics; Proceedings of Machine Learning Research; Journal of Machine Learning Research; Advances in Neural Information Processing Systems; and Driving Simulation Association proceedings.
We exclude venues whose peer-review process cannot be verified, including non-reviewed technical-paper series, edited-book chapters, and generic proceedings series with unconfirmed conference-level review. The replication package provides the executable metadata-based classification rule and supporting evidence for each publisher and venue decision. The resulting Scopus query is shown below.
TITLE-ABS-KEY( “test*” OR “assessing” OR “quality” OR “validat*” ) AND TITLE-ABS-KEY(( “simulation” AND ( “real-world” OR “real” ) ) OR “sim2real” OR “real2sim” OR “sim-to-real” OR “real-to-sim” OR “sim to real” OR “real to sim” OR “simulation gap” OR “simulation-gap” OR “reality gap” OR “reality-gap” OR “synthetic gap” OR “realism” OR “vil” OR “hil” OR “scil” OR “sil” OR “in the loop” OR “-in-the-loop”) AND TITLE-ABS-KEY( “self-driving” OR “autonomous vehicle” OR “adaptive cruise control” OR “acc” OR “adas” OR “advanced driver assistance systems” OR “automated driving” OR “ads” OR “automated-driving” OR “self driving” ) AND PUBYEAR > 2017 AND ( LIMIT-TO ( LANGUAGE,“English” ) )
When multiple versions of the same publication were available, the most complete version was retained. This affects two included studies: for each, the search retrieved a conference version, but we extracted and cited the extended journal version. Their demographic metadata in therefore also refers to the journal versions.
Search Results
The search was performed on 4 May 2026, applying the language and publication year constraints described above. The Scopus query returned 4,195 publications. Applying the peer-review constraint (SC3) excluded 1,009 publications, leaving 3,186 candidate publications for study selection.
Study Selection
Study selection was performed in two phases. The preliminary phase was used to obtain a larger set of included studies and refine the review procedure. The full selection phase then applied the refined procedure to the remaining candidate publications. Throughout both phases, publications failing the relevance criteria were distinguished from surveys and reviews excluded because they were not primary studies. The complete selection process is summarised in Figure 2.

Figure 2. Study identification, preliminary selection, and full selection process.
Preliminary Phase
The candidate publications were divided equally among the three reviewers. Each reviewer independently screened their assigned publications until identifying roughly 30 primary studies that satisfied the selection criteria. These 91 newly included studies, together with the 10 pilot studies, formed a first set of 101 studies.
The preliminary phase accounted for 1,112 publications: the 10 pilot studies, 91 newly included studies, 999 publications excluded based on the relevance criteria, and 12 surveys or reviews excluded as secondary studies. This left 2,074 candidate publications for the full selection phase. We excluded a publication based on the relevance criteria if its primary focus was not on ADS, if it did not measure or mitigate a simulation-to-reality difference relevant to evaluating or testing an ADS, or if it focused only on training or improving ADS without addressing this difference. Surveys, reviews, tutorials, and position papers without an original testing or validation approach were excluded as secondary studies.
The three reviewers then jointly examined the 91 newly included studies. They discussed the interpretation and classification of each study and refined the extraction and coding rules where necessary. This examination showed that six of the 91 did not belong in the corpus: five were not relevant on closer examination, and one was the conference version of a paper whose journal version was also selected. The initial corpus therefore comprised 95 studies. The 999 publications excluded based on the relevance criteria during this phase were independently checked by a second reviewer to reduce the risk of false-negative exclusions.
4.2.2 Full Selection The remaining 2,074 candidate publications were divided equally among the three reviewers and screened independently using the refined selection criteria. Uncertain cases were discussed by all three reviewers until consensus was reached. This phase resulted in 140 additional included studies, 1,894 relevance exclusions, and 40 surveys or reviews excluded as secondary studies. A small number of candidates shared the Driving Simulation Association’s publisher and venue metadata with its peer-reviewed Driving Simulation Conference proceedings but were themselves vendor product demonstrations or conference front-matter listings that were not peer-reviewed publications. These were identified manually from their content during screening and excluded without changing the SC3 rule.
Across the 3,186 candidate publications, 2,899 were excluded based on the relevance criteria (999 during the preliminary phase, 6 at the joint examination of the selected studies, and 1,894 during full selection), while 52 surveys or reviews were excluded as secondary studies (12 and 40, respectively). The initial corpus of 95 studies and the 140 additional studies identified during full selection therefore form the final corpus of 235 primary studies.
Data Extraction
Each included study is analysed to answer the research questions defined in § 3. The extracted information is recorded in a structured spreadsheet, with one row per study and one column per extraction field. The spreadsheet was defined before extraction, calibrated using the pilot studies, further refined during the preliminary phase, and then applied to the full corpus. The extraction fields use either predefined categorical values or free-text descriptions. Predefined fields include the study objective, gap aspect, simulation-to-reality direction, and execution classifications. For fields whose possible values could not be defined in advance, the reviewers instead recorded a short description of what the study reported. These free-text fields include the research target, proposed technique, evaluation procedure, and evaluation metrics. Their categories were derived later during data analysis, as described in § 4.6.
During the pilot study, all three reviewers independently coded the 10 pilot studies to calibrate the extraction process. Inter-rater agreement was measured using Fleiss’ [65] before disagreements were resolved. The agreement calculation covered the primary objective, direction, and execution level of each proposed-approach and evaluation-reference component. The pooled agreement across the 10 pilot studies was , indicating substantial agreement. The reviewers then discussed disagreements and refined the extraction guidelines until consensus was reached. The extraction procedure was further refined during the preliminary phase by jointly examining the 91 newly included studies. The three reviewers read and discussed these studies together, resolving ambiguities in the extraction fields, classifications, and coding rules. The resulting procedure was then applied to the remaining studies included during full selection. Ambiguities and borderline cases encountered during full extraction were discussed by all three reviewers until consensus was reached.
Data Analysis
Predefined categorical fields are analysed using the categories defined during data extraction. For each free-text field, we first record a short description of what each study reports. At this stage, the description is not assigned to a predefined category. For example, the research-target field describes what the study investigates in relation to the simulation–reality gap. The proposed-technique field describes how the study attempts to mitigate the gap. The evaluation-procedure field describes how the study measures the gap or validates a mitigation. The evaluation-metrics field records the quantities used in that evaluation. After extraction, descriptions referring to the same concept are grouped together. Different terminology does not create a separate group when it describes the same concept. Conversely, descriptions that refer to different concepts remain separate even when they occur in the same study. A study can therefore contribute to more than one group when it reports several distinct targets, techniques, procedures, or metrics.
The three reviewers then jointly examine the complete set of groups for each field. They compare their meaning, merge groups that represent the same concept, separate groups that contain distinct concepts, and refine their names and boundaries until consensus is reached. The resulting groups form the lowest level of the classification used in the analysis. Where needed, related groups are then organised into broader categories. For , related research targets are organised into macro categories and subcategories. For , mitigation techniques are organised into macro categories, subcategories, and, where useful, subsubcategories. and use separate classifications derived in the same way from their evaluation procedures and metrics. The hierarchy is therefore used to organise the categories after they have been derived from the extracted descriptions.
After the category structure is finalised, the assignments are checked against the original extracted descriptions and the final category definitions are applied consistently across the corpus. The categories are then used to compute the counts and distributions reported in the results. A study may have several values for a field when the corresponding research question allows multiple assignments. It is counted once in each applicable category and at most once within the same category. Counts across non-mutually-exclusive categories can therefore exceed the number of studies included in the corresponding analysis. Table 3 summarises the data used for each research question. The remainder of this subsection defines the analysis performed for each field.
| RQ | Data type | Data content |
| Predefined categorical | Measure only, Mitigate only, Measure and mitigate | |
| Predefined categorical, combinable | Scenario, Behaviour, Perception, Localization, Actuation, Entire; a study addressing several aspects equally is recorded as their combination | |
| Predefined categorical | Sim to real, Real to sim, Both | |
| Free text, categorised | Research target, grouped into macro categories and subcategories | |
| Free text, categorised | Mitigation technique, grouped into macro categories, subcategories, and optional subsubcategories | |
| Predefined categorical, one value per component | Scenario, Actors, Perception, Localization, Actuation, ECU/control, each recorded as Not modelled, Simulated, Pseudo-sim, Pseudo-real, Mixed, Both, or Real; Partially real additionally arises at group level; each setup also carries one or more execution-source tags (real vehicle, real hardware, real dataset, simulator, simulated dataset), or None | |
| Free text, categorised | Evaluation procedure, including its explicit absence | |
| Predefined categorical, one value per component | The same components and execution levels as ; the evaluation reference has its own execution-source tags | |
| Free text, categorised | Evaluation metrics, grouped by the quantity being measured | |
| Free text, named entities | Simulation platforms and reference datasets | |
| Free text, categorised | Hardware, grouped by its role in the experimental setup | |
| Free text, categorised | System under test and the specific technology implementing it | |
| Free text, categorised | Origin of evaluation scenarios or test cases | |
| Predefined categorical | Code, Dataset, Code and dataset, Other artefact, None; mutually exclusive and assigned by priority |
Table 3. Type and content of the data used for each research question.
Context
For , the predefined primary objective categories are measure, mitigate, and both. For , the predefined gap aspect categories are scenario, behaviour, perception, localization, actuation, and entire. When a study addresses several aspects with equal importance, their combination is recorded directly rather than derived through the free-text categorisation used elsewhere. For , the predefined direction categories are sim to real, real to sim, and both.
Approaches
analyses the research-target descriptions extracted from the studies. Descriptions referring to the same research target are grouped together using the procedure described above. Related targets are then organised into macro categories and subcategories. The classification describes what the study investigates in relation to the simulation–reality gap, rather than the technique used to address it.
analyses the techniques used to mitigate the simulation–reality gap and therefore applies only to studies whose objective includes mitigation. Descriptions referring to the same mitigation technique are first grouped together. Related techniques are then organised into six macro categories: Real-equipment integration, Generative content synthesis, Traffic/agent behaviour modelling, Real-world evidence reconstruction, Sensor physics modelling, and Ego policy/control transfer. These are further divided into subcategories and, where recurring techniques require an additional distinction, subsubcategories. A study may use more than one mitigation technique. It is assigned to every applicable macro category and may also appear in several subcategories within the same macro category.
Manuscript submitted to ACM
analyses the execution of the proposed approach across six components: scenario, actors, perception, localization, actuation, and ECU/control. Unlike and , these categories are predefined. Each component is assigned one of the execution levels defined in Table 4. This classification describes how a component is provided during execution, not how it was created. For example, calibrating a simulator or a generative model with real-world data does not by itself change how the resulting component is provided.
| Category | Definition |
| Not modelled | The component is not present. |
| Simulated | The component is provided by a general-purpose simulation platform or physics model. This holds even if the platform is calibrated, tuned, or partly trained using real-world data. |
| Pseudo-sim | The component is provided by a standalone model or generator, not a general-purpose simulation platform, whose representation is not constructed from real-world observations or measured real-world characteristics. |
| Pseudo-real | The component is provided by a standalone model, reconstruction, or generator whose representation is constructed from real-world observations, measurements, or measured real-world characteristics, but the real-world component is not present live during execution. |
| Mixed | Real and simulated elements are combined within the same execution. |
| Both | Separate simulated and real-world executions are performed in parallel. |
| Real | The component uses entirely real-world content. |
| Partially real | Group level only: one component is Real while the other is neither Real, Both, nor Mixed. |
Table 4. Categorical execution levels used for RQ2.3 and RQ3.2.
A general-purpose simulation platform remains Simulated even when it is calibrated, tuned, or partly trained using real-world data. The classification depends on whether the platform provides the component, not on how its parameters were set. Pseudo-sim and Pseudo-real both describe standalone models or generators used without such a platform. They differ in the source of their representation. For example, a physical model built from first-principles equations and only validated against real measurements is Pseudo-sim. The same type of model is Pseudo-real if its error or degradation behaviour is fitted to real measurements. This distinction applies across all six components. A component is Mixed when real and simulated elements are combined within the same execution. Examples include a real vehicle or sensor operating with simulated content, or a general-purpose simulator combined with real-world-derived content within the same component.
The six components are also grouped into three functional groups: scene, comprising scenario and actors; sensing, comprising perception and localization; and control, comprising actuation and ECU/control. Execution is analysed at both component and group level.
We use Partially real at group level when one component is Real while the other is neither Real, Both, nor Mixed. For example, real perception combined with simulated localization produces a Partially real sensing group. Mixed and Both describe the use of simulation and reality for the same component. Mixed combines them within one execution, while Both uses separate simulated and real executions. Partially real instead describes a difference between the two components of the same functional group. This category is derived from the component classifications and is never assigned to an individual component.
Component execution levels and execution sources are recorded separately. The execution level describes how a component is provided, while the execution source records the type of setup used to obtain it. Real describes real-world component content and does not by itself establish physical execution. For example, a study can use a recorded real-world dataset without operating a physical vehicle. The corresponding component is then Real, while its execution source is a Real dataset.
For both the proposed approach and its evaluation reference, we assign any applicable study-level execution sources:
Real vehicle. A physical vehicle operated in the study, including small-scale vehicles; Real hardware. A live sensor, controller, rig, or participant without an operated vehicle;
Real dataset. Pre-existing recorded real-world data;
Simulator. A model or generator executed by the study, including the generation of a new synthetic dataset;
Simulated dataset. Pre-existing synthetic data used without executing its generator.
None. It marks the absence of an execution source for the proposed approach or the absence of a comparison reference for evaluation. Recordings collected through vehicle or hardware experiments performed in the study retain the corresponding physical-source tag. Recordings from an earlier campaign are classified as a Real dataset. Approach tags include sources used for training or calibration, whereas reference tags describe sources used for comparison.
Evaluation
RQ: Evaluation analyses the free-text description of each study’s evaluation procedure. Descriptions of equivalent evaluation procedures are grouped using the procedure defined at the beginning of § 4.6. Studies with no recorded evaluation are checked again to distinguish an explicitly absent evaluation from missing extraction information. The same classification of evaluation procedures is used for all study objectives, but its interpretation depends on the objective. For a Measure only study, the procedure describes how the study measures the simulation–reality gap. For a Mitigate only study, it describes how the study validates the proposed mitigation. For a Measure and Mitigate study, the measurement of the gap and the validation of the mitigation are considered separately.
analyses the evaluation reference using the same predefined components, functional groups, and execution levels as . We match the proposed approach and evaluation reference within each study and compare the same component on both sides. Component-wise comparisons include only studies in which that component is present in the proposed approach. This restriction controls for differences in the components included in experimental setups. However, the presence of a component does not establish that it is the research target. We therefore interpret these comparisons alongside and do not treat an absent reference component as inadequate evidence without considering the study’s claim. The meaning of the evaluation reference depends on the study objective. For a mitigation study, it is the reference against which the proposed mitigation is validated. For a measurement study, it is the system or execution treated as the reference when measuring the simulation–reality gap. Not modelled ordinarily indicates that a component is absent from an otherwise present reference setup. When a study performs no evaluation, no reference setup exists. At component level, we represent this case using Not modelled for all components rather than introducing another execution-level category. The separate None source tag distinguishes the absence of an evaluation reference from the absence of an individual component.
analyses the free-text descriptions of the reported evaluation metrics. Metrics are first grouped according to the quantity they measure rather than the mathematical statistic used. Metrics that measure the same type of quantity therefore belong to the same higher-level category even when they use different numerical measures. Where needed, the specific metric is retained as a lower-level type. This separates what is being evaluated from the particular measure used to quantify it.
Experimental Components
RQ: Experimental Components records each named simulation platform and reference dataset used by a study. These are retained as named entities rather than grouped into derived categories. analyses the free-text hardware descriptions. Hardware with the same role in the experimental setup is grouped into the same category. analyses the free-text description of the system under test. Systems performing the same function are grouped together, while the specific model, architecture, or technique is retained where reported. analyses how the evaluation scenarios or test cases are obtained. The extracted descriptions are grouped into six categories: predefined test cases, simulation-based generation, reconstruction from real-world data, replay of recorded data, live real-world execution, and hybrid or mixed generation. assigns each study exactly one of five mutually exclusive artefact categories: Code and dataset when both are reported, Code or Dataset when only one is reported, Other artefact when the reported artefact is neither, and None when no artefact is reported. Categories are assigned in that priority order.
Threats to Validity
Search and selection validity The corpus may omit relevant studies because the search relies on Scopus and requires terms related to ADS, testing or validation, and the simulation–reality relationship in titles, abstracts, or keywords. Studies using different terminology may therefore remain undiscovered. Retrieving all ten pilot studies provides an initial check of search coverage, but this golden set cannot establish completeness. Restricting the corpus to English-language publications from 2018 onward and venues with verifiable peer review also excludes earlier work, industrial reports, and potentially relevant publications whose review process cannot be established from venue metadata. Our findings therefore characterise the retrieved peer-reviewed literature within these boundaries.
Reviewer bias and reliability Study selection and extraction involve judgement, particularly when distinguishing reality gap research from work concerned only with ADS training or improvement. We mitigate this threat through pilot calibration, joint examination of the preliminary corpus, independent checking of preliminary relevance exclusions, and consensus discussions. However, full-phase screening was divided among reviewers rather than independently duplicated. The reported Fleiss’ concerns selected categorical fields in ten pilot studies and does not establish agreement across the full corpus or the subsequently derived categories. Residual differences in interpretation may therefore affect inclusion decisions and category assignments.
Construct validity Concepts such as realism, fidelity, and the reality gap are used inconsistently across disciplines. Mapping them onto common categories requires interpretation and may obscure distinctions between studies. Separately classifying the proposed approach and its evaluation reference across six components reduces ambiguity, but execution categories describe experimental configurations rather than evidence adequacy. Similarly, physical equipment, recorded observations, or multiple metric families do not by themselves establish correspondence in ADS behaviour or test outcomes. Incomplete reporting may also constrain classification: an unreported element does not necessarily indicate its absence from the underlying experiment.
Synthesis and generalisability The reported frequencies describe publications rather than independent approaches or demonstrations of effectiveness. Related studies may reuse methods, datasets, or platforms, and a study may contribute to several categories. Category counts are therefore non-exclusive and should not be interpreted as comparative effectiveness or methodological quality. The synthesis also depends on published evidence and may underrepresent unsuccessful approaches or unpublished industrial practice. Consequently, the identified patterns apply primarily to the included literature. The mapping does not establish causal relationships between gap mitigation, behavioural correspondence, and agreement between simulated and physical test verdicts.
Results
Context
RQ: Primary Objective Mitigation is the most common objective: it accounts for 103 studies (43.8%), followed by Measure and mitigate with 76 studies (32.3%) and Measure only with 56 studies (23.8%). Overall, 76.2% of the studies include some form of mitigation.
RQ1.2: Addressed Gap Aspect
A study is assigned either a single gap aspect or, when it addresses several aspects equally, a combination of them, such as scenario and behaviour. The single aspects are scenario (the representation of the driving scenario), behaviour (the behaviour of traffic participants), perception (the sensing process), localization, actuation (vehicle actuation), and the entire gap (the system as a whole). Each study appears once, so the counts across all single aspects and combinations sum to 235 studies.
A single aspect is addressed by 193 studies (82.1%). Perception is the most common with 84 studies (35.7%), followed by scenario with 51 (21.7%). The entire gap, rather than a specific aspect, is addressed by 22 studies (9.4%), while 21 (8.9%) focus on actuation and 15 (6.4%) on behaviour. No study addresses localization, alone or in combination.
The remaining 42 studies (17.9%) address several aspects, in seven distinct combinations. Scenario and behaviour is the most common, with 10 studies (4.3%), followed by scenario and actuation, and perception and behaviour (8 each, 3.4%). Perception and scenario, and perception, scenario and actuation, occur in 5 studies each (2.1%), while perception and actuation, and actuation and behaviour, occur in 3 each (1.3%). Scenario is the aspect most often combined with others (28 of the 42 studies), ahead of perception and behaviour (21 each) and actuation (19).
RQ1.3: Direction
Real2sim accounts for 97 studies (41.3%) and sim2real for 95 studies (40.4%), while 43 studies (18.3%) address both directions.
Approach
RQ2.1: Research Target
The research targets and the corresponding primary studies are shown in Table 5.
| Research target | Studies | Primary studies |
| Scenario and actor realism | 109 | |
| Traffic/NPC realism | 50 | [3, 4, 18, 21, 23, 30, 31, 32, 35, 43, 46, 54, 75, 76, 83, 85, 105, 107, 115, 120, 122, 136, 140, 142, 148, 149, 151, 152, 162, 165, 188, 208, 215, 217, 218, 226, 237, 240, 245, 246, 260, 264, 267, 270, 277, 279, 281, 284, 292, 295] |
| Test-scenario validity | 26 | [10, 15, 40, 41, 59, 67, 78, 84, 86, 121, 126, 135, 152, 163, 183, 189, 212, 214, 221, 225, 248, 250, 260, 275, 276, 304] |
| Crash-scenario validity | 13 | [1, 19, 72, 104, 175, 216, 272, 273, 278, 290, 294, 297, 302] |
| Pedestrian/VRU realism | 12 | [50, 68, 69, 137, 146, 150, 153, 160, 228, 272, 283, 306] |
| Road/environment realism | 11 | [20, 28, 113, 145, 180, 195, 211, 238, 241, 263, 296] |
| Perception | 105 | |
| Camera perception | 56 | [4, 6, 11, 12, 17, 34, 45, 50, 55, 56, 58, 60, 68, 71, 79, 93, 102, 133, 135, 141, 144, 147, 163, 168, 172, 177, 178, 179, 182, 193, 194, 207, 213, 214, 219, 222, 223, 229, 232, 243, 247, 248, 249, 251, 252, 253, 257, 266, 268, 280, 286, 289, 293, 300, 301, 307] |
| LiDAR perception | 23 | [8, 16, 36, 70, 88, 99, 106, 108, 119, 143, 158, 159, 184, 199, 200, 202, 206, 210, 255, 262, 265, 283, 304] |
| Radar perception | 10 | [89, 90, 112, 156, 157, 166, 167, 258, 259, 303] |
| Camera + LiDAR perception | 8 | [33, 41, 47, 76, 96, 132, 269, 282] |
| Camera + LiDAR + Radar perception | 6 | [38, 94, 109, 130, 139, 152] |
| Camera + LiDAR + Event camera perception | 1 | [5] |
| Multi-sensor perception | 1 | [10] |
| Ego system behaviour and planning | 62 | |
| Ego-vehicle dynamics/actuation | 37 | [2, 4, 10, 15, 28, 29, 38, 44, 46, 48, 51, 52, 53, 57, 64, 78, 95, 113, 114, 123, 129, 132, 139, 154, 172, 180, 181, 183, 189, 191, 209, 212, 214, 246, 261, 263, 285] |
| SUT behaviour realism | 28 | [15, 28, 29, 33, 35, 40, 50, 52, 64, 84, 92, 138, 139, 158, 172, 173, 189, 194, 208, 209, 214, 221, 227, 231, 241, 242, 269, 300] |
| ADS-policy transfer | 9 | [5, 38, 64, 74, 87, 100, 222, 268, 287] |
| Ego localization/pose estimation | 7 | [18, 38, 116, 216, 227, 256, 269] |
| Communication/network fidelity | 3 | [101, 227, 305] |
| Small scale vehicle suitability | 1 | [186] |
Table 5. RQ research targets and supporting primary studies. Bars show the number of studies assigned to each target, and the final column lists the corresponding primary studies. Studies may be assigned to several targets, so counts across categories are not mutually exclusive and subcategory counts may not sum to the parent count.
Scenario and actor realism (109 studies). Traffic/NPC realism is the most common target with 50 studies, followed by test scenario validity with 26. Crash scenario validity accounts for 13 studies, pedestrian/VRU realism for 12, and road/environment realism for 11. Langer et al. [136] calibrate an urban microscopic traffic simulation against real traffic data before using it as a testing environment. Fremont et al. [67] specify scenarios in simulation and re-execute matching scenarios on a real vehicle. Oladimeji et al. [175] reconstruct real crash scenarios and compare them with the original accidents. Zofka et al. [306] place a real pedestrian inside a mixed-reality testbed. Bateman et al. [20] test an HD-map prediction model on real-world map changes not seen during training.
Perception (105 studies). Camera perception is the most common target with 56 studies, followed by LiDAR perception with 23 and radar perception with 10. The remaining 16 studies combine multiple sensing modalities, including Camera + LiDAR + Radar, Camera + LiDAR, and Camera + LiDAR + Event camera. Reway et al. [194] compare the performance of a camera-based object detector on simulated and real driving footage. Rosenberger et al. [200] analyse real-world LiDAR sensor behaviour to improve the physical fidelity of a simulated LiDAR sensor model. Wachtel et al. [259] validate a radar sensor model under non-ideal conditions.
Ego system behaviour and planning (62 studies). Ego vehicle dynamics/actuation is the most common target with 37 studies, followed by SUT behaviour realism with 28 and ADS policy transfer with 9. Lu et al. [154] build a vehicle-in-the-loop test system in which a real vehicle responds to an L3 autopilot operating against simulated surroundings. Tettamanti et al. [246] drive a real vehicle inside a microscopic traffic simulation and compare its behaviour with the same manoeuvres performed in simulation. Gutiérrez-Moreno et al. [87] adapt a decision-making policy developed in simulation for transfer to a real autonomous driving stack.
Ego localization/pose estimation (7 studies). These studies target the estimation of the ego vehicle’s position/pose.
Communication/network fidelity (3 studies). These studies target V2X or vehicle network communication behaviour.
Small scale vehicle suitability (1 study). This study investigates whether a small-scale or robot platform is a valid substitute for a full-size vehicle.
5.2.2 RQ: Mitigation Techniques Of the 179 studies with a mitigation objective, 164 use techniques from one mitigation family and 15 use techniques from several families. Studies using several families are counted in each, so the family counts in Table 6 exceed 179. The table also shows the techniques within each family; the corresponding primary studies are listed in our Appendix. A study can also use several techniques within the same family, so subcategory counts can exceed the family total.
| Mitigation technique | Studies |
| Real-equipment integration (ViL/HiL/MR/XiL) | 56 |
| Vehicle-in-the-loop (on-road) | 28 |
| Mixed reality | 21 |
| Controller/ECU hardware-in-the-loop | 6 |
| Co-simulation | 5 |
| Vehicle-in-the-loop (testbench) | 4 |
| Camera sensor-in-the-loop | 3 |
| Human-in-the-loop | 3 |
| Physical scale-model / surrogate-platform validation | 2 |
| LiDAR sensor-in-the-loop | 1 |
| Modular X-in-the-loop framework | 1 |
| Radar sensor-in-the-loop | 1 |
| Small-scale vehicle-in-the-loop | 1 |
| Steering-actuator hardware-in-the-loop | 1 |
| XiL/HIL | 1 |
| Generative content synthesis | 41 |
| Diffusion-based generation | 14 |
| GAN-based generation/translation | 12 |
| Neural rendering (NeRF/3DGS) | 6 |
| Synthetic data generation/augmentation | 4 |
| Synthetic perturbation/augmentation | 3 |
| Image compositing | 1 |
| Physics-based multi-sensor object insertion | 1 |
| Point-cloud/image reprojection & warping | 1 |
| Traffic/agent behaviour modelling | 29 |
| Learning-based (imitation/RL) traffic models | 7 |
| Traffic/behaviour-model calibration | 5 |
| Route/trajectory-conditioned behaviour modelling | 4 |
| Learned/generative trajectory models | 3 |
| Rule-based/behaviour-tree traffic models | 3 |
| Optimization/search-based generation | 2 |
| Statistical/distributional traffic calibration | 2 |
| Field-test/traffic-flow calibration | 1 |
| Human-reactive traffic models | 1 |
| Traffic/NPC behaviour modelling | 1 |
| Real-world evidence reconstruction | 28 |
| Recorded driving/sensor-log reconstruction | 12 |
| Crash/accident-evidence reconstruction | 6 |
| Map/aerial-derived reconstruction | 5 |
| Scenario-generation framework/toolchain | 2 |
| Data-driven simulator construction | 1 |
| Domain adaptation | 1 |
| Reconstruction-strategy comparison (classical vs. neural assets) | 1 |
| Retrieval-augmented domain adaptation | 1 |
| Sensor physics modelling | 28 |
| LiDAR sensor modelling | 14 |
| Camera sensor modelling | 6 |
| Radar sensor modelling | 5 |
| Camera+LiDAR+radar sensor modelling | 2 |
| Compute/timing-fidelity emulation | 2 |
| Ego policy/control transfer | 13 |
| Reinforcement-learning-based policy transfer | 5 |
| Vehicle-dynamics/controller calibration | 5 |
| Imitation-learning-based policy transfer | 2 |
| Generative-video-assisted policy transfer | 1 |
Table 6. mitigation technique families and techniques, among the studies with a mitigation objective. Bars show the number of studies using each family or technique. A study can use several families and several techniques within a family, so counts are not mutually exclusive and technique counts may exceed the family total.
Real-equipment integration (56 studies). These approaches combine simulation with physical vehicles, sensors, controllers, or human participants during testing. On-road vehicle-in-the-loop and mixed reality are the two largest subcategories. They include virtual traffic or actors surrounding a real vehicle, studies of real-vehicle testing platforms, and virtual or synthetic content injected into a vehicle’s sensing or planning stack. Smaller subcategories include controller/ECU hardware-in-the-loop, co-simulation, camera sensor-in-the-loop, non-road vehicle-in-the-loop testbenches, and human-in-the-loop testing.
Generative content synthesis (41 studies). These approaches create or modify sensor observations or scene content to make simulated inputs more representative of reality. Diffusion-based and GAN-based methods are the two largest
subcategories. Diffusion approaches generate or refine sensor imagery, video, traffic scenarios, or multi-agent trajectories. GAN-based approaches mainly modify camera inputs for downstream perception or detection tasks, with one LiDAR case. Saleh et al. [206] use a GAN to adapt simulated bird’s-eye-view LiDAR point clouds toward the real domain before training a vehicle detector. A further subset uses neural rendering, including neural radiance fields (NeRF) and three-dimensional Gaussian splatting (3DGS).
Traffic/agent behaviour modelling (29 studies). These approaches model how surrounding vehicles and other road users move and react. Techniques include learned behaviour, predefined rules, and calibration from observed traffic. Learned approaches include imitation learning from demonstrations, reinforcement learning through interaction or feedback, and trajectory-generation models. Trajectory-conditioned models generate behaviour subject to a specified path or motion pattern. Bergamini et al. [21] learn a reactive traffic-agent simulator from real-world driving logs.
Real-world evidence reconstruction (28 studies). These approaches use driving recordings, crash evidence, or other real-world information to construct simulation content or adapt models across domains. Recorded driving or sensor-log reconstruction is the largest subcategory, followed by crash or accident-evidence reconstruction and map- or aerial-derived reconstruction. Most studies reconstruct trajectories or agent behaviour from recorded logs. Others reconstruct complete scenes or road geometry, evaluate domain-adaptation techniques, or generate scenarios from real driving or accident databases. Kim et al. [122] extract naturalistic driving situations from real recordings to build training scenarios for an automated driving system.
Sensor physics modelling (28 studies). These approaches model how sensors produce measurements and how objects or environmental conditions affect them. LiDAR sensor modelling is the largest subcategory. The models cover effects such as weather-related attenuation, intensity, reflectance, and point dropout and are calibrated or validated against real LiDAR measurements. Smaller subsets model camera, radar, or combined camera, LiDAR, and radar sensing. Two studies model compute-hardware timing fidelity: one calibrates it against real CPU measurements rather than a live-integrated ECU, and one injects model and edge computation latency. Rosenberger et al. [199] decompose an automotive LiDAR sensor model into benchmarked functional components representing physical sensor behaviour.
Ego policy/control transfer (13 studies). These approaches adapt driving policies or control models so that behaviour developed in simulation transfers to a physical vehicle. Gutiérrez-Moreno et al. [87] adapt a decision-making policy developed in simulation for transfer to a real autonomous driving stack.
Execution Level of the Proposed Approach
Figure 3 shows how the proposed approaches are executed across the six components and the three functional groups. At the functional-group level, 48 of 235 studies (20.4%) use simulated content across Scene, Sensing, and Control. This is the most common configuration. Only 3 studies (1.3%) use real content across all three groups. The remaining studies use many different combinations of simulated, real, and intermediate execution levels.
Figure 3. RQ2.3 execution level of the proposed approach: execution levels of the six individual components (top), and the observed combinations for Scene, Sensing, and Control ranked by frequency (bottom). In the lower panel, colours encode study objective and icons encode execution level.
Simulation is the most common execution level at the component level. Across the 1,410 component classifications, 580 (41.1%) are Simulated. Another 398 (28.2%) are Not modelled, while 176 (12.5%) are Real. The remaining classifications are Pseudo-real (89, 6.3%), Mixed (88, 6.2%), Both (72, 5.1%), and Pseudo-sim (7, 0.5%).
The distribution also differs substantially between components. Scenario and Actors are most often Simulated, with 118 and 112 studies, respectively. Only 11 studies do not model Scenario, while 40 do not model Actors. Perception is also commonly Simulated (93 studies), but it has a more varied distribution. It is Pseudo-real in 23 studies, Mixed in 26, Both in 22, and Real in 19.
Later components are more often absent from the experimental setup. Localization is Not modelled in 105 studies, Actuation in 99, and ECU/control in 92. Nevertheless, simulation remains common when these components are included. Localization is Simulated in 82 studies, while Actuation is Simulated in 87 studies and ECU/control in 88. Real execution is also more common for these components than for Perception. The Real counts are 32 for Localization, 37 for Actuation, and 35 for ECU/control, compared with 19 for Perception.
The intermediate execution levels represent a substantial part of the corpus. Pseudo-real, Mixed, and Both together account for 249 of the 1,410 component classifications (17.7%). Their use also differs across components. Pseudo-real occurs most often for Scenario (34), Actors (29), and Perception (23). Mixed follows a similar pattern, with 26 studies for Scenario, 21 for Actors, and 26 for Perception. Both is most common for Perception (22), followed by Localization (14). Pseudo-sim is rare, with only 7 classifications across all six components.
These categories capture different relationships between simulation and reality. Pseudo-real represents real-world-derived content without the corresponding real component being present live. Mixed combines real and simulated elements within the same execution. Both instead represents separate simulated and real executions of the same component. Their frequency shows that a binary simulated-versus-real classification would hide a substantial part of the experimental setups in the corpus.
The lower panel of Figure 3 shows the combinations of execution levels across Scene, Sensing, and Control. The fully simulated configuration is the largest single group with 48 studies, but the overall distribution is fragmented. The next configurations contain only 12, 11, and 8 studies. A further 37 combinations occur in only one study each. Thus, no execution profile other than the fully simulated configuration represents a large share of the corpus.
These results describe component content rather than the physical setup used to obtain it. In particular, Real does not necessarily mean live physical execution. A component can be classified as Real when it uses recorded real-world data.
Evaluation
This group covers evaluation procedures, the execution level of the evaluation reference, and reported metrics. A study can use several evaluation procedures or metric categories in RQ and RQ. RQ reports one execution level per component for the evaluation reference.
5.3.1 RQ: Evaluation Procedure Nearly all studies report a validation or measurement procedure: 222 of 235 (94.5%), while 13 (5.5%) report none. The evaluation techniques and the corresponding primary studies are shown in Table 7. Direct or matched simulation-to-reality comparison (132 studies) and task/SUT-performance comparison (130) are the two most common techniques. They are followed by quantitative or distributional comparison (65), feature or image-quality comparison (24), and human or qualitative assessment (21). Sixteen studies have no substantive evaluation technique. This includes the 13 studies reporting no evaluation procedure and three that report an activity, such as a functional demonstration, without a comparison covered by the coded evaluation techniques.
| Evaluation technique | Studies | Primary studies |
| Direct or matched sim-to-real comparison | 132 | [2, 3, 4, 6, 8, 10, 11, 12, 16, 19, 20, 21, 23, 28, 34, 35, 38, 41, 43, 46, 47, 48, 51, 52, 54, 57, 59, 67, 69, 70, 74, 75, 76, 84, 85, 86, 87, 88, 90, 92, 93, 94, 99, 101, 102, 106, 108, 109, 112, 113, 114, 116, 120, 121, 129, 132, 135, 136, 137, 138, 139, 140, 141, 143, 147, 148, 149, 150, 156, 157, 158, 159, 162, 165, 166, 167, 168, 172, 173, 178, 179, 180, 181, 183, 188, 189, 194, 199, 200, 202, 207, 212, 213, 214, 215, 217, 218, 221, 226, 228, 231, 232, 240, 246, 248, 250, 253, 255, 256, 258, 259, 260, 261, 262, 263, 264, 265, 270, 272, 273, 277, 279, 280, 281, 282, 283, 287, 297, 301, 303, 304, 307] |
| Task/SUT performance comparison | 130 | [2, 4, 5, 6, 10, 11, 12, 15, 16, 17, 18, 19, 20, 28, 29, 35, 36, 38, 40, 41, 44, 45, 46, 47, 50, 51, 52, 53, 60, 67, 68, 69, 70, 72, 75, 78, 79, 86, 87, 88, 89, 90, 92, 93, 94, 100, 101, 102, 106, 107, 108, 112, 113, 114, 123, 132, 133, 135, 141, 142, 145, 146, 159, 160, 163, 166, 168, 172, 175, 177, 178, 179, 180, 182, 183, 184, 186, 188, 189, 191, 193, 194, 202, 206, 210, 211, 212, 213, 214, 215, 216, 217, 219, 221, 222, 226, 227, 229, 231, 232, 238, 240, 243, 245, 246, 248, 249, 250, 252, 257, 259, 263, 264, 265, 266, 268, 280, 282, 283, 284, 289, 290, 293, 294, 295, 296, 300, 301, 306, 307] |
| Quantitative or distributional comparison | 65 | [6, 32, 45, 46, 47, 48, 51, 54, 55, 56, 57, 60, 64, 72, 74, 75, 76, 83, 85, 90, 102, 105, 114, 115, 116, 119, 126, 133, 135, 136, 138, 142, 143, 148, 156, 157, 160, 162, 165, 172, 173, 191, 194, 215, 219, 222, 225, 226, 229, 237, 240, 249, 253, 260, 263, 265, 266, 267, 275, 281, 282, 283, 290, 292, 295] |
| Feature or image-quality comparison | 24 | [12, 17, 31, 41, 55, 58, 71, 84, 96, 102, 130, 140, 144, 175, 191, 193, 211, 223, 229, 247, 248, 265, 286, 300] |
| Human or qualitative assessment | 21 | [1, 12, 17, 30, 32, 35, 57, 69, 120, 121, 136, 149, 237, 251, 276, 285, 286, 287, 295, 302, 306] |
| No substantive evaluation | 16 | [33, 95, 104, 122, 151, 152, 153, 154, 195, 208, 209, 241, 242, 269, 278, 305] |
Table 7. RQ evaluation techniques and supporting primary studies. Bars show the number of studies using each technique, and the final column lists the corresponding primary studies. Studies may use several techniques, so counts are not mutually exclusive.
Most studies combine evaluation techniques. Ninety-eight studies (41.7%) use exactly one, 92 (39.1%) combine two, and 29 (12.3%) combine three or more. The most common exact combination is task/SUT-performance comparison with direct or matched simulation-to-reality comparison (49 studies, 20.9%). This is followed by task/SUT-performance comparison alone (38, 16.2%) and direct or matched simulation-to-reality comparison alone (35, 14.9%).
The next most common exact combinations are quantitative or distributional comparison + direct or matched sim-to-real comparison (17); quantitative or distributional comparison + task/SUT performance comparison + direct or matched sim-to-real comparison (16); no substantive evaluation (16); quantitative or distributional comparison (11); quantitative or distributional comparison + task/SUT performance comparison (11); feature or image-quality comparison (8); human or qualitative assessment (6); feature or image-quality comparison + task/SUT performance comparison (4); and human or qualitative assessment + direct or matched sim-to-real comparison (4).
Evaluation techniques differ across study objectives. Mitigate only studies most often use task/SUT-performance comparison (53 of 103, 51.5%), followed by direct or matched simulation-to-reality comparison (45, 43.7%). Direct or matched comparison is the most common technique among Measure and mitigate studies (54 of 76, 71.1%). Among Measure only studies, task/SUT-performance comparison and direct or matched comparison occur at similar rates, with 35 of 56 studies (62.5%) and 33 (58.9%), respectively. Quantitative or distributional comparison occurs in 20.4%, 32.9%, and 33.9% of the three groups, respectively.
No substantive evaluation occurs in 15 Mitigate only studies, no Measure and mitigate studies, and one Measure only study. For mitigation studies, this means that the effectiveness of the proposed mitigation is not explicitly assessed. For the Measure only study, no substantive procedure for measuring the gap is reported.
Execution Level of the Evaluation Reference
5.3.2 RQ3.2: Execution Level of the Evaluation Reference Figure 4 shows the execution level of the evaluation reference across the six components and the three functional groups. At the functional-group level, the most common reference profile has real content across Scene, Sensing, and Control (55 of 235 studies, 23.4%). It is followed by real Scene and Sensing with Control not modelled (49, 20.9%), and by references in which all three groups are absent (40, 17.0%). Only 14 studies (6.0%) have a fully simulated reference. In total, 36 distinct profiles occur, 19 of them in a single study, as the lower panel of the figure shows.
Figure 4. Figure 4. execution level of the evaluation: six-component execution-state counts (top), and the observed functional-group state combinations ranked by frequency (bottom). In the lower panel, colours encode study objective, and icons encode execution category.
At the component level, references are mostly real or absent. Of the 1,410 component classifications, 550 (39.0%) are Real, and 671 (47.6%) are Not modelled, while only 131 (9.3%) are Simulated. The intermediate levels are rare: Both (39, 2.8%), Mixed (13, 0.9%), Pseudo-real (5, 0.4%), and Pseudo-sim (1, 0.1%). By contrast, only 176 of the 1,410 classifications for the proposed approach (12.5%) are Real.
The distribution differs between components. The component counts coded Real are Scenario (141), Actors (112), Perception (112), Localization (67), Actuation (64), ECU/Control (54). Reference content is absent for Scenario (53), Actors (87), Perception (88), Localization (144), Actuation (147), and ECU/Control (152), so Localization, Actuation, and ECU/Control are Not modelled in more than half of the studies. The counts coded Simulated are Scenario (24), Actors (24), Perception (21), Localization (18), Actuation (20), and ECU/Control (24). Both is the most frequent intermediate level, with Scenario (9), Actors (7), Perception (9), Localization (6), Actuation (4), ECU/Control (4).
To see how these references are obtained, we count the study-level reference sources among the studies whose proposed approach includes each component. A source may support only part of the setup, so these counts are not attributed to individual components, and sources may overlap. A real dataset is the most frequent source for Scenario (98 of 224, 43.8%), Actors (89 of 195, 45.6%), and Perception (73 of 184, 39.7%). A real vehicle is the most frequent source for Localization (56 of 130, 43.1%), Actuation (58 of 136, 42.6%), and ECU/Control (61 of 143, 42.7%). Real hardware is rarely the source, in 7 to 18 studies per component. A simulator is the reference source in 17.7% to 22.8% of these studies, and no executed reference is reported in 16.1% to 22.3%.
Of the 55 studies with real content across all three reference groups, 50 include physical vehicle or hardware experiments and 5 use recorded real data without a physical vehicle or hardware experiment. A real-content profile is therefore not a count of live tests. Recorded images or point clouds can support a perception comparison; a closed-loop driving claim additionally requires evidence about the interaction of commands, motion, and subsequent observations.
5.3.3 RQ: Evaluation Metrics Seven categories capture the reported metrics, while an eighth records studies reporting no metric. The categories, their most frequent specific metrics, and the corresponding primary studies are shown in Table 8; all specific metrics are listed in the Appendix. Sensor/perception comparison (107 studies) and ADS behaviour/trajectory comparison (96) are the two most common categories. They are followed by Vehicle dynamics/actuation comparison (37), Safety/event outcome (34), No metric reported (23), Runtime/computational (19), Qualitative assessment (16), and Human/participant behavioural comparison (1).
| Metric category / metric | Studies | Primary studies |
| Sensor/perception comparison | 107 | |
| Detection accuracy/rate | 30 | [12, 16, 18, 19, 36, 47, 56, 89, 102, 141, 158, 159, 175, 177, 184, 194, 199, 202, 210, 212, 213, 219, 226, 248, 255, 257, 264, 265, 282, 283] |
| Average precision (mAP/AP) | 27 | [6, 20, 36, 68, 79, 85, 89, 102, 108, 140, 141, 158, 159, 168, 184, 186, 229, 243, 248, 257, 280, 282, 283, 289, 294, 301, 307] |
| IoU/segmentation overlap | 20 | [11, 60, 70, 88, 132, 140, 159, 166, 182, 214, 229, 243, 248, 249, 265, 282, 283, 294, 300, 307] |
| Precision/recall/F1 | 19 | [18, 20, 28, 36, 50, 79, 100, 102, 108, 140, 142, 158, 168, 193, 194, 206, 229, 249, 283] |
| Distributional image/video realism (FID/FVD/KID) | 18 | [31, 41, 88, 168, 178, 179, 223, 229, 237, 248, 265, 266, 268, 282, 286, 289, 300, 301] |
| Structural similarity (SSIM) | 15 | [12, 41, 94, 102, 140, 147, 168, 193, 214, 229, 248, 265, 282, 297, 300] |
| RMSE/MAE-family error | 13 | [6, 12, 70, 88, 140, 142, 184, 193, 226, 229, 265, 282, 297] |
| Learned perceptual similarity (LPIPS) | 12 | [41, 140, 147, 168, 178, 214, 229, 265, 282, 286, 300, 301] |
| ADS behaviour/trajectory comparison | 96 | |
| Speed/velocity comparison | 36 | [2, 3, 18, 29, 30, 43, 44, 48, 52, 53, 54, 64, 74, 78, 85, 86, 87, 90, 96, 112, 115, 136, 149, 150, 173, 183, 221, 226, 240, 246, 256, 259, 267, 281, 290, 302] |
| Trajectory error (ADE/FDE/RMSE/ATE) | 15 | [21, 53, 74, 75, 114, 116, 158, 179, 183, 189, 240, 268, 277, 282, 295] |
| Trajectory/path point-wise distance (sim vs. real) | 13 | [4, 21, 29, 32, 67, 78, 86, 129, 132, 145, 173, 188, 245] |
| RMSE/MAE-family error | 11 | [2, 48, 53, 64, 74, 115, 226, 265, 277, 282, 284] |
| Lane-keeping/car-following behaviour | 9 | [100, 136, 144, 145, 149, 212, 221, 226, 269] |
| Distribution-distance test | 6 | [32, 135, 136, 150, 226, 240] |
| Inter-vehicle/relative-spacing distance | 5 | [64, 85, 183, 240, 267] |
| Real-vs-simulated driving-behaviour comparison | 5 | [5, 189, 207, 217, 284] |
| Vehicle dynamics/actuation comparison | 37 | |
| Acceleration | 17 | [29, 30, 32, 48, 57, 64, 85, 136, 149, 150, 183, 221, 226, 240, 256, 273, 290] |
| Steering | 8 | [44, 74, 87, 123, 173, 183, 221, 231] |
| Jerk | 6 | [32, 76, 150, 273, 290, 295] |
| RMSE/MAE-family error | 6 | [48, 53, 57, 64, 74, 226] |
| Distribution-distance test | 5 | [32, 136, 150, 226, 240] |
| Braking/deceleration | 4 | [3, 191, 214, 246] |
| Yaw | 4 | [30, 76, 173, 221] |
| Correlation coefficient | 3 | [57, 64, 226] |
| Safety/event outcome | 34 | |
| Collision/crash rate | 26 | [15, 19, 21, 40, 45, 67, 84, 85, 100, 115, 150, 191, 212, 218, 226, 238, 240, 267, 269, 272, 273, 279, 290, 295, 300, 302] |
| Time-to-collision/PET | 12 | [45, 84, 85, 150, 191, 218, 226, 267, 273, 290, 300] |
| Pass/fail or disengagement rate | 8 | [10, 45, 218, 227, 228, 231, 290, 300] |
| Safety distance/near-miss | 3 | [57, 222, 240] |
| Scenario criticality score (composite risk indicators) | 1 | [104] |
| Traffic-safety distributional statistics | 1 | [281] |
| No metric reported | 23 | [33, 38, 69, 95, 109, 120, 122, 139, 151, 152, 153, 154, 163, 180, 208, 209, 241, 242, 250, 251, 278, 293, 304] |
| Runtime/computational | 19 | |
| Memory/compute utilization | 7 | [17, 70, 101, 147, 168, 219, 286] |
| Latency/runtime | 6 | [36, 44, 76, 100, 165, 214] |
| Frame rate (FPS) | 3 | [70, 147, 186] |
| Inference time | 2 | [28, 168] |
| Network/communication load | 2 | [216, 305] |
| Accuracy-vs-compute tradeoff (Pareto efficiency) | 1 | [219] |
| Model training time | 1 | [307] |
| ODD-coverage & ADS-failure testing overhead | 1 | [17] |
| Qualitative assessment | 16 | |
| Visual/qualitative inspection | 11 | [35, 47, 94, 108, 129, 195, 231, 258, 260, 287, 306] |
| Likert-scale rating | 3 | [1, 289, 295] |
| Human/user study | 2 | [121, 186] |
| Whiteness/subjective realism score | 1 | [186] |
| Human/participant behavioural comparison | 1 | [160] |
Table 8. RQ3.3 evaluation metric categories and their eight most frequent specific metrics (ties alphabetical), with primary studies.
Sensor/perception comparison (107 studies). Detection accuracy/rate is the most common metric with 30 studies, followed by average precision/mAP/AP with 27, IoU/segmentation overlap with 20, precision/recall/F1 with 19, and FID/FVD/KID with 18. Image-similarity metrics include SSIM (15), RMSE/MAE-based error (13), LPIPS (12), and PSNR (11). Manivasagam et al. [158] report detection average precision to quantify how closely a realistic LiDAR simulator reproduces real sensing conditions.
ADS behaviour/trajectory comparison (96 studies). Speed/velocity comparison is the most common metric with 36 studies, followed by trajectory error using ADE/FDE/RMSE/ATE (15), trajectory/path distance (13), and RMSE/MAE-based error (11). Lane keeping/car following behaviour appears in 9 studies and distribution-distance tests in 6. Inter-vehicle distance and direct real-versus-simulated driving-behaviour comparison each appear in 5 studies. Bui et al. [29] validate a model-predictive adaptive cruise controller by comparing commanded and real vehicle speed in a real-time system.
Vehicle dynamics/actuation comparison (37 studies). Acceleration is the most common measure with 17 studies, followed by steering (8), jerk (6), RMSE/MAE-based error (6), distribution-distance tests (5), and yaw (4). Stadler et al. [226] include acceleration among the metrics used to assess a scenario-based virtual test relative to reality.
Safety/event outcome (34 studies). Collision/crash rate is the most common measure with 26 studies, followed by time to collision or post-encroachment time (PET) with 12. Pass/fail, or disengagement rate, appears in 8 studies. Safety distance/near miss appears in 3 studies, while a composite scenario criticality score and traffic-safety distributional statistics appear in one study each. Bai et al. [15] measure collision rate in their AR-based vehicle-road cooperation testing system.
Runtime/computational (19 studies). Memory/compute utilization appears in 7 studies, latency/runtime in 6, frame rate/FPS in 3, and inference time and network/communication load in 2 each. Several other measures occur once.
Qualitative assessment (16 studies). Visual or qualitative inspection is the most common approach with 11 studies, followed by Likert-scale ratings (3), human/user studies (2), and subjective realism scoring (1).
Human/participant behavioural comparison (1 study). Martín-Serrano et al. [160] compare human driver behaviour in a real vehicle with behaviour in a virtual-reality driving scenario.
No metric reported (23 studies). These studies report no evaluation metric. This includes two studies that report only a general “comparability of results” between two setups without naming a specific quantity.
Experimental Components
Simulation Platforms
5.4.1 RQ: Simulation Platforms Table 9 lists the simulation platforms and datasets named by more than four studies. The complete list can be found in our Appendix. CARLA is the most common named simulator with 45 studies. Nineteen studies report no named simulation platform or reference dataset. Unity and IPG CarMaker follow with 16 studies each, while every other named platform appears in 13 studies or fewer. Among the reference datasets, KITTI is the most common with 18 studies. nuScenes follows with 10 studies, Cityscapes and the Waymo Open Dataset with 9 each, and Virtual KITTI with 8.
| Simulation platform / dataset | Studies |
| Simulation platforms | 182 |
| CARLA | 45 |
| IPG CarMaker | 16 |
| Unity | 16 |
| MATLAB/Simulink | 13 |
| SUMO | 12 |
| Custom/research simulator | 11 |
| VIRES VTD | 9 |
| SVL/LGSVL | 7 |
| Gazebo | 7 |
| LidarSim | 6 |
| Autoware simulator | 6 |
| Dataset/reference data | 66 |
| KITTI | 18 |
| nuScenes | 10 |
| Cityscapes | 9 |
| Waymo Open Dataset | 9 |
| Virtual KITTI | 8 |
| Custom/study-collected dataset | 7 |
| Traffic-trajectory dataset (NGSIM/HighD/rounD/INTERACTION) | 7 |
| No simulation platform or reference dataset reported | 19 |
Table 9. RQ4.1 simulation platforms and reference datasets reported by at least five studies.
Manuscript submitted to ACM
Hardware Platforms
RQ4.2 records the physical equipment used by each study, from full-size or small-scale vehicles to individual sensors, controllers, and compute hardware. A study can include several pieces of equipment. The hardware categories and the corresponding primary studies are shown in Table 10. The No hardware/unspecified category represents the absence of reported hardware.
| Hardware platform | Studies | Primary studies |
| No hardware or unspecified | 91 | [1, 6, 11, 12, 19, 20, 21, 23, 31, 32, 41, 45, 47, 55, 58, 60, 68, 71, 72, 75, 79, 83, 85, 88, 92, 93, 96, 101, 102, 104, 105, 107, 115, 121, 122, 126, 133, 135, 136, 138, 140, 148, 152, 153, 162, 163, 178, 182, 188, 206, 207, 210, 211, 215, 219, 223, 225, 226, 229, 237, 238, 240, 241, 243, 245, 247, 249, 257, 260, 265, 267, 268, 269, 270, 272, 275, 276, 277, 278, 279, 280, 281, 284, 286, 292, 293, 294, 296, 301, 302, 303] |
| Full-size vehicle | 76 | [2, 3, 5, 10, 15, 28, 33, 34, 35, 38, 40, 43, 46, 48, 50, 51, 52, 53, 54, 56, 57, 64, 67, 74, 78, 87, 90, 94, 106, 109, 112, 113, 114, 116, 120, 129, 139, 143, 149, 151, 156, 157, 158, 159, 166, 167, 172, 173, 180, 181, 183, 184, 189, 191, 194, 208, 209, 212, 214, 216, 217, 218, 221, 228, 242, 246, 248, 255, 259, 282, 283, 285, 297, 305, 306, 307] |
| Camera | 48 | [3, 4, 5, 15, 18, 28, 33, 34, 38, 40, 50, 52, 67, 74, 86, 90, 94, 100, 109, 116, 132, 139, 141, 144, 145, 146, 149, 151, 175, 177, 179, 186, 189, 193, 194, 208, 212, 214, 217, 222, 246, 252, 253, 263, 264, 282, 283, 297] |
| LiDAR | 44 | [5, 8, 15, 16, 28, 33, 36, 38, 40, 67, 74, 86, 90, 94, 99, 106, 108, 109, 119, 132, 139, 143, 158, 173, 184, 186, 189, 199, 200, 202, 208, 214, 242, 246, 255, 259, 262, 263, 264, 282, 283, 287, 304, 306] |
| Localization/positioning hardware | 44 | [4, 15, 35, 38, 50, 51, 52, 67, 69, 74, 84, 86, 87, 90, 94, 109, 116, 129, 139, 143, 151, 154, 156, 166, 167, 173, 179, 181, 183, 186, 189, 194, 208, 214, 216, 217, 218, 221, 228, 250, 256, 263, 297, 305] |
| Compute hardware | 25 | [3, 17, 18, 30, 44, 70, 76, 95, 100, 142, 147, 150, 165, 168, 186, 227, 228, 248, 266, 273, 289, 290, 295, 300, 307] |
| ECU/embedded controller | 20 | [2, 29, 33, 34, 38, 44, 48, 51, 172, 173, 181, 184, 189, 193, 194, 214, 217, 221, 227, 305] |
| Radar | 19 | [15, 38, 40, 74, 86, 89, 90, 94, 109, 112, 139, 156, 157, 166, 167, 208, 212, 214, 258] |
| Sensor-stimulation equipment | 11 | [18, 33, 67, 86, 112, 130, 139, 193, 216, 228, 253] |
| Roadside/communication infrastructure | 9 | [35, 146, 151, 183, 216, 217, 242, 256, 264] |
| Small-scale vehicle/robot | 9 | [18, 100, 132, 186, 231, 232, 251, 261, 287] |
| VR/MR or motion-capture equipment | 9 | [50, 69, 86, 132, 160, 208, 228, 285, 306] |
| Human driving controls | 8 | [30, 69, 84, 139, 189, 208, 228, 263] |
| Testbench/HIL equipment | 6 | [48, 51, 53, 59, 123, 139] |
| Drone/aerial platform | 4 | [137, 145, 189, 195] |
| Atmospheric test chamber | 3 | [94, 213, 255] |
Table 10. RQ4.2 hardware platforms and supporting primary studies.
No hardware or unspecified hardware is the most common category with 91 studies. These studies use simulation or existing data without reporting physical equipment. Among studies reporting hardware, full-size vehicles are the most common platform (76), followed by cameras (48), LiDAR (44), localization/positioning hardware such as GNSS/IMU units (44), and radar (19). Compute hardware, such as embedded GPUs or in-vehicle processing units, is reported by 25 studies and ECU/embedded controllers by 20.
The remaining categories are sensor-stimulation equipment (11); roadside/communication infrastructure, VR/MR or motion-capture equipment, and small-scale vehicles/robots (9 each); human driving controls (8); testbench/HIL equipment (6); drone/aerial platforms (4); and atmospheric test chambers (3). Atmospheric test chambers generate artificial rain, fog, or spray water to test real sensors or vehicles under controlled adverse-weather conditions. They are used by Hasirlioglu and Riener [94], Vargas Rivero et al. [255], and the PAVIN-platform study [213]. Lu et al. [154] build a vehicle-in-the-loop test system around a real full-size autopilot-equipped vehicle. Pahk et al. [179] evaluate a lane-keeping-assist system using real camera hardware after translating simulated images toward the real domain.
Hardware use differs across research targets. Studies with no reported hardware most frequently address camera perception and traffic/NPC realism, with 30 studies each. Full-size vehicles occur most often in ego vehicle dynamics/actuation (28) and SUT behaviour realism (19). These two targets also account for most uses of ECU/embedded controllers, with 10 and 9 studies, and nearly all uses of testbench/HIL equipment, with 5 and 1 of the 6 studies. LiDAR hardware occurs mainly in LiDAR perception (16) and SUT behaviour realism (11), followed by multi-sensor perception and ego vehicle dynamics/actuation (9 each). Camera hardware occurs across camera perception (13), multi-sensor perception (8), traffic/NPC realism (9), ego vehicle dynamics/actuation (12), and SUT behaviour realism (11). Localization/positioning hardware occurs most often in ego vehicle dynamics/actuation (13) and SUT behaviour realism (12). The complete set of of data is in our Appendix.
Systems Under Test
The systems under test are grouped into ten categories. The categories and specific technologies are shown in Table 11.
| System under test | Studies |
| No SUT | 89 |
| Camera perception | 40 |
| YOLO-family detector | 15 |
| Object detector | 12 |
| Segmentation model | 7 |
| BEV-based detector/feature extractor | 4 |
| Lane detection | 3 |
| VRU (pedestrian/cyclist) detection & driver-warning system | 3 |
| Faster R-CNN detector | 2 |
| 3D human pose estimation | 1 |
| ADAS smart front camera (sensor and embedded software) | 1 |
| Depth estimation | 1 |
| Object detection + tracking | 1 |
| Full ADS | 37 |
| Modular | 29 |
| End-to-end | 8 |
| Black-box AV (architecture undisclosed) | 1 |
| Simplified driving function with PID vehicle controller (path following, TTC-based braking) | 1 |
| Ego-vehicle control/actuation | 36 |
| Lane keeping/changing control | 11 |
| Adaptive cruise control | 10 |
| Automatic emergency braking | 6 |
| Steering control | 4 |
| Vehicle control stack/ECU | 3 |
| Model predictive control | 2 |
| Ground-robot dynamics control system | 1 |
| Integrated trajectory-planner + vehicle controller | 1 |
| PID trajectory-tracking controller | 1 |
| Rule-based reactive control policy | 1 |
| Waypoint-following control | 1 |
| Planning/trajectory prediction | 18 |
| Path/motion planning | 14 |
| Decision-making/planning policy | 3 |
| Trajectory/motion prediction | 2 |
| Automated valet-parking planner | 1 |
| LiDAR perception | 17 |
| 3D object detection | 7 |
| PV-RCNN | 3 |
| BEV feature extractor (LiDAR/camera fusion) | 2 |
| LiDAR object detection + tracking | 2 |
| PointPillars | 2 |
| Sensor simulation/generative model | 2 |
| LiDAR intensity simulation model | 1 |
| Point-cloud segmentation | 1 |
| PointNet(++) | 1 |
| PolarNet | 1 |
| RangeNet++ | 1 |
| VoxelNet | 1 |
| Radar perception | 7 |
| Localization | 5 |
| Modular-ADS localization component | 2 |
| GNSS/GPS-based localization | 1 |
| NDT-based mapping/localization | 1 |
| SLAM-based localization | 1 |
| LLM/foundation-model reasoning agent | 3 |
| Multi-sensor fusion perception/tracking | 2 |
Table 11. RQ4.3 systems under test and specific technologies.
No SUT (89 studies). These studies evaluate simulation or testbed fidelity without a vehicle or ADS-side driving function. This category also includes studies in which the testing infrastructure, environment-side behaviour model, or human-participant comparison is the object of study rather than a downstream vehicle-side function.
Camera perception (40 studies). These studies evaluate camera-based perception systems. The table distinguishes YOLO-family detectors (15), generic object detectors (12), segmentation models (7), BEV-based detectors or feature extractors (4), lane detection and VRU/pedestrian detection (3 each), and several techniques occurring in one study.
Full ADS (37 studies). These studies evaluate complete driving systems from sensing to actuation. The table distinguishes modular architectures (29), end-to-end architectures (8), and black-box or undisclosed architectures (1). Samak et al. [209] validate a modular autonomous driving stack across simulation and real scales using an integrated digital-twin framework.
Ego vehicle control/actuation (36 studies). These studies evaluate low-level control systems for steering, braking, or acceleration. The table distinguishes lane keeping/changing control (11), adaptive cruise control (10), automatic emergency braking (6), steering control (4), vehicle control stack/ECU (3), model predictive control (2), and several techniques occurring in one study.
Planning/trajectory prediction (18 studies). These studies evaluate decision-making, motion-planning, or trajectory-prediction systems. The table distinguishes path/motion planning (14), decision-making/planning policy (3), trajectory/motion prediction (2), and automated valet parking planning (1).
LiDAR perception (17 studies). These studies evaluate LiDAR-based perception systems. The table includes 3D object detection (7), PV-RCNN (3), LiDAR object detection and tracking (2), PointPillars (2), sensor simulation or generative models evaluated as systems under test in their own right (2), and several architectures occurring in one study.
Radar perception (7 studies). These studies evaluate radar-based perception systems.
Localization (5 studies). These studies evaluate localization and pose-estimation systems. The table distinguishes modular-ADS localization components (2), GNSS/GPS-based localization (1), NDT-based mapping/localization (1), and SLAM-based localization (1).
LLM/foundation model reasoning agent (3 studies). These studies evaluate driving systems based on LLM or foundation-model reasoning.
Multi-sensor perception and tracking (2 studies). These studies evaluate perception systems combining multiple sensing modalities.
5.4.4 RQ4.4: Test Case Generation Test cases are obtained or generated in several ways. The test-case origins and the corresponding primary studies are shown in Table 12.
Table 12. RQ4.4 test-case origin and supporting primary studies.
Simulation-generated test cases are the most common, appearing in 157 studies. They are followed by real-world execution (87 studies), real-world reconstruction (73), and recorded or replayed test cases (63). Manual or predefined test cases appear in 37 studies and hybrid or mixed generation in 16.
Nalić et al. [163] generate test scenarios within a co-simulation framework. Kettelgerdes et al. [119] take in-field measurements on a real test track to study precipitation effects on solid-state LiDAR sensors. Liu et al. [152] construct a digital-twin reconstruction of real-world driving conditions. Kim et al. [122] extract and replay naturalistic driving situations recorded from real traffic. Brogle et al. [28] run predefined test cases in a hardware-in-the-loop simulation without real-time constraints. Argui et al. [11] combine real and simulated navigation elements in a vision-based mixed-reality framework.
RQ₄.₅: Testing Artefacts Public testing artefacts reported by the studies are listed with citations in Table 13. Most studies report no public artefact (188 studies, 80.0%). Source code is reported by 26 studies (11.1%), another type of artefact by 14 (6.0%), a public dataset by 5 (2.1%), and both code and a dataset by 2 (0.9%). The 14 studies with another type of artefact are heterogeneous: they report open-source tools, platforms, or replication packages recorded without a repository link (6), a tool released together with the data it generates (1), demonstration videos or project web pages (3), and scenario or measurement data that is described as available without a confirmed release (4). One study classified under Source code [21] reports a repository that no longer resolves. We retain the Source code classification because the paper reports the repository, although the artefact is no longer retrievable.
Table 13. RQ₄.₅ testing artefacts and citations.
Discussion
The central implication of this mapping is that the importance of the reality gap depends on how simulation results are used. Failure discovery, comparison of ADS versions, pass/fail assessment, and estimation of operational risk require different forms of evidence and may be affected by different discrepancies. A discrepancy may affect ADS behaviour and change the testing conclusion. Future studies should therefore specify the decision being supported, the ADS and scenario conditions considered, the reference used for comparison, and the criteria used to evaluate correspondence. This would help identify which discrepancies matter, what level of fidelity is sufficient, and which additional measurements can reduce uncertainty most effectively. The following research directions address these questions based on the findings of our mapping.
Gap Mitigation and Evaluation Adequacy
Of the 235 studies, 179 address mitigation, alone or alongside measurement. However, improving over an unmitigated baseline does not mean that the remaining discrepancy is acceptable. Reducing one discrepancy also leaves others unresolved: a more accurate sensor model can coexist with unrealistic actor responses that affect ADS behaviour. The value of a mitigation for testing therefore depends on which aspect of the reality gap it addresses and how it interacts with the remaining discrepancies.
Likewise, different evaluation setups provide different types of evidence. Recorded observations allow sensor-level comparisons, but a fixed recording cannot capture how ADS actions affect subsequent observations. Physical execution provides evidence only for the aspects and conditions that were actually compared. Using physical equipment or describing a system as a digital twin does not, by itself, establish that the evaluation demonstrates the claimed correspondence.
Manuscript submitted to ACM
Studies of simulation fidelity investigate which aspects of a model require greater detail [176]. For example, Sagmeister et al. [205] compare vehicle models against real-world data and examine how model simplifications affect closed-loop motion-control evaluation. Such studies help identify fidelity requirements for specific applications, but do not establish that greater fidelity always leads to more reliable testing conclusions. Multi-simulator approaches investigate whether combining models improves testing. Digital siblings [24] combine predictions and evaluate them against a digital twin of a scaled vehicle, while MultiSim [224] searches for failures across simulators. However, the reported benefits do not establish that agreement between simulators implies agreement with physical outcomes. Further research should investigate when using multiple simulators improves the prediction of real-world test outcomes and how dependencies between simulators affect the reliability of these predictions.
Researchers should evaluate improvements in simulation fidelity and combinations of simulators against an independent reference and a clearly defined testing objective. These evaluations should distinguish between finding failures, obtaining consistent results across simulators, and agreement with physical outcomes. This research is relevant to ISO21448 [110] (SOTIF), which addresses safety assessment, and the UNECE NATM guidelines [254], which address the evaluation of virtual toolchains. Both provide a context for assessing simulation evidence in relation to the claims it supports. Future work should translate these assessment objectives into measurable criteria for specific claims, defining which discrepancies are acceptable and how these limits can be established experimentally. Tool developers can support this process by documenting model assumptions, the conditions under which models are applicable, and dependencies shared between simulators.
A concrete research objective is to determine how much model fidelity is needed for simulation and physical testing to support the same engineering decision, such as selecting between ADS versions or applying a pass/fail criterion, using independent physical measurements within a defined class of scenarios.
From Discrepancy Measurements to Testing Consequences
Our metric analysis identifies 61 studies using at least two of the four metric families, nine using at least three, and three using all four. However, using multiple metric families does not necessarily mean that a study investigates whether discrepancies in sensor observations affect ADS behaviour and, in turn, change test verdicts. Our extraction does not allow us to determine how many studies establish these relationships.
These relationships form a causal chain [187]: a discrepancy may affect ADS behaviour, which may in turn change the test verdict. Each causal link requires empirical evidence. An image-quality improvement can leave behaviour unchanged, while a small timing discrepancy can change a verdict near a braking threshold. Evaluation should therefore establish whether, and under which conditions, a discrepancy propagates to the testing conclusion [66]. This resembles fault propagation analysis, which investigates whether a fault leads to a detectable failure [13]. Controlled interventions and sensitivity analysis could help identify consequential discrepancy types and magnitudes.
Future studies should investigate this causal chain in a single experimental setting, with a clearly defined scenario class, ADS under test, and acceptance criterion. Where feasible, simulated and physical executions should use the same ADS, corresponding scenarios, and consistent evaluation criteria. Comparisons should account for differences caused by closed-loop divergence, variability between executions, time alignment, and oracle implementation. Reporting how frequently discrepancies change test verdicts would help quantify their impact under the evaluated conditions, provided that the procedure used to select discrepancies is also reported.
Researchers should distinguish between cases where an ADS passes in simulation but fails in reality and cases where it fails in simulation but passes in reality. The first case indicates potential false confidence, while the second may reflect simulator artefacts, execution variability, or differences between the simulated and physical experiments. These errors have different consequences: the former can leave physical failures undetected, while the latter can lead to unnecessary rejection, investigation, or retesting. Evaluations should report the frequency of both types of disagreement and specify which error type a method aims to minimise. When false confidence has more serious consequences, stricter acceptance criteria may be appropriate, but the resulting increase in false alarms should also be reported.
This motivates the creation of new benchmarks of paired simulated and physical executions that report the full pass/fail contingency table, define conditional disagreement rates, and distinguish routine scenarios from cases near an acceptance boundary [164].
Uncertainty About the Remaining Gap
The reality gap does not need to be completely eliminated for simulation results to be useful, but its effect on testing conclusions must be considered. Uncertainty estimates should distinguish between variability within the simulation and uncertainty about how well the simulation corresponds to reality. Repeating simulations can reduce sampling uncertainty but does not eliminate systematic modelling errors. Werling et al. [274] combine uncertainty quantification, stochastic simulation, and sensitivity analysis to assess system-level risk. Sim2Val [155] uses paired measurements and control variates to improve estimates of real-world metrics. These approaches provide ways to use imperfect simulation models under explicit assumptions, but do not provide a general confidence score for simulation results.
Researchers should first clarify what an uncertainty estimate represents. Confidence in an estimate of physical performance, the probability that simulation and physical testing produce the same verdict, and the maturity of an evaluation process are different concepts. Predictions of agreement between simulated and physical verdicts should be evaluated using independent paired executions, including cases near acceptance thresholds. Uncertainty intervals should be evaluated against physical measurements to determine whether they achieve the expected coverage and how informative they are. Their applicability under changed conditions should be assessed separately. Conformal prediction is one possible approach for constructing prediction sets or intervals using paired simulated and physical executions [9]. Its standard finite-sample coverage guarantees require exchangeability and generally apply across test cases rather than to each scenario individually. Future research should investigate appropriate calibration procedures and how dependencies between test cases and changes in data distributions affect these guarantees.
Assessors should also consider how uncertainty about the remaining reality gap affects the evidence supporting a claim. Building on the physical qualification of simulation evidence in the UNECE NATM guidelines and Euro NCAP protocols [62, 254], researchers and assessment bodies should develop reporting conventions that connect uncertainty estimates to acceptance criteria, applicability conditions, and unresolved verdicts. This would make probabilistic evidence easier to interpret and compare without treating a confidence value as a substitute for its supporting assumptions.
Tool developers should report uncertainty alongside simulation results and the conditions under which they are applicable. When plausible discrepancies could change a pass/fail decision, the test verdict may remain unresolved. ADS developers can use these cases to prioritise additional measurements, model improvements, or physical testing.
Future work should investigate how physical experiments can be selected to reduce uncertainty about a stated claim under a fixed budget, comparing adaptive selection with random or coverage-based selection using an independent evaluation set.
ADS Architecture and Gap Propagation
Our architectural comparison includes 29 studies of modular ADS and eight of end-to-end ADS, with two studies included in both groups. Direct or matched comparisons are more common in studies of modular ADS, while comparisons based on task or SUT performance are more common in studies of end-to-end ADS. However, the small number of studies and the overlap between groups do not allow general conclusions about evaluation quality.
Explicit interfaces provide a methodological distinction. In modular ADS, detections, trajectories, and commands can be compared or substituted at defined boundaries, making some discrepancies easier to isolate. These interfaces also allow evaluations that bypass relevant parts of the reality gap: a planner evaluated with idealised detections leaves the effect of realistic perception errors unresolved. However, component-level correspondence does not necessarily establish system-level correspondence because timing, correlated errors, and feedback affect component interactions.
Learned components introduce an additional challenge. Even when synthetic and real inputs look similar, they can produce different internal representations and downstream responses [203]. Comparing inputs alone may therefore fail to identify differences that affect ADS behaviour. Comparing internal representations can complement behavioural measurements, but these differences should also be evaluated for their effect on ADS decisions and test verdicts.
End-to-end ADS do not necessarily expose conventional functional boundaries, making it harder to identify which functions are responsible for observed discrepancies. Large backbones and vision-language planners further complicate this analysis [77], as architectural labels alone do not indicate which internal representations or interfaces can be evaluated. Controlled perturbations can help investigate how changes in simulated inputs affect ADS behaviour without requiring access to internal states. However, these effects must also be compared with those observed under corresponding physical conditions [132, 134].
Two research directions emerge: determining when correspondence between individual components also leads to correspondence at the system level, and developing methods to identify discrepancies that affect testing outcomes without relying on modular architectures. Applying comparable perturbations to different ADS architectures would help investigate how discrepancies propagate through these systems. Modular simulation environments such as AlpaSim [169] can support these experiments even when evaluating end-to-end ADS. Tool developers should therefore provide synchronised observations, reproducible configurations, and controlled model substitution to help ADS developers identify discrepancies and determine their effects on testing conclusions.
Future work should have controlled experiments that vary discrepancies individually and jointly to establish when their effects amplify or cancel, and whether component-level correspondence remains sufficient to predict system behaviour and test verdicts.
Evidence for Credibility and Real-World Claims
Regulators and assessors need to determine whether a testing campaign provides sufficient evidence to support a specific claim about real-world performance. Reducing a reality gap metric is only useful for this purpose if its relationship to the claim is established. Agreement between simulation and reality is only one part of this assessment, as requirements, scenario selection, test oracles, and conditions not covered by testing also affect the conclusions that can be drawn.
The UNECE NATM guidelines recommend virtual–physical comparisons and justification of extrapolation beyond validation scenarios [254]. Euro NCAP’s virtual-testing protocol combines model qualification with physical verification and permits idealised perception within its specified procedure [62]. Such qualification supports a bounded assessment; it does not establish perception robustness or unrestricted ADS performance. NVIDIA’s Halos SAFE documentation and Bosch’s credibility framework similarly relate simulation assessment to the claim and decision it supports [170, 171, 197].
These approaches raise three research questions: which physical comparisons are sufficient to evaluate the discrepancies relevant to a specific claim, how can evidence be extended beyond the conditions used for comparison, and how should the remaining uncertainty influence acceptance decisions? Researchers should develop experimental designs and methods for generalising results to address these questions. Assessment bodies can define what evidence and justification are required, while industry provides the comparisons, applicability conditions, and traceability needed to evaluate whether these requirements are met.
Campaign-level conclusions additionally require a sampling argument. Scenarios selected to expose failures do not directly estimate their operational frequency. Waymo’s collision-avoidance methodology connects scenarios and acceptance criteria to a defined objective [131], while the Safety First for Automated Driving white paper identifies representativeness and simulation uncertainty as challenges for statistical safety arguments [204]. Results cannot be treated as independent evidence when tests share modelling errors, and repeated success does not resolve conditions omitted from the campaign.
A clearly defined operational design domain helps establish the scope of a testing campaign and interpret its coverage. ODD specifications and structured scenario descriptions [190] should therefore guide scenario generation, selection, and reporting. However, coverage alone is not sufficient to establish confidence in a performance estimate. Operational exposure, sampling probabilities, dependencies between tests, and the remaining reality gap must also be considered. Researchers should investigate how these factors affect the estimated performance, while assessors should distinguish between evidence that a testing campaign can identify failures and evidence that supports estimates of operational risk.
A future research objective is to develop and evaluate claim-specific qualification protocols that link comparison references, acceptance criteria, sampling assumptions, and applicability conditions to engineering decisions.
Maintaining Evidence as the Reality (Gap) Changes
The reality gap depends on the ADS, simulation models, and operating conditions being compared. Changes such as retraining the ADS, replacing a sensor model, or modifying the vehicle can affect which discrepancies are relevant. Previous evidence of correspondence between simulation and reality may therefore remain valid for some claims but require reassessment for others. Real-world observations may also reveal conditions that were not considered in the original evaluation. Learned simulators introduce an additional concern: matching recorded observations does not establish correspondence when the ADS departs from the recorded trajectory. Controllable multi-sensor generation, as described in Waymo’s world-model announcement [271], motivates evaluating novel viewpoints, actor reactions, temporal consistency, and cross-sensor agreement. Evaluation data should remain distinct from data used to train or calibrate the policy and simulator.
Simulator updates can improve correspondence in some conditions while changing it in others. The applicability of earlier evidence must therefore be reconsidered for claims affected by the update [239]. Continual learning also raises questions about retaining previously established behaviour [233, 234]. Researchers should develop change-impact methods that identify which evidence remains applicable and which claims require new paired measurements, rather than assume either complete reuse or complete revalidation.
Changes in operating conditions are harder to identify than versioned software updates. Distribution monitoring and misbehaviour detection [81, 230, 235] offer starting points for identifying conditions that warrant reassessment. However, a distribution shift does not necessarily change a verdict, and a consequential discrepancy may escape an aggregate distance measure. Monitoring methods should therefore be evaluated for their ability to detect changes relevant to supported claims, including missed changes, false alarms, and detection delay.
For tool developers, versioned configurations, data provenance, and records linking claims to supporting comparisons would enable targeted reassessment. ASAM’s work on exchanging simulation-quality information provides a relevant standardisation direction [14]; the relationship between exchanged metrics and testing validity still requires evaluation.
For ADS developers, field observations and unresolved disagreements should guide additional measurements and physical comparisons. Shared paired executions and documented changes would help assess whether these procedures preserve credible evidence as the reality gap evolves.
Future work should compare targeted and full revalidation after controlled changes to the ADS, simulator, or sensors, measuring both the physical-testing effort saved and the consequential verdict disagreements missed on an independent evaluation set.
Related Work
Existing secondary studies examine ADS testing, simulation, and transfer between simulated and real environments from complementary perspectives. Table 14 compares their scope with this mapping, focusing on whether they systematically characterise reality gap targets, mitigation, evaluation, execution configurations, and component-level reference evidence.
| Secondary study scope | reality gap focus | Gap targets | Mitigation | Evaluation | Execution | Component evidence |
| ADS testing surveys [244, 298] | – | Partial | Partial | Partial | – | – |
| Scenario-based testing surveys [299] | – | Partial | Partial | – | – | – |
| Testing ML-based systems mappings [196, 291] | – | Partial | – | Partial | – | – |
| Simulation platform surveys [118, 201] | – | – | Partial | – | – | – |
| Sim-to-real and digital-twin surveys [103] | Partial | Partial | Yes | Partial | – | – |
| This mapping | Yes | Yes | Yes | Yes | Yes | Yes |
Table 14. Scope of related secondary studies and this mapping. “Yes” indicates systematic classification, “Partial” discussion without classification, and “–” a dimension outside the scope or not systematically addressed.
ADS and scenario-based testing. Tang et al. [244] organise ADS testing into module-level and system-level techniques, while Zhao et al. [298] place testing within a broader overview of autonomous driving architectures and functions. Zhong et al. [299] focus on scenario-based testing in simulation, formalising the approach and classifying the corresponding literature. These surveys establish the broader testing landscape, but do not systematically relate reality gap claims to the simulated and real components used in their evaluation.
Testing ML-based systems. Riccio et al. [196] map test adequacy criteria, input generation, and oracles across ML-based systems, including autonomous driving. Zhang et al. [291] organise the field around testing properties, components, and workflows. These studies provide relevant perspectives on evaluating learned components, but their scope extends across application domains and does not centre on correspondence between simulated and physical ADS execution.
Simulation platforms. Kaur et al. [118] compare driving simulators against defined requirements. Rosique et al. [201] review perception systems and simulation capabilities for autonomous vehicle research. Their unit of analysis is primarily the platform or sensing technology. Our mapping instead examines how individual studies configure simulated and real components and which references and measurements they report.
Sim-to-real transfer and digital twins. Hu et al. [103] provide the closest perspective to our study, reviewing technologies for bridging the reality gap in autonomous driving, including transfer methods, digital twins, and parallel intelligence. Their synthesis is organised primarily around technologies for transferring knowledge and constructing virtual representations. We complement this perspective by examining the reality gap as a testing problem: which discrepancies are targeted, how studies measure or mitigate them, and how the proposed approach differs from its evaluation reference. Gu et al. [82] map 34 studies on software testing for extended reality (XR) applications, covering testing techniques, objectives, evaluation methodologies, and supporting tools and datasets. Their study concerns the correctness and quality of applications operating in virtual, augmented, or mixed environments. Our mapping addresses a different relationship: whether simulated representations and executions correspond to their real-world counterparts in ADS testing. Mixed reality can form part of an ADS test configuration without making the XR application itself the system under test.
The contribution of this mapping is therefore the joint analysis of research claims, execution configurations, and reported evaluation evidence. Applying the same component-level classification to an approach and its reference makes their relationship explicit, while distinguishing the presence of real-world content from a comparison that addresses the targeted discrepancy. This characterisation supports interpretation of the reported evidence; it does not rank study quality or establish whether discrepancies propagate to changed ADS behaviour or test verdicts.
Conclusion
This systematic mapping study examined how ADS research measures and mitigates the simulation-to-reality gap in testing and validation. We analysed 235 primary studies selected from 4,195 publications, characterising their objectives, targets, mitigation techniques, experimental setups, and evaluation evidence. Our component-level classification distinguishes the proposed approach from its evaluation reference across scenario, actors, perception, localization, actuation, and ECU/control.
The literature concentrates on mitigation, addressed by 179 studies alone or alongside measurement, with scenario and actor realism and perception as the most common targets. Experimental setups combine simulated models, recorded observations, and physical equipment in ways that a single simulated-versus-real label cannot adequately describe. These combinations must instead be interpreted against the discrepancy targeted and the claim evaluated. Reducing a discrepancy does not by itself establish credible test results or ADS trustworthiness. Recorded observations can support sensor or perception comparisons, whereas closed-loop claims require evidence addressing feedback between system actions and subsequent observations. Similarly, physical equipment and multiple metric families do not establish that discrepancies are traced through ADS behaviour to test verdicts. Our mapping characterises the reported evidence; it does not determine how frequently these causal relationships are established or rank individual studies by quality.
Future research should connect gap mitigation to its testing consequences, quantify uncertainty about the remaining gap, and establish when evidence remains applicable across architectures, configurations, and operating conditions. Where feasible, matched simulated and physical executions should assess behavioural correspondence and verdict agreement. The classification and study mappings provide a basis for organising these comparisons. The central challenge is to establish which discrepancies can change a testing conclusion and what evidence justifies relying on that conclusion in the real world.
Acknowledgements
This research was funded by the Bavarian Ministry of Economic Affairs, Regional Development and Energy. We thank Donghwan Shin, Neil Walkinshaw, Christopher M. Poskitt, and Amir Soltani for insightful discussions that helped shape parts of the discussion in this paper.
References
Manuscript submitted to ACM
//proceedings.mlr.press/v288/leung25a.html Manuscript submitted to ACM
Manuscript submitted to ACM
References
- [1]M.U. Abbasi, J. Ma, and S. Moussa. 2025. Acr2Xodr: From Accident Reports to OpenDRIVE - Road Scenario Generation Using Large Language Models. In 2025 5th International Conference on Computer Systems, ICCS 2025. 65–69. https://doi.org/10.1109/ICCS67844.2025.11292219
- [2]J.P. Allamaa, P. Patrinos, H. Van der Auweraer, and T.D. Son. 2025. Learning-Based NMPC Adaptation for Autonomous Driving Using Parallelized Digital Twin. IEEE Transactions on Control Systems Technology 33, 4 (2025), 1317–1332. https://doi.org/10.1109/TCST.2024.3437163
- [3]M. Ameli, S.T. Mcquade, J.W. Lee, M. Bunting, M.W. Nice, H. Wang, W. Barbour, R. Weightman, C. Denaro, R. Delorenzo, S. Hornstein, J.F. Davis, D. Timsit, R. Wagner, R. Xu, M. Mahmood, M. Mahmood, M.L.D. Monache, B. Seibold, D. Work, J. Sprink le, B. Piccolo, and A.M. Bayen. 2025. Design, Preparation, and Execution of the 100-AV Field Test for the CIRCLES Consortium: Methodology and Implementation of the Largest Mobile Traffic Control Experiment to Date. IEEE Control Systems 45, 1 (2025), 139–155. https://doi.org/10.1109/MCS.2024.3500332
- [4]A. Amini, I. Gilitschenski, J. Phillips, J. Mosayebi, R. Banerjee, S. Karaman, and D. Rus. 2020. Learning Robust Control Policies for End-to-End Autonomous Driving from Data-Driven Simulation. IEEE Robotics and Automation Letters 5, 2 (2020), 1143–1150. https://doi.org/10.1109/LRA.2020.2966414
- [5]A. Amini, T.-H. Wang, I. Gilitschenski, W. Schwarting, Z. Liu, S. Han, S. Karaman, and D. Rus. 2022. VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles. In Proceedings - IEEE International Conference on Robotics and Automation, Vol. 2022-January. 2419–2426. https://doi.org/10.1109/ICRA46639.2022.9812276
- [6]M.H. Amini and S. Nejati. 2024. Bridging the Gap between Real-world and Synthetic Images for Testing Autonomous Driving Systems. In Proceedings - 2024 39th ACM/IEEE International Conference on Automated Software Engineering, ASE 2024. 732–744. https://doi.org/10.1145/3691620.3695067
- [7]Mohammad Hossein Amini, Shervin Naseri, and Shiva Nejati. 2024. Evaluating the impact of flaky simulators on testing autonomous driving systems. Empirical Softw. Engg. 29, 2 (Feb. 2024), 30 pages. https://doi.org/10.1007/s10664-023-10433-5
- [8]V. Anand, B. Lohani, R. Mishra, V. Kumar, and G. Pandey. 2026. Toward Closing the Sim-to-Real Gap for Autonomous Vehicles: A Physics-Guided Learning Approach for LiDAR Intensity Simulation. IEEE Transactions on Intelligent Transportation Systems (2026). https://doi.org/10.1109/TITS.2026.3681982
- [9]Anastasios N Angelopoulos and Stephen Bates. 2023. Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning 16, 4 (2023), 494–591.
- [10]J.F. Arango, R. Gutiérrez-Moreno, P.A. Revengá, Á. Llamazares, E. López-Guillén, and L.M. Bergasa. 2026. Closing Sim2Real Gaps: A Versatile Development and Validation Platform for Autonomous Driving Stacks. Sensors 26, 4 (2026). https://doi.org/10.3390/s26041338
- [11]I. Argui, M. Gueriau, and S. Ainouz. 2023. Building a Vision-Based Mixed-Reality Framework for Autonomous Driving Navigation. In 9th 2023 International Conference on Control, Decision and Information Technologies, CoDIT 2023. 1960–1965. https://doi.org/10.1109/CoDIT58514.2023.10284251
- [12]I. Argui, M. Gueriau, and S. Ainouz. 2025. Exploring augmentation strategies in mixed reality for autonomous driving with depth cameras. Transactions of the Institute of Measurement and Control 47, 7 (2025), 1455–1465. https://doi.org/10.1177/01423312241296919
- [13]J. Arlat, M. Aguera, L. Amat, Y. Crouzet, J.-C. Fabre, J.-C. Laprie, E. Martins, and D. Powell. 1990. Fault injection for dependability validation: a methodology and some applications. IEEE Transactions on Software Engineering 16, 2 (1990), 166–182. https://doi.org/10.1109/32.44380
- [14]ASAM. [n.d.]. P_2025_04 ASAM QSQ: Quantifying Simulation Quality. Ongoing standardisation project; project proposal. https://www.asam.net/index.php?id= dumpFile&f=9578&t=f&token=7faa36af2481adb98db248c260f3bc20de994484 Accessed 20 October 2026; not a released standard.
- [15]X. Bai, P. Dong, Y. Huang, S. Kumar, H. Yu, and Y. Ren. 2024. An AR-Based Mechanism for Training Framework, Components Modeling, and an Implementation Example. IEEE Internet of Things Journal 11, 13 (2024), 23460–23474. https://doi.org/10.1109/JIOT.2024.3386691
- [16]M.A. Balm, C. Hanilci, and N. Acir. 2025. Benchmarking domain adaptation for LiDAR-based 3D object detection in autonomous driving. Signal, Image and Video Processing 19, 12 (2025). https://doi.org/10.1007/s11760-025-04580-z
- [17]L. Baresi, D.Y. Xian Hu, A. Stocco, and P. Tonella. 2025. Efficient Domain Augmentation for Autonomous Driving Testing Using Diffusion Models. In Proceedings - International Conference on Software Engineering. 398–410. https://doi.org/10.1109/ICSE55347.2025.00206
- [18]R. Barker, A. Hurst, R. Shrubsall, G.M. Hassan, and T. French. 2018. A low-cost hardware-in-the-loop agent-based simulation testbed for autonomous vehicles. In IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM, Vol. 2018-July. 1015–1020. https://doi.org/10.1109/AIM.2018.8452376
- [19]S.K. Bashetty, H. Ben Amor, and G. Fainekos. 2020. DeepCrashTest: Turning Dashcam Videos into Virtual Crash Tests for Automated Driving Systems. In Proceedings - IEEE International Conference on Robotics and Automation. 11353–11360. https://doi.org/10.1109/ICRA40945.2020.9197053
- [20]S.M. Bateman, N. Xu, H.C. Zhao, Y. Ben Shalom, V. Gong, G. Long, and W. Maddern. 2024. Exploring Real World Map Change Generalization of Prior-Informed HD Map Prediction Models. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 4568–4578. https://doi.org/10.1109/CVPRW63382.2024.00459
- [21]L. Bergamini, Y. Ye, O. Scheel, L. Chen, C. Hu, L. Del Pero, B. Osinski, H. Grimmett, and P. Ondruska. 2021. Simnet: Learning Reactive Self-driving Simulations from Real-world Observations. In Proceedings - IEEE International Conference on Robotics and Automation, Vol. 2021-May. 5119–5125. https://doi.org/10.1109/ICRA48506.2021.9561666
- [22]Alex Bewley, Jessica Rigley, Yuxuan Liu, Jeffrey Hawke, Richard Shen, Vinh-Diêu Lam, and Alex Kendall. 2019. Learning to drive from simulation without real world labels. In 2019 International conference on robotics and automation (ICRA). IEEE, 4818–4824.arxiv.org/abs/1812.03823
- [23]D. Bi, Y. Zhao, Z. Gu, T. Mihalj, J. Hu, and A. Eichberger. 2025. Toward a Full-Stack Co-Simulation Platform for Testing of Automated Driving Systems. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC. 980–986. https://doi.org/10.1109/ITSC60802.2025.11423564
- [24]Matteo Biagiola, Andrea Stocco, Vincenzo Riccio, and Paolo Tonella. 2024. Two is better than one: digital siblings to improve autonomous driving testing. Empirical Softw. Eng. 29, 4 (May 2024), 33 pages. https://doi.org/10.1007/s10664-024-10458-4
- [25]Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, and Karol Zieba. 2016. End to End Learning for Self-Driving Cars. arXiv:1604.07316 [cs.CV]
- [26]Markus Borg, Raja Ben Abdesselam, Shiva Nejati, François-Xavier Jegeden, and Donghwan Shin. 2020. Digital Twins Are Not Monozgyotic - Cross-Replicating ADAS Testing in Two Industry-Grade Automotive Simulators. CoRR abs/2012.06822 (2020). arXiv:2012.06822 https://arxiv.org/abs/2012.06822
- [27]Markus Borg, Jens Henriksson, Kasper Socha, Olof Lennartsson, Elias Sonnsjö Lönegren, Thanh Bui, Piotr Tomaszewski, Sankar Raman Sathyamoorthy, Sebastian Brink, and Mahshid Helali Moghadam. 2022. Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System. https://doi.org/10.48550/ARXIV.2204.07874
- [28]C. Brogle, C. Zhang, K.L. Lim, and T. Braunl. 2019. Hardware-in-the-Loop Autonomous Driving Simulation without Real-Time Constraints. In IEEE Transactions on Intelligent Vehicles, Vol. 4. 375–384. https://doi.org/10.1109/TIV.2019.2919457
- [29]D.-T. Bui, H.D. Nguyen, Z. Gu, and A. Eichberger. 2024. Validation of Adaptive Cruise Control based on Model Predictive Control for Autonomous Vehicles in Real-Time System. In 10th 2024 International Conference on Control, Decision and Information Technologies, CoDIT 2024. 2621–2626. https://doi.org/10.1109/CoDIT62066.2024.10708081
- [30]J-D. Buzdagan, C. Antonyan, R.G. Boboc, and I-A. Rus. 2025. Human-Centered Simulations: Designing Reactive Traffic Scenarios for Enhanced Fidelity in Autonomous Vehicle Testing. In 2025 13th International Conference on Control, Mechatronics and Automation, ICCMA 2025. 496–501. https://doi.org/10.1109/ICCMA67641.2025.11369611
- [31]J. Cao, S. Liu, X. Yang, Y. Li, and S. Du. 2024. RAREST: Text-Driven Automatic Realistic Simulator for Autonomous Traffic. IEEE Signal Processing Letters 31 (2024), 3049–3053. https://doi.org/10.1109/LSP.2024.3481151
- [32]Y. Cao, B. Ivanovic, C. Xiao, and M. Pavone. 2024. Reinforcement Learning with Human Feedback for Realistic Traffic Simulation. In Proceedings - IEEE International Conference on Robotics and Automation. 14428–14434. https://doi.org/10.1109/ICRA57147.2024.10610878
- [33]M.G. Capdevila, N. Vitorino, Y. Poledna, B. Malena, M.F. Drechsler, G.G. Albuquerque, T. Furlan, and R.S. Netto. 2024. Smart&Safe Mobility Lab: Mixed Reality Environment with HIL, CV2X for VRU Detection. In 2024 IEEE 13th International Conference on Cloud Networking, CloudNet 2024. https://doi.org/10.1109/CloudNet62663.2024.10815791
- [34]S.M. Cardozo and V. Hlaváč. 2025. CARLA Hardware-in-the-Loop Simulation Environment for Autonomous Driving. SAE International Journal of Connected and Automated Vehicles 9, 2 (2025). https://doi.org/10.4271/12-09-02-0015
- [35]X. Che, C. Li, and Z. Zhang. 2021. A Test Method for Self-driving Vehicle Based on Mixed Reality. In Proceedings – 5th IEEE International Conference on Smart Internet of Things, SmartIoT 2021. 401–405. https://doi.org/10.1109/SmartIoT52359.2021.00075
- [36]H. Chen and S. Muller. 2022. Analysis of Real-Time LiDAR Sensor Simulation for Testing Automated Driving Functions on a Vehicle-in-the-Loop Testbench. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2022-June. 1605–1614. https://doi.org/10.1109/IV51971.2022.9827048
- [37]Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. 2023. End-to-end Autonomous Driving: Challenges and Frontiers. arXiv:2306.16927 [cs.RO]arxiv.org/abs/2306.16927
- [38]S. Chen, Y. Chen, S. Zhang, and N. Zheng. 2019. A Novel Integrated Simulation and Testing Platform for Self-Driving Cars with Hardware in the Loop. In IEEE Transactions on Intelligent Vehicles, Vol. 4. 425–436. https://doi.org/10.1109/TIV.2019.2919470
- [39]Xiaoyu Chen, Jiachen Hu, Chi Jin, Lihong Li, and Liwei Wang. 2021. Understanding Domain Randomization for Sim-to-real Transfer. CoRR abs/2110.03239 (2021). arXiv:2110.03239
- [40]Y. Chen, S. Chen, T. Xiao, S. Zhang, Q. Hou, and N. Zheng. 2020. Mixed Test Environment-based Vehicle-in-the-loop Validation - A New Testing Approach for Autonomous Vehicles. In IEEE Intelligent Vehicles Symposium, Proceedings. 1283–1289. https://doi.org/10.1109/IV47402.2020.9304658
- [41]Y. Chen, J. Zhang, Z. Xie, W. Li, F. Zhang, J. Lu, and L. Zhang. 2025. S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation. IEEE Transactions on Pattern Analysis and Machine Intelligence 47, 6 (2025), 4358–4376. https://doi.org/10.1109/TPAMI.2025.3543072
- [42]Felipe Codevilla, Antonio M. López, Vladlen Koltun, and Alexey Dosovitskiy. 2018. On Offline Evaluation of Vision-based Driving Models. CoRR abs/1809.04843 (2018). arXiv:1809.04843 https://arxiv.org/abs/1809.04843
- [43]A. Coppola, A. Mungiello, G. Pane, A. Petrillo, and S. Santini. 2025. On the Virtual Testing of ADAS in CCAM environment via Vehicle-in-the-Loop framework. In IFAC-PapersOnLine, Vol. 59. 151–156. https://doi.org/10.1016/j.ifacol.2025.07.097
- [44]G.T.F. Da Rocha, R.M. Bacurau, and J.V. Ferreira. 2025. MicroAutoware: An Autoware Vehicle Interface Designed for Real-Time Embedded Systems with Hardware-In-the-Loop (HIL) Support. In IEEE Intelligent Vehicles Symposium, Proceedings. 1546–1551. https://doi.org/10.1109/IV64158.2025.11097536
- [45]Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, Andreas Geiger, and Kashyap Chitta. 2024. NAVSIM: data-driven non-reactive autonomous vehicle simulation and benchmarking. In Proceedings of the 38th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS ’24). Curran Associates Inc., Red Hook, NY, USA, Article 902, 14 pages.
- [46]I.G. Daza, R. Izquierdo, L.M. Martínez, O. Benderius, and D.F. Llorca. 2023. Sim-to-real transfer and reality gap modeling in model predictive control for autonomous driving. Applied Intelligence 53, 10 (2023), 12719–12735. https://doi.org/10.1007/s10489-022-04148-1
- [47]D. Deter, C. Wang, A. Cook, and N.K. Perry. 2021. Simulating the Autonomous Future: A Look at Virtual Vehicle Environments and How to Validate Simulation Using Public Data Sets. IEEE Signal Processing Magazine 38, 1 (2021), 111–121. https://doi.org/10.1109/MSP.2020.2984428
- [48]R. Dona, S. Vass, K. Mattas, M.C. Galassi, and B. Ciuffo. 2022. Virtual Testing in Automated Driving Systems Certification. A Longitudinal Dynamics Validation Example. IEEE Access 10 (2022), 47661–47672. https://doi.org/10.1109/ACCESS.2022.3171180
- [49]Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. 2017. CARLA: An Open Urban Driving Simulator. In Proceedings of the 1st Annual Conference on Robot Learning. CoRR abs/1711.03938, 1–16. http://arxiv.org/abs/1711.03938
- [50]M.F. Drechsler, J. Peintner, F. Reway, G. Seifert, A. Riener, and W. Huber. 2022. RIME, A Mixed Reality Environment for Testing of Automated Driving Functions. IEEE Transactions on Vehicular Technology 71, 4 (2022), 3443–3456. https://doi.org/10.1109/TVT.2022.3160353
- [51]M.F. Drechsler, Y. Poledna, M. Hjort, S. Kharrazi, and W. Huber. 2024. Vehicle Dynamics Parameter Estimation Methodology for Virtual Automated Driving Testing. In IAVVC 2024 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC63304.2024.10786416
- [52]M.F. Drechsler, G. Seifert, J. Peintner, F. Reway, A. Riener, and W. Huber. 2022. How Simulation based Test Methods will substitute the Proving Ground Testing?. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2022-June. 903–908. https://doi.org/10.1109/IV51971.2022.9827394
- [53]M.F. Drechsler, C. Sell, Y. Poledna, and W. Huber. 2025. Accelerating the Approval of Automated Driving Vehicles through standardized XiL test environments. In Proceedings of the 2025 IEEE International Conference on Vehicular Electronics and Safety, ICVES 2025. 183–188. https://doi.org/10.1109/ICVES65691.2025.1137566
- [54]M.F. Drechsler, V. Sharma, F. Reway, C. Schütz, and W. Huber. 2022. Dynamic Vehicle-in-the-Loop: A Novel Method for Testing Automated Driving Functions. SAE International Journal of Connected and Automated Vehicles 5, 4 (2022). https://doi.org/10.4271/12-05-04-0029
- [55]A. Duminil, S.-S. Jeng, and D. Gruyer. 2024. Assessing fidelity in synthetic datasets: A multi-criteria combination methodology. In FUSION 2024 - 27th International Conference on Information Fusion. https://doi.org/10.23919/FUSION9988.2024.10706451
- [56]A. Duminil, S.-S. Jeng, and D. Gruyer. 2024. A Comprehensive Exploration of Fidelity Quantification in Computer-Generated Images. Sensors 24, 8 (2024). https://doi.org/10.3390/s24082463
- [57]B. Durukal, S. Kinay, N. Zengin, B. Gunaydm, B. Ozturk, and S.K. Yetkin. 2022. A Digital Twin Study: Particle Swarm Optimization of ACC Controller for Follow Acceleration Maneuver. In 2022 IEEE 21st International Conference on Sciences and Techniques of Automatic Control and Computer Engineering, STA 2022 - Proceedings. 146–153. https://doi.org/10.1109/STA56120.2022.10019084
- [58]T.D. Eberhardt, T. Bruhl, R. Schwager, T.S. Sohn, and W. Stork. 2025. Beyond the Smudge: Simulating Opaque and Transparent Automotive Camera Lens Soiling. In 2025 IEEE Symposium on Computational Intelligence for Energy, Transport and Environmental Sustainability, CIETES 2025. https://doi.org/10.1109/CIETES63869.2025.10995030
- [59]M. Elgharbawy, A. Schwarzhaüpt, M. Frey, and F. Gauterin. 2019. Ontology-based adaptive testing for automated driving functions using data mining techniques. Transportation Research Part F: Traffic Psychology and Behaviour 66 (2019), 234–251. https://doi.org/10.1016/j.trf.2019.07.021
- [60]A. Elmquist, R. Serban, and D. Negrut. 2025. A Methodology to Quantify Simulation- Versus-Reality Differences in Images for Autonomous Robots. IEEE Sensors Journal 25, 4 (2025), 6522–6533. https://doi.org/10.1109/JSEN.2024.3522050
- [61]Elsevier. 2026. Elsevier. https://www.vvv.acs.com Accessed: 2026-03-09.
- [62]Euro NCAP. 2026. Safe Driving & Crash Avoidance: Virtual Testing. Supporting Protocol Version 1.1. Euro NCAP. https://cdn.euroncap.com/cars/assets/Euro_NCAP_Supporting_Protocol_Safe_Driving_and_Crash_Avoidance_Virtual_Testing_v1_1_5f51c1897.pdf
- [63]Hosam K Fathy, Zoran S Filipi, Jonathan Hagena, and Jeffrey L Stein. 2006. Review of hardware-in-the-loop simulation and its prospects in the automotive area. In Modeling and simulation for military applications, Vol. 6228. SPIE, 117–136.
- [64]Z. Fei, M. Andersson, and A. Tingberg. 2024. Correlation of Software-in-the-Loop Simulation with Physical Testing for Autonomous Driving. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC. 2050–2057. https://doi.org/10.1109/ITSC58415.2024.10919543
- [65]Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin 76, 5 (1971), 378.DOI
- [66]Michael Foster, Christopher Wild, Robert M. Hierons, and Neil Walkinshaw. 2024. Causal Test Adequacy. In 2024 IEEE Conference on Software Testing, Verification and Validation (ICST). 161–172. https://doi.org/10.1109/ICST60714.2024.00023
- [67]D.J. Fremont, E. Kim, Y.V. Pant, S.A. Seshia, A. Acharya, X. Bruso, P. Wells, S. Lemke, Q. Lu, and S. Mehta. 2020. Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real World. In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems, ITSC 2020. https://doi.org/10.1109/ITSC45102.2020.9294368
- [68]D. Fu, J. Hu, D. Zhou, F. Wang, Z. Wang, and W. Liao. 2025. Controllable Pedestrian Video Editing for Multi-View Driving Scenarios via Motion Sequence. In Proceedings - 2025 IEEE/CVF International Conference on Computer Vision Workshops, ICCV-W 2025. 1861–1870. https://doi.org/10.1109/ICCVW69036.2025.00196
- [69]Z. Fu, Y. Xu, M. Yang, J. Huang, D. Yang, and Z. Zhong. 2024. Immersive Traffic Interactive Simulator for Multi-Agent. In Proceedings - 2024 IEEE 24th International Conference on Software Quality, Reliability and Security Companion, QRS-C 2024. 1004–1013. https://doi.org/10.1109/QRS-C63300.2024.00133
- [70]M.K. Gaddam, S. Kapoor, J. Vedula, M. Manu, M.A. Usmani, and S. Polvanov. 2025. LiDAR Sensor Simulation in Unity: Real-Time Performance for Autonomous Systems and Virtual Environments. In 2025 International Conference on Information, Implementation, and Innovation in Technology, I2ITCON 2025. https://doi.org/10.1109/I2ITCON65200.2025.11210718
- [71]N. Gadipudi, I. Elamvazuthi, M. Sannugam, L.I. Ihzar, T. Prasetyo, R. Jegadeeshwaran, and S.S.A. Ali. 2022. Synthetic to Real Gap Estimation of Autonomous Driving Datasets using Feature Embedding. In 2022 IEEE 5th International Symposium in Robotics and Manufacturing Automation, Manuscript submitted to ACM ROMA 2022. https://doi.org/10.1109/ROMA55875.2022.9915679
- [72]A. Gambi, T. Huynh, and G. Fraser. 2019. Generating effective test cases for self-driving cars from police reports. In ESEC/FSE 2019 - Proceedings of the 2019 27th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 257–267. https://doi.org/10.1145/3338906.3338942
- [73]Alessio Gambi, Pascale Maul, Marc Mueller, Lefteris Stamatogiannakis, Thomas Fischer, and Sebastiano Panichella. 2019. Soft-body Simulation and Procedural Generation for the Development and Testing of Cyber-physical Systems. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) (ICSE-SEIP ’19). IEEE.
- [74]M. Ganesan, B. Chokkalingam, S. Kandhasamy, R. Verma, and L. Mihet-Popa. 2025. Bridging the Sim-to-Real Gap in Motion Planning for Autonomous Electric Vehicles Using Autoware: A Comparative Study of Simulation and Real-World Deployment. IEEE Access 13 (2025), 136210–136225. https://doi.org/10.1109/ACCESS.2025.3587991
- [75]H. Gao, T. Qu, X. Gong, P. Wang, and H. Chen. 2025. MSIE-Transformer: A novel driving behavior modeling approach for virtual simulation test environment. Accident Analysis and Prevention 217 (2025). https://doi.org/10.1016/j.aap.2025.108039
- [76]X. Gao, Z. Wang, Y. Feng, C. Wang, Z. Zhou, Y. Huang, L. Ma, Z. Chen, and B. Xu. 2026. Driving realistic Driving Sequence Synthesis for Testing Multi-Sensor Fusion Perception Systems. IEEE Transactions on Software Engineering 52, 4 (2026), 1176–1197. https://doi.org/10.1109/TSE.2025.3650399
- [77]Yuan Gao, Mattia Piccinini, Yuchen Zhang, Dingrui Wang, Korbinian Moller, Roberto Brusnicki, Baha Zarrouki, Alessio Gambi, Jan Frederik Totz, Kai Storms, Steven Peters, Andrea Stocco, Bassam Alrifaee, Marco Pavone, and Johannes Betz. 2026. Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis. IEEE Open Journal of Intelligent Transportation Systems (2026). https://arxiv.org/abs/2506.11526
- [78]Y. Gao, Z. Xu, X. Zhao, G. Wang, and Q. Yuan. 2020. Hardware-in-the-Loop Simulation Platform for Autonomous Vehicle AEB Prototyping and Validation. In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems, ITSC 2020. https://doi.org/10.1109/ITSC45102.2020.9294704
- [79]Z. Gao, J. Zhou, Q. Xue, and H. Shen. 2025. A Data-Centric Image Enhancement Framework for Reducing Domain Discrepancies in Autonomous Driving Safety Evaluation. In Proceedings - IEEE Global Communications Conference, GLOBECOM. 296–301. https://doi.org/10.1109/GLOBECOM56902.2025.11432011
- [80]Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. 2013. Vision meets robotics: The KITTI dataset, In International Journal of Robotics Research. International Journal of Robotics Research (IJRR).DOI
- [81]Ruben Grewal, Paolo Tonella, and Andrea Stocco. 2024. Predicting Safety Misbehaviours in Autonomous Driving Systems using Uncertainty Quantification. In Proceedings of 17th IEEE International Conference on Software Testing, Verification and Validation (ICST ’24). IEEE, 12 pages.arxiv.org/abs/2404.18573
- [82]Ruizhen Gu, José Miguel Rojas, and Donghwan Shin. 2025. Software testing for extended reality applications: a systematic mapping study. Automated Software Engineering 32, Article 56 (2025). https://doi.org/10.1007/s10515-025-00523-7
- [83]X. Gu, S. Wu, B. Li, H. Wang, X. Jiao, Z. Wang, W. Lu, W. Yu, L. Chen, J. Hu, and S. Zhao. 2023. Copula-Based Interactive Behavior Modeling for Naturalistic Test Scenarios Generation. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC. 5940–5947. https://doi.org/10.1109/ITSC57777.2023.10421919
- [84]Chunshi Guo, Renaud Deborne, Vincent Ravaud, Arnaud Franjou, Romain Rocca, Elodie Bouillot, Hakim Molleheli, Zhou Fang, Didier Wautier, and Andras Kemeny. 2021. A Seamless Model-In-the-Loop Simulation Framework for the Design and Testing of ADAS and AD Software. In Proceedings of the Driving Simulation Conference 2021 Europe VR (2021-09-14), Andras Kemeny, Jean-Rémy Chardonnet, and Florent Clombet (Eds.). Driving Simulation Association, Munich, Germany, 17–24.
- [85]Z. Guo, X. Gao, J. Zhou, X. Cai, X. Yang, L. Wen, and X. Sun. 2025. SceneDM: Consistent Diffusion Models for Coherent Multi-agent Trajectory Generation. In IEEE International Conference on Automation Science and Engineering. 954–960. https://doi.org/10.1109/CASE58245.2025.11163744
- [86]P. Gupta, R. Wang, T. R. J. H. Han, D. Karbowski, A. Vahidi, and Y. Jia. 2023. An X-in-the-Loop (XIL) Testing Framework for Validation of Connected and Autonomous Vehicles. In IAVVC 2023 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC57316.2023.10328040
- [87]R. Gutiérrez-Moreno, R. Barea, E. López-Guillén, F. Arango, P. Revenga, and L.M. Bergasa. 2024. Decision Making for Autonomous Driving Stack: Shortening the Gap from Simulation to Real-World Implementations. In IEEE Intelligent Vehicles Symposium, Proceedings. 3107–3113. https://doi.org/10.1109/IV55156.2024.10588560
- [88]H. Haghighi, Y. Wang, M. Dianati, K. Debattista, and V. Donzella. 2026. A Unified Generative Framework for Realistic LiDAR Simulation in Autonomous Driving Systems. IEEE Sensors Journal 26, 3 (2026), 5013–5025. https://doi.org/10.1109/JSEN.2025.3642208
- [89]A. Haider, A. Eryildirim, M. Pigniczki, L. Haas, B. Schlager, T. Zeh, D. Nickel, and A.W. Koch. 2025. Modeling and Simulation of Automotive FMCW RADAR Sensor for Environmental Perception. IEEE Open Journal of Intelligent Transportation Systems 6 (2025), 433–455. https://doi.org/10.1109/OJITS.2025.3554452
- [90]A. Haider, L. Haas, A. Eryildirim, T. Zeh, S. Koyama, N.H. Mullner, M. Jakobi, and A.W. Koch. 2024. A Methodology for Verifying and Validating the Functional Performance Evaluation of High-Fidelity RADAR Sensor Models at Raw Data and Detection Levels. In 2024 IEEE 8th International Conference on Vision, Image and Signal Processing, ICVISP 2024. https://doi.org/10.1109/ICVISP64524.2024.10959585
- [91]Fitash Ul Haq, Donghwan Shin, Shiva Nejati, and Lionel Briand. 2020. Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study. In Proceedings of 13th IEEE International Conference on Software Testing, Verification and Validation (ICST ’20). IEEE.DOI
- [92]Fitash Ul Haq, Donghwan Shin, Shiva Nejati, and Lionel Briand. 2021. Can offline testing of deep neural networks replace their online testing? a case study of automated driving systems. Empirical Software Engineering 26, 5 (2021), 90. https://doi.org/10.1007/s10664-021-09982-4
- [93]F. Harhammer, A.T. Müller, P. Graf, and N.C. Stache. 2025. Sim-to-Real Domain Adaptation with Randomized Data for Autonomous Driving. In IAVVC 2025 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC61942.2025.12119640
- [94]S. Hasirlioglu and A. Riener. 2018. A Model-Based Approach to Simulate Rain Effects on Automotive Surround Sensor Data. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 2018-November. 2609–2615. https://doi.org/10.1109/ITSC.2018.8569907
- [95]J.E. Heikkinen, S. Gafurow, S. Kopylov, T. Minav, S. Grebennikov, and A. Kurbanov. 2019. Hardware-in-the-loop platform for testing autonomous vehicle control algorithms. In Proceedings - International Conference on Developments in eSystems Engineering, DeSE, Vol. October-2019. 906–911. https://doi.org/10.1109/DeSE.2019.00168
- [96]G. Hess, C. Lindstrom, M. Fatemi, C. Pettersson, and L. Svensson. 2025. SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 11982–11992. https://doi.org/10.1109/CVPR52734.2025.01119
- [97]Carl Hildebrandt and Sebastian Althoff. 2021. World-in-the-Loop Simulation for Autonomous Systems Validation. In 2021 IEEE International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/ICRA48506.2021.9561240
- [98]Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A. Efros, and Trevor Darrell. 2017. CyCADA: Cycle-Consistent Adversarial Domain Adaptation. CoRR abs/1711.03213 (2017). arXiv:1711.03213arxiv.org/abs/1711.03213
- [99]K. Hofrichter, L. Elster, C. Linnhoff, T. Ruppert, and S. Peters. 2025. Introducing a Development Method for Active Perception Sensor Simulations Using Continuous Verification and Validation. Sensors 25, 24 (2025). https://doi.org/10.3390/s25247642
- [100]Z. Hong. 2025. Effective Learning Mechanism Based on Reward-Oriented Hierarchies for Sim-to-Real Adaption in Autonomous Driving Systems. IEEE Transactions on Intelligent Transportation Systems 26, 3 (2025), 3527–3542. https://doi.org/10.1109/TITS.2024.3254882
- [101]M. Hoppe, J.C. Kirchhoff, E. Kusmenko, C.Y. Lee, and B. Rumpe. 2022. Embedded Autonomous Vehicle Simulation with Hardware Emulation in the Loop. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2022-June. 16–21. https://doi.org/10.1109/IV51971.2022.9827215
- [102]H. Hsiang, K.-C. Chen, and Y.-Y. Chen. 2022. Development of Simulation-Based Testing Scenario Generator for Robustness Verification of Autonomous Vehicles. In Proceedings of the 2022 5th International Conference on Advanced Systems and Emergent Technologies, IC_ASET 2022. 210–215. https://doi.org/10.1109/IC_ASET53395.2022.9765910
- [103]Xuemín Hu, Shen Li, Tingyu Huang, Bo Tang, and Long Chen. 2023. Sim2real and Digital Twins in Autonomous Driving: A Survey. arXiv preprint arXiv:2305.01263 (2023).
- [104]Y. Hu and D. Zhao. 2025. A simulation-based testing framework for autonomous driving: ensuring realism and priority of test scenarios. Journal of Intelligent Transportation Systems: Technology, Planning, and Operations 29, 6 (2025), 654–673. https://doi.org/10.1080/15472450.2025.2497510
- [105]P. Huang, W. Ding, B. Stoler, J. Francis, B. Chen, and D. Zhao. 2025. CADRE: Controllable and Diverse Generalization of Safety-Critical Driving Scenarios Using Real-World Trajectories. In Proceedings - IEEE International Conference on Robotics and Automation. 5474–5481. https://doi.org/10.1109/ICRA55743.2025.11127319
- [106]T. Huang, S. Song, H. Hu, Y. Gao, G. Shao, and Q. Zhu. 2024. A Hierarchical LiDAR Simulation Framework Incorporating Physical Attenuation Response in Autonomous Driving Scenarios. IEEE Transactions on Intelligent Transportation Systems 25, 6 (2024), 6200–6211. https://doi.org/10.1109/TITS.2023.3340676
- [107]Z. Huang, Z. Zhang, A. Vaidya, Y. Chen, J. Fernandez Fisac, and C. Lv. 2026. Versatile Behavior Diffusion for Generalized Traffic Agent Simulation. IEEE Transactions on Intelligent Transportation Systems (2026). https://doi.org/10.1109/TITS.2026.3662886
- [108]S. Huch, L. Scalera ndi, E. Rivera, and M. Lienkamp. 2023. Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level. IEEE Transactions on Intelligent Vehicles 8, 4 (2023), 2970–2982. https://doi.org/10.1109/TIV.2023.3251650
- [109]Y.-S. Hwang, H.-S. Yun, M.-S. Lee, S.-M. Ha, Y.-K. Yoon, and K.-S. Yun. 2024. Implementation of a Simulation-Based Evaluation Environment for Autonomous Driving Service Verification. In International Conference on ICT Convergence. 570–573. https://doi.org/10.1109/ICTC60822.2024.10827399
- [110]International Organization for Standardization. 2022. Road vehicles—Safety of the intended functionality. Technical Report ISO 21448:2022. International Organization for Standardization, Geneva, Switzerland. https://www.iso.org/standard/77490.html
- [111]Tech. Rep. ISO/PAS 21448:2019 International Organization for Standardization. 2019. Road Vehicles - Safety of the Intended Functionality.
- [112]S.B. Jayapal Gowdu, P. Aust, A. Schwind, F. Hau, and M.A. Hein. 2022. Evaluation of scenario-based automotive radar testing in virtual environment using real driving data. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 2022-October. 2379–2384. https://doi.org/10.1109/ITSC55140.2022.9922366
- [113]B. Jekl, Z. Deutsch, M. Kopasz, E. Tóth, Á. Fehér, D. Doba, B. Németh, S. Aradi, T. Tettamanti, V.V. Tan, and P. Gáspár. 2025. Speed planner implementation for autonomous vehicles in an urban context using mixed reality. European Transport Research Review 17, 1 (2025). https://doi.org/10.1186/s12544-025-00754-5
- [114]S. Jiang, Y. Wang, W. Lin, Y. Cao, L. Lin, J. Miao, and Q. Luo. 2021. A High-accuracy Framework for Vehicle Dynamic Modeling in Autonomous Driving. In IEEE International Conference on Intelligent Robots and Systems. 6680–6687. https://doi.org/10.1109/IROS51168.2021.9636861
- [115]S. Jing, Y. Zhao, X. Zhao, F. Hui, and A.J. Khattak. 2025. An Efficient High-Risk Lane-Changing Scenario Edge Cases Generation Method for Autonomous Vehicle Safety Testing. IEEE Transactions on Intelligent Vehicles 10, 1 (2025), 459–471. https://doi.org/10.1109/TIV.2024.3414828
- [116]S. Jung, Y.-S. Lee, and K.-T. Lee. 2025. Real-Virtual Synchronized Localization in Autonomous Vehicle Testbeds Using SLAM and Virtual Simulation. In Proceedings of the 18th IEEE/ACM International Conference on Utility and Cloud Computing, UCC 2025. https://doi.org/10.1145/3773274.3774659
- [117]Nidhi Kalra and Susan M. Paddock. 2016. Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? RAND Corporation, Santa Monica, CA. https://doi.org/10.7249/RR1478
- [118]Prabhjot Kaur, Samira Taghavi, Zhaofeng Tian, and Weisong Shi. 2021. A Survey on Simulators for Testing Self-Driving Cars. CoRR abs/2101.05337 (2021). arXiv:2101.05337 https://arxiv.org/abs/2101.05337
- [119]M. Kettelgerdes and G. Elger. 2023. Fine-Field Measurement and Methodology for Modeling and Validation of Precipitation Effects on Solid-State LiDAR Sensors. IEEE Journal of Radio Frequency Identification 7 (2023), 192–202. https://doi.org/10.1109/JRFID.2023.3234999
- [120]R. Khraire, H. Guo, H. Gangwar, N.K. Saravanan, J. Wishart, and J. Zhao. 2025. Augmented Reality-Enhanced Digital Twin for Automated Vehicle Testing. In IFAC-PapersOnLine, Vol. 59. 174–179. https://doi.org/10.1016/j.ifacol.2025.07.030
- [121]E. Kim, J. Shenoy, S. Junges, D.J. Fremont, A. Sangiovanni-Vincentelli, and S.A. Seshia. 2022. Querying Labelled Data with Scenario Programs for Sim-to-Real Validation. In Proceedings - 13th ACM/IEEE International Conference on Cyber-Physical Systems, ICCPS 2022. 34–45. https://doi.org/10.1109/ICCPS54341.2022.00014
- [122]J. Kim, H. Ko, I. Choi, S. Ji, and J. Jung. 2024. Designing a Naturalistic Driving Situation Extraction Method to Improve the Learning Process of Automated Driving Technology. In International Conference on Electrical, Computer, and Energy Technologies, ICECET 2024. https://doi.org/10.1109/ICECET61845.2024.10698152
- [123]S. Kim, G. Kim, T. Kim, C. Jeong, and C.M. Kang. 2025. Autonomous Vehicle Control Using CARLA Simulator, ROS, and EPS HILS. In 2025 International Conference on Electronics, Information, and Communication, ICEIC 2025. https://doi.org/10.1109/ICEIC64792.2025.10879635
- [124]B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A. Al Sallab, Senthil Yogamani, and Patrick Pérez. 2021. Deep Reinforcement Learning for Autonomous Driving: A Survey. arXiv:2002.00444 [cs.LG]arxiv.org/abs/2002.00444
- [125]B. Kitchenham and S. Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering.
- [126]N. Kolb, F. Huber, and A. Pretschner. 2025. Automatically Improving Scenario Descriptions Derived From Recorded Traffic. IEEE Transactions on Intelligent Vehicles 10, 12 (2025), 5126–5135. https://doi.org/10.1109/TIV.2024.3514296
- [127]Phillip Kooman. 2024. Simulating for Safety for the Evaluation of Autonomous Products.
- [128]Town Kramer, Ed Murphy, Chris Anderson, Rahul Ravikumar, Adam Conway, et al. 2023. Donkeycar: an open source hardware and software platform to build a small scale self driving car. https://www.donkeycar.com/. Accessed: 2023-11-23.
- [129]R. Krecht, M. Unger, and A. Ballagli. 2023. Validation Process of the Computer Simulation of a Test-Purpose Self-Driving Vehicle. In IAVVC 2023 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC57316.2023.10328107
- [130]H. Kuroda, Y. Amano, M. Mizukoshi, N. Hiruma, K. Sato, and H. Inoue. 2025. Analysis of Snow Properties in Optical and Millimeter Wave for High-Fidelity Sensor Modelling. In IAVVC 2025 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC61942.2025.11219532
- [131]K. D. Kusano, K. Beatty, S. Schnelle, F. Favaro, C. Crary, and T. Victor. 2026. Collision Avoidance Testing of the Waymo Automated Driving System. Waymo Research publication page. https://www.waymo.com/research/collision-avoidance-testing-of-the-waymo-automated/ Publication details limited to metadata verified on the research page.arxiv.org/abs/2212.08148
- [132]S.C. Lambertenghi, M. Flores Valdez, and A. Stocco. 2025. A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems. In Proceedings - 2025 40th IEEE/ACM International Conference on Automated Software Engineering, ASE 2025. 2808–2820. https://doi.org/10.1109/ASE63991.2025.00230
- [133]S.C. Lambertenghi and A. Stocco. 2024. Assessing Quality Metrics for Neural Reality Gap Input Mitigation in Autonomous Driving Testing. In Proceedings - 2024 IEEE Conference on Software Testing, Verification and Validation, ICST 2024. 173–184. https://doi.org/10.1109/ICST60714.2024.00024
- [134]Stefano Carlo Lambertenghi, Hannes Leonhard, and Andrea Stocco. 2025. Benchmarking Image Perturbations for Testing Automated Driving Assistance Systems. In Proceedings of the 18th IEEE International Conference on Software Testing, Verification and Validation (ICST ’25). IEEE, 12 pages.arxiv.org/abs/2501.12269
- [135]T. Lan, N. Schade, and J. Pannek. 2024. Assessing Reliability of Virtual Testing for Deep Learning Models in Autonomous Vehicles. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC. 537–542. https://doi.org/10.1109/ITSC58415.2024.1029065
- [136]M. Langer, M. Harth, L. Preitschaft, R. Kates, and K. Bogenberger. 2021. Calibration and Assessment of Urban Microscopic Traffic Simulation as an Environment for Testing of Automated Driving. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 2021-September. 3210–3216. https://doi.org/10.1109/ITSC48978.2021.9564743
- [137]S. Larter, R. Queiroz, S. Edwards, A. Sarkar, and K. Czarnecki. 2022. A Hierarchical Pedestrian Behavior Model to Generate Realistic Human Behavior in Traffic Simulation. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2022-June. 533–541. https://doi.org/10.1109/IV51971.2022.9827035
- [138]S. Lee, T. Oh, S. Han, J. Won, and J. Yoo. 2025. Model-Centric Quantitative Methodology for Evaluating Simulation Reliability in Autonomous-Vehicle Testing. IEEE Access 13 (2025), 182238–182256. https://doi.org/10.1109/ACCESS.2025.3622162
- [139]Y.-H. Lee and J.-H. Won. 2024. Study on Full Scale Testbench Based Vehicle-In-The-Loop Simulation Testing for Automated Vehicles. In IAVVC 2024 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC63304.2024.10786476
- [140]Johnathan Leung, Guansen Tong, Parasara Sridhar Duggirala, and Praneeth Chakravarthula. 2025. From Road to Code: Neuro-Symbolic Program Synthesis for Autonomous Driving Scene Translation and Analysis. In Proceedings of the International Conference on Neuro-symbolic Systems (Proceedings of Machine Learning Research, Vol. 288), George Pappas, Pradeep Ravikumar, and Sanjit A. Seshia (Eds.). PMLR, 331–351. https:// Manuscript submitted to ACM //proceedings.mlr.press/v288/leung25a.html
- [141]B. Li, H. Wang, Y. Wang, P.H. Chan, D. Perks, and V. Donzella. 2025. Data-Driven Headlight Flare Model for Automotive Cameras. IEEE Open journal of Vehicular Technology 6 (2025), 2709–2720. https://doi.org/10.1109/OJVT.2025.3601400
- [142]C. Li, X. Wang, Z. Ye, J. Ye, S. Chen, and S. Zhao. 2025. Generative Reconstruction Model for Trajectory Anomaly Detection and Virtual Traffic Flow Authenticity Evaluation. In Proceedings of the International Joint Conference on Neural Networks. https://doi.org/10.1109/IJCNN64981.2025.11228692
- [143]H. Li, N. Bamminger, L. Wan, and A. Eichberger. 2024. Multi-Level and Metrics Evaluation Approach for Data-Driven Based Sensor Models. Automotive Innovation 7, 2 (2024), 248–257. https://doi.org/10.1007/s42154-023-00275-8
- [144]H. Li, C. Wu, M. Yuan, Y. Zhang, C. Zhao, C. Song, H. Feng, E. Ding, D. Zhang, and J. Wang. 2025. XLD: A Cross-Lane Dataset for Benchmarking Novel Driving View Synthesis. In Proceedings - 2025 International Conference on 3D Vision, 3DV 2025. 659–669. https://doi.org/10.1109/3DV66043.2025.00066
- [145]T. Li, B. Carstens, and X. Liu-Henke. 2025. Autonomous Driving Test Scenario Generation and Integration Using UAV-Based Visual Data. In ISSE 2025 - 11th IEEE International Symposium on Systems Engineering, Symposium Proceedings. https://doi.org/10.1109/ISSE65546.2025.11369991
- [146]X. Li, J. Wei, H. Wang, L. Dong, R. Chen, C. Yi, J. Cai, D. Niyato, and X. Shen. 2025. Toward Intelligent Transportation With Pedestrians and Vehicles In-the-Loop: A Surveillance Video-Assisted Federated Digital Twin Framework. IEEE Network 39, 6 (2025), 305–315. https://doi.org/10.1109/MNET.2025.3574228
- [147]Z. Li, C. Wu, L. Zhang, and J. Zhu. 2025. DGNR: Density-Guided Neural Point Rendering of Large Driving Scenes. IEEE Transactions on Automation Science and Engineering 22 (2025), 4394–4407. https://doi.org/10.1109/TASE.2024.3410891
- [148]Z. Liang, J. Wang, T. Zhang, and X. Yong. 2025. DTTF-Sim: A Digital Twin-Based Simulation System for Continuous Autonomous Driving Testing. Sensors 25, 11 (2025). https://doi.org/10.3390/s25113447
- [149]N. Lichtle, E. Vinitsky, M. Nice, R. Bhadani, M. Bunting, F. Wu, B. Piccoli, B. Seibold, D.B. Work, J.W. Lee, J. Sprinkle, and A.M. Bayen. 2024. From Sim to Real: A Pipeline for Training and Deploying Traffic Smoothing Cruise Controllers. IEEE Transactions on Robotics 40 (2024), 4490–4505. https://doi.org/10.1109/TRO.2024.3463407
- [150]X. Liu, S. Li, X. Luo, H. Huang, J. Bian, and R. Zhou. 2026. Enhancing Autonomous Vehicle Testing Through High-Risk Powered Two-Wheeler Trajectory Generation Using a Conditional Denoising Diffusion Probabilistic Model. IEEE Internet of Things Journal 13, 8 (2026), 16466–16479. https://doi.org/10.1109/JIOT.2026.3659671
- [151]Y. Liu, Y. Yan, Y. Fang, and H. Min. 2024. Intelligent Co-Simulation Framework Combined with Real Test Fields for Collaborative Driving. In Proceedings of the 2024 8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024. https://doi.org/10.1109/CVCI63518.2024.10830232
- [152]Y. Liu, K. Zhang, and Z. Li. 2021. Application of digital twin and parallel system in automated driving testing. In Proceedings 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence, DTPI 2021. 123–126. https://doi.org/10.1109/DTPI52967.2021.9540143
- [153]T. Long, M. Zong, C. Zhang, Y. Liu, S. Ma, and L. Li. 2024. Generating Realistic VRU-AV Scenarios via Social Force-Based Gradient Optimization. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC. 4216–4221. https://doi.org/10.1109/ITSC58415.2024.10919513
- [154]Z. Lu, Q. Kongjian, and Z. Qing. 2022. Design of Vehicle in the Loop Test System Based on L3 Autopilot. In MCECE 2022 - IEEE 5th Advanced Information Management, Communications, Electronic and Automation Control Conference. 727–733. https://doi.org/10.1109/MCECE5388.2022.10020004
- [155]Rachel Luo, Heng Yang, Michael Watson, Apoorva Sharma, Sushant Veer, Edward Schmerling, and Marco Pavone. 2025. SimVal: Leveraging Correlation Across Test Platforms for Variance-Reduced Metric Estimation. Conference on Robot Learning (CoRL), arXiv:2506.20553. https://doi.org/10.48550/arXiv.2506.20553
- [156]Z.F. Magosi and A. Eichberger. 2023. A Novel Approach for Simulation of Automotive Radar Sensors Designed for Systematic Support of Vehicle Development. Sensors 23, 6 (2023). https://doi.org/10.3390/s23063227
- [157]Z.F. Magosi, C. Wellerhaus, V.R. Tihanyi, P. Luley, and A. Eichberger. 2022. Evaluation Methodology for Physical Radar Perception Sensor Models Based on On-Road Measurements for the Testing and Validation of Automated Driving. Energies 15, 7 (2022). https://doi.org/10.3390/en15072545
- [158]S. Manivasagam, I.A. Barsan, J. Wang, Z. Yang, and R. Urtasun. 2023. Towards Zero Domain Gap: A Comprehensive Study of Realistic LiDAR Simulation for Autonomous Testing. In Proceedings of the IEEE International Conference on Computer Vision. 8238–8248. https://doi.org/10.1109/ICCV51070.2023.00760
- [159]S. Manivasagam, S. Wang, K. Wong, W. Zeng, M. Sazanovich, S. Tan, B. Yang, W-C. Ma, and R. Urtasun. 2020. LiDARsim: Realistic LiDAR simulation by leveraging the real world. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 11164–11173. https://doi.org/10.1109/CVPR42600.2020.01118
- [160]S. Martín Serrano, R. Izquierdo, I. García Daza, M.Á. Sotelo, and D. Fernández-Llorca. 2025. Behavioural Gap Assessment of Human-Vehicle Interaction in Real and Virtual Reality-Based Scenarios in Autonomous Driving. International Journal of Human-Computer Interaction 41, 11 (2025), 6879–6892. https://doi.org/10.1080/10447318.2024.2385514
- [161]Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.arxiv.org/abs/2003.08934
- [162]H. Naing, W. Cai, J. Yu, J. Zhong, and L. Yu. 2025. Fine-Grained Trajectory Reconstruction by Microscopic Traffic Simulation With Dynamic Data-Driven Evolutionary Optimization. IEEE Transactions on Intelligent Transportation Systems 26, 2 (2025), 1930–1950. https://doi.org/10.1109/TITS.2024.3502213
- [163]D. Nalic, A. Eichberger, G. Hanzl, M. Fellendorf, and B. Rogic. 2019. Development of a Co-Simulation Framework for Systematic Generation of Scenarios for Testing and Validation of Automated Driving Systems. In 2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019. Manuscript submitted to ACM 1895–1901. https://doi.org/10.1109/ITSC.2019.8916839
- [164]Molla Mohammad Abid Naziri, Stefano Carlo Lambertenghi, Andrea Stocco, and Marcelo d’Amorim. 2026. Misbehaviour Forecasting for Focused Autonomous Driving Systems Testing. In Proceedings of the 48th International Conference on Software Engineering (ICSE ’26). ACM/IEEE.arxiv.org/abs/2512.18823
- [165]N. Neis, J. Ziehn, M. Roschani, and J. Beyerer. 2025. Data-Driven Lane Change Modeling for Automated Driving Function Validation. In Proceedings of the 2025 IEEE International Conference on Vehicular Electronics and Safety, ICVES 2025. 147–154. https://doi.org/10.1109/ICVES65691.2025.11376528
- [166]A. Ngo, M.P. Bauer, and M. Resch. 2021. A Multi-Layered Approach for Measuring the Simulation-to-Reality Gap for Radar Perception for Autonomous Driving. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 2021-September. 4008–4014. https://doi.org/10.1109/ITSC48978.2021.9564521
- [167]Anthony Ngo, Max Paul Bauer, and Michael M. Resch. 2021. Deep Evaluation Metric: Learning to Evaluate Simulated Radar Point Clouds for Virtual Testing of Autonomous Driving. 2021 IEEE Radar Conference (RadarConf21) (2021), 1–6. https://doi.org/DOI:10.1109/RadarConf2147009.2021.9455235
- [168]I. Nosheen, M.A. Farooq, P. Corcoran, C. Ennis, and M.G. Madden. 2025. Enhancing Synthetic Image Realism with Controlled Diffusion Models. In Proceedings of the International Joint Conference on Neural Networks. https://doi.org/10.1109/IJCNN64981.2025.11228158
- [169]NVIDIA. 2026. AlpaSim: A Modular, Lightweight, and Data-Driven Research Simulator for Autonomous Driving. Open-source software repository. https://github.com/NVlabs/alpasim Accessed 20 October 2026.
- [170]NVIDIA. [n. d.]. Making Halos SAFE Credible and Auditable. Technical documentation. https://docs.nvidia.com/halos-safe/technical-brief/making-halos-safe-credible-and-auditable Accessed 20 October 2026.
- [171]NVIDIA. [n. d.]. Our Architecture: NVIDIA Halos SAFE Technical Brief. Technical documentation. https://docs.nvidia.com/halos-safe/technical-brief/our-architecture Accessed 20 October 2026.
- [172]T. Oh, S. Cho, and J. Yoo. 2024. A Reliability Evaluation Methodology for X-in-the-Loop Simulation in Autonomous Vehicle Systems. IEEE Access 12 (2024), 193622–193640. https://doi.org/10.1109/ACCESS.2024.3519713
- [173]T. Oh, Y. Ha, D. Yoo, and J. Yoo. 2024. AD-VILS: Implementation and Reliability Validation of Vehicle-in-the-Loop Simulation Platform for Evaluating Autonomous Driving Systems. IEEE Access 12 (2024), 164190–164209. https://doi.org/10.1109/ACCESS.2024.3492162
- [174]Matthew O’Kelly, Hongrui Zheng, Dhruv Karthik, and Rahul Mangharam. 2020. FiTENH: An Open-source Evaluation Environment for Continuous Control and Reinforcement Learning. In Proceedings of the NeurIPS 2019 Competition and Demonstration Track (Proceedings of Machine Learning Research, Vol. 123). PMLR, 77–89. https://proceedings.mlr.press/v123/o-kelly20a.html
- [175]D. Oladimeji, C. Femi-Adeyinka, J. Dutta, and M.K. An. 2024. Enhanced Approaches to Similarity Analysis on Car Accident Reconstruction. In 2024 IEEE 5th World AI IoT Congress, AIoT 2024. 84–90. https://doi.org/10.1109/AIoT61789.2024.10578996
- [176]Olek Osikowicz, Phil McMinn, Wei Xing, and Donghwan Shin. 2026. Multi-Fidelity Bayesian Optimization for Simulation Based Autonomous Driving Systems Testing. In 2026 IEEE Intelligent Vehicles Symposium (IV). 187–194. https://doi.org/10.1109/IV66570.2026.11623850
- [177]L.M. Otto, M. Kaiser, D. Seebacher, and S. Müller. 2025. Validation of Ai-Based 3D Human Pose Estimation in a Cyber-Physical Environment. In IAVVC 2025 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC61942.2025.11219441
- [178]J. Pahk, S. Park, J. Shim, S. Son, J. Lee, J. An, Y. Lim, and G. Choi. 2024. Lane Segmentation Data Augmentation for Heavy Rain Sensor Blockage Using Realistically Translated Raindrop Images and CARLA Simulator. IEEE Robotics and Automation Letters 9, 6 (2024), 5488–5495. https://doi.org/10.1109/LRA.2024.3390587
- [179]J. Pahk, J. Shim, M. Baek, Y. Lim, and G. Choi. 2023. Effects of Sim2Real Image Translation via DCGAN on Lane Keeping Assist System in CARLA Simulator. IEEE Access 11 (2023), 33915–33927. https://doi.org/10.1109/ACCESS.2023.3262991
- [180]C. Park, S. Chung, and H. Lee. 2020. Vehicle-in-the-loop in global coordinates for advanced driver assistance system. Applied Sciences (Switzerland) 10, 8 (2020). https://doi.org/10.3390/app10082645
- [181]A. Parra, A.J. Rodriguez, A. Zubizarreta, and J. Perez. 2020. Validation of a Real-Time Capable Multibody Vehicle Dynamics Formulation for Automotive Testing Frameworks Based on Simulation. IEEE Access 8 (2020), 213253–213265. https://doi.org/10.1109/ACCESS.2020.3040232
- [182]S. Pasios and N. Nikolaidis. 2025. Enhancing Photorealism in CARLA Autonomous Driving Simulator. In European Signal Processing Conference. 671–675. https://doi.org/10.23919/EUSIPCO63237.2025.11226820
- [183]J. Peintner, M. Funk Drechsler, F. Reway, G. Seifert, W. Huber, and A. Riener. 2021. Mixed Reality Environment for Complex Scenario Testing. In ACM International Conference Proceedings Series. 605–608. https://doi.org/10.1145/3473856.3474034
- [184]C. Peng and Z. Wang. 2025. Method and Applications of Solid-State Lidar Modeling for X-in-the-Loop Testing of Autonomous Vehicles. In MM 2025 - Proceedings of the 33rd ACM International Conference on Multimedia, Co-Located with MM 2025. 8135–8143. https://doi.org/10.1145/3746027.3755321
- [185]Kai Petersen, Robert W. G. Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology 64 (2015), 1–18. https://doi.org/10.1016/j.infsof.2015.03.007
- [186]U. Petrovic and N. Muhammad. 2025. On the Suitability of Small-Sized Vehicles for Autonomous-Driving Research. In International Conference for Artificial Intelligence: Applications, Innovation and Ethics, AI2E 2025. https://doi.org/10.1109/AI2E64943.2025.10982988
- [187]Roberto Pietrantuono, Luca Giammatei, Stefano Russo, Julien Siebert, and Neil Walkinshaw. 2026. Causal Software Engineering: A Vision and Roadmap. In Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering (Montreal, QC, Canada) (FSE Companion ’26). Association for Computing Machinery, New York, NY, USA, 1346–1350. https://doi.org/10.1145/3803437.3805585
- [188]R. Queiroz, D. Sharma, R. Caldas, K. Czarnecki, S. Garcia, T. Berger, and P. Pelliccione. 2024. A Driver-Vehicle Model for ADS Scenario-Based Testing. IEEE Transactions on Intelligent Transportation Systems 25, 8 (2024), 8641–8654. https://doi.org/10.1109/TITS.2024.3373531
- [189]J. Reckenzaun, R. Rott, O. Nikolic, P. Innerwinkler, M. Kirchengast, D.J. Ritter, S. Solmaz, S. Ladstatter, P. Luley, B. Toth, A.M. Hee, L. Hormann, A. Potsch, M. Rothacher, A. Rovid, Z. Vincze, and M. Csontho. 2024. Transnational Testing, Operation and Certification of Automated Driving Systems - Building bricks for the NATM multi-pillar framework. In IAVVC 2024 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC63304.2024.10786414
- [190]Jan Reich, Daniel Hillen, Joshua Frey, Nishanth Laxman, Takehito Ogata, Donato Di Paola, Satoshi Otsuka, and Natsumi Watanabe. 2023. Concept and metamodel to support cross-domain safety analysis for ODD expansion of autonomous systems. In International Conference on Computer Safety, Reliability, and Security. Springer, 165–178.
- [191]Felix Reisgys, Mohamed Elgharbayy, Andreas Schwarzhaupt, and Eric Sax. 2021. Argumentation on ADAS Simulation Validity using Aleatory and Epistemic Uncertainty Estimation. In Proceedings of the Driving Simulation Conference 2021 Europe VR (2021-09-14), Andras Kemeny, Jean-Rémy Chardonnet, and Florent Colombet (Eds.). Driving Simulation Association, Munich, Germany, 25–32.
- [192]Replication package 2026. Replication package. https://github.com/ast-fortiss-tum/The_Reality_Gap_in_Automated_Driving_Systems_Testing.
- [193]F. Reway, M. Drechsler, R. Murthy, Y. Poledna, W. Huber, and C. Icking. 2022. Simulation-based test methods with an automotive camera-in-the-loop for automated driving algorithms. In International Conference on Electrical, Computer, Communications and Mechatronics Engineering, ICECCME 2022. https://doi.org/10.1109/ICECCME55909.2022.9988437
- [194]F. Reway, A. Hoffmann, D. Wachtel, W. Huber, A. Knoll, and E. Ribeiro. 2020. Test Method for Measuring the Simulation-to-Reality Gap of Camera-based Object Detection Algorithms for Autonomous Driving. In IEEE Intelligent Vehicles Symposium, Proceedings. 1249–1256. https://doi.org/10.1109/IV47402.2020.9304567
- [195]M.Y. Rezaldi, A.F. Rozie, D. Krisnandi, and A.H.A.M. Siagian. 2021. 3D Orthomosaic from Unmanned Aerial Vehicle (UAV) Image Data for Real-World Environment 3D Map Creation Autonomous Vehicles Simulation System. In ACM International Conference Proceeding Series. 16–20. https://doi.org/10.1145/3489088.3489129
- [196]Vincenzo Riccio, Gunel Jahangirova, Andrea Stocco, Nargiz Humbatova, Michael Weiss, and Paolo Tonella. 2020. Testing machine learning based systems: a systematic mapping. Empirical Software Engineering 25 (2020), 5193–5254.
- [197]Robert Bosch. 2025. Advances in Credibility of Modeling and Simulation: A Generic Framework Approach at Bosch. NAFEMS World Congress 2025, paper NWC25-0007201. https://www.nafems.org/publications/resource_center/nwc25-0007201-paper/ Publisher abstract; corporate attribution used for this record.
- [198]Guodong Rong, Byung Hyun Shin, Hadi Tabatabaee, Qiang Lu, Steve Lemke, Märtīns Možeiko, Eric Boise, Geehoon Uhm, Mark Gerow, Shalin Mehta, et al. 2020. A High Fidelity Simulator for Autonomous Driving. arXiv preprint arXiv:2005.03778 (2020).arxiv.org/abs/2005.03778
- [199]P. Rosenberger, M. Holder, S. Huch, H. Winner, T. Fleck, M.R. Zofka, J. Marius Zollner, T. D’Hondt, and B. Wassermann. 2019. Benchmarking and functional decomposition of automotive lidar sensor models. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2019-June. 632–639. https://doi.org/10.1109/IVS.2019.8814081
- [200]P. Rosenberger, M. Holder, M. Zirulnik, and H. Winner. 2018. Analysis of Real World Sensor Behavior for Rising Fidelity of Physically Based Lidar Sensor Models. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2018-June. 611–616. https://doi.org/10.1109/IVS.2018.8500511
- [201]Francisca Rosique, Pedro J. Navarro, Carlos Fernández, and Antonio Padilla. 2019. A Systematic Review of Perception System and Simulators for Autonomous Vehicles Research. Sensors 19, 3 (2019). https://doi.org/10.3390/s19030648
- [202]K. Saad, A. Yousif, and C.F. Elias. 2025. A Digital Twin Approach for Perception Development and Validation in Autonomous Vehicles. In IEEE Intelligent Vehicles Symposium, Proceedings. 2577–2584. https://doi.org/10.1109/IV64158.2025.11097489
- [203]Danial Safaei, Siddartha Khastgir, Mohsen Alirezaei, Jeroen Ploeg, Chih-Hong Cheng, Son Tong, and Xingyu Zhao. 2026. Quantifying Fidelity: A Decisive Feature Approach to Comparing Synthetic and Real Imagery. In 2026 IEEE Intelligent Vehicles Symposium (IV). 847–854. https://doi.org/10.1109/IV66570.2026.11624013
- [204]Safety First for Automated Driving consortium. 2019. Safety First for Automated Driving. Multi-company white paper. https://www.mercedes-benz.com/content/dam/brandhub/innovation/safety-first-for-automated-driving/safety-first-for-automated-driving-whitepaper_en.pdf
- [205]Simon Sagmeister, Panagiotis Kounatidis, Sven Gollbrisch, and Markus Lienkamp. 2026. Analyzing the Impact of Simulation Fidelity on the Evaluation of Autonomous Driving Motion Control. arXiv:2602.07984. https://arxiv.org/abs/2602.07984 Preprint deposited in 2026; related IEEE IV 2027 publication; doi:10.1109/IVS55156.2024.1058858.
- [206]K. Saleh, A. Abobakr, M. Attia, J. Iskander, D. Nahavandi, M. Hossny, and S. Nahvandi. 2019. Domain adaptation for vehicle detection from bird’s eye view lidar point cloud data. In Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019. 3235–3242. https://doi.org/10.1109/ICVCW.2019.00404
- [207]C. Samak, T. Samak, B. Li, and V. Krovi. 2026. Sim2Real Diffusion: Leveraging Foundation Vision Language Models for Adaptive Automated Driving. IEEE Robotics and Automation Letters 11, 1 (2026), 177–184. https://doi.org/10.1109/LRA.2025.3632723
- [208]T.V. Samak, C.V. Samak, and V.N. Krovi. 2024. Metaverse for Safer Roadways: An Immersive Digital Twin Framework for Exploring Human-Autonomy Coexistence in Urban Transportation Systems. In 2024 IEEE Conference on Telepresence, Telepresence 2024. 160–165. https://doi.org/10.1109/Telepresence63209.2024.10841549
- [209]T.V. Samak, C.V. Samak, and V.N. Krovi. 2024. Towards Validation of Autonomous Vehicles Across Scales using an Integrated Digital Twin Framework. In IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM. 1068–1075. https://doi.org/10.1109/AIM55361.2024.10637205
- [210]E. Schabus, S. Genser, and D. Watzenig. 2025. Object-Based Lidar Model for Enhanced Sensor Simulation in Automated Driving. In IAVVC 2025 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC61942.2025.11219463
- [211]P. Schulz, P. Zwick, J. Gamerdinger, S. Reiter, A. Viehl, and O. Bringmann. 2025. Virtual KITTI NextGen: Procedural Environment Reconstruction for Test Coverage Expansion of Connected and Autonomous Vehicles. In IAVVC 2025 - IEEE International Automated Vehicle Validation Conference, Proceedings. https://doi.org/10.1109/IAVVC61942.2025.11219632
- [212]C. Schyr, H. Inoue, and Y. Nakaoka. 2022. Vehicle-in-the-Loop Testing-a Comparative Study for Efficient Validation of ADAS/AD Functions. In ICCVE 2022 - IEEE International Conference on Connected Vehicles and Expo. https://doi.org/10.1109/ICCVE52871.2022.9742737
- [213]C. Segonne and P. Duthon. 2023. Qualification of the PAVIN Fog and Rain Platform and Its Digital Twin for the Evaluation of a Pedestrian Detector in Fog. Journal of Imaging 9, 10 (2023). https://doi.org/10.3390/jimaging9100211
- [214]Y. Shen, B. Chandaka, Z.-H. Lin, A. Zhai, H. Cui, D. Forsyth, and S. Wang. 2023. Sim-on-Wheels: Physical World in the Loop Simulation for Self-Driving. IEEE Robotics and Automation Letters 8, 12 (2023), 8912–8919. https://doi.org/10.1109/LRA.2023.3325689
- [215]Y. Shi, Z. Liu, Z. Wang, J. Ye, W. Tong, and Z. Liu. 2022. An Integrated Traffic and Vehicle Co-Simulation Testing Framework for Connected and Autonomous Vehicles. IEEE Intelligent Transportation Systems Magazine 14, 6 (2022), 26–40. https://doi.org/10.1109/MITS.2022.3188566
- [216]M.U. Shoukat, L. Yan, Y. Yan, F. Zhang, Y. Zhai, P. Han, S.A. Nawaz, M.A. Raza, M.W. Akbar, and A. Hussain. 2024. Autonomous driving test system under hybrid reality: The role of digital twin technology. Internet of Things (Netherlands) 27 (2024). https://doi.org/10.1016/j.iot.2024.101301
- [217]L. Shuguang, L. Zhonglin, W. Wenbo, Z. Yang, H. Jierui, and C. Hong. 2022. Vehicle-in-the-Loop Intelligent Connected Vehicle Simulation System Based on Vehicle-Road-Cloud Collaboration. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 2022-October. 1711–1716. https://doi.org/10.1109/ITSC55140.2022.9922190
- [218]Martin Sigl, Sebastian Wagner, Mohammad Bahram, Christoph Schütz, and Daniel Watzinger. 2021. Experiencing data-driven simulation. In Proceedings of the Driving Simulation Conference 2021 Europe VR (2021-09-14), Andras Kemeny, Jean-Rémy Chardonnet, and Florent Colombert (Eds.). Driving Simulation Association, Munich, Germany, 11–15.
- [219]G. Simakov, V. Stepanyants, and A. Martysheva. 2025. Effects of Simulator Fidelity on Automated Vehicle Object Perception Accuracy. In Proceedings - 2025 International Russian Smart Industry Conference, SmartIndustryCon 2025. 794–798. https://doi.org/10.1109/SmartIndustryCon65166.2025.10986291
- [220]Society of Automotive Engineers. 2021. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. SAE International. https://www.sae.org/standards/content/j3016_202104/ J3016_202104.
- [221]S. Solmaz, M. Rudigier, and M. Mischinger. 2020. A Vehicle-in-the-Loop Methodology for Evaluating Automated Driving Functions in Virtual Traffic. In IEEE Intelligent Vehicles Symposium, Proceedings. 1465–1471. https://doi.org/10.1109/IV47402.2020.9304811
- [222]Z. Song, Y. Li, X. Tao, W. Li, H. Ding, and D. Cao. 2026. Vision-language model-based scene understanding and decision-making for autonomous vehicles with a tailored augmented reality vehicle-in-the-loop testing platform. Transportation Research Part C: Emerging Technologies 186 (2026). https://doi.org/10.1016/j.trc.2026.105587
- [223]R.K. Soni and B.B. Nair. 2021. Deep Learning Based Approach to Generate Realistic Data for ADAS Applications. In 2021 5th International Conference on Computer, Communication, and Signal Processing, ICCCCSP 2021. 181–185. https://doi.org/10.1109/ICCCSP52374.2021.9465529
- [224]Lev Sorokin, Matteo Biagiola, and Andrea Stocco. 2026. Simulator ensembles for trustworthy autonomous driving systems testing. Empirical Software Engineering 31, 4 (2026), 80. https://doi.org/10.1007/s10664-026-10821-7
- [225]C. Stadler, R. German, and A. Djanatliev. 2022. Towards a Multivariate Scoring Metric for the Evaluation of Simulation Scenarios in Automated Driving. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 2022-October. 2508–2513. https://doi.org/10.1109/ITSC55140.2022.9922082
- [226]C. Stadler, F. Montanari, W. Baron, C. Sippl, and A. Djanatliev. 2022. A Credibility Assessment Approach for Scenario-Based Virtual Testing of Automated Driving Functions. IEEE Open Journal of Intelligent Transportation Systems 3 (2022), 45–60. https://doi.org/10.1109/OJITS.2022.3140493
- [227]Y. Stangherlin, A. Rofini, V. Sulli, V. Vannoucci, F. Valentini, E. Cinque, and M. Pratesi. 2025. The X-in-the-Loop Approach in P-CAR for Verification and Validation of Automated Vehicles. In IEEE Vehicular Technology Conference. https://doi.org/10.1109/VTC2025-Fall65116.2025.11310205
- [228]M. Steimle, V. Stange, M. Vollrath, and M. Maurer. 2025. Human&vehicle-in-the-loop test method in human factors research on automated vehicles: Conceptualization, realization, case-study, and lessons learned. Transportation Research Interdisciplinary Perspectives 29 (2025). https://doi.org/10.1016/j.trip.2024.101282
- [229]C. Steinhauser, P. Reis, H. Padusinski, J. Langer, and E. Sax. 2025. Data Quality Matters: Quantifying Image Quality Impact on Machine Learning Performance. In IEEE Intelligent Vehicles Symposium, Proceedings. 1564–1570. https://doi.org/10.1109/IV64158.2025.11097603
- [230]Andrea Stocco, Paulo J. Nunes, Marcelo d’Amorim, and Paolo Tonella. 2022. ThirdEye: Attention Maps for Safe Autonomous Driving Systems. In Proceedings of 37th IEEE/ACM International Conference on Automated Software Engineering (ASE ’22). IEEE/ACM. https://doi.org/10.1145/3551349.3556968
- [231]A. Stocco, B. Pulfer, and P. Tonella. 2023. Mind the Gap! A Study on the Transferability of Virtual Versus Physical-World Testing of Autonomous Driving Systems. IEEE Transactions on Software Engineering 49, 4 (2023), 1928–1940. https://doi.org/10.1109/TSE.2022.3202311
- [232]A. Stocco, B. Pulfer, and P. Tonella. 2023. Model vs system level testing of autonomous driving systems: a replication and extension study. Empirical Software Engineering 28, 3 (2023). https://doi.org/10.1007/s10664-023-10306-x
- [233]Andrea Stocco and Paolo Tonella. 2020. Towards Automated Defectors that Learn Continuously. In Proceedings of 31st International Symposium on Software Reliability Engineering Workshops (ISSREW 2020). IEEE.
- [234]Andrea Stocco and Paolo Tonella. 2021. Confidence-driven Weighted Retraining for Predicting Safety-Critical Failures in Autonomous Driving Systems. Journal of Software: Evolution and Process (2021). https://doi.org/10.1002/smr.2386
- [235]Andrea Stocco, Michael Weiss, Marco Calzana, and Paolo Tonella. 2020. Misbehaviour prediction for autonomous driving systems. In Proceedings of the ACM/IEEE 42nd international conference on software engineering. 359–371.arxiv.org/abs/1910.04443
- [236]Pei Sun, Henrik Kretschmar, Xerxes Dotiwalla, Aurelien Choudhary, Vijaysai Patnaik, Paul Tsui, James Guo, Yuan-Chen Chang, Vincent Anguelov, Dmitri Dolgov, et al. 2020. Scalability in perception for autonomous driving: Waymo open dataset. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020).arxiv.org/abs/1912.04838
- [237]S. Sun, Z. Gu, T. Sun, J. Cun, C. Yuan, Y. Han, D. Li, and M.H. Ang. 2024. DriveSceneGen: Generating Diverse and Realistic Driving Scenarios From Scratch. IEEE Robotics and Automation Letters 9, 8 (2024), 7007–7014. https://doi.org/10.1109/LRA.2024.3416792
- [238]T. Sun, C. Li, R. Yan, and Y. Cai. 2025. Testing Autonomous Driving Systems with Irregular Junctions Extracted from OpenStreetMap. In IEEE International Conference on Software Quality, Reliability and Security, QRS. 600–611. https://doi.org/10.1109/QRS65678.2025.00065
- [239]Yang Sun, Christopher M. Poskitt, Kun Wang, and Jun Sun. 2025. RiskyDIVER: Automatically Repairing Autonomous Vehicle Driving Behaviour for $0.08 per Violation. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). 1921–1933. https://doi.org/10.1109/ICSE55347.2025.00216
- [240]S. Suo, K. Wong, J. Xu, J. Tu, A. Cui, S. Casas, and R. Urtasun. 2023. MIXSIM: A Hierarchical Framework for Mixed Reality Traffic Simulation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2023-June. 9622–9631. https://doi.org/10.1109/CVPR52729.2023.00928
- [241]M. Szalai, B. Tettamanti, and V. Tihanyi. 2020. Mixed reality test environment for autonomous cars using Unity 3D and SUMO. In SAMI 2020 - IEEE 18th World Symposium on Applied Machine Intelligence and Informatics, Proceedings. 73–78. https://doi.org/10.1109/SAMI48414.2020.9108745
- [242]Z. Szalay, M. Szalai, B. Toth, T. Tettamanti, and V. Tihanyi. 2019. Proof of concept for Scenario-in-the-Loop (SciL) testing for autonomous vehicle technology. In 2019 8th IEEE International Conference on Connected Vehicles and Expo, ICCVE 2019 - Proceedings. https://doi.org/10.1109/ICCVE45908.2019.8965086
- [243]T. D. S. Guruswamy, N. Ravipati, and M. Eirinaki. 2020. Evaluating Validity of Synthetic Data in Perception Tasks for Autonomous Vehicles. In Proceedings - 2020 IEEE International Conference on Artificial Intelligence Testing, AITest 2020. 73–80. https://doi.org/10.1109/AITEST49225.2020.00018
- [244]Shuncheng Tang, Zhenya Zhang, Yi Zhang, Jixian Zhou, Yan Guo, Shuang Liu, Shengjian Guo, Yan-Fu Li, Lei Ma, Yinxing Xue, and Yang Liu. 2023. A Survey on Automated Driving System Testing: Landscapes and Trends. ACM Trans. Softw. Eng. Methodol. 32, 5, Article 124 (July 2023), 62 pages. https://doi.org/10.1145/3579642
- [245]A. Tenbrock, A. Konig, T. Keutgens, and H. Weber. 2021. The ConSCenD Dataset: Concrete Scenarios from the highD Dataset According to ALKS Regulation UNECE R157 in OpenX. In IEEE Intelligent Vehicles Symposium, Proceedings. 174–181. https://doi.org/10.1109/IVWorkshops54471.2021.9669219
- [246]T. Tettamanti, M. Szalai, S. Vass, and V. Tihanyi. 2018. Vehicle-In-the-Loop Test Environment for Autonomous Driving with Microscopic Traffic Simulation. In 2018 IEEE International Conference on Vehicular Electronics and Safety, ICVES 2018. https://doi.org/10.1109/ICVES.2018.8519486
- [247]J. Thieling, P. Elspas, and J. Rosmann. 2019. Neural networks for end-to-end refinement of simulated sensor data for automotive applications. In SysCon 2019 - 13th Annual IEEE International Systems Conference, Proceedings. https://doi.org/10.1109/SYSCON.2019.8836918
- [248]Y. Tian, W. Chu, W. Zhou, X. Tang, and K. Li. 2026. HGSim: High-fidelity and generalizable simulation framework for autonomous driving scenes. Neurocomputing 660 (2026). https://doi.org/10.1016/j.neucom.2025.131784
- [249]M. Tim, M. Szemenyei, and R. Moni. 2020. Simulation to Real Domain Adaptation for Lane Segmentation. In 2020 23rd IEEE International Symposium on Measurement and Control in Robotics, ISMCR 2020. https://doi.org/10.1109/ISMCR51255.2020.9263406
- [250]B. Toth and Z. Szalay. 2023. Development and Functional Validation Method of the Scenario-in-the-Loop Simulation Control Model Using Co-Simulation Techniques. Machines 11, 11 (2023). https://doi.org/10.3390/machines11111028
- [251]S.S. Ulhas, S. Kannapiran, and S. Berman. 2024. GAN-Based Domain Adaptation for Creating Digital Twins of Small-Scale Driving Testbeds: Opportunities and Challenges. In IEEE Intelligent Vehicles Symposium, Proceedings. 137–143. https://doi.org/10.1109/IV55156.2024.10588594
- [252]F. Ulreich, M.F. Drechsler, Y. Poldna, P.H. Chan, T. Herranen, M. Ebert, A. Kaup, and W. Huber. 2025. Validating Camera Sensor Models for Virtual Testing of Vision Systems in Automated Driving. In Proceedings of the 2025 IEEE International Conference on Vehicular Electronics and Safety, ICVES 2025. 57–64. https://doi.org/10.1109/ICVES65691.2025.11376403
- [253]F. Ulreich, A. Kaup, and M. Ebert. 2024. Novel Test Bench for End-to-End Validation of Monocular Depth Estimation Under the Influence of Glaring Situations. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC. 2426–2432. https://doi.org/10.1109/ITSC58415.2024.10920009
- [254]United Nations Economic Commission for Europe. 2023. New Assessment/Test Method for Automated Driving (NATM) Guidelines for Validating Automated Driving System (ADS). Technical Report ECE/TRANS/WP.29/2023/44/Rev.1. UNECE. https://unece.org/transport/documents/2023/06/working-documents/grva-new-assessmenttest-method-automated-driving-natm
- [255]J. Vargas Rivero, T. Gerbich, B. Buschardt, and J. Chen. 2022. The Effect of Spray Water on an Automotive LIDAR Sensor: A Real-Time Simulation Study. IEEE Transactions on Intelligent Vehicles 7, 1 (2022), 57–72. https://doi.org/10.1109/TIV.2021.3067892
- [256]M.K. Verma, P. Panchal, U. Hariharan, S. Rehan, D. Singh, and M. Yadav. 2025. Ego Vehicle Localization with GPS-IMU Fusion: A Key to Effective Scenario Generation. In 2025 3rd International Conference on Communication, Security, and Artificial Intelligence, ICCSAI 2025. 1678–1680. https://doi.org/10.1109/ICCSAI64074.2025.11063982
- [257]A. Von Bernuth, G. Volk, and O. Bringmann. 2018. Rendering Physically Correct Raindrops on Windshields for Robustness Verification of Camera-based Object Recognition. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2018-June. 922–927. https://doi.org/10.1109/IVS.2018.8500494
- [258]D. Wachtel, T. Rothmeier, L. Cristofoli, M. Vossiek, and W. Huber. 2024. Simulation and Validation of Automotive Radar Performance with Water Spray Over Radome. In Proceedings of IEEE Sensors. https://doi.org/10.1109/SENSORS60989.2024.10784774
- [259]D. Wachtel, S. Schroder, F. Reway, W. Huber, and M. Vossiek. 2021. Validation of a Radar Sensor Model under non-ideal Conditions for Testing Automated Driving Systems. In IEEE Intelligent Vehicles Symposium, Proceedings, 83–89. https://doi.org/10.1109/IVWorkshops54471.2021.9669205
- [260]S. Wagner, K. Groh, T. Kuhbeck, and A. Knoll. 2019. Towards cross-verification and use of simulation in the assessment of automated driving. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2019-June. 1589–1596. https://doi.org/10.1109/IVS.2019.8814268
- [261]A. Waheed, M. Areti, L. Gallantree, and Z. Hasnain. 2025. Quantifying the Sim2Real Gap: Model-Based Verification and Validation in Autonomous Ground Systems. IEEE Robotics and Automation Letters 10, 4 (2025), 3819–3826. https://doi.org/10.1109/LRA.2025.3546126
- [262]A. Wallace, A. Khastgir, X. Zhang, S. Brewerton, B. Anctil, P. Burns, D. Charlebois, and P. Jennings. 2022. Validating Simulation Environments for Automated Driving Systems Using 3D Object Comparison Metric. In IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2022-June. 860–866. https://doi.org/10.1109/IV51971.2022.9827354
- [263]G. Wang, Z. Zhang, and A. Nazarova. 2025. A digital twin-based platform for testing and optimization of path planning algorithms. Simulation Modelling Practice and Theory 145 (2025). https://doi.org/10.1016/j.simpat.2025.103206
- [264]H. Wang, P. Xiao, A. Liu, J. Zhang, and Q. Shan. 2026. UTDT: A Baseline Benchmark for Urban Traffic Flow Digital Twins via Roadside LiDAR–Camera Fusion. IEEE Transactions on Intelligent Transportation Systems (2026). https://doi.org/10.1109/TITS.2026.3651290
- [265]Jingkang Wang, Sivabalan Manivasagam, Yun Chen, Ze Yang, Ioan Andrei Barsan, Anqi Joyce Yang, Wei-Chiu Ma, and Raquel Urtasun. 2022. CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulation. In Proceedings of the 6th Conference on Robot Learning (CoRL) (Proceedings of Machine Learning Research, Vol. 205). https://proceedings.mlr.press/v205/wang23b.htmlarxiv.org/abs/2311.01447
- [266]J. Wang, M. Ouyang, X. Zhang, and Y. Jiang. 2025. Generative Testing Scenario Image Synthesis for Virtual-Reality Fusion-Based Autonomous Vehicle Testing. In 2025 5th International Conference on Artificial Intelligence, Big Data and Algorithms, CAIBDA 2025. 982–990. https://doi.org/10.1109/CAIBDA65784.2025.11182778
- [267]J. Wang, X. Yan, Y. Mu, H. Sun, Z. Cao, and H.X. Liu. 2025. RADE: Learning Risk-Adjustable Driving Environment via Multi-Agent Conditional Diffusion. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC. 1925–1932. https://doi.org/10.1109/ITSC60802.2025.11423555
- [268]J. Wang, Z. Yang, Y. Bai, Y. Li, Y. Zou, B. Sun, A. Kundu, J. Lezama, L.Y. Huang, G. Zhu, J.-J. Hwang, D. Anguelov, M. Tan, and C.M. Jiang. 2025. Drive&Gen: Co-Evaluating End-to-End Driving and Video Generation Models. In IEEE International Conference on Intelligent Robots and Systems. 8934–8941. https://doi.org/10.1109/IROS60139.2025.11247678
- [269]R. Wang, Z. Wang, Y. Huang, and L. Ma. 2023. When Simulator Meets Natural Deviation: A Study on Deviations in Simulation-based ADS Testing. In Proceedings - 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshop, ISSREW 2023. 83–88. https://doi.org/10.1109/ISSREW60843.2023.00052
- [270]S.-H. Wang, C.-H. Tu, and J.-C. Juang. 2022. Automatic traffic modelling for creating digital twins to facilitate autonomous vehicle development. Connection Science 34, 1 (2022), 1018–1037. https://doi.org/10.1080/09540091.2021.1997914
- [271]Waymo. 2026. The Waymo World Model: A New Frontier for Autonomous Driving Simulation. Company technical announcement. https://community.waymo.com/blog/2026/02/the-waymo-world-model-a-new-frontier-for-autonomous-driving-simulation/ Capability announcement, not independent validation.
- [272]Z. Wei, J. Bian, H. Huang, R. Zhou, and H. Zhou. 2025. Generating risky and realistic scenarios for autonomous vehicle tests involving powered two-wheelers: A novel reinforcement learning framework. Accident Analysis and Prevention 218 (2025). https://doi.org/10.1016/j.aap.2025.108038
- [273]Z. Wei, H. Huang, G. Zhang, R. Zhou, X. Luo, S. Li, and H. Zhou. 2025. Interactive Critical Scenario Generation for Autonomous Vehicles Testing Based on In-Depth Crash Data Using Reinforcement Learning. IEEE Transactions on Intelligent Vehicles 10, 3 (2025), 1471–1482. https://doi.org/10.1109/TIV.2024.3415961
- [274]Moritz Werling, Rainer Faller, Wolfgang Betz, and Daniel Straub. 2025. Safety Integrity Framework for Automated Driving. arXiv:2503.20544. https://doi.org/10.48550/arXiv.2503.20544
- [275]J. Wu, C. Lu, A. Arrieta, and S. Ali. 2025. A Tool for Benchmarking Large Language Models’ Robustness in Assessing the Realism of Driving Scenarios. In Proceedings - 2025 2nd IEEE/ACM International Conference on AI-powered Software, AIware 2025. 263–267. https://doi.org/10.1109/AIware69974.2025.00007
- [276]J. Wu, C. Lu, A. Arrieta, T. Yue, and S. Ali. 2024. Reality Bites: Assessing the Realism of Driving Scenarios with Large Language Models. In Proceedings - 2024 IEEE/ACM 1st International Conference on AI Foundation Models and Software Engineering, FORGE 2024. 40–51. https://doi.org/10.1145/3650105.3652296
- [277]Y. Wu and L. Ma. 2025. Fusing Temporal Convolutional Networks and Physical Car-Following Model for Realistic Traffic Flow Simulation. In 2025 3rd International Conference on Artificial Intelligence and Automation Control, AIAC 2025. 508–512. https://doi.org/10.1109/AIAC68175.2025.11332380
- [278]Z. Xinxin, L. Fei, and W. Xiangbin. 2020. CSG: Critical Scenario Generation from Real Traffic Accidents. In IEEE Intelligent Vehicles Symposium, Proceedings. 1330–1336. https://doi.org/10.1109/IV47402.2020.9304609
- [279]D. Xu, Y. Chen, B. Ivanovic, and M. Pavone. 2023. BITS: Bi-level Imitation for Traffic Simulation. In Proceedings - IEEE International Conference on Robotics and Automation, Vol. 2023-May. 2929–2936. https://doi.org/10.1109/ICRA48891.2023.10161167
- [280]W. Xu, N. Souly, and P.P. Brahma. 2021. Reliability of GAN Generated Data to Train and Validate Perception Systems for Autonomous Vehicles. In Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2021. 171–180. https://doi.org/10.1109/WACVW52041.2021.00023
- [281]X. Yan, S. Feng, H. Sun, and H.X. Liu. 2025. Distributionally Consistent Simulation of Naturalistic Driving Environment for Autonomous Vehicle Testing. IEEE Transactions on Intelligent Transportation Systems 26, 7 (2025), 9187–9200. https://doi.org/10.1109/TITS.2025.3571966
- [282]Z. Yang, Y. Chen, J. Wang, S. Manivasagam, W.-C. Ma, A.J. Yang, and R. Urtasun. 2023. UniSim: A Neural Closed-Loop Sensor Simulator. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2023-June. 1389–1399. https://doi.org/10.1109/CVPR57229.2023.00140
- [283]Ze Yang, Sivabalan Manivasagam, Ming Liang, Bin Yang, Wei-Chiu Ma, and Raquel Urtasun. 2021. Recovering and Simulating Pedestrians in the Wild. In Proceedings of the 2020 Conference on Robot Learning (Proceedings of Machine Learning Research, Vol. 155), Jens Kober, Fabio Ramos, and Claire Tomlin (Eds.). PMLR, 419–431. https://proceedings.mlr.press/v155/yang21a.htmlarxiv.org/abs/2011.08106
- [284]Z. Yang, S. Masuda, and M. Tatsubori. 2021. Online Adaptation of Parameters using GRU-based Neural Network with BO for Accurate Driving Model. In GIS: Proceedings of the ACM International Symposium on Advances in Geographic Information Systems. 33–36. https://doi.org/10.1145/3474717.3483632
- [285]D. Yeo, G. Kim, and S. Kim. 2019. MAXIM: Mixed-reality automotive driving XMIulation. In Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019. 460–464. https://doi.org/10.1109/ISMAR-Adjunct.2019.00124
- [286]B. Yu and D. Wang. 2026. Trajectory-Based Video Generation Model for Autonomous Driving. In 2026 2nd International Conference on Electrical Automation and Artificial Intelligence, ICEAAI 2026. 515–519. https://doi.org/10.1109/ICEAAI68545.2026.1142436
- [287]M. Yuan, J. Shan, and K. Mi. 2024. From Naturalistic Traffic Data to Learning-Based Driving Policy: A Sim-To-Real Study. IEEE Transactions on Vehicular Technology 73, 1 (2024), 245–257. https://doi.org/10.1109/TVT.2023.3307409
- [288]Ekim Yurtsever, Jacob Lambert, Alexander Carballo, and Kazuya Takeda. 2020. A survey of autonomous driving: Common practices and emerging technologies. IEEE access 8 (2020), 58443–58469.DOI
- [289]Y. Zeng, S. Yan, J. Li, T. Huang, S. Ai, and H. Zeng. 2025. Controllable Rainy Image Generation for Autonomous Driving via a GAN-Based Framework. In 2025 5th International Symposium on Computer Technology and Information Science, ISCTIS 2025. 1265–1272. https://doi.org/10.1109/ISCTIS65944.2025.11065211
- [290]G. Zhang, H. Huang, R. Zhou, S. Li, and J. Bian. 2025. High-Risk Trajectories Generation for Safety Testing of Autonomous Vehicles Based on In-Depth Crash Data. IEEE Transactions on Intelligent Transportation Systems 26, 8 (2025), 11619–11630. https://doi.org/10.1109/TITS.2025.3576236
- [291]Jie M. Zhang, Mark Harman, Lei Ma, and Yang Liu. 2022. Machine Learning Testing: Survey, Landscapes and Horizons. IEEE Transactions on Software Engineering 48, 1 (2022), 1–36. https://doi.org/10.1109/TSE.2019.2962027
- [292]Q. Zhang, Y. Gao, Y. Zhang, Y. Guo, D. Ding, Y. Wang, P. Sun, and D. Zhao. 2022. TrajGen: Generating Realistic and Diverse Trajectories With Reactive and Feasible Agent Behaviors for Autonomous Driving. IEEE Transactions on Intelligent Transportation Systems 23, 12 (2022), 24474–24487. https://doi.org/10.1109/TITS.2022.3202185
- [293]Q. Zhang, C. Guo, D. Pan, Z. Meng, C. Fu, and G. Li. 2025. A Pixel-level Image Synthesis Method Based on Vehicle Position Semantics and Motion Attitude. In 2025 IEEE 2nd International Conference on Electronics, Communications and Intelligent Science, ECIS 2025 - Proceeding. https://doi.org/10.1109/ECIS65594.2025.11086908
- [294]X. Zhang and Y. Cai. 2023. Building Critical Testing Scenarios for Autonomous Driving from Real Accidents. In ISSTA 2023 - Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 462–474. https://doi.org/10.1145/3597926.3598070
- [295]Y. Zhang, S. Song, and C. Chen. 2025. End-to-End Learning Based Traffic Flow Agent Generation and Its Application in Simulation Environments. In 2025 7th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2025. 306–311. https://doi.org/10.1109/RICAI68060.2025.11384875
- [296]Y. Zhang, X. Zheng, and H. Liang. 2022. Self-driving simulation scene reconstruction using self-supervised depth completion. In Proceedings - 2022 7th International Conference on Automation, Control and Robotics Engineering, CACRE 2022. 254–263. https://doi.org/10.1109/CACRE54574.2022.9834190
- [297]J. Zhao, W. Li, B. Zhu, P. Zhang, and R. Tang. 2025. A Tachograph-Based Approach to Restoring Accident Scenarios From the Vehicle Perspective for Autonomous Vehicle Testing. IEEE Transactions on Intelligent Transportation Systems 26, 9 (2025), 13909–13926. https://doi.org/10.1109/TITS.2025.356980
- [298]Jingyuan Zhao, Wenyi Zhao, Bo Deng, Zhenghong Wang, Feng Zhang, Wenxiang Zheng, Wanke Cao, Jinrui Nan, Yubo Lian, and Andrew F. Burke. 2024. Autonomous driving system: A comprehensive survey. Expert Systems with Applications 242 (2024), 122836. https://doi.org/10.1016/j.eswa.2023.122836
- [299]Ziyuan Zhong, Yun Tang, Yuan Zhou, Vania de Oliveira Neves, Yang Liu, and Baishakhi Ray. 2021. A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation. arXiv:2112.00964 [cs.SE]arxiv.org/abs/2112.00964
- [300]H. Zhou, L. Lin, J. Wang, Y. Lu, D. Bai, B. Liu, Y. Wang, A. Geiger, and Y. Liao. 2026. HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving. IEEE Transactions on Pattern Analysis and Machine Intelligence 48, 4 (2026), 4673–4691. https://doi.org/10.1109/TPAMI.2025.3647952
- [301]M. Zhou, X. Zhang, J. Yin, Y. Hu, W. Liu, and J. Li. 2026. Diffusion Models for Autonomous Driving in Smart Parking: A Data Synthesis Framework. IEEE Transactions on Automation Science and Engineering 23 (2026), 3586–3595. https://doi.org/10.1109/TASE.2025.3590040
- [302]R. Zhou, Z. Lin, G. Zhang, H. Huang, H. Zhou, and J. Chen. 2024. Evaluating Autonomous Vehicle Safety Performance Through Analysis of Pre-Crash Trajectories of Powered Two-Wheelers. IEEE Transactions on Intelligent Transportation Systems 25, 10 (2024), 13560–13572. https://doi.org/10.1109/TITS.2024.3392673
- [303]B. Zhu, Y. Sun, J. Zhao, S. Zhang, P. Zhang, and D. Song. 2022. Millimeter-Wave Radar in-the-Loop Testing for Intelligent Vehicles. IEEE Transactions on Intelligent Transportation Systems 23, 8 (2022), 11126–11136. https://doi.org/10.1109/TITS.2021.3100894
- [304]M.R. Zorfka, M. Essinger, T. Fleck, R. Kohlhaas, and J.M. Zollner. 2018. The sleepwalker framework: Verification and validation for autonomous vehicles by mixed reality LiDAR stimulation. In 2018 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots, SIMPAR 2018. 151–157. https://doi.org/10.1109/SIMPAR.2018.8376285
- [305]M.R. Zorfka, T. Fleck, P. Schorner, M. Heinrich, and J.M. Zollner. 2023. Paving the Ways for Vehicles-in-the-loop: Wireless Integration of Autonomous Vehicles into Mixed Reality Proving Grounds and Test Areas. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC. 2626–2633. https://doi.org/10.1109/ITSC57777.2023.10421917
- [306]M.R. Zorfka, S. Ulbrich, D. Karl, T. Fleck, R. Kohlhaas, A. Rönnau, R. Dillmann, and J.M. Zöllner. 2018. Traffic Participants in the Loop: A Mixed Reality-Based Interaction Testbed for the Verification and Validation of Autonomous Vehicles. In IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 2018-November. 3583–3590. https://doi.org/10.1109/ITSC.2018.8569226
- [307]J. Zuo, H. Hu, Z. Zhou, Y. Cui, Z. Liu, J. Wang, N. Guan, J. Wang, and C.J. Xue. 2025. RALAD: Bridging the Real-to-Sim Domain Gap in Autonomous Driving with Retrieval-Augmented Learning. In IEEE International Conference on Intelligent Robots and Systems. 17001–17007. https://doi.org/10.1109/IROS60139.2025.11245837

