betaFeedback

WorksComputer Science

BugGen 2.0: Rethinking the Roles of LLMs for Scalable RTL Bug Synthesis

Abstract

AI-assisted RTL localization and debugging need ground-truth bug datasets with known locations and observable failures, while testbench grading uses injected bugs to measure detection capability. Creating such datasets at scale remains a major EDA challenge. BugGen 1.0 shows that Large Language Models (LLMs) can generate context-aware RTL mutations. Still, it relies on the model for both choosing where to mutate and generating the edit, which increases inference costs and can lead to invalid choices. BugGen 2.0 changes this division of labor. Deterministic algorithms handle structural analysis, target selection, and validation, while the LLM focuses on localized semantic rewriting. Across four OpenTitan IP blocks and FabScalar with three LLMs, BugGen 2.0 reaches every requested quota of compilable bug scenarios that cause at least two test failures, with 14-16 of 16 configured mutation types represented. In completed same-model comparisons, it reduces token use by 6.8x-15.4x relative to BugGen 1.0 and 1.5x-7.9x relative to Firefly. BugGen 2.0 also expands the dataset without simulating every new bug. After finding accepted scenarios with observable failures, it generates alternative mutations at the same locations, simulates only a subset, and estimates acceptance for the rest. This skips 63.6% of AES and 33.9% of FabScalar variant simulations. Full simulation finds that 77.7% of AES variants and 95.2% of FabScalar variants are accepted, with a 1.09% estimation error on the directly comparable FabScalar set. These results show that focusing LLMs on semantic rewriting lowers inference and simulation cost while improving consistency across models and designs.

Introduction

Growing design complexity makes verification and debugging a dominant part of integrated circuit (IC) development, with design functional verification (DFV) alone accounting for more than half of the overall design effort [1]. Bug datasets can support both machine learning (ML)-based localization and debugging [2], [3], and systematic grading of verification environments, but obtaining useful ground truth at scale remains difficult. Manual fault injection does not scale, while predefined RTL mutation testing frameworks [4] can generate many faults but capture limited semantic and architectural context. More broadly, preparing open datasets remains a major obstacle to applying ML in EDA [5]. The practical need is therefore for a scalable way to generate varied, context-aware RTL bugs and evaluate them in industrial-style verification flows.

We refer to BugGen [6] as BugGen 1.0 throughout this paper. Its LLM-centered workflow partitions RTL modules, then uses three agents to select a region, choose a mutation type and target, and generate the edit. A shared cache records successful outcomes, but this history influences later decisions only through in-context reasoning rather than enforced constraints. As a result, incompatible choices can consume several model calls before compilation rejects them, and rollback may lead the agents back to similar unsuccessful decisions. Since prior outcomes and prompt constraints remain advisory, the LLM can repeat earlier mistakes or generate edits that violate those constraints.

BugGen 2.0 explicitly handles decisions with deterministic mechanisms rather than relying on the LLM. Type-aware filtering removes incompatible targets before generation using regex rules and abstract syntax tree (AST)-derived types. Stratified line sampling spreads selections across the RTL file, while compile learning uses previous outcomes to steer mutation type selection. Batched generation requests several localized edits in one model call. Finally, once a bug scenario is accepted, variant augmentation generates alternative mutations at its locations, simulates only a subset, and estimates the outcomes of the rest.

This paper makes three major contributions that, to our knowledge, are new to RTL bug synthesis. First, it introduces a hybrid workflow that assigns structural analysis, target selection, and validation to deterministic algorithms while reserving the LLM for localized rewriting. Second, it introduces variant augmentation, which reduces simulation by evaluating only a subset of generated alternatives and estimating the rest. Third, it characterizes model dependence by evaluating efficiency and consistency across different LLM backends. Across five designs and three LLMs, BugGen 2.0 reaches every requested quota of accepted bug scenarios and covers 14–16 of the 16 configured mutation types. On AES and FabScalar, it also uses substantially fewer tokens than LLM-centered baselines.

Traditional Hardware Mutation Testing

Traditional RTL mutation testing frameworks [4] apply predefined syntactic mutations primarily to assess verification quality, with undetected mutants revealing gaps in the testbench. Although scalable, these approaches capture limited semantic and architectural context. Prior research on mutation testing in software has also explored reducing execution costs through historical operator performance [7] and predictive detection models [8]. BugGen 2.0 applies these ideas to RTL bug synthesis, using localized RTL context to generate diverse mutations that produce observable failures for debugging and learning. It therefore combines context-aware RTL mutation with history-guided, budgeted evaluation.

LLM-Based RTL Mutation

Recent approaches use LLMs to move beyond predefined mutation operators by generating edits from RTL context. Firefly [9] prompts an LLM with complete RTL files and relies on the model to select both mutation locations and types. BugGen 1.0 [6] adds a closed-loop multi-agent workflow with cached history, but still delegates structural decomposition and target selection to LLM reasoning. Related work, such as RTLFixer [10], instead uses EDA tool feedback to guide LLM-based RTL repair rather than to generate bugs. Thus, existing LLM-based bug synthesis systems continue to rely on the model for both target selection and edit generation. BugGen 2.0 separates these roles, using explicit structural analysis and selection mechanisms to reduce inference costs while retaining a broad representation of mutation types.

Introducing BugGen 2.0

Figure 1 shows the base generation and variant augmentation pipelines. BugGen 2.0 first determines which RTL locations and mutation types are structurally compatible, and then asks the LLM to generate only localized edits. Generated candidates pass through syntax, compilation, and simulation checks. This ordering removes unsuitable choices before they consume more expensive inference and verification resources.

Figure 1. BugGen 2.0 pipelines.

A mutation is one RTL code edit whose injection location is known by construction, while a bug scenario is a modified RTL instance containing N≥1N \ge1 mutations. The main pipeline produces base scenarios. Once a base scenario is accepted, augmentation can generate alternative mutations at the same locations, producing variants. For example, after a scenario containing a mutation of type tt at line ℓ\ell causes observable test failures, augmentation can keep ℓ\ell fixed and generate other compatible edits there. A scenario is accepted if it compiles and satisfies the user-defined criterion for its generation stage. In our experiments, base generation requires at least two failing tests, and variant augmentation requires at least one.

Scope and Filtering

Before generation, BugGen 2.0 extends the BugGen 1.0 rulebook with mutation-specific regex rules and a signal-type database, TypeDB, extracted from the Verible AST [11] using a technique similar to VCDiag [2]. These checks map each RTL line to compatible mutation types and remove known incompatibilities, including mismatched user-defined types, enums, or structs, before invoking the LLM or compiler. A signal swap, for instance, is rejected when the candidate signals have incompatible types or widths.

Target Selection

After filtering, BugGen 2.0 samples among the compatible mutation types using a weight updated from prior outcomes, as described in Section III-D. To avoid repeatedly targeting the same part of a file, the system divides each file into RR approximately equal strata, or contiguous line ranges. For each of the NN mutations, it selects the least-used eligible stratum not already used in the scenario, breaking ties uniformly at random, then selects the least-used compatible line within that stratum.

Batch Generation and Pre-checks

For each selected mutation type and line pair (t,ℓ)(t,\ell), BugGen 2.0 extracts a local context window around ℓ\ell. All selected windows are packed into one prompt, so that a three-mutation scenario can request all three edits in one LLM call. The LLM returns the edits as a JSON array, similar to Firefly [9]. If an entry is missing or malformed, the valid edits are retained, and only the failed slot is regenerated using another compatible mutation type.

Each returned edit is checked against the original RTL and its compatibility rules. Valid edits are then applied one at a time to a clean copy and parsed with Verible [11] before full

compilation. An earlier edit can change the source text around a later target, so a later slot is omitted if its target can no longer be found. The resulting scenario may therefore contain fewer than the requested NN mutations.

Compile Learning

Compile learning uses prior outcomes to guide future mutation type selection. Intuitively, a mutation type that repeatedly produces invalid RTL becomes less likely to be selected again within the same module. Each type has a nonnegative penalty score ftf_t: failures increase the score, accepted scenarios decrease it, and larger scores reduce future selection.

A scenario may contain several mutations, so the system cannot always determine which mutation caused a failure. It therefore applies the same scenario-level update to every mutation type used in the scenario. This avoids relying on noisy, tool-specific compiler logs, but does not assign causal blame to a particular mutation.

For the currently eligible mutation type set TeT_e, the sampling probability of type tt is

π(t)=wt∑j∈Tewj,wt=αmin⁡(ft,fmax⁡)ρt,(1)\pi(t) = \frac{w_t}{\sum_{j \in T_e} w_j}, \qquad w_t = \alpha^{\min(f_t, f_{\max})}\rho_t, \tag*{(1)}

where α\alpha suppresses types with prior failures, ρt\rho_t penalizes reuse within the current scenario, and fmax⁡f_{\max} caps the penalty. By default, α=0.75\alpha= 0.75, so each additional failure multiplies the base weight by 0.75. We set ρt=0.7\rho_t = 0.7 if type tt has already been selected in the current scenario and 1 otherwise, with fmax⁡=20f_{\max} = 20. Scores begin at zero and reset at module boundaries so that feedback remains local to each module.

After observing scenario outcome ss, the score is updated as

ft←max⁡(0,ft+δs),(2)f_t \leftarrow\max(0, f_t + \delta_s), \tag*{(2)}

where δs\delta_s is +β+\beta, −γ-\gamma, or +δu+\delta_u for a failure, accepted scenario, or compiled but unaccepted scenario, respectively. We use (β,γ,δu)=(1,1,0)(\beta, \gamma, \delta_u) = (1, 1, 0). Failures include LLM no-operation outputs, Verible syntax failures, and compiler failures. A compiled but unaccepted scenario leaves the score unchanged because the result may reflect testbench observability rather than mutation quality.

Variant Augmentation for Data Expansion

Variant augmentation expands the dataset by reusing locations from accepted base scenarios. Since these locations come from base scenarios with observable failures, BugGen 2.0 explores different compatible mutations there rather than searching for new targets. Specifically, if a base scenario modifies lines ℓ1\ell_1 and ℓ2\ell_2, augmentation keeps those locations fixed and replaces the original mutation types with alternatives.

For each accepted base, augmentation generates up to vmax⁡v_{\max} variants. At each location, the original type is excluded, while the same regex, TypeDB, and no-operation checks determine compatible replacements. Types are sampled using compile learning weights with a reuse penalty ρt\rho_t for types already selected in the scenario. All variants are generated in one LLM call, applied to clean RTL copies, and checked for compatibility and Verible validity. Invalid or duplicate variants are discarded.

Simulating every variant would be expensive, so BugGen 2.0 evaluates only a subset and estimates acceptance for the rest. For retained variant set VsV_s of base ss and nominal simulation fraction bb, the per-base budget is

Bs={0,∣Vs∣=0,max⁡(1,round⁡(∣Vs∣b)),∣Vs∣>0.(3)B_s = \begin{cases} 0, & |V_s| = 0,\\ \max(1, \operatorname{round}(|V_s|b)), & |V_s| > 0. \end{cases} \tag*{(3)}

At least one simulation ensures that every base contributes measured evidence, so the total fraction can exceed bb. Consider 125 bases, each producing four or five variants, yielding 600 in total. With b=0.2b = 0.2, the per-base minimum gives 125 simulations instead of 120. The remaining question is which variants to simulate. A type with weight wtw_t near 1 has accumulated little penalty and is already favored by the generation policy, while a type near 0 has accumulated repeated penalties and is strongly suppressed. BugGen 2.0, therefore, spends more of the limited simulation budget on types between these extremes using ut=4wt(1−wt)u_t = 4w_t(1-w_t). The score peaks at ut=1u_t = 1 when wt=0.5w_t = 0.5 and decreases toward 0 at either extreme. At wt=0.1w_t = 0.1 or 0.9, for instance, ut=0.36u_t = 0.36. Since wtw_t is a ranking weight, not a probability, this is a mid-weight selection heuristic and does not represent uncertainty estimation.

Unsimulated variants are estimated from prior simulated outcomes. For each mutation type, BugGen 2.0 records the observed acceptance rate yt=dt/sty_t = d_t/s_t, where dtd_t and sts_t are the counts of accepted and simulated variants, respectively. An unsimulated variant receives the mean rate of its mutation types, or the design’s base acceptance rate if no history exists. A variant may contain multiple mutations, so yty_t reflects association with acceptance, not the causal effect of one type. Measured and estimated outcomes are reported separately, and the cache persists across runs within a design.

Experimental Setup

We evaluate BugGen 2.0 on four OpenTitan [12] IP blocks (AES, OTP Controller, ROM Controller, and USB) and the FabScalar [13] processor core. OpenTitan uses SystemVerilog UVM testbenches with Synopsys VCS [14]. FabScalar uses Verilog, a C++ instruction-set simulator as its reference model, and Cadence Xcelium [15]. Table I summarizes these flows. Each experiment uses a predefined accepted-scenario quota. A scenario contains at most N=3N = 3 simultaneous mutations, and base acceptance requires compilation followed by at least τ=2\tau= 2 failing tests.

DesignRTL LinesVerificationSimulator
AES17,198UVMVCS
OTPCtrl12,747UVMVCS
RomCtrl2,691UVMVCS
USB15,942UVMVCS
FabScalar40,575ISS Ref.Xcelium

Table I. Evaluated designs and verification flows.

For comparison, we rerun BugGen 1.0 and Firefly [9] on the same RTL and LLM backends. We adapt only their interfaces to the verification flows, preserving their prompting, target selection, and mutation generation procedures. AES also includes a commercial RTL mutation testing tool as a traditional reference. All LLM-based frameworks use the same 16 mutation types in Table II.

Mutation Types
missing_assignmentsignal_swap
bitwise_corruptionstuck_at_fault
control_flow_bugdefault_value_change
wrong_assignmentpipeline_hazard
incorrect_data_sizechecker_corruption
fsm_transitionregister_address_corruption
adjacent_field_swaphandshake_corruption
loop_modificationmasking_corruption

Table II. Mutation types available for all LLM-based frameworks.

Experiments use Google Gemma 4 31B (G4) [16], OpenAI gpt-oss-120b (OSS) [17], and DeepSeek v4 Flash (D4F) [18]. BugGen 2.0 uses a 32,768-token context window, an 8,192-token output cap, disabled reasoning, provider-default sampling parameters, and no model-specific tuning. Token counts include prompts and completions from every request, including retries and unsuccessful outputs. We report token-reduction factors only for the same model.

The systems use different stopping rules. BugGen 1.0 and BugGen 2.0 run toward accepted scenario quotas, whereas Firefly receives one attempt per configured scenario. As a result, attempts should not be compared as a common budget across all rows. Tokens per accepted scenario (kTok/A) instead normalizes inference volume by accepted output.

Results

The results answer three questions: how much inference and wall-clock time the hybrid workflow uses, how its components affect validation and line-position spread, and how much additional data variant augmentation adds under a limited simulation budget.

End-to-End Performance

Table III reports AES and FabScalar results. A/Att.A/Att. is accepted scenarios over top-level attempts, where one attempt may contain several internal generation retries. CC is the fraction of generated candidates that pass Verible and compilation. kTok/AkTok/A is the total number of LLM tokens per accepted scenario, including retries and unsuccessful outputs. TT counts mutation types represented in accepted scenarios, and SS counts post-compilation simulation evaluations. Each evaluation may execute several tests.

SystemModelA/Att.CkTok/ATS
AES
FireflyG456/14045%16.2863
OSS94/14074%10.511103
D4F56/14042%41.61359
BG 1.0G4†62/10050%89.41095
OSS†62/8562%80.811101
D4F140/20385%81.211333
BG 2.0G4140/14780%4.6215163
OSS140/15371%7.1515163
D4F140/14878%5.2615173
FabScalar‡
FireflyG4339/340100%12.110340
OSS333/34098%10.612333
D4F324/34096%14.615325
BG 1.0G4360/503100%80.69859
OSS360/36196%41.913387
D4F360/37196%45.27435
BG 2.0G4360/39093%8.3114481
OSS360/36486%6.1714381
D4F360/36190%4.1714394

Table III. End-to-end performance on AES and FabScalar.

BugGen 2.0’s efficiency, as shown in Table III, is less sensitive to the choice of LLM. Across both designs, its token cost remains within 4.17–8.31 kTok/A, while Firefly varies more strongly on AES and BugGen 1.0 incurs much higher cost when repeated retries fail to make progress. In same-model

comparisons, BugGen 2.0 reduces token use by 15.4× on AES with D4F and 6.8×–10.9× on FabScalar, while the reduction relative to Firefly is 1.5×–7.9×.

DesignModelA/Att.CkTok/ATS
OTPCtrlG4120/12467%4.2416120
OSS120/12178%4.8916120
D4F120/12374%3.8416120
RomCtrlG4150/16073%4.1714165
OSS150/15671%6.1615170
D4F150/15777%3.7515170
USBG4160/16677%4.2314173
OSS160/16980%5.4814182
D4F160/16585%3.7415180

Table IV. BugGen 2.0 on additional OpenTitan designs.

The additional OpenTitan results show that this consistency extends beyond AES and FabScalar. All nine model and design combinations meet their quotas with low token cost and 14–16 represented mutation types across substantially different RTL blocks.

Figure 2 complements these aggregate results with convergence trajectories and mutation type distributions. Firefly generates its full candidate set before simulation, so its final yield depends on the quality of that initial set. This works well on FabScalar, where 339 of 340 are accepted, but poorly on AES, where only 56 of 140 survive with both G4 and D4F. BugGen 1.0 instead retries unsuccessful scenarios, but because prior outcomes remain advisory, requeued scenarios can revisit similar targets and mutation choices, producing long retry loops with little progress. BugGen 2.0 avoids much of this behavior through explicit filtering and history-guided selection, leading to steadier convergence. The lower panels further show that this efficiency does not come from concentrating on a small subset of mutation types. BugGen 2.0 maintains both broad and highly consistent mutation type distributions across LLM backends, suggesting that the pipeline itself plays a larger role in preserving diversity than model-specific behavior.

Accepted scenarios over time and mutation-type distributions for AES and FabScalar

Figure 2. End-to-end convergence and mutation type representation with Gemma 4 and DeepSeek v4 Flash. (a)–(c) Cumulative accepted bug scenarios over run time, with the dotted horizontal lines marking target quotas. (d)–(f) Mutation type distributions in accepted scenarios.

Line-Position Spread and Component Effects

Table V gives a descriptive comparison of AES outcomes and line-position spread for the G4 runs in Table III. For the LLM-based systems, D/F/U denotes accepted candidates, failed Verible or compiler validation, and compiled but subthreshold candidates. For the commercial tool, the same letters denote detected, dropped, and undetected individual faults. The outcome categories, therefore, partition candidates within a system but are not equivalent across systems.

To measure how evenly accepted mutations are distributed across the RTL, each target file is divided into M=10M = 10 equal line bins. These bins are used only for evaluation and are separate from the strata used during target selection. For LLM systems, pip_i is the fraction of accepted mutation locations in bin ii. For the commercial tool, it is the fraction of reported static fault locations. We compute the normalized Shannon entropy as

Spread⁡=−∑i=1Mpilog⁡2pilog⁡2M(4)\operatorname{Spread} = \frac{-\sum_{i=1}^{M} p_i \log_2 p_i}{\log_2 M} \tag*{(4)}

Scores are averaged across the seven AES target files. A score of 0 indicates complete concentration, while 1 indicates uniform placement. We assign 0 to target files with no accepted location. The metric measures source-line spread, not semantic or architectural coverage.

SystemD/F/USpread
Commercial tool523/335/10.726
Firefly56/77/70.514
BugGen 1.062/94/330.327
BugGen 2.0140/40/230.858

Table V. AES outcomes and line-position spread with Gemma 4.

BugGen 2.0 has the highest line-position spread in Table V, while the partial BugGen 1.0 run leaves three target files with no accepted location. This pattern is consistent with stratified sampling favoring underexplored line ranges instead of repeatedly revisiting already explored ones. While differences in candidate conditioning, sample size, and scenario-versus-fault units prevent these statistics from serving as direct measures of overall fault quality, they provide a useful view of how broadly each method distributes accepted bugs across RTL locations.

Table VI reports independent ablations of pre-filtering, compile learning, and stratification on AES. The independent full run was conducted after the runs in Tables III and V, so the results differ slightly. The run without pre-filtering terminated after the two-day wall-clock limit, at 117 accepted scenarios.

The ablations separate the roles of the three mechanisms. Pre-filtering yields the greatest inference savings, as removing it causes no-op outputs and model requests to increase sharply, despite the higher validation rate among the candidates that survive generation. Compile learning reduces repeated invalid choices, while stratification determines how accepted mutations are distributed across the RTL. These results support using separate deterministic mechanisms for different sources of inefficiency rather than asking the LLM to manage them all.

ConfigurationAcc.CkTok/ASimsTypesSpread
Full BugGen 2.014081%4.4159150.862
Without pre-filtering11791%22.1182150.713
Without compile learning14078%4.8164150.871
Without stratification14083%4.2160150.823

Table VI. Component ablation on OpenTitan AES with Gemma 4.

Variant Augmentation

Table VII evaluates how variant augmentation expands the dataset while reducing simulation effort. Budget simulates only the subset selected in Section III-E, while All and GT evaluate every retained variant. On AES, Budget and All are independent runs with different generated variants, so they characterize the overall tradeoff but cannot validate the variants skipped by Budget on the same set. On FabScalar, GT continues the Budget run by simulating exactly the variants it skipped, enabling direct validation of the estimate.

To quantify estimation accuracy, let AvA_v be the measured number of accepted variants, A^v\widehat{A}_v be the estimate, and ∣V∣|V| be the number of retained variants. The error EvE_v measures the relative difference between measured and estimated acceptance, while Eall-accE_{\mathrm{all\text{-}acc}} measures the error from assuming that every retained variant is accepted.

Ev=∣A^v−Av∣Av,Eall-acc=∣V∣−AvAv.(5)E_v = \frac{|\widehat{A}_v-A_v|}{A_v}, \qquad E_{\mathrm{all\text{-}acc}} = \frac{|V|-A_v}{A_v}. \tag*{(5)}

AES illustrates the simulation tradeoff. Budget skips 63.6% of the retained variant simulations, but its skipped variants are not later evaluated, so its estimation error cannot be measured directly. The independent All run instead checks the estimator on a separate fully simulated set. Its estimate predicts 258 accepted variants, while the full simulation accepts 300 of 386, yielding a 14.0% error, compared to 28.7% when every retained variant is assumed to succeed. Budget and All take 12.3 and 27.3 hours, respectively. While their different base runs and variant sets preclude a controlled speedup comparison, the measurements still illustrate the practical runtime benefit that motivates budgeted simulation.

FabScalar provides the same-set validation unavailable for AES Budget. After simulating 636 of 962 variants, Budget estimates 906 accepted variants across the full set. GT then evaluates the remaining 326 variants and obtains 916 accepted variants, giving a 1.09% estimation error while avoiding 33.9% of the simulations during the budgeted run. This also improves on the 5.0% error obtained by assuming that every retained variant is accepted.

The FabScalar estimate uses a cache of 437 prior outcomes from earlier base generation runs. Table VII excludes the cost of collecting this history, so the cache has an upfront cost but can be reused across subsequent augmentation runs. The cache is frozen before GT evaluates the skipped variants, preventing those outcomes from influencing the estimate and providing an independent test of how well previously collected evidence transfers to new variants.

MetricOpenTitan AESOpenTitan AESOpenTitan AESFabScalarFabScalarFabScalar
BaseBudgetAllBaseBudgetGT
Data expansion
Variants retained03853860962962
Variants simulated—140386—636962
Variant sims skipped—63.6%0%—33.9%0%
Variants accepted—114300—600916
Variant acc. rate—81.4%77.7%—94.3%95.2%
Execution
RTL instances1405255263601,3221,322
Evaluation attempts1813405924511,0871,413
Run time (in hours)—12.327.3—15.017.8
Acceptance estimation
Estimated accepted—298258—906906
Estimated error (EvE_v)——14.0%——1.09%
Assume all error (Eall-accE_{\mathrm{all\text{-}acc}})——28.7%——5.0%

Table VII. Variant augmentation with Gemma 4.

Conclusion

In conclusion, BugGen 2.0 shows that scalable RTL bug synthesis benefits from separating structural decisions from semantic rewriting. Deterministic filtering, targeting, validation, and feedback reduce the number of repeated low-value decisions, while the LLM is reserved for localized edits. Across five designs and three LLMs, BugGen 2.0 reaches every quota with broad mutation type representation and consistent efficiency. In completed same-model comparisons on AES and FabScalar, it reduces token use by 6.8×\times–15.4×\times relative to BugGen 1.0 and 1.5×\times–7.9×\times relative to Firefly. Variant augmentation addresses the remaining simulation bottleneck by reusing locations from accepted scenarios, simulating only a subset of alternatives, and estimating the rest. Together, these mechanisms make ground-truth RTL bug generation more practical and scalable for debugging, evaluation, and testbench grading.

The current evaluation establishes compilation and test observability, but does not characterize how individual bugs propagate through a module or affect architectural behavior. Stronger pre-simulation predictors and propagation-aware analysis could further reduce simulation cost while providing richer ground truth for debugging and verification. Such analysis could distinguish bugs that merely trigger failures from those with deeper functional or architectural effects, extending BugGen 2.0 toward more informative bug characterization.

Acknowledgment

The complete acknowledgment and funding information will be included in the final version. Additional acknowledgment text may be added before submission.

References

  1. [1]H. D. Foster, “2024 Wilson Research Group IC/ASIC functional verification trend report,” Siemens EDA White Paper, 2024.
  2. [2]M. Luu, S. Jasper, K. Le, E. Pan, M. Quinn, A. Tyagi, J. Hu et al., “VC-Diag: Classifying erroneous waveforms for failure triage acceleration,” in Proc. 2025 ACM/IEEE 7th Symposium on Machine Learning for CAD (MLCAD), 2025, pp. 1–8.
  3. [3]J. Wang, Y. Lei, J. Chen, H. Xie, and C. Liu, “SimFree: A simulation-free HDL bug localization approach via deep static representation fusion,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2026.DOI
  4. [4]J. Wu et al., “Mantra: Mutation testing of hardware design code based on real bugs,” in Proc. 60th ACM/IEEE Design Automation Conference (DAC), 2023, pp. 1–6.DOI
  5. [5]Y. Chen, V. A. Chhabria, C. Hao, R. Harjani, J. Hu, A. B. Kahng, M. Quinn, S. S. Sapatnekar, and A. Tyagi, “NSF workshop on shared infrastructure for machine learning EDA: Summary report,” National Science Foundation (NSF), Grant CCF-2310319, 2023.
  6. [6]S. Jasper, M. Luu, E. Pan, A. Tyagi, M. Quinn, J. Hu, and D. Houngninou, “BugGen: A self-correcting multi-agent LLM pipeline for realistic RTL bug synthesis,” in Proc. 2025 ACM/IEEE 7th Symposium on Machine Learning for CAD (MLCAD), 2025, pp. 1–9.arxiv.org/abs/2506.10501
  7. [7]G. Petrović, M. Ivanković, G. Fraser, and R. Just, “Practical mutation testing at scale: A view from Google,” IEEE Transactions on Software Engineering, vol. 48, no. 10, pp. 3900–3912, 2022.DOI
  8. [8]K. Jain, U. Alon, A. Groce, and C. L. Goues, “Contextual predictive mutation testing,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2023, pp. 250–261.arxiv.org/abs/2309.02389
  9. [9]B. Skakov, S. Abdurraimov, O. Nurman, and N. Kabylkas, “Firefly: Shedding light on verification gaps using LLM-powered mutation testing,” in Proc. 2025 ACM/IEEE 7th Symposium on Machine Learning for CAD (MLCAD), 2025, pp. 1–12.DOI
  10. [10]Y. Tsai, M. Liu, and H. Ren, “RTLFixer: Automatically fixing RTL syntax errors with large language model,” in Proceedings of the 61st ACM/IEEE Design Automation Conference, 2024, pp. 53:1–53:6.DOI
  11. [11]Verible, “Verible: Verilog lint, format, and language server,” GitHub repository, https://github.com/chipsalliance/verible.
  12. [12]OpenTitan Project, “OpenTitan: Open source silicon root of trust (RoT),” GitHub repository, 2026, https://github.com/lowrisc/opentitan.
  13. [13]N. K. Choudhary, S. V. Wadhavkar, T. A. Shah et al., “FabScalar: Composing synthesizable RTL designs of arbitrary cores within a canonical superscalar template,” in Proc. 38th Annual International Symposium on Computer Architecture (ISCA), 2011, pp. 11–22.
  14. [14]Synopsys, Inc., “Synopsys VCS: Verilog compiler simulator,” https://www.synopsys.com/verification/simulation/vcs.html, 2024.
  15. [15]Cadence Design Systems, Inc., “Cadence Xcelium: Logic simulation,” https://www.cadence.com/en_US/home/tools/system-design-and-verification/simulation-and-testbench-verification/xcelium-simulator.html, 2024.
  16. [16]Gemma Team, S. E. Abd et al., “Gemma 4 technical report,” arXiv preprint arXiv:2607.02770, 2026, https://arxiv.org/abs/2607.02770.
  17. [17]OpenAI, “gpt-oss-120b & gpt-oss-20b model card,” arXiv preprint arXiv:2508.10925, 2025, https://arxiv.org/abs/2508.10925.
  18. [18]DeepSeek-AI, “DeepSeek-V4: Towards highly efficient million-token context intelligence,” arXiv preprint arXiv:2606.19348, 2026, https://arxiv.org/abs/2606.19348.

Paper details

vlnc:W_zubj8hnf
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents