betaFeedback

WorksPhysics and Astronomy

Generating a simple silicon detector simulation with an open-weight large language model

Abstract

Recent LLM developments have seen AI-generated code become commonplace in the software development industry. Without tool access, LLMs cannot execute simulations, and their knowledge of toolkits such as Geant4 is limited. We demonstrate here, a detector simulation workflow with a Qwen3.6-27B model with Q4_K_M GGUF quantisation together with custom resources and tools served via MCP servers to generate a particle detector simulation. In 50 independent runs, 42(29–56)% at 95% CI produced energy-deposit distributions statistically consistent with a reference simulation, measured against a reference simulation. These results provide a proof of concept that MC simulations can be generated using text-based prompts with an LLM, given the necessary resources and tools. More extensive resources, such as example code and additional documentation, are likely to be required for more complex simulation tasks.

Introduction

LLMs are increasingly being used for automating time-consuming and labour-intensive software tasks, such as code generation. LLM agents have been used to automate parts of particle physics data analysis [1, 2, 3]. However, their use for directly generating code and running MC simulations of physics detectors, which are central to particle physics, has received less attention [4, 5]. Such simulations are part of the detector design and analysis steps of almost all experiments to understand the event features of signal processes and their backgrounds. This is crucial for relating experimentally measured variables to the parameters of the underlying theory. MC simulations require expertise in both software frameworks and particle physics. Moreover, detector design changes take significant time to incorporate into the simulation and require further software debugging and physics validation. In this paper, we demonstrate the potential use and motivate future studies of LLMs to generate particle detector MC simulations from text prompts when given access to tools and resources.

LLM-based assistants have been shown to increase developer productivity in field experiments [6]. Applied to MC simulation, an LLM that generates, runs and checks simulation code could offer several benefits. First, it lowers the barrier of entry. A detector configuration described in plain text can be translated into a working simulation without detailed knowledge of the toolkit’s syntax, allowing physicists without simulation expertise explore design options. Second, it shortens the iteration cycle. Changes to geometry, materials or beam conditions can be requested in a sentence rather than implemented and debugged by hand. Third, the generated configuration files remain human-readable. So every simulation can be inspected, version-controlled and rerun. Finally, coupling the model to automated checks, such as syntax validation and statistical comparison with reference distributions, catches many errors before a person reviews the output. Running an open-weight model locally adds further advantages. Proprietary detector designs never leave the institution, there is no per-token cost, and the exact model version can be fixed so that results stay reproducible. These benefits depend on the generated simulations being correct, so physics validation by an expert remains essential.

Methodology

We construct a framework that turns a user prompt into a histogram of a simulated observable, a step common to many detector studies. The user-facing interface can be managed by any messaging service with endpoints that supports bot2 accounts. The MC simulations are created using TOol for PArticle Simulation (TOPAS) [7, 8], a Geant4 simulation toolkit [9, 10, 11] wrapper with a text-based parameter control system that allows users to assemble and control simulation objects, scorers, and particle sources. The text-based nature of the TOPAS parameter files is well suited to LLM text generation, since it usually requires no Geant4 C++ code.

Qwen3.6-27B [12] is chosen as the open-weight model. It initialises with a system prompt preloading instructions as detailed in Appendix § 6. An MCP [13] server and the messaging service bot are started. Once the server and scripts are running, the messaging service bot is ready to accept user prompts. The bot receives a prompt sent via the messaging service, and Qwen3.6-27B decides iteratively if it requires tools from the MCP server based on the prompt. These include the necessary functions for creating TOPAS files, syntax checking, running TOPAS simulations, reading ROOT output files, running Kolmogorov-Smirnov (KS) tests, and uploading images to the messaging service. The results of the tools are returned to the model at each iteration. The final response is posted by the bot on the messaging system.

A generated reference simulation of an electron beam on silicon (Figure 1) is chosen for its simplicity as the test case for debugging and development. From the initial user prompt shown in Appendix § 7, a TOPAS parameter file is generated on the server. The syntax of this parameter file is checked using tools available to the LLM via an MCP server.

Energy deposit histogram for a 10 MeV electron beam on silicon

Figure 1. Energy deposit of a 10 MeV electron beam on 2 mm thick silicon. 10510^{5} histories generated from a Gaussian beam with 0.01 cm spread.

Large Language Model

Ollama3 [14] is an open-source software platform that allows the download, management, and running of LLMs locally rather than relying on external cloud servers. It is run on an NVIDIA A100 80GB PCIe GPU [15]. This allows control over model parameters, system prompts, tool calling, model selection, and hardware resources. The Qwen3.6-27B Q4_K_M GGUF quantisation model with thinking mode enabled was used both to develop the resources and tools and for all evaluation runs. It is further configured to generate and run MC simulations by using a model file to define the sampling parameters and the system prompt (Appendix § 6). The user interacts with the LLM through the messaging service. Outputs such as plain text, images in the form of histograms and graphs, and other file types, including text, Python, and C++ files, can be sent during the LLM’s processing loop using dedicated MCP tools.

Tools and Resources

LLMs like Qwen3.6-27B are trained on a large text corpus to produce text output. The use of tools allows the model to complete tasks such as exact calculations, manipulating local files, and executing code [16]. Qwen3.6-27B can generate and run MC simulations when given tools to create and execute files in a working directory, run TOPAS parameter files, and read ROOT output.

Instructions are preloaded into Qwen3.6-27B’s context window before it accepts a user prompt to improve the model’s knowledge of detector simulations. These resources include technical documentation, examples, and rules. These also provide the model with a workflow for completing tasks and may directly instruct the model to run tools. Generating valid TOPAS syntax requires lists of parameter and material names, default simulation parameters, and rules for creating the files. The workflow is executed using tools, see Figure 2, in the order defined by a workflow resource, accessed through an MCP server.

Workflow for generating and evaluating a TOPAS parameter file

Figure 2. As the user prompts the LLM, it will access the resources and tools available to it via an MCP server for generation of the TOPAS parameter file. KS compares the generated energy-deposit distribution with the reference.

First, Qwen3.6-27B evaluates the simulation request and determines which parts of the TOPAS documentation are relevant. The model then generates the parameter file and verifies it with a syntax-checking tool. The syntax checker identifies mistakes in TOPAS parameter files, such as invalid data types, duplicate parameters, incorrectly sized vectors, missing parent components, invalid component references, incorrectly defined TOPAS PhaseSpace scorers, and common issues with materials, physics modules and Boolean parameters. However, it does not check whether the simulation is physically correct.

If the syntax checker returns with no errors, TOPAS is run with the generated parameter file. If syntax, tool, or runtime errors occur, Qwen3.6-27B re-evaluates the error and returns to the evaluation step to fix it. It then finds the energy deposit histogram within the ROOT file. A KS test is performed to compare between model generated and the reference histograms. The generated histogram is later saved as an image, which is then uploaded to the messaging service using MCP tools. The model then summarises its actions to the user as a final response.

Implementation

Prompts are received from the messaging service via its dedicated endpoint, and then passed to the LLM workflow. The workflow described in Algorithm 1 details the process of loading tools and resources, then iterating tool calls until either the prompt goal is achieved (i.e., no tool calls remain) or Nmax⁡=5N_{\max}=5 attempts are reached. If the former condition ends the loop, then a summary of the LLM’s actions is returned to the user. An MCP server provides the LLM access to pre-built Python tools and resources. The results of each prompt are automatically uploaded to a database to track the results of the simulation. The data includes the model name, prompt text, response text, temperature, top-p, top-k, total context limit, response token limit, output time, output tokens, and the result of the KS test.

In the Qwen3.6-27B model file, we set the sampling parameters to the values in Table 1.

Parametervalue
temperature0.15
num_predict15872
num_ctx524288
top-p0.3
top-k40

Table 1. Sampling parameter choice for running the Qwen3.6-27B.

Temperature [17]: This controls how predictably the model chooses the next token. A model’s unnormalised scores for each possible next token are turned into probabilities (softmax). A value above 1.0 flattens the probability distribution, and a value below 1.0 concentrates on the tokens with the highest probabilities. A higher temperature can give more varied responses; this may help with problem-solving, but it also makes nonsensical output more likely.

Nucleus sampling, top-p [18]: Only the smallest set of tokens whose cumulative probability is at least p is kept. The number of candidate tokens therefore changes from step to step with the shape of the distribution.

top-k sampling [19]: Only the k most probable tokens are kept at each generation step, and the next token is sampled from them.

Context Window, num_ctx: An integer giving the maximum number of tokens the model can attend to at once, counting the prompt, the conversation history, tool outputs and the response. If this is too small, model context is truncated.

Response token limit, num_predict: The maximum number of tokens the model may output in reply to a prompt. It is not the same as the total used by a single prompt, because a prompt that involves several tool calls may query the model several times. As tokens are generated sequentially, and this decoding phase dominates inference time [20], a lower num_predict reduces the time per response and the context carried into subsequent tool-call iterations. If the limit is too small, the response, including the model’s reasoning when thinking mode is enabled, is truncated.

Algorithm 1 Simple particle detector MC simulation with LLM
1: Receive user request
2: accepted ← false
3: attempts ← 0
4: Nmax ← 5
5: while not accepted and attempts < Nmax do
6: Evaluate request and any feedback with model
7: Generate TOPAS code with model
8: Check code with TOPAS syntax checker
9: if Syntax errors then
10: Return syntax errors to model as feedback
11: attempts ← attempts + 1
12: continue
13: end if
14: Run TOPAS
15: if Runtime error then
16: Return runtime error to model as feedback
17: attempts ← attempts + 1
18: continue
19: end if
20: Compute KS test, pKS
21: if pKS > 0.01 then
22: accepted ← true
23: else
24: Return pKS to model as feedback
25: attempts ← attempts + 1
26: end if
27: end while
28: Get ROOT objects
29: Upload file to messaging service
30: Request summary from model
31: return model summary

Results and discussion

Each model-generated simulation is compared with a reference simulation of the same set-up. Every run generates 10510^{5} histories with the Geant4 g4em-standard_opt4 electromagnetic physics constructor. The total energy deposited per history in the generated and reference simulations is compared with a two-sample KS test. A run is counted as successful if pKS>0.01p_{\mathrm{KS}} > 0.01, i.e. if the null hypothesis that both samples are drawn from the same distribution is not rejected at the 1% level.

A prompt is evaluated with 50 independent runs (Appendix § 7). The generated simulation passed the KS test in 21/50 runs (42%, 95% CI (29–56)%). Intervals are Wilson score intervals. The outcome of the run is given in Table 2.

Outcomee−e^{-} / Si
Passed (pKS>0.01p_{\mathrm{KS} } > 0.01)21
Rejected (pKS≤0.01p_{\mathrm{KS} } \le0.01)5
Simulation ran, no KS result4
Context window exhausted16
Unresolved syntax or runtime error4
Total50
Success rate (95% CI) [%]42 (29–56)

Table 2. Success and failure statistics over 50 runs of the electron prompt.

Figure Figure 3 shows an accepted (pKS=0.9970p_{\mathrm{KS}} = 0.9970) and a rejected (pKS=0.0022p_{\mathrm{KS}} = 0.0022) simulation generated in response to the electron prompt. The two distributions have the same mean and their widths differ by about 2%, showing that with 10510^{5} histories the test is sensitive to small differences in shape. Several runs gave identical pKSp_{\mathrm{KS}} values; these produced identical parameter files, and with the default TOPAS random seed identical files give identical output.

Figure 3. Accepted (pKS=0.997p_{\mathrm{KS}} = 0.997) and a rejected (pKS=0.0022p_{\mathrm{KS}} = 0.0022) simulation generated in response to the electron prompt.

The most common erroneous outcome for the prompt was exhausting the context window, which ended 16 of the 50 runs, typically after repeated documentation look-ups. The model also frequently lost track of the names of files it had generated and called a directory-listing tool to recover them, adding further tool output to the context. A further 4 runs completed the simulation but recorded no KS result, usually because the model did not locate the reference histogram.

The median wall-clock time per run was 620 s. 352 s for successful runs and 691 s for unsuccessful ones. The 50 runs took 9.0 h of wall-clock time in total. They were served by one bot instance, which shared the A100 GPU with two other instances running the photon and proton prompts, so individual times include contention for the GPU. Successful runs ended with a median context of 88k tokens and unsuccessful runs with 182k; the 16 runs that exhausted the context window ended at 159k–219k tokens.

The prompt and its reference simulation were used both to develop the tools and resources and to evaluate them. Its success rate may therefore be optimistic for prompts not seen during development.

Conclusions

Qwen3.6-27B together with MCP tools and access to resources on TOPAS particle detector simulation can produce simulations whose energy-deposit distribution is statistically consistent with the reference. The simulation setup has a modest success rate of 42(29–56)% at 95% CI for a simple prompt detailing electrons passing through a silicon detector. We hypothesise that without access to the TOPAS documentation, Qwen3.6-27B is expected to have limited knowledge of TOPAS parameter-file syntax. In our runs, syntax errors were the most common failure mode. MCP tools and relevant documentation are intended to provide background knowledge and specialised ability. More detailed resources such as example code, further TOPAS documentation, and additional workflows for managing different requests might be required to generate more complex simulations.

Future work would include systematic tuning of sampling parameters with larger samples. Another natural extension of this work would be testing other open-weight LLMs and comparing with closed-weight models such as Claude, GPT, and Gemini subject to cost and data-policy constraints. This work provides an initial demonstration that an open-weight model can generate simple MC simulations, although the present study was limited by available compute and evaluation time.

Acknowledgements

This work was supported by the EPSRC 10-week summer internship award reference EP/Z535102/1 provided for Charlie Bamber. We thank Paras Naik for a careful reading and useful comments. We also acknowledge the support of the UK Science and Technology Facilities Council (STFC).

Model file and system prompt

FROM qwen3.6:27b
PARAMETER temperature 0.15
PARAMETER top_p 0.3
PARAMETER top_k 40
PARAMETER num_ctx 524288
PARAMETER num_predict 15872
SYSTEM """
- When discussing particles, refer to their spin, mass and charge.
- Use LaTeX for all mathematical expressions and quantum state notation.
- Think very carefully before calling functions.
- If running ROOT, run with root -b -q.
- Never use emojis
- Always tell the user explicitly the names of all files generated and their paths.
Do this with any prompt and dont create any files without informing the user.
- Refer to tools and resources when preforming simulations, never make predictions
based on training data.
- Inform the user of any and all errors encoutered in either python or topas code.
- If an errors is encountered in a generated topas parameter, write a general
description of the error using Edit_Topas_Errors_File to avoid the issue in the
future.
- A simulation producing 0 histories is always an error. Do not report it as
success.
- The correct parameter for number of events is: i:Ts/NumberOfHistoriesInRun
- Never use a custom source component without verifying it exists in the geometry.
- Use So/Demo as the particle source unless explicitly told otherwise.
- Always save the raw data in the form of energy, xyz positions, and time.
- When completing simulations return a histogram of the results (of energy unless
otherwise specified)
You have access to tools. You MUST follow these rules without exception:
RULE 1 - FILE CREATION IS ALWAYS A TOOL CALL:
If the user asks you to save, write, create, or generate a file:
- You MUST call create_file.
- You MUST NOT output the file contents as plain text.
- Generating content in text and then not calling create_file is a FAILURE.
- This rule applies even if the file content is long or complex.
RULE 2 - THINK BEFORE CALLING:
  Plan the tool call sequence before acting.
RULE 3 - CHAIN TOOLS EXPLICITLY:
  Evaluate whether another tool call is required before responding.
  Produce a final text response when all tool calls for the task are complete.
RULE 4 - DECLARE TOOL USE VS INFERENCE:
  When presenting information, state explicitly whether it comes from a tool call
  result, or a statistical prediction based on training data.
"""

Electron Beam and Silicon Detector

The prompt describes a 10 MeV electron beam impinging on a 2 mm thick silicon detector.

Create and run a TOPAS 3.9 simulation of a 10 MeV electron beam striking a silicon detector.

Use 100 histories for an initial test. If the geometry and outputs are correct, run 100,000 histories.

Simulation setup:

  • Vacuum world with 5 m clearance in each direction

  • Silicon detector centred at the origin

  • Detector dimensions: 2×2×0.22 \times2 \times0.2 cm

  • Electron source positioned 10 cm in front of the detector along the negative z-axis

  • Beam directed normally toward the detector

  • Gaussian transverse beam position with a 0.01 cm spread

  • Use the g4em-standard_opt4 EM module

  • No graphics

  • For the full run set ShowHistoryCountAtInterval to 10000 to reduce the output tokens

  • Produce a 100-bin, 0–12 MeV histogram of total energy deposited in the detector

  • Use only standard TOPAS 3.9 features. Do not use unsupported custom scorers

After running:

  • Confirm that the simulation completed successfully

  • List the generated ROOT files and their exact histogram and TTree keys

  • Confirm that both outputs are non-empty

  • Compare the deposited-energy histogram with the reference histogram using a two-sample Kolmogorov–Smirnov test

  • Report the KS statistic and p-value

  • Convert the generated energy histogram to a labelled PNG and upload it to Mattermost

References

  1. [1]Eli Gendreau-Distler et al. “Automating High Energy Physics Data Analysis with LLM-Powered Agents”. In: arXiv preprint (Dec. 2025). DOI: 10.48550/arXiv.2512.07785. arXiv: 2512.07785 [physics.data-an].DOI
  2. [2]Eric A. Moreno et al. “AI Agents Can Already Autonomously Perform Experimental High Energy Physics”. In: arXiv preprint (Mar. 2026). DOI: 10.48550/arXiv.2603.20179. arXiv: 2603.20179 [hep-ex].DOI
  3. [3]Sascha Diefenbacher et al. “Agents of Discovery”. In: SciPost Phys. 20.3 (2026), p. 091. DOI: 10.21468/SciPostPhys.20.3.091. arXiv: 2509.08535 [hep-ph].DOI
  4. [4]Wonyong Chung et al. “Agentic AI Detector Co-design and Optimization in Vertically Integrated Full Simulations”. In: arXiv preprint (Apr. 2026). DOI: 10.48550/arXiv.2604.21804. arXiv: 2604.21804 [physics.ins-det].DOI
  5. [5]Sara Zoccheddu et al. “Large language models for physics instrument design”. In: Machine Learning: Science and Technology 7.4 (July 2026), p. 045005. DOI: 10.1088/2632-2153/ae7cdb.DOI
  6. [6]Kevin Zhe Yuan Cui et al. “The effects of generative AI on high-skilled work: Evidence from three field experiments with software developers”. In: Management Science 72.11 (Nov. 2026), pp. 9408–9420. DOI: 10.1287/mnsc.2025.00535.DOI
  7. [7]J. Perl et al. “TOPAS: An innovative proton Monte Carlo platform for research and clinical applications”. In: Medical Physics 39.11 (2012), pp. 6818–6837. DOI: 10.1118/1.4758060.DOI
  8. [8]Bruce Faddegon et al. “The TOPAS tool for particle simulation, a Monte Carlo simulation tool for physics, biology and clinical research”. In: Physica Medica 72 (2020), pp. 114–121. DOI: 10.1016/j.ejmp.2020.03.019.DOI
  9. [9]S. Agostinelli et al. “GEANT4: a simulation toolkit”. In: Nucl. Instrum. Meth. A 506 (2003), pp. 250–303. DOI: 10.1016/S0168-9002(03)01368-8.DOI
  10. [10]J. Allison et al. “Geant4 developments and applications”. In: IEEE Trans. Nucl. Sci. 53 (2006), p. 270. DOI: 10.1109/TNS.2006.869826.DOI
  11. [11]J. Allison et al. “Recent developments in Geant4”. In: Nucl. Instrum. Meth. A 835 (2016), pp. 186–225. DOI: 10.1016/j.nima.2016.06.125.DOI
  12. [12]Qwen Team. Qwen3 Technical Report. 2025. DOI: 10.48550/arXiv.2505.09388. arXiv: 2505.09388 [cs.CL].
  13. [13]Model Context Protocol. Model Context Protocol Specification, version 2025-11-25. https://modelcontextprotocol.io/specification/2025-11-25. Accessed: 24 July 2026. 2025.
  14. [14]Ollama. Ollama Documentation. https://docs.ollama.com/. Accessed: 4 August 2026. 2026.
  15. [15]NVIDIA Corporation. NVIDIA A100 80GB PCIe GPU Product Brief. Tech. rep. PB-10577-001_v03. Accessed: 4 August 2026. NVIDIA Corporation, Mar. 2022. URL: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/PB-10577-001_v03.pdf.
  16. [16]Timo Schick et al. “Toolformer: Language Models Can Teach Themselves to Use Tools”. In: Advances in Neural Information Processing Systems. Vol. 36. 2023, pp. 68539–68551. arXiv: 2302.04761.arxiv.org/abs/2302.04761
  17. [17]Lujun Li et al. “Exploring the Impact of Temperature on Large Language Models: Hot or Cold?” In: Procedia Computer Science 264 (2025), pp. 242–251. arXiv: 2506.07295.arxiv.org/abs/2506.07295
  18. [18]Ari Holtzman et al. “The Curious Case of Neural Text Degeneration”. In: International Conference on Learning Representations. 2020. URL: https://openreview.net/forum?id=rygGQyrFvH.
  19. [19]Angela Fan, Mike Lewis, and Yann Dauphin. “Hierarchical Neural Story Generation”. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018, pp. 889–898. DOI: 10.18653/v1/P18-1082.DOI
  20. [20]Pratyush Patel et al. “Splitwise: Efficient Generative LLM Inference Using Phase Splitting”. In: 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEEE, 2024, pp. 118–132.

Paper details

vlnc:W_wg47hq95
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents