WorksComputer Science
Toward Full Autonomous Laboratory Instrumentation Control with Large Language Models
Abstract
The control of complex laboratory instrumentation often requires significant programming expertise, creating a barrier for researchers lacking computational skills. This work explores the potential of large language models (LLMs), such as ChatGPT, and LLM-based artificial intelligence (AI) agents to enable efficient programming and automation of scientific equipment. Through a case study involving the implementation of a setup that can be used as a single-pixel camera or a scanning photocurrent microscope, we demonstrate how ChatGPT can facilitate the creation of custom scripts for instrumentation control, significantly reducing the technical barrier for experimental customization. Building on this capability, we further illustrate how LLM-assisted tools can be extended into autonomous AI agents capable of independently operating laboratory instruments and iteratively refining control strategies. This approach underscores the transformative role of LLM-based tools and AI agents in democratizing laboratory automation and accelerating scientific progress.
This is the accepted manuscript of the following article:
Yong Xie, Kexin He, Andres Castellanos-Gomez,
“Toward full autonomous laboratory instrumentation control with large language models,”
Small Structures, 2025, 6(8), 2500173,
which has been published in final form at
https://doi.org/10.1002/sstr.202500173
Introduction
Scientific instrumentation is a cornerstone of modern research, enabling discoveries across a wide range of disciplines. Innovations such as the scanning tunneling microscope (STM), super-resolution microscopy, and transmission electron microscopy (TEM) have enabled atomic-scale imaging, advanced biological imaging, and structural analysis of quantum materials, respectively [1, 2, 3, 4, 5, 6, 7, 8, 9]. These breakthroughs exemplify how instrumentation drives progress by providing researchers with tools to probe the unknown.
However, while the development of new instruments is critical, effectively utilizing existing laboratory equipment remains challenging for many research groups. Scientific instruments often rely on sophisticated control software for workflow automation and data acquisition [10]. Developing such software traditionally demands significant programming expertise, creating a barrier for researchers who may lack computational backgrounds. Consequently, many laboratories depend on commercially available equipment with proprietary software. While these tools simplify basic operation, they often lack the flexibility needed for custom experimental setups, limiting their potential for innovation.
Recent advancements in artificial intelligence (AI) have revolutionized scientific workflows, enabling tasks such as predicting material properties, uncovering structure-property relationships, and accelerating materials discovery [11, 12, 13]. However, these AI-driven solutions frequently require custom software and programming expertise, rendering them inaccessible to groups without strong computational resources. This gap highlights the need for more accessible approaches to leveraging AI technologies for laboratory automation and instrumentation control.
Large language model (LLM)-based tools, such as ChatGPT, provide a promising solution to this challenge. These models have already demonstrated their utility in diverse research tasks, including drafting scientific manuscripts [14, 15], uncovering structure-property relationships [16], proposing novel scientific hypotheses [17, 18], and even contributing as peer reviewers [19, 20]. Beyond these applications, LLM-based tools offer an intuitive interface for generating and refining programming code, streamlining the automation of experimental workflows. By translating natural language instructions into executable commands, these models empower researchers to communicate effectively with scientific equipment, enabling custom workflows without extensive programming knowledge. For instance, systems like ORGANA (a robotic assistant that autonomously performs lab tasks based on natural language instructions) demonstrate how LLM-based tools can automate diverse chemistry experiments, reducing manual workload and enhancing efficiency [21]. Similarly, studies have highlighted the role of LLM-based tools in accelerating materials discovery through high-throughput experimentation and data-driven strategies. These developments signify a shift towards more autonomous, flexible, and efficient laboratory environments, aligning with the goals of self-driving labs and AI-assisted research [21, 22, 23, 24].
LLM-based tools can also address a critical pain point in laboratories: the difficulty of controlling specialized instruments that are poorly supported by user communities. To date, LLM-based programming has already demonstrated control capabilities over relatively simple devices (e.g. power supplies, robotic arms, or basic measurement units) typically through static scripts with limited interaction or adaptability [25, 26, 27]. More significantly, only a few studies have systematically explored the use of AI agents capable of autonomously interpreting user intent, managing experimental context, and executing complete control sequences, representing an essential step toward achieving intelligent and autonomous laboratory systems.
In this paper, we demonstrate the application of ChatGPT to streamline the control of commercially available scientific equipment. Through a case study involving the control of a set of hardware that can be used to implement a single-pixel camera system or a scanning photocurrent microscope, we illustrate how researchers can use LLM-based tools to iteratively develop functional scripts, simplifying the programming of complex setups. This approach highlights the potential of LLM-based tools to democratize laboratory automation and make advanced experimental configurations accessible to a wider scientific community.
Results and Discussion
We begin with the implementation of the hardware setup to demonstrate a single-pixel camera system, providing an illustrative example of how LLM-based tools can assist in hardware control and automation. Figure 1 outlines the pipeline for using LLM-based tools in instrumentation control. The process starts with the researcher selecting a programming language (MATLAB or Python) and planning the hardware configuration.

Figure 1. Workflow of the LLM (ChatGPT) based instrumentation control. After identifying the computer-instrument interfaces, the instruments are controlled using scripts generated in response to human prompts through a large language model (e.g., ChatGPT).
We assembled a minimal working setup to construct either a scanning photocurrent system or a single-pixel camera, both widely used in optoelectronics research [28].
In this setup, the Keithley 2450 source measure unit (SMU) applies a bias voltage and measures the current of a photodetector, while a motorized stage (Standa 8MTF-75LS05, 107291, 107293, Standa Controller model 8SMC4-USB-B9-2) provides precise XY movement. The communication protocols for the equipment were identified based on manufacturer documentation: the Keithley 2450 communicates via VISA protocol over USB connection, and the Standa XY stage communicates through a USB serial (COM) port.
Control scripts for both instruments were iteratively generated through interactions with ChatGPT. While the VISA-based communication of the Keithley SMU is widely supported by general-purpose instrument libraries, the Standa stage required handling a more specialized protocol. To demonstrate the versatility of the LLM-based approach, we applied the same prompt-driven code generation method to this non-standard interface, incorporating information from the manufacturer’s documentation into the LLM prompts and refining the code iteratively based on system feedback. Once data were collected, the datasets were saved and visualized using scripts refined through user prompts.
After assembling the hardware components and making the necessary connections, we outlined the system's operation to guide code generation. The XY stage moves the sample in a raster scan pattern, covering the entire area to be imaged. At each pixel position, the stage halts momentarily for measurement. The SMU applies a bias voltage to the CdS photodetector and measures the photocurrent generated by light reflected from the sample. A fiber-based reflection probe (Thorlabs RP24) connected to an LED is used for illumination and collection of reflected light. This photocurrent directly correlates with the intensity of light reflected from the sample at each pixel. As the scan progresses, variations in photocurrent are recorded, enabling the construction of a detailed reflectance map that highlights areas with differing optical properties and reveals surface features.
Figure 2a shows a flowchart that outlines how the XY stage is integrated with the Keithley 2450 source measurement unit, breaking the instrumentation control into distinct tasks. At each step in the flowchart, a user prompt generated by the LLM helps configure and test the relevant hardware control or data acquisition code. Figure 2b presents a typical example of such a prompt, which focuses on dividing the coding procedure into small, manageable segments. This segmentation enables rapid testing and iterative user feedback at each stage, ensuring that any issues can be identified and resolved before proceeding.

Figure 2. Typical prompts used in LLM-based instrumentation illustrate the STEP approach (Segment, Test, Evaluate, Proceed). (a) The flow chart of the process to combine the movement of stage with the Keithley 2450 source measurement unit. (b) The typical prompt used for the corresponding interaction with each block in (a). The key point is breaking the coding process into small steps so the user can quickly review and adjust the code through interaction with ChatGPT.
A key factor in this iterative approach is obtaining feedback early in the process to verify that the generated code functions correctly. To achieve this balance between efficiency and accuracy, the STEP Approach (Segment, Test, Evaluate, Proceed) was developed. The typical prompt “Can you write from the very basic, simple code, and step by step, stop at every step for me to confirm it works (and give me the feedback if it is successful, then proceed to the next step?” was employed during the first three steps shown in Figure 2. This approach demonstrates how LLM-based methods can simplify and streamline instrument control, even for users with limited programming experience.
Figure 3(a) illustrates the hardware configuration used to implement a single-pixel camera, which includes the fiber coupled LED light source, the Standa XY stage, the photodetector, and the Keithley 2450 source measurement unit. The optical path relies on a Thorlabs reflection probe (RP25) with a leg that directs and collects the light signal. Figure 3(b) provides a 3D rendering, showing the fiber positioned over the sample. The setup employs a “snake-like” raster scanning, as depicted in Figure 3(c). The code generated by ChatGPT acquires and plots the measured photocurrents, producing a final image of the scanned region. The inset of Figure 3(c) shows an actual photograph of the laser-cut aluminum foil mounted on black paper, which corresponds well with the features observed in the scanning photocurrent map. This demonstration highlights how LLM-based instrumentation can streamline optoelectronic measurements and data visualization, even for users with minimal coding experience. Scanning photocurrent mapping systems are widely used as spatially resolved tools for probing photoresponse channels [10]. Figure 4 illustrates the adaptation of the ChatGPT-based instrumentation to perform scanning photocurrent mapping with minor modifications to the setup. In this configuration, a fiber coupled LED light source illuminates with a focused spot, using a focusable collimator, onto the surface of a CdS photodetector. During the raster scanning process, photocurrent measurements are recorded using the Keithley 2450 while the Standa XY stage moves across the sample. The same ChatGPT-generated code was employed for this setup, enabling a "snake-like" raster scanning pattern with steps. The resulting photocurrent mapping, shown in Figure 4(c), clearly resolves spatial inhomogeneities in the photodetector's photocurrent response. This demonstration highlights the effectiveness of ChatGPT-based instrumentation for producing high-resolution, spatially resolved measurements, even in straightforward experimental setups.

Figure 3. The hardware and results of the scanning photocurrent image building using the ChatGPT-based instrumentation. (a) The hardware assembly of the LED, photodetector, and the scanning Standa XY stage connected with the Thorlabs reflection probe with linear leg (RP25). (b) A zoomed-in 3D representation of the optical fiber on the “2D Foundry” logo. A cartoon explaining the snake-like pattern raster scanning is shown besides. (c) The scanning photocurrent image captured by the code generated with ChatGPT. The inset shows a photograph of the test sample, fabricated by laser cut aluminum foil onto a black paper.

Figure 4. Scanning photocurrent mapping of a commercial photodetector using the ChatGPT-based instrumentation. (a) The hardware assembly includes an LED light source, a photodetector, and the scanning Standa XY stage, connected via a Thorlabs optical fiber (M28L03). (b) A close-up view of the collimator directing collimated light onto the CdS photodetector. (c) The photocurrent mapping image generated using code created with ChatGPT. The inset shows an optical image of the CdS photodetector.
To demonstrate the effectiveness of our methodology, we employed ChatGPT-4.1 to develop an autonomous AI agent capable for laboratory instrumentation control. The agent itself was built using the same iterative, prompt-driven approach described in Figure 2. As shown in Figure 5a, we began by establishing a Python interface to the OpenAI API and then used ChatGPT to generate the initial version of the agent code.

Figure 5. Towards a fully autonomous laboratory workflow for instrumentation control using an AI Agent. (a) Overview of the automated control pipeline: a human operator interacts with ChatGPT to generate code that interfaces with the Keithley 2450 SourceMeter via the OpenAI API. The system iteratively refines the code based on execution feedback. (b) Structure of the AI agent, including the initial system prompt and the autonomous execution loop guided by goal reminders. (c) Example Python code segment generated by the agent to identify VISA resources and initialize communication with the instrument. (d) Current–voltage (I–V) sweep measured using the final Python script produced by the agent, showing successful instrument control and data acquisition.
The resulting agent, illustrated in Figure 5b, operates in a closed loop: it sends prompts to the OpenAI API, receives Python code in response, executes the code to interact with a Keithley 2450 SourceMeter, and then uses the resulting output or error messages to formulate refined follow-up prompts. This process continues autonomously until the task is completed.
The agent’s operation relies on a structured sequence of messages sent to the OpenAI API: a system message defines its role and behavior (e.g., "You are an expert Python lab "automation agent..."), while user messages provide instructions and guide the task (e.g., "Start by listing VISA resources..." and "Continue building and refining the – sweep script until complete"). In this framework, the system message sets the assistant’s identity and constraints, whereas user messages simulate instructions from a human operator. Through this iterative dialogue, the agent progressively generated a series of executable scripts (autolab_code_iter*.py) to automate an – sweep with the Keithley 2450, as shown in Figure 5c. The final result, the – characteristics of a photoresistor, is shown in Figure 5d. Full agent logs and generated code are provided in the Supplementary Information.
While our ChatGPT-based agent demonstrates reliable performance in automating stepwise experimental tasks such as – sweeps, current LLMs are not yet optimized for real-time control or experiments requiring rapid feedback and sub-second decision-making. Additionally, LLM-based automation raises ethical and safety concerns, including the risk of misoperation, data security, and regulatory compliance issues. To mitigate these risks, we recommend testing generated code in sandboxed environments under human supervision, adhering to laboratory safety protocols. For applications involving sensitive data, the use of locally deployed models is advisable. Future developments may benefit from hybrid architectures combining LLM agents with real-time control systems and rule-based safety layers.
Conclusions
This study demonstrates the transformative potential of large language models, exemplified by ChatGPT, in laboratory automation. By enabling researchers to efficiently program and control complex instrumentation, LLMs address a critical challenge in experimental science. Our case study of a single-pixel camera system and scanning photocurrent mapping system underscores the feasibility and effectiveness of this approach, paving the way for broader adoption of AI-driven tools in scientific research. The integration of LLM-based tools into laboratory workflows holds the promise of fostering innovation and accessibility, driving a new era of flexible, efficient, and customized scientific experimentation.
MATERIALS AND METHODS
Code generation with AI language models: ChatGPT (GPT-4o, and o3, ChatGPT 4.1) has been used to generate the code used in this manuscript.
Supporting Information: Detailed code examples, ChatGPT prompt links, and a recommended workflow for end-to-end measurement automation for controlling a Keithley 2450 SourceMeter and a Standa XY stage in MATLAB. Sample scripts are also provided to facilitate replication of the experimental procedures for the AI agent. The supporting data and code are available at Zenodo: https://doi.org/10.5281/zenodo.15065601
AUTHOR INFORMATION
Corresponding Authors
(YX) [email protected] (ACG) [email protected]
Author Contributions
Y.X. and A.C.G. conceived the idea and designed the experiments. Y.X. performed the experiments. Y.X., K.H. and A.C.G. prepared the figures. Y.X. and A.C.G. analysed the data and wrote the manuscript.
Acknowledgements
This work was supported by the Open Fund of State Key Laboratory of Infrared Physics (Grant No. SITP-NLIST-ZD-2024-01). A.C-G. acknowledges funding by the Ministry of Science and Innovation (Spain) through the projects TED2021-132267B-I00, PID2020-115566RB-I00, PDC2023-145920-I00 and PID2023-151946OB-I00 and the European Research Council through the ERC-2024-PoC StEnSo (grant agreement 101185235) and the ERC-2024-SyG SKIN2DTRONICS (grant agreement 101167218). The author (Y.X.) acknowledges Prof. Ju Li (MIT) and Dr. Álvaro Blanco (ICMM-CSIC) for enlightening discussion.
References
- [1]G. Binnig, H. Rohrer, C. Gerber, E. Weibel, Tunneling through a controllable vacuum gap, Appl. Phys. Lett. (1982) 40, 178–180. https://doi.org/10.1063/1.92999
- [2]G. Binnig, H. Rohrer, Scanning tunneling microscopy, Surf. Sci. (1983),126 (1-3), 236–244. https://doi.org/10.1016/0039-6028(83)90716-1
- [3]G. Binnig, H. Rohrer, C. Gerber, E. Weibel, 7×7 Reconstruction on Si (111) resolved in real space, Phys. Rev. Lett. (1983), 50 (2), 120–123. https://doi.org/10.1103/PhysRevLett.50.120
- [4]E. Betzig, G.H. Patterson, R. Sougrat, O.W. Lindwasser, S. Olenych, J.S. Bonifacino, M.W. Davidson, J. Lippincott-Schwartz, H.F. Hess, Imaging intracellular fluorescent proteins at nanometer resolution, Science (2006), 313 (5793), 1642–1645. https://doi.org/10.1126/science.1127344
- [5]W.E. Moerner, L. Kador, Optical detection and spectroscopy of single molecules in a solid, Phys. Rev. Lett. (1989), 62 (21), 2535–2538. https://doi.org/10.1103/PhysRevLett.62.2535
- [6]J. Dubochet, J. Lepault, R. Freeman, J. Berriman, J.-C. Homo, Electron microscopy of frozen water and aqueous solutions, J. Microsc. (1982), 128 (3), 219–237. https://doi.org/10.1111/j.1365-2818.1982.tb04625.x
- [7]J. Frank, Averaging of low exposure electron micrographs of nonperiodic objects, Ultramicroscopy (1975), 8, 159–162. https://doi.org/10.1016/S0304-3991(75)80020-9
- [8]R. Henderson, P.N.T. Unwin, Three-dimensional model of purple membrane obtained by electron microscopy, Nature (1975), 257 (5521), 28–32. https://doi.org/10.1038/257028a0
- [9]S. Frank, P. Poncharal, Z.L. Wang, W.A. de Heer, Carbon Nanotube Quantum Resistors, Science (1998), 280 (5370),1744–1746. https://doi.org/10.1126/science.280.5370.1744
- [10]C. Reuter, R. Frisenda, D.Y. Lin, T.S. Ko, D. Perez de Lara, A. Castellanos-Gomez, A versatile scanning photocurrent mapping system to characterize optoelectronic devices based on 2D materials, Small Methods (2017), 1(7), 1700119. https://doi.org/10.1002/smtd.201700119
- [11]T. Dai, S. Vijayakrishnan, F. T. Szczypiński, J.-F. Ayme, E. Simaei, T. Fellowes, R. Clowes, L. Kotopanov, C. E. Shields, Z. Zhou, J. W. Ward, & A. I. Cooper, Autonomous mobile robots for exploratory synthetic chemistry, Nature (2024), 635, 890–897. https://doi.org/10.1038/s41586-024-08173-7
- [12]B. Hou, J. Wu, & D. Y. Qiu, Unsupervised representation learning of Kohn–Sham states and consequences for downstream predictions of many-body effects, Nature Communications (2024), 15, 9481. https://doi.org/10.1038/s41467-024-53748-7
- [13]H. Yang, R. Hu, H. Wu, X. He, Y. Zhou, Y. Xue, K. He, W. Hu, H. Chen, M. Gong, X. Zhang, P.-H. Tan, E. R. Hernández, Y. Xie, Identification and Structural Characterization of Twisted Atomically Thin Bilayer Materials by Deep Learning, Nano Lett. (2024), 24 (9), 2789–2797. https://doi.org/10.1021/acs.nanolett.3c04815
- [14]J.M. Buriak, D. Akinwande, N. Artzi, et al., Best Practices for Using AI When Writing Scientific Manuscripts, ACS Nano (2023), 17(5), 4091–4093. https://doi.org/10.1021/acsanano.3c01544
- [15]A. Castellanos-Gomez, Good Practices for Scientific Article Writing with ChatGPT and Other Artificial Intelligence Language Models, Nanomanufacturing (2023), 3(2), 135–138. https://doi.org/10.3390/nanomanufacturing3020009
- [16]X. Zhang, Z. Zhou, C. Ming, Y.-Y. Sun, GPT-Assisted Learning of Structure–Property Relationships by Graph Neural Networks: Application to Rare-Earth-Doped Phosphors, J. Phys. Chem. Lett. (2023), 14 (50), 11342–11349. https://doi.org/10.1021/acs.jpclett.3c02848
- [17]Y.J. Park, S.E. Jerng, S. Yoon, J. Li, 1.5 Million Materials Narratives Generated by Chatbots, Scientific Data, (2024), 11(1), 1060. https://doi.org/10.1038/s41597-024-03886-w
- [18]Y.J. Park, D. Kaplan, Z. Ren, C.-W. Hsu, C. Li, H. Xu, S. Li, J. Li, Can ChatGPT Be Used to Generate Scientific Hypotheses? J. Materiomics (2024), 10 (3), 578–584. https://doi.org/10.1016/j.jmat.2023.08.007
- [19]J.M. Buriak, M.C. Hersam, P.V. Kamat, Can ChatGPT and Other AI Bots Serve as Peer Reviewers? ACS Energy Lett. (2023), 9 (1), 191–192. https://pubs.acs.org/doi/10.1021/acsenergylett.3c02586DOI
- [20]Z. Ren, Z. Ren, Z. Zhang, T. Buonassisi, J. Li, Autonomous experiments using active learning and AI, Nature Reviews Materials (2023), 8 (9), 563–564. https://doi.org/10.1038/s41578-023-00588-4
- [21]K. Darvish, M. Skreta, Y. Zhao, Y. Naruki, S. Sagnik, B. Miroslav, C. Yang, H. Han, X. Haoping, A. Alán, G. Animesh, S. Florian, ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterization, arXiv preprint (2024) arXiv:2401.06949. https://doi.org/10.48550/arXiv.2401.06949
- [22]S. Tao, L. Man, Z. Xiao, C. Lin, H. Yan, C. Jia, Z. Qing, L. Dao, Z. Bai, Z. Gang, Z. Guo, Z. Fei, S. Wei, F. Yao, J. Jiang, L. Yi, A multiagent-driven robotic ai chemist enabling autonomous chemical research on demand, Journal of the American Chemical Society, (2025), 147(15),12534-12545. https://doi.org/10.1021/jacs.4c17738
- [23]X. Shan, Y. Pan, F. Cai, H. Gao, J. Xu, D. Liu, Q. Zhu, P. Li, Z. Jin, J. Jiang, M. Zhou, Accelerating the Discovery of Efficient High-Entropy Alloy Electrocatalysts: High-Throughput Experimentation and Data-Driven Strategies, Nano Lett. (2024) 24(37), 11632-11640. https://doi.org/10.1021/acs.nanolett.4c03208
- [24]Y. Pan, X. Shan, F. Cai, H. Gao, J. Xu, M. Zhou, Accelerating the Discovery of Oxygen Reduction Electrocatalysts: High - Throughput Screening of Element Combinations in Pt - Based High - Entropy Alloys, Angewandte Chemie International Edition, (2024), 63(37) , e202407116. https://doi.org/10.1002/anie.202407116
- [25]D. Fébba, K. Egbo, W.A. Callahan, A. Zakutayev, From text to test: AI-generated control software for materials science instruments, Digital Discovery (2025), 4 , 35–45. https://www.sciencedirect.com/science/article/pii/S2635098X24002134DOI
- [26]D.A. Boiko, R. MacKnight, B. Kline, G. Gomes, Autonomous chemical research with large language models, Nature (2023), 624, 570–578. https://www.nature.com/articles/s41586-023-06792-0DOI
- [27]N. Yoshikawa, M. Skreta, K. Darvish, S. Arellano-Rubach, Z. Ji, L.B. Kristensen, A.Z. Li, Y. Zhao, H. Xu, A. Kuramshin, A. Aspuru-Guzik, F. Shkurti, A. Garg, Large language models for chemistry robotics, Autonomous Robots (2023), 47, 1057–1086. https://link.springer.com/article/10.1007/s10514-023-10136-2DOI
- [28]S. G. Baird, T. D. Sparks, What is a minimal working example for a self-driving laboratory? Matter(2022), 5 (12), 4170–4178. https://doi.org/10.1016/j.matt.2022.11.007