WorksPhysics and Astronomy
Precise large-scale chemical transformations on surfaces: deep learning meets scanning probe microscopy with interpretability
Abstract
Scanning Probe Microscopy (SPM) techniques have shown great potential in fabricating nanoscale structures endowed with exotic quantum properties achieved through various manipulations of atoms and molecules. However, precise control requires extensive domain knowledge, which is not necessarily transferable to new systems and cannot be readily extended to large-scale operations. Therefore, efficient and autonomous SPM techniques are needed to learn optimal strategies for new systems, in particular for the challenge of controlling chemical reactions and hence offering a route to precise atomic and molecular construction. In this paper, we developed a software infrastructure named AutoOSS (Autonomous On-Surface Synthesis) to automate bromine removal from hundreds of Zn(II)-5,15-bis(4-bromo-2,6-dimethylphenyl)porphyrin (\ch{ZnBr2Me4DPP}) on Au(111), using neural network models to interpret STM outputs and deep reinforcement learning models to optimize manipulation parameters. This is further supported by Bayesian Optimization Structure Search (BOSS) and Density Functional Theory (DFT) computations to explore 3D structures and reaction mechanisms based on STM images.
Introduction
Precisely and controllably manipulating atoms or molecules on surfaces offers the potential for assembling nanomaterials with tuneable exotic properties for novel applications in optoelectronics and spintronics. [1, 2, 3, 4, 5, 6, 7] Recently, scanning probing microscopy (SPM), including scanning tunnelling microscopy (STM) and atomic force microscopy (AFM), has shown great potential in nanofabrication through complex manipulations including pulling, pushing, pick–transfer–drop and dissociation. [8, 9, 10, 11, 12] These manipulations are predominantly controlled through the tip position (tip, tip, tip), bias voltage (V) and tunnelling current (I) in STM. However, the selection and optimization of such parameters is a time-consuming and repetitive process and strongly depends on the domain knowledge, which is not necessarily transferable to new systems. Therefore, efficient and autonomous SPM techniques are needed to reduce the reliance on human supervision and efficiently learn optimal strategies for the fabrication of functional nanostructures, particularly to the scale that would have an impact in real technologies.
Advanced machine learning techniques, especially image classification, image segmentation and reinforcement learning (RL), have recently emerged as promising methods to automate various tasks in SPM, including the identification of optimal sample regions, the evaluation of the quality of scanning images, tip conditioning and the selection of manipulation parameters and the detection of reaction sites. [13, 14, 15, 16, 17, 18, 19] For example, RL decision-making agents have been developed using discrete actions to find the proper trajectories to lift a large molecule, [20] and also to laterally manipulate a polar molecule. [21] In contrast to making decisions within a set of discrete actions, Chen and coworkers developed a Deep Reinforcement Learning (DRL) approach capable of selecting parameters from continuous action space including tip-start and -end positions, bias voltage and tunnelling conductance to steer the motion of atoms. [22] The advancements in SPM automation mentioned above pave the way for the next step in nanostructure assembly – the automation of chemical reactions.
For the engineering of new organic materials, on-surface synthesis (OSS), which is based on chemical reactions, has developed into a powerful tool for the controllable formation of molecular structures on solid surfaces. [23] In particular, the ability to control chemical reactions using temperature [24] and light [25] in combination with careful selection of molecular precursors has allowed for breakthrough work in the fabrication of carbon nanostructures and organic molecular networks. [26] Partnered with the high-resolution characterisation SPM offers, sequences of on-surface reactions now provide molecular assembly options that are impossible in solution. [27, 28, 29, 30] Alongside this, the concept of single molecule engineering, to control all the elementary steps of a molecular chemical reaction via SPM manipulations, was first introduced in 2000. [31] Yet the potential of SPM for single molecule engineering has only emerged in recent years. [28, 32, 33, 34, 35] Despite these exciting results, it is clear that the technical challenges and time demands of manual manipulation approaches are not suitable for fabrication beyond a few molecules, and scaling these procedures beyond single manipulations and reactions to fabricate large molecular assemblies and engineer complex electronic states requires autonomous SPM operation. [30, 36]
In this paper, we establish a deep learning workflow to automate STM manipulations and optimize manipulation parameters to efficiently and selectively break C–Br bonds in organobromides. Breaking these bonds is the first step of the Ullmann reaction, [23] and an important intermediary step in OSS of complex molecules. This is then applied to -5,15-bis(4-bromo-2,6-dimethylphenyl)porphyrin () on Au(111) as a model system to study autonomous tip-induced reactions in STM. Meanwhile, Density Functional Theory (DFT) [37] calculations and Bayesian Optimization Structure Search (BOSS) [38] serve as an auxiliary tool to explore adsorption structures and reaction mechanisms in combination with SPM results and DRL models.
Results and discussion
The overall architecture of our software infrastructure AutoOSS (Automated On-Surface Synthesis) consists of three components (Figure 1): Target detection–search and identify targeted fragments based on STM images; Interpretation—Models to interpret the STM output during manipulation; Decision-making—DRL agent for selecting SPM parameters.

Figure 1. AutoOSS workflow. AutoOSS consists of three key modules: target detection, decision-making and interpretation. The target detection module is responsible for detecting individual candidate molecules from a larger scanning image by evaluating distances and areas of image contrast. The interpretation module aims at understanding the effect of manipulation parameters implemented on molecules through identifying products based on STM output (images and signals). The identification of products determines the next step. The decision-making module generates manipulation parameters. Here, we primarily employed two methods—a random generator and a DRL approach. The DRL approach searches for optimal STM manipulation parameters towards a goal using a reward system based on the state. Finally, a substantive number of 2D scanning images, reflecting various configurations of molecules on , collected during the whole process, can be used to analyze the geometric and electronic structures and potential reaction mechanisms with BOSS and DFT.
Target Detection
To efficiently detect promising candidate molecules to test C-Br bond dissociation, we acquired an STM image containing several molecules and molecular clusters (see Methods for details about the sample preparation). We then analyzed the distance between them (default: 2.5 nm, comparable to the size of molecules) and the area of the associated contrast patterns (default: 1.5 - 2.5 nm2) to exclude clusters or fragmented molecules in Figure 2a. However, many individual fragments share similar areas, especially the dissociated products resulting from the loss of one or two bromine atoms (Figure S9), which are hard to distinguish from one another. Therefore, we developed a neural network model to identify molecules more precisely based on magnified images focusing on the targeted patterns, where we zoomed in on a smaller scanning region of nm – still large enough to accommodate the target molecules measuring around 2.3 nm (Figure 2b). Furthermore, the complexity introduced by adsorbing a non-planar 3D structure onto a 2D surface, where the target molecule can undergo rotations and bind to the substrate at various sites and configurations, inevitably leads to diversity in observed STM contrasts. To understand the features of the molecule for target detection purposes, we correlated the observed STM contrast with multiple configurations aided by simulated STM images (see Methods and Figure S5). Among these, we found that the most commonly seen contrast patterns in STM images (four lobes (2, 3, 4, 5) symmetrically around a larger lobe (1) in Figure 2c, Figure 2d) match well with the three most stable adsorption structures (structures 1, 38, 73, 110, 115, 150 and 158 in Figure S5), which have nearly isoenergetic computed energies ranging from -2.43 eV to -2.32 eV.

Figure 2. Search and identification of . (a) Detection of individual molecules from large images by the distance between molecules and the area of the contour. The blue points indicate the detected molecules in contours. Red points represent the centre point of contours, whose areas are marked by values. (b) Architecture of the neural network used to predict whether the image includes an individual . (c) Example of targeted contrast patterns in STM images at different rotation angles. (d) 3D view of DFT simulated STM with superimposed molecular structure. (e) Side view of adsorbed on .

Figure S5. Various configurations of adsorbed on searched by BOSS. The energy diagram represents the searched structures and corresponding adsorption energies obtained from BOSS. Some stable configurations with possible corresponding STM scanning images are shown. From left to right, the columns represent 3D structures optimized by DFT calculations from top view and from side view, DFT-simulated STM, experimental scanning images (setpoint: A, bias voltage: 1 V). The values on the side structures indicate the adsorption energies obtained by DFT calculations.

Figure S9. Properties of contrast patterns in STM images. (a) Distribution of areas of patterns of pristine and their corresponding products ( or ) after implementing manipulations. (b) Distributions of angles, (c) heights and widths for the approximated ellipses from patterns of reactants . Data obtained by analysing 209 pairs of images among successful cases.
To improve the ability of the models in identifying molecules, we defined the most frequently observed molecular features in the STM images as target objects while allowing minor deviations in tip condition and molecular rotation (Figure 2c). The 3D structure and corresponding STM images (Figure 2d, Figure 2e) revealed that the central lobe 1 represents the upper periphery of the porphyrin ring, the two lobes of 2 and 3 at the ends are partly due to the presence of Br atoms, and the other two lobes of 4 and 5 originate mainly from the methyl fragments on the phenyl ring. Based on these characteristics, we manually constructed a labelled dataset of 1350 images, and an image classifier (Figure 2b) was trained to evaluate whether the scanning image includes an individual – the ultimate accuracy of the model was 98.5% on the test dataset (more details in the method and Figure S12).

Figure S12. Performance metrics of for distinguishing or not when detecting targeted reactants, with 1389 samples in the training dataset and 198 samples in the test dataset.
Interpretation
After finding and identifying the target molecules, we are in a position to initiate the C–Br dissociation process by placing the STM tip on a specific site and applying a voltage bias (ramp pattern or pulse pattern, details shown in Figure S16) and a current. Varying parameters among these four (, , V and I) may lead to various effects on the molecules, as shown by the representative selection in Figure 3a. The dissociation of the C–Br bond(s) – resulting in the corresponding dissociated molecules and – is the goal of the manipulation. However, as reflected in STM images, there are multiple possible outcomes of the manipulation process. For example, the contrast pattern of a Br atom (lobe 2 or lobe 3 in Figure 2d) may disappear or co-exist near the contrast patterns of or . Besides, the appearance and contrast of these patterns may vary due to the possible changes in the STM tip apex during the manipulation process. In addition to changes in the chemical structure of the molecule, some manipulation parameters kept molecules intact simply resulting in its rotation or translation along the Au(111) surface as well as subtle shape or contrast changes due to different tip conditions. On the other hand, more extreme manipulation parameters can cause destructive damage to molecules and induce breaking of other bonds than C–Br, significant changes such as complete flips of the molecular configuration, large movements of molecules far away from the initial positions, and serious problems in tips like contamination, instabilities and multiple apices (see Figure S8).

Figure 3. Interpretation of reaction. (a) Possible states after dissociation. The images in the first and third columns refer to STM images before and after dissociation. Red points in images indicate the tip positions for dissociation. The second column represents the topography change during the implementation of parameters, where the red point marks the initial tip-sample distance. All values are relative to the initial tip-sample distance. (b) Illustration of signal classifier for evaluating whether the dissociation happens. (c) Three categories for evaluating products using image classifiers.

Figure S8. Example of indeterminate cases in STM images.

Figure S16. Performance of randomly generating actions at two types of bias patterns with tip positions referred to the centre point of the ellipse constrained to within 0.6 nm along x (or y) axis. (a) 8 s pulse pattern. (b) 42 s ramp pattern. The first column illustrates the bias voltage patterns, the second column indicates the distribution of Suc (red dots), Ind (blue dots), and Int species (grey dots) upon varying the current and the voltage, the third column shows the corresponding distribution upon varying the tip position (tip and tip), the fourth column represents the evolution of dissociation steps over multiple episodes, where red points symbolize successful dissociation.
One of the major challenges for the automation of chemical reactions in SPM is to understand and recognize the consequences of applying manipulation parameters as outlined in the previous section. Due to the possibility of many complex outcomes, we opted to simply classify all of these into three categories: successful dissociation (Suc), intact molecule (Int) and indeterminate status (Ind), as shown in Figure 3c. This is used to determine if the manipulation action on a targeted molecule has to be continued (in the case of Int) or stopped (in the case of Suc and Ind), and whether the – bond dissociation succeeds (in the case of Suc). For the Int category, the molecule may rotate but keep the typical characteristics of the target molecules, indicating that manipulation can continue. Meanwhile, Ind status and Suc status mean that the image pattern cannot be characterized as the targeted molecule any more, and the manipulation process is terminated. The difference between the two is that in the former we cannot establish if the – bond has been dissociated, while in the latter it has clearly succeeded, either resulting in ZnBrMeDPP or ZnMeDPP.
Aiming at automating this evaluation process of products, we analyzed over 5000 cases from the STM output (see Methods). The most straightforward way is to inspect the images after dissociation. Therefore, we trained classification models ( and ) with experts labelling images to predict whether the products are Suc molecules (ZnBrMeDPP or ZnMeDPP), ZnBrMeDPP (Int molecules) or belonging to the Ind category (accuracy higher than 97%, more performance matrices and algorithmic details available in Fig S11-14 and Methods).
Another obvious signal to consider as a classifier is the bias voltage (V)-topography (Z) curve – it clearly exhibits different characteristics when resulting in different products during the dissociation (Figure 3b). Generally, successful dissociation tends to be accompanied by a larger hysteresis in their V-Z curves. While a small hysteresis or even an overlapped curve emerges for manipulation parameters keeping molecules intact, especially those occurring at low voltage or current. Furthermore, we quantitatively estimated the three categories by analyzing the difference in topography between ramping up and down () calculated by Eq. (5) in the section Signal classification. As shown in Figure S11, there is some overlap in the distribution of the for the three categories. However, the values among Int cases are usually smaller than 3.0 nm, and for Suc cases values tend to be larger, even reaching 20 nm, whereas a broader range of values (0 - 63 nm) is observed in Ind cases. While this offers useful insight into the dissociation process in some cases, we found that it was not a reliable classifier for DRL in general, as it was difficult to distinguish among different manipulation effects.

Figure S11. Distribution of during dissociation for three categories of products. 200 samples were obtained by a random selection from each category.
Decision-making
Random action
Developing models capable of interpreting STM manipulation outcomes is an essential precondition to finding the optimal parameters to reach the desired goal. Initially, we employed the most straightforward method — random action — to generate the four most relevant manipulation parameters for dissociation of the C–Br bonds: , , , .
We approximate the contrast pattern of a target molecule as an ellipse, whose centre is defined as the reference tip position. Based on the size of the patterns in STM images (about 2.4 nm, Figure S17), we limited the range of possible tip positions to within a radius of 1.3 nm from the reference position, sufficient to cover the whole pattern. In addition, the ranges of voltages and currents are set to 1200 - 4000 mV and 0 - 1200 pA based on domain knowledge. Figure 4 demonstrates the effects of 573 dissociation attempts on 150 molecules. Of these, 34% of the molecules (Figure 4c) were successfully dissociated into either [1] or [2], while the majority of molecules were categorized as Ind cases. The voltage and current distribution (Figure 4a) revealed that successful dissociation reactions tend to occur at higher voltages (above 2400 mV), but are also accompanied with a high chance of unwanted reactions. However, the possibility of unwanted reactions could be reduced to some extent by using a lower current. We suspect that a higher current leads to multiple electrons being injected into the molecule, which excites multiple bonds, thus resulting in products which are difficult to analyze. Yet, the dependency of the applied current on the frequency of Ind cases is too noisy to make any clear conclusion in this regard. Meanwhile, lower voltages sometimes result in rotation of the molecule, or no change at all. On the other hand, the dissociation reaction does not seem highly sensitive to the tip position, even when the tip is not directly on top of the molecule, it could still break the C–Br bond as desired. The spectra detected at points 3 and 9 of the molecule (Figure S7) indicate the characteristics of the Au(111) substrate, suggesting that the C–Br bond should be located between point 2 (or 8) and point 3 (or 9), whose distance referred to the centre point is less than 0.9 nm (Figure S17). Moreover, we compared the result of the effect of random actions with the tip position constrained to be over molecules by reducing the radius from 1.3 nm to 0.6 nm in Figure S16 – the success rate slightly increased to 0.39. Meanwhile, the consequence of changing the voltage pattern from a pulse of 8 s to a ramp of 42 s demonstrated a comparable success rate of 0.40 in Figure S16 (more details of the voltage patterns are illustrated in Methods, the pulse pattern is the default if not said otherwise).
Figure 4. Performance of random action. (a) Distribution of bias voltages and currents during 573 dissociation events. (b) Distribution of tip positions. A molecule in green is superposed as a reference, where the yellow point represents the center point of the molecule, approximated as an ellipse. (c) The dissociation attempts for each episode before termination, where red points indicate successful dissociation.

Figure S7. (a,b) Experimental spectra at points 0 - 11 on the pattern corresponding to a BrMeDPP molecule in (c) an STM image (STS feedback loop at A and 2 V for all cases). (d, e,f) maps recorded at -1.35 V (HOMO-1), -1.15 V (HOMO) and 1.85 V (LUMO), the setpoint is A, constant current mode.

Figure S17. Schematic illustration of the distance of each part of ZnBrMeDPP within a pattern approximated as an ellipse, referenced to its center point.
We further attempted to constrain the tip position near the C–Br bond based on Eq. (6), which ensures consistent positioning regardless of the rotational state of the contrast patterns in images, and also applied randomly generated bias voltages and currents to dissociate the molecule. The result for 164 molecules in Figure S18 showed a similar trend in the distribution of voltage and current as with random tip position previously used. However, the success rate increases from 0.34 to 0.43, implying that specific tip positions could somewhat reduce the possibility of unwanted reactions of the molecules.

Figure S18. Performance of random action at a fixed tip position. (a) Left panel: the pattern corresponding to a molecule is approximated as an ellipse. Tip position referred to the center point of the ellipse. Right panel: the distribution of tip position in this test. (b) Distribution of voltages and currents. Three categories of products (Int, Ind, Suc) under the voltage-current pairs are marked by grey, red and blue. (c) The evolution of dissociation steps over multiple episodes, red points indicate successful dissociation. The specific tip position is set based on Equation (6).
Optimize action by DRL
By definition, the random generator lacks the ability to optimize the dissociation parameters. Generally, this kind of decision-making problem can be formalized as a Markov decision process, where the manipulation parameters (action) depend solely on the current STM image (state). Therefore, we employed a DRL approach based on the Soft Actor-Critic (SAC) algorithm [39] to optimize parameters for breaking the C–Br covalent bond, using a rational reward design based on interpreting the SPM scanning images during manipulations. To simplify the issue, we hypothesize states in DRL are the same with a state space for all selected molecules, regardless of tip condition and slight changes in the molecular conformations on the surface. The goal in our DRL models is to optimize bias voltage and current at the same specific tip position (Figure 5a) under the reward system in Eq. (7).

Figure 5. Performance of DRL model. (a) Upper panel: illustration of the fixed tip position referred to the center point of a molecule, approximated as an ellipse (left) and the distribution of all tip positions referred to center points during DRL training process (right). down panel: architecture of DRL model based on the SAC algorithm. It consists of policy network, critic networks (Q-value function), temperature parameter and target Q-Networks and replay buffer. (b) Real STM trajectory while detecting targeted molecules on the Au(111) sample and corresponding dissociation results. Red points indicate successful dissociation for the molecules, while gray points represent failed dissociation for the molecules after at most 20 attempts with varying various parameters. Here xy axes correspond to the STM measurement coordinates. (c) Performance of image classifier on unknown cases. (d) Evolution of dissociate steps (top) and rewards (bottom) over episodes. (e) Evolution of bias voltages (top) and currents (bottom) over dissociation times. (f) Distribution of the pairs of voltage and current implemented on molecules for 968 dissociation times. (g) Repeatability test: dissociate 49 molecules at 3900.36 mV and 97.75 pA.
Figure 5b displays the trajectory of 328 episodes with a total of 968 dissociation manipulations on molecules. The coordinates correspond to the real coordinates in the STM, reflecting the distribution of molecules in this region. The red points indicate molecules which underwent successful dissociation, while gray points indicate indeterminate cases. Note that the heterogeneity in success rate across the surface is a function of non-uniform distribution of molecules on the surface, interruptions in scanning for technical reasons and also the influence of regions used for tip conditioning, and it is difficult to make any inferences on the role of the surface itself. The confusion matrix in Figure 5c further confirms the high accuracy of image classifier models on the unknown dataset. Figure 5d illustrates that the model starts to converge after 60 episodes, with a success rate before 60 episodes of 0.43, consistent with that in the tests using random voltage and current. After 60 episodes, the success rate increases to 0.62 and the dissociation steps per episode are
fewer than 3 in most cases, also with a higher occurrence of larger accumulated rewards.
The fluctuation of rewards between 1 and -10 could be attributed to the high proximity between parameters leading to successful and indeterminate dissociation. Due to differences in molecular conformations on Au(111) and tip condition, parameters that lead to successful C–Br dissociation for one molecule may result in an indeterminate dissociation for another. This is confirmed by repeatedly testing these successful parameters to dissociate molecules (Figure S19), where the success rate is just 0.42, comparable to that in random dissociation. Meanwhile, the voltage converged to higher values (more than 3500 mV) whereas the current narrows to lower values (less than 400 pA), despite some fluctuations, as shown in Figure 5e, Figure 5f. Such narrowing of the range of voltage and current guided by the reward, decreases the dissociation steps per episode and increases the success rate. Furthermore, the analysis for the distribution of these parameters in Figure S20 implies that the sets of parameters with voltage higher than 3800 mV and current less than 200 pA offer a higher possibility to successfully dissociate molecules and reduce indeterminate cases.

Figure S19. Performance of randomly selecting actions from successful dissociation parameters. (a) Upper panel: Distribution of the tip position, Lower pane: Evolution of dissociation steps over multiple episodes, red points indicate successful dissociation. (b) Distribution of voltages and currents.

Figure S20. Distribution of bias voltages (a) and currents (b) under three categories of products (Int, Ind and Suc), marked by grey, red and blue, in the DRL training
To explore whether we can further increase the success rate, we randomly selected a set of parameters with a lower current value (97.75 pA) and higher voltage value (3900.36 mV) from those associated with a high likelihood of successful dissociation in DRL training, and repeatedly applied these parameters to 49 molecules, as shown in Figure 5g, obtaining an increase of the success rate up to 0.8. This demonstrates the feasibility of our model applied on long-term, selective, efficient operations for autonomous on-surface synthesis in STM. Furthermore, the orbital energies of the highly localized C–Br states of the adsorbed ZnBrMeDPP molecule with respect to the Fermi level at 3.7 eV (Figure S10), comparable to the voltage bias applied to promote successful dissociation, suggest that selective bond dissociation is probably achieved by tunneling electrons into the corresponding antibonding states, consistent with previous literature. [32, 40, 41]

Figure S10. Interpretation of reaction. (a) STM images of , and . (b) Corresponding DFT-simulated STM images. (c) Corresponding 3D structures (top view and side view). (d) Corresponding energy diagrams from to radical to biradical. (e) Atom-Projected Density of States (PDOS) of adsorbed on Au(111), all energies are relative to the Fermi level (-5.16 eV). The Au DOS are scaled by 0.05. (f) C–Br antibonding orbitals and states for the molecule in the gas-phase as well as on surface.
Conclusions
To summarize, we have demonstrated the capability of a deep learning model to identify reactants and products based on STM outputs, enabling a DRL agent to evaluate various manipulation parameters. Furthermore, the establishment of a deep reinforcement learning approach allows the agent to optimize these parameters. These advancements address key challenges in STM automation and molecular synthesis. Ultimately, the integration of target detection module, interpretation module and decision-making module into the AutoOSS workflow achieved the automation of tip-induced C–Br bond breaking from in STM. AutoOSS enables long-term, selective and efficient operations without human intervention. Moreover, the extensive dataset accumulated from experiments, combined with big-data analysis, DFT calculations and BOSS, offers the opportunity to uncover hidden physical information, explore 3D molecular conformations and investigate reaction mechanisms despite the limits of resolution in STM images.
AutoOSS paves the way for automating manipulations in on-surface synthesis, thus pioneering a new paradigm in single molecular engineering. Moving forward, we anticipate the possibility of extending AutoOSS to a diverse array of molecules and applications pertinent to complex chemical reactions, encompassing various chemical bonds, molecules, tip, and manipulation types. For similar reaction processes, moving to different molecule and substrate combinations requires only retraining the model on appropriate classifiers of reaction success. Furthermore, there is the potential to enhance the model’s selectivity and precision by using a refined tip, optimized bias voltage pattern or incorporating AFM signals into the workflow to provide atom-level resolution scanning images.
Methods
Experimental preparation and STM microscopy
molecules (chemical structure shown in Figure 6) were synthesized via the precursors 5,15-bis(4-bromo-2,6-dimethylphenyl)porphyrin () from 2,6-dimethyl-4-bromobenzaldehyde, as shown in Figure S22. Characterizations associated with molecules and precursors (Figure S23-Figure S30), including and nuclear magnetic resonance (-NMR and -NMR) spectroscopy, mass spectrometry (MS) and ultraviolet-visible (UV/vis) spectroscopy, were implemented. Then, molecules were evaporated from a Knudsen cell heated to 230 °C onto a Au(111) sample kept below 7 K temperature. The STM scaned and dissociations were performed in constant current mode on a Createc LT-STM system with a gold-coated PtIr tip. The STM images recorded at different scales from 100 nm to 3.5 nm are shown in Figure S1. Contrast-adjusted STM images in Figure S3 are available to show the adsorption sites of individual molecules on various regions either in the herringbone of the Au(111) or on the Au(111) surface. Ultimately, we chose nm to detect promising targeted molecules and nm to make further identification by neural networks and dissociation manipulations. The scanning speed and the number of pixels for all images are and 128, resulting in around 42 s per image.

Figure 6. Chemical structure of .

Figure S1. STM images at difference scales. (a) , (b) , (c) , (d) , (e) .

Figure S3. (a, b) STM image of target molecules adsorbed at the hexagonal close-packed (hcp), face-centered cubic (fcc) and elbow regions of the reconstructed surface. {sup}? In (b), the herringbones are visible by adjusting the image contrast.

Figure S22. Synthetic route towards ZnBrMeDPP.

Figure S23. H-NMR spectrum of HBrMeDPP in CDCl (// = residual solvent signals).

Figure S30. UV/vis spectrum of in (conc. = 10 ).
Spiral path planning
The approach area was about nm, where four nm squares near the boundary were set aside to form tips. Therefore, the manipulation region usually corresponded to the XY coordinates in STM from -300 nm to 300 nm, where the centre point of the region of nm for detecting target candidates was updated by the shortest distance away from the reference point beyond forbidden area. It was formulated as: , where and indicate Euclidean distance and Manhattan distance , respectively, and the coefficient of is set as 1.
Detect target candidates
We firstly converted raw scanning images from STM to greyscale images and then made further analyses to detect target candidates using two methods. One method is to limit the distance between image contrast patterns, where binary images with the threshold pixel values of 50 (pixels less than 50 were set to 0) and 150 (pixels greater than 150 were set to 255) were obtained to find the individual molecules through a thresholding distance (default: 2.5 nm) between points to get rid of dimer, trimer or clusters. Another method is to limit the area of patterns, for which we detected contours by the Otsu algorithm [42, 43] with a clear outline (another two algorithms—global thresholding and Otsu thresholding after Gaussian filtering were compared by corresponding areas in Figure S2). Based on statistical analysis of candidates, we restricted the area of patterns within 1.5-3.0 nm² to further exclude some individual fragments.

Figure S2. Comparison of three image thresholding methods. (a) Global, (b) Otsu and (c) Otsu Gaussian for detecting the contrast pattern in an STM image.
BOSS
We employed the Bayesian Optimization Structure Search (BOSS) method [38] to reduce the number of DFT evaluations needed to map out the configurational phase space. Data points were initialized with a quasi-random Sobol sequence and the GP-Lower Confidence Bound acquisition function with increasing exploration (elcb) were used on all runs. The kernels for rotation and -translation were standard periodic kernel (stdp), while the -coordinate used radial basis functions (rbf). The surface symmetry was exploited to multiply the acquired data points by applying symmetry operations to the adsorbate at high-symmetry sites, where the Au(111) surface has three rotationally equivalent sites in addition to two translationally equivalent ones. Initially, a conformational search was conducted on the isolated gas-phase molecule, with the search variables being full rotation of the phenyl moieties and their methyl substituents (6D search). The surrogate model was constructed on 407 DFT data points. The search resulted in one single main conformer in terms of phenyl rotation, as shown in Figure S4, which was subsequently employed as the molecular building block in the adsorption structure search. The same structure was used as the building block for and , since loss of the terminal Br atoms does not result in significant rearrangement of the rest of the molecule following DFT relaxation. The adsorption structure search was done by constructing a surrogate model of the DFT () PES for the translational and rotational degrees of freedom (6D search), and subsequently relaxing the lowest energy surrogate model local minima with DFT, thus accounting for any changes in the structures of the isolated rigid molecular building blocks enabled by the surface interaction. The molecular adsorbate building blocks for the search were the lowest energy , , and species as described above, which were combined with the relaxed Au(111) substrate building block. The surrogate models for the adsorption structures were constructed out of 262, 94 and 108 data entries for , , and , respectively. The global minimum predictions were oscillating between the symmetrically equivalent rotational configurations ().

Figure S4. Conformational analysis of ZnBrMeDPP using BOSS. (a) Schematic illustration of rotatable C(porphyrin)-C(phenyl) bond (purple shadow) and C(phenyl)-C(methyl) bond (green shadow). (b) Corresponding energy diagrams by scanning rotation angles. The inset magnifies the energy curve for the rotation around the C(phenyl)-C(methyl) bond. (c) Structure with the phenyl group at 90° angles around the C(porphyrin)-C(phenyl) bonds. (d) Structure with the phenyl group at 0° (Methyl groups in phenyls bump into the porphyrin ring).
DFT calculations
All DFT computations were performed using FHI-aims. [37] For the initial conformational search using BOSS, we employed the B3LYP functional [44, 45] with light defaults and first tier basis functions. Subsequently, the resulting global minimum conformers, substrate, and all adsorption structures were relaxed to a forces less than using the Perdew-Burke-Ernzerhof (PBE) functional augmented with the van der Waals dispersion correction, including collective screening effects of the substrate electrons (), fully denoted (). [46, 47] This choice of functional for both isolated and adsorbed molecules was motivated by the properties of an adsorbed configuration being of interest, for which this functional has been demonstrated accurate in comparison with experiments. [48] The same functional was also used during BOSS data acquisition iterations for the adsorption structures. The Brillouin zone was sampled using a Monkhorst-Pack grid, and the slab was constructed using four layers of gold atoms as the Au(111) surface, of which the two lowest layers were kept fixed during all computations. The length of the box in the -direction was in total , ensuring sufficient vacuum space. This relatively large slab size was chosen to avoid interactions with adjacent adsorbates. Spin polarization was used for the dissociated and species.
The STM images were simulated using FHI-aims with the Tersoff-Hamann approximation as implemented therein. [49] The simulation bias was kept at for all images, which were created with VESTA[50] using an isovalue between and a.u. to match experimental STM images.
The calculations for the C–Br dissociation model reactions were performed using the climbing image nudged elastic band and growing strings methods [51, 52] as implemented in https://gitlab.com/cest-group/aimsChain-py3. The pathways of both bond cleavage reactions were modeled using 12 images in total, where the growing strings force threshold was , while the climbing image threshold was . The initial structure for the reaction was the global minimum adsorption configuration as determined by BOSS, while the dissociated final structures of each step in the reaction were determined by moving the Br atom 5 Å away from the rest of the porphyrin, and relaxing with DFT to the force threshold as the initial image.
The adsorption energy is formulated as: , where denotes molecule on the substrate, the isolated molecule and the isolated substrate.
Image classification
All image classifiers were developed based on the ResNet18 model [53] (the architecture of the neural network shown in Figure 2b and Figure S6), taking STM images with a size of 3.5 3.5 nm and the pixel numbers of 128 as input. To ensure the intact pattern of fragment in the image, we adjust the scanning region based on the centre of the pattern in STM images and scan again if the centre point is beyond the threshold region. The default criterion for the centre in the pattern is less than 0.438 nm along both axes, referred to as the centre point of the scanning region.

Figure S6. Architectures of Block 1 and Block 2 in ResNet18. Block 1 is stacked by two sets of convolution layer, normalization layer and Relu layer, which is a regular ResNet block. Block2 enhances a regular ResNet block by incorporating a convolution with a normalization.
The image classifiers consist of three binary models (, , ) and one multiclass model () with the numbers of corresponding datasets shown in Table 1, intended for detecting reactants and distinguishing products. Due to the complexity in products, caused by variable tip conditions, various conformations as well as subtle differences for dissociated molecules and pristine molecules, we trained another two binary models ( and ) for more elaborate distinctions to supplement . In brief, these models were designed to distinguish and non (), to distinguish indeterminate and non indeterminate () and to distinguish intact molecules () and dissociated molecules ( or ) ().
| Target or not | Indeterminate or not | ||||
| class | train | test | class | train | test |
| True mol | 273 | 39 | Indeterminate | 1186 | 270 |
| Non true | 1116 | 159 | Non indeterminate | 2764 | 607 |
| Dissociation or not | Products | ||||
| class | train | test | class | train | test |
| Original mol | 2023 | 438 | Original mol | 2023 | 438 |
| Diss mol | 741 | 169 | Diss mol | 741 | 169 |
| Indeterminate | 1186 | 270 |
Table 1. Dataset for four images classifiers
The Adam optimizer [54] with cross entropy loss function and StepLR were used to optimize parameters in models. In addition, Bayesian optimization was introduced to optimize learning rate based on the converged loss values under corresponding learning rate values. Eventually, the optimal learning rates in Adam optimizer are , , , 0.0001 for , , , , respectively. All models perform decently with the accuracy more than 94% and the under the curve (AUC) higher than 98% (more performance metrics are shown in Figure S12-15 and Table S1-4). A confusion matrix divides the classification results into 4 categories through comparing the true values and predicted values: True Position (TP, both real values and predicted values are 1), True Negative (TN, both real values and predicted values are 0), False Positive (FP, real values are 0, but predicted values are 1), False Negative (FN, real values are 1, but predicted values are 0).
| class | precision | recall | f1-score | support |
| True_mol | 0.93 | 1.00 | 0.96 | 39 |
| Non_true | 1.00 | 0.98 | 0.99 | 159 |
| accuracy | 0.98 | 198 | ||
| macro avg | 0.96 | 0.99 | 0.98 | 198 |
| weighted avg | 0.99 | 0.98 | 0.99 | 198 |
Table S1. Performance metrics of
Signal classification
We tested two types of voltage bias patterns—a ramp of 42s and a pulse of 8s, with similar processes, as shown in Figure S16. Times were divided into 1024 steps. The initial voltages are 1 V: for a ramp pattern, the voltage starts to increase to the specific voltages from point 20 until point 512, symmetrically, then decrease to 1 V at point 1004; while for a pulse pattern, the voltage directly jumps to the specific voltage at point 20, which is maintained until point 1004, then back to 1 V. We analyzed signal changes during the dissociation by the difference of topography, formulated as follows:
where indicates the value of topography at a point during the process of voltage variation along 1024 points.
Specific tip position
where H and is the height and the angle of an ellipse evaluated by the fitEllipse function in OpenCV, [55] is a coefficent (default: 0.3), and are random noise ranging from -0.1 Å to 0.1 Å.
Reward design
The assessments from image classifiers are applied to evaluate the reward and make further decisions. The reward is defined as:
where indicates the dissociate times in an episode, and is a coefficient with a default value of 0.2.
Success rate
The success rate in a test is defined as:
In this equation, and represent, respectively, the number of successful dissociation and the number of the indeterminate cases for all episodes in one test. The value is used to evaluate the ability of the model optimizing parameters to avoid indeterminate cases. Dissociation steps in an episode can be used to assess how fast the model can find successful dissociation parameters. Therefore, the unchanged dissociation is not necessarily considered here.
Tip conditioning
The tip may suffer bluntness, contamination, instability, damage or multiple tips during the scanning process. Correspondingly, different parameters related to voltage, indentation depth and time may be needed to identify the condition for a sharp and stable tip. To maintain a good tip, the workflow monitors the tip condition and reforms when needed. Experience in this task demonstrates that a random approach heights ranging from 2 nm to 5.5 nm and constant voltage of 1 V usually can achieve a decent tip. As the criteria for successful tip conditioning, we search for candidate molecules in a 20 20 nm image – if this fails many times (default: 4), our algorithm tries deeper immersion with 10 nm. This strategy allows a long-term operations in the whole workflow ranging from detection to dissociation.
Tip conditioning is activated only when detecting molecules. Once the targeted molecule is found, it consecutively tests different dissociation parameters until it terminates, without the interruption from conditioning tip. On one hand, the movement during the tip conditioning may lead to the shift of coordinates in STM and the tip status after forming may be complex and still effectively bad, which may damage targeted molecules. On the other hand, if the molecules are damaged by a dissociation manipulation rather than tip itself, we treat it as a failed manipulation, and classify it as a indeterminate case. Furthermore, empirically, gentle manipulations during dissociation in our task sometimes even make tip better in obtaining high-quality scanning images. Therefore, tip conditioning during dissociation attempts is not necessary.
To avoid moving the tip long distances, four square regions with the length of 100 nm near the edge of approach area were set, among which the tip moves towards the closest one for conditioning. To reduce the time on scanning the tip conditioning region, we just condition the tip at a random point in the square in practice. As an option, the algorithm of detecting point from the blank region to avoid molecules (Figure S21) is available.

Figure S21. Schematic illustration of our algorithm detecting blank regions and selecting tip position from specific regions for conditioning tips.
Soft Actor-Critic
The SAC approach consists of policy sampling module for mapping a state to an action, two state value q network for evaluating the state value, and one state-action value q network. The maximum entropy RL in the model maximizes the cumulative rewards and also pursues the diversity of policy through introducing the entropy term:
Value network: The loss function of value:
Policy network: The action was determined by policy network, which generated the mean and std of Gaussian distribution by 1 linear layer (hidden_dim: 512) and then sampled action from the Gaussian distribution. The loss function of policy was set based on the Kullback-Leibler (KL) Divergence:[56]
Entropy regularization: To maximize the entropy, the corresponding loss function was set as follows:
where is the temperature parameter.
In addition, the advanced sampling technique, Hindsight Experience Replay (HER),[57] was used to improve the data efficiency. The optimal hyperparameters found in testing are learning rate lr of 0.0003, discount factor of 0.99 and target smoothing pf of 0.1.
Acknowledgement
We thank the valuable discussion and assistance from Roman Fasel, Bruno Schuler, Hongxiang Xu, Shuning Cai, Zhenrong Zhao, Xueyong Jia, Zhengmao Li, Mohammad Amini and Büşra Gamze Arslan. The authors acknowledge fundings from the Academy of Finland (project no. 318995, 320555, 346824, 347319). A.S.F. was supported by the World Premier International Research centre Initiative (WPI), MEXT, Japan. This research was part of the Finnish centre for Artificial Intelligence FCAI. The authors acknowledge the computational resources provided by the Aalto Science-IT project and CSC, Helsinki. T.T. ad G.B. acknowledge financial support from the Spanish MCIN/AEI/10.13039/501100011033 and European Union NextGenerationEU/ PRTR (PID2020-116490GB-I00, TED2021-131255B-C43), MICIU /AEI /10.13039/501100011033 / FEDER, UE (PID2023-151167NB-I00), the Comunidad de Madrid and the Spanish State through the Recovery, Transformation and Resilience Plan [“Materiales Disruptivos Bidimensionales (2D)” (MAD2D-CM) (UAM1)-MRR Materiales Avanzados], and the European Union through the Next Generation EU funds. IMDEA Nanociencia is appreciative of support from the “Severo Ochoa” Programme for Centers of Excellence in R&D (CEX2020-001039-S).
Supporting Information Available
The Supporting Information is available free of charge.
Detection of individual ZnBrMeDPP: STM images at different scales, Measuring the area of contrast patterns, Individual molecules on Au(111), Conformational analysis of ZnBrMeDPP, Various configurations of ZnBrMeDPP on Au(111), ResNet18 Block, dI/dV spectra and maps of ZnBrMeDPP on Au(111);
Interpretation: Example of indeterminate cases, Properties of contrast patterns in STM images, Reaction energies and possible dissociation mechanisms, Signal classifier, Performance metrics of M, Performance metrics of M, Performance metrics of M, Performance metrics of M$_{\mathrm{Performance metrics of ;
Decision-making: Random action at different bias patterns, Contrast pattern measurement, Random action with fixed tip position, Random selection from successful dissociation parameters, Dissociation parameters from the DRL model, Tip conditioning region;
Synthesis and characterization: Materials and Methods, Synthesis of precursors and , as well as their corresponding -NMR, -NMR, MS and UV/vis.
Data and Software availability
A video demonstrating AutoOSS’s ability to autonomously and selectively control the reaction, all training dataset and parameters in machine learning models and input and output of BOSS and DFT calculations can be obtained on the Zenodo repository at https://doi.org/10.5281/zenodo.13761822. The source codes and examples are available on the GitHub repository at https://github.com/SINGROUP/AutoOSS.
TOC Graphic
AutoOSS automates tip-induced Br removal from large-scale molecules on Au(111) in scanning tunneling microscopy with interpretability and selectivity by combining theoretical calculations and advanced machine learning techniques.
Supplementary Information for: Precise large-scale chemical transformations on surfaces: deep learning meets scanning probe microscopy with interpretability
Nian Wu; Markus Aapro; Joakim S. Jestilä; Robert Drost; Miguel Martínez García; Tomás Torres; Feifei Xiang; Nan Cao; Zhijie He; Giovanni Bottari; Peter Liljeroth; Adam S. Foster
Nian Wu, Markus Aapro, Joakim S. Jestilä, Robert Drost, Miguel Martínez García, Tomás Torres, Feifei Xiang, Nan Cao, Zhijie He, Giovanni Bottari, Peter Liljeroth, and Adam S. Foster.
†Department of Applied Physics, Aalto University, Helsinki, 02150, Finland.
‡Departamento de Química Orgánica, Universidad Autónoma de Madrid, Madrid, 28049, Spain
¶IMDEA-Nanociencia, Campus de Cantoblanco, Madrid, 28049, Spain
§Institute for Advanced Research in Chemical Sciences, Universidad Autónoma de Madrid, Madrid, 28049, Spain
∥nanotech@surfaces Laboratory, Empa-Swiss Federal Laboratories for Materials Science and Technology, Dübendorf, 8600, Switzerland
⊥Department of Computer Science, Aalto University, Helsinki, 02150, Finland.
#WPI Nano Life Science Institute, Kanazawa University, Kanazawa, 610101, Japan.
List of abbreviations
AcOEt = Ethyl acetate
AFM = Atomic Force Microscopy
AUC = Area under the curve
BOSS = Bayesian Optimization Structure Search
CREST = Conformer-Rotamer Ensemble Sampling Tool
DFT = Density Functional Theory
DCM = Dichloromethane
DCTB = trans-2-[3-(4-tert-butylphenyl)-2-methyl-2-propenylidene]malononitrile
DDQ = 2,3-Dichloro-5,6-dicyano-1,4-benzoquinone
DRL = Deep reinforcement learning
HER = Hindsight Experience Replay
MALDI-TOF = Matrix-assisted laser desorption/ionization-time of flight
MS = Mass spectrometry
NMR = Nuclear magnetic resonance
OSS = On-surface synthesis
Rf = Retention factor
RL = Reinforcement learning
SAC = Soft Actor-Critic
SPM = Scanning Probe Microscopy
STM = Scanning Tunnelling Microscopy
THF = Tetrahydrofurane
UV/vis = Ultraviolet-visible
= 5,15-bis(4-bromo-2,6-dimethylphenyl)porphyrin
= Zn(II)-5,15-bis(4-bromo-2,6-dimethylphenyl)porphyrin
Detection of individual
STM images at different scales
Measuring the area of contrast patterns
Individual molecules on
Conformational analysis of ZnBrMeDPP
The most important conformational change for the ZnBrMeDPP molecule is due to rotation of the CH(CH)Br (phenyl) groups relative to the central porphyrin. It should be noted that free rotation of the CH (methyl) groups around the C-C bond that connect them to the phenyl is possible, but this rotation does not change the overall conformations of the porphyrin enough to warrant a separate analysis of the corresponding rotamers on the surface, as these minute structural changes are likely superseded by those induced by adsorbate-substrate interactions. The main conformer of ZnBrMeDPP has the phenyl moieties orthogonal to the central porphyrin backbone, with free rotation being hindered by the methyl groups on the former (rotation barrier around 3.0 eV corresponding to clash between atoms, see Figure S4). Futhermore, the backbone of the molecule has some limited flexibility, but since changes in this structure is most accurately described by collective motion of the molecule, it is difficult to assign simple variables required by BOSS, such as bond stretch, bending or torsion to this motion. Therefore, we also used the Conformer-Rotamer Ensemble Sampling Tool (CREST),{sup}? which employs metadynamics including collective variables to check for alternative gas phase conformers. However, this analysis also yielded one conformer only, in addition to a large number (around 300) of similar rotamers, corresponding to the aforementioned collective movement of the porphyrin backbone, the phenyl side groups, and the methyl groups.
Various configurations of on
ResNet18 Block
spectra and maps of ZnBrMeDPP on Au(111)
Interpretation
Example of indeterminate cases
Properties of contrast patterns in STM images
Reaction energies and possible dissociation mechanisms
In Figure S10, the STM images of pristine (left), (center), (right) are represented. Based on these, we explored the corresponding 3D structures of the or products by DFT. When removing a Br atom from , the adjacent C atom in the product tends to bind with an atom in the Au(111) surface, simultaneously losing its initial radical character, as corroborated by spin-polarized DFT computations. Similar effects are observed after removing the second Br atom, and the two terminals of the product bind strongly with the surface, the whole molecule adopting an "arc-type" conformation on the surface. After inspecting the -coordinates of specific atoms before and after dissociation, we note that some atoms are elevated, while others are lowered with respect to their initial positions (Figure S10), which could be reflected in the experimental topography (Figure 3a, red dot is the initial height of tip). Furthermore, the energy diagrams (Figure S10) obtained by DFT calculations for the removals of two bromines from in both concurrent and consecutive models show that the activation barrier energy for the first Br removal (2.05 eV) is close to the second Br removal (1.82 eV), while a higher activation energy barrier of 3.66 eV needs to be surmounted for the concurrent removal of two bromines. Therefore, the dissociation of two bromine atoms from could occur sequentially or simultaneously during the dissociation procedure.
To gain further insight on the possible reaction mechanism leading to the C–Br bond dissociation, we analyzed the electronic structure of . Highly localized C–Br antibonding orbitals were identified based on the feature in the PDOS corresponding to Br and C atoms at approximately 3.7 eV above the Fermi level (Figure S10). For the isolated gas-phase molecule, these correspond to LUMO+7 and LUMO+8. From visual inspection, the symmetries of the orbitals appear correct for a - bond, representing the antibonding counterpart for the single bond between the C and Br atoms. Based on the agreement with the most successful bias voltage determined by RDL at 3.9 V, and the energies of the antibonding states, we surmise that one probable mechanism is dissociative electron attachment, where the tunneling electrons are supplied to the specific C–Br orbitals. Subsequently, the charge is neutralized, which is accompanied by vibrational excitation of the C–Br bond. Furthermore, this provides us with a way to rationalize the observation that in some instances, both C–Br bonds are dissociated during manipulation, as the antibonding states are delocalized over both C–Br bonds, in contrast to the gas-phase molecule, where one end is more prominent. Despite this consistent model, we cannot fully rule out other mechanisms, as we observe successful dissociation with a wide range of parameters and tip positions. Considering that the computed barriers for dissociation are significantly lower than the bond energies at 2.6 and 3.3 eV (Figure S10), respectively, another possible
mechanism proceeds via tunneling into orbitals of lower energy, followed by the vibrational excitation. The resulting excess vibrational energy can quickly be redistributed into the other modes, including the one corresponding to C–Br bond vibration. This is consistent with the more stochastic nature (lower rate of success) of the dissociation observed at these lower bias values, as sufficient vibrational energy needs to be partitioned into these specific bonds. Furthermore, this also rationalizes why higher bias values induce more reactions in general, as more energy becomes available for reactions.
Signal classifier
Performance metrics of
Performance metrics of

Figure S13. Performance metrics of for classifying , dissociated molecules, indeterminate when evaluating products, with 3950 samples in the training dataset and 877 samples in the test dataset.
| class | precision | recall | f1-score | support |
| Original_mol | 0.96 | 0.98 | 0.97 | 438 |
| Diss_mol | 0.93 | 0.88 | 0.90 | 169 |
| Indeterminate | 0.93 | 0.94 | 0.94 | 270 |
| accuracy | 0.95 | 877 | ||
| macro avg | 0.94 | 0.93 | 0.94 | 877 |
| weighted avg | 0.95 | 0.95 | 0.95 | 877 |
Table S2. Performance metrics of
Performance metrics of

Figure S14. Performance metrics of for distinguishing indeterminate products among products, with 3950 samples in the training dataset and 877 samples in the test dataset.
| class | precision | recall | f1-score | support |
| Non_indeterminate | 0.98 | 0.98 | 0.98 | 607 |
| Indeterminate | 0.95 | 0.95 | 0.95 | 270 |
| accuracy | 0.97 | 877 | ||
| macro avg | 0.96 | 0.96 | 0.96 | 877 |
| weighted avg | 0.97 | 0.97 | 0.97 | 877 |
Table S3. Performance metrics of
Performance metrics of

Figure S15. Performance metrics of for distinguishing and dissociated molecules when evaluating products, with 2764 samples in the training dataset and 607 samples in the test dataset.
| class | precision | recall | f1-score | support |
| Original_mol | 0.99 | 0.99 | 0.99 | 438 |
| Diss_mol | 0.98 | 0.98 | 0.98 | 169 |
| accuracy | 0.99 | 607 | ||
| macro avg | 0.98 | 0.98 | 0.98 | 607 |
| weighted avg | 0.99 | 0.99 | 0.99 | 607 |
Table S4. Performance metrics of
Decision-making
Random action at different bias patterns
Contrast pattern measurement
Random action with fixed tip position
Random selection from successful dissociation parameters
Dissociation parameters from the DRL model
Tip conditioning region
Synthesis and characterization
Materials and Methods
Chemicals and solvents were purchased from commercial suppliers (Aldrich, BLDpharm, Thermo Scientific Chemicals and Scharlab) and used without further purification. All dry solvents were freshly distilled under argon over an appropriate drying agent before use. Column chromatography was carried out on silica gel VWR-60 (40–63 µm). Analytical TLC was performed on aluminum sheets precoated with silica gel 60 F-254 from Merck. - and -NMR spectra were recorded with a Bruker XRD-500 (500 MHz) instrument at room temperature (25 °C). Chemical shifts values () are referred to the corresponding deuterated solvent (CDCl). UV/vis experiments were carried out by using quartz cells with a 1 cm optical path length in a JASCO-V660 UV-vis spectrophotometer. MALDI-TOF MS spectra were obtained in a Bruker ULTRAFLEX III (MALDI-TOF/TOF) spectrometer.
Synthesis of precursors HBrMeDPP
HBrMeDPP was synthesized using a modified procedure from Zhao and co-workers. ?
meso-H-dipyrromethane (533 mg, 2.5 mmol, 1 eq.) and 2,6-dimethyl-4-bromobenzaldehyde (365 mg, 2.5 mmol, 1 eq.) were loaded in a flame-dried 500 mL round-bottom flask. (250 mL) and dry ethanol (1.87 mL) were added to the flask and the mixture was vigorously degassed with argon for 40 minutes under stirring and in the dark. Then, (0.1 mL) was added under argon, whereupon the mixture turned dark red. The mixture was stirred in the dark for 80 minutes at room temperature, then DDQ (908 mg, 4 mmol, 1.6 eq.) was added in one batch and the mixture was heated to reflux for 1 h. After that, the mixture was allowed to cool down to room temperature, triethylamine (2 mL) was added and the solution stirred for 5 minutes. After this time, the solvent was removed under reduced pressure and the resulting crude purified by column chromatography using DCM/n-heptane (3:2) as eluent. From the column, a red-coloured fraction was isolated (Rf = 0.68) which upon removal of the solvent under reduced pressure afforded a solid which was suspended in n-pentane, sonicated, filtered, washed with n-pentane (4 mL), and dried under vacuum to yield as a dark-violet powder (225 mg, 27%). H-NMR (500 MHz, ): (s, 2H), 9.37 (d, J = 4.6 Hz, 4H), 8.87 (d, J = 4.6 Hz, 4H), 7.69 (s, 4H), 1.86 (s, 12H), (s, 2H); C NMR (125 MHz, ): , 139.7, 132.4, 130.1, 129.9, 122.4, 115.9, 105.1, 21.7; MALDI-TOF MS (DCTB matrix): m/z (% intensity) = 674.1 – 681.1 m/z [M] (100%); UV/vis (): (log ) = 405 (5.32), 500 (4.21), 576 (3.72).
H-NMR
C-NMR

Figure S24. C-NMR spectrum of HBrMeDPP in CDCl (// = residual solvent signals).
MS

Figure S25. MALDI-TOF mass spectrum of . Inset: (a) experimental isotopic resolution of the MALDI-TOF main peak at . (b) Calculated isotopic pattern for .
UV/vis

Figure S26. UV/vis spectrum of in (conc. = ).
Synthesis of
(20 mg, 0.03 mmol) and (54 mg, 0.30 mmol, 10 eq.) were loaded in a flame-dried 10 mL Schlenk tube and three cycles of vacuum/argon backfilling were applied. Then, dry THF (5 mL) was added, and the mixture was heated to reflux while stirring under argon for 24 h. After this time, the mixture was then allowed to cool to room temperature, quenched with water (10 mL) and extracted with DCM (3 15 mL). The combined organic layers were washed with water (25 mL) and brine (25 mL), dried over anhydrous , filtered and the filtrate dried under reduced pressure. The resulting crude was subjected to a short column chromatography using DCM/n-heptane (3:2) as eluent, isolating a red-coloured fraction () which solvent was removed under reduced pressure. The resulting solid was suspended in n-pentane, sonicated, filtered, washed with n-pentane (3 2 mL), and dried under vacuum to yield as a purple powder (21 mg, 95%). -NMR (500 MHz, ): (s, 2H), 9.42 (d, J = 4.5 Hz, 4H), 8.94 (d, J = 4.5 Hz, 4H), 7.68 (s, 4H), 1.83 (s, 12H); -NMR (125 MHz, ): , 149.5, 141.9, 140.9, 132.8, 131.1, 129.9, 122.1, 116.9, 106.1, 21.7; MALDI-TOF MS (DCTB matrix): (% intensity) = 736.0 – 746.0 [M] (100%); UV/vis (): (log ) = 410 (5.34), 540 (4.22).
H-NMR

Figure S27. H-NMR spectrum of ZnBrMeDPP in CDCl ( = residual solvent signals).
C-NMR

Figure S28. C-NMR spectrum of ZnBrMeDPP in CDCl ( = residual solvent signals).
MS

Figure S29. MALDI-TOF mass spectrum of . Inset: (a) experimental isotopic resolution of the MALDI-TOF main peak at 738.0 m/z. (b) Calculated isotopic pattern for .
UV/vis
References
- [1]Khajetoorians, A. A.; Wegner, D.; Otte, A. F.; Swart, I. Creating designer quantum states of matter atom-by-atom. Nat. Rev. Phys. 2019, 1, 703–715.DOI
- [2]Pitters, J.; Croshaw, J.; Achal, R.; Livadarou, L.; Ng, S.; Lupoiu, R.; Chutora, T.; Huff, T.; Walus, K.; Wolkow, R. A. Atomically precise manufacturing of silicon electronics. ACS nano 2024, 18, 6766–6816.DOI
- [3]Gross, L.; Schuler, B.; Pavliček, N.; Fatayer, S.; Majzik, Z.; Moll, N.; Peña, D.; Meyer, G. Atomic force microscopy for molecular structure elucidation. Angew. Chem. Int. Ed. 2018, 57, 3888–3908.
- [4]Sanvito, S. Molecular spintronics. Chem. Soc. Rev. 2011, 40, 3336–3355.
- [5]Li, L. et al. Manipulating the insulator–metal transition through tip-induced hydrogeneration. Nat. Mater. 2022, 21, 1246–1251.DOI
- [6]Li, P.; Chen, S.; Dai, H.; Yang, Z.; Chen, Z.; Wang, Y.; Chen, Y.; Peng, W.; Shan, W.; Duan, H. Recent advances in focused ion beam nanofabrication for nanostructures and devices: Fundamentals and applications. Nanoscale 2021, 13, 1529–1565.DOI
- [7]Ren, Y.; Chen, Q.; He, M.; Zhang, X.; Qi, H.; Yan, Y. Plasmonic optical tweezers for particle manipulation: principles, methods, and applications. ACS nano 2021, 15, 6105–6128.DOI
- [8]Whitman, L.; Stroscio, J. A.; Dragoset, R. A.; Celotta, R. Manipulation of adsorbed atoms and creation of new structures on room-temperature surfaces with a scanning tunneling microscope. Science 1991, 251, 1206–1210.
- [9]Zhang, F.; Zhang, Y.; Li, L.; Mou, X.; Peng, H.; Shen, S.; Wang, M.; Xiao, K.; Ji, S.-H.; Yi, D.; Nan, T.; Tang, J.; Yu, P. Nanoscale multistate resistive switching in WO3 through scanning probe induced proton evolution. Nat. Commun. 2023, 14, 3950.DOI
- [10]Li, M.; Xun, K.; Zhu, X.; Liu, D.; Liu, X.; Jin, X.; Wu, M. Research on AFM tip-related nanofabrication of two-dimensional materials. Nanotechnol. Rev. 2023, 12, 20230153.DOI
- [11]Simpson, G. J.; García-López, V.; Daniel Boese, A.; Tour, J. M.; Grill, L. How to control single-molecule rotation. Nat. Commun. 2019, 10, 4631.
- [12]Zhong, Q.; Mardyukov, A.; Solel, E.; Ebeling, D.; Schirmeisen, A.; Schreiner, P. R. On-Surface Synthesis and Real-Space Visualization of Aromatic P3N3. Angew. Chem. Int. Ed. 2023, 135, e202310121.DOI
- [13]Kalinin, S. V.; Ziatadinov, M.; Hinkle, J.; Jesse, S.; Ghosh, A.; Kelley, K. P.; Lupini, A. R.; Sumpter, B. G.; Vasudevan, R. K. Automated and autonomous experiments in electron and scanning probe microscopy. ACS nano 2021, 15, 12604–12627.DOI
- [14]Rashidi, M.; Wolkow, R. A. Autonomous scanning probe microscopy in situ tip conditioning through machine learning. ACS nano 2018, 12, 5185–5189.
- [15]Krull, A.; Hirsch, P.; Rother, C.; Schiffrin, A.; Krull, C. Artificial-intelligence-driven scanning probe microscopy. Commun. Phys. 2020, 3, 54.DOI
- [16]Alldritt, B.; Urtev, F.; Oinonen, N.; Aapro, M.; Kannala, J.; Liljeroth, P.; Foster, A. S. Automated tip functionalization via machine learning in scanning probe microscopy. Comput. Phys. Commun. 2022, 273, 108258.
- [17]Su, J.; Li, J.; Guo, N.; Peng, X.; Yin, J.; Wang, J.; Lyu, P.; Luo, Z.; Mouthaan, K.; Wu, J.; Zhang, C.; Wang, X.; Lu, J. Intelligent synthesis of magnetic nanographenes via chemist-intuited atomic robotic probe. Nat. Synth. 2024, 3, 466–476.DOI
- [18]Zhu, Z.; Yuan, S.; Yang, Q.; Jiang, H.; Zheng, F.; Lu, J.; Sun, Q. Autonomous Scanning Tunneling Microscopy Imaging via Deep Learning. J. Am. Chem. Soc. 2024, 146, 29199–29206.
- [19]Ramsauer, B.; Cartus, J. J.; Hofmann, O. T. MAM-STM: A software for autonomous control of single moieties towards specific surface positions. Comput. Phys. Commun. 2024, 109264.arxiv.org/abs/2404.09694
- [20]Leinen, P.; Esders, M.; Schütt, K. T.; Wagner, C.; Müller, K.-R.; Tautz, F. S. Autonomous robotic nanofabrication with reinforcement learning. Sci. Adv. 2020, 6, eabb6987.
- [21]Ramsauer, B.; Simpson, G. J.; Cartus, J. J.; Jeindl, A.; García-López, V.; Tour, J. M.; Grill, L.; Hofmann, O. T. Autonomous Single-Molecule Manipulation Based on Reinforcement Learning. J. Phys. Chem. A 2023, 127, 2041–2050.DOI
- [22]Chen, I.-J.; Aapro, M.; Kipnis, A.; Ilin, A.; Liljeroth, P.; Foster, A. S. Precise atom manipulation through deep reinforcement learning. Nat. Commun. 2022, 13, 7499.DOI
- [23]Clair, S.; de Oteyza, D. G. Controlling a chemical coupling reaction on a surface: tools and strategies for on-surface synthesis. Chem. Rev. 2019, 119, 4717–4776.DOI
- [24]Grill, L.; Dyer, M.; Lafferentz, L.; Persson, M.; Peters, M. V.; Hecht, S. Nano-architectures by covalent assembly of molecular building blocks. Nat. Nanotechnol. 2007, 2, 687–691.DOI
- [25]Grossmann, L.; King, B. T.; Reichlmaier, S.; Hartmann, N.; Rosen, J.; Heckl, W. M.; Björk, J.; Lackinger, M. On-surface photopolymerization of two-dimensional polymers ordered on the mesoscale. Nat. Chem. 2021, 13, 730–736.DOI
- [26]Grill, L.; Hecht, S. Covalent on-surface polymerization. Nat. Chem. 2020, 12, 115–130.
- [27]Sweetman, A.; Champness, N. R.; Saywell, A. On-surface chemical reactions characterised by ultra-high resolution scanning probe microscopy. Chem. Soc. Rev. 2020, 49, 4189–4202.DOI
- [28]Zhong, Q.; Ihle, A.; Ahles, S.; Wegner, H. A.; Schirmeisen, A.; Ebeling, D. Constructing covalent organic nanoarchitectures molecule by molecule via scanning probe manipulation. Nat. Chem. 2021, 13, 1133–1139.DOI
- [29]Ren, J. et al. On-surface synthesis of ballbot-type N-heterocyclic carbene polymers. Nat. Chem. 2023, 15, 1737–1744.DOI
- [30]Sun, K.; Silveira, O. J.; Ma, Y.; Hasegawa, Y.; Matsumoto, M.; Kera, S.; Krejčí, O.; Foster, A. S.; Kawai, S. On-surface synthesis of disilabenzene-bridged covalent organic frameworks. Nat. Chem. 2023, 15, 136–142.DOI
- [31]Hla, S.-W.; Bartels, L.; Meyer, G.; Rieder, K.-H. Inducing all steps of a chemical reaction with the scanning tunneling microscope tip: towards single molecule engineering. Phys. Rev. Lett. 2000, 85, 2777.
- [32]Pavlicek, N.; Majzik, Z.; Collazos, S.; Meyer, G.; Pérez, D.; Guitián, E.; Peña, D.; Gross, L. Generation and characterization of a meta-aryne on Cu and NaCl surfaces. ACS nano 2017, 11, 10768–10773.DOI
- [33]Kaiser, K.; Scriven, L. M.; Schulz, F.; Gawel, P.; Gross, L.; Anderson, H. L. An sp-hybridized molecular carbon allotrope, cyclo[18] carbon. Science 2019, 365, 1299–1301.
- [34]Albrecht, F.; Fatayer, S.; Pozo, I.; Tavernelli, I.; Repp, J.; Peña, D.; Gross, L. Selectivity in single-molecule reactions by tip-induced redox chemistry. Science 2022, 377, 298–301.DOI
- [35]Ruan, Z.; Schramm, J.; Bauer, J. B.; Naumann, T.; Bettinger, H. F.; Tonner-Zech, R.; Gottfried, J. M. Synthesis of Tridecacene by Multistep Single-Molecule Manipulation. J. Am. Chem. Soc. 2024, 146, 3700–3709.DOI
- [36]Gao, H.-J.; Gao, L. Scanning tunneling microscopy of functional nanostructures on solid surfaces: Manipulation, self-assembly, and applications. Prog. Surf. Sci. 2010, 85, 28–91.DOI
- [37]Blum, V.; Gehrke, R.; Hanke, F.; Havuk, P.; Havu, V.; Ren, X.; Reuter, K.; Scheffler, M. Ab initio molecular simulations with numeric atom-centered orbitals. Comput. Phys. Commun. 2009, 180, 2175–2196.
- [38]Todorović, M.; Gutmann, M. U.; Corander, J.; Rinke, P. Bayesian inference of atomistic structure in functional materials. npj Comput. Mater. 2019, 5, 35.DOI
- [39]Haarnoja, T.; Zhou, A.; Hartikainen, K.; Tucker, G.; Ha, S.; Tan, J.; Kumar, V.; Zhu, H.; Gupta, A.; Abbeel, P.; Levine, S. Soft Actor-Critic Algorithms and Applications. 2019; https://arxiv.org/abs/1812.05905, (accessed 01 29, 2019).
- [40]Lesnard, H.; Bocquet, M.-L.; Lorente, N. Dehydrogenation of aromatic molecules under a scanning tunneling microscope: Pathways and inelastic spectroscopy simulations. J. Am. Chem. Soc. 2007, 129, 4298–4305.
- [41]Kawai, S.; Krejčí, O.; Nishiuchi, T.; Sahara, K.; Kodama, T.; Pawlak, R.; Meyer, E.; Kubo, T.; Foster, A. S. Three-dimensional graphene nanoribbons as a framework for molecular assembly and local probe chemistry. Sci. Adv. 2020, 6, eaay8913.DOI
- [42]Cheriet, M.; Said, J. N.; Suen, C. Y. A recursive thresholding technique for image segmentation. IEEE Trans. Image Process. 1998, 7, 918–921.DOI
- [43]Gong, J.; Li, L.; Chen, W. Fast recursive algorithms for two-dimensional thresholding. Pattern Recognit. 1998, 31, 295–300.DOI
- [44]Becke, A. D. Density-functional exchange-energy approximation with correct asymptotic behavior. Phys. Rev. A 1988, 38, 3098–3100.
- [45]Lee, C.; Yang, W.; Parr, R. G. Development of the Colle-Salvetti correlation-energy formula into a functional of the electron density. Phys. Rev. B 1988, 37, 785–789.
- [46]Tkatchenko, A.; Scheffler, M. Accurate Molecular Van Der Waals Interactions from Ground-State Electron Density and Free-Atom Reference Data. Phys. Rev. Lett. 2009, 102, 073005.
- [47]Ruiz, V. G.; Liu, W.; Zojer, E.; Scheffler, M.; Tkatchenko, A. Density-Functional Theory with Screened van der Waals Interactions for the Modeling of Hybrid Inorganic-Organic Systems. Phys. Rev. Lett. 2012, 108, 146103.
- [48]Liu, W.; Maaß, F.; Willenbockel, M.; Bronner, C.; Schulze, M.; Soubatch, S.; Tautz, F. S.; Tegeder, P.; Tkatchenko, A. Quantitative Prediction of Molecular Adsorption: Structure and Binding of Benzene on Coinage Metals. Phys. Rev. Lett. 2015, 115, 036104.
- [49]Tersoff, J.; Hamann, D. R. Theory of the scanning tunneling microscope. Phys. Rev. B 1985, 31, 805.
- [50]Momma, K.; Izumi, F. VESTA 3 for three-dimensional visualization of crystal, volumetric and morphology data. J. Appl. Crystallogr. 2011, 44, 1272–1276.DOI
- [51]Jónsson, H.; Mills, G.; Jacobsen, K. W. Classical and quantum dynamics in condensed phase simulations; World Scientific, 1998; pp 385–404.
- [52]Peters, B.; Heyden, A.; Bell, A. T.; Chakraborty, A. A growing string method for determining transition states: Comparison to the nudged elastic band and string methods. J. Chem. Phys. 2004, 120, 7877–7886.
- [53]He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2016; pp 770–778.
- [54]Kingma, D. P.; Ba, J. Adam: A Method for Stochastic Optimization. 2017; https://arxiv.org/abs/1412.6980, (accessed 01 30, 2017).
- [55]Bradski, G.; Kaehler, A. Learning OpenCV: Computer vision with the OpenCV library; "O’Reilly Media, Inc.", 2008; pp 334–352.
- [56]Kullback, S.; Leibler, R. A. On information and sufficiency. Ann. Math. Stat. 1951, 22, 79–86.
- [57]Andrychowicz, M.; Wolski, F.; Ray, A.; Schneider, J.; Fong, R.; Welinder, P.; McGrew, B.; Tobin, J.; Pieter Abbeel, O.; Zaremba, W. Hindsight experience replay. Adv. Neural Inf. Process. Syst. 2017, 30, 5048–5058.arxiv.org/abs/1707.01495