WorksComputer Science
Mechanism-Level Chemical Reaction Simulation with Electron Bookkeeping Transformer
Abstract
Modeling chemical reaction mechanisms is central to understanding and predicting chemical reactivity. We present Electron Bookkeeping Transformer (Bookkeeper), a data-driven model that simulates chemical reactions at mechanistic resolution by generating electron movement trajectories. Following the symbolic arrow-pushing formalism in chemistry, Bookkeeper explicitly models each electron as a token and predicts its movement sequentially, ensuring mass and charge conservation during both elementary step and reaction pathway predictions by construction. On reaction-size-balanced test sets, Bookkeeper improves top-1 elementary-step accuracy, top-1 pathway rank accuracy, and top-1 to top-5 product prediction accuracy over FlowER. On the entire test set, it achieves higher elementary-step top-1 accuracy but lower pathway accuracy than FlowER. Bookkeeper also samples elementary steps substantially faster, adapts to several unseen reaction types with minimal fine-tuning, and generates candidate chemical transformations for further screening. These results suggest that explicit electron bookkeeping is a useful framework for virtually screening chemical reactions across diverse reagents, validating computer-assisted synthesis planning predictions, and providing mechanistic interpretations of reactivity.
Introduction
Chemists explain transformations from reactants to products in a reaction in terms of their mechanisms, which are ordered sequences of elementary steps where electrons move, and bonds form and break. This mechanistic approach is central to reaction development and screening, impurity analysis, and the discovery of new transformations [41, 22, 20, 8]. Historically, however, machine learning models for reaction prediction are trained to predict only the final major product from a set of reactants, often using SMILES translation, graph edits, or template classification [40, 33, 32, 37, 17, 10, 11, 29, 44]. Although such models can be highly accurate in endpoint prediction, they generally do not explain how products form, often disregard reagents and catalysts, and provide limited information about intermediates, byproducts, or competing pathways compared to those that operate mechanistically [20, 36, 24, 12].
Recent efforts to construct mechanistic reaction datasets have made it increasingly feasible to revisit reaction prediction at the mechanism level. In particular, large-scale mechanistic datasets generated from expert-curated elementary reaction templates provide sequences of intermediates between experimentally reported reactants and products, enabling models to predict elementary steps rather than only global outcomes [20, 7]. For example, Young et al. [20] constructed such a dataset by applying expert elementary reaction templates to reactions in the USPTO database, recovering experimentally recorded products and yielding millions of elementary mechanistic steps [20]. Mechanistic models built on similar data have also recovered reaction pathways, identified reagent and catalyst roles, and predicted byproducts or impurities that endpoint models often omit [20, 8, 36].
However, reaction prediction models can still inherit limitations from their representations. Sequence-and string-based models have flexible input formats, most often operating on SMILES strings, but are often not mass-conserved and can delete or hallucinate atoms [32, 37, 25]. Graph-edit models can conserve atoms through their more constrained data structures, but do not directly model electron flow as the primary generative action and are reported to produce nonphysical steps when bond, hydrogen, and charge edits are predicted independently [29, 18, 17, 10]. Earlier models such as ELECTRO and NERF explicitly connect reaction prediction to electron movement, but often rely on graph representations of molecules and represent electron flow indirectly as redistributions [4, 3]. In more recent work, Reactron predicts reaction outcomes and accompanying arrow-pushing explanations from reactants, showing the value of using chemical mechanisms as a medium for interpretable generation [8]. FlowER further models electron movements as a continuous flow and provides a recent open-source baseline for generative mechanism prediction, but its continuous BE-matrix formulation requires sum-safe rounding after inference to recover chemically valid states [19]. In contrast, our formulation generates relocations of already-existing individual electron tokens as discrete operations, obviating the need for rounding. The model’s representation, sequential state-updating training objective, and inference procedure are distinct from FlowER, while both build on previous electron flow modeling.
We thus introduce Electron Bookkeeping Transformer (BOOKKEEPER), a generative transformer for chemical reaction mechanism prediction. The model represents a mechanistic state as a set of atoms and a multiset of valence electrons, where each electron token is assigned to either a single atom as a lone electron or between two atoms as a bonding electron. An elementary arrow-pushing action moves one existing electron from a source location to a destination location, and repeating such actions updates the electron bag and produces a sequence of electron movements. Because generation moves existing electron tokens over a fixed atom set, BOOKKEEPER preserves both atoms and the total number of electrons by construction. Empirically, BOOKKEEPER improves elementary-step prediction and top-1 through top-5 pathway recovery on reaction-size-balanced test subsets relative to FlowER [19] and MechSMILES T5 [25]. On the full test set, however, FlowER achieves overall higher pathway accuracy, while BOOKKEEPER retains a smaller top-1 elementary-step advantage. BOOKKEEPER also offers 100 faster elementary-step sampling, which is useful for large-scale reaction screening or exploration of deep or broad mechanistic pathways. We also find that BOOKKEEPER performs similarly to FlowER on out-of-distribution reactions and can be effectively fine-tuned with minimal additional data. Furthermore, we conduct combinatorial virtual reaction screening following Chen et al. [8] to demonstrate that our approach can predict novel reactivity as well as extrapolate known mechanisms. We also examine predictions of representative regioselectivity examples and responses to stoichiometric perturbations, suggesting that our model can learn chemically meaningful preferences.
Our contributions can be summarized as follows:
We introduce Electron Bookkeeping Transformer (BOOKKEEPER), which represents electrons as tokens and generates chemical reaction mechanisms as sequences of electron movements. This tokenized representation preserves mass and charge conservation by construction and produces interpretable arrow-pushing trajectories.
We characterize the accuracy of BOOKKEEPER on a large mechanistic dataset used in prior work and reaction-size-balanced subsets, identifying strengths in top-ranked elementary-step prediction as well as limitations in full-pathway recovery. Its generative architecture enables efficient elementary-step sampling and complete trajectory generation without beam search.
We demonstrate BOOKKEEPER’s adaptation to out-of-distribution reaction types and corroborate its chemically meaningful behavior through combinatorial reaction discovery, regioselectivity, and robustness to excess reagents.
Related work
Mechanisms in chemical reaction prediction Machine-learning methods for reaction prediction have typically focused on predicting the final major product from a set of reactants and reagents. Approaches in the past decade have posed reaction prediction as template selection, reaction-center identification, or learned chemical reactivity prediction [40, 9, 33]. Later models improved flexibility by representing reactions as SMILES-to-SMILES translation problems, graph-to-sequence problems, molecular graph edits, or related graph-based formulations for retrosynthesis and synthesis planning [17, 10, 32, 37, 29]. These endpoint-prediction models have converged to high accuracies on typical benchmark datasets, but they abstract away the elementary steps by which transformations occur, providing little information about intermediates, byproducts, or the mechanistic basis of selectivity.
Mechanistic reaction prediction models address this limitation by predicting sequences of elementary intermediate states rather than only final products [20, 8]. No ELECTRON LEFT BEHIND first demonstrated the value of mechanistic, electron-based rules before the current deep-learning era [5]. More recent models and representations such as RMechRP, DeepMech, and MechSMILES predict radical mechanisms through contrastive learning over reactive atom or orbital pairs [36], use mechanistic supervision for interpretable stepwise prediction [12, 24], or formulate arrow-pushing mechanisms into text to fine-tune large language models [25]. Since balanced chemical reactions can be fully described by changes in electron distribution among atoms of reacting molecules, some mechanistic approaches further focus on modeling electron movement: ELECTRO introduced a generative model of pseudo-electron paths [4], predictors of electron redistribution and movement represent reactions through changes in electron or bond structure [3, 8], and FlowER formulates mechanism prediction as electron flow matching over continuous bond-electron matrices [19]. Notably, Reactron [8] predicts reaction mechanisms by iteratively generating electron-pair movements between source and sink sites, updating the molecular state after each predicted electron path. BOOKKEEPER similarly generates reactions through successive electron-movement predictions. Unlike Reactron, BOOKKEEPER explicitly represents individual valence electrons as tokens, updates their locations one movement at a time, and employs shuffled-prefix training to accommodate the unordered nature of electron movements within an elementary step.
Autoregressive and discrete generative modeling Autoregressive generative models have been widely used in language modeling [2, 35, 39, 43], and related trajectory-modeling approaches similarly treat decision processes as sequences of actions [6, 16]. Discrete generative modeling has also been studied through permutation-aware training, denoising, masked generation, and constrained decoding [42, 1, 13, 30, 21]. This is especially chemically relevant, as reaction mechanisms do not have a unique canonical arrow order when multiple electrons move in a concerted manner (i.e., simultaneously) within a single mechanistic step. To account for this, BOOKKEEPER is trained on shuffled partially completed electron-flow states, encouraging mechanism completion without relying on one arbitrary total ordering of arrows. During inference, we use frequency-ranked stochastic sampling instead of relying on raw model scores [14, 23].
Methods
Formulation
Representation for molecules A structure in a mechanistic reaction state includes the set of atoms present in a reaction , where each atom is defined by its element type and indexed. The set of atoms is conserved through any prediction, as mechanisms may move electrons and change bonding but do not create or delete atoms. A valence electron in a reaction is either part of a lone pair or radical on a single atom or a bonding electron shared between two atoms, so we represent each electron by its location with , where denotes an electron localized on atom , and denotes an electron shared between atoms and . Since electrons in covalent bonds are shared symmetrically, and represent electrons in the same set of bonds. During training, for each electron are randomly swapped with probability 0.5, augmenting both equivalent endpoint orderings. An electron bag is thus a multiset denoting an electron configuration over electron locations, where is the space of possible electron locations over the atom set. The multiplicity of a location records how many electrons occupy that lone or bonding position. For example, two electrons between atoms correspond to a single bond, four to a double bond, and six to a triple bond.

Figure 1. Electron bookkeeping formulation for mechanism-level reaction simulation. (a) BOOKKEEPER converts an atom-mapped molecular state in SMILES format into an atom structure and an electron bag. (b) Given the current electron bag, BOOKKEEPER predicts an electron movement in stages through a transformer architecture by selecting a source electron and then predicting the first and second atom indices of the destination location. (c) Example mechanistic steps in a reaction pathway for an electrophilic aromatic substitution reaction.
Updates to reaction states Any state can therefore be represented as , denoting a specific discrete distribution of electrons in over . An arrow denotes an arrow-pushing action that moves one electron from a source location to a destination location . Each arrow updates the electron bag by removing one electron from the source and adding one electron to the destination, yielding . The atom set is unchanged, so the updated state is .
Mass-conserved reaction steps and trajectory An elementary step is an unordered set of arrows that results in the next reaction state , where is typically between 1 and 6. A mechanistic reaction pathway is a finite sequence of elementary steps, represented as a trajectory of states . Because each update removes and adds exactly one electron and the atom set is conserved, , for all . The numbers of atoms and electrons are invariant by construction.
Chemical validity We define a generated elementary step to be chemically valid if all its arrows can be applied to the current electron bag and the resulting electron configuration satisfies valence and bond formation rules for each atom. A generated pathway is chemically valid if every generated elementary step yields a valid molecular state.
Autoregressive transformer model to predict electron movements
BOOKKEEPER defines a stochastic policy over electron-movement actions. Given a current mechanistic state , the model samples one electron movement at a time, applies the movement to the electron bag, and repeats until an elementary step is completed. Thus, an elementary mechanism with arrows is generated as , with , , and . Atom and electron representations are tokenized, and each electron is additionally appended with a copy index that distinguishes multiple electrons occupying the same location. Tokens are then concatenated and passed through a transformer encoder [39], allowing electron tokens to attend to atoms, other electrons, and the current global mechanistic context.
The model predicts an electron movement in three sequential stages. First, it predicts a gate distribution over all current electron tokens plus a stop action, with . The gate determines whether generation should continue and, if so, which source electron should move next. If the stop action is selected, no additional electron movement is generated for the current elementary step. If an electron token is selected, that token defines the source electron and its current source location . Second, conditioned on the selected source electron and the current state, the model predicts the first atom index of the destination location with . Third, conditioned additionally on this first destination atom, the model predicts the second atom index with .
Equivalently, the model factorizes the electron-movement probability as
The electron bag is then updated with the sampled action to arrive at the next state , from which the model samples the next action.
Training
Electron movements are considered simultaneous in the same elementary step. We train BOOKKEEPER with shuffled-prefix supervision to avoid introducing unnecessary order dependence among arrows in the same step. During training, for an elementary step with ground-truth arrows , we sample a partial mechanistic step ("prefix") and require the model to generate a valid continuation. Specifically, we choose a random ordering of the arrows, sample a prefix length , apply the prefix actions to the initial electron bag, and train the model from the resulting partial state . The remaining actions define the set of valid next generative decisions from .
The model is then trained using the three-stage action factorization described in the previous section: gate/source selection, first destination atom prediction, and second destination atom prediction. During training, the gate distribution is supervised with a multi-positive target over all remaining ground-truth source electrons. Let denote the set of electron-token indices corresponding to remaining source locations in the current electron bag. If , every element of is treated as a valid positive gate choice. If , the stop class is the only positive gate target. The gate loss is therefore
where when no arrow remains. This multi-positive gate objective allows any remaining valid electron movement to be selected next, rather than forcing the model to predict one arbitrary next arrow.
For each moved electron, the destination objective supervises the two atom indices defining its target electron location. If a remaining arrow has source gate index and destination location , the destination contribution gives objective
Therefore, the full training objective is . By exposing the model to many shuffled partial mechanistic steps, shuffled-prefix training encourages generation without reliance on a single imposed arrow order.
Generative prediction of elementary steps and reaction pathways
Elementary step sampling At inference time, BOOKKEEPER generates mechanistic elementary steps by stochastic autoregressive sampling. Given an initial state , the model repeatedly samples a source electron and destination location and updates the electron bag with the predicted arrow. Because the arrows within an elementary step are modeled sequentially even though they collectively describe a single chemical transformation, intermediate electron configurations may not correspond to complete molecular states. We therefore identify completed elementary steps using chemical integrity criteria. Specifically, after each sampled arrow update, we examine atoms touched by the generated arrows and require that: (i) each touched atom has nonzero associated electrons, (ii) each touched atom has an even number of bonding electrons, so that bonds correspond to paired-electron bond orders, and (iii) the total number of lone plus bonding electrons around each touched atom does not exceed its valence-shell capacity. Since the model also predicts a stop action as described in Section 3.2, elementary-step prediction terminates either when the stop action is predicted or when it passes these integrity checks, whichever happens first. This procedure ensures that extracted elementary steps satisfy the implemented chemical constraints, without guaranteeing physical or kinetic feasibility. Generated samples that contain trivial reverse loops, where an electron movement is later immediately reversed by the opposite arrow, are discarded.
Pathway generation We can directly generate complete reaction pathways by repeatedly sampling arrows as described above until the model predicts the stop action for all electrons or the maximum iteration limit terminates the trajectory. Using this method, the model samples an electron trajectory from overall reactants in state to overall predicted products in , through a series of arrows describing a sequence of elementary steps. To extract intermediate states from the trajectory, we apply the same chemical validity criteria checks as above to identify elementary steps.
Besides the natural sequential trajectory formulation that predicts an entire mechanism end-to-end as a consecutive sequence of arrows, BOOKKEEPER is also compatible with other inference algorithms such as beam search. At each pathway depth, the model generates and frequency-ranks candidate next elementary steps, and beam search retains a set of partial pathways for further expansion. Thus, the same elementary-step generation process can be broadly used as a local generative transition model for broader pathway search.
Experiments
We design experiments to address four aspects of BOOKKEEPER: First, following Joung et al. [19], we evaluate its in-distribution performance on both top- elementary-step prediction and top- reaction pathway prediction, as well as demonstrate its higher inference speed compared to benchmark models. Second, also following Joung et al. [19], we evaluate its out-of-distribution generalization and effectiveness of data-efficient fine-tuning. Third, following Chen et al. [8], we examine its application to combinatorial reaction discovery to assess whether the model can predict plausible and diverse reactivity from enumerated reactant pools. Fourth, we probe whether BOOKKEEPER learns chemically meaningful principles by investigating its regioselective behavior and robustness to excess reagents.
Performance
We evaluate BOOKKEEPER on two in-distribution test settings: reaction-size-balanced subsets and the entire FlowER test set. The full FlowER test set contains 28,049 unique chemical reactions. To facilitate evaluation with end-to-end trajectory sampling, we construct smaller subsets of 1,010 reactions balanced across six atom-count bins, increasing the representation of larger reactions relative to the full test distribution. We compare against FlowER [19] and MechSMILES T5 [25], the two closest open source baselines for generative mechanistic prediction. FlowER is a continuous BE-matrix electron-flow model, while MechSMILES T5 represents mechanisms as textual arrow-pushing sequences, then uses this formulation to train a T5 encoder-decoder model [27]. The balanced-subset comparisons use the released baseline checkpoints of both benchmarking models on the mechanistic dataset of Joung et al. [19]; the full test set results compare BOOKKEEPER with FlowER.
Elementary-step prediction For elementary-step prediction, each model is given a mechanistic state and asked to predict the next state . For BOOKKEEPER, we sample 64 candidate elementary steps, convert each resulting electron bag into a molecule, canonicalize the predicted molecule, group duplicate predictions, and rank candidates by empirical frequency. A prediction is counted as correct if the canonicalized SMILES of the next intermediate or product after one generated elementary step exactly matches the ground-truth next intermediate or product.
Pathway accuracy Full-pathway accuracy evaluates the prediction bottleneck of the ground-truth elementary steps along a reference pathway [19]. Following the pathway accuracy definition of FlowER, for a reaction with different reference pathways, let be the length of the -th pathway, and be the rank of the ground truth predicted from the starting state at step of pathway . The rank of the reaction is . Top- full-pathway accuracy is the percentage of reactions with , requiring every ground-truth elementary step along at least one reference pathway to rank within the top .
Product prediction For product prediction, we sample complete pathways from reactants through intermediates to final products. We canonicalize the final products, group equivalent product predictions, and rank them by frequency. Top- product accuracy is the percentage of reactions whose ground-truth final product appears among the top ranked products. We use independent linear trajectories for BOOKKEEPER generation and compare it against beam search strategies used by FlowER and MechSMILES.
These three prediction tasks test complementary aspects of mechanistic modeling. Elementary-step accuracy measures recovery of the next mechanistic state; full-pathway accuracy assesses recovery of all steps along at least one reference pathway per reaction under a rank threshold; and product accuracy measures the recovery of the final product from sampled pathways. On the reaction-size-balanced subsets (Table 1), BOOKKEEPER achieves the highest elementary-step top-1 accuracy (86.9%), while FlowER performs better at top-5 and top-10. For pathway prediction, BOOKKEEPER slightly outperforms FlowER at top-1, whereas FlowER achieves higher accuracy at top-5 and top-10. In product prediction, BOOKKEEPER achieves the highest top-1 (59.9%) and top-5 (89.1%) accuracies, while FlowER leads at top-10. On the full test set (Table 2), BOOKKEEPER achieves 90.72% elementary-step top-1 accuracy versus 88.48% for FlowER, but FlowER is higher at top-2 through top-10. For complete pathways, FlowER outperforms BOOKKEEPER at every reported rank. The contrasting rankings between the two test settings demonstrate that overall model performances are sensitive to the test set reaction-size distribution. We further attribute the low reaction pathway accuracy of BOOKKEEPER (Table 2) to the sequential generation of complete pathways rather than through a beam search or top- sampling.
| Model | Step accuracy (%) Top-1 | Step accuracy (%) Top-5 | Step accuracy (%) Top-10 | Path accuracy (%) Top-1 | Path accuracy (%) Top-5 | Path accuracy (%) Top-10 | Product accuracy (%) Top-1 | Product accuracy (%) Top-5 | Product accuracy (%) Top-10 |
| MechSMILES T5 | 68.3 | 82.6 | 82.8 | - | - | - | 47.6 | 77.4 | 78.2 |
| FlowER | 80.3 | 98.8 | 99.1 | 81.9 | 95.7 | 95.7 | 42.7 | 87.5 | 95.3 |
| BOOKKEEPER | 86.9 | 93.4 | 93.7 | 82.9 | 85.1 | 85.1 | 59.9 | 89.1 | 90.1 |
Table 1. Top- elementary-step, full-pathway, and product prediction accuracies (%) on the balanced test set. Path accuracy uses the best route per reaction under the worst-step rank criterion, following FlowER. Product accuracy is the percentage of reactions whose ground-truth final product appears among the top ranked products.
| Model | Step accuracy (%) Top-1 | Step accuracy (%) Top-2 | Step accuracy (%) Top-5 | Step accuracy (%) Top-10 | Path accuracy (%) Top-1 | Path accuracy (%) Top-2 | Path accuracy (%) Top-5 | Path accuracy (%) Top-10 |
| FlowER | 88.48 | 96.42 | 98.60 | 98.82 | 88.97 | 94.75 | 97.39 | 97.70 |
| BOOKKEEPER | 90.72 | 94.08 | 95.11 | 95.37 | 88.48 | 93.37 | 96.14 | 96.26 |
Table 2. Top- elementary-step and full-pathway prediction accuracies (%) on the full FlowER test set. Path accuracy uses the best route per reaction under the worst-step rank criterion, following FlowER. FlowER results are quoted from its paper [19].
Inference speed We also compare elementary-step inference speed as a function of reaction size. We measure speed only at the elementary-step level so that differences in pathway-level search algorithms do not confound the comparison. All models are evaluated on the same dataset using the same NVIDIA RTX 4090 GPU and the same software environment. We group test examples by the number of atoms in the input reaction state and report average sampling time per elementary-step prediction. As shown in Figure 2 (a), BOOKKEEPER is consistently faster than FlowER and MechSMILES T5 across reaction sizes, reflecting the efficiency of directly sampling discrete electron movements rather than generating continuous BE-matrix trajectories or textual mechanism strings.

Figure 2. Inference speed and out-of-distribution fine-tuning of BOOKKEEPER compared with benchmark models. (a) Mean elementary-step inference time as a function of the number of atoms in the reacting molecule set. (b) Top-1 elementary-step accuracy on 12 out-of-distribution reaction types before and after fine-tuning.
Out-of-distribution generalization and data-efficient fine-tuning
We evaluate whether BOOKKEEPER generalizes to reaction types excluded from training and whether it can adapt to these new regimes with limited data (Appendix A.4). We first evaluate BOOKKEEPER trained on only in-distribution data directly on the OOD test reactions, without any reaction-type-specific training. We compare against FlowER without fine-tuning on the same OOD splits. This evaluates zero-shot or direct-transfer generalization to unseen reaction classes. After fine-tuning BOOKKEEPER as described in Section 3, each fine-tuned model is evaluated again on the same held-out OOD test set.
Figure 2(b) shows that BOOKKEEPER generalizes competitively to held-out reaction types before fine-tuning, achieving 56.9% average top-1 accuracy and outperforming FlowER on 7 of 12 classes. With only 32 fine-tuning reactions per type, its average top-1 accuracy increases to 80.1%, improving on 10 of 12 classes. These results suggest that BOOKKEEPER learns reusable electron-movement motifs that can be adapted to new reaction regimes with limited supervision, though certain reaction classes (e.g., Negishi coupling) may require broader coverage of organometallic or reagent-specific chemistry.
Combinatorial screening for potential reaction discovery
As an exploratory application, we use BOOKKEEPER for virtual screening of reagent-substrate combinations and generation of mechanistic hypotheses not represented by templates in the training data. Taking inspiration from the combinatorial reaction prediction setting used by Chen et al. [8], we identify 315 distinct reagent sets from 1,200 expert-curated reaction templates from the mechanistic dataset used in this work [20], and then isolate 387 substrate sets from the test set by removing reagents from each reaction. Pairing every substrate set with every reagent set yields 121,905 combinations. For each combination, BOOKKEEPER generates reaction pathways using frequency-ranked stochastic sampling. The predicted reactions are then categorized into three classes, following the spirit of prior combinatorial reaction discovery analyses [8]: (i) no reactivity, where the model fails to generate a chemically valid transformation; (ii) known reactivity, where the predicted mechanism matches a transformation already in the FlowER templates; and (iii) new or unseen reactivity, where the predicted mechanism corresponds to an unseen reagent-substrate transformation not in the FlowER templates.
Across 121,905 reagent-substrate combinations, BOOKKEEPER predicts 10,575 possible reactive combinations, corresponding to 8.7% of the search space. Of these, 479 are new transformations, representing 4.5% of all predicted reactive combinations and 0.4% of all combinations. This sparsity is expected, as most arbitrary reagent-substrate pairings should not react, and the model should not force products for implausible inputs. Figure 3 shows that BOOKKEEPER can both recover known mechanistic analogs and identify a set of candidate new transformations. Because each prediction includes an explicit electron-movement trajectory, these candidates can be inspected mechanistically and prioritized for expert review or experimental validation.

Figure 3. Combinatorial putative reaction discovery using BOOKKEEPER. (a) Representative subset of the reaction matrix, with each cell corresponding to one set of reagents or substrates. (b) Details of representative predicted new overall transformations identified from the reaction matrix. “New reactivity” is defined as a reaction that does not correspond to any of the FlowER templates, rather than chemistry that is truly new.
Learned chemical principles
Regioselectivity We next examine whether BOOKKEEPER learns chemically meaningful behavior beyond exact-match prediction by investigating its regioselectivity. Regioselective reactions are a direct probe of chemical context because although multiple sites may be formally reactive, steric, electronic, or directing effects often favor one site. Prior work has designed specialized machine learning models to account for nuanced regioselectivity for complex substrates [28, 31, 26].
We examine BOOKKEEPER’s predictions on representative examples, and Figure 4 shows that BOOKKEEPER indeed assigns high probability to chemically preferred reactive sites. In (a), the substrate contains three plausible bromination sites, but the model’s top-ranked prediction exclusively selects the experimentally preferred site. This behavior could be indicative of the model utilizing the surrounding substituent context, including directing effects and electronic differences across the aromatic ring. Figure 4 (b) probes site selectivity in a Michaelis-Arbuzov reaction with competing carbon centers. BOOKKEEPER predicts nucleophilic attack at the accessible carbon with high frequency, generating the correct intermediate, while the sterically hindered carbon is assigned a no-reactivity outcome. This demonstrates the model’s ability to suppress reactivity when the broader molecular environment makes a nominally possible attack unfavorable. These examples suggest that BOOKKEEPER’s electron-movement policy captures context-specific chemical preferences, which are essential for reaction mechanism prediction.

Figure 4. BOOKKEEPER learns chemically meaningful behavior from reaction mechanisms. (a) Regioselective prediction in a substrate with three possible bromination sites for electrophilic aromatic substitution. (b) Site-selective reactivity prediction for a Michaelis-Arbuzov reaction with different carbon hybridizations. (c) Example of balanced vs. unbalanced reactions with imperfect stoichiometry and excess molecules.
Imperfect stoichiometry and excess reagents A practical limitation of mass-conserved mechanistic models is that all reacting species must be present in the reactant pool, often at the exact molecular ratio required by the reaction. Indeed, prior formulations such as FlowER often assume that the input contains the exact number of molecules participating in the reaction [19]. In screening settings, however, the exact molecular ratios of reagents, solvents, additives, or even substrates may be unknown. It is thus useful for a mechanistic model to be robust against excess reactants, preserving nonparticipating species while still predicting the same mechanistic transformation among the reactive components. We test this by increasing the number of reagent and solvent molecules for elementary-step inputs and measuring whether top- exact-match accuracy deteriorates. Specifically, we test a fixed perturbation level with 4 additional copies of a selected nonparticipating reagent or solvent appended, each with distinct atom-map indices. The original reacting species remain unchanged.
The results show that BOOKKEEPER remains robust under this setting, with an elementary-step top-1 accuracy of 84.3%, top-5 of 92.2%, and top-10 of 92.7%. This corresponds to a 2.6-point reduction at top-1 and a 1.0-point reduction at top-10 relative to the original inputs. Thus, even when the input contains additional molecules, the model retains nearly the same frequency-ranked prediction performance as in the perfectly specified setting.
Taken together, these experiments provide complementary evidence that BOOKKEEPER has learned some chemically meaningful behavior beyond benchmark exact-match accuracy. The regioselectivity cases show that the model can distinguish favored and disfavored reaction sites using molecular context, while the excess-species experiments show that it can avoid spurious reactivity when additional molecules are present. These properties support the use of BOOKKEEPER for reaction discovery and screening, where models must evaluate imperfectly specified reactant pools, propose mechanistically interpretable transformations, and avoid overreacting to chemically irrelevant species.
Limitations
BOOKKEEPER predicts symbolic arrow-pushing hypotheses rather than experimentally validated mechanisms or energy landscapes, and as such, conservation of atoms and electrons does not necessarily imply kinetic or thermodynamic feasibility. The model does not explicitly capture excited states, stereochemistry, or any three-dimensional considerations explicitly. Pathway accuracy depends on the sample budget, decoding temperature, and canonicalization. For screening new and unseen reactivity, candidate reactions require further independent literature, computational, and experimental validation. The regioselectivity results are illustrative examples rather than an aggregate selectivity benchmark.
The model’s generalization is also ultimately bounded by the coverage and fidelity of available mechanistic training data. Rare mechanisms, poorly represented reagent classes, and chemistry outside templated reactions may require broader datasets or more targeted fine-tuning. The full-test-set comparison reveals lower overall pathway accuracy than FlowER despite the more favorable top-1 ranking on reaction-size-balanced subsets. Therefore, our model’s performance against benchmarks also depend on the evaluation distribution.
Conclusions
We introduced BOOKKEEPER, which uses a discrete electron-bookkeeping formulation to preserve atoms and electrons by construction while generating interpretable arrow-pushing trajectories. It enables efficient mechanism generation without requiring continuous flow integration or beam search. Across the evaluated reaction sizes, the model achieves substantially faster elementary-step inference than FlowER and MechSmiLES T5, highlighting its computational advantages for large-scale mechanistic reaction modeling. BOOKKEEPER improves top-ranked elementary-step prediction and pathway recovery on reaction-size-balanced benchmarks, while mostly matching FlowER on full-test-set results. BOOKKEEPER also adapts to held-out reaction types with limited data and produces mechanistic hypotheses that can be inspected for reaction screening. Importantly, its efficient sampling makes it particularly promising for applications requiring evaluation of large numbers of reagent–substrate combinations, where inference cost can be a major practical bottleneck. As mechanistic reaction datasets expand, the combination of explicit electron conservation, efficient autoregressive generation, and scalable candidate screening provides a promising foundation for computational exploration of chemical reaction space.
Acknowledgments
This work was supported by the Machine Learning for Pharmaceutical Discovery and Synthesis Consortium and the National Science Foundation under grant no. CHE-2144153. S.L. received additional funding from the MIT Health and Life Sciences Collaborative (HEALS).
Technical appendices and supplementary material
Code and data availability
We include the code and instructions to access the mechanistic dataset [19] at https://github.com/scaoyx/bookkeeper.
Mechanistic data
We train BOOKKEEPER on elementary steps derived from applying templates to experimentally reported reactions, as curated by Joung et al. [20], using the same splits as FlowER [19]. Each training example corresponds to one state transition , where and share the same atom set but differ in their electron configuration. The goal of the model is to generate a set of arrows that transforms into .
Each elementary step is converted from kekulized SMILES strings into a structure and electron bags. We compute the multiset difference between electron bags and , and construct ground truth arrows by matching each removed electron location in to an added electron location in , yielding arrows . The matching is done by constructing source-destination pairs that share the atom through which the electron is transferred, e.g., a lone electron on atom moving to bond , or a bonding electron moving to a lone location on one of its endpoint atoms, with deterministic tie-breaking when there are multiple identical electron locations. Trivial resonance moves within the same aromatic ring are removed.
We further characterize the number of single-electron arrows associated with each elementary step in the mechanistic dataset. Figure 5 shows the resulting distribution. The modal elementary step contains four single-electron arrows, accounting for 65.1% of all steps. The 18.0% containing no electron movements are the products-to-products static steps used to represent stopping conditions for the model to learn.

Figure 5. Distribution of the number of single-electron arrows per elementary step in the mechanistic corpus. The inset shows the sparse long tail spanning 15-38 arrows on a logarithmic count scale.
Architecture, optimization, and training
The BOOKKEEPER encoder has 8 bidirectional transformer layers with 8 attention heads, 512 hidden dimensions, 2048 feed-forward dimensions, and dropout 0.1, totaling 27.42 million trainable parameters. The model supports up to 384 atom tokens. Training uses AdamW with a learning rate of , 100 optimizer-step linear warm-up followed by a constant learning rate, global-norm gradient clipping at 100, and 4 GPUs with 64 examples per GPU (global batch size 256). The random seed is 2026, validation occurs every 1,000 optimizer steps, and training lasted 167 epochs (422,458 optimizer steps, approximately 4 days 19 hours).
Out-of-distribution fine-tuning
We evaluate data-efficient adaptation on the same 12 out-of-distribution (OOD) reaction types and data splits used by FlowER [19]. Starting from a BOOKKEEPER checkpoint pre-trained only on the in-distribution training set, we fine-tune a separate model for each held-out reaction type. For each type, fine-tuning uses only 32 overall reactions from the corresponding OOD training split, while evaluation is performed on the held-out OOD test set. This protocol measures whether a broadly trained electron-movement policy can be adapted to a new mechanistic regime with minimal reaction-type-specific supervision, rather than requiring full retraining on a large specialized dataset.
Evaluation protocol
For evaluation, we generate 64 raw candidate predictions per test input for each compared method while retaining each model’s native generators. Successful outputs are reconstructed and canonicalized. Equivalent molecular outcomes or complete intermediate sequences are grouped by chemical identity, and unique predictions are ranked by descending sample frequency. Invalid, missing, or timed-out raw outputs remain failures and are not replaced. Table 3 and Table 4 report the reaction-size-balanced subset results, with uncertainty estimated by standard deviations from repeated sampling of such balanced subsets. The full test set comparison (Table 2) uses all available test examples and is reported as point estimates without bootstrap error bars. Note that the following results are obtained by running the original FlowER model, not FlowER-large, which was the one evaluated in § 4.
| Model | Top-1 | Top-2 | Top-5 | Top-10 |
| MechSMILES T5 | ||||
| FlowER | ||||
| BOOKKEEPER |
Table 3. Top- elementary-step prediction accuracy (%) on reaction-size-balanced test subsets with standard deviations.
| Model | Top-1 | Top-2 | Top-5 | Top-10 |
| MechSMILES T5 | ||||
| FlowER | ||||
| BOOKKEEPER |
Table 4. Top- reaction pathway prediction accuracy (%) on reaction-size-balanced test subsets with standard deviations.
Shuffled-prefix supervision ablation
To evaluate the contribution of shuffled-prefix supervision, we compare BOOKKEEPER against a fixed-order training variant without shuffled-prefix augmentation. All other experimental settings are held constant. Shuffled-prefix supervision exposes the model to different valid orderings of electron movements within an elementary step, encouraging it to predict the remaining movements from arbitrary partially updated states rather than relying on a predetermined arrow order.
Table 5 reports the top- prediction accuracies for both variants. Shuffled-prefix supervision consistently improves performance across all evaluated . For reaction pathway prediction, top-1 accuracy increases from 54.2% to 59.9%, while top-10 accuracy increases from 87.9% to 90.1%. The effect is considerably larger in the elementary step evaluation, where top-1 accuracy increases from 16.1% to 88.6% and top-10 accuracy increases from 64.6% to 94.3%.
| Task | Training | Top-1 | Top-2 | Top-5 | Top-10 |
| Reaction pathway | Shuffled-prefix | 59.9 | 79.7 | 89.1 | 90.1 |
| No shuffle | 54.2 | 77.1 | 86.2 | 87.9 | |
| Elementary step | Shuffled-prefix | 88.6 | 92.6 | 94.2 | 94.3 |
| No shuffle | 16.1 | 55.2 | 63.5 | 64.6 |
Table 5. Ablation of shuffled-prefix supervision. Top- prediction accuracy (%) with and without shuffled-prefix training. All other experimental settings are held constant.
These results demonstrate the importance of accounting for the permutation invariance of electron movements within an elementary step. Training on a single fixed arrow ordering can introduce dependencies on an arbitrary serialization, whereas shuffled-prefix supervision allows the model to learn from a broader range of chemically equivalent partial-step configurations.
Inference efficiency and robustness experiments
Inference efficiency across reaction sizes We examine elementary-step inference time as a function of the number of atoms in the input reaction state. Table 6 presents timing measurements for BOOKKEEPER, FlowER, and MechSMILES T5 across ten reaction-size groups. The measurements are obtained at the elementary-step level rather than for end-to-end reaction pathway generation, avoiding differences in pathway-level search procedures.
| Mean atoms | BOOKKEEPER | FlowER | MechSMILES T5 |
| 32 | 9.67 ± 0.02 | 110.70 ± 60.73 | 221.37 ± 1.01 |
| 51 | 10.98 ± 0.08 | 94.23 ± 0.96 | 316.37 ± 2.21 |
| 79 | 18.56 ± 0.31 | 94.43 ± 0.19 | 212.33 ± 2.20 |
| 96 | 18.71 ± 0.21 | 95.55 ± 1.75 | 282.26 ± 5.07 |
| 126 | 24.99 ± 0.23 | 92.66 ± 0.50 | 460.17 ± 99.03 |
| 148 | 31.82 ± 0.70 | 95.50 ± 3.06 | 1188.75 ± 353.51 |
| 180 | 29.61 ± 0.05 | 93.21 ± 0.46 | 1851.50 ± 13.14 |
| 204 | 40.85 ± 0.91 | 95.65 ± 1.71 | 586.73 ± 5.71 |
| 223 | 42.78 ± 0.33 | 94.91 ± 0.26 | 1431.76 ± 63.59 |
| 261 | 47.88 ± 0.30 | 95.49 ± 0.52 | 1926.87 ± 126.80 |
Table 6. Elementary-step inference time (ms) across reaction-size groups. Reaction size is measured by the mean number of atoms in each group.
BOOKKEEPER achieves lower elementary-step inference times than both baselines across all evaluated reaction sizes. Its measured inference time increases from 9.67 ms for reactions with an average of 32 atoms to 47.88 ms for reactions with an average of 261 atoms. In contrast, FlowER requires approximately 93-111 ms across these groups, while MechSMILES T5 exhibits substantially higher and more variable inference times, particularly for larger reaction systems.
Robustness to excess reagents and solvents We evaluate the robustness of BOOKKEEPER to imperfectly specified reactant stoichiometry by introducing additional copies of nonparticipating reagents or solvents into elementary-step inputs. For each perturbed reaction, 4 additional copies of the selected nonparticipating species are appended, giving five copies in total. The reacting components remain unchanged, and the appended copies receive unique atom-mapping indices. Exact-match evaluation requires the additional molecules to remain unchanged in the predicted state.
Table 7 compares elementary-step prediction accuracy on the original and perturbed inputs. Adding excess species reduces top-1 accuracy from 86.9% to 84.3% and top-10 accuracy from 93.7% to 92.7%, corresponding to decreases of 2.6 and 1.0 percent, respectively. These results indicate that BOOKKEEPER can preserve nonparticipating molecules while recovering the intended transformation with relatively small accuracy degradation at the tested perturbation level.
| Setting | Top-1 | Top-2 | Top-3 | Top-4 | Top-5 | Top-6 | Top-7 | Top-8 | Top-9 | Top-10 |
| Original | 86.9 | 90.7 | 92.8 | 93.2 | 93.4 | 93.6 | 93.6 | 93.6 | 93.7 | 93.7 |
| Excess | 84.3 | 88.1 | 90.3 | 91.2 | 92.2 | 92.3 | 92.3 | 92.4 | 92.7 | 92.7 |
Table 7. Elementary-step top- accuracy (%) with the original reactant pool and with 4 additional copies of nonparticipating reagents or solvents.
Endpoint ordering and predicted stop actions
BOOKKEEPER’s shuffled-prefix training featurizer draws a random orientation for two-atom electron locations, including the destination of each remaining movement. In the molecular representation, an electron shared by atoms and is equal to one shared by and , independent of orientation.
During elementary-step sampling, chemical integrity checks test atoms appearing in sampled arrows and confirm that each must have a nonzero associated electron count, an even number of bonding electrons, and total incident occupancy no greater than the configured valence-shell capacity. The candidate is accepted immediately when these conditions hold. Furthermore, trajectories containing both a move and its reverse are rejected, whether or not the moves are consecutive. In the linear full-trajectory sampler, no-move, reverse-loop detection, or the maximum number of iterations ends a trajectory. The integrity check is also used as a method to identify elementary steps in the pathway.
Full pathway evaluation
We evaluate full reaction pathway prediction by requiring exact recovery of each ground-truth elementary step in a pathway, rather than only the final reaction products. Starting from the reactants, BOOKKEEPER autoregressively generates individual electron movements and updates the electron configuration after each action. Completed elementary steps are identified using the chemical integrity criteria described in § 3.4 and Appendix § 8.8, yielding a sequence of elementary steps from the initial reactants to the predicted products.
For each test reaction, we generate 64 independent stochastic trajectories using linear autoregressive sampling rather than beam search. The same raw-output budget and post-generation ranking procedure are applied to all benchmark models, while each retains its native generation algorithm. Generated outputs are canonicalized, grouped by equivalent predictions, and ranked according to their empirical frequencies among the 64 samples. Invalid, incomplete, timed-out, or missing predictions are not replaced by additional samples, and all test examples remain in the evaluation denominator.
Sampling temperature sensitivity
We investigate the sensitivity of BOOKKEEPER to sampling temperature by varying the temperature while maintaining a fixed budget of 64 raw predictions per test input. All other experimental settings remain unchanged across different temperatures. We evaluate temperatures of 0.7, 1.0, and 2.0 for both elementary-step prediction and complete reaction pathway generation.
Table 8 presents elementary-step prediction accuracies. Performance remains relatively stable across the evaluated temperatures. Top-1 accuracy varies from 88.4% to 89.0%, while top-10 accuracy ranges from 93.9% to 94.3%. The largest difference across temperatures at any reported is 0.9 percentage points, suggesting that single-step prediction is relatively insensitive to temperature within the investigated range.
| Top- | |||
| 1 | 89.0 | 88.6 | 88.4 |
| 2 | 92.7 | 92.6 | 92.4 |
| 3 | 93.2 | 93.6 | 92.7 |
| 4 | 93.4 | 93.9 | 93.2 |
| 5 | 93.4 | 94.2 | 93.6 |
| 6 | 93.6 | 94.3 | 93.8 |
| 7 | 93.6 | 94.3 | 94.0 |
| 8 | 93.7 | 94.3 | 94.0 |
| 9 | 93.9 | 94.3 | 94.0 |
| 10 | 93.9 | 94.3 | 94.0 |
Table 8. Elementary-step top- prediction accuracy (%) at different sampling temperatures, using 64 samples per input.
In contrast, complete pathway prediction exhibits greater sensitivity to sampling temperature, as shown in Table 9. At , BOOKKEEPER achieves 59.9% top-1 and 90.1% top-10 accuracy. Increasing the temperature to reduces these values to 51.0% and 87.5%, respectively, while yields 57.3% and 81.8%. The lowest evaluated temperature achieves the highest pathway accuracy at every reported .
| Top- | |||
| 1 | 59.9 | 51.0 | 57.3 |
| 2 | 79.7 | 78.6 | 76.6 |
| 3 | 84.9 | 83.3 | 79.2 |
| 4 | 87.0 | 84.4 | 80.2 |
| 5 | 89.1 | 86.5 | 80.7 |
| 6 | 89.6 | 87.0 | 81.3 |
| 7 | 89.6 | 87.5 | 81.3 |
| 8 | 89.6 | 87.5 | 81.8 |
| 9 | 90.1 | 87.5 | 81.8 |
| 10 | 90.1 | 87.5 | 81.8 |
Table 9. Reaction pathway top- prediction accuracy (%) at different sampling temperatures, using 64 samples per input.
The greater temperature sensitivity of pathway prediction is consistent with the accumulation of sampling differences over multiple autoregressive electron movements. Under a fixed sampling budget, lower temperatures concentrate probability mass on higher-probability trajectories, increasing their frequency in the resulting candidate ranking. Higher temperatures likely distribute samples more broadly across possible trajectories, which can reduce exact-match accuracy when predictions are ranked by empirical frequency.
Therefore, we demonstrate a tradeoff between concentrating predictions on high-probability outcomes and exploring alternative reaction trajectories. The preferred sampling temperature may thus depend on whether the application prioritizes exact pathway recovery or broader candidate generation.
Forward major product prediction comparison
Although BOOKKEEPER is designed for mechanistic reaction prediction, the accuracy of its final reaction products provides an additional measure of practical utility. We therefore compare BOOKKEEPER with two established forward reaction prediction models, Graph2SMILES (G2S) [38] and Chemformer [15], on the same major product prediction task.
As in the reaction-size-balanced mechanism evaluation in § 4.1, we partition the mechanistic test set into six bins by input atom count, spanning 50 to 384 atoms, and sample 168 reactions from each bin. Each evaluation sample therefore contains 1,010 reactions. This procedure reduces the dominance of smaller reactions in the original test distribution and provides greater representation of larger molecular systems.
Because conventional forward predictors generally target the major reaction product rather than the complete set of products and byproducts, we heuristically identify the major product for each reference reaction and evaluate all three models against this common target. Predictions are scored using exact major product matching, with invalid SMILES, failed predictions, and missing outputs retained in the evaluation denominator. This metric differs from the complete pathway criteria used elsewhere in this work.
Table 10 reports the resulting top- accuracies. BOOKKEEPER achieves 87.8% top-1 and 94.7% top-10 accuracy, compared with 79.7% and 86.0% for Chemformer and 64.4% and 90.3% for Graph2SMILES. BOOKKEEPER obtains the highest mean accuracy at every reported . Chemformer outperforms Graph2SMILES at top-1, while Graph2SMILES achieves higher accuracy from top-4 onward.
| Top- | BOOKKEEPER | Chemformer | Graph2SMILES |
| 1 | 87.8 ± 3.9 | 79.7 ± 3.2 | 64.4 ± 3.9 |
| 2 | 92.4 ± 2.7 | 83.1 ± 3.1 | 77.8 ± 2.7 |
| 3 | 93.4 ± 2.4 | 84.5 ± 2.4 | 83.0 ± 2.4 |
| 4 | 93.8 ± 2.3 | 84.9 ± 2.2 | 87.4 ± 2.3 |
| 5 | 93.8 ± 2.3 | 85.3 ± 2.0 | 89.4 ± 2.1 |
| 6 | 94.4 ± 2.1 | 85.9 ± 2.0 | 90.3 ± 2.1 |
| 7 | 94.6 ± 2.1 | 86.0 ± 2.0 | 90.3 ± 2.1 |
| 8 | 94.6 ± 2.1 | 86.0 ± 2.0 | 90.3 ± 2.1 |
| 9 | 94.7 ± 2.1 | 86.0 ± 2.0 | 90.3 ± 2.1 |
| 10 | 94.7 ± 2.1 | 86.0 ± 2.0 | 90.3 ± 2.1 |
Table 10. Top- major product prediction accuracy (%) on balanced test sets. Values are reported as mean ± standard deviation across evaluation samples.
We note that the two forward-prediction baselines differ in their pretraining and fine-tuning histories. For Graph2SMILES, we use the baseline checkpoint previously employed by FlowER, initialized from a USPTO-full pretrained model and subsequently fine-tuned on the same mechanistic training distribution as BOOKKEEPER. For Chemformer, we use the publicly released checkpoint fine-tuned on USPTO-mixed, without additional fine-tuning on our mechanistic training set. Consequently, the comparison does not control for differences in training data, pretraining procedures, or model architecture. Potential overlap between the Chemformer training data and our evaluation subset also cannot be excluded.
The different checkpoint histories may partially explain the models’ rank-dependent performance. Chemformer achieves substantially higher top-1 accuracy than Graph2SMILES, whereas Graph2SMILES recovers more reference products at larger . Similar relative trends have been observed in previous comparisons of these architectures, although differences in benchmark distributions make direct attribution of the observed gap to a particular modeling choice difficult.
Overall, these results indicate that the explicit mechanistic representation of BOOKKEEPER does not necessarily compromise major product prediction accuracy on the test set.
Model development and negative results
The design of BOOKKEEPER emerged from exploring alternative formulations of mechanism-level reaction generation. In particular, we investigated whether the complete set of electron movements within an elementary step could be predicted in parallel, rather than generating individual movements autoregressively.
We developed a discrete denoising formulation that predicts the destination of every electron after each elementary step. Starting from the initial electron bag, a featurization module constructs per-electron destination targets together with move/stop indicators. During training, destination coordinates and movement indicators are corrupted, and the denoising model learns to reconstruct the complete post-step electron configuration. At inference time, all electron destinations are predicted through 100 denoising iterations.
We implemented transformer variants with 8 and 12 layers, both using a hidden dimension of 512. In these experiments, predictions were dominated by the no-move outcome; i.e., the model frequently predicted that nearly all electrons should retain their original locations. This behavior is consistent with the strong class imbalance between moving and stationary electrons in typical elementary reactions. Moreover, independently generated electron destinations could produce mutually inconsistent changes when interpreted as a simultaneous set of arrows.
These difficulties motivated the autoregressive formulation used in BOOKKEEPER. Rather than predicting the entire electron redistribution in parallel, the model selects one electron source, predicts its destination, applies the movement to the current state, and conditions subsequent predictions on the updated electron configuration. This sequential formulation allows each prediction to incorporate the consequences of previously generated movements.
Because the arrows within a single elementary step are chemically unordered, sequential generation also introduces a potential dependence on arbitrary arrow serialization. We address this through shuffled-prefix supervision, which exposes the model to multiple orderings and partial configurations of the same elementary step. The ablation in Appendix A.6 demonstrates the importance of this training strategy.
Another practical design choice is the use of copy indices for electron tokens. Multiple electrons can occupy the same location, such as a lone pair or a covalent bond. Assigning distinct computational indices allows the model to select, update, and track individual tokens while preserving the total electron count. These indices distinguish elements of the model representation and do not imply physically distinguishable electron identities.
Our final architecture uses a bidirectional transformer to encode the complete current atom structure and electron bag, and an autoregressive action factorization to predict successive electron movements. An alternative decoder-only transformer could also generate serialized electron-movement sequences, but would require an explicit treatment of the arbitrary ordering of simultaneous arrows. We did not train a matched decoder-only control. Accordingly, the present experiments motivate our architectural choices without necessarily establishing that BOOKKEEPER’s architecture is uniquely optimal.
Template coverage and literature supporting new reactivity candidates
Mechanistic template coverage The mechanistic dataset is predominantly organic chemistry transformations, with comparatively limited representation of organometallic and transition-metal-catalyzed chemistry. To characterize this imbalance, we categorize the reaction templates according to their chemical transformation classes. Table 11 summarizes the resulting distribution.
| Category | Representative reactions | Templates | Share (%) |
| Transition metal catalyzed cross-coupling | Suzuki, Buchwald-Hartwig, Miyaura borylation | 13 | 5.2 |
| Other cross-coupling | Sonogashira, Stille, Kumada | 11 | 4.4 |
| Main group organometallic | Grignard | 10 | 4.0 |
| Organosilicon | Hosomi-Sakurai | 6 | 2.4 |
| Other related organometallic transformations | Barbier, Petasis, Weinreb ketone synthesis, olefin metathesis | 5 | 2.0 |
| Organic transformations | Remaining organic reaction types | 207 | 82.1 |
| Total | 252 | 100.0 |
Table 11. Distribution of chemical transformation classes in the mechanistic template catalog
Of the 252 categorized templates, 207 (82.1%) correspond to other organic reaction types, while the remaining 45 (17.9%) span transition-metal-catalyzed cross-couplings, main-group organometallic transformations, organosilicon chemistry, and related reactions.
This uneven coverage provides a plausible explanation for the comparatively weak performance observed when adapting BOOKKEEPER to Negishi cross-coupling reactions. Although the electron bookkeeping representation can describe electron redistribution involving metal-containing species, reliable prediction of such chemistry possibly depends on sufficient and representative mechanistic training data. The observed limitation therefore partially relates to the chemical coverage of the available corpus rather than an inherent inability of the representation to encode organometallic transformations.
Literature-supported candidate transformation As an illustrative case from combinatorial reaction screening, BOOKKEEPER predicts the methylation of diethyl ether by methyl trifluoromethanesulfonate (methyl triflate, MeOTf), yielding a methyldiethyloxonium triflate species.
The proposed elementary step involves nucleophilic attack by the ether oxygen on the methyl group of MeOTf, accompanied by cleavage of the carbon-triflate bond. This transformation is consistent with an -type methyl transfer and produces a trialkyloxonium cation with triflate as its counteranion. The associated carbon dioxide molecule in the screened reactant combination remains unchanged.
The predicted product belongs to the trialkyloxonium/Meerwein-type family of compounds, for which related species have been documented experimentally. In particular, trialkyloxonium chemistry reported by Staneke et al. [34] [34] provides literature support for the chemical plausibility of this prediction.
This example illustrates how BOOKKEEPER can propose mechanistically interpretable candidate transformations whose plausibility can be assessed against established chemical literature. However, we emphasize that a candidate transformation absent from the training templates is not necessarily a previously undiscovered reaction. Establishing chemical novelty and experimental feasibility requires additional literature analysis, expert evaluation, and computational or downstream experimental validation.
References
- [1]Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Systems, volume 34, pages 17981–17993. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper_files/paper/2021/hash/958c530554f78cdc8e97125b70e6973d-Abstract.html.arxiv.org/abs/2107.03006
- [2]Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model.
- [3]Hangrui Bi, Hengyi Wang, Chence Shi, Connor Coley, Jian Tang, and Hongyu Guo. Non-autoregressive electron redistribution modeling for reaction prediction. In Proceedings of the 38th International Conference on Machine Learning, pages 904–913. PMLR. URL https://proceedings.mlr.press/v139/bi21a.html.arxiv.org/abs/2106.07801
- [4]John Bradshaw, Matt J. Kusner, Brooks Paige, Marwin H. S. Segler, and José Miguel Hernández-Lobato. A generative model for electron paths. URL http://arxiv.org/abs/1805.10970.
- [5]Jonathan H. Chen and Pierre Baldi. No electron left behind: A rule-based expert system to predict chemical reactions and reaction mechanisms. 49(9):2034–2043. ISSN 1549-5956, 1549-960X. doi: 10.1021/ci900157k. URL https://pubs.acs.org/doi/10.1021/ci900157k.DOI
- [6]Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. In Advances in Neural Information Processing Systems, volume 34, pages 15084–15097. Curran Associates, Inc., . URL https://proceedings.neurips.cc/paper_files/paper/2021/hash/7f489f642a0ddb10272b5c31057f0663-Abstract.html.arxiv.org/abs/2106.01345
- [7]Shuan Chen, Ramil Babazade, Taewan Kim, Sunkyu Han, and Yousung Jung. A large-scale reaction dataset of mechanistic pathways of organic reactions. 11(1):863, . ISSN 2052-4463. doi: 10.1038/s41597-024-03709-y. URL https://www.nature.com/articles/s41597-024-03709-y.DOI
- [8]Shuan Chen, Kye Sung Park, Taewan Kim, Sunkyu Han, and Yousung Jung. Predicting chemical reaction outcomes based on electron movements using machine learning, . URL http://arxiv.org/abs/2503.10197.
- [9]Connor W Coley, Regina Barzilay, Tommi S Jaakkola, William H Green, and Klavs F Jensen. Prediction of organic reaction outcomes using machine learning. ACS Central Science, 3(5):434, 2017.DOI
- [10]Connor W. Coley, Wengong Jin, Luke Rogers, Timothy F. Jamison, Tommi S. Jaakkola, William H. Green, Regina Barzilay, and Klavs F. Jensen. A graph-convolutional neural network model for the prediction of chemical reactivity. 10(2):370–377. ISSN 2041-6520, 2041-6539. doi: 10.1039/C8SC04228D. URL https://xlink.rsc.org/?DOI=C8SC04228D.DOI
- [11]Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, and Le Song. Retrosynthesis prediction with conditional graph logic network. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper_files/paper/2019/hash/0d2b2061826a5df3221116a508a6052-Abstract.html.arxiv.org/abs/2001.01408
- [12]Manajit Das, Ajnabiul Hoque, Mayank Baranwal, and Raghavan B. Sunoj. DeepMech: A machine learning framework for chemical reaction mechanism prediction. URL http://arxiv.org/abs/2509.15872.
- [13]Zhengfu He, Tianxiang Sun, Qiong Tang, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. Diffusion-BERT: Improving generative masked language models with diffusion models. In Anna Rogers, Jordan Boyd-Graber, and Naoki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4521–4534. Association for Computational Linguistics. doi: 10.18653/v1/2023.acl-long.248. URL https://aclanthology.org/2023.acl-long.248/.DOI
- [14]Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. URL http://arxiv.org/abs/1904.09751.
- [15]Ross Irwin, Spyridon Dimitriadis, Jiazhen He, and Esben Jannik Bjerrum. Chemformer: a pre-trained transformer for computational chemistry. Machine Learning: Science and Technology, 3(1):015022, 2022.DOI
- [16]Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. In Advances in Neural Information Processing Systems, volume 34, pages 1273–1286. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper_files/paper/2021/hash/099fe6b0b444c23836c4a5a07346082b-Abstract.html.arxiv.org/abs/2106.02039
- [17]Wengong Jin, Connor Coley, Regina Barzilay, and Tommi Jaakkola. Predicting organic reaction outcomes with weisfeiler-lehman network. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper_files/paper/2017/hash/ced556cd9f9c0c8315cfbe0744a3baf0-Abstract.html.arxiv.org/abs/1709.04555
- [18]Kjell Jorner, Tore Brinck, Per-Ola Norrby, and David Buttar. Machine learning meets mechanistic modelling for accurate prediction of experimental activation energies. 12(3):1163–1175. ISSN 2041-6520, 2041-6539. doi: 10.1039/D0SC04896H. URL https://xlink.rsc.org/?DOI=D0SC04896H.DOI
- [19]Joonyoung F. Joung, Mun Hong Fong, Nicholas Casetti, Jordan P. Liles, Ne S. Dassanayake, and Connor W. Coley. Electron flow matching for generative reaction mechanism prediction. 645(8079):115–123. . ISSN 1476-4687. doi: 10.1038/s41586-025-09426-9. URL https://www.nature.com/articles/s41586-025-09426-9.
- [20]Joonyoung F. Joung, Mun Hong Fong, Jihye Roh, Zhengkai Tu, John Bradshaw, and Connor W. Coley. Reproducing reaction mechanisms with machine-learning models trained on a large-scale mechanistic dataset. 63(43):e202411296, . ISSN 1521-3773. doi: 10.1002/anie.202411296. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/anie.202411296. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/anie.202411296.
- [21]Terry Koo, Frederick Liu, and Luheng He. Automata-based constraints for language model decoding. URL http://arxiv.org/abs/2407.08103.
- [22]Johannes T. Margraf, Hyunwook Jung, Christoph Scheurer, and Karsten Reuter. Exploring catalytic reaction networks with machine learning. 6(2):112–121. ISSN 2520-1158. doi: 10.1038/s41929-022-00896-y. URL https://www.nature.com/articles/s41929-022-00896-y.
- [23]Tsvetomila Mihaylova and André F. T. Martins. Scheduled sampling for transformers. In Fernando Alva-Manchego, Eunsol Choi, and Daniel Khashabi, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 351–356. Association for Computational Linguistics. doi: 10.18653/v1/P19-2049. URL https://aclanthology.org/P19-2049/.DOI
- [24]Ryan J. Miller, Alexander E. Dashuta, Brayden Rudisill, David Van Vranken, and Pierre Baldi. Mechanism-aware deep learning for polar reaction prediction. 147(44):4168–4176. ISSN 0002-7863, 1520-5126. doi: 10.1021/jacs.5c16838. URL https://pubs.acs.org/doi/10.1021/jacs.5c16838.DOI
- [25]Théo A. Neukomm, Zlatko Jončev, and Philippe Schwaller. Teaching language models mechanistic explainability through MechSMILES. URL http://arxiv.org/abs/2512.05722. version: 2.
- [26]Giorgio Pesciullesi, Philippe Schwaller, Teodoro Laino, and Jean-Louis Reymond. Transfer learning enables the molecular transformer to predict regio- and stereoselective reactions on carbohydrates. 11 (1):4874. ISSN 2041-1723. doi: 10.1038/s41467-020-18671-7. URL https://www.nature.com/articles/s41467-020-18671-7.DOI
- [27]Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.arxiv.org/abs/1910.10683
- [28]Nicolai Ree, Andreas H. Göller, and Jan H. Jensen. RegioML: predicting the regioselectivity of electrophilic aromatic substitution reactions using machine learning. 1(2):108–114. ISSN 2635-098X. doi: 10.1039/D1DD00032B. URL https://xlink.rsc.org/?DOI=D1DD00032B.
- [29]Mikołaj Sacha, Mikołaj Błaż, Piotr Byrski, Paweł Dąbrowski-Tumański, Mikołaj Chromiński, Rafał Loska, Paweł Włodarczyk-Pruszyński, and Stanisław Jastrzębski. Molecule edit graph attention network: Modeling chemical reactions as sequences of graph edits. 61(7):3273–3284. ISSN 1549-9596, 1549-960X. doi: 10.1021/acs.jcim.1c00537. URL https://pubs.acs.org/doi/10.1021/acs.jcim.1c00537.DOI
- [30]Subham Sekhar Sahoo, Marianne Arriola, and Yair Schiff. Simple and effective masked diffusion language models.arxiv.org/abs/2406.07524
- [31]Jules Schleinitz, Alba Carretero-Cerdán, Anjali Gurajapu, Yonatan Harnik, Gina Lee, Amitesh Pandey, Anat Milo, and Sarah E. Reisman. Designing target-specific data sets for regioselectivity predictions on complex substrates. 147(9):7476–7484. ISSN 0027-863, 1520-5126. doi: 10.1021/jacs.4c15902. URL https://pubs.acs.org/doi/10.1021/jacs.4c15902.DOI
- [32]Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Christopher A. Hunter, Costas Bekas, and Alpha A. Lee. Molecular transformer: A model for uncertainty-calibrated chemical reaction prediction. 5(9):1572–1583. ISSN 2374-7943, 2374-7951. doi: 10.1021/acscentsci.9b00576. URL https://pubs.acs.org/doi/10.1021/acscentsci.9b00576.DOI
- [33]Marwin H. S. Segler and Mark P. Waller. Neural-symbolic machine learning for retrosynthesis and reaction prediction. 23(25):5966–5971. ISSN 1521-3765. doi: 10.1002/chem.201605499. URL https://chemistry-europe.onlinelibrary.wiley.com/doi/abs/10.1002/chem.201605499. _eprint: https://chemistry-europe.onlinelibrary.wiley.com/doi/pdf/10.1002/chem.201605499.DOI
- [34]PO Staneke, G Groothuis, S Ingemann, and NMM Nibbering. Competing electron transfer, proton abstraction and nucleophilic substitutions in gas-phase reactions of (radical) anions with chloro-and bromomethanes. Journal of physical organic chemistry, 9(7):471–486, 1996.doi.org/10.1002/(sici)1099-1395(199607)9:7<471::aid-poc807>3.0.co;2-i
- [35]Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper_files/paper/2014/hash/5a18e133cbf9f257297f410bb7eca942-Abstract.html.arxiv.org/abs/1409.3215
- [36]Mohammadamin Tavakoli, Yin Ting T Chiu, Alexander Shmakov, and Ann Marie Carlton. AI FOR INTERPRETABLE CHEMISTRY: PREDICTING RADICAL MECHANISTIC PATHWAYS VIA CONTRASTIVE LEARNING.arxiv.org/abs/2311.01118
- [37]Igor V. Tetko, Pavel Karpov, Ruud Van Deursen, and Guillaume Godin. State-of-the-art augmented NLP transformer models for direct and single-step retrosynthesis. 11(1):5575. ISSN 2041-1723. doi: 10.1038/s41467-020-19266-y. URL https://www.nature.com/articles/s41467-020-19266-y.DOI
- [38]Zhengkai Tu and Connor W Coley. Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction. Journal of chemical information and modeling, 62(15):3503–3513, 2022.DOI
- [39]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Lion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd30531c4a845aa-Abstract.html.arxiv.org/abs/1706.03762
- [40]Jennifer N. Wei, David Duvenaud, and Alán Aspuru-Guzik. Neural networks for the prediction of organic chemistry reactions. 2(10):725–732. ISSN 2374-7943, 2374-7951. doi: 10.1021/acscentsci.6b00219. URL https://pubs.acs.org/doi/10.1021/acscentsci.6b00219.DOI
- [41]Mingjian Wen, Evan Walter Clark Spotte-Smith, Samuel M. Blau, Matthew J. McDermott, Aditi S. Krishnapriyan, and Kristin A. Persson. Chemical reaction networks and opportunities for machine learning. 3(1):12–24. ISSN 2662-8457. doi: 10.1038/s43588-022-00369-z. URL https://www.nature.com/articles/s43588-022-00369-z.DOI
- [42]Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper_files/paper/2019/hash/dc6a7e655d7e5840e66733e9ee67cc69-Abstract.html.arxiv.org/abs/1906.08237
- [43]Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec. GraphRNN: Generating realistic graphs with deep auto-regressive models. In Proceedings of the 35th International Conference on Machine Learning, pages 5708–5717. PMLR. URL https://proceedings.mlr.press/v80/you18a.html.arxiv.org/abs/1802.08773
- [44]Weihe Zhong, Ziduo Yang, and Calvin Yu-Chian Chen. Retrosynthesis prediction using an end-to-end graph generative architecture for molecular graph editing. 14(1):3009. ISSN 2041-1723. doi: 10.1038/s41467-023-38851-5. URL https://www.nature.com/articles/s41467-023-38851-5.DOI