Development of a Deep Reinforcement Learning Approach for Transition State Search in Molecular Systems: A Transferable Model Across Structural and Dimensional Variations

We developed a DRL approach for TS searches combining the deep Q-network and the bidirectional long short-term memory architectures to handle variable-sized molecular systems. We applied our method to 1,3,5-hexatriene (Triene), alanine dipeptide (Ala2), and 1,3,5,7-octatetraene (Tetraene) as target systems with the search space limited to backbone dihedral angles. Our model successfully learned the TS search tasks for all examined systems. More importantly, we demonstrate that our model pre-trained on Triene exhibits distinct transfer-learning capabilities across both backbone modifications (Triene to Ala2) and chain extensions (Triene to Tetraene). Such transferability is highly promising for future applications to molecules in general, as it allows an agent trained on simple systems to accelerate TS searches in larger or more complex ones. We expect that these findings on cross-system transferability represent an important step forward in developing DRL-based approaches for TS searches.