WorksPhysics and Astronomy
Alpha Zero for Physics: Application of Symbolic Regression with Alpha Zero to find the analytical methods in physics
Abstract
Machine learning with neural networks is now becoming a more and more powerful tool for various tasks, such as natural language processing, image recognition, winning the game, and even for the issues of physics. Although there are many studies on the application of machine learning to numerical calculation and assistance of experiments, the methods of applying machine learning to find the analytical method are poorly studied. In this paper, we propose the frameworks of developing analytical methods in physics by using the symbolic regression with the Alpha Zero algorithm, that is Alpha Zero for physics (AZfP). As a demonstration, we show that AZfP can derive the high-frequency expansion in the Floquet systems. AZfP may have the possibility of developing a new theoretical framework in physics.
Introduction
Recently, machine learning with neural networks has shown the immensely high performance in various areas, such as the natural language processing[1, 2, 3, 4, 5], image recognition[6, 7, 8, 9] and generation[10, 11, 12], mastering games[13, 14, 15, 16, 17]. Also in the area of physics, the intensive studies have begun for the application of machine learning techniques to physics problems, those are the detection of the phase transition[18, 19, 20, 21, 22, 23], calculation of equilibrium states[24, 25, 26, 27] or steady states[28], and materials informatics[29, 30, 31, 32, 33]. In these previous studies, the machine learning techniques are used for the support of experiments or the numerical calculations. On the other hands, one may comes to a question:
Can we use the machine learning techniques for developing the theoretical analysis methods?
Before considering how to use, let us consider what are the theoretical analysis methods. One of the main keywords may be the scale separation and the reduction, which are used in almost all areas. For examples, in nonlinear systems, we can integrate out the fast variables and perturbatively derive the effective model only with the slow variables when there is the relaxation time-scale separation among the variables. However, in general, it is a non-trivial problem to find and separate the fast process and the slow process of the system, and we have to perform an appropriate transformation or projection and get the frame in which the scale separation is apparent.
In this paper, we focus on utilizing reinforcement learning to find the symbolic formulation of the appropriate transformation. Recently, the symbolic physics learner is proposed by Ref.[34], in which they utilize the Monte-Carlo tree search[35] to perform the symbolic regression of the double pendulum’s dynamics, and this method overcomes the conventional methods such as the genetic algorithms[36, 37] or the deep symbolic regression[38].
Inspired by this work, we propose the framework of the application of Alpha Zero[15] to the symbolic regression of the appropriate transformation, which we call “Alpha Zero for Physics” (AZfP).
Tree representation of equations and the game of its construction
Here, we introduce three types of nodes for describing equations: function nodes, branch nodes, and variable nodes, shown in Figure 1. The function nodes catch one edge and pass another edge toward the leaf. The function nodes can describe, for examples, , , , , and the like. The branch nodes catch one edge and pass two edges toward the leaf. The branch nodes operate such as , , , and . The variable nodes are the leaf nodes, which catch one edge from the root and pass no edges. The variable nodes describe the variables or operators in the physics model we consider, such as , , or . We also include constants in the variable nodes.
Figure 1. (a)An example of the tree construction and its procedures. (b) An example of the forbidden tree.
We can construct a tree graph by starting from the root edge, adding the nodes and edges, and finishing by connecting all edges to (the variable) nodes. The completed tree describes a specific equation. An example is shown in Figure 1. We can describe any equation by a tree with these three types of nodes.
Therefore, finding the equation with some desirable properties is equal to finding a series of the tree’s nodes with the desirable properties from the possible nodes or finding the best strategy from the action space. If we can define the reward, which becomes large when the equation has the desirable properties, the problem is a game to find the best strategy with the highest score. We note that how to define the score depends on the kind of physics problem.
For the well-defined game, we also pose the additional rules as follows:
We must add the nodes from the top to the bottom. (Definition of the procedure of the game.)
| node type | Function | Branch | Variable |
| Graph | Fn | Br | Var |
| Examples |
Table I. Table of the types of the nodes for describing equations in a tree graph.
The number of the variable nodes () must be equal or less than the number of the branch nodes () plus one, that is . The equation is completed when the equality is satisfied, and the game ends. (Definition of the condition for the termination of the game.)
The same branch cannot be consecutive. (Due to the symmetry of the operation. For an example, must be expressed in , not in as shown in Figure 1).
If the function nodes include a certain function and its inverse function, they cannot be consecutive. (Forbid of the redundancy of the game as same as “Sennichite” in Shogi game.)
Obeying these rules, we search for the best strategy with AZfP.
Algorithms of Alpha Zero for Physics
Here, we briefly introduce the algorithms of AZfP; further details can be found in the Supplementary Materials(SM).[39] The algorithms of AZfP are based on those of original Alpha Zero[15]. Alpha Zero is divided into the search algorithms and the architecture of the neural network. We save and update the statistics of the results of the search in the nodes of the policy tree, which are , , while searching for the best policy. We note that represents the state of the making equation, and describes the action or the next node to be added to the equation tree. is the visit count of the nodes of policy trees in the search. is the action value where is the next state after performing in the state . is the prior probability of selecting the action in the state , and is given by the neural network. By utilizing these statistics, we perform the PUCT algorithms for search, which reads,
where and are the hyperparameters which decide the agent’s personality for searching. We note that the PUCT algorithm is a kind of the MCTS algorithms. After searching and storing the statistics (we call this PUCT simulations), AZfP play the game by performing the action which is most explained in the simulation.
In the SPL, they use the MCTS without neural networks, while they include the transplantation of the high-score trees for searching deep trees efficiently, which may be the remnant of the genetic algorithms. Although the transplantation may leads high score trees quickly, it should equal to the action space change in the simulation and may ruin the MCTS(UCB) algorithm. AZfP instead utilizes the neural networks for efficient search, without causing such situations. Moreover, inspired by the SPL, the agent in AZfP holds the memory of the score of some equation trees and utilizes it to guess . If the agent does not have the memory of the score of the tree focused on, the agent evaluates the estimation value from the neural network.
We represent the state of the making tree for an equation in -dimension binary vector as shown in Figure 3, where is the number of kinds of action and is the allowed maximum turn (tree length). We input this state vector to the neural networks and get for as the output. By using this output and Eq. (1), AZfP chooses the next node to be added to the tree. The neural network is trained from the results which the agent has in the memory. We describe the flow of the procedure of AZfP in Figure III. We also show the detail algorithm in SM[39].
Figure 3. vector representation of equation state as an input for the case where the action space consists of exp, , and the maximum turn is 3.
Application of Alpha Zero for Physics to periodically-driven systems
Finally, we demonstrate that the AZfP can derive the theoretical analysis methods in periodically-driven systems, which is the high-frequency expansion to calculate the Floquet unitary transformation.
Figure 2. The picture of the five step procedure of Alpha Zero for Physics. Step1: Input the state vector to the neural network; Step2: We choose the action by Eq.(1) in the simulation of PUCT search or choose the most-simulated action in the self-play; Step3: Update the state and the equation; Step4: If the equation is completed, calculate the score of the equation; Step5: Update the PUCT statistics in the simulation or train the neural network using the self-play statistics.
In periodically-driven systems, we consider the time-dependent Hamiltonian as
where is the system Hamiltonian without the driving and is the driving term which depends on time. Under the time-dependent unitary transformation, the Schrödinger equation can be written as
where we call as the dressed Hamiltonian, which describes the dynamics of from the reference frame of .
When we focus on the periodically-driven systems, where and are satisfied, it is known that there is the periodic time-dependent unitary transformation , which make the dressed Hamiltonian time-independent, that is, . We call Floquet Hamiltonian, , whose dynamics is described by the Floquet Hamiltonian, the exact rotating frame (RF), and the micromotion operator or the kick operator. In the high-frequency regime, we can construct this exact rotating frame perturbatively with high-frequency expansion such as the Floquet-Magnus expansion[40, 41] and van-Vleck expansion[42, 43]. If we calculate up to the finite order , the dressed Hamiltonian slightly depends on time by . Fortunately, when we calculate the heating rate of the system under the high-frequency driving, it is more beneficial to calculate the high-frequency expansion up to the second or third order [44] and we call this beneficial RF an appropriate RF.
In the following, supposing that the physicists in the world do not know the high-frequency expansion methods and the appropriate RF but want to derive the appropriate time-periodic unitary transformation in which the dressed Hamiltonian slightly depends on time, we “derive” the high-frequency expansion and the appropriate RF with AZfP.
We set and as the variable nodes. Because variables and the target equation are Hermitian, we set , , as the branch nodes which keep the Hermiticity. We set as the function nodes, while it cannot include time-derivative because it cause divergence when the driving is proportional to the step function. AZfP constructs by combining these operators and variables under the boundary condition , and calculate the dressed Hamiltonian
Because we want to get the RF in which the time-dependence of the dressed Hamiltonian is small, we define the reward of the game as,
where and is the total time step.
Under this setup, AZfP selects the next adding node, completes the equation, and searches for the best symbolic construction of Hermitian which makes the time-dependence of smallest.
In order to calculate the reward, we have to introduce the concrete model and demonstrate the concrete procedure. Here, AZfP derive the symbolic representation for and it should be useful even when the system is different. Thus, we should use the easiest model to calculate the reward, although we have to check in what parameter regime and under what kind of symmetries the derived symbolic representation can be used.
For simplicity, in this letter, we analyze an interacting two-spin model under the periodic driving, which reads,
where represents the quantum spin operator at site , represents a static magnetic fields, represents an interaction between the two spins, and represents an AC magnetic fields. For the simplicity, we set , , and .
Next, we show the results by AZfP. We set the hyperparameters as , , which is roughly optimized by the genetic algorithm. We use Adam [45] as the optimization function. The detail setup for deep neural networks in AZfP is written in SM[39].
Figure 4 shows the best unitary transformation and its reward (score) which AZfP found when setting the maximum tree length . AZfP, in order, found the first-, second-, and third-order Floquet-Magnus expansion for the rotating frame. We set the iterations of calculating the score as the x-axis because the calculation of the scores is the bottleneck for the calculation time. Figure 4 also shows that AZfP can efficiently search for the appropriate rotating frame because AZfP is able to find the third-order Floquet-Magnus expansion by almost times search in case the searching space roughly consists of points to search (because and the variable nodes are the terminal nodes.). We note that other reinforcement algorithms could not find these frames efficiently as AZfP could.[46] The performance comparison with -greedy algorithm and the Actor-Critic algorithm with the proximal policy optimization[47] is written in SM[39]. We also
Figure 4. Searching dynamics by AZfP when we set . The y-axis represents the max score found in the search, and the x-axis represents the iteration number of calculated scores in the search. Because the search includes randomness, we have checked in 20 trials. The blue lines represent the performance of each trial, and the red line represents their average. AZfP found the symbolic representation of the unitary transformation corresponding to the first-, second-, and third-order Floquet-Magnus expansion in order and also found some unphysical rotating frames with good scores. We set the model parameters as follows: , , , , .
note that, when we set the , AZfP also found the unphysical frame as the best frame, whose physical units of terms are not consistent. (See the SM[39].)
Application of Alpha Zero for Physics to other physical problem.
Finally, we argue how the framework of AZfP can be applied to other general problems. One way is the construction of the symbolic representation effective model of unknown nonlinear dynamics such as the dynamics of active matters and weather. This direction is similar to the previous symbolic regression research. In these cases, the score is defined as the smallness of the difference between the target dynamics and the constructed equations. Another direction is the derivation of the effective model from the original Hamiltonian or nonlinear dynamics. Also, in these cases, the score should be the smallness of difference between the target dynamics and the constructed equations when the time-scale or space-scale is coarse-grained. (For example, we set the target dynamics as the five time-step by the original dynamics, and AZfP searches for the effective Hamiltonian or the nonlinear equation, which gives the target dynamics in one step.) We note that this direction has been studied recently in the context of quantum computation, in which reinforcement learning searches for the effective and possible computation in noisy quantum devices from the original computation[48]. We note that the time for calculating the reward often becomes the bottleneck of the calculation time of AZfP, and therefore, we should define the reward to be calculated in not-so-large time.
Conclusion and Remarks
In this letter, we have proposed the method to construct the appropriate transformation using the Alpha Zero algorithm and demonstrate the concrete process and the results for the periodic-driven system. Just by defining the reward as it becomes large when the tree of the equation has the desirable properties, AZfP can search for its symbolic representation.
We remark on some of the future developments of AZfP. In this work, we do not include the arbitrary constant as the variable nodes, assuming that the appropriate unitary transformation should be constructed only by the physical operators and their combinations and not include certain constants. When we use AZfP to derive an effective model, we should include the constant as the variable nodes and optimize constants after completing the equation tree.
In this work, we can use an -dimensional vector as an input because it is not so large. We note that we should set not-so-large because a too-long equation is difficult for humans to use and it may be useless. When becomes too large due to the large action space, we may have to use the recurrent architecture such as the recurrent neural networks[38, 49, 50] as the previous researches use or the RetNet[51]. However, we note that the recurrent architecture often makes the learning difficult[52].
Some previous research about symbolic regression in physics [50, 53] considers the physical unit and imposes its constraint on trees. The constraint on the search space usually leads to performance improvement. The application of this technique to AZfP is left for future work.
We believe our proposed method, Alpha Zero for Physics, is applicable even in other systems and problems, and this work paves a new way to utilize machine learning to develop analytical methods in physics.
Acknowledge
YM deeply appriciates Akinori Tanaka for fruitiful discussion. We use Julia[54] and its library Flux.jl[55] for all calculation, and the codes we have used are uploaded in the Github[56]. This work is supported by KAKENHI 23H04527 and RIKEN Special Postdoctoral Researcher Program. Computer simulations were partially done on the supercomputer of Tokyo University at the ISSP.
Algorithm of Alpha Zero for Physics
Algorithm 1: Searching Algorithm with PUCT and the training of the neural networks
Input: Action space , reward function ;
Parameters: parameters of player’s properties for PUCT , noise parameter in exploring , maximum turn ;
Output: optimal tree and its equivalent equation ;
for each episode do
for each self-play do
Initialize: , , ;
for each turn for self-play do
for each simulation do
while expandable and do
Expand child nodes and evaluate with the neural networks;
Choose ;
Take action , update ;
note as visited, ;
end
if the equation tree is completed then
Calculate the reward (score) and update , by the back propagation;
save the history and its values in buffer;
end
end
Selection: ;
;
end
end
for each training do
make the image and target data by sampling from the buffer;
Training: train the neural networks with the loss function defined by Eq. (12);
end
end
Architecture of the neural networks and its learning
As same as the Alpha Zero[15], the neural networks of AZfP consist of a “body” followed by both the policy and value “heads.” The body consists of the input layer and 12 residual blocks, each of which consists of the batch-normalized layer and a single relu layer whose width is 64 with the skip connection. The width of the input layer is . The policy head consists of 3 residual blocks, and the output relu layer whose width is and the softmax function. The value head consists of 3 residual blocks and the output layer whose width is one and activation function is .
After playing 100 games with the PUCT algorithm, we stock the equation trees . We create batches of images, which is input vector converted from a tree () as in Figure 3, and targets, which is of the node, from this stock by randomly choosing games and nodes (). We define the loss function using these images and targets of the batches as follows:
where , is the estimated value by the neural networks, for , is the policy output from the neural network, is the hyperparameter, and is the weight parameters of the neural networks. The first term is the mean square error for the action value, the second term is the cross entropy term for the policy, and the last term is the weight decay term. We rescale the mean square error for the action value because its norm is less than 1 in the original Alpha Zero.
Comparison with other reinforcement learning techniques
Here, we show the results by the other reinforcement learning algorithms, which are the -greedy algorithm and the Actor-Critic algorithm with the proximal policy optimization[47, 57] when setting the maximum length is 14. Figure S1 shows that the other algorithm usually cannot find the third-order Floquet-Magnus expansion while Alpha Zero for Physics can find. The behavior of -greedy algorithm should be similar to those of genetic algorithm because they explore the proximity of the state which gives the maximum reward at that time. Thus, our AZfP should show better performance than the conventional genetic algorithm for symbolic regression. It is known that the the On-Policy algorithm such as PPO often show a bad sample efficiency. In figure.S1, AC+PPO shows the worst results and it may stem from the smallness of the samples.
Figure S1. The scores via evaluation iteration by the -greedy algorithm and the actor-critic method with PPO. We have calculated 20 trials by each methods. The blue lines show each score dynamics, the red line shows the average of them, and the grey line shows the average performance of AZfP.
Search for the longer equation
Here, we show the results when we set the maximum length . When the maximum length becomes larger, the exploration space becomes larger and it becomes more difficult to find good rotating frames. In case, the unphysical rotating frame in which the physical unit is inconsistent can have better score than the Flouqet-Magnus expansion, and our AZfP can also find them.
Figure S2. Searching dynamics by AZfP when we set . The y-axis represents the max score found in the search, and the x-axis represents the iteration number of calculated scores in the search. Because the search includes randomness, we have checked in 20 trials. The blue lines represent the performance of each trial, and the red line represents their average. Because the exploration space is wider than case, the performance in the same evaluation iteration becomes worse.
References
- [1]J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” (2019), arXiv:1810.04805 [cs.CL].arxiv.org/abs/1810.04805
- [2]Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, ArXiv abs/1907.11692 (2019).arxiv.org/abs/1907.11692
- [3]T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, in Advances in Neural Information Processing Systems, Vol. 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (Curran Associates, Inc., 2020) pp. 1877–1901.
- [4]H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” (2023), arXiv:2302.13971 [cs.CL].arxiv.org/abs/2302.13971
- [5]H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Koreney, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungeta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, “Llama 2: Open foundation and fine-tuned chat models,” (2023), arXiv:2307.09288 [cs.CL].arxiv.org/abs/2307.09288
- [6]A. Krizhevsky, I. Sutskever, and G. E. Hinton, in Advances in Neural Information Processing Systems, Vol. 25, edited by F. Pereira, C. J. Burges, L. Bottou, and K. Weinberger (Curran Associates, Inc., 2012).
- [7]K. He, X. Zhang, S. Ren, and J. Sun, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) pp. 770–778.
- [8]O. Ronneberger, P. Fischer, and T. Brox, in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, edited by N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi (Springer International Publishing, Cham, 2015) pp. 234–241.
- [9]J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,” (2021), arXiv:2102.04306 [cs.CV].arxiv.org/abs/2102.04306
- [10]S. Bond-Taylor, A. Leach, Y. Long, and C. G. Willcocks, IEEE Transactions on Pattern Analysis and Machine In- telligence 44, 7327 (2022).
- [11]G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan, “Normalizing flows for probabilistic modeling and inference,” (2021), arXiv:1912.02762 [stat.ML].arxiv.org/abs/1912.02762
- [12]Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” (2021), arXiv:2011.13456 [cs.LG].arxiv.org/abs/2011.13456
- [13]D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lancot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, Nature 529, 484 (2016).
- [14]D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, Y. Chen, T. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis, Nature 550, 354 (2017).
- [15]D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lancot, L. Sifre, D. Kumaran, T. Graepel, T. Lillicrap, K. Simonyan, and D. Hassabis, Science 362, 1140 (2018), https://www.science.org/doi/pdf/10.1126/science.aar6404.
- [16]J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, T. Lillicrap, and D. Silver, Nature 588, 604 (2020).
- [17]S. Kapturowski, G. Ostrovski, W. Dabney, J. Quan, and R. Munos, in International Conference on Learning Representations (2019).
- [18]L. Wang, Phys. Rev. B 94, 195105 (2016).
- [19]J. Carrasquilla and R. G. Melko, Nature Physics 13, 431 (2017).
- [20]E. van Nieuwenburg, Y.-H. Liu, and S. Huber, Nature Physics 13, 435 (2017).
- [21]W. Hu, R. R. P. Singh, and R. T. Scalettar, Phys. Rev. E 95, 062122 (2017).
- [22]N. Yoshioka, Y. Akagi, and H. Katsura, Phys. Rev. B 97, 205110 (2018).
- [23]Y.-H. Liu and E. P. L. van Nieuwenburg, Phys. Rev. Lett. 120, 176401 (2018).
- [24]Y. Nomura, A. S. Darmawan, Y. Yamaji, and M. Imada, Phys. Rev. B 96, 205152 (2017).
- [25]G. Carleo and M. Troyer, Science 355, 602 (2017), https://www.science.org/doi/pdf/10.1126/science.aag2302.
- [26]G. Carleo, Y. Nomura, and M. Imada, Nature Communications 9, 5322 (2018).
- [27]A. Joshi, R. Peters, and T. Posske, Phys. Rev. B 108, 094410 (2023).
- [28]N. Yoshioka and R. Hamazaki, Phys. Rev. B 99, 214306 (2019).
- [29]F. Brockherde, L. Vogt, L. Li, M. E. Tuckerman, K. Burke, and K.-R. Müller, Nature Communications 8, 872 (2017).
- [30]J. Kirkpatrick, B. McMorrow, D. H. P. Turban, A. L. Gaunt, J. S. Spencer, A. G. D. G. Matthews, A. Obika, L. Thiry, M. Fortunato, D. Pfau, L. R. Castellanos, S. Petersen, A. W. R. Nelson, P. Kohli, P. Mori-Sánchez, D. Hassabis, and A. J. Cohen, Science 374, 1385 (2021), https://www.science.org/doi/pdf/10.1126/science.abj6511.
- [31]J. Guo, Z. Chen, Z. Liu, X. Li, Z. Xie, Z. Wang, and Y. Wang, Scientific Reports 12, 15326 (2022).
- [32]R. Pederson, B. Kalita, and K. Burke, Nature Reviews Physics 4, 357 (2022).
- [33]M. Tsukamoto, S. Ito, K. Ogawa, Y. Ashida, K. Sasaki, and K. Kobayashi, Scientific Reports 12 (2022), 10.1038/s41598-022-18115-w.
- [34]F. Sun, Y. Liu, J. Wang, and H. Sun, in The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 (OpenReview.net, 2023).
- [35]R. Coulom, in Computers and Games, edited by H. J. van den Herik, P. Ciancarini, and H. H. L. M. J. Donkers (Springer Berlin Heidelberg, Berlin, Heidelberg, 2007) pp. 72–83.
- [36]J. R. Koza, Statistics and Computing 4, 87 (1994).
- [37]L. Billard and E. Diday, Journal of The American Statistical Association - J AMER STATIST ASSN 98, 470 (2003).
- [38]T. Mundhenk, M. Landajuela, R. Glatt, C. P. Santiago, D. fais sol, and B. K. Petersen, in Advances in Neural Information Processing Systems, Vol. 34, edited by M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan (Curran Associates, Inc., 2021) pp. 24912–24923.
- [39]See Supplemental Material http://link.aps.org/supplemental/ for (i) Algorithms of Alpha Zero for Physics; (ii) architecture of neural networks and its learning; (iii) comparison with other approach.
- [40]T. Mori, T. Kuwahara, and K. Saito, Phys. Rev. Lett. 116, 120401 (2016).
- [41]T. Kuwahara, T. Mori, and K. Saito, Annals of Physics 367, 96 (2016).
- [42]M. Bukov, L. D’Alessio, and A. Polkovnikov, Advances in Physics 64, 139 (2015), https://doi.org/10.1080/00018732.2015.1055918.
- [43]A. Eckardt and E. Anisimovas, New Journal of Physics 17, 093039 (2015).
- [44]T. Mori, Phys. Rev. Lett. 128, 050604 (2022).
- [45]D. P. Kingma and J. Ba, in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, edited by Y. Bengio and Y. LeCun (2015).
- [46]We could not compare our results with the previous studies, such as the deep symbolic optimization[38] and SPL[34]. There are some reasons. First, their codes are not open or do not work when following their instruction in the tutorial text. Second, their codes are written for the symbolic regression of the nonlinear equation and the case in which the variable node only consists of a single variable x. Therefore, they seem unable to be used for our calculation.
- [47]J. Schulman, F. Wolksi, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” (2017), arXiv:1707.06347 [cs.LG].arxiv.org/abs/1707.06347
- [48]S. Daimon, K. Tsunekawa, R. Takeuchi, T. Sagawa, N. Yamamoto, and E. Saitoh, “Quantum circuit distillation and compression,” (2023), arXiv:2309.01911 [quant-ph].DOI
- [49]J. L. Elman, Cognitive Science 14, 179 (1990).
- [50]W. Tenachi, R. Ibata, and F. I. Diakogiannis, “Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws,” (2023), arXiv:2303.03192 [astro-ph.IM].DOI
- [51]Y. Sun, L. Dong, S. Huang, S. Ma, Y. Xia, J. Xue, J. Wang, and F. Wei, “Retentive network: A succes- sor to transformer for large language models,” (2023), arXiv:2307.08621 [cs.CL].
- [52]R. Pascanu, T. Mikolov, and Y. Bengio, in Proceedings of the 30th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 28, edited by S. Dasgupta and D. McAllester (PMLR, Atlanta, Georgia, USA, 2013) pp. 1310–1318.
- [53]S.-M. Udrescu, A. Yan, J. Feng, O. Neto, T. Wu, and M. Tegmark, in Advances in Neural Information Processing Systems, Vol. 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (Curran Associates, Inc., 2020) pp. 4860–4871.
- [54]J. Bezanson, A. Edelman, S. Karpinski, and V. B. Shah, SIAM review 59, 65 (2017).
- [55]M. Innes, Journal of Open Source Software 3, 602 (2018).
- [56]Please check https://github.com/YoshihiroMichishita/julia/tree/master/AlphaZeroForPhysics.
- [57]J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, in Proceedings of the 32nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 37, edited by F. Bach and D. Blei (PMLR, Lille, France, 2015) pp. 1889–1897.





