WorksPhysics and Astronomy
Data-driven discovery of multiscale chemical reactions governed by the law of mass action
Abstract
In this paper, we propose a data-driven method to discover multiscale chemical reactions governed by the law of mass action. First, we use a single matrix to represent the stoichiometric coefficients for both the reactants and products in a system without catalysis reactions. The negative entries in the matrix denote the stoichiometric coefficients for the reactants and the positive ones for the products. Second, we find that the conventional optimization methods usually get stuck in the local minima and could not find the true solution in learning the multiscale chemical reactions. To overcome this difficulty, we propose a partial-parameters-freezing (PPF) technique to progressively determine the network parameters by using the fact that the stoichiometric coefficients are integers. With such a technique, the dimension of the searching space is gradually reduced in the training process and the global mimina can be eventually obtained. Several numerical experiments including the classical Michaelis-Menten kinetics, the hydrogen oxidation reactions, and the simplified GRI-3.0 mechanism verify the good performance of our algorithm in learning the multiscale chemical reactions. The code is available at \url{https://github.com/JuntaoHuang/multiscale-chemical-reaction}.
Introduction
Chemical reactions are fundamental in many scientific fields including biology, material science, chemical engineering and so on. To identity the reactions from experimental data, the traditional methods are mainly based on some empirical laws and expert knowledge [10]. Recently, thanks to the rapid development of machine learning [19] and data-driven modeling [34, 22, 6, 29, 20, 30, 14] [3], it is desirable to develop a data-driven method of discovering the underlying chemical reactions from massive data automatically.
Consider a reaction system with species participating in reactions:
for . Here is the chemical symbol for the -th species, the nonnegative integers and are the stoichiometric coefficients of the -th species in the -th reaction, and and are the direct and reverse reaction rates of the -th reaction. The reaction is reversible if both and are positive. Strictly speaking, all elementary chemical reactions are reversible due to microscopic reversibility. However, in real applications, some of the rate constants are negligible, thus the corresponding reactions can be omitted and the retained ones can be considered as irreversible.
Denote by the concentration of the -th species at time for . According to the law of mass action [38], the evolution of obeys the ordinary differential equations (ODEs) [27]
for . Given the concentration time series data , our goal is to learn the stoichiometric coefficients , and reaction rates and .
In the literature there are already some works on this topic. In [5], the authors applied linear regressions to infer the chemical reactions, with the assumption that the reactions are at most the result of bimolecular collisions and the total reaction order is not greater than two. In [40], the linear regression was utilized with an L1 objective, which transforms the problem into a mixed-integer linear programming (MILP). This approach suffers from the same restrictive assumptions as in [5]. In [18], the authors presented an approach to infer the stoichiometric subspace of a chemical reaction network from steady-state concentration data profiles, which is then cast as a series of MILP. In [25], some chemically reasonable requirements were considered such as the mass conservation and the principle of detailed balance. The deep neural networks (DNNs) were applied to extract the chemical reaction rate information in [31, 32], but the weights are difficult to interpret physically. In [13], the authors adapted the sparse identification of nonlinear dynamics (SINDy) method [4, 9] to the present problem. However, the approach relies on expert knowledge, which precludes the application in a new reaction system with unknown reaction pathways. Within the framework of SINDy, other works are [24] [2, 1]. In order to improve the performance of SINDy, two additional steps including least-squares regression and stepwise regression in the identification were introduced in [2], which are based on the traditional statistical methods. In [1], SINDy was combined with the DNNs to adaptively model and control the process dynamics. An implicit-SINDy was proposed and applied to infer the Michaelis-Menten enzyme kinetics in [24]. Additionally, a statistical learning framework was proposed based on group-sparse regression which leverage prior knowledge from physical principles in [23]. For example, the mass conservation is enforced in the JAK-STAT reaction pathway for signal transduction in [23].
Our work is mainly motivated by [15], where the authors proposed a Chemical Reaction Neural Network (CRNN) by resorting to the feature of the equations in (1). The discovery of chemical reactions usually involves two steps: the identification of the reaction pathways (i.e., the stoichiometric coefficients) and the determination of the reaction rates. For complex reaction processes, one could not even identify the reaction pathways and has to infer both the stoichiometric coefficients and the rate constants from data. The work in [15] presents a neural network approach for discovering unknown reaction pathways from concentration data. The parameters in CRNN correspond to the stoichiometric coefficients and reaction rates and the network has only one hidden layer with the exponential activation functions.
Different from CRNN in [15], we use a single matrix of order to represent the stoichiometric coefficients for both the forward and reverse reactions by assuming no catalysis reactions. The negative entries in the matrix denote the stoichiometric coefficients for the reactants and the positive for the products.
On the other hand, the reaction rates often differ in a wide range of magnitudes, which causes a lot of troubles in learning the multiscale chemical reactions. To provide some insights into this difficulty, we design a nonlinear regression problem to fit a polynomial with two terms, see (2.1) in § 2. The given coefficients of the polynomial differ in several orders of magnitudes and the polynomial degree is to be determined. We find numerically that the conventional optimization algorithm usually gets stuck in the local minima and could not find the true solution. Another observation in the numerical experiment is that the learned polynomial degree of the terms with larger coefficient is close to the true solution. Inspired by this observation, we propose a partial-parameters-freezing (PPF) technique to escape from the local minima. Specifically, we perform a round operation on the learned polynomial degree which are close to integer in the optimization process if the loss function does not decrease. The revised algorithm works well for this problem. Some theoretical analysis is also provided to explain the numerical phenomenon.
We then generalize the PPF technique to learn the multiscale chemical reactions. Notice that the stoichiometric coefficients are integers. In the training process, if the loss function stops to decrease, the stoichiometric coefficients which are close to integers are rounded and then frozen afterwards. With such a treatment, the stoichiometric coefficients are gradually determined, the dimension of the searching space is reduced in the training process, and eventually the global minima can be obtained. Several numerical experiments including the classical Michaelis–Menten kinetics, the hydrogen oxidation reactions and the simplified GRI-3.0 mechanism verify that our method performs much better in learning the mutiscale chemical reactions.
This paper is organized as follows. In § 2, we investigate a multiscale nonlinear regression problem numerically and theoretically. Our algorithm for learning the multiscale chemical reactions is presented in § 3. In § 4, the performance of the algorithm is validated through several numerical examples. Finally, conclusions and the outlook of future work are presented in § 5.
Multiscale nonlinear regression problem
To provide some insights into the difficulties in learning the multiscale chemical reactions, we consider a nonlinear regression problem to fit the following function:
Here and are two given constants satisfying , and are two integers to be determined. This simple toy model captures two key features of the multiscale chemical reactions. The first feature is that the right-hand side of the chemical reaction ODEs (1) is polynomials and the stoichiometric coefficients are integers. The second one is that the multiscale chemical reactions often have reaction rates which differ in several orders of magnitudes.
Given the dataset , we define the loss function to be the mean squared error (MSE):
to estimate the parameters and . Next, conventional optimization methods can be used to obtain the estimation of and .
In the numerical experiment, we take and . The ground truth solutions are and . The data for are randomly sampled from a uniform distribution in with the number of data , and . The Adam optimization method [17] is applied with the full batch gradient decent. The learning rate is taken to be . The initial guess of and is randomly chosen in .
For this toy model, we numerically find that the naive implementation will get stuck in the local minima and could not find the true solution. The history of the loss function and the parameters and in the training is presented in Figure 2.1, see the dashed lines.

Figure 2.1. Multiscale nonlinear regression problem: the history of loss function in (3) and the parameters and in the training process. Solid lines: the method with the PPF technique; dashed lines: the method without the PPF technique.
Although the naive optimization could not find the global minima, we notice that in this local minima is close to the true solution . Inspired by this observation, we propose a partial-parameters-freezing (PPF) technique to escape from the local minima. To be more specific, we keep track of the loss function in the training. If the loss does not decrease, we check the parameters and : if any of these is close to its nearest integer with a given threshold, we round it to the integer and do not update it in the afterwards optimization process.
For comparison, we also plot the history of the loss function and the parameters with the PPF technique in Figure 2.1, see the solid lines. The threshold is taken to be 0.05 in this test. The loss stops decreasing with the epoch around 7000. Then is rounded to 2 and only is updated afterwards. The true solution is eventually obtained when the epoch is around 10000.
To better understand why it is easy to get stuck in the local minima without the PPF treatment, we investigate the landscape of the loss function. In Figure 2.2, we plot the 3D surface and the contour map for the loss as a function of . In Figure 2.2a, it is observed that the loss function has several local minima in which is close to 2. Moreover, the local minima in the naive implementation is also labeled in Figure 2.2 (b).

(a) loss function surface plot

(b) loss function contour map
Figure 2.2. Multiscale nonlinear regression problem: the landscape of the loss function in (3). Left: loss function surface plot (in log scale); right: loss function contour map (in log scale), local minima .
We also plot the profiles of the loss function with fixed , 2 and 2.01 in Figure 2.3. It is observed that slight perturbations in have a considerable impact on the minima of the loss function. Moreover, the loss as a 1D function with fixed is well-behaved. This explains why our algorithm is easy to find the global minima after freezing the integer parameter .

Figure 2.3. Multiscale nonlinear regression problem: loss function in (3) with fixed parameters , and .
We mention that we also test other cases with different coefficients and satisfying and different integers and . The results are similar and thus omitted here.
We conclude this section with some theoretical analysis to explain the local minima phenomenon observed above. By taking gradient of the loss function in (3), we have
Here denotes the true solution of the parameter for . From the expression (4), we can provide some insights on the phenomenon that the local minima is close to the true solution . To reach the local minima, the gradient should be zero. Refer to the expression (4). Since , whether or not the gradient is close to zero depends mainly on the fact that , instead of , is close to the ground truth.
Algorithm
In this section, we present our algorithm for learning the multiscale chemical reactions. First, we use a single matrix to represent the stoichiometric coefficients for both the reactants and products. Each row of the matrix represents one reaction, where the negative entries denote the stoichiometric coefficients for the reactants and the positive ones for the products. This setup is valid for systems without catalysis reactions. In addition, we adapt the PPF technique for the multiscale nonlinear regression problem proposed in § 2 to learn the multiscale chemical reactions (1).
We assume that the data are given in the form of the concentrations and the time derivatives in different time snapshots , our goal is to learn the stoichiometric coefficients and the reaction rates. Realistically, often only is available, and the time derivatives could be approximated using numerical differentiations [33, 7].
To better illustrate the algorithm, we firstly introduce some vector notations. We denote the forward and reverse reaction rates in (1) by and . The stoichiometric coefficients in (1) are collected in two matrices:
Assume that there is no catalysis reactions. Therefore, only one of and can be non-zero for any . In this case, the matrix satisfies
According to this property, we only need to pin down the matrix . Then and can be recovered by and , respectively.
Next, we define the neural network which has the input and the parameters , and :
for . Here the parameters and denote the logarithms of the reaction rates and [15]. This change of variables technique has two advantages. The first one is that the positivity of the reaction rates is guaranteed automatically. The second one is that the reaction rates for the multiscale chemical reactions usually differ in several orders of magnitudes. The slight changes of and will make and change a lot, which could potentially make the neural network to be more robust in the training process.
The loss function is defined as the mean squared error (MSE) between the data for the time derivatives and the output of the neural network:
Here is a regularization term with the regularization constant, and
Here both and regularization terms are included.
This neural network works quite well for non-stiff chemical reactions. However, for stiff reactions, we observe that the optimization usually gets stuck in the local minima in the training process and could not find the true solution. The common techniques such as the mini-batch and reducing the learning rate do not work in such a situation. To attack this problem, we adapt the PPF technique proposed in the previous section.
The training procedure is split into two parts. The first part is to learn the matrix . To better illustrate the algorithm, we introduce some notation. Denote the vector in the -th row of by for . Define the distance to the nearest integer for any vector as
where denotes the function rounding an arbitrary real number to its nearest integer and it is defined to work element-wise on vectors. We keep track of the loss function in the training process. If the loss function stops decreasing, we check if any row of is close to the nearest integers, i.e., . Here, is a hyperparameter and we take in all the numerical examples in § 4. If the -th row of satisfies the condition , then we round to and do not update it in the afterwards training. In addition, to help the optimization algorithm escape from the local minima, we randomly reinitialize other non-integer entries in when the loss stops decreasing. After all the entries in reach integer, we freeze them and then learn the parameters and related to the reaction rates. We remark that the SINDy algorithms [4, 13] can also be applied in learning the reaction rates when the stoichiometric coefficients are known. The algorithm is summarized in Algorithm 1.
Remark 3.1. Here we assume that all the reactions are reversible. However, the algorithm can be also applied to irreversible reactions without any modification. The expected result is that the learned reverse reaction rates for the irreversible reactions will be close to zero. This will be demonstrated numerically in Example 4.2 in § 4.
Remark 3.2. The number of reactions can be learned by repeatedly executing the algorithm with different . The ground truth of can be inferred from the best one. This will be shown in the numerical examples in the next section.
Remark 3.3. In many chemical reaction systems, the rate constants usually depend on the temperature. For example, the Arrhenius law can describe such a dependence:
where is the reaction rate, is the pre-exponential factor, is the activation energy and is the gas constant. In this case, the unknown parameters will include the pre-exponential factor, the activation energy and the stoichiometric coefficients. Our PPF technique can be directly applied without much modification. The performance will be verified numerically in the test in the next section.
Algorithm 1: Algorithm for learning chemical reactions
Input: time series data
Output: stoichiometric coefficient matrix , chemical reaction rates and
initialize hyperparameters: number of reactions , total number of epoch , learning rate , regularization coefficient , integer threshold ;
initialize parameters: , and ; // step 1: learning
; // record loss function in each epoch
;
for do
Compute loss ;
Compute by backpropagation;
Update parameters (excluding the integer entries in ) by Adam method; // if loss increase, then check if any row of is close to integer
if then
for do
if then
;
;
end
if then
; // random reinitialize non-integer entries in
end
end
end // if all the entries in are integers, then stop learning
if then
break;
end
end // step 2: learning and
for do
Compute loss ;
Compute by backpropagation;
Update parameters (excluding ) by Adam method;
end
for do
;
;
end
Numerical results
Here the performance of our algorithm will be shown with five examples. The first example is an artificial reaction mechanism with two reactions [21]. The second one is the well-known Michaelis-Menten kinetics [16] in biochemistry. The third one is the hydrogen oxidation reactions [11, 8]. The fourth one is the extended Zeldovich mechanism, a typical chemical mechanism describing the oxidation of nitrogen and NOx formation [41]. The last one is the simplified GRI-3.0 mechanism, a chemical mechanism describing the methane oxidation [37].
In each numerical example, we randomly take 100 different initial conditions to generate the data. For each initial condition, we take uniform time snapshots at with and . The data is generated by solving the governing ODEs numerically using implicit Runge-Kutta method of the Radau IIA family of the fifth order [39] with small enough tolerance. The datasets are randomly split into the training datasets and the validation datasets by a ratio of 4:1. It is worthy to note that here we do not take to be too small so that the datasets could be potentially replaced by the experiment data in the future. The algorithm is implemented with PyTorch [28].
Now we present some details of the training and validation for the following four numerical tests. In the training process, all the parameters in the neural network are first randomly initialized from the uniform distribution in the interval . Then, we update the parameters by minimizing the loss in (6) using the standard Adams algorithm [17]. The learning rate is taken to be and the regularization coefficient in (6) is . Recall the training method following (8), we take the integer threshold to be 0.05. Besides, the total epoch number is and the mini-batch gradient descent is applied with the batch size 10. For the validation, we use the following relative error:
Here the 's come from the validation dataset. For the other details, we refer the interested readers to our code in https://github.com/JuntaoHuang/multiscale-chemical-reaction.
Example 4.1 (hypothetical stiff reaction network). The first test case is an artificial reaction network with two reactions, taken from [21]:
Here , and indicate the fuel, radical and product in combustions, respectively. The reaction rates are taken to be and . The two reversible reactions in (4.1) have dramatically different reaction rates. Thus, the second reaction (4.1b) will quickly approach to equilibrium after a transient period, after which the first one (4.1a) becomes rate-limiting. This simple model is chosen to test the correctness of our code for stiff reactions.
The corresponding ODE system for (4.1) is linear. The eigenvalues of the coefficient matrix are , and , which differ in several orders of magnitudes. This indicates that the ODE system is stiff [39].
To illustrate the advantage of the PPF technique, we compare the performance of the algorithm with and without this technique. The history of the training and validation errors is shown in Figure 4.4. The relative error stays around without this technique, and decreases to after applying this technique. The learned parameters are listed in Table 4.1. The upper part of the table is the learned parameters with the PPF technique, which agrees well with the ground truth in (4.1). By contrast, the algorithm without imposing this technique could not generate the correct result. Moreover, it is interesting to see that, without using the technique, the learned stoichiometric coefficients in the first reaction and the second one has the opposite sign. We also notice that the summation of the forward rate of the first reaction and the reverse rate of the second one is close to the true reaction rate . The same holds true for the reverse rate of the first reaction and the forward rate of the second one. This indicates that the effect of these two learned reactions is identical to the fast reaction (11) and the slow reaction (10) is not captured here. This is similar to the phenomenon we observed in the multiscale nonlinear regression problem in § 2.

Figure 4.4. Example 4.1: the history of the relative error for the training data and the verification data. Solid line: the method with the PPF technique; dashed line: the method without the PPF technique.
| freezing | |||||
| 1 | 0.000 | 1.000 | −1.000 | 1.000e+03 | 1.000e+03 |
| 2 | −1.000 | 1.000 | 0.000 | 1.000e+00 | 1.000e+00 |
| no freezing | |||||
| 1 | −0.001 | 0.999 | −0.999 | 7.448e+02 | 5.731e+02 |
| 2 | 0.000 | −0.999 | 0.999 | 4.277e+02 | 2.559e+02 |
Table 4.1. Example 4.1: learned parameters. Upper part: with the PPF technique; lower part: without the PPF technique. Here denotes the row vector of the matrix .
Next, we test the algorithm with different number of chemical reactions. We take the number of reactions ranging from 1 to 4. The relative errors in the training data and the validation data are shown in Figure 4.5. The relative error decreases by three magnitudes when increasing the number of proposed reactions from one to two and reaches a plateau after that. Moreover, it is observed from Table 4.2 that some of the learned stoichiometric coefficients or reaction rates are close to zero if the number of reactions are larger than two. It then can be inferred that the kinetics could be well described with two reactions.
| reaction num 1 | |||||
| 1 | 0.000 | −1.000 | 1.000 | 1.000e+03 | 1.000e+03 |
| reaction num 2 | |||||
| 1 | 0.000 | 1.000 | −1.000 | 1.000e+03 | 1.000e+03 |
| 2 | −1.000 | 1.000 | 0.000 | 1.000e+00 | 1.000e+00 |
| reaction num 3 | |||||
| 1 | 0.000 | 1.000 | −1.000 | 1.000e+03 | 1.000e+03 |
| 2 | −1.000 | 1.000 | 0.000 | 9.217e+02 | 1.218e+02 |
| 3 | 0.750 | −0.101 | 0.384 | 7.887e−04 | 1.813e−03 |
| reaction num 4 | |||||
| 1 | 0.000 | −1.000 | 1.000 | 1.000e+03 | 1.000e+03 |
| 2 | 0.000 | 0.000 | 0.000 | 5.931e+01 | 5.929e+01 |
| 3 | 0.000 | 0.000 | 0.000 | 2.526e+01 | 2.524e+01 |
| 4 | 1.000 | −1.000 | 0.000 | 1.000e+00 | 1.000e+00 |
Table 4.2. Example 4.1: learned parameters with different number of reactions. Here denotes the row vector of the matrix .

Figure 4.5. Example 4.1: relative error for the training data and the validation data with different number of reactions.
Example 4.2 (enzyme kinetics). In this example, we consider the Michaelis–Menten kinetics [16], one of the best-known models of enzyme kinetics in biochemistry. It involves an enzyme E, binding to a substrate S, to form a complex ES, which in turn releases a product P, regenerating the original enzyme. This can be represented schematically as [16]
Here denotes the forward rate constant, the reverse rate constant, and the catalytic rate constant. This model is used in a variety of biochemical situations other than enzyme-substrate interaction, including antigen–antibody binding, DNA-DNA hybridization, and protein–protein interaction [26]. Moreover, the reaction rates vary widely between different enzymes. In our test case, we follow [36] and take , and .
Note that the second reaction in (12) is not reversible. Here, we show that the exactly same algorithm can be applied to this situation. The results with and without the PPF technique are listed in Table 4.3. In the upper part of the table, the reverse rate for the second reaction is . It then can be inferred that the system could be well described using two reactions with the second one to be irreversible. Again, the algorithm without this treatment could only get the correct result for the first faster reaction in (12). The evolution of the loss function is similar to that in Example 4.1 and thus omitted here.
| freezing | ||||||
| 1 | −1.000 | −1.000 | 1.000 | 0.000 | 1.000e+06 | 1.000e+03 |
| 2 | 1.000 | 0.000 | −1.000 | 1.000 | 1.000e+01 | 1.949e-04 |
| no freezing | ||||||
| 1 | −0.999 | −0.999 | 0.989 | 0.000 | 9.921e+05 | 9.929e+02 |
| 2 | −1.001 | −1.000 | 2.385 | 0.000 | 7.956e+03 | 1.392e+01 |
Table 4.3. Example 4.2: learned parameters. Upper part: with the PPF technique; lower part: without the PPF technique. Here denotes the row vector of the matrix .
Next, we test the performance of the algorithm when the reaction rates depend on temperature. We assume that the rate constants in (12) satisfy the Arrhenius law:
where the pre-exponential factors are given by
and the activation energy are
and the gas constant . The temperature is randomly taken in a uniform distribution in the interval . In this case, the unknown parameters will include the pre-exponential factor, the activation energy in the Arrhenius law, and the stoichiometric coefficients. Our PPF technique can be directly applied without much modification.
We compare the performance of the algorithm with and without the PPF technique. The history of the relative error for the training data and the verification data with variable temperature is shown in Figure 4.6. We see clearly that the errors with the PPF technique are much smaller than those without the technique. We also show the learned parameters in Table 4.4. The upper part of the table is the learned parameters with the PPF technique, which agrees well with the ground truth. By contrast, the algorithm without imposing this technique could not generate the correct result.

Figure 4.6. Example 4.2: the history of the relative error for the training data and the verification data with variable temperature. Solid line: the method with the PPF technique; dashed line: the method without the PPF technique.
| freezing | ||||||||
| 1 | 1.000 | 0.000 | −1.000 | 1.000 | 1.000e+03 | 1.801e−05 | 2.240e+03 | 5.203e+03 |
| 2 | −1.000 | −1.000 | 1.000 | 0.000 | 1.000e+00 | 4.000e+00 | 1.600e+03 | 3.680e+03 |
| no freezing | ||||||||
| 1 | 1.061 | 0.001 | −1.000 | 2.887 | 3.391e+02 | 1.665e−06 | 2.240e+03 | 2.244e+03 |
| 2 | 0.969 | 0.002 | −1.000 | 0.032 | 6.640e+02 | 2.689e−01 | 6.727e+01 | 6.684e+02 |
Table 4.4. Example 4.2: learned parameters with variable temperatures. Upper part: with the PPF technique; lower part: without the PPF technique. Here denotes the row vector of the matrix .
Example 4.3 (hydrogen oxidation reaction). In this example, we consider a model for hydrogen oxidation reaction where six species H (hydrogen), O (oxygen), HO (water), H, O, OH (radicals) are involved in six steps in a closed system under constant volume and temperature
with the reaction rates , , , , , , , , , . The system (4.6) corresponds to the simplified picture of this chemical process and the reaction rates reflect only orders of magnitude for relevant real-word systems. The magnitude of the reaction rates vary from to , which leads to the multiscale phenomena. This reaction network has much more reactions and is more realistic than the first two test cases.
We first compare the performance of our algorithm with and without the PPF treatment. The history of the training and the validation error is shown in Figure 4.7. Again, we observe that this technique greatly reduces the training and validation errors. The learned parameters are listed in Table 4.5. The algorithm can generate the correct result with this technique. On the other hand, without using this technique, the phenomenon of the opposite signs observed in Table 4.1 also appears.

Figure 4.7. Example 4.3: the history of the relative error for the training data and the verification data. Solid line: the method with the PPF technique; dashed line: the method without the PPF technique.
| freezing | ||||||||
| 1 | 0.000 | 1.000 | 0.000 | 1.000 | 3.375e+04 | 1.000e+03 | ||
| 2 | 1.000 | 0.000 | 0.000 | 1.000 | 1.080e+04 | 1.000e+03 | ||
| 3 | 0.000 | 0.000 | 1.000 | 0.000 | 1.400e+03 | 1.000e+00 | ||
| 4 | 0.000 | 1.000 | 0.000 | 0.000 | 0.000 | 3.375e+02 | 1.000e+00 | |
| 5 | 1.000 | 0.000 | 0.000 | 0.000 | 0.000 | 2.160e+02 | 2.000e+00 | |
| 6 | 0.000 | 1.000 | 0.000 | 0.000 | 1.000e+02 | 7.714e-01 | ||
| no freezing | ||||||||
| 1 | 0.000 | 0.938 | 0.000 | 0.923 | 1.971e+04 | 5.512e+02 | ||
| 2 | 0.000 | 1.087 | 0.000 | 1.108 | 1.407e+04 | 4.488e+02 | ||
| 3 | 0.885 | 0.000 | 0.115 | 0.885 | 1.217e+04 | 2.112e+01 | ||
| 4 | 0.000 | 0.087 | 0.910 | 0.913 | 1.071e+03 | 2.926e+01 | ||
| 5 | 0.000 | 0.996 | 0.008 | 0.000 | 0.000 | 3.379e+02 | 8.912e-01 | |
| 6 | 0.987 | 0.000 | 0.007 | 0.004 | 0.004 | 2.170e+02 | 3.377e+00 |
Table 4.5. Example 4.3: learned parameters. Upper part: with the PPF technique; lower part: without the PPF technique. Here denotes the row vector of the matrix .
We also test the performance of the algorithm with Gaussian noise. The algorithm can get the correct prediction of the stoichiometric coefficients with the noise level and . The learned reaction rates with noise are shown in Table 4.6. The relative errors for reaction rates are typically less than the order of for noise and for noise.
| noise | relative error | relative error | ||
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 | ||||
| 5 | ||||
| 6 | ||||
| noise | relative error | relative error | ||
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 | ||||
| 5 | ||||
| 6 |
Table 4.6. Example 4.3: learned reaction rates with noise.
Moreover, we plot the evolution of the concentrations of the six species with the noise level
in Figure 4.8. We observe a good agreement of the solution generated by our learned model and the exact solution. We also measure the prediction errors of the learned model at 100 uniformly points in the time interval . The prediction errors are with zero noise, with noise level and with noise level .

Figure 4.8. Example 4.3: the evolution of the concentration of the 6 species in the hydrogen oxidation reaction problem obtained by solving the original ODEs (4.6) and our learned ODEs. noise level .
Example 4.4 (extended Zeldovich mechanism). In this example, we test our algorithm on the extended Zeldovich mechanism, which is a chemical mechanism describing the oxidation of nitrogen and NOx formation [41]. Similar to Example 4.3, this is another realistic test case. The reaction mechanisms read as
and the reaction rates are given by the Arrhenius law [12]:
with the temperature.
In the numerical test, we fix the temperature to be , which is a reasonable temperature in real applications [12]. At this temperature, the reaction rates are
Then, we follow the same procedure in the previous examples to generate the data and execute the algorithm to discover the stoichiometric coefficients and the reaction rates. Again, the algorithm with the PPF treatment can predict the correct result, which is shown in Table 4.7. We observe that the accurate reaction rates are obtained.
| freezing | |||||||||
| 1 | 0.000 | 0.000 | 1.000 | 0.000 | 1.000 | 6.111e+10 | 4.732e+07 | ||
| 2 | 1.000 | 1.000 | 0.000 | 0.000 | 0.000 | 3.298e+10 | 5.019e+05 | ||
| 3 | 0.000 | 1.000 | 1.000 | 0.000 | 0.000 | 3.782e+06 | 3.931e+03 |
Table 4.7. Example 4.4: learned parameters with the PPF technique. Here denotes the row vector of the matrix .
Example 4.5 (simplified GRI-3.0 mechanism). In this example, we test our algorithm on the simplified GRI-3.0 mechanism, which is a chemical mechanism describing the methane oxida-
tion [37]. This is the most complicated reaction system tested in the paper. The mechanism includes 16 species with 12 reactions and reads as
The reaction rates are given in [37], which are derived from the reaction rates of the standard GRI-3.0 Mech [35]. We compute the reaction rates with the temperature and list them in Table 4.8. Here, the reaction rates are normalized such that the smallest one is of order 1.
| 5.088e+00 | 1.891e+00 | 2.607e+00 | 6.268e+00 | 5.446e+00 | 1.283e+01 |
| 1.349e+00 | 5.264e+03 | 3.268e+01 | 4.873e+03 | 2.978e+02 | 5.227e+03 |
Table 4.8. Example 4.5: reaction rates in simplified GRI-3.0 Mech
Note that all the reactions in (4.10) are not reversible. Here, we apply exactly the same algorithm to this situation, similar to Example 4.2. To illustrate the advantage of the PPF technique, we first compare the performance of the algorithm with and without this technique.
The history of the training and validation errors is shown in Figure 4.9. The relative error stays around without this technique, and decreases to after applying this technique.

Figure 4.9. Example 4.5: the history of the relative error for the training data and the verification data. Solid line: the method with the PPF technique; dashed line: the method without the PPF technique.
We also list the learned parameters with the PPF technique in Table 4.9. Here, the learned stoichiometric coefficients are the same with the true coefficients in (4.10) and they are omitted here. The upper part of the table is the learned rates in the forward reactions with the PPF technique, which agrees well with the ground truth in Table 4.8. The learned rates in the reverse reactions are in the magnitude of to . It then can be inferred that the system can be well described using only forward reactions. By contrast, the algorithm
caption: Example 4.5: learned reaction rates in simplified GRI-3.0 Mech. Upper part: reaction rates in the forward reaction; lower part: reaction rates in the reverse reaction.
:::
without imposing this technique could not generate the correct result and we omit the results here.
Conclusion
In this paper, we propose a data-driven method to discover multiscale chemical reactions governed by the law of mass action. The method mainly contains two novel points. First, we use a single matrix to represent the stoichiometric coefficients for both the reactants and products in a system without catalysis reactions. The negative entries in the matrix denote the stoichiometric coefficients for the reactants and the positive ones for the products. Second, by considering a multiscale nonlinear regression problem, we find that the conventional optimization methods usually get stuck in the local minima and could not find the true solution. To escape from the local minima, we propose a PPF technique. Notice that the stoichiometric coefficients are integers. In the training process, if the loss function stops to decrease, the stoichiometric coefficients which are close to integers are rounded and then frozen afterwards. With such a treatment, the stoichiometric coefficients are gradually determined, the dimension of the searching space is reduced in the training process, and eventually the global mimina can be obtained. Several numerical experiments including the classical Michaelis–Menten kinetics, the hydrogen oxidation reactions and simplified GRI-3.0 mechanism verify the validity of our algorithm in learning the multiscale chemical reactions.
There are still some problems to be addressed in order to develop a robust and general framework for discovering multiscale chemical reactions from data. We shall highlight some of the challenges that could guide future advances. First, it is interesting to generalize the PPF technique to the catalysis reactions. Second, the number of species cannot be determined from our algorithm. In principle, to infer the unknown chemical reaction systems, we should have the concentration time series data for all the species. Our algorithm cannot treat the problem when the concentrations for some partial species are unknown. This difficulty may be overcomed by combining the current algorithm with the Neural ODE approach in [15]. The third challenge is that for very complex reaction networks with large number of reactions (hundreds or thousands), our algorithm may not always find out the correct solution. New ideas are needed at this point.
References
- [4]Academy of Sciences, 113(15):3932–3937, 2016.
- [5]S. C. Burnham, D. P. Searson, M. J. Willis, and A. R. Wright. Inference of chemical reaction networks. Chemical Engineering Science, 63(4):862–873, 2008.DOI
- [6]K. Champion, B. Lusch, J. N. Kutz, and S. L. Brunton. Data-driven discovery of co-ordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45):22445–22451, 2019.
- [7]R. Chartrand. Numerical differentiation of noisy, nonsmooth data. ISRN Applied Mathematics, 2011, 2011.DOI
- [8]E. Chiavazzo and I. V. Karlin. Quasi-equilibrium grid algorithm: Geometric construction for model reduction. Journal of Computational Physics, 227(11):5535–5560, 2008.DOI
- [9]B. de Silva, K. Champion, M. Quade, J.-C. Loiseau, J. N. Kutz, and S. Brunton. PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data. Journal of Open Source Software, 5(49):1–4, 2020.DOI
- [10]C. W. Gao, J. W. Allen, W. H. Green, and R. H. West. Reaction mechanism generator: Automatic construction of chemical kinetic mechanisms. Computer Physics Communications, 203:212–225, 2016.DOI
- [11]A. N. Gorban and I. V. Karlin. Invariant manifolds for physical and chemical kinetics, volume 660. Springer Science & Business Media, 2005.
- [12]R. K. Hanson and S. Salimian. Survey of rate constants in the n/h/o system. In Combustion chemistry, pages 361–421. Springer, 1984.DOI
- [13]M. Hoffmann, C. Fröhner, and F. Noé. Reactive SINDy: Discovering governing reactions from concentration data. The Journal of Chemical Physics, 150(2):025101, 2019.DOI
- [14]J. Huang, Z. Ma, Y. Zhou, and W.-A. Yong. Learning thermodynamically stable and galilean invariant partial differential equations for non-equilibrium flows. Journal of Non-Equilibrium Thermodynamics, 2021.DOI
- [15]W. Ji and S. Deng. Autonomous discovery of unknown reaction pathways from data by chemical reaction neural network. The Journal of Physical Chemistry A, 125(4):1082–1092, 2021.DOI
- [16]J. P. Keener and J. Sneyd. Mathematical Physiology, volume 1. Springer, 1998.
- [17]D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.arxiv.org/abs/1412.6980
- [18]D. Langary and Z. Nikoloski. Inference of chemical reaction networks based on concentration profiles using an optimization framework. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(11):113121, 2019.DOI
- [19]Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature, 521(7553):436–444, 2015.DOI
- [20]L. Lu, X. Meng, Z. Mao, and G. E. Karniadakis. DeepXDE: A deep learning library for solving differential equations. arXiv preprint arXiv:1907.04502, 2019.DOI
- [21]T. Lu and C. K. Law. On the applicability of directed relation graphs to the reduction of reaction mechanisms. Combustion and Flame, 146(3):472–483, 2006.DOI
- [22]B. Lusch, J. N. Kutz, and S. L. Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications, 9(1):1–10, 2018.DOI
- [23]S. Maddu, B. L. Cheeseman, C. L. Müller, and I. F. Sbalzarini. Learning physically consistent mathematical models from data using group sparsity. arXiv preprint arXiv:2012.06391, 2020.DOI
- [24]N. M. Mangan, S. L. Brunton, J. L. Proctor, and J. N. Kutz. Inferring biological networks by sparse identification of nonlinear dynamics. IEEE Transactions on Molecular, Biological and Multi-Scale Communications, 2(1):52–63, 2016.
- [25]T. Nagy, J. Tóth, and T. Ladics. Automatic kinetic model generation and selection based on concentration versus time curves. International Journal of Chemical Kinetics, 52(2):109–123, 2020.DOI
- [26]D. L. Nelson, A. L. Lehninger, and M. M. Cox. Lehninger principles of biochemistry. Macmillan, 2008.
- [27]H. G. Othmer. Analysis of complex reaction networks. Lecture Notes, School of Mathematics, University of Minnesota, 2003.
- [28]A. Paszke, S. Gross, F. Massa, A. Lehrer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32:8026–8037, 2019.arxiv.org/abs/1912.01703
- [29]M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.DOI
- [30]M. Raissi, A. Yazdani, and G. E. Karniadakis. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science, 367(6481):1026–1030, 2020.DOI
- [31]R. Ranade, S. Alqahtani, A. Farooq, and T. Echekki. An ANN based hybrid chemistry framework for complex fuels. Fuel, 241:625–636, 2019.DOI
- [32]R. Ranade, S. Alqahtani, A. Farooq, and T. Echekki. An extended hybrid chemistry framework for complex hydrocarbon fuels. Fuel, 251:276–284, 2019.DOI
- [33]L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1-4):259–268, 1992.DOI
- [34]S. H. Rudy, S. L. Brunton, J. L. Proctor, and J. N. Kutz. Data-driven discovery of partial differential equations. Science Advances, 3(4):e1602614, 2017.
- [35]G. P. Smith, D. M. Golden, M. Frenklach, N. W. Moriarty, B. Eiteneer, M. Goldenberg, C. T. Bowman, R. K. Hanson, S. Song, W. C. J. Gardiner, V. V. Lissianski, and Z. Qin. Gri-mech home page, http://www.me.berkeley.edu/gri_mech/.
- [36]V. Srinivasan and R. Aiken. Stage-wise parameter estimation for stiff differential equations. AIChE journal, 32(2):195–199, 1986.DOI
- [37]C. Sung, C. Law, and J.-Y. Chen. Augmented reduced mechanisms for no emission in methane oxidation. Combustion and Flame, 125(1-2):906–919, 2001.DOI
- [38]E. O. Voit, H. A. Martens, and S. W. Omholt. 150 years of the mass action law. PLoS Comput Biol, 11(1):e1004012, 2015.DOI
- [39]G. Wanner and E. Hairer. Solving Ordinary Differential Equations II. Springer Berlin Heidelberg, 1996.
- [40]M. J. Willis and M. von Stosch. Inference of chemical reaction networks using mixed integer linear programming. Computers & Chemical Engineering, 90:31–43, 2016.DOI
- [41]I. Zeldovich, G. I. Barenblatt, V. Librovich, and G. Makhviladze. Mathematical theory of combustion and explosions. 1985.DOI