WorksPhysics and Astronomy
Numerical Integration Instabilities in Likelihood Fits: Mechanisms and Remedies
Abstract
Numerical instabilities in the normalization of probability density functions can produce artificially lower negative log-likelihood (NLL) values than the true minimum of a likelihood fit, even when the resulting fit provides a poorer description of the data. Motivated by anomalously low NLL values found in non-converged fits associated with a recent CMS search for fully charmed tetraquark candidates, we use a pseudo-dataset generated from a likelihood model based on that analysis to investigate how numerical integration can yield unreliable normalization results, how these errors affect likelihood minimization, and how they can be addressed. We find that, even at fixed integration tolerance, the stopping decision of a numerical integrator can change with the fit parameters, introducing discontinuities in the normalization integrals that propagate to the likelihood surface. Such discontinuities can trap the minimizer at a numerical step and yield an artificially lower NLL. Tightening the integration tolerance mitigates these numerical discontinuities, while adaptive integration methods, such as Gauss–Kronrod quadrature, can also improve the numerical stability. A more severe failure occurs when the integration grid cannot adequately resolve a sharply localized contribution. In this case, the integrator may satisfy its estimated-error criterion despite substantially underestimating the normalization, allowing the minimizer to enter a region with an artificially low NLL. Tightening integration tolerances or using alternative adaptive integration methods does not necessarily resolve sharply localized contributions. We demonstrate that a suitable variable transformation can improve their numerical resolution, yielding more accurate normalization integrals and restoring fit convergence.
Introduction
Maximum-likelihood fitting is widely used for parameter estimation in many fields. In many applications, the normalization integrals of probability density functions (PDFs) depend on the fit parameters and must be evaluated numerically to calculate the likelihood. Numerical inaccuracies in these integrals can distort the evaluated likelihood, potentially affecting parameter estimation and fit convergence.
Likelihood fitting is extensively used in experimental particle physics, and we draw on the case of fitting structures in the mass spectrum reported by three LHC experiments [1, 2, 3, 4, 5]. In particular, CMS has reported a family of fully charmed tetraquark candidates whose masses exhibit a Regge pattern, with possible connections to structures observed in the spectrum [3, 4, 5, 6, 7]. Extracting resonance properties from these spectra involves complex likelihood fits with multiple contributions, as exemplified by the CMS analysis [3, 4, 5], which includes three interfering resonances and yields multiple fit solutions [8]. The fits were performed using RooFit [9], with RooIntegrator1D for numerical normalization and MIGRAD in Minuit [10] for minimization. In maximum-likelihood fits, a lower negative log-likelihood (NLL) ordinarily indicates a better description of the data within the assumed model. However, during the development of this analysis, some fits returned NLL values substantially lower than those obtained from normally converged fits, despite failing standard convergence criteria. Likelihood scans of these supposed minima exhibited discontinuities. This behavior was traced back to problems with numerical integrations performed in the minimization algorithm and motivated a broader investigation into the conditions under which numerical integration can produce inaccurate normalization results, how the resulting errors affect likelihood minimization, and how such failures can be identified and corrected.
In this work, we investigate this problem using a pseudo-dataset based on the CMS analysis [3, 4]. Other technical challenges, such as the estimation of large global significances, have been discussed elsewhere [11] and are beyond the scope of this study. We examine how the criteria used to terminate numerical integration affect the accuracy and continuity of the normalization integrals and the resulting likelihood, and show that tightening the integration tolerance or using adaptive integration methods can mitigate the numerical discontinuities. We further investigate a more severe failure in which a sharply localized contribution is not adequately resolved by the integration grid, causing the normalization to be substantially underestimated despite satisfying the estimated-error criterion. By examining the minimization process, we trace how the resulting likelihood distortion leads the minimizer into an anomalous parameter region. Finally, we compare different numerical integration approaches and demonstrate that a suitable variable transformation can improve the normalization accuracy and restore fit convergence.
Model and Fit Setup
The numerical study considers the mass spectrum, , in the range GeV. Following the CMS analysis [12], the model consists of four components: nonresonant single-parton scattering (NRSPS), double-parton scattering (DPS), a noninterfering threshold component BW, and a coherent contribution from three resonances, , , and . The total event density is
where is the yield of component , and each is a PDF normalized over GeV. All four component densities vanish for , where is the production threshold [13]. The DPS shape is described by a fixed parametrization derived from simulated events [3, 4]. The threshold component BW is modeled by a relativistic -wave Breit–Wigner with floating mass and width and does not interfere with the other components. The three signal resonances are described by relativistic -wave Breit–Wigner amplitudes, combined coherently following the CMS three-resonance interference model [3, 4]:
where BW, BW, and BW correspond to , , and , respectively. The amplitude is taken as the reference, with and . The parameters and are dimensionless relative magnitudes, while and are relative phases in radians. Detector-resolution convolution and efficiency corrections are not applied in this study.
The NRSPS component is described by
where the parameters , , and are fixed, while the dimensionless parameter is allowed to float within . The normalized NRSPS density is given by . As approaches zero, the distribution becomes increasingly concentrated near threshold.
To examine numerical instabilities beyond a particular fit configuration of the CMS analysis [3], we retain the background parametrizations and their fixed parameters, while using different input values for the resonance masses, widths, interference parameters, component yields, and the NRSPS shape parameter . A pseudo-dataset of 14,049 events, matching the event count in the CMS Run 2 dataset [3], is generated from this model. Figure 1 shows the resulting mass spectrum and the generation inputs are summarized in Table 1. The same sample is used throughout the numerical studies.
Figure 1. The mass spectrum of the pseudo-dataset together with the result of a converged fit. The black points represent the pseudo-dataset, while the solid blue curve shows the total fit. The individual NRSPS, DPS, BW, and coherent signal contributions are shown separately. The lower panel shows the pull distribution, defined as the difference between the data and the fit divided by the uncertainty in each bin.
Table 1: Generation inputs, allowed parameter ranges, and fitted values for the converged fit shown in Figure 1. Masses and widths are given in GeV and relative phases in radians. The parameters and are fixed, while the remaining 17 parameters are allowed to float in the fit. The fitted values are broadly consistent with the generation inputs, with the largest deviation approximately twice the corresponding fit uncertainty.
| Parameter | Unit | Input | Fit range | Returned fit value |
| – | 8153 | |||
| – | 3645 | |||
| – | 1020 | |||
| – | 1231 | |||
| GeV | 6.323 | |||
| GeV | 0.334 | |||
| GeV | 6.587 | |||
| GeV | 0.222 | |||
| – | 1.217 | |||
| rad | −0.334 | |||
| GeV | 6.871 | |||
| GeV | 0.134 | |||
| – | 1 | fixed | 1 | |
| rad | 0 | fixed | 0 | |
| GeV | 7.134 | |||
| GeV | 0.097 | |||
| – | 0.481 | |||
| rad | −0.951 | |||
| – | 0.091 |
Table 1.
The pseudo-dataset is fitted over the mass range of 6–15 GeV using an extended unbinned maximum-likelihood fit. The negative log-likelihood (NLL) is defined as
where , and the index runs over the four model components. Each PDF is normalized numerically over the fitted mass range. In the fit, a total of 17 parameters are allowed to vary: the four component yields, the masses and widths of BW, BW, BW, and BW, the relative interference parameters , , , and , and the NRSPS shape parameter . Here and () denote the masses and widths of the three interfering resonances, while and describe the noninterfering threshold component. For the coherent signal, and are fixed. The remaining NRSPS shape parameters in Eq. (3) and the DPS shape parameters are also held fixed. A converged fit, consistent with the generation inputs, is overlaid in Figure 1. The allowed fit ranges and the fitted parameters are summarized in Table 1.
The likelihood fits are performed using RooFit and Minuit in ROOT 6.20.06 [9, 10, 14]. The NLL is minimized using the MIGRAD algorithm in Minuit, with RooIntegrator1D used to evaluate the numerical normalization integrals. MIGRAD is a variable-metric minimization algorithm that uses gradient information to iteratively approach the minimum of the NLL. The convergence of MIGRAD is primarily assessed using the estimated distance to minimum (EDM), which estimates the expected further decrease in the NLL. In the default RooFit configuration, the EDM is required to be below for convergence. The integrator uses abso-
lute and relative tolerances to determine when the estimated numerical error is sufficiently small to terminate integration. A more detailed explanation of the tolerance is given in § 4. In the baseline configuration, both tolerances are set to , corresponding to the RooFit configuration used in this study. The integration method and tolerances can be configured through RooFit’s RooAbsReal::defaultIntegratorConfig() interface. The HESSE procedure [10] is subsequently used to estimate the Hessian matrix, whose elements are the second derivatives of the NLL with respect to the fit parameters at the fitted minimum, and to obtain the parameter covariance matrix and uncertainties from the inverse Hessian matrix.
Observation of Anomalous NLL Behavior
To ensure that the true global minimum of the NLL is found, the parameter space is explored by repeating the fit of the mass distribution 500 times on the same pseudo-dataset, with randomized initial parameter values drawn from the allowed fit ranges (Table 1). All fits use the numerical configuration of § 2, with both the absolute and relative integration tolerances set to , meaning that integration stops when the estimated absolute or relative numerical error falls below this value. For each fit, we track the minimum NLL (minNLL), the EDM estimated by MIGRAD from the NLL gradient and curvature, the status codes of the minimization and subsequent covariance-matrix evaluation, and the covariance-matrix quality (covQual). A status code of 0 indicates successful minimization or covariance-matrix computation, while a code of 4 reflects failure. A covQual of 3 indicates an accurate covariance matrix, while 2 corresponds to a matrix modified to ensure positive definiteness. Lower values reflect an approximate covariance matrix, potentially containing only diagonal elements, a matrix that was not calculated, or one of unknown quality. A fit is regarded as reliably converged only if both the minimization and covariance-matrix evaluation return status codes of 0, the EDM is below , the covQual is 3, and no floating parameter lies at the boundary of its allowed range.
Among the 500 fits, 110 satisfy these criteria and converge to a common minNLL value of approximately , which is used as a reference for comparison with the minNLL values obtained from the non-converged fits. Some non-converged fits, however, report substantially lower minNLL values. Figure 2 shows the distribution of minNLL values obtained from the 500 fits with randomized initial parameters, distinguishing fits that satisfy the convergence criteria from those that do not.
Figure 2. Distribution of minNLL for the 500 randomized fits, shown separately for fits that satisfy and that fail the convergence criteria. (a) Full range. (b) Zoom around the converged NLL region.
Because each PDF is normalized by a numerically evaluated integral, integration errors can distort the NLL and affect the minimization. To test whether such errors contribute to the anomalous results, we select two sets of initial parameter values from the 500 fits performed at : one associated with a converged fit reaching minNLL , and the other with a non-converged fit reporting an anomalously low minNLL. These are referred to as the reference and runaway starting points, respectively. To examine whether the anomalous fit behavior is sensitive to numerical integration accuracy, the minimization is repeated from each starting point with different absolute and relative integration tolerances of RooIntegrator1D, while all other settings remain unchanged. The results are summarized in Table 2.
| Reference starting point | Runaway starting point | |||||||
| status | covQual | EDM | minNLL | status | covQual | EDM | minNLL | |
| 0 | 3 | 4 | 2 | |||||
| 4 | 2 | 4 | 2 | |||||
| 0 | 3 | 0 | 3 | |||||
| 0 | 3 | 0 | 3 | |||||
| 0 | 3 | 4 | 3 |
Table 2. Fit results obtained from two sets of initial parameter values, referred to as the reference and runaway starting points. Both the absolute and relative RooIntegrator1D integration tolerances are set to the same value, denoted by . The first row () shows the original results of two fits selected from the 500 fits with randomized initial parameters: the fit from the reference starting point converged to minNLL , whereas the fit from the runaway starting point failed to converge and returned an anomalously low minNLL. Below the horizontal line, the fits are repeated using the same respective initial parameter values, with varied from the looser to the tighter . For each entry we report the fit status (0 for successful minimization and covariance-matrix evaluation, 4 for failure), the covariance-matrix quality (covQual; 3 for an accurate matrix, 2 for a matrix modified to ensure positive definiteness), the estimated distance to minimum (EDM) estimated by MIGRAD which estimates the expected further decrease in the NLL, and the minimized NLL (minNLL). The two cases investigated in detail in § 4 and § 5, corresponding to the reference starting point at and the runaway starting point at , respectively, are highlighted in bold.
At , both starting points yield non-converged fits with minNLL , about 1.3 NLL units below the reference value of obtained from the 110 converged fits. At and , all fits return minNLL values consistent with the reference NLL and have fit status zero. At , the runaway starting point produces a much more severe anomaly, with minNLL and an extremely large EDM. These results show that the reported NLL and the convergence behavior are highly sensitive to the numerical integration tolerance, even when the dataset, model, and starting parameters remain unchanged. To understand the origin of these anomalies, the two cases highlighted in bold in Table 2 are investigated in the following sections, focusing first on the modest NLL reduction for the reference starting point at and then on the much more severe anomaly for the runaway starting point at .
Numerical Discontinuities in Likelihood Evaluation and Their Origin
To investigate the numerical behavior of the likelihood, a one-dimensional scan of the X(6900) mass, , is first examined as an illustrative example. For the non-converged fit obtained from the reference starting point using RooIntegrator1D at (Table 2), all parameters are fixed to the values returned by that fit except , which is scanned over 6.8–6.9 GeV in steps of GeV. The scan exhibits pronounced discontinuities in the NLL (Figure 3(a)). To investigate their origin, we examine the numerical evaluation of the likelihood and its individual components in the vicinity of these discontinuities.
Figure 3. Numerical discontinuities and integrator stopping behavior in the scan for the reference starting point, with the other 16 fit parameters fixed at the values obtained from the fit using RooIntegrator1D (RI1D) at . (a) NLL scans comparing RI1D with tolerances of and , and adaptive Gauss–Kronrod (GK) integration with a tolerance of . (b) Enlarged view of the largest NLL discontinuity () obtained with RI1D at . (c) Corresponding signal normalization integral returned by RI1D, , together with the integral estimates and obtained at refinement levels and , respectively. RI1D evaluates the integral using the trapezoidal rule and progressively refines the integration grid by doubling the number of equal-width subintervals at each level, giving subintervals at level . Here, denotes the integral estimate obtained through Neville extrapolation, and is the level at which the integrator stops. (d) Numerical error estimates and for the signal normalization integral at refinement levels and , respectively, compared with the integration tolerance . Panels (b)–(d) share the same range. The vertical dotted lines indicate the interval over which the integrator stopping level switches from to at a tolerance of .
The largest NLL jump occurs between and GeV, where (Figure 3(b)),
compared with changes of approximately 0.1–0.4 in the NLL over a GeV step in away from the discontinuities. Table 3 presents the NLL and the signal normalization integral around this discontinuity. As can be seen from the table, the jump is accompanied by an abrupt decrease of approximately 0.91% in , compared with changes of only about 0.01–0.02% between other neighboring scan points. Although changes by less than 1%, its effect on the NLL is amplified by the presence of 2541 signal events at this scan point. Varying only between the two adjacent scan points and 6.8495 GeV, while holding the unnormalized densities fixed, yields . The remaining contribution from the change in the unnormalized signal density is +0.6, giving a total NLL change of −22.7, in agreement with the observed jump. The normalization integrals of the DPS, NRSPS, and BW components remain unchanged, as expected because these components do not depend on .
| [GeV] | NLL | ||||
| 6.8493 | −93064.6 | 0.21085 | — | — | — |
| 6.8494 | −93064.5 | 0.21081 | 0.1 | −0.00004 | −0.02% |
| 6.8495 | −93087.2 | 0.20890 | −22.7 | −0.00191 | −0.91% |
| 6.8496 | −93086.8 | 0.20888 | 0.4 | −0.00002 | −0.01% |
| 6.8497 | −93086.4 | 0.20886 | 0.4 | −0.00002 | −0.01% |
| 6.8498 | −93086.0 | 0.20883 | 0.4 | −0.00002 | −0.01% |
| 6.8499 | −93063.8 | 0.21061 | +22.2 | +0.00177 | +0.85% |
| 6.8500 | −93063.7 | 0.21056 | 0.1 | −0.00004 | −0.02% |
Table 3. NLL and signal normalization integral () evaluated in the scan around the largest NLL discontinuity, using RooIntegrator1D at with all other fit parameters held fixed. The differences and are calculated relative to the preceding scan point, and is the corresponding relative change with respect to at that preceding point, reported as a percentage. The pairs of adjacent scan points spanning the two NLL discontinuities are highlighted in bold, together with the corresponding large changes in , , and .
To understand why the numerical normalization changes abruptly as varies, we examine how RooIntegrator1D determines when to terminate numerical integration. To evaluate an integral , RooIntegrator1D initially applies the trapezoidal rule over the full integration range using a single interval, giving . At each successive refinement level , the number of equal-width subintervals is doubled, yielding a trapezoidal estimate based on subintervals [15]:
As the grid spacing approaches zero, the trapezoidal estimates converge to the integral, , and in practice this limit is estimated by extrapolating a finite number of estimates. Once the first five estimates, , have been obtained, Neville’s algorithm [16] is used to extrapolate these estimates to the limit of vanishing grid spacing. At each level , the five most recent trapezoidal estimates are used to obtain an extrapolated estimate of the normalization integral . The difference between the extrapolation using the four most recent estimates and that using all five, denoted , serves as an internal error estimate. Refinement continues until the estimated error falls below either the absolute tolerance or the relative tolerance ,
or until the maximum refinement level is reached. Once either tolerance is satisfied at level , the integrator stops and returns the extrapolated estimate . If the predefined maximum refinement level is reached without convergence, the integrator reports non-convergence and returns the final trapezoidal estimate [14]. The RooFit default configuration sets the absolute and relative integration tolerances equal, with . We retain this convention and vary to study the dependence on integration precision. Since measures the last extrapolation correction rather than the deviation from the exact integral, a small does not necessarily guarantee an accurate result.
At the largest NLL discontinuity (Figure 3(b)), the stopping level changes from to between and 6.8495 GeV. At the first point, the integration proceeds to level and returns the extrapolated estimate . At the neighboring point, the internal error estimate falls below the stopping threshold of (Figure 3(d)), causing the integrator to terminate at level and return instead. The switch between and introduces a discontinuity in the returned normalization integral, as shown in Figure 3(c). When rises above the stopping threshold again, the stopping level switches back to , producing the second discontinuity near GeV.
Except for the discontinuity between and 6.8495 GeV, there are four more discontinuities in the NLL for the scan, as shown in Figure 3(a) and summarized in Table 4. All five NLL discontinuities coincide with a change in the refinement level at which RooIntegrator1D terminates. To examine these stopping-level changes, Figure 4(a) shows the estimated integration errors at different refinement levels across the scan, while Figure 4(b) provides enlarged views near the stopping-level transitions. At these transitions, an error estimate crosses the stopping threshold, changing the first refinement level that satisfies the stopping criterion. This causes the integrator to return a different normalization estimate, producing the corresponding NLL discontinuity.
| transition [GeV] | transition | ||
| 6.8115 → 6.8116 | 9 → 10 | −2.2 | |
| 6.8494 → 6.8495 | 10 → 7 | −22.7 | |
| 6.8498 → 6.8499 | 7 → 10 | +22.2 | |
| 6.8575 → 6.8576 | 10 → 8 | −1.0 | |
| 6.8606 → 6.8607 | 8 → 9 | +2.8 |
Table 4. Changes in the NLL and signal normalization integral (), together with the corresponding stopping-level transitions, across the five discontinuities identified in the scan in Figure 3(a), using RooIntegrator1D at with all other fit parameters held fixed. The transition column gives the two adjacent scan points spanning each discontinuity, while the transition indicates the corresponding change in the integration stopping level. The differences and are calculated by subtracting the values at the first scan point from those at the second.

Figure 4. Numerical error estimates and stopping-level transitions in the scan using RooIntegrator1D at . (a) Magnitudes of the Neville extrapolation corrections, , for refinement levels –. The horizontal dashed line indicates the integration stopping threshold. (b) Enlarged views of the error estimates near the stopping-level transitions associated with the NLL discontinuities in Figure 3(a). The solid curves show (i) , (ii) , (iii) , and (iv) , while the horizontal dashed lines indicate the stopping threshold. The shaded regions distinguish different stopping levels , with transitions of (i) , (ii) , (iii) , and (iv) . These transitions occur where the corresponding error estimates cross the stopping threshold.
Although the scan reveals how numerical integration can introduce artificial discontinuities in the NLL, the largest jump occurs approximately 24 MeV away from the value GeV at which the minimizer stopped, where the NLL is locally smooth along this parameter direction. The role of this discontinuity in the convergence failure is therefore not immediately clear. To investigate this further, we examine the NLL as a function of the other floating parameters while keeping the remaining parameters fixed at their returned values. Among these scans, a numerical step is observed along the NRSPS shape parameter , close to its returned value (Figure 5(a) and Figure 5(b)). We therefore examine this step in detail to assess its role in the convergence failure.
Figure 5. Numerical behavior of the likelihood as a function of the NRSPS shape parameter , with the other 16 fit parameters fixed at the values returned by the non-converged fit using RooIntegrator1D (RI1D) at . (a) NLL as a function of over the full range, comparing RI1D at (red solid) with adaptive Gauss–Kronrod (GK) integration at (blue dashed). The red box indicates the region enlarged in panels (b) and (c). (b) Enlarged NLL scan using RI1D at , showing a numerical discontinuity of near the returned fit point.(c) Corresponding NRSPS normalization integrals evaluated using RI1D (red solid) and adaptive GK (blue dashed) with the same respective tolerances as in panel (a). The red dotted and black dashed vertical lines mark the numerical discontinuity at obtained with RI1D at and the corresponding returned fit value, respectively.
The discontinuity occurs near , just below the returned value of , and is accompanied by a corresponding jump in the NRSPS normalization (Figure 5(b) and Figure 5(c)). Between two adjacent scan points, and 0.128263, the NLL decreases by approximately 1.7, an unphysically large change for so fine a step in , and the returned NRSPS normalization decreases by approximately 0.021%. This discontinuity arises from a change in the RooIntegrator1D stopping level from to between these two points. In the fit, MIGRAD performed approximately 2730 function evaluations, moving from its initial value of 0.144 to 0.128285, but failed to converge. At termination, the numerical first derivative associated with was approximately , whereas those associated with the other 16 floating parameters were of order unity or smaller. This derivative is consistent with a finite-difference evaluation crossing the NLL discontinuity of approximately 1.7 and is much larger in magnitude than the local slope of the otherwise smooth NLL curve. MIGRAD failed to find a further decrease in the evaluated NLL and returned this point with an EDM of . The subsequent HESSE calculation failed to obtain a positive-definite error matrix. The large EDM and the absence of an accurate covariance matrix both fall short of the convergence criteria defined in § 3, indicating that the fit had not reached a trustworthy minimum.
To investigate the role of the numerical discontinuity in the convergence failure, additional minimization tests were performed. Starting from the returned fit point, both a fit with only floating and a full 17-parameter fit failed to move away from that point using RooIntegrator1D at . In contrast, repeating the full fit from the original initial values with the RooIntegrator1D refinement level fixed at restored convergence, with successful completion of the minimization and covariance-matrix evaluation (status = 0), an accurate covariance matrix (covQual = 3), and an estimated further decrease in the NLL below the threshold (EDM = ). These results indicate that the integration stopping-level switching was responsible for the observed convergence failure.
To mitigate these numerical discontinuities in the NLL, one approach is to tighten the integration tolerance . Alternatively, the integration can be improved by using a different numerical integration method. Since the accuracy of an integral depends on how the integration domain is sampled, adaptive Gauss–Kronrod quadrature [17], for example, uses two numerical approximations of the same integral to estimate local integration errors and selectively refine subintervals, in contrast to the uniform refinement used by RooIntegrator1D. Both approaches yield smooth NLL curves in the scan, as shown in Figure 3(a). The discontinuity near the returned value is also absent with adaptive Gauss–Kronrod integration at (Figure 5). To assess the impact on minimization, the full 17-parameter fit is repeated from the same initial val-
ues using adaptive Gauss–Kronrod integration at . The fit converges with status = 0, covQual = 3, and EDM = , reaching minNLL = , consistent with the reference value obtained in § 3. The recovery of convergence supports the interpretation that numerical discontinuities in the PDF normalization, and the resulting artificially lower NLL, contributed to the original minimization failure.
Severe Numerical Integration Errors from Unresolved Narrow Structures
Mechanism of Normalization Underestimation in the Presence of Narrow Structures
As shown in Table 2, repeating the fit from the same runaway starting point with progressively tighter integration tolerances leads to markedly different results. At and , the fits return minNLL values near , consistent with the reference solutions. However, further tightening the tolerance to produces a non-converged fit with an anomalously low minNLL of . In this fit, the NRSPS shape parameter decreases to , much smaller than the input value of 0.091, while the NRSPS yield increases to approximately , which is even larger than the total input yield of 14049. These extreme values are clearly not a physically preferred description of the spectrum.
To investigate this behavior, we examine the NLL as a function of the floating parameters while keeping the remaining parameters fixed at the values returned by the runaway fit. Unlike the other parameter scans, the scan reveals a pronounced decrease in the NLL toward very small values (Figure 6(a)), accompanied by a strong suppression of the NRSPS normalization integral (Figure 6(b)). Both RooIntegrator1D and adaptive Gauss–Kronrod integration exhibit this behavior, although the suppression is substantially more severe with RooIntegrator1D. Discontinuities are also observed in the NLL and normalization curves. For RooIntegrator1D, the integrator first checks whether the estimated integration error satisfies the prescribed tolerance at refinement level , using obtained from the extrapolation of successive trapezoidal approximations. As varies, crosses the integration tolerance (Figure 6(c)). On one side of the crossing, the integrator terminates at , while on the other side, it continues to higher refinement levels. The resulting difference in the returned normalization estimates produces the observed discontinuities in the normalization and NLL. Similar discontinuities in adaptive Gauss–Kronrod integration also arise from changes in the integration stopping decision. These stopping transitions explain the observed discontinuities, but not why the NRSPS normalization becomes extremely small at low .
Figure 6. Numerical behavior of the NRSPS normalization and likelihood as a function of . (a) NLL as a function of , with the other 16 fit parameters fixed at the values returned by the runaway fit at . Results are shown for RooIntegrator1D (RI1D) at , , and , and adaptive Gauss–Kronrod (GK) integration at . (b) Corresponding NRSPS normalization integral evaluated with the same integration settings. The vertical dotted lines in (a) and (b) mark the RI1D stopping transitions at , , and , with the corresponding values annotated. (c) Estimated integration error at refinement level as a function of . The intersections of the curve with the horizontal dashed lines mark the stopping-transition positions for the three integration tolerances. (d) NRSPS mass distributions for different values, each normalized to its maximum. The vertical dashed line indicates the threshold at GeV. The distributions become increasingly narrow near threshold as decreases.
The severe underestimation of the NRSPS normalization originates from the increasingly narrow distribution near the threshold. As shown in Figure 6(d), decreasing concentrates the distribution into a progressively smaller mass interval above the threshold. At refinement level , RooIntegrator1D divides the integration range of 6–15 GeV into 16 equal subintervals. The first sampling point above the threshold is therefore at 6.5625 GeV, approximately 369 MeV above threshold, whereas the NRSPS peak lies only 2.5 MeV above threshold at . The integration grid therefore misses the region that provides most of the integral and underestimates the true integral by many orders of magnitude. Since the estimated integration errors are calculated from the sampled function values, the errors may remain small even when the integration grid fails to resolve the dominant contribution, causing the integrator to terminate prematurely. As a result, tightening the integration tolerance does not necessarily prevent this failure. The same limitation can affect other numerical integration methods, including adaptive Gauss–Kronrod quadrature, whose error estimates also depend on sampled function values.
We further investigate how the minimizer moves from its initial value of , bypasses the expected minimum near , and eventually enters the anomalous small- region. As shown in Table 2, the fit from the same runaway starting point first fails to converge at looser tolerances. After the integration tolerance is tightened, the fits at and return minNLL values near , consistent with the reference solution, whereas the fit at returns a much lower value of . To understand why the two integration tolerances lead to different fit outcomes, we examine how MIGRAD [10] explores
the parameter space. During minimization, MIGRAD uses changes in the NLL to determine the direction in which the parameters should move and then evaluates the NLL at different distances along that direction to find a parameter point with a lower NLL. To keep within its specified range of , Minuit employs an internal parameter for minimization, which is mapped to through , following the Minuit standard transformation for parameters with finite lower and upper bounds. Because MIGRAD performs a line search in the internal parameter space, the nonlinear and periodic sine transformation allows a finite step in to reach a very small without evaluating the NLL at intermediate values between its current and new values, including intermediate values near the value of the normally converged fit.
For the fits from the same runaway starting point at and , with only the former converging, the change in integration tolerance produces small differences in the evaluated NLL, typically of order when the parameter values are nearly identical. These differences can affect the estimated gradients and consequently alter the search directions and the subsequent parameter points evaluated by MIGRAD. Although these two fits reach nearly identical parameter points, with , at the start of a line search, their subsequent evaluations probe different points in the 17-dimensional parameter space, with the trial values of differing by a factor of approximately 2.7 and the other fit parameters also taking different values. At , severe underestimation of the NRSPS normalization at the trial point produces an artificially low NLL of approximately , which is accepted as an improvement because the underestimated normalization inflates the NRSPS PDF contribution to the likelihood. In contrast, at , the trial at a different small- point gives an NLL of approximately , about 4055 higher than the lowest NLL previously found during that fit, and is therefore rejected. The minimizer subsequently returns to the vicinity of the normally converged region and eventually converges to the reference solution at . The different outcomes of these two fits show that changes in the integration tolerance can introduce small differences in the evaluated NLL, which in turn affect the parameter-space search performed by MIGRAD.
Improving Numerical Integration of Narrow Structures through Variable Transformations
The numerical failure identified above arises because the integration grid does not adequately resolve the narrow NRSPS contribution near threshold. To improve the sampling of that narrow region, we apply a Möbius variable transformation [18] to concentrate numerical sampling points near this region, without changing the NRSPS function or the likelihood definition.
The integration variable is transformed to through
where is the threshold, with GeV and . This choice increases the sampling density near threshold as decreases. The NRSPS normalization integral then becomes
For small , the transformation concentrates sampling points near threshold, where the NRSPS contribution is concentrated. Figure 7(a) compares the NRSPS normalization integrals obtained using the original RooIntegrator1D with , Möbius-transformed RooIntegrator1D with , and Möbius-transformed adaptive Gauss–Kronrod integration with . At , the value returned by the non-converged fit (Tab. 2 runaway case), the Möbius-transformed calculation gives , compared with from the original RooIntegrator1D. The two transformed integration methods give consistent results across the scanned range, supporting the reliability of the numerical integration with suitable variable transformation and eliminating the severe underestimation observed with the original method. The corresponding NLL scans in Figure 7(b) are also smooth after the transformation, without the anomalously low NLL values at small .
Figure 7. Improved evaluation of the NRSPS normalization and the likelihood using the Möbius variable transformation. (a) NRSPS normalization integral as a function of , comparing the original RooIntegrator1D (RI1D) with , Möbius-transformed RI1D with , and Möbius-transformed adaptive Gauss–Kronrod integration with . (b) Corresponding NLL scans for using the same three integration configurations, with the other 16 fit parameters fixed at the values returned by the non-converged fit using the original RI1D at (the runaway case in Table 2). (c) NLL scan for around the corrected best-fit solution using Möbius-transformed adaptive Gauss–Kronrod integration with , with the other 16 parameters fixed at their corrected best-fit values. The green marker indicates the minimum of the scan. The variable transformation removes the severe underestimation of the normalization integral and the associated anomalously low NLL values observed with the original integration method.
The full 17-parameter fit is then repeated from the same initial parameter values in the runaway case, using Möbius-transformed adaptive Gauss–Kronrod integration with for the NRSPS normalization. The pseudo-dataset and all other aspects of the fit remain unchanged. The fit converges with minNLL and , consistent with the reference solution in Table 2. The corresponding NLL scan for exhibits a smooth minimum, as shown in Figure 7(c). As an independent check, we also test the alternative transformation , where , a simple quadratic map that likewise concentrates sampling near threshold. Using adaptive Gauss–Kronrod integration with , this transformation also recovers the reference likelihood minimum. These results show that improving the numerical sampling near threshold can stabilize the NRSPS normalization and restore convergence of the full likelihood fit without modifying the underlying mass-spectrum model.
Robustness and Discussion
The two numerical failures examined in § 4 and § 5 illustrate how inaccurate numerical evaluation of the PDF normalization can distort the likelihood. In the first case, the integrator’s decision to stop or continue can change between neighboring parameter points, producing discontinuities in the normalization and hence the NLL, even when the underlying physical density is smooth, which can stall MIGRAD at a spuriously low NLL caused by the incorrect numerical integral. In the second case, the NRSPS distribution becomes increasingly concentrated near the threshold as decreases. The integration grid may then fail to resolve the narrow peak, leading to a severely underestimated normalization and an artificially low-NLL region into which the minimization can be drawn.
To evaluate the robustness of the improved numerical integration strategies discussed above—tightening the integration tolerances, adopting an alternative adaptive integration algorithm (using Gauss–Kronrod integration as an example), and applying a suitable variable transformation (as demonstrated using the Möbius transformation for the NRSPS integral in § 5.2)—we repeat the 500 fits for each of the three strategies, using the same randomly generated initial parameter values as in the original study (Figure 2). The performance of the three strategies is summarized in Table 5, including their effectiveness in recovering anomalously low NLL values and improving fit convergence. Tightening the integration tolerances in RooIntegrator1D from to , , or recovers nonnegative values for 86% of the 69 fits originally exhibiting anomalously low NLL values with , where and denotes the reference global minimum. Among the 390 fits that did not reach with RooIntegrator1D at , 54% satisfy the convergence criteria and 53% converge to with these tighter tolerances. Adaptive Gauss–Kronrod integration at yields the corresponding anomalous-NLL recovery rate of 96%, the convergence rate of 81%, and the fraction converging to of 78%. Further applying the Möbius transformation to the NRSPS integral with adaptive Gauss–Kronrod eliminates the remaining anomalously low-NLL cases and achieves convergence for 18% of the fits that did not converge with adaptive Gauss–Kronrod integration alone, with 14% of those that did not reach under adaptive Gauss–Kronrod integration now converging to . Combining the results from the three strategies eliminates all anomalously low NLL values and achieves an overall convergence rate of 87% among the 390 fits that did not converge to with the original configuration, with 84% converging to .
| Method | Anomalous-NLL recovery | Converged | Converged to |
| () (%) | () (%) | () (%) | |
| Default RI1D | 0/69 (0%) | 112/500 (22%) | 110/500 (22%) |
| Tighter RI1D | 59/69 (86%) | 212/390 (54%) | 206/390 (53%) |
| Adaptive GK | 66/69 (96%) | 314/390 (81%) | 306/390 (78%) |
| Möbius NRSPS + adaptive GK | 3/3 (100%) | 14/76 (18%) | 12/84 (14%) |
| Combined (best result per fit) | 69/69 (100%) | 339/390 (87%) | 327/390 (84%) |
Table 5. Robustness of three numerical integration strategies using the same 500 sets of randomly generated initial parameter values as in Figure 2. The strategies include tightening the integration tolerances with RooIntegrator1D (RI1D) at , , , adopting adaptive Gauss–Kronrod (adaptive GK) integration at , and applying the Möbius transformation to the NRSPS integral with adaptive GK integration at . These strategies are evaluated on the 390 fits that did not converge to the reference global minimum () using RI1D at . The three performance metrics are defined as follows. “Anomalous-NLL recovery” denotes the fraction of the 69 original fits with for which is recovered, where and is the NLL at . “Converged” requires status = 0, covQual = 3, and EDM , while “Converged to ” additionally requires reaching the reference minimum. Each entry is given as () (%), where and denote the numbers of fits satisfying the corresponding criterion and considered for that metric, respectively, with the percentage given in parentheses. The Möbius transformation is applied only to cases unresolved by adaptive GK, where counts the fits still failing that metric after adaptive GK. “Combined (best result per fit)” represents the combined performance, selecting one result per fit by prioritizing convergence to , followed by convergence and smallest nonnegative .
Figure 8 further compares the original distribution with that obtained by selecting one result for each of the 500 initial-parameter sets from the three strategies. For each set, the fit results obtained using tighter RooIntegrator1D tolerances, adaptive Gauss–Kronrod integration, and Möbius-transformed adaptive Gauss–Kronrod integration are compared. Priority is given to results converging to the reference minimum . If no result reaches , a result satisfying the convergence criteria is selected. Otherwise, the result with the smallest nonnegative is retained. The histograms are stacked with different colors indicating the selected strategies, with each fit assigned the same color in both panels. As shown in Figure 8, the resulting distribution no longer contains the anomalously low NLL values below the reference global minimum. Together with the individual recovery rates in Table 5, these results demonstrate that the numerical integration strategies examined here effectively suppress spurious low-NLL solutions arising from numerical integration errors and improve the robustness of the fitting procedure.
Figure 8. Comparison of the distributions for the 500 fits obtained with (a) the original RooIntegrator1D (RI1D) at and (b) the selected results from the three improved numerical integration strategies: tighter RI1D tolerances with (orange), adaptive Gauss–Kronrod (GK) integration (green), and Möbius-transformed GK integration of the NRSPS integral (purple). Here, , where denotes the NLL at the reference global minimum (black dashed). The annotations and indicate the numbers of fits with negative and those converging to , respectively. For each fit, the value shown in panel (b) is taken from the selected result, prioritizing convergence to , followed by convergence, and finally the smallest nonnegative . Fits already converged to with the original method are shown in gray. The histograms are stacked with different colors indicating the selected strategies, with each fit assigned the same color in both panels.
The remaining convergence failures are associated with the strong correlations among fit parameters, local minima, and numerical integration errors. Strong parameter correlations can allow changes in one parameter to be compensated by changes in others with little variation in the NLL, while numerical integration errors can affect the estimation of gradients and curvature. These effects can hinder gradient-based minimization, which may become trapped in local minima or fail to converge. A search using the SIMPLEX algorithm in Minuit [10] can explore alternative regions of the parameter space by comparing NLL values at different points, without relying on gradient or curvature estimates. This makes the search less sensitive to unreliable local derivative estimates and can provide a better starting point for subsequent refinement with MIGRAD. Additional improvements can be achieved by adjusting parameter bounds to limit excursions along weakly constrained directions, where large changes in parameter values have little effect on the likelihood, and by reparameterizing strongly correlated parameters to reduce disparities in curvature across parameter directions, thereby improving the stability of the minimization. With these additional treatments, all 500 fits were successfully brought to convergence at the reference global minimum.
Although demonstrated using a specific pseudo-dataset and Minuit, the numerical difficulties identified here reflect general challenges in likelihood minimization. Extended unbinned likelihood fits relying on numerical PDF normalization, particularly for distributions that can become sharply peaked, may encounter similar problems when integration grids fail to resolve important contributions or when stopping decisions change. Practical safeguards include examining the likelihood and component normalization integrals along sensitive parameters, adjusting numerical integration tolerances, comparing independent integration algorithms, monitoring shape parameters whose values can produce sharply localized features, and applying suitable integration variable transformations where necessary.
These safeguards, however, come with practical trade-offs. Tighter integration tolerances can substantially increase computational cost. For a representative set of values, the average evaluation time of the NRSPS normalization integral using RooIntegrator1D increases by factors of approximately 70 and 260 when the tolerance is reduced from to and , respectively. As normalization integrals are evaluated repeatedly during minimization, this additional cost can become substantial. Furthermore, tighter tolerances may offer little improvement in accuracy for numerically stable fits and do not necessarily resolve integration errors when sharply localized features are insufficiently sampled by the integration algorithm. The effectiveness of alternative integration algorithms and variable transformations also depends on the integrand and parameter region, while convergence difficulties associated with strong parameter correlations or local minima may require separate treatments. We therefore recommend retaining the default settings when numerical stability has been verified and applying more stringent tolerances or alternative integration methods when numerical instabilities are identified. Regardless of the numerical settings used, an anomalously low NLL accompanied by failed convergence diagnostics should be treated as a potential numerical warning rather than evidence for a better physical solution.
Conclusions
Anomalously low NLL values have been investigated using a pseudo-dataset based on the CMS analysis [3, 4, 12]. Numerical errors in PDF normalization can introduce artificial structures in the likelihood surface and lead to unreliable minimization results. In particular, changes in the integrator’s stopping decision as the fit parameters vary can produce discontinuities in the returned normalization, even at fixed integration tolerance. The problem becomes particularly severe when the PDF develops a sharply localized structure that is poorly resolved by the integration grid, allowing the numerical error estimate to indicate convergence despite a substantial normalization error. Such errors can cause the minimization algorithm (in our example, MIGRAD [10]) to encounter and retain parameter configurations with spuriously low NLL values.
Tightening the integration tolerance or using alternative adaptive numerical integration can mitigate these instabilities, but these approaches do not necessarily resolve sharply localized contributions. When evaluating PDF normalizations, an appropriate variable transformation that concentrates the sampling points in the critically sampled region can stabilize the numerical integral and restore reliable minimization. Across the 500 fits studied, these strategies eliminate all anomalously low NLL values and substantially improve convergence to the reference global minimum (Table 5). These results imply that an apparently lower NLL need not signal a better description of the data. It may instead arise from an inaccurate PDF normalization, even when the numerical integrator reports that its stopping criterion has been satisfied. In particular, non-converged fits that report unusually low NLL values should not be regarded as mathematically preferred solutions. Care is therefore required when interpreting such outcomes in fits that rely on numerically evaluated PDF normalizations.
Acknowledgements
This work was supported by the National Natural Science Foundation of China under Grant No. 12535004.
References
- [1]LHCb Collaboration, Sci. Bull. 65, 1983 (2020). DOI 10.1016/j.scib.2020.08.032. [arXiv:2006.16957]DOI
- [2]ATLAS Collaboration, Phys. Rev. Lett. 131, 151902 (2023). DOI 10.1103/PhysRevLett.131.151902. [arXiv:2304.08962]DOI
- [3]CMS Collaboration, Phys. Rev. Lett. 132, 111901 (2024). DOI 10.1103/PhysRevLett.132.111901. [arXiv:2306.07164]DOI
- [4]CMS Collaboration, Nature 648, 58 (2025). DOI 10.1038/s41586-025-09711-7. [arXiv:2506.07944]DOI
- [5]CMS Collaboration, (2026). Submitted to Science Advances, [arXiv:2602.02252]arxiv.org/abs/2602.02252
- [6]F. Zhu, G. Bauer, K. Yi, Chin. Phys. Lett. 41, 111201 (2024). DOI 10.1088/0256-307X/41/11/111201. [arXiv:2410.11210]DOI
- [7]H. Zhou, X. Wang, F. Zhu, L. Zhang, G. Bauer, K. Yi, Chin. Phys. Lett. 42, 120202 (2025). DOI 10.1088/0256-307X/42/12/120202. [arXiv:2511.08495]DOI
- [8]Y. Bai, D.Y. Chen, Phys. Rev. D 99, 072007 (2019). DOI 10.1103/PhysRevD.99.072007. [arXiv:1901.01394]DOI
- [9]W. Verkerke, D.P. Kirkby, eConf C0303241, MOLT007 (2003). [arXiv:physics/0306116]
- [10]F. James, M. Roos, Comput. Phys. Commun. 10, 343 (1975). DOI 10.1016/0010-4655(75)90039-9
- [11]L. Chen, Y. Chen, G. Bauer, L.G. Spiegel, Z. Hu, K. Yi, Symmetry 17, 57 (2025). DOI 10.3390/sym17010057. [arXiv:2412.20777]DOI
- [12]CMS Collaboration, Observation of new structures in the J/ψψ̄/ψ mass spectrum in pp collisions at √s = 13 TeV. Tech. Rep. CMS-PAS-BPH-21-003, CERN (2022). Https://cds.cern.ch/record/2815336
- [13]F. Takahashi, et al., Int. J. Mod. Phys. A 41, 2630011 (2026). DOI 10.1142/S0217751X26300115DOI
- [14]ROOT Project. ROOT: data analysis framework. https://root.cern. Accessed: 2026-11-09
- [15]W. Romberg, Det Kongelige Norske Videnskabers Selskab Forhandlinger 28(7), 30 (1955)
- [16]E.H. Neville, J. Indian Math. Soc. 20, 87 (1934)
- [17]M. Galassi, et al., GNU Scientific Library (Network Theory Ltd., 2019)
- [18]H.H.H. Homeier, E.O. Steinborn, J. Comput. Phys. 87(1), 61 (1990). DOI 10.1016/0021-9991(90)90225-PDOI






