betaFeedback

WorksPhysics and Astronomy

Numerical Integration Instabilities in Likelihood Fits: Mechanisms and Remedies

Abstract

Numerical instabilities in the normalization of probability density functions can produce artificially lower negative log-likelihood (NLL) values than the true minimum of a likelihood fit, even when the resulting fit provides a poorer description of the data. Motivated by anomalously low NLL values found in non-converged fits associated with a recent CMS search for fully charmed tetraquark candidates, we use a pseudo-dataset generated from a likelihood model based on that analysis to investigate how numerical integration can yield unreliable normalization results, how these errors affect likelihood minimization, and how they can be addressed. We find that, even at fixed integration tolerance, the stopping decision of a numerical integrator can change with the fit parameters, introducing discontinuities in the normalization integrals that propagate to the likelihood surface. Such discontinuities can trap the minimizer at a numerical step and yield an artificially lower NLL. Tightening the integration tolerance mitigates these numerical discontinuities, while adaptive integration methods, such as Gauss–Kronrod quadrature, can also improve the numerical stability. A more severe failure occurs when the integration grid cannot adequately resolve a sharply localized contribution. In this case, the integrator may satisfy its estimated-error criterion despite substantially underestimating the normalization, allowing the minimizer to enter a region with an artificially low NLL. Tightening integration tolerances or using alternative adaptive integration methods does not necessarily resolve sharply localized contributions. We demonstrate that a suitable variable transformation can improve their numerical resolution, yielding more accurate normalization integrals and restoring fit convergence.

Introduction

Maximum-likelihood fitting is widely used for parameter estimation in many fields. In many applications, the normalization integrals of probability density functions (PDFs) depend on the fit parameters and must be evaluated numerically to calculate the likelihood. Numerical inaccuracies in these integrals can distort the evaluated likelihood, potentially affecting parameter estimation and fit convergence.

Likelihood fitting is extensively used in experimental particle physics, and we draw on the case of fitting structures in the J/ψψJ/\psi\psi mass spectrum reported by three LHC experiments [1, 2, 3, 4, 5]. In particular, CMS has reported a family of fully charmed tetraquark candidates whose masses exhibit a Regge pattern, with possible connections to structures observed in the J/ψψJ/\psi\psi spectrum [3, 4, 5, 6, 7]. Extracting resonance properties from these spectra involves complex likelihood fits with multiple contributions, as exemplified by the CMS J/ψψJ/\psi\psi analysis [3, 4, 5], which includes three interfering resonances and yields multiple fit solutions [8]. The fits were performed using RooFit [9], with RooIntegrator1D for numerical normalization and MIGRAD in Minuit [10] for minimization. In maximum-likelihood fits, a lower negative log-likelihood (NLL) ordinarily indicates a better description of the data within the assumed model. However, during the development of this analysis, some fits returned NLL values substantially lower than those obtained from normally converged fits, despite failing standard convergence criteria. Likelihood scans of these supposed minima exhibited discontinuities. This behavior was traced back to problems with numerical integrations performed in the minimization algorithm and motivated a broader investigation into the conditions under which numerical integration can produce inaccurate normalization results, how the resulting errors affect likelihood minimization, and how such failures can be identified and corrected.

In this work, we investigate this problem using a pseudo-dataset based on the CMS analysis [3, 4]. Other technical challenges, such as the estimation of large global significances, have been discussed elsewhere [11] and are beyond the scope of this study. We examine how the criteria used to terminate numerical integration affect the accuracy and continuity of the normalization integrals and the resulting likelihood, and show that tightening the integration tolerance or using adaptive integration methods can mitigate the numerical discontinuities. We further investigate a more severe failure in which a sharply localized contribution is not adequately resolved by the integration grid, causing the normalization to be substantially underestimated despite satisfying the estimated-error criterion. By examining the minimization process, we trace how the resulting likelihood distortion leads the minimizer into an anomalous parameter region. Finally, we compare different numerical integration approaches and demonstrate that a suitable variable transformation can improve the normalization accuracy and restore fit convergence.

Model and Fit Setup

The numerical study considers the J/ψJ/ψJ/\psi J/\psi mass spectrum, m=mJ/ψJ/ψm = m_{J/\psi J/\psi}, in the range 6<m<156 < m < 15 GeV. Following the CMS analysis [12], the model consists of four components: nonresonant single-parton scattering (NRSPS), double-parton scattering (DPS), a noninterfering threshold component BW0_0, and a coherent contribution from three resonances, X(6600)X(6600), X(6900)X(6900), and X(7100)X(7100). The total event density is

F(m)=NNRSPSPNRSPS(m)+NDPSPDPS(m)+NBW0PBW0(m)+NsigPsig(m),(1)F(m) = N_{\mathrm{NRSPS}}P_{\mathrm{NRSPS}}(m) + N_{\mathrm{DPS}}P_{\mathrm{DPS}}(m) + N_{\mathrm{BW}_0}P_{\mathrm{BW}_0}(m) + N_{\mathrm{sig}}P_{\mathrm{sig}}(m), \tag*{(1)}

where NiN_i is the yield of component ii, and each Pi(m)P_i(m) is a PDF normalized over [6,15][6,15] GeV. All four component densities vanish for m<mthm < m_{\mathrm{th}}, where mth=2mJ/ψm_{\mathrm{th}} = 2m_{J/\psi} is the J/ψJ/ψJ/\psi J/\psi production threshold [13]. The DPS shape is described by a fixed parametrization derived from simulated events [3, 4]. The threshold component BW0_0 is modeled by a relativistic SS-wave Breit–Wigner with floating mass and width and does not interfere with the other components. The three signal resonances are described by relativistic SS-wave Breit–Wigner amplitudes, combined coherently following the CMS three-resonance interference model [3, 4]:

Psig(m)∝∣r1eiϕ1BW1(m)+BW2(m)+r3eiϕ3BW3(m)∣2,(2)P_{\mathrm{sig}}(m) \propto\left|r_1 e^{i\phi_1}\mathrm{BW}_1(m) + \mathrm{BW}_2(m) + r_3 e^{i\phi_3}\mathrm{BW}_3(m)\right|^2, \tag*{(2)}

where BW1_1, BW2_2, and BW3_3 correspond to X(6600)X(6600), X(6900)X(6900), and X(7100)X(7100), respectively. The X(6900)X(6900) amplitude is taken as the reference, with r2=1r_2 = 1 and ϕ2=0\phi_2 = 0. The parameters r1r_1 and r3r_3 are dimensionless relative magnitudes, while ϕ1\phi_1 and ϕ3\phi_3 are relative phases in radians. Detector-resolution convolution and efficiency corrections are not applied in this study.

The NRSPS component is described by

fNRSPS(m)=(m−mth)α[1−(1(15−mth)2−p110)(15−m)2]exp⁡[−(m−mth)p32p2p3],(3)f_{\mathrm{NRSPS}}(m) = (m-m_{\mathrm{th}})^\alpha\left[1-\left(\frac{1}{(15-m_{\mathrm{th}})^2}-\frac{p_1}{10}\right)(15-m)^2\right]\exp\left[-\frac{(m-m_{\mathrm{th}})^{p_3}}{2p_2^{p_3}}\right], \tag*{(3)}

where the parameters α=0.80567\alpha= 0.80567, p1=0.32647p_1 = 0.32647, and p3=0.53968p_3 = 0.53968 are fixed, while the dimensionless parameter p2p_2 is allowed to float within [0,1][0,1]. The normalized NRSPS density is given by PNRSPS(m)∝fNRSPS(m)P_{\mathrm{NRSPS}}(m) \propto f_{\mathrm{NRSPS}}(m). As p2p_2 approaches zero, the distribution becomes increasingly concentrated near threshold.

To examine numerical instabilities beyond a particular fit configuration of the CMS analysis [3], we retain the background parametrizations and their fixed parameters, while using different input values for the resonance masses, widths, interference parameters, component yields, and the NRSPS shape parameter p2p_2. A pseudo-dataset of 14,049 events, matching the event count in the CMS Run 2 dataset [3], is generated from this model. Figure 1 shows the resulting mass spectrum and the generation inputs are summarized in Table 1. The same sample is used throughout the numerical studies.

Figure 1. The J/ψJ/ψJ/\psi J/\psi mass spectrum of the pseudo-dataset together with the result of a converged fit. The black points represent the pseudo-dataset, while the solid blue curve shows the total fit. The individual NRSPS, DPS, BW0_0, and coherent signal contributions are shown separately. The lower panel shows the pull distribution, defined as the difference between the data and the fit divided by the uncertainty in each bin.

Table 1: Generation inputs, allowed parameter ranges, and fitted values for the converged fit shown in Figure 1. Masses and widths are given in GeV and relative phases in radians. The parameters r2r_2 and ϕ2\phi_2 are fixed, while the remaining 17 parameters are allowed to float in the fit. The fitted values are broadly consistent with the generation inputs, with the largest deviation approximately twice the corresponding fit uncertainty.

ParameterUnitInputFit rangeReturned fit value
NNRSPSN_{\mathrm{NRSPS}}–8153[0,105][0,10^5]8071±3358071 \pm335
NDPSN_{\mathrm{DPS}}–3645[0,105][0,10^5]3168±2353168 \pm235
NBW0N_{\mathrm{BW}_0}–1020[0,104][0,10^4]1334±5281334 \pm528
NsigN_{\mathrm{sig}}–1231[0,104][0,10^4]1476±3631476 \pm363
mBW0m_{\mathrm{BW}_0}GeV6.323[6.20,6.50][6.20,6.50]6.339±0.0456.339 \pm0.045
ΓBW0\Gamma_{\mathrm{BW}_0}GeV0.334[0,1.50][0,1.50]0.413±0.1990.413 \pm0.199
m1m_1GeV6.587[6.40,6.80][6.40,6.80]6.635±0.0256.635 \pm0.025
Γ1\Gamma_1GeV0.222[0,1.50][0,1.50]0.215±0.0840.215 \pm0.084
r1r_1–1.217[0,103][0,10^3]0.846±0.4100.846 \pm0.410
ϕ1\phi_1rad−0.334[−π,π][-\pi,\pi]0.481±0.6760.481 \pm0.676
m2m_2GeV6.871[6.80,7.00][6.80,7.00]6.873±0.0276.873 \pm0.027
Γ2\Gamma_2GeV0.134[0,0.50][0,0.50]0.190±0.0560.190 \pm0.056
r2r_2–1fixed1
ϕ2\phi_2rad0fixed0
m3m_3GeV7.134[7.00,7.50][7.00,7.50]7.096±0.0347.096 \pm0.034
Γ3\Gamma_3GeV0.097[0,0.50][0,0.50]0.176±0.0770.176 \pm0.077
r3r_3–0.481[0,103][0,10^3]0.653±0.2600.653 \pm0.260
ϕ3\phi_3rad−0.951[−π,π][-\pi,\pi]−1.547±0.445-1.547 \pm0.445
p2p_2–0.091[0,1][0,1]0.107±0.0100.107 \pm0.010

Table 1.

The pseudo-dataset is fitted over the mass range of 6–15 GeV using an extended unbinned maximum-likelihood fit. The negative log-likelihood (NLL) is defined as

NLL=Ntot−∑j=1Nobsln⁡[∑iNiPi(mj)].(4)\mathrm{NLL}=N_{\mathrm{tot}}-\sum_{j=1}^{N_{\mathrm{obs}}}\ln\left[\sum_i N_iP_i(m_j)\right]. \tag*{(4)}

where Ntot=∑iNiN_{\mathrm{tot}}=\sum_iN_i, and the index ii runs over the four model components. Each PDF is normalized numerically over the fitted mass range. In the fit, a total of 17 parameters are allowed to vary: the four component yields, the masses and widths of BW0_0, BW1_1, BW2_2, and BW3_3, the relative interference parameters r1r_1, r3r_3, ϕ1\phi_1, and ϕ3\phi_3, and the NRSPS shape parameter p2p_2. Here mim_i and Γi\Gamma_i (i=1,2,3i=1,2,3) denote the masses and widths of the three interfering resonances, while mBW0m_{\mathrm{BW}_0} and ΓBW0\Gamma_{\mathrm{BW}_0} describe the noninterfering threshold component. For the coherent signal, r2=1r_2=1 and ϕ2=0\phi_2=0 are fixed. The remaining NRSPS shape parameters in Eq. (3) and the DPS shape parameters are also held fixed. A converged fit, consistent with the generation inputs, is overlaid in Figure 1. The allowed fit ranges and the fitted parameters are summarized in Table 1.

The likelihood fits are performed using RooFit and Minuit in ROOT 6.20.06 [9, 10, 14]. The NLL is minimized using the MIGRAD algorithm in Minuit, with RooIntegrator1D used to evaluate the numerical normalization integrals. MIGRAD is a variable-metric minimization algorithm that uses gradient information to iteratively approach the minimum of the NLL. The convergence of MIGRAD is primarily assessed using the estimated distance to minimum (EDM), which estimates the expected further decrease in the NLL. In the default RooFit configuration, the EDM is required to be below 10−310^{-3} for convergence. The integrator uses abso-

lute and relative tolerances to determine when the estimated numerical error is sufficiently small to terminate integration. A more detailed explanation of the tolerance is given in § 4. In the baseline configuration, both tolerances are set to 10−710^{-7}, corresponding to the RooFit configuration used in this study. The integration method and tolerances can be configured through RooFit’s RooAbsReal::defaultIntegratorConfig() interface. The HESSE procedure [10] is subsequently used to estimate the Hessian matrix, whose elements are the second derivatives of the NLL with respect to the fit parameters at the fitted minimum, and to obtain the parameter covariance matrix and uncertainties from the inverse Hessian matrix.

Observation of Anomalous NLL Behavior

To ensure that the true global minimum of the NLL is found, the parameter space is explored by repeating the fit of the mass distribution 500 times on the same pseudo-dataset, with randomized initial parameter values drawn from the allowed fit ranges (Table 1). All fits use the numerical configuration of § 2, with both the absolute and relative integration tolerances set to ε=10−7\varepsilon= 10^{-7}, meaning that integration stops when the estimated absolute or relative numerical error falls below this value. For each fit, we track the minimum NLL (minNLL), the EDM estimated by MIGRAD from the NLL gradient and curvature, the status codes of the minimization and subsequent covariance-matrix evaluation, and the covariance-matrix quality (covQual). A status code of 0 indicates successful minimization or covariance-matrix computation, while a code of 4 reflects failure. A covQual of 3 indicates an accurate covariance matrix, while 2 corresponds to a matrix modified to ensure positive definiteness. Lower values reflect an approximate covariance matrix, potentially containing only diagonal elements, a matrix that was not calculated, or one of unknown quality. A fit is regarded as reliably converged only if both the minimization and covariance-matrix evaluation return status codes of 0, the EDM is below 10−310^{-3}, the covQual is 3, and no floating parameter lies at the boundary of its allowed range.

Among the 500 fits, 110 satisfy these criteria and converge to a common minNLL value of approximately −93079.8-93079.8, which is used as a reference for comparison with the minNLL values obtained from the non-converged fits. Some non-converged fits, however, report substantially lower minNLL values. Figure 2 shows the distribution of minNLL values obtained from the 500 fits with randomized initial parameters, distinguishing fits that satisfy the convergence criteria from those that do not.

Figure 2. Distribution of minNLL for the 500 randomized fits, shown separately for fits that satisfy and that fail the convergence criteria. (a) Full range. (b) Zoom around the converged NLL region.

Because each PDF is normalized by a numerically evaluated integral, integration errors can distort the NLL and affect the minimization. To test whether such errors contribute to the anomalous results, we select two sets of initial parameter values from the 500 fits performed at ε=10−7\varepsilon= 10^{-7}: one associated with a converged fit reaching minNLL ≈−93079.8\approx-93079.8, and the other with a non-converged fit reporting an anomalously low minNLL. These are referred to as the reference and runaway starting points, respectively. To examine whether the anomalous fit behavior is sensitive to numerical integration accuracy, the minimization is repeated from each starting point with different absolute and relative integration tolerances of RooIntegrator1D, while all other settings remain unchanged. The results are summarized in Table 2.

Reference starting pointRunaway starting point
ε\varepsilonstatuscovQualEDMminNLLstatuscovQualEDMminNLL
10−710^{-7}034.9×10−54.9 \times10^{-5}−93079.8-93079.8422.7×1072.7 \times10^{7}−449012.9-449012.9
10−610^{-6}425.0×1025.0 \times10^{2}−93081.1-93081.1423.1×1023.1 \times10^{2}−93081.1-93081.1
10−810^{-8}035.1×10−45.1 \times10^{-4}−93079.8-93079.8032.3×10−42.3 \times10^{-4}−93079.8-93079.8
10−1010^{-10}034.2×10−44.2 \times10^{-4}−93079.8-93079.8034.1×10−44.1 \times10^{-4}−93079.8-93079.8
10−1210^{-12}031.1×10−41.1 \times10^{-4}−93079.8-93079.8434.7×10144.7 \times10^{14}−119890.2-119890.2

Table 2. Fit results obtained from two sets of initial parameter values, referred to as the reference and runaway starting points. Both the absolute and relative RooIntegrator1D integration tolerances are set to the same value, denoted by ε\varepsilon. The first row (ε=10−7\varepsilon= 10^{-7}) shows the original results of two fits selected from the 500 fits with randomized initial parameters: the fit from the reference starting point converged to minNLL ≈−93079.8\approx-93079.8, whereas the fit from the runaway starting point failed to converge and returned an anomalously low minNLL. Below the horizontal line, the fits are repeated using the same respective initial parameter values, with ε\varepsilon varied from the looser 10−610^{-6} to the tighter 10−1210^{-12}. For each entry we report the fit status (0 for successful minimization and covariance-matrix evaluation, 4 for failure), the covariance-matrix quality (covQual; 3 for an accurate matrix, 2 for a matrix modified to ensure positive definiteness), the estimated distance to minimum (EDM) estimated by MIGRAD which estimates the expected further decrease in the NLL, and the minimized NLL (minNLL). The two cases investigated in detail in § 4 and § 5, corresponding to the reference starting point at ε=10−6\varepsilon= 10^{-6} and the runaway starting point at ε=10−12\varepsilon= 10^{-12}, respectively, are highlighted in bold.

At ε=10−6\varepsilon= 10^{-6}, both starting points yield non-converged fits with minNLL ≈−93081.1\approx-93081.1, about 1.3 NLL units below the reference value of −93079.8-93079.8 obtained from the 110 converged fits. At ε=10−8\varepsilon= 10^{-8} and 10−1010^{-10}, all fits return minNLL values consistent with the reference NLL and have fit status zero. At ε=10−12\varepsilon= 10^{-12}, the runaway starting point produces a much more severe anomaly, with minNLL ≈−119890.2\approx-119890.2 and an extremely large EDM. These results show that the reported NLL and the convergence behavior are highly sensitive to the numerical integration tolerance, even when the dataset, model, and starting parameters remain unchanged. To understand the origin of these anomalies, the two cases highlighted in bold in Table 2 are investigated in the following sections, focusing first on the modest NLL reduction for the reference starting point at ε=10−6\varepsilon= 10^{-6} and then on the much more severe anomaly for the runaway starting point at ε=10−12\varepsilon= 10^{-12}.

Numerical Discontinuities in Likelihood Evaluation and Their Origin

To investigate the numerical behavior of the likelihood, a one-dimensional scan of the X(6900) mass, m2m_2, is first examined as an illustrative example. For the non-converged fit obtained from the reference starting point using RooIntegrator1D at ε=10−6\varepsilon= 10^{-6} (Table 2), all parameters are fixed to the values returned by that fit except m2m_2, which is scanned over 6.8–6.9 GeV in steps of 10−410^{-4} GeV. The scan exhibits pronounced discontinuities in the NLL (Figure 3(a)). To investigate their origin, we examine the numerical evaluation of the likelihood and its individual components in the vicinity of these discontinuities.

Figure 3. Numerical discontinuities and integrator stopping behavior in the m2m_2 scan for the reference starting point, with the other 16 fit parameters fixed at the values obtained from the fit using RooIntegrator1D (RI1D) at ε=10−6\varepsilon= 10^{-6}. (a) NLL scans comparing RI1D with tolerances of 10−610^{-6} and 10−810^{-8}, and adaptive Gauss–Kronrod (GK) integration with a tolerance of 10−610^{-6}. (b) Enlarged view of the largest NLL discontinuity (ΔNLL=−22.7\Delta\mathrm{NLL} = -22.7) obtained with RI1D at ε=10−6\varepsilon= 10^{-6}. (c) Corresponding signal normalization integral returned by RI1D, I^\hat{I}, together with the integral estimates E7E_7 and E10E_{10} obtained at refinement levels n=7n = 7 and n=10n = 10, respectively. RI1D evaluates the integral using the trapezoidal rule and progressively refines the integration grid by doubling the number of equal-width subintervals at each level, giving 2n−12^{n-1} subintervals at level nn. Here, EnE_n denotes the integral estimate obtained through Neville extrapolation, and n∗n^* is the level at which the integrator stops. (d) Numerical error estimates ∣δ7∣|\delta_7| and ∣δ10∣|\delta_{10}| for the signal normalization integral at refinement levels n=7n = 7 and n=10n = 10, respectively, compared with the integration tolerance ε=10−6\varepsilon= 10^{-6}. Panels (b)–(d) share the same m2m_2 range. The vertical dotted lines indicate the interval over which the integrator stopping level switches from n∗=10n^* = 10 to n∗=7n^* = 7 at a tolerance of ε=10−6\varepsilon= 10^{-6}.

The largest NLL jump occurs between m2=6.8494m_2 = 6.8494 and 6.84956.8495 GeV, where ΔNLL=−22.7\Delta\mathrm{NLL} = -22.7 (Figure 3(b)),

compared with changes of approximately 0.1–0.4 in the NLL over a 10−410^{-4} GeV step in m2m_2 away from the discontinuities. Table 3 presents the NLL and the signal normalization integral IsigI_{\mathrm{sig}} around this discontinuity. As can be seen from the table, the jump is accompanied by an abrupt decrease of approximately 0.91% in IsigI_{\mathrm{sig}}, compared with changes of only about 0.01–0.02% between other neighboring scan points. Although IsigI_{\mathrm{sig}} changes by less than 1%, its effect on the NLL is amplified by the presence of 2541 signal events at this scan point. Varying only IsigI_{\mathrm{sig}} between the two adjacent scan points m2=6.8494m_2 = 6.8494 and 6.8495 GeV, while holding the unnormalized densities fixed, yields ΔNLL=−23.3\Delta\mathrm{NLL} = -23.3. The remaining contribution from the change in the unnormalized signal density is +0.6, giving a total NLL change of −22.7, in agreement with the observed jump. The normalization integrals of the DPS, NRSPS, and BW0_0 components remain unchanged, as expected because these components do not depend on m2m_2.

m2m_2 [GeV]NLLIsigI_{\mathrm{sig}}ΔNLL\Delta\mathrm{NLL}ΔIsig\Delta I_{\mathrm{sig}}ΔIsig/Isig\Delta I_{\mathrm{sig}}/I_{\mathrm{sig}}
6.8493−93064.60.21085———
6.8494−93064.50.210810.1−0.00004−0.02%
6.8495−93087.20.20890−22.7−0.00191−0.91%
6.8496−93086.80.208880.4−0.00002−0.01%
6.8497−93086.40.208860.4−0.00002−0.01%
6.8498−93086.00.208830.4−0.00002−0.01%
6.8499−93063.80.21061+22.2+0.00177+0.85%
6.8500−93063.70.210560.1−0.00004−0.02%

Table 3. NLL and signal normalization integral (IsigI_{\mathrm{sig}}) evaluated in the m2m_2 scan around the largest NLL discontinuity, using RooIntegrator1D at ε=10−6\varepsilon= 10^{-6} with all other fit parameters held fixed. The differences ΔNLL\Delta\mathrm{NLL} and ΔIsig\Delta I_{\mathrm{sig}} are calculated relative to the preceding scan point, and ΔIsig/Isig\Delta I_{\mathrm{sig}}/I_{\mathrm{sig}} is the corresponding relative change with respect to IsigI_{\mathrm{sig}} at that preceding point, reported as a percentage. The pairs of adjacent scan points spanning the two NLL discontinuities are highlighted in bold, together with the corresponding large changes in ΔNLL\Delta\mathrm{NLL}, ΔIsig\Delta I_{\mathrm{sig}}, and ΔIsig/Isig\Delta I_{\mathrm{sig}}/I_{\mathrm{sig}}.

To understand why the numerical normalization changes abruptly as m2m_2 varies, we examine how RooIntegrator1D determines when to terminate numerical integration. To evaluate an integral I=∫abf(x) dxI = \int_a^b f(x)\,\mathrm{d}x, RooIntegrator1D initially applies the trapezoidal rule over the full integration range [a,b][a,b] using a single interval, giving T1=(b−a)[f(a)+f(b)]/2T_1 = (b-a)[f(a)+f(b)]/2. At each successive refinement level nn, the number of equal-width subintervals is doubled, yielding a trapezoidal estimate TnT_n based on 2n−12^{n-1} subintervals [15]:

Tn=Tn−12+hn∑k=12n−2f(a+(2k−1)hn),hn=b−a2n−1,n≥2.T_n = \frac{T_{n-1}}{2} + h_n \sum_{k=1}^{2^{n-2}} f\left(a + (2k-1)h_n\right), \qquad h_n = \frac{b-a}{2^{n-1}}, \qquad n \ge2.

As the grid spacing approaches zero, the trapezoidal estimates converge to the integral, I=lim⁡n→∞TnI = \lim_{n\to\infty} T_n, and in practice this limit is estimated by extrapolating a finite number of estimates. Once the first five estimates, T1,…,T5T_1,\ldots,T_5, have been obtained, Neville’s algorithm [16] is used to extrapolate these estimates to the limit of vanishing grid spacing. At each level n≥5n \ge5, the five most recent trapezoidal estimates are used to obtain an extrapolated estimate of the normalization integral EnE_n. The difference between the extrapolation using the four most recent estimates and that using all five, denoted δn\delta_n, serves as an internal error estimate. Refinement continues until the estimated error ∣δn∣|\delta_n| falls below either the absolute tolerance εabs\varepsilon_{\mathrm{abs}} or the relative tolerance εrel∣En∣\varepsilon_{\mathrm{rel}}|E_n|,

∣δn∣≤max⁡(εabs,εrel∣En∣),|\delta_n| \le\max\left(\varepsilon_{\mathrm{abs}},\varepsilon_{\mathrm{rel}}|E_n|\right),

or until the maximum refinement level is reached. Once either tolerance is satisfied at level n∗n^*, the integrator stops and returns the extrapolated estimate En∗E_{n^*}. If the predefined maximum refinement level nmax⁡n_{\max} is reached without convergence, the integrator reports non-convergence and returns the final trapezoidal estimate Tnmax⁡T_{n_{\max}} [14]. The RooFit default configuration sets the absolute and relative integration tolerances equal, with εabs=εrel≡ε=10−7\varepsilon_{\mathrm{abs}} = \varepsilon_{\mathrm{rel}} \equiv\varepsilon= 10^{-7}. We retain this convention and vary ε\varepsilon to study the dependence on integration precision. Since δn\delta_n measures the last extrapolation correction rather than the deviation from the exact integral, a small ∣δn∣|\delta_n| does not necessarily guarantee an accurate result.

At the largest NLL discontinuity (Figure 3(b)), the stopping level changes from n∗=10n^* = 10 to n∗=7n^* = 7 between m2=6.8494m_2 = 6.8494 and 6.8495 GeV. At the first point, the integration proceeds to level n=10n = 10 and returns the extrapolated estimate E10E_{10}. At the neighboring point, the internal error estimate ∣δ7∣|\delta_7| falls below the stopping threshold of 10−610^{-6} (Figure 3(d)), causing the integrator to terminate at level n=7n = 7 and return E7E_7 instead. The switch between E10E_{10} and E7E_7 introduces a discontinuity in the returned normalization integral, as shown in Figure 3(c). When ∣δ7∣|\delta_7| rises above the stopping threshold again, the stopping level switches back to n∗=10n^* = 10, producing the second discontinuity near m2=6.8499m_2 = 6.8499 GeV.

Except for the discontinuity between m2=6.8494m_2 = 6.8494 and 6.8495 GeV, there are four more discontinuities in the NLL for the m2m_2 scan, as shown in Figure 3(a) and summarized in Table 4. All five NLL discontinuities coincide with a change in the refinement level n∗n^* at which RooIntegrator1D terminates. To examine these stopping-level changes, Figure 4(a) shows the estimated integration errors ∣δn∣|\delta_n| at different refinement levels across the m2m_2 scan, while Figure 4(b) provides enlarged views near the stopping-level transitions. At these transitions, an error estimate crosses the stopping threshold, changing the first refinement level that satisfies the stopping criterion. This causes the integrator to return a different normalization estimate, producing the corresponding NLL discontinuity.

m2m_2 transition [GeV]n∗n^* transitionΔNLL\Delta\mathrm{NLL}ΔIsig\Delta I_{\mathrm{sig}}
6.8115 → 6.81169 → 10−2.2−2.75×10−4-2.75 \times10^{-4}
6.8494 → 6.849510 → 7−22.7−1.91×10−3-1.91 \times10^{-3}
6.8498 → 6.84997 → 10+22.2+1.77×10−3+1.77 \times10^{-3}
6.8575 → 6.857610 → 8−1.0−1.32×10−4-1.32 \times10^{-4}
6.8606 → 6.86078 → 9+2.8+1.87×10−4+1.87 \times10^{-4}

Table 4. Changes in the NLL and signal normalization integral (IsigI_{\mathrm{sig}}), together with the corresponding stopping-level transitions, across the five discontinuities identified in the m2m_2 scan in Figure 3(a), using RooIntegrator1D at ε=10−6\varepsilon= 10^{-6} with all other fit parameters held fixed. The m2m_2 transition column gives the two adjacent scan points spanning each discontinuity, while the n∗n^* transition indicates the corresponding change in the integration stopping level. The differences ΔNLL\Delta\mathrm{NLL} and ΔIsig\Delta I_{\mathrm{sig}} are calculated by subtracting the values at the first scan point from those at the second.

Numerical error estimates and stopping-level transitions in the $m_2$ scan

Figure 4. Numerical error estimates and stopping-level transitions in the m2m_2 scan using RooIntegrator1D at ε=10−6\varepsilon= 10^{-6}. (a) Magnitudes of the Neville extrapolation corrections, ∣δn∣|\delta_n|, for refinement levels n=6n = 6–1212. The horizontal dashed line indicates the integration stopping threshold. (b) Enlarged views of the error estimates near the stopping-level transitions associated with the NLL discontinuities in Figure 3(a). The solid curves show (i) ∣δ9∣|\delta_9|, (ii) ∣δ7∣|\delta_7|, (iii) ∣δ8∣|\delta_8|, and (iv) ∣δ8∣|\delta_8|, while the horizontal dashed lines indicate the stopping threshold. The shaded regions distinguish different stopping levels n∗n^*, with transitions of (i) 9→109 \to10, (ii) 10→7→1010 \to7 \to10, (iii) 10→810 \to8, and (iv) 8→98 \to9. These transitions occur where the corresponding error estimates cross the stopping threshold.

Although the m2m_2 scan reveals how numerical integration can introduce artificial discontinuities in the NLL, the largest jump occurs approximately 24 MeV away from the value m2=6.8252m_2 = 6.8252 GeV at which the minimizer stopped, where the NLL is locally smooth along this parameter direction. The role of this discontinuity in the convergence failure is therefore not immediately clear. To investigate this further, we examine the NLL as a function of the other floating parameters while keeping the remaining parameters fixed at their returned values. Among these scans, a numerical step is observed along the NRSPS shape parameter p2p_2, close to its returned value (Figure 5(a) and Figure 5(b)). We therefore examine this step in detail to assess its role in the convergence failure.

Figure 5. Numerical behavior of the likelihood as a function of the NRSPS shape parameter p2p_2, with the other 16 fit parameters fixed at the values returned by the non-converged fit using RooIntegrator1D (RI1D) at ε=10−6\varepsilon= 10^{-6}. (a) NLL as a function of p2p_2 over the full range, comparing RI1D at ε=10−6\varepsilon= 10^{-6} (red solid) with adaptive Gauss–Kronrod (GK) integration at ε=10−8\varepsilon= 10^{-8} (blue dashed). The red box indicates the region enlarged in panels (b) and (c). (b) Enlarged NLL scan using RI1D at ε=10−6\varepsilon= 10^{-6}, showing a numerical discontinuity of ΔNLL=−1.7\Delta\mathrm{NLL} = -1.7 near the returned fit point.(c) Corresponding NRSPS normalization integrals evaluated using RI1D (red solid) and adaptive GK (blue dashed) with the same respective tolerances as in panel (a). The red dotted and black dashed vertical lines mark the numerical discontinuity at p2=0.128263p_2 = 0.128263 obtained with RI1D at ε=10−6\varepsilon= 10^{-6} and the corresponding returned fit value, respectively.

The discontinuity occurs near p2=0.128263p_2 = 0.128263, just below the returned value of p2=0.128285p_2 = 0.128285, and is accompanied by a corresponding jump in the NRSPS normalization (Figure 5(b) and Figure 5(c)). Between two adjacent scan points, p2=0.128262p_2 = 0.128262 and 0.128263, the NLL decreases by approximately 1.7, an unphysically large change for so fine a step in p2p_2, and the returned NRSPS normalization decreases by approximately 0.021%. This discontinuity arises from a change in the RooIntegrator1D stopping level from n∗=11n^* = 11 to n∗=10n^* = 10 between these two points. In the fit, MIGRAD performed approximately 2730 function evaluations, moving p2p_2 from its initial value of 0.144 to 0.128285, but failed to converge. At termination, the numerical first derivative associated with p2p_2 was approximately −8×103-8 \times10^3, whereas those associated with the other 16 floating parameters were of order unity or smaller. This derivative is consistent with a finite-difference evaluation crossing the NLL discontinuity of approximately 1.7 and is much larger in magnitude than the local slope of the otherwise smooth NLL curve. MIGRAD failed to find a further decrease in the evaluated NLL and returned this point with an EDM of 5.0×1025.0 \times10^2. The subsequent HESSE calculation failed to obtain a positive-definite error matrix. The large EDM and the absence of an accurate covariance matrix both fall short of the convergence criteria defined in § 3, indicating that the fit had not reached a trustworthy minimum.

To investigate the role of the numerical discontinuity in the convergence failure, additional minimization tests were performed. Starting from the returned fit point, both a fit with only p2p_2 floating and a full 17-parameter fit failed to move away from that point using RooIntegrator1D at ε=10−6\varepsilon= 10^{-6}. In contrast, repeating the full fit from the original initial values with the RooIntegrator1D refinement level fixed at n=11n = 11 restored convergence, with successful completion of the minimization and covariance-matrix evaluation (status = 0), an accurate covariance matrix (covQual = 3), and an estimated further decrease in the NLL below the 10−310^{-3} threshold (EDM = 7.3×10−57.3 \times10^{-5}). These results indicate that the integration stopping-level switching was responsible for the observed convergence failure.

To mitigate these numerical discontinuities in the NLL, one approach is to tighten the integration tolerance ε\varepsilon. Alternatively, the integration can be improved by using a different numerical integration method. Since the accuracy of an integral depends on how the integration domain is sampled, adaptive Gauss–Kronrod quadrature [17], for example, uses two numerical approximations of the same integral to estimate local integration errors and selectively refine subintervals, in contrast to the uniform refinement used by RooIntegrator1D. Both approaches yield smooth NLL curves in the m2m_2 scan, as shown in Figure 3(a). The discontinuity near the returned p2p_2 value is also absent with adaptive Gauss–Kronrod integration at ε=10−8\varepsilon= 10^{-8} (Figure 5). To assess the impact on minimization, the full 17-parameter fit is repeated from the same initial val-

ues using adaptive Gauss–Kronrod integration at ε=10−8\varepsilon= 10^{-8}. The fit converges with status = 0, covQual = 3, and EDM = 1.6×10−41.6 \times10^{-4}, reaching minNLL = −93079.8-93079.8, consistent with the reference value obtained in § 3. The recovery of convergence supports the interpretation that numerical discontinuities in the PDF normalization, and the resulting artificially lower NLL, contributed to the original minimization failure.

Severe Numerical Integration Errors from Unresolved Narrow Structures

Mechanism of Normalization Underestimation in the Presence of Narrow Structures

As shown in Table 2, repeating the fit from the same runaway starting point with progressively tighter integration tolerances leads to markedly different results. At ε=10−8\varepsilon= 10^{-8} and 10−1010^{-10}, the fits return minNLL values near −93079.8-93079.8, consistent with the reference solutions. However, further tightening the tolerance to ε=10−12\varepsilon= 10^{-12} produces a non-converged fit with an anomalously low minNLL of −119890.2-119890.2. In this fit, the NRSPS shape parameter p2p_2 decreases to 4.33×10−64.33 \times10^{-6}, much smaller than the input value of 0.091, while the NRSPS yield increases to approximately 9.0×1049.0 \times10^{4}, which is even larger than the total input yield of 14049. These extreme values are clearly not a physically preferred description of the spectrum.

To investigate this behavior, we examine the NLL as a function of the floating parameters while keeping the remaining parameters fixed at the values returned by the runaway fit. Unlike the other parameter scans, the p2p_2 scan reveals a pronounced decrease in the NLL toward very small p2p_2 values (Figure 6(a)), accompanied by a strong suppression of the NRSPS normalization integral (Figure 6(b)). Both RooIntegrator1D and adaptive Gauss–Kronrod integration exhibit this behavior, although the suppression is substantially more severe with RooIntegrator1D. Discontinuities are also observed in the NLL and normalization curves. For RooIntegrator1D, the integrator first checks whether the estimated integration error satisfies the prescribed tolerance at refinement level n=5n = 5, using ∣δ5∣|\delta_5| obtained from the extrapolation of successive trapezoidal approximations. As p2p_2 varies, ∣δ5∣|\delta_5| crosses the integration tolerance (Figure 6(c)). On one side of the crossing, the integrator terminates at n=5n = 5, while on the other side, it continues to higher refinement levels. The resulting difference in the returned normalization estimates produces the observed discontinuities in the normalization and NLL. Similar discontinuities in adaptive Gauss–Kronrod integration also arise from changes in the integration stopping decision. These stopping transitions explain the observed discontinuities, but not why the NRSPS normalization becomes extremely small at low p2p_2.

Figure 6. Numerical behavior of the NRSPS normalization and likelihood as a function of p2p_2. (a) NLL as a function of p2p_2, with the other 16 fit parameters fixed at the values returned by the runaway fit at ε=10−12\varepsilon= 10^{-12}. Results are shown for RooIntegrator1D (RI1D) at ε=10−10\varepsilon= 10^{-10}, 10−1210^{-12}, and 10−1410^{-14}, and adaptive Gauss–Kronrod (GK) integration at ε=10−10\varepsilon= 10^{-10}. (b) Corresponding NRSPS normalization integral INRSPSI_{\mathrm{NRSPS}} evaluated with the same integration settings. The vertical dotted lines in (a) and (b) mark the RI1D stopping transitions at ε=10−14\varepsilon= 10^{-14}, 10−1210^{-12}, and 10−1010^{-10}, with the corresponding p2p_2 values annotated. (c) Estimated integration error ∣δ5∣\lvert\delta_5\rvert at refinement level n=5n = 5 as a function of p2p_2. The intersections of the ∣δ5∣\lvert\delta_5\rvert curve with the horizontal dashed lines mark the stopping-transition positions for the three integration tolerances. (d) NRSPS mass distributions for different p2p_2 values, each normalized to its maximum. The vertical dashed line indicates the J/ψJ/ψJ/\psi J/\psi threshold at mth=6.1938m_{\mathrm{th}} = 6.1938 GeV. The distributions become increasingly narrow near threshold as p2p_2 decreases.

The severe underestimation of the NRSPS normalization originates from the increasingly narrow distribution near the J/ψ/ψJ/\psi/\psi threshold. As shown in Figure 6(d), decreasing p2p_2 concentrates the distribution into a progressively smaller mass interval above the threshold. At refinement level n=5n = 5, RooIntegrator1D divides the integration range of 6–15 GeV into 16 equal subintervals. The first sampling point above the J/ψ/ψJ/\psi/\psi threshold is therefore at 6.5625 GeV, approximately 369 MeV above threshold, whereas the NRSPS peak lies only 2.5 MeV above threshold at p2≃3.33×10−4p_2 \simeq3.33 \times10^{-4}. The integration grid therefore misses the region that provides most of the integral and underestimates the true integral by many orders of magnitude. Since the estimated integration errors are calculated from the sampled function values, the errors may remain small even when the integration grid fails to resolve the dominant contribution, causing the integrator to terminate prematurely. As a result, tightening the integration tolerance does not necessarily prevent this failure. The same limitation can affect other numerical integration methods, including adaptive Gauss–Kronrod quadrature, whose error estimates also depend on sampled function values.

We further investigate how the minimizer moves from its initial value of p2=0.123p_2 = 0.123, bypasses the expected minimum near p2≃0.107p_2 \simeq0.107, and eventually enters the anomalous small-p2p_2 region. As shown in Table 2, the fit from the same runaway starting point first fails to converge at looser tolerances. After the integration tolerance is tightened, the fits at ε=10−8\varepsilon= 10^{-8} and 10−1010^{-10} return minNLL values near −93079.8-93079.8, consistent with the reference solution, whereas the fit at ε=10−12\varepsilon= 10^{-12} returns a much lower value of −119890.2-119890.2. To understand why the two integration tolerances lead to different fit outcomes, we examine how MIGRAD [10] explores

the parameter space. During minimization, MIGRAD uses changes in the NLL to determine the direction in which the parameters should move and then evaluates the NLL at different distances along that direction to find a parameter point with a lower NLL. To keep p2p_2 within its specified range of [0,1][0,1], Minuit employs an internal parameter uu for minimization, which is mapped to p2p_2 through p2=(1+sin⁡u)/2p_2 = (1 + \sin u)/2, following the Minuit standard transformation for parameters with finite lower and upper bounds. Because MIGRAD performs a line search in the internal parameter space, the nonlinear and periodic sine transformation allows a finite step in uu to reach a very small p2p_2 without evaluating the NLL at intermediate p2p_2 values between its current and new values, including intermediate values near the p2p_2 value of the normally converged fit.

For the fits from the same runaway starting point at ε=10−10\varepsilon= 10^{-10} and ε=10−12\varepsilon= 10^{-12}, with only the former converging, the change in integration tolerance produces small differences in the evaluated NLL, typically of order 10−310^{-3} when the parameter values are nearly identical. These differences can affect the estimated gradients and consequently alter the search directions and the subsequent parameter points evaluated by MIGRAD. Although these two fits reach nearly identical parameter points, with p2≃0.123p_2 \simeq0.123, at the start of a line search, their subsequent evaluations probe different points in the 17-dimensional parameter space, with the trial values of p2p_2 differing by a factor of approximately 2.7 and the other fit parameters also taking different values. At ε=10−12\varepsilon= 10^{-12}, severe underestimation of the NRSPS normalization at the trial point produces an artificially low NLL of approximately −119890-119890, which is accepted as an improvement because the underestimated normalization inflates the NRSPS PDF contribution to the likelihood. In contrast, at ε=10−10\varepsilon= 10^{-10}, the trial at a different small-p2p_2 point gives an NLL of approximately −88994-88994, about 4055 higher than the lowest NLL previously found during that fit, and is therefore rejected. The minimizer subsequently returns to the vicinity of the normally converged region and eventually converges to the reference solution at p2≃0.107p_2 \simeq0.107. The different outcomes of these two fits show that changes in the integration tolerance can introduce small differences in the evaluated NLL, which in turn affect the parameter-space search performed by MIGRAD.

Improving Numerical Integration of Narrow Structures through Variable Transformations

The numerical failure identified above arises because the integration grid does not adequately resolve the narrow NRSPS contribution near threshold. To improve the sampling of that narrow region, we apply a Möbius variable transformation [18] to concentrate numerical sampling points near this region, without changing the NRSPS function or the likelihood definition.

The integration variable mm is transformed to t∈[0,1]t \in[0,1] through

m(t)=mth+Ltλ+(1−λ)t,(5)m(t) = m_{\mathrm{th}} + \frac{L t}{\lambda+ (1-\lambda)t}, \tag*{(5)}

where mth=2mJ/ψm_{\mathrm{th}} = 2m_{J/\psi} is the J/ψJ/ψJ/\psi J/\psi threshold, L=mmax⁡−mthL = m_{\max} - m_{\mathrm{th}} with mmax⁡=15m_{\max} = 15 GeV and λ=L/p2\lambda= L/p_2. This choice increases the sampling density near threshold as p2p_2 decreases. The NRSPS normalization integral then becomes

INRSPS=∫01fNRSPS(m(t))Lλ[λ+(1−λ)t]2 dt.(6)I_{\mathrm{NRSPS}} = \int_0^1 f_{\mathrm{NRSPS}}(m(t)) \frac{L\lambda}{[\lambda+ (1-\lambda)t]^2}\,dt. \tag*{(6)}

For small p2p_2, the transformation concentrates sampling points near threshold, where the NRSPS contribution is concentrated. Figure 7(a) compares the NRSPS normalization integrals obtained using the original RooIntegrator1D with ε=10−12\varepsilon= 10^{-12}, Möbius-transformed RooIntegrator1D with ε=10−12\varepsilon= 10^{-12}, and Möbius-transformed adaptive Gauss–Kronrod integration with ε=10−10\varepsilon= 10^{-10}. At p2=4.33×10−6p_2 = 4.33 \times10^{-6}, the value returned by the non-converged fit (Tab. 2 runaway case), the Möbius-transformed calculation gives INRSPS≈2.77×10−8I_{\mathrm{NRSPS}} \approx2.77 \times10^{-8}, compared with 3.05×10−1003.05 \times10^{-100} from the original RooIntegrator1D. The two transformed integration methods give consistent results across the scanned p2p_2 range, supporting the reliability of the numerical integration with suitable variable transformation and eliminating the severe underestimation observed with the original method. The corresponding NLL scans in Figure 7(b) are also smooth after the transformation, without the anomalously low NLL values at small p2p_2.

Figure 7. Improved evaluation of the NRSPS normalization and the likelihood using the Möbius variable transformation. (a) NRSPS normalization integral as a function of p2p_2, comparing the original RooIntegrator1D (RI1D) with ε=10−12\varepsilon= 10^{-12}, Möbius-transformed RI1D with ε=10−12\varepsilon= 10^{-12}, and Möbius-transformed adaptive Gauss–Kronrod integration with ε=10−10\varepsilon= 10^{-10}. (b) Corresponding NLL scans for p2p_2 using the same three integration configurations, with the other 16 fit parameters fixed at the values returned by the non-converged fit using the original RI1D at ε=10−12\varepsilon= 10^{-12} (the runaway case in Table 2). (c) NLL scan for p2p_2 around the corrected best-fit solution using Möbius-transformed adaptive Gauss–Kronrod integration with ε=10−10\varepsilon= 10^{-10}, with the other 16 parameters fixed at their corrected best-fit values. The green marker indicates the minimum of the scan. The variable transformation removes the severe underestimation of the normalization integral and the associated anomalously low NLL values observed with the original integration method.

The full 17-parameter fit is then repeated from the same initial parameter values in the runaway case, using Möbius-transformed adaptive Gauss–Kronrod integration with ε=10−10\varepsilon= 10^{-10} for the NRSPS normalization. The pseudo-dataset and all other aspects of the fit remain unchanged. The fit converges with minNLL ≈−93079.8\approx-93079.8 and p2=0.107±0.010p_2 = 0.107 \pm0.010, consistent with the reference solution in Table 2. The corresponding NLL scan for p2p_2 exhibits a smooth minimum, as shown in Figure 7(c). As an independent check, we also test the alternative transformation m=mth+p2z2m = m_{\mathrm{th}} + p_2 z^2, where z∈[0,L/p2]z \in[0,\sqrt{L/p_2}], a simple quadratic map that likewise concentrates sampling near threshold. Using adaptive Gauss–Kronrod integration with ε=10−10\varepsilon= 10^{-10}, this transformation also recovers the reference likelihood minimum. These results show that improving the numerical sampling near threshold can stabilize the NRSPS normalization and restore convergence of the full likelihood fit without modifying the underlying mass-spectrum model.

Robustness and Discussion

The two numerical failures examined in § 4 and § 5 illustrate how inaccurate numerical evaluation of the PDF normalization can distort the likelihood. In the first case, the integrator’s decision to stop or continue can change between neighboring parameter points, producing discontinuities in the normalization and hence the NLL, even when the underlying physical density is smooth, which can stall MIGRAD at a spuriously low NLL caused by the incorrect numerical integral. In the second case, the NRSPS distribution becomes increasingly concentrated near the J/ψ/ψJ/\psi/\psi threshold as p2p_2 decreases. The integration grid may then fail to resolve the narrow peak, leading to a severely underestimated normalization and an artificially low-NLL region into which the minimization can be drawn.

To evaluate the robustness of the improved numerical integration strategies discussed above—tightening the integration tolerances, adopting an alternative adaptive integration algorithm (using Gauss–Kronrod integration as an example), and applying a suitable variable transformation (as demonstrated using the Möbius transformation for the NRSPS integral in § 5.2)—we repeat the 500 fits for each of the three strategies, using the same randomly generated initial parameter values as in the original study (Figure 2). The performance of the three strategies is summarized in Table 5, including their effectiveness in recovering anomalously low NLL values and improving fit convergence. Tightening the integration tolerances in RooIntegrator1D from ε=10−7\varepsilon= 10^{-7} to 10−810^{-8}, 10−1010^{-10}, or 10−1210^{-12} recovers nonnegative ΔNLL\Delta\mathrm{NLL} values for 86% of the 69 fits originally exhibiting anomalously low NLL values with ΔNLL<0\Delta\mathrm{NLL} < 0, where ΔNLL=NLL−NLLG\Delta\mathrm{NLL} = \mathrm{NLL} - \mathrm{NLL}_{G} and GG denotes the reference global minimum. Among the 390 fits that did not reach GG with RooIntegrator1D at ε=10−7\varepsilon= 10^{-7}, 54% satisfy the convergence criteria and 53% converge to GG with these tighter tolerances. Adaptive Gauss–Kronrod integration at ε=10−7\varepsilon= 10^{-7} yields the corresponding anomalous-NLL recovery rate of 96%, the convergence rate of 81%, and the fraction converging to GG of 78%. Further applying the Möbius transformation to the NRSPS integral with adaptive Gauss–Kronrod eliminates the remaining anomalously low-NLL cases and achieves convergence for 18% of the fits that did not converge with adaptive Gauss–Kronrod integration alone, with 14% of those that did not reach GG under adaptive Gauss–Kronrod integration now converging to GG. Combining the results from the three strategies eliminates all anomalously low NLL values and achieves an overall convergence rate of 87% among the 390 fits that did not converge to GG with the original configuration, with 84% converging to GG.

MethodAnomalous-NLL recoveryConvergedConverged to GG
(Nok/NtestN_{\rm ok}/N_{\rm test}) (%)(Nok/NtestN_{\rm ok}/N_{\rm test}) (%)(Nok/NtestN_{\rm ok}/N_{\rm test}) (%)
Default RI1D ε=10−7\varepsilon= 10^{-7}0/69 (0%)112/500 (22%)110/500 (22%)
Tighter RI1D59/69 (86%)212/390 (54%)206/390 (53%)
Adaptive GK ε=10−7\varepsilon= 10^{-7}66/69 (96%)314/390 (81%)306/390 (78%)
Möbius NRSPS + adaptive GK ε=10−7\varepsilon= 10^{-7}3/3 (100%)14/76 (18%)12/84 (14%)
Combined (best result per fit)69/69 (100%)339/390 (87%)327/390 (84%)

Table 5. Robustness of three numerical integration strategies using the same 500 sets of randomly generated initial parameter values as in Figure 2. The strategies include tightening the integration tolerances with RooIntegrator1D (RI1D) at ε=10−8\varepsilon= 10^{-8}, 10−1010^{-10}, 10−1210^{-12}, adopting adaptive Gauss–Kronrod (adaptive GK) integration at ε=10−7\varepsilon= 10^{-7}, and applying the Möbius transformation to the NRSPS integral with adaptive GK integration at ε=10−7\varepsilon= 10^{-7}. These strategies are evaluated on the 390 fits that did not converge to the reference global minimum (GG) using RI1D at ε=10−7\varepsilon= 10^{-7}. The three performance metrics are defined as follows. “Anomalous-NLL recovery” denotes the fraction of the 69 original fits with ΔNLL<0\Delta\mathrm{NLL} < 0 for which ΔNLL≥0\Delta\mathrm{NLL} \ge0 is recovered, where ΔNLL=NLL−NLLG\Delta\mathrm{NLL} = \mathrm{NLL} - \mathrm{NLL}_{G} and NLLG\mathrm{NLL}_{G} is the NLL at GG. “Converged” requires status = 0, covQual = 3, and EDM <10−3< 10^{-3}, while “Converged to GG” additionally requires reaching the reference minimum. Each entry is given as (Nok/NtestN_{\rm ok}/N_{\rm test}) (%), where NokN_{\rm ok} and NtestN_{\rm test} denote the numbers of fits satisfying the corresponding criterion and considered for that metric, respectively, with the percentage given in parentheses. The Möbius transformation is applied only to cases unresolved by adaptive GK, where NtestN_{\rm test} counts the fits still failing that metric after adaptive GK. “Combined (best result per fit)” represents the combined performance, selecting one result per fit by prioritizing convergence to GG, followed by convergence and smallest nonnegative ΔNLL\Delta\mathrm{NLL}.

Figure 8 further compares the original ΔNLL\Delta\mathrm{NLL} distribution with that obtained by selecting one result for each of the 500 initial-parameter sets from the three strategies. For each set, the fit results obtained using tighter RooIntegrator1D tolerances, adaptive Gauss–Kronrod integration, and Möbius-transformed adaptive Gauss–Kronrod integration are compared. Priority is given to results converging to the reference minimum GG. If no result reaches GG, a result satisfying the convergence criteria is selected. Otherwise, the result with the smallest nonnegative ΔNLL\Delta\mathrm{NLL} is retained. The histograms are stacked with different colors indicating the selected strategies, with each fit assigned the same color in both panels. As shown in Figure 8, the resulting distribution no longer contains the anomalously low NLL values below the reference global minimum. Together with the individual recovery rates in Table 5, these results demonstrate that the numerical integration strategies examined here effectively suppress spurious low-NLL solutions arising from numerical integration errors and improve the robustness of the fitting procedure.

Figure 8. Comparison of the ΔNLL\Delta\mathrm{NLL} distributions for the 500 fits obtained with (a) the original RooIntegrator1D (RI1D) at ε=10−7\varepsilon= 10^{-7} and (b) the selected results from the three improved numerical integration strategies: tighter RI1D tolerances with ε=10−8/10−10/10−12\varepsilon= 10^{-8}/10^{-10}/10^{-12} (orange), adaptive Gauss–Kronrod (GK) integration (green), and Möbius-transformed GK integration of the NRSPS integral (purple). Here, ΔNLL=NLLfit−NLLG\Delta\mathrm{NLL} = \mathrm{NLL}_{\mathrm{fit}} - \mathrm{NLL}_{G}, where NLLG\mathrm{NLL}_{G} denotes the NLL at the reference global minimum GG (black dashed). The annotations N(ΔNLL<0)N(\Delta\mathrm{NLL} < 0) and NGN_{G} indicate the numbers of fits with negative ΔNLL\Delta\mathrm{NLL} and those converging to GG, respectively. For each fit, the ΔNLL\Delta\mathrm{NLL} value shown in panel (b) is taken from the selected result, prioritizing convergence to GG, followed by convergence, and finally the smallest nonnegative ΔNLL\Delta\mathrm{NLL}. Fits already converged to GG with the original method are shown in gray. The histograms are stacked with different colors indicating the selected strategies, with each fit assigned the same color in both panels.

The remaining convergence failures are associated with the strong correlations among fit parameters, local minima, and numerical integration errors. Strong parameter correlations can allow changes in one parameter to be compensated by changes in others with little variation in the NLL, while numerical integration errors can affect the estimation of gradients and curvature. These effects can hinder gradient-based minimization, which may become trapped in local minima or fail to converge. A search using the SIMPLEX algorithm in Minuit [10] can explore alternative regions of the parameter space by comparing NLL values at different points, without relying on gradient or curvature estimates. This makes the search less sensitive to unreliable local derivative estimates and can provide a better starting point for subsequent refinement with MIGRAD. Additional improvements can be achieved by adjusting parameter bounds to limit excursions along weakly constrained directions, where large changes in parameter values have little effect on the likelihood, and by reparameterizing strongly correlated parameters to reduce disparities in curvature across parameter directions, thereby improving the stability of the minimization. With these additional treatments, all 500 fits were successfully brought to convergence at the reference global minimum.

Although demonstrated using a specific pseudo-dataset and Minuit, the numerical difficulties identified here reflect general challenges in likelihood minimization. Extended unbinned likelihood fits relying on numerical PDF normalization, particularly for distributions that can become sharply peaked, may encounter similar problems when integration grids fail to resolve important contributions or when stopping decisions change. Practical safeguards include examining the likelihood and component normalization integrals along sensitive parameters, adjusting numerical integration tolerances, comparing independent integration algorithms, monitoring shape parameters whose values can produce sharply localized features, and applying suitable integration variable transformations where necessary.

These safeguards, however, come with practical trade-offs. Tighter integration tolerances can substantially increase computational cost. For a representative set of p2p_2 values, the average evaluation time of the NRSPS normalization integral using RooIntegrator1D increases by factors of approximately 70 and 260 when the tolerance is reduced from 10−710^{-7} to 10−1010^{-10} and 10−1210^{-12}, respectively. As normalization integrals are evaluated repeatedly during minimization, this additional cost can become substantial. Furthermore, tighter tolerances may offer little improvement in accuracy for numerically stable fits and do not necessarily resolve integration errors when sharply localized features are insufficiently sampled by the integration algorithm. The effectiveness of alternative integration algorithms and variable transformations also depends on the integrand and parameter region, while convergence difficulties associated with strong parameter correlations or local minima may require separate treatments. We therefore recommend retaining the default settings when numerical stability has been verified and applying more stringent tolerances or alternative integration methods when numerical instabilities are identified. Regardless of the numerical settings used, an anomalously low NLL accompanied by failed convergence diagnostics should be treated as a potential numerical warning rather than evidence for a better physical solution.

Conclusions

Anomalously low NLL values have been investigated using a pseudo-dataset based on the CMS J/ψJ/ψJ/\psi J/\psi analysis [3, 4, 12]. Numerical errors in PDF normalization can introduce artificial structures in the likelihood surface and lead to unreliable minimization results. In particular, changes in the integrator’s stopping decision as the fit parameters vary can produce discontinuities in the returned normalization, even at fixed integration tolerance. The problem becomes particularly severe when the PDF develops a sharply localized structure that is poorly resolved by the integration grid, allowing the numerical error estimate to indicate convergence despite a substantial normalization error. Such errors can cause the minimization algorithm (in our example, MIGRAD [10]) to encounter and retain parameter configurations with spuriously low NLL values.

Tightening the integration tolerance or using alternative adaptive numerical integration can mitigate these instabilities, but these approaches do not necessarily resolve sharply localized contributions. When evaluating PDF normalizations, an appropriate variable transformation that concentrates the sampling points in the critically sampled region can stabilize the numerical integral and restore reliable minimization. Across the 500 fits studied, these strategies eliminate all anomalously low NLL values and substantially improve convergence to the reference global minimum (Table 5). These results imply that an apparently lower NLL need not signal a better description of the data. It may instead arise from an inaccurate PDF normalization, even when the numerical integrator reports that its stopping criterion has been satisfied. In particular, non-converged fits that report unusually low NLL values should not be regarded as mathematically preferred solutions. Care is therefore required when interpreting such outcomes in fits that rely on numerically evaluated PDF normalizations.

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 12535004.

References

  1. [1]LHCb Collaboration, Sci. Bull. 65, 1983 (2020). DOI 10.1016/j.scib.2020.08.032. [arXiv:2006.16957]DOI
  2. [2]ATLAS Collaboration, Phys. Rev. Lett. 131, 151902 (2023). DOI 10.1103/PhysRevLett.131.151902. [arXiv:2304.08962]DOI
  3. [3]CMS Collaboration, Phys. Rev. Lett. 132, 111901 (2024). DOI 10.1103/PhysRevLett.132.111901. [arXiv:2306.07164]DOI
  4. [4]CMS Collaboration, Nature 648, 58 (2025). DOI 10.1038/s41586-025-09711-7. [arXiv:2506.07944]DOI
  5. [5]CMS Collaboration, (2026). Submitted to Science Advances, [arXiv:2602.02252]arxiv.org/abs/2602.02252
  6. [6]F. Zhu, G. Bauer, K. Yi, Chin. Phys. Lett. 41, 111201 (2024). DOI 10.1088/0256-307X/41/11/111201. [arXiv:2410.11210]DOI
  7. [7]H. Zhou, X. Wang, F. Zhu, L. Zhang, G. Bauer, K. Yi, Chin. Phys. Lett. 42, 120202 (2025). DOI 10.1088/0256-307X/42/12/120202. [arXiv:2511.08495]DOI
  8. [8]Y. Bai, D.Y. Chen, Phys. Rev. D 99, 072007 (2019). DOI 10.1103/PhysRevD.99.072007. [arXiv:1901.01394]DOI
  9. [9]W. Verkerke, D.P. Kirkby, eConf C0303241, MOLT007 (2003). [arXiv:physics/0306116]
  10. [10]F. James, M. Roos, Comput. Phys. Commun. 10, 343 (1975). DOI 10.1016/0010-4655(75)90039-9
  11. [11]L. Chen, Y. Chen, G. Bauer, L.G. Spiegel, Z. Hu, K. Yi, Symmetry 17, 57 (2025). DOI 10.3390/sym17010057. [arXiv:2412.20777]DOI
  12. [12]CMS Collaboration, Observation of new structures in the J/ψψ̄/ψ mass spectrum in pp collisions at √s = 13 TeV. Tech. Rep. CMS-PAS-BPH-21-003, CERN (2022). Https://cds.cern.ch/record/2815336
  13. [13]F. Takahashi, et al., Int. J. Mod. Phys. A 41, 2630011 (2026). DOI 10.1142/S0217751X26300115DOI
  14. [14]ROOT Project. ROOT: data analysis framework. https://root.cern. Accessed: 2026-11-09
  15. [15]W. Romberg, Det Kongelige Norske Videnskabers Selskab Forhandlinger 28(7), 30 (1955)
  16. [16]E.H. Neville, J. Indian Math. Soc. 20, 87 (1934)
  17. [17]M. Galassi, et al., GNU Scientific Library (Network Theory Ltd., 2019)
  18. [18]H.H.H. Homeier, E.O. Steinborn, J. Comput. Phys. 87(1), 61 (1990). DOI 10.1016/0021-9991(90)90225-PDOI

Paper details

vlnc:W_twb6x49t
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents