betaFeedback

WorksMathematics

A Modified Quasi-Newton Method for Unconstrained Optimization with Nonpositive Curvature Detection and Structured Complexity

Abstract

We consider the convergence, worst-case complexity, and implementation of R2N, a modified quasi-Newton method for large-scale unconstrained smooth optimization. At each iteration, we compute a step that approximately minimizes the sum of a quadratic model of the objective with an adaptive quadratic regularization, and that results in sufficient decrease. The objective is only assumed to be C1C^1, although second derivatives may be used if they are available. The implementation features a broad range of step-computation strategies, from sparse factorization to matrix-free Krylov methods, that satisfy global convergence requirements. When detected, directions of nonpositive curvature are exploited. Our analysis allows unbounded model Hessians, provided they satisfy a mild growth condition. We derive a worst-case evaluation complexity bound that separates the progress made along positive- and nonpositive-curvature directions. By forcing the regularization parameter to vanish near isolated local minimizers, we obtain superlinear local convergence. The implementation specializes naturally to nonlinear least-squares problems. We provide a modular Julia implementation, and use it in a numerical study on the CUTEst test set that assesses the treatment of nonpositive curvature, the subproblem solver, the update of the regularization parameter, and direct versus iterative subproblem solves. An Armijo-Goldstein line search along nonpositive-curvature directions, MINRES-QLP as subproblem solver, and a regularization parameter driven to zero near a solution form our recommended configuration. With exact second derivatives, R2N is competitive with a trust-region method, solving slightly more problems with fewer objective and gradient evaluations. With convex models, the trust-region method performs as well or better.

Farhad Rahbarnia was partially supported by Alexander Graham Bell Canada Graduate Scholarships - Doctoral number 568744-2022. Dominique Orban was partially supported by NSERC Discovery Grant RGPIN–2020–06535.

Farhad Rahbarnia GERAD and Department of Mathematics and Industrial Engineering, Polytechnique Montréal. E-mail: [email protected]

Dominique Monnet Université de Rennes, INSA Rennes, CNRS, IRMAR-UMR 6625, F-35000 Rennes, France. E-mail: [email protected]

Dominique Orban GERAD and Department of Mathematics and Industrial Engineering, Polytechnique Montréal. E-mail: [email protected]

Introduction

We consider the solution of

minimize⁡x∈Rn f(x),(1)\underset{x\in\mathbb{R}^{n}}{\operatorname{minimize}}\ f(x), \tag*{(1)}

where f:Rn→Rf:\mathbb{R}^{n}\to\mathbb{R} is C1C^{1} by way of a modified Newton or quasi-Newton method with quadratic regularization. At each iteration, a step is computed by approximately minimizing the sum of a quadratic model of ff about the current iterate and a multiple of the squared step norm. The multiple in question is a regularization parameter that is updated according to the decrease in ff induced by the step. Such methods recently received renewed attention, e.g., from Kanzow and Steck [30], who study direct solvers based on limited-memory quasi-Newton approximations, and Diouane et al. [15], who derive convergence and complexity properties under weak assumptions, even when the objective includes a nonsmooth regularizer. Quadratic regularization is related to line search methods and is a well-known alternative to trust-region methods [10]. One of its appealing features is that subproblems are unconstrained. It is famously used in the method of Levenberg [34] and Marquardt [40] for nonlinear least squares.

We propose solver R2N, whose salient features are

  • a global convergence analysis that follows from [15] under the sole assumption that f∈C1f\in C^{1}, with model Hessians allowed to grow unbounded at a controlled rate, and under a lower bound on Armijo-Goldstein step lengths when directions of nonpositive curvature are exploited;

  • a structured worst-case evaluation complexity analysis that separates positive- and nonpositive-curvature steps under Lipschitz continuity of ∇f\nabla f and ∇2f\nabla^{2}f at the iterates along the steps computed by the algorithm that corrects misleading bounds from [39]:

    • if the sequence of model Hessians {Bk}\{B_k\} remains bounded, we show that accepted steps along which curvature is positive contribute a term proportional to ε−2\varepsilon^{-2} to the total number of evaluations to bring ∥∇f(xk)∥\lVert\nabla f(x_k)\rVert below ε>0\varepsilon>0;

    • when {Bk}\{B_k\} is allowed to grow unbounded, this first bound deteriorates—see Proposition 13;

    • when Bk=∇2f(xk)B_k=\nabla^{2}f(x_k), accepted steps along which curvature is nonpositive contribute a term proportional to ε−3/2\varepsilon^{-3/2};

  • this second bound may further deteriorate when the model curvature is inexact along the step—see Theorem 2;

  • superlinear or quadratic local convergence near an isolated local minimizer for a variant in which the regularization parameter is driven to zero;

  • a modular implementation that is suited for large-scale problems, allows multiple iterative and direct subproblem solvers, and accommodates exact, limited-memory quasi-Newton, or other Hessian approximations;

  • a variant for nonlinear least squares named R2NLS that may be viewed as an implementation of the method of Levenberg [34] and Marquardt [40], to which our global convergence and complexity results apply, and our local convergence analysis applies when ∑ifi(xk)∇2fi(xk)→0\sum_i f_i(x_k)\nabla^2 f_i(x_k) \to0, in particular for zero-residual problems or when the residuals are linear;

  • a thorough numerical study, on unconstrained and nonlinear least-squares problems from the CUTEst collection, of the choices that our analysis leaves open: the treatment of nonpositive curvature, the subproblem solver, the update of the regularization parameter, and direct versus iterative subproblem solves, together with a comparison with a trust-region method.

Our current implementation supports the following iterative solvers: the truncated conjugate-gradient method of Dembo and Steihaug [12], the truncated conjugate residuals method of Dahito and Orban [11], the truncated minimum residual method of Liu and Roosta [38], where truncated refers to the identification of directions of nonpositive curvature. R2N currently supports the direct solvers MA57 [18] and MA97 [29]. When using limited-memory BFGS model Hessians, we also support the direct solver of Erway et al. [20].

One of our objectives is to settle, through a thorough numerical study, the choices that our analysis leaves open. We use exact second derivatives, limited-memory BFGS approximations and Gauss-Newton models, and compare R2N with the trust-region solver Trunk using the same model, subproblem solver and stopping test. Although R2N also supports MA97, only MA57 results are reported. Exploiting nonpositive curvature with the Armijo-Goldstein line search is more efficient and more robust than falling back on the Cauchy step, and comparable to the other strategies we consider. MINRES-QLP is more efficient than truncated CG, and driving σk\sigma_k to zero near a solution improves efficiency without affecting robustness. Exact subproblem solves, whether by sparse factorization or by the solver of Erway et al. [20], bring no consistent advantage over MINRES-QLP. With exact second derivatives, R2N solves slightly more problems than Trunk and requires fewer objective and gradient evaluations, while Trunk requires fewer Hessian-vector products. With convex models, Trunk performs as well or better.

Li et al. [35], Li and Li [36] and Polyak [51] study the properties of the regularized Newton method under the assumption that ff is convex. Ueda and Yamashita [53] further study the properties of a regularized Newton method with Armijo line search for nonconvex ff under the assumption that ff is C2C^2. At each iteration, they set the regularization parameter based on the smallest eigenvalue of the Hessian and the gradient norm, thereby ensuring that subproblems are convex. They establish a O(ε−2)O(\varepsilon^{-2}) complexity bound if ∇2f\nabla^2 f is Lipschitz continuous and superlinear convergence under a local error bound condition. Under similar assumptions, Ueda and Yamashita [54] replace the Armijo line search with a parameter update that plays a role in the definition of the regularization parameter. They establish a complexity bound in O(ε−2)O(\varepsilon^{-2}), but do not show that it is sharp. Zhang and Ni [55] combine a similar strategy with a non-monotone Armijo line search.

Kanzow and Steck [30] study a method similar to ours but rely on limited-memory quasi-Newton methods and variants of the direct method developed by Erway et al. [20] to solve the first-order stationarity conditions of the model exactly. They assume that model Hessians remain bounded, and do not exploit nonpositive-curvature directions, but merely increase the regularization parameter when they occur.

Gratton et al. [27] propose a regularized Newton method for nonconvex ff in which the shift has the form σk∥∇f(xk)∥\sqrt{\sigma_k}\lVert\nabla f(x_k)\rVert, and σk\sigma_k is updated as in R2N. When the smallest eigenvalue of the Hessian, possibly restricted to a Krylov subspace, is sufficiently negative, they take a step of prescribed length along an approximate eigenvector, and otherwise absorb the negative curvature into the shift. They establish a bound in O(∣log⁡ε∣ε−3/2)O(\lvert\log\varepsilon\rvert\varepsilon^{-3/2}) under Lipschitz continuity of the exact Hessian. By contrast, R2N uses σk\sigma_k itself as the shift, requires no eigenvalue estimate, exploits any direction of nonpositive curvature detected by the subproblem solver in a line search, and allows quasi-Newton models.

Sharp complexity bounds for the regularized (quasi-)Newton method were first established by Diouane et al. [15], even when model Hessians, whether they are exact or not, are allowed to grow unbounded. Because their focus is on nonsmooth problems, they solve subproblems with a first-order method and have no concept of curvature. We build upon their convergence analysis, which is conducted under the sole assumption that f∈C1f \in C^1, and discuss multiple subproblem solvers. We also benefit from their complexity analysis, and supplement it to emphasize the contribution of nonpositive-curvature steps to the overall complexity bound. Indeed, contrary to the references above, we allow models to be nonconvex, and we exploit nonpositive curvature wherever possible, in a manner similar to trust-region methods. Our results correct several results by Liu and Roosta [39], who study a regularized Newton method with line search and MINRES as a subproblem solver, and claim an overall O(ε−3/2)O(\varepsilon^{-3/2}) complexity bound.

Organization of the Paper

The paper is organized as follows. In § 3, we review quasi-Newton and regularization methods. In § 4, we introduce the models and the Cauchy decrease condition, and state Algorithm R2N in § 5. In § 6, we describe subproblem solvers that satisfy the Cauchy decrease condition, and our treatment of nonpositive curvature. We establish global convergence in § 7. In § 8, we derive the structured complexity bound, and show in § 9 and § 10 that its two terms are attained, at least when exact derivatives are used. In § 11, we present a variant with fast local convergence, and specialize R2N to nonlinear least squares § 12. Finally, we describe our implementation and reports numerical experiments § 13.

Notation

We use uppercase and lowercase Latin letters for matrices and vectors, respectively. Lowercase Greek letters denote scalars. The Euclidean norm of vector xx is denoted ∥x∥\lVert x\rVert. Accordingly, the spectral norm of matrix AA is ∥A∥\lVert A\rVert. Sets are denoted by uppercase calligraphic letters such as S\mathcal{S}, and their cardinality by ∣S∣\lvert\mathcal{S}\rvert. A subscript kk denotes an index or iteration number. For positive sequences {αk}\{\alpha_k\} and {βk}\{\beta_k\}, αk=O(βk)\alpha_k = O(\beta_k) means that there exists a constant C>0C > 0 such that αk≤Cβk\alpha_k \le C\beta_k for all kk large enough, and αk=o(βk)\alpha_k = o(\beta_k) means that lim⁡k→∞αk/βk=0\lim_{k\to\infty}\alpha_k/\beta_k = 0. If AA is symmetric, A≻0A \succ0 and A⪰0A \succeq0 mean that it is positive definite and positive semi-definite, respectively.

Background

Traditional quasi-Newton methods for unconstrained optimization compute steps sks_k that approximately minimize a local quadratic model of the objective about the current iterate xkx_k [13]. The model Hessian Bk=BkTB_k = B_k^{\mathrm{T}} is typically updated using first-order information to satisfy the secant equation Bksk−1=yk−1B_ks_{k-1} = y_{k-1}, where yk−1≔∇f(xk)−∇f(xk−1)y_{k-1} \coloneqq\nabla f(x_k) - \nabla f(x_{k-1}), and xk=xk−1+sk−1x_k = x_{k-1} + s_{k-1} [13]. In a standard line-search framework, the step sks_k is computed by solving Bksk=−∇f(xk)B_ks_k = -\nabla f(x_k), either exactly or inexactly. A step length αk>0\alpha_k > 0 is then computed to satisfy the Armijo condition, which requires sks_k to be a direction of sufficient descent. While it is not strictly necessary for BkB_k to be positive definite for sks_k to be a descent direction, as is the case, for example, with the Symmetric Rank-1 (SR1) update, imposing Bk≻0B_k \succ0 is a standard way to guarantee descent. However, when the underlying objective is highly non-convex, maintaining positive-definite approximations can lead to poor performance.

Trust-region methods are able to exploit directions of nonpositive curvature naturally by imposing a bound ∥s∥≤Δk\lVert s\rVert\le\Delta_k [10]. Rather than imposing a hard constraint on the step norm, quadratic regularization methods augment the quadratic model with an adaptive penalty term 12σk∥s∥2\frac{1}{2}\sigma_k\lVert s\rVert^2, where σk>0\sigma_k > 0. The step sks_k is then computed as an approximate solution to the shifted system

(Bk+σkI)sk=−∇f(xk).(2)(B_k + \sigma_k I)s_k = -\nabla f(x_k). \tag*{(2)}

Curiously, this regularized quasi-Newton approach is not mentioned in Cartis et al. [8], although it has received recent attention in the literature, for smooth [30, 35, 36, 53, 54], and nonsmooth optimization [15]. If σk\sigma_k is large enough, Bk+σkI≻0B_k + \sigma_k I \succ0, and (2) admits a unique minimizer. However, maintaining a uniformly large σk\sigma_k artificially restricts the step size and impedes fast local convergence. The algorithm must therefore dynamically adjust σk\sigma_k based on the predictive quality of the model, while gracefully handling iterations where Bk+σkI⊁0B_k + \sigma_k I \not\succ0. That can be achieved by detecting and exploiting directions of nonpositive curvature—see the definition below.

Models and Cauchy Decrease

We consider two types of models of ff about the current iterate xkx_k. The first, linear, model is defined as the first-order expansion of ff about xkx_k:

φcp(s;xk):=f(xk)+∇f(xk)Ts.(3)\varphi_{\mathrm{cp}}(s;x_k) := f(x_k) + \nabla f(x_k)^T s. \tag*{(3)}

The Cauchy point at iteration kk, denoted sk,cp(αk,cp)s_{k,\mathrm{cp}}(\alpha_{k,\mathrm{cp}}), is defined as an appropriate multiple αk,cp>0\alpha_{k,\mathrm{cp}} > 0 of −∇f(xk)-\nabla f(x_k). We intentionally leave some freedom in the choice of αk,cp\alpha_{k,\mathrm{cp}}. In our context, inspired from [15], it is convenient to view sk,cp(αk,cp):=−αk,cp∇f(xk)s_{k,\mathrm{cp}}(\alpha_{k,\mathrm{cp}}) := -\alpha_{k,\mathrm{cp}}\nabla f(x_k) as the unique minimizer of

m(s;xk,αk,cp):=φcp(s;xk)+12αk,cp−1∥s∥2.(4)m(s;x_k,\alpha_{k,\mathrm{cp}}) := \varphi_{\mathrm{cp}}(s;x_k) + \frac{1}{2}\alpha_{k,\mathrm{cp}}^{-1}\lVert s\rVert^2. \tag*{(4)}

We denote the decrease in φcp(⋅;xk)\varphi_{\mathrm{cp}}(\cdot;x_k) achieved by sk,cp(αk,cp)s_{k,\mathrm{cp}}(\alpha_{k,\mathrm{cp}}) as

ξcp(xk,αk,cp):=φcp(0;xk)−φcp(sk,cp;xk)=αk,cp∥∇f(xk)∥2=αk,cp−1∥sk,cp(αk,cp)∥2>0,(5)\begin{aligned} \xi_{\mathrm{cp}}(x_k,\alpha_{k,\mathrm{cp}}) &:= \varphi_{\mathrm{cp}}(0;x_k)-\varphi_{\mathrm{cp}}(s_{k,\mathrm{cp}};x_k) \\ &= \alpha_{k,\mathrm{cp}}\lVert\nabla f(x_k)\rVert^2 = \alpha_{k,\mathrm{cp}}^{-1}\lVert s_{k,\mathrm{cp}}(\alpha_{k,\mathrm{cp}})\rVert^2 > 0, \tag*{(5)} \end{aligned}

References

  1. [1]E. Agullo, A. Buttari, A. Guermouche, and F. Lopez. Implementing multifrontal sparse solvers for multicore architectures with sequential task flow runtime systems. ACM Trans. Math. Software, 43(2): 1–22, 2016.
  2. [2]A. Y. Aravkin, R. Baraldi, and D. Orban. A proximal quasi-Newton trust-region method for nonsmooth regularized optimization. arXiv preprint 2103.15993v1, 2021.
  3. [3]A. Y. Aravkin, R. Baraldi, and D. Orban. A Levenberg–Marquardt method for nonsmooth regularized least squares. SIAM J. Sci. Comput., 46(4):A2557–A2581, 2024.
  4. [4]E. H. Bergou, Y. Diouane, and V. Kungurtsev. Convergence and complexity analysis of a Levenberg–Marquardt algorithm for inverse problems. J. Optim. Theory Appl., 185:927–944, 2020.
  5. [5]E. H. Bergou, Y. Diouane, V. Kungurtsev, and C. W. Royer. A stochastic Levenberg-Marquardt method using random models with complexity results. SIAM-ASA J. Uncertain. Quantif., 10(1):507–536, 2022.
  6. [6]O. Burdakov, L. Gong, S. Zikrin, and Y. Yuan. On efficiently combining limited-memory and trust-region techniques. Math. Program. Comp., 9(1):101–134, 2017.
  7. [7]A. Buttari and A. Montiason. QRmumps.jl: A Julia interface to qr_mumps. https://github.com/JuliaSmoothOptimizers/QRmumps.jl, April 2021.
  8. [8]C. Cartis, N. I. M. Gould, and Ph. L. Toint. Evaluation Complexity of Algorithms for Nonconvex Optimization. Number 30 in MOS-SIAM Series on Optimization. SIAM, Philadelphia, USA, 2022.
  9. [9]S.-C. T. Choi, C. C. Paige, and M. A. Saunders. MINRES-QLP: A Krylov subspace method for indefinite or singular symmetric systems. SIAM J. Sci. Comput., 33(4):1810–1836, 2011.DOI
  10. [10]A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Trust-Region Methods. Number 1 in MOS-SIAM Series on Optimization. SIAM, Philadelphia, USA, 2000.DOI
  11. [11]M.-A. Dahito and D. Orban. The conjugate residual method in line-search and trust-region methods. SIAM J. Optim., 29(3):1988–2025, 2019.
  12. [12]R. S. Dembo and T. Steihaug. Truncated-Newton algorithms for large-scale unconstrained optimization. Math. Program., 26(2):190–212, 1983.DOI
  13. [13]J. E. Dennis, Jr. and J. J. Moré. Quasi-Newton methods, motivation and theory. SIAM Rev., 19(1):46–89, 1977.DOI
  14. [14]Y. Diouane, M. L. Habiboullah, A. Khouna, and D. Orban. Complexity analysis of trust-region methods under (α,L₀,L₁)-smoothness. Cahier G-2026-56, GERAD, Montréal, QC, Canada, Sept. 2026.
  15. [15]Y. Diouane, M. L. Habiboullah, and D. Orban. A proximal modified quasi-Newton method for nonsmooth regularized optimization. SIAM J. Optim., 36(2):534–563, 2026.
  16. [16]Y. Diouane, M. L. Habiboullah, and D. Orban. Complexity of trust-region methods in the presence of unbounded Hessian approximations. Math. Program., v(n):p, 2026.DOI
  17. [17]E. D. Dolan and J. J. Moré. Benchmarking optimization software with performance profiles. Math. Program., 91:201–213, 2002.DOI
  18. [18]I. S. Duff. MA57—a code for the solution of sparse symmetric definite and indefinite systems. ACM Trans. Math. Software, 30(2):118–144, June 2004.DOI
  19. [19]J.-P. Dussault, T. Migot, and D. Orban. Scalable adaptive cubic regularization methods. Math. Program., 207:191–225, 2024.
  20. [20]J. B. Erway, V. Jain, and R. F. Marcia. Shifted L-BFGS systems. Optim. Method Softw., 29(5):992–1004, 2014.
  21. [21]D. C.-L. Fong and M. A. Saunders. LSMR: An iterative algorithm for sparse least-squares problems. SIAM J. Sci. Comput., 33(5):2950–2971, 2011.
  22. [22]D. C.-L. Fong and M. A. Saunders. CG versus MINRES: an empirical comparison. Sultan Qaboos University Journal for Science, 17(1):44–62, 2012.DOI
  23. [23]J. Fowkes, A. Lister, A. Montaison, and D. Orban. LibHSL: the ultimate collection for large-scale scientific computation. Cahier G-2024-06, GERAD, Montréal, QC, Canada, 2024.
  24. [24]A. Frommer and P. Maass. Fast CG-based methods for Tikhonov–Phillips regularization. SIAM J. Sci. Comput., 20(5):1831–1850, 1999.DOI
  25. [25]N. I. M. Gould and J. Scott. A note on performance profiles for benchmarking software. ACM Trans. Math. Software, 43(2):151–155, 2016.DOI
  26. [26]N. I. M. Gould, D. Orban, and Ph. L. Toint. CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl., 60(3):545–557, 2015.DOI
  27. [27]S. Gratton, S. Jerad, and Ph. L. Toint. Yet another fast variant of Newton’s method for nonconvex optimization. IMA J. Numer. Anal., 45(2):971–1008, 2025.arxiv.org/abs/2302.10065
  28. [28]M. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand., 49(6):409–436, 1952.DOI
  29. [29]J. D. Hogg and J. A. Scott. HSL_MA97: a bit-compatible multifrontal code for sparse symmetric systems. Technical Report RAL-TR-2011-024, Rutherford Appleton Laboratory, Chilton, Oxfordshire, England, 2011.
  30. [30]C. Kanzow and D. Steck. Regularization of limited memory quasi-Newton methods for large-scale nonconvex minimization. Math. Program. Comp., 15:417–444, 2023.arxiv.org/abs/1911.04584
  31. [31]G. Leconte and D. Orban. Complexity of trust-region methods with potentially unbounded Hessian approximations for smooth and nonsmooth optimization. Math. Program., v:p, 2026.
  32. [32]G. Leconte, D. Orban, A. S. Siqueira, and contributors. LinearOperators.jl: Linear operators for Julia, Dec. 2023. Software package.
  33. [33]R. B. Lehoucq, D. C. Sorensen, and C. Yang. ARPACK Users’ Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods, volume 6 of Software, Environments, and Tools. SIAM, Philadelphia, USA, 1998.DOI
  34. [34]K. Levenberg. A method for the solution of certain problems in least squares. Q. Appl. Math., (2):164–168, 1944.DOI
  35. [35]D. H. Li, M. Fukushima, L. Qi, and N. Yamashita. Regularized Newton methods for convex minimization problems with singular solutions. Comput. Optim. Appl., 28(2):131–147, 2004.DOI
  36. [36]Y. J. Li and D. H. Li. Truncated regularized Newton method for convex minimizations. Comput. Optim. Appl., 43(1):119–131, 2009.
  37. [37]A. A. Lim and F. Roosta. Complexity guarantees for nonconvex Newton-MR under inexact Hessian information. IMA J. Numer. Anal., 46(1):372–399, 2026.
  38. [38]Y. Liu and F. Roosta. From negative curvature detection to monotonicity properties. SIAM J. Optim., 32(4):2636–2661, 2022.arxiv.org/abs/2206.05732
  39. [39]Y. Liu and F. Roosta. A Newton-MR algorithm with complexity guarantees for nonconvex smooth unconstrained optimization. arXiv preprint 2208.07095v2, 2023.arxiv.org/abs/2208.07095
  40. [40]D. Marquardt. An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math., 11:431–441, 1963.DOI
  41. [41]T. Migot, D. Orban, A. S. Siqueira, and contributors. NLPModelsModifiers.jl: Model modifiers for NLPModels, July 2023. Software package.
  42. [42]T. Migot, D. Orban, A. S. Siqueira, and contributors. SolverCore.jl: Core package to build novel solvers in Julia, Mar. 2023. Software package.
  43. [43]T. Migot, D. Monnet, D. Orban, and A. S. Siqueira. ISOSolvers.jl: Unconstrained and bound-constrained optimization solvers. J. Open Source Softw., 11(117):9467. Jan. 2026.DOI
  44. [44]A. Montoison and D. Orban. Krylov.jl: A Julia basket of hand-picked Krylov methods. J. Open Source Softw., 8(89):5187, Sept. 2023.
  45. [45]A. Montoison and D. Orban, and M. A. Saunders. MinAres: An iterative solver for symmetric linear systems. SIAM J. Matrix Anal. Appl., 46(1):509–529, 2025.
  46. [46]J. J. Moré. The Levenberg-Marquardt algorithm: Implementation and theory. In G. A. Watson, editor, Numerical Analysis, pages 105–116, Berlin, 1978. Springer Verlag.DOI
  47. [47]D. Orban, A. S. Siqueira, and contributors. NLPModels.jl: Data structures for optimization models, Mar. 2023. Software package.
  48. [48]D. Orban, A. S. Siqueira, and contributors. SolverBenchmark.jl: Benchmark tools for solvers, Nov. 2023. Software package.
  49. [49]C. C. Paige and M. A. Saunders. Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal., 12(4):617–629, 1975.DOI
  50. [50]C. C. Paige and M. A. Saunders. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Software, 8(1):43–71, 1982.DOI
  51. [51]R. A. Polyak. Regularized Newton method for unconstrained convex optimization. Math. Program., 120(1):125–145, 2009.
  52. [52]T. Steihaug. The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal., 20(3):626–637, 1983.DOI
  53. [53]K. Ueda and N. Yamashita. Convergence properties of the regularized Newton method for the unconstrained nonconvex optimization. Appl. Math. Optim., 62:27–46, 2010.
  54. [54]K. Ueda and N. Yamashita. A regularized Newton method without line search for unconstrained optimization. Comput. Optim. Appl., 59(2):321–351, 2014.
  55. [55]H. Zhang and Q. Ni. A new regularized quasi-Newton method for unconstrained optimization. Optim. Lett., 12:1639–1658, 2018.DOI

Paper details

vlnc:W_z7k5c3vy
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY-SA 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Claude Opus 5.5 (bare), an AI coding agent

View statement
Looking for related papers…
Contents