LEVENBERG-MARQUARDT METHOD WITH A GENERAL LM PARAMETER AND A NONMONOTONE TRUST REGION TECHNIQUE

Luyao Zhao; Jingyong Tang

doi:10.11948/20220441

2024 Volume 14 Issue 4

Article Contents

Article navigation > Journal of Applied Analysis & Computation > 2024 > 14(4): 1959-1976

Next Article Previous Article

Luyao Zhao, Jingyong Tang. LEVENBERG-MARQUARDT METHOD WITH A GENERAL LM PARAMETER AND A NONMONOTONE TRUST REGION TECHNIQUE[J]. Journal of Applied Analysis & Computation, 2024, 14(4): 1959-1976. doi: 10.11948/20220441

Citation:

Luyao Zhao, Jingyong Tang. LEVENBERG-MARQUARDT METHOD WITH A GENERAL LM PARAMETER AND A NONMONOTONE TRUST REGION TECHNIQUE[J]. Journal of Applied Analysis & Computation, 2024, 14(4): 1959-1976. doi: 10.11948/20220441

LEVENBERG-MARQUARDT METHOD WITH A GENERAL LM PARAMETER AND A NONMONOTONE TRUST REGION TECHNIQUE

Luyao Zhao^1,,
Jingyong Tang^1, ,

1.
College of Mathematics and Statistics, Xinyang Normal University, 464000 Xinyang, China

Author Bio: Email: zly976557281@163.com(L. Zhao)
Corresponding author: Email: tangjy@xynu.edu.cn(J. Tang)
Fund Project: This work was supported by the Henan Province Natural Science Foundation (222300420520) and the Key Scientific Research Projects of Higher Education of Henan Province (22A110020)

Abstract

Abstract

We propose a new Levenberg-Marquardt (LM) method for solving the nonlinear equations. The new LM method takes a general LM parameter $ \lambda_k=\mu_k[(1-\theta)\|F_k\|^\delta+\theta\|J_k^TF_k\|^\delta] $ where $ \theta\in[0,1] $ and $ \delta\in(0,3) $ and adopts a nonmonotone trust region technique to ensure the global convergence. Under the local error bound condition, we prove that the new LM method has at least a superlinear convergence rate with the order $ \min\{1+\delta,4-\delta,2\} $. We also apply the new LM method to solve the nonlinear equations arising from the weighted linear complementarity problem. Numerical experiments indicate that the new LM method is efficient and promising.
- Nonlinear equations /
- Levenberg-Marquardt method /
- nonmonotone technique /
- local error bound /
- weighted linear complementarity problem
MSC: 65K05, 90C30

FullText(HTML)

References

[1]	M. Ahookhosh and K. Amini, An efficient nonmonotone trust-region method for unconstrained optimization, Numer. Algor., 2012, 59(4), 523–540. doi: 10.1007/s11075-011-9502-5 CrossRef Google Scholar
[2]	K. Amini, F. Rostami and G. Caristi, An efficient Levenberg-Marquardt method with a new LM parameter for systems of nonlinear equations, Optim., 2018, 67(5), 637–650. doi: 10.1080/02331934.2018.1435655 CrossRef Google Scholar
[3]	S. Asadi, Z. Darvay and G. Lesaja, et al., A full-Newton step interior-point method for monotone weighted linear complementarity problems, J. Optim. Theory Appl., 2020, 186(3), 864–878. doi: 10.1007/s10957-020-01728-4 CrossRef Google Scholar
[4]	R. Behling and A. Iusem, The effect of calmness on the solution set of systems of nonlinear equations, Math. Program., 2013, 137(1–2), 155–165. doi: 10.1007/s10107-011-0486-7 CrossRef Google Scholar
[5]	J. S. Chen, The semismooth-related properties of a merit function and a descent method for the nonlinear complementarity problem, J. Global Optim., 2006, 36(4), 565–580. doi: 10.1007/s10898-006-9027-y CrossRef Google Scholar
[6]	X. N. Chi, Z. P. Wan and Z. J. Hao, A full-modified-Newton step $O(n)$ infeasible interior-point method for the special weighted linear complementarity problem, J. Ind. Manag. Optim., 2022, 18(4), 2579–2598. doi: 10.3934/jimo.2021082 CrossRef Google Scholar
[7]	X. N. Chi and G. Q. Wang, A full-Newton step infeasible interior-point method for the special weighted linear complementarity problem, J. Optim. Theory Appl., 2021, 190(1), 108–129. doi: 10.1007/s10957-021-01873-4 CrossRef Google Scholar
[8]	N. Y. Deng, Y. Xiao and F. J. Zhou, Nonmonotonic trust region algorithm, J. Optim. Theory Appl., 1993, 76(2), 259–285. doi: 10.1007/BF00939608 CrossRef Google Scholar
[9]	J. Y. Fan, A modified Levenberg-Marquardt algorithm for singular system of nonlinear equations, J. Comput. Math., 2003, 21(5), 625–636. Google Scholar
[10]	J. Y. Fan and J. Y. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition, Comput. Optim. Appl., 2006, 34(1), 47–62. doi: 10.1007/s10589-005-3074-z CrossRef Google Scholar
[11]	J. Y. Fan and J. Y. Pan, A note on the Levenberg-Marquardt parameter, Appl. Math. Comput., 2009, 207(2), 351–359. Google Scholar
[12]	J. Y. Fan and Y. X. Yuan, On the convergence of a new Levenberg-Marquardt method, AMSS, Chinese Academy of Sciences, 2001, Report NO. 005, 1–11. Google Scholar
[13]	J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Comput., 2005, 74(1), 23–39. doi: 10.1007/s00607-004-0083-1 CrossRef Google Scholar
[14]	M. S. Gowda, Weighted LCPs and interior point systems for copositive linear transformations on Euclidean Jordan algebras, J. Global Optim., 2019, 74(2), 285–295. doi: 10.1007/s10898-019-00760-7 CrossRef Google Scholar
[15]	C. F. Ma and L. H. Jiang, Some research on Levenberg-Marquardt method for the nonlinear equations, Appl. Math. Comput., 2007, 184(2), 1032–1040. Google Scholar
[16]	F. A. Potra, Weighted complementarity problems-a new paradigm for computing equilibria, SIAM J. Optim., 2012, 22(4), 1634–1654. doi: 10.1137/110837310 CrossRef Google Scholar
[17]	F. A. Potra, Sufficient weighted complementarity problems, Comput. Optim. Appl., 2016, 64(2), 467–488. doi: 10.1007/s10589-015-9811-z CrossRef Google Scholar
[18]	G. W. Stewart and J. G. Sun, Matrix Perturbation Theory, Academic Press, Inc., Boston, MA, San Diego, CA, 1990. Google Scholar
[19]	J. Y. Tang, A variant nonmonotone smoothing algorithm with improved numerical results for large-scale LWCPs, Comput. Appl. Math., 2018, 37(3), 3927–3936. doi: 10.1007/s40314-017-0554-6 CrossRef Google Scholar
[20]	J. Y. Tang and H. C. Zhang, A nonmonotone smoothing Newton algorithm for weighted complementarity problems, J. Optim. Theory Appl., 2021, 189(3), 679–715. doi: 10.1007/s10957-021-01839-6 CrossRef Google Scholar
[21]	J. Y. Tang and J. C. Zhou, A modified damped Gauss-Newton method for non-monotone weighted linear complementarity problems, Optim. Methods Softw., 2022, 37(3), 1145–1164. doi: 10.1080/10556788.2021.1903007 CrossRef Google Scholar
[22]	J. Y. Tang and J. C. Zhou, Quadratic convergence analysis of a nonmonotone Levenberg-Marquardt type method for the weighted nonlinear complementarity problem, Comput. Optim. Appl., 2021, 80(1), 213–244. doi: 10.1007/s10589-021-00300-8 CrossRef Google Scholar
[23]	Ph. L. Toint, Non-monotone trust-region algorithm for nonlinear optimization subject to convex constraints, Math. Program., 1997, 77(1), 69–94. Google Scholar
[24]	H. Y. Wang and J. Y. Fan, Convergence rate of the Levenberg-Marquardt method under Hölderian local error bound, Optim. Methods Softw., 2020, 35(4), 767–786. doi: 10.1080/10556788.2019.1694927 CrossRef Google Scholar
[25]	H. Y. Wang and J. Y. Fan, Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound, J. Ind. Manag. Optim., 2021, 17(4), 2265–2275. doi: 10.3934/jimo.2020068 CrossRef Google Scholar
[26]	N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, Comput. Suppl., 2001, 15, 239–249. Google Scholar
[27]	Y. X. Yuan, Recent advances in trust region algorithms, Math. Program., 2015, 151(1), 249–281. doi: 10.1007/s10107-015-0893-2 CrossRef Google Scholar
[28]	M. L. Zeng and G. H. Zhou, Improved convergence results of an efficient Levenberg-Marquardt method for nonlinear equations, J. Appl. Math. Comput., 2022, 68(6), 3655–3671. doi: 10.1007/s12190-021-01599-6 CrossRef Google Scholar
[29]	J. Zhang, A smoothing Newton algorithm for weighted linear complementarity problem, Optim. Lett., 2016, 10(3), 499–509. doi: 10.1007/s11590-015-0877-4 CrossRef Google Scholar
[30]	L. Zheng, L. Chen and Y. F. Ma, A variant of the Levenberg-Marquardt method with adaptive parameters for systems of nonlinear equations, AIMS Math., 2022, 7(1), 1241–1256. Google Scholar
[31]	W. J. Zhou, On the convergence of the modified Levenberg-Marquardt method with a nonmonotone second order Armijo type line search, J. Comput. Appl. Math., 2013, 239, 152–161. doi: 10.1016/j.cam.2012.09.025 CrossRef Google Scholar