CC BY
38
Численные методы и их приложения
UDC 519.6
A Local and Semilocal Convergence of the Continuous Analogy of Newton's Method
T. Zhanlav*, O. Chuluunbaatart
* Department of Applied Mathematics National University of Mongolia Ulan-Bator, Mongolia ^ Laboratory of Information Technologies Joint Institute for Nuclear Research, Joliot-Curie 6, 141980 Dubna, Moscow region, Russia
In this paper, a region of convergence of the continuous analogy of Newton's method is defined and an optimal choice of the parameter t is proposed. For the damped Newton's method a global convergence is proved and error bounds are obtained. The damping strategies allow one to extend the convergence domain of the initial guesses. Several damping strategies were compared. Numerical examples are given and confirm the theoretical results.
Key words and phrases: nonlinear equations in Banach spaces; damped Newton's method; recurrence relations; error bounds.
1. Introduction
One of the modifications of Newton method is the well-known continuous analogy of Newton's method (CANM) or damped Newton's method
Xk+1 = Xk + Tk Vk, F'(xk )vk = -F (Xk), k = 0,1,... (1)
to solve nonlinear equations F(x) = 0 in Banach spaces. In this case F : fi C X ^ Y is an operator defined in an open convex subset fi of the Banach space X into the Banach space Y. If Tk = 1 the CANM leads to the Newton method. In our paper [1] we have shown that the iteration (1) is convergent if Tk G (0; 2). In the present paper, we give a r-region of convergence of CANM and propose an optimal choice for the parameter Tk in (1). We also prove the semilocal convergence theorem for the iteration (1) by using a technique consisting of a new system of recurrence relations [2].
2. The r—region of Convergence of the CANM
We study the influence of the parameter t on the convergence of method (1). Consider the open level set La = {x G fi; ||F(s)|| < 'q/f3 = a} and let L C fi be bounded. In that follows we assume that
(ci) HF'(y) - F'(x)H < L Hy - xl x,y G fi,
(C2) ||(F'(so))-1l < P,
(cs) H(F'(xo))-1F(so)! < V, ao = LPV,
(C4) L H(F'(xk ^HKF^Xk ))-1F (xk )y < ak, k = 0,1,...
Received 27th September, 2011.
O. Chuluunbaatar acknowledges a financial support from the RFBR Grant No. 11-01-00523, and the theme 09-6-1060-2005/2013 "Mathematical support of experimental and theoretical studies conducted by JINR"..
Theorem 1. Let us assume that the conditions (c1)-(c4) are satisfied and Tk G Ik, where
("• -' +) = («■ 2). f »<<*<(2)
0, — ) . for 1 <ak < M < to.
flfe /
Then the sequence [xk] obtained by (1) is well defined and remains in La and converges to some x* with F(x*) = 0.
Proof. We notice first of all that
Tkak < 1, к = 0,1,..., as Tk G Ik. By induction we prove that there exists r&+i = (F'(xk+i))-i-, such that
Iiri+,|| < (S)
Now from the conditions (c1) and (c4) we have
||J - rfcF'(xk+i)II < |Г|| ||F'f(xk) - F'(xk+i)II < L ||rfc|| ||®fc+i - xk|| =
= rk L ||rfc || ||rfc F (xk)H < тк ak < 1, к = 0,1,...
So, by the Banach perturbation lemma there should exist Г^+1 satisfying (3). Thus the sequence {xk} obtained from the damped Newton iteration is well defined. The Taylor expansion of F(x) about Xk gives
F(xk+i) = F(xk) + F'(£fc)(xk+i - xk) =
= (1 - rk)F(xk) + (F'(&) - F'(xk))(xk+i - xk), (4)
where £k = 6xk + (1 - 6)xk+i, в G (0,1). Using the conditions (ci) and (c4) in (4) we have
||F(®fc+i)|| < ^(тк)||F(хк)Ц, (5)
■ф(тк) = |1 - Tk| + akTk. (6)
It is easy to show that
Ф(тк) < 1 (7)
under the condition Tk G Ik. Therefore, from (5) we deduce the following
||F (xk )|| < ip(Tk-i)ip(Tk-2) ••• ^(ro)|F (®c)|. (8)
According to (7) we have
||F (Xk )|| < ||F Ы|| < f, к = 1, 2,...,
i.e. xk G La for all к = 1, 2,... From (8) we obtaine
||F(xk)|| — 0 at к —У то.
Since La is bounded, there exists an accumulation point x* of the sequence [xk] with F (x*) = 0. □
We call the interval Ik by the r-region of convergence, as well as call the value Tk, such that
■^(Tk) = min ^(t),
Tfce(0,2)
the optimal one and denote it by Topt. The direct calculation gives us
( 1
1, for 0 < ak < ^,
1 r 1 2
Topt =< , for 2 <ak < 1
— — e, for ak > 1,
ak (9)
ak, for 0 < ak < 1,
^°Pt)=\ 1 — ± < 4 for 1 <ak <1,
1 — £ + e2ak, for ak > 1.
Here £ is a small number. Some particular choices of the parameter, in contrast to the optimal one, may be useful. We consider
-1 + Vl + 8afc
Tk =-41-, (10)
which is a middle point of the admissible interval Ik. Since Tk < 1, then from (6) we obtain
3
HTk) = 3 (1 — Tk). (11)
In (11) we can see that ^(rk) < 1 if Tk > 1/3. On the other hand, the Tk is decreasing and ^(rk) is increasing with respect to ak. Therefore from (10) and (11) we get
1 <Tk < 1, 0 < ^(Tk) < 1, Tk ak < 1
under condition 0 < ak < 3. Thus, the choice (10) allows us to weaken the condition imposed on ak. From (10), (11) it follows that Tk ^ 1 and ^(rk) ^ 0 at ak ^ 0. Moreover, if ak+i < ak for k = 0,1,..., then we have
To <T1 < ••• <Tk < ••• < 1, 0 < ip(Tk) < • • • < ^(n) < ^(to).
It should be pointed out that, according to affine invariance property [3] we can also take
= -1^1 + 26afc, for yb> 0 k = 0,1,..., (12)
bak
which has the same properties as Tk given by (10), i.e. Tk : (0; to) ^ (0,1) and decreasing with respect to ak.
Theorem 2. Suppose the conditions (ci) — (c4) are satisfied and Tk is given by b + 2
(12) with 0 < ak < —2— • Then the sequence {xk} obtained by (1) is well defined and remains in La and converges to some x* with F(x*) = 0.
Proof. By virtue of (6) and (12) we have
^(Tk) = (1 — Tk).
Since Tk(ak) is a decreasing function with respect to ak and ^(rk) is decreasing • , _ , „ b + 2 , 2 _ , too with respect to tk, and 0 < ak < —2—, then we get y+2 < Tk < 1, and
0 < ^(rk) < 1, Tkak < 1. The proof follows immediately from (5) and (8). □
The above derived admissible interval (2) and the theoretical optimal value as well as the particular choices (10), (12) cannot be implemented directly, since the arising quantities ak are computationally unavailable due to the arising Lipschitz constant, IITfc|| and ||rkF(xk)||. However, the obtained theoretical results can be useful for the construction of computational strategies. Namely, from the assumption (c4) it is evident that ||F(xk)H ^ 0 if ak ^ 0, and ak ^ to if ||F(xk)|| ^ to.
Therefore, replacing ak in (12) by yk = ||F(xk)||, we obtain
2
= ' + V + 2tryk G (0. "> 0 (13)
The function t* = T*(yk) decreases with respect to yk. From (13) it is also clear that t* ^ 1 if yk ^ 0, and t* ^ 0 if yk ^ to. We have the following theorem.
Theorem 3. Suppose the conditions (c1) — (c4) are satisfied and Tk is given by
(13). Suppose also that 0 < ak < —' . Then the sequence [xk] obtained by (1)
is well defined and remains in La and converges to some x* with F(x*) = 0.
Proof. By virtue of (6) and (13) we have
li *\ 6^fc + 2ak
^k) = ' + byk + V1 + 2^. (14) From (13), (14) it is easy to show that the next two inequalities hold simultaneously
T**ak < 1 and ^(t**) < 1 (15)
under the condition
0 <ak < ^ Mk (14a)
or equivalently
yk > ^ ~ ' . (146)
Since 0 < t* < 1 and T*ak < 1 then t* G Ik. As a consequence, the assertion of the theorem follows directly from Theorem 1. □
Corollary. Theorem 3 is valid for any choice rk G (0, r*j.
Indeed, Tkak < T*ak < 1 and for such Tk we have ^(rk) = 1 — Tk + ak< 1, that assures a reduction of the residual norm, i.e. ||F(xk+1 )|| < ||F(xk)H. However the choice t* is preferable in the sense that t* ^ 1 as yk ^ 0 and the method (1) asymptotically leads to quadratically convergent Newton method. Nevertheless, there arises a question of the choice of the parameter b in (13). In our opinion, it must be done in such a way that the damped Newton's method converges rapidly.
3. A Semilocal Convergence of CANM
We define the sequence
an+i = f (an,Tn)2g(an,Tn)an, ao = Lfir], n = 0,1,..., (16)
where
f (X,T) = -^-, g(x,r) = |1 — r | + r2x. (17)
1 — TX
We need the following auxiliary lemmas, whose proofs are trivial from [2].
Lemma 1. Let f and g be two real functions given by (17). Then
(i) For a fixed t G (0,1], f (x,t) and g(x,r) increase as a function of x G (0, 1/t) and f (x, t) > 1,
(ii) f (ix; t) < f (x; t) and g(rix,T) < 7pg(x,r) for- 7 G (0,1],
where
' 0, if t = 1,
P 1 1, if r = 1.
3 — \/5
Lemma 2. Let 0 < a0 < —2— and rk G (0,1) for all k = 0,1,... Then the sequence {an} is decreasing, i.e.
n 3 —
0 < • • • < an < an-1 < • • • < a1 < a0 < —2—.
Lemma 3. Let us suppose the conditions of Lemma, 2 are satisfied and define
7 = a1/a0. Then
(i) 1 = f (a0,T0)2g(a0,T0) G (0,1),
, , (1Ip)»-i (I+P)"-1
(ii) an < 7(1+P) an-1 < ••• < 7 p a0,
(iii) f (an,Tn)g(an,Tn) < A with A = ^^1 ^) < 1. Notice that
L ||r0N H^F(S0)|| < 00, Il®1 — ®0| < H^F(®0)| < n<R'n with R = Y"1^^ > 1,
i.e. x1 G B(x0,Rrj) = {x G X; ||s — s0|| < Rq}. In these conditions we prove, for n > 1, the following statements:
(In) ||rJ < f (an-1,Tn-1) ||r n 1 | , (IIn) ||r nF(xn)H < f (an-1,Tn-1)g(an-1,Tn-1) ||rn-1^(xn-1)||,
(IIIn) L ||rj ||rnF(xn)H < an, (IVn) Xn+1 G B(x0,Rrj). Assuming T0a0 < 1 and x1 G Q, we have
||/ — ^F'(®1)| < ||r0| HF'(X0) — F'(®1)| < L H^H — »0! =
= T0L ||r0| H^F(®0)| < T0a0 < 1.
Then by the Banach lemma, r1 exists and
"r1" ^ 1 — ||ro|||№) — F'(X!)|| < f ^ M,
and (I1) is true. On the other hand, according to (1), we have
F (Xn+1) = (1 — Tn)F (Xn) + (F' (£n) — F' (Xn))(Xn+1 — Xn), where £n = dxn + (1 — 6)xn+1, 6 G (0,1), and
F' (xn+1) = F' (xn)(I — Pn), Pn = r n(F' (xn) — F' (xn+1)).
If xn,xn+1 G fi following
HPn|| < L ||rn|| ||sn+i — xn|| < rnL ||rn|| ||rnF(xn)H < rnan < 1. Then r„+1 = (I — Pn) — 1rn, and for n = 0, if x0,x1 G fi, we have
riF (®1) = (I — Po)-1ro[(1 — ro )F (so) + (F' (&) — F' (®o))(®1 — ®o)] =
= (I — Po)-1[(1 — ro) — roro(F' (&) — F' (xo))]roF (xo).
Thus, we get
H^F0n)|| < -[|1 — ro| + ro2ao] H^F(®o)|| = f(ao,To)g(ao,ro) H^F(®o)||,
' — Toao
i.e. (II1) is true. To prove (III1) and (IV1), we use
L ||r11 H^F(®1)|| < f (oo,to)2g(a0,T0)L ||r|| H^F(®0)|| < f (ao,To)2g(a0,T0)a0 = «1 and
||®2 — x0|| < ||®2 — ®1| + ||®1 — x0|| < nH^F(®1)| + ro||roF(®0)|| < < [r1/(a0,T0)ff(a0,T0) + t0] H^F(®0)|| < 1+ 7
f(«0 ,To) = ^(1 + A7) < ^ = Rq,
i.e. x2 G B(ao, Rq) and this proof holds by induction for all n G N. Now following an inductive procedure and assuming xn+1 G fi the items (In)-(IVn) are proved.
To establish the convergence of [xn], we only have to prove that it is a Cauchy sequence and that xn+1 G fi. We note that
n— 1
||r„F(xn)H < ||roF(®0)HGn, Gn = n fK,rk)g(ak,rk).
k=o
As a consequence of Lemma 3 it follows that
Gn < A""]——17= A"7.
k=o
So, from A < 1, we deduce that Gn = 0. We can now formulate the following
result on convergence for the iteration (1).
Theorem 4. In the conditions indicated for the operator F, let us assume that ro = (F'(xq))—1 G L(Y,X) exists at some xq G fi and (c1) — (c4) are satisfied. Suppose that Tk is given by
* —' + V + 26yfc *
rfc ={ Tk =-W*-, f°r 1 — Tfc >£, (18)
1, for 1 — T* < e.
Then if B(xa,Rq) = [x G X; ||s — xq,|| < Rq] G fi, the sequence [xn] defined by (1) and starting at xq has, at least, R-order (p + 1) and converges to a solution x* of the equation F(x) = 0. In this case, the solution x* and the iterations xn
belong to B(x0, Rri) and x* is the only solution to F(x) = 0 in B , — R^j H Q. Furthermore, we can give the following error estimates:
Xn+rr y ^
A'
n+m
1 - A7
2™-1+m
lircF (®c)|, 7 =
|1 - tq| + apTo (1 - rpap)2
(19)
Proof. We have proved above that the assumption (c4) is satisfied. Then by virtue of Theorem 3 the damped Newton's iteration converges. Namely the residual norm || F(xk)|| decrease as k increase. Then t** tends to units as k increases. Hence the condition 1 — tk < £ will holds starting at some number k = m. Then by virtue of (18) we have Tk = 1 for all k > m. On the other hand, xn G B(x0, Rq) for all n G N, then xn G Q, n G N. Now we prove that {xn} is a Cauchy sequence. To do this, we consider n,m > 1:
e-1 e-1
^Xm+n+e — ^m+n^ ^ \lXm+n+l+1 — Xm+n+l1 ^ ||rm+n+ZF(Xm+n+l)| ^
I=0 1=0
e-1
e- 1
< ||rro+nF(xm+n)|| £ A '72m+"(2 -1) < ||rro+nF(xm+n)|| £(A7)e-1 =
i=0
i=0
1 - (A7)e
1 - A7
in which we have used the well-known inequality (1 + x)k > 1 + kx
||rm+nF (xm+n)||, (20)
According to Lemma 3, we have
Hrm+nF(xm+n)H < An72m(2"-1)|rmF(xm)H < A
n„ ,2™-1
(X r
and
llTmF(Xm)ll < (A7)m|ToF(so)||. Substituting (21), (22) into (20), we obtain
n+e n^ ^
1 - (A7r
1 - ¿7
Ar+n7
2™-1+r
||roF (xo )||,
(21)
(22)
(23)
then {xn} is a Cauchy sequence. Now by letting e ^ to in (23), we obtain (19).
To prove that F(x*) = 0, notice that ||rnF(xn)H ^ 0 by letting n ^ to. As ||^X®n)|| ^ ll^" (^n)||TnF (xn)H and F' (xn) is a bounded sequence, we deduce ||F(sn)|| ^ 0 and then F(x*) = 0 by the continuity of F. Now, to show the uniqueness, suppose
that y* G B , -j-^ — R^j H Q is another solution to F(x) = 0. Then
0 = F (y*) - F (x* ) = / F' (x* + t(y* - x*))dt(y* - x*)
(24)
Also we use the estimation
I - ro 1 F'(x* + t(y* - x*))dt < ||ro|| /||F'(x* + t(y* - x*)) - F'(œo)||di <
1
1
1
1 1 < LP f ||s* - t(y* - x*) - xo\\dt < LP f ((1 - t)\\x* - £01| + t\\y* - ®o||)di <
0 0
< ZT (^ + Zp - ^) = 1. (25)
1
We see that the operator JF'(x* + t(y* -s*))dt has an inverse and consequently from
0
(24), we get y* = x*. □
Remark. In [4] the following choice was proposed
= ^(¡T)1!!1 T-1' n = 1,2,..., ^ - 0.1, (26)
and were the error bounds of kind of (19) derived under the conditions ||F'(x)-11| < P, ||F''(x)|| < M. Here we derive the error bounds (19) without such conditions. There is a closed relationship between (13) and (26). Namely, (13) gives
< _ ||F(x„-i)|| = -1 + y/l + 2b\\F(xn)\\
^n -
<-1 HF(xn)| n -1^1 + 26|F(x„-i)||'
It is easy to show that An ^ 1 at max(||F(xn-1)H, ||F(xn)||) ^ 0, i.e. in the limit these two choices coincide. Since Tk = 1 for all k > m our iteration (1) can be considered as a Newton iteration starting at a0 = am, where
3 — \/5
am = L |\rm|| \\rmF(sm)|| < a0lm < , (27)
which obviously, holds for large m, and for any a0. On the other hand, the Kantorovich semilocal convergence theorem was proved under the condition a0 < 1/2, i.e. the local Newton methods, require sufficiently good initial guesses. Unlike, our iteration (1), as Newton's method is able to compensate for bad initial guesses by virtue of damping strategies. The inequality (27) has shown that the damped Newton's method at first m-stage allows one to extend the convergence domain of the initial guesses.
4. Numerical Results and Discussion
Now let us give some numerical examples that confirm the theoretical results. We use the following test examples fi(x) = 0, i = 1,..., 5, which are the same as in [5]: /1(s) = ln x = 0, x* = 1.0,
h(x) = ex2+7x-30 - 1 = 0, x* = 3.0, f3(x) = 1/x - 1 = 0, s* = 1,
fi(x) = s3 + 4s2 - 10 = 0, s* = 1.3652300134140968457, f5(x) = arctan x = 0, x* = 0.0.
All these computations are carried out with a double arithmetic precision and the number of iterations n such that |/(xn)l < 1.0e - 16 is tabulated in Table 1. Note that for the last two calculations for (10) and (9) used ak = lf'(xk)(f'(xk))-2f(xk)|. Table 1 also gives the number of iterations of the simple Newton's method, and damped Newton's method with [6]
5(0)
Tk =
5(0) + 5(1)'
W)= f2 (-k - *Hg) . (28)
Table 1
The number of iterations at e = le — 16
Functions
f2{x) h{x) h{x) h{x)
x0 6.4 4.0 2.0 3.5 4.2 5.55 0.9 2.01 2.4 -0.5 0.1 1.0 2.0 1.7 1.4 1.0
Newton - - 6 12 22 45 4 - - 108 10 5 - - - 5
Tk by (28) 11 10 5 13 25 50 5 - - - 141 5 5 4 3 4
3.0 5 6 6 81 - - 5 8 12 19 23 10 7 7 6 5
Tk by (13) b 2.0 8 6 5 68 - - 5 8 24 480 24 9 7 6 6 5
1.0 9 8 5 51 - - 5 9 - 73 25 7 9 7 6 5
0.1 - - 6 23 2260 - 5 13 - 40 22 5 - - 7 5
Tk by (10) 7 7 6 20 41 86 6 7 7 - 7 6 5 5 5 5
Tk by (9) 19 20 1 15 26 48 5 4 6 - 8 5 5 5 5 3
to
CO ¡=
3
o
a fo
a £
I
-5
3
a
a
o S CO
Site
iZ o
o to
V
CO
References
1. Zhanlav T., Chuluunbaatar O. Convergence of the Continuous Analogy of Newton's Method for Solving Nonlinear Equations // Numerical Methods and Programming, Moscow State University. — 2009. — Vol. 10. — Pp. 402-407.
2. Hernandez M. A., Salanova M. A. Modification of the Kantorovich Assumptions for Semilocal Convergence of the Chebyshev Method // J. Comput. Appl. Math. — 2000. — Vol. 126. — Pp. 131-143.
3. Deuflhard P. Newton Methods for Nonlinear Problems. Affine Invariance and Adoptive Algorithms. — Springer International, 2002.
4. Zhanlav T., Puzynin I. V. The Convergence of Iteration Based on a Continuous Analogue of Newton's Method // Comput.Math and Math. Phys. — 1992. — Vol. 32. — Pp. 729-737.
5. Zhanlav T., Chuluunbaatar O. High-Order Convergent Iterative Methods for Solving Nonlinear Equations // Bulletin of Peoples' Friendship University of Russia. Series "Mathematics. Information Sciences. Physics". — 2009. — No 3. — Pp. 7078.
6. Ermakov V. V., Kalitkin N. N. The Optimal Step and Regularization for Newton's Method // USSR Comp. Phys. and Math. Phys. — 1981. — No 21(2). — Pp. 235242.
УДК 519.6
Локальная и полулокальная сходимость непрерывного аналога метода Ньютона
Т. Жанлав*, О. Чулуунбаатар^
* Кафедра прикладной математики Монгольский государственный университет, Монголия ^ Лаборатория информационных технологий Объединённый институт ядерных исследований ул. Жолио-Кюри д.6, Дубна, Московская область, 141980, Россия
В данной работе определена область сходимости непрерывного аналога метода Ньютона и предложен оптимальный выбор параметра т. Для затухающего метода Ньютона доказана глобальная сходимость и получены оценки погрешности. Стратегии затухания позволяют расширить область начальных параметров, при которых метод сходится. Дано сравнение различных стратегий затухания. Приведённые численные примеры подтверждают теоретические результаты.
Ключевые слова: нелинейные уравнения в банаховых пространствах; затухающий метод ньютона; рекуррентные соотношения; оценка погрешности.
