Smoothing iterative method for a class of nonsmooth equations Текст научной статьи по специальности «Математика»

Вычислительные технологии

Том 11, № 6, 2006

SMOOTHING ITERATIVE METHOD FOR A CLASS OF NONSMOOTH EQUATIONS*

H. Wang

Nanjing University of Aeronautics and Astronautics, China College of Sciences, China University of Mining and Technology, Xuzhou

e-mail: [email protected]

D. Cao

College of Sciences, China University of Mining and Technology, Xuzhou

S. Li

College of Information and Electrical Engineering, China University of Mining and Technology, Xuzhou

Статья посвящена решению одного класса негладких уравнений c использованием функции-максимума. На основе принципа максимума энтропии строится гладкая функция, аппроксимирующая исходную негладкую. Предложена новая итерационная формула и дан алгоритм решения аппроксимирующей задачи. Указаны условия, при которых обеспечивается сходимость. Теоретический анализ и результаты расчетов показали эффективность и надежность алгоритма.

Introduction

We consider the following nonsmooth equations

F (x) = 0 (0.1)

where F : D С Rn ^ Rn, F (x) = (fi(x), ^(x), ••• , fn(x))T and /¿(x) = ma xiej. fj (x)(i = 1,... ,n), /¿j(x)(j £ Ji) are continuously differentiable on D, Ji for i = 1,... ,n are finite index sets. Obviously, fi(x)(i = 1,... , n) are nondifferentiable functions on D.

Recently, the problem (0.1) has been discussed by several authors. In papers [1-3], Newton method and quasi-Newton method for solving nonsmooth equations have been described. Based on the same rate of convergence in [1-3], an improved algorithm with less computing cost was suggested in [4] and [6]. But the above algorithm is hard to accomplish in actual numerical computation. In this paper, we propose a new idea of solving nonsmooth equations (0.1) and our method can improve these defects. The following two steps are required.

First, we use the maximum entropy method to transform nonsmooth functions fi(x)(i = 1,... ,n) to smooth functions.

*This work is supported by the National Science Foundation of China (60304016) and by the Youth Foundation and Science Foundation of CUMT (A200401, A200410).

© Институт вычислительных технологий Сибирского отделения Российской академии наук, 2006.

Second, we apply known techniques and new iterative formula to solve the resulting smooth equations.

As a particular case of this idea, we propose a new algorithm for solving problem (0.1). The structure of the paper is as follows: In Section 1, we will describe maximum entropy function and design smooth approximating function. In Section 2, the proposed algorithm and its convergence property are described. In Section 3, we give numerical tests to illustrate our theoretical results. Finally, the last section contains the conclusions and an outline of the directions for future research.

1. Design Smooth Approximating Function

For nondifferentiable problem this is hard to solve it by traditional methods including derivative. Usually we handle it with a differentiable function approaching to objective function. A maximum entropy function of uniform approximation to objective function is given in [5] and [9, 10].

Maximum Entropy Function. The maximum entropy function originated in the work of L. Boltzmann [10] who studied the relation between entropy and probability in physical systems. Subsequently, entropy concepts and ideas have been introduced in physics, chemistry, meteorology, information science, etc. The maximum entropy function, which was introduced by Jayned [9] has proved to be useful for obtaining convergent upper bounds on nonsmooth objective functions.

Definition 1.1. Let p be a positive integer and let < : D ^ Rl(i = 1,... , k) be given continuously differentiable functions and let <(x) = maxi<i<k{^(x)}. The maximum entropy function <p : D ^ R1 of ... , is defined by

<p(x) = 1l^exp (p<i(x)) > . p It! J

Obviously, <p(x) is also continuously differentiable on D.

Lemma 1.1 (Jaynes [9]). Given £ > 0, zip > 0 such that Vp > p and Vx £ D

(1.1)

Also, if 1 < p < q, then

I<p(x) - <(x)l < £-

<q(x) < <p(x)-

(1.2)

(1.3)

Proof. Let Vj(x) = exp(<i(x)) (i = 1,...,k) and let v(x) = (vi(x))kx1. For each x £ D

(Vp > 1)

lv(x)lli

i= 1

1/p

^exp(p<i(x))

i=1

1/p

So

Now

<p(x) = ln(||v(x)y.

|v(x)||^ = lim ||v(x)||p = max{exp(<i(x))}.

1<i<fc

p

So

ln(IKx)ID = max {<i(x)} 1 <i<k

lim <p(x) = max ^(x).

p—x 1 <i<k

This proves (1.2). Finally, according to Jensen's inequality, (1 < p < q) ^ (||v(x)||q < ||v(x)||p), so ln(||v(x)||q < ln ||v(x)||p) whence <q(x) < <p(x). So (1.3) is proved. □

Lemma 1.2. Vx G D, we have

<(x) < <p(x) < <(x) + —. (1.4)

p

Proof.

k

(x)) > - <(x)

<p(x) - <(x) = 1 ln \ Y] exp(p^i (x)U - <(x) p Ife '

= "ln \ ¿exp(p(^i(x) - ^(x))) p 11=1

By definition of <(x), ^i(x) — <(x) < 0 (i = 1,... , k) and there is at least one j G {1,... , k}

k

such that <(x) — <(x) = 0. Therefore 1 < exp(p(^i(x) — <(x))) < k. Therefore we have

i= 1

1 k 1 0 < - ln < N exp(p(<i(x) — <(x))) > < - ln(k).

p 11=1 J p

So we complete the proof.

Because of differentiability and other good properties of maximum entropy function, it is extensively applied to the field of nonlinear programming, stochastic programming, nondifferentiable programming, multiobjective programming, etc.

Smooth Approximating Function. Let mi be the cardinality of set Ji (i = 1,..., n), so maximum entropy function of fij (x) (j G Ji) is defined as follows

Zip(x) = 1 ln \ exp (Pfij (x)) f (1.5)

P IjeJ )

ln mi

where p(> 1) is entropy factor. And Vx G D, we also have fi(x) < fip(x) < fi(x) +---,

p

fiq (x) < fip(x) V q > p > 1, i = 1,... ,n. And the function fip (x) is continuously differentiable on D.

From Lemma 1.1 and Lemma 1.2, we know that

lim fip(x) = fi(x), i = 1,... , n. p—

Namely, fip(x) is uniform approximation of fi (x). Obviously, it is reasonable and feasible that we substitute a differentiable function fip (x) for fi(x). With the idea of maximum entropy function, we can transform (0.1) to the approximation differentiable equations:

Fp(x) = 0, (1.6)

where Fp(x) = (/ip(x),/p(x),

/np(x)) •

Theorem 1.2. Given £ > 0, 3pk > 1, such that ( p = pk >

ln m

m = max (mil £ ' 1<i<1

||F(x) - F(x*)|U < £ or ||Fp(x) - Fp(x*)|U < £

where x is a solution of problem (1.6), x* is a solution of problem (0.1). Proof. Because x is a solution of problem (0.1), we have

/i(x*) = max/(x)| = 0, i = 1,..., n, i€ Ji

(1.7)

l/ip(x*) - /i(x*)| = |/ip(x*)|

p

ln {exp (p/ij(x*))

o'eJi

<

ln m, ln m

< -<-, i = 1,...,n. (1.8)

pp

And x is a solution of problem (1.6), so /ip(x) = 0 (i = 1,... , n). Then we can prove

ln '

yi(x) < /ip(x) < /i(x) +-', i = 1

l/i(x)| <

p

ln m, ln m -' < -,

n,

i = 1,..., n.

p

p

Let p = pfc >

ln m

from (1.8) and (1.9), we have

(1.9)

||F(x) - F(x*) ||Fp(x) - Fp(x*

||F(x)|U < £, = ||Fp(x*)|U < £.

So we complete the proof.

□

2. Algorithm and Convergence

Convergence. Let y G R(y = 0). We consider fixed point problem as follows

x, - Y/,p(x) = x,, i =1,...,n. (2.1) Obviously, fixed point problem (2.1) and problem (1.6) are equivalent. So we have

xf+1) = x(k) - Y(fc)/,p(x(fc)), i = 1,..., n, k = 0,1,... (2.2)

According to properties of maximum entropy function, it is clear that the solution of problem

(2.2) approaches the solution of problem (0.1) under the condition of iteration convergence with p ^

Let

G(x(k))

/ x1k) - y(fc)/ip(x(k))\

V xik) - Y(fc)/np(x(fc))/

then G (x(k)) = I - Y(k)U(k)M(k)

1

£

CO

CO

where U(k) = diag(uik),..., uifc)), u(k)

1

E exp(p/ij (x(k)) '

jeJi

( E <|exp(p/lj(x(k)))

M(k)

Then we have

jeJi

dx(

E |exp(p/ij(x(k))) jeJi I ax«

E exp(p/« (x(k)))

nj

\ jeJn

dx'

(k)

E <j exp(p/„j(x(k))) jeJ

nj

dx«k)

x

(k+i)

G(x(k)), k = 0,1,

(2.3)

Theorem 2.1 [8]. The sufficient condition of iteration convergence of (2.3) is p(G' (x(k))) < 1, where p(G'(x(k))) = y^max A((G'(x(k))T(G'(x(k))) is spectral radius of G'(x(k)).

Theorem 2.2. Suppose x* C int(D) is a fixed point of the map G : D C Rn ^ Rn. If

Y(k) < , (k), (k = 0,1,...), then Vx(0) G D, x* is an attracting point of iterative sequence

(2.3).

p(M(k))

Proof. Suppose A is an arbitrary eigenvalue of the matrix

(U(k) M(k))T U(k)M(k) = (M(k))T (U(k))2M(k) and z G Rn is an eigenvector corresponding to A, then (M(k))T(U(k))2M(k)z = Az and zT ((M(k))T (U(k) )2M (k))z (M(k)z)T (U (k))2(M (k)z)

A

z T z

zT z

nn

E u2(M(k)z)2 E maxi<i<n(Ui}2(M(k)z)2

i=i

zT z

<

i=1

zT z

<

(M(k)z)T (M(k)z) zT((M(k))TM(k))z

zT z

zT z

Obviously, we can deduce that p2(U(k)M(k)) < p2(M(k)) (namely p(U(k)M(k)) < p(M(k))), and Y(k) < p(M1(k)), such that p(7(k)U(k)M(k)) = Y(k)p(U(k)M(k)) < Y(k)p(M(k)) < 1. Therefore

p(G'(x(k))) = p(1 - 7(k)U(k)M(k)) < 1.

From Theorem 2.1, we know that Vx(0) G D, x* is an attracting point of iterative sequence (2.3).

Algorithm. With the preceding ideas, we suggest the following algorithm for solving problem (0.1).The detailed steps of the algorithm are as follows.

Step 1. Construct corresponding maximum entropy function and transform the nonsmooth equations (0.1) to the following smooth equations

/ip(x) = 1 P

■4 £

IjeJi

exp (P/ij(x))

, n.

Step 2. With given initial point x(0) G D and entropy factor p(> 1), solve the above smooth

equations by following iteration (letY(k) < ——ttt-

y p(M(k))

x(k+i) = x(k) _ y

(k)

P

lni^exp/(x(k))) I , i = 1,...,n, k = 0,1,...

o'eJi

i

1

Step 3. If ||F(x(fc))||^ > e and ||x(k) - x(k-1)|| > e, then go to Step 2. Step 4- Output end.

In the process of actual computing, we ought to notice the following three points:

ln

— according to Theorem 1.2, let p > -;

e 1

— in numerical computation, we can select y(k) = -—-¡Tr?;

' ||M (fc)||

— in order to accelerate the convergence speed of iteration, we can also improve the algorithm by applying Gauss — Seidel iterative technique:

(fc+1) (fc) 7(fc) ! I V^ f f ( (fc+!) (fc+!) (k) xi = xi--in<exp(p/ij(x1 ;))

P IjeJi

i = 1,..., n, k = 0,1,...

3. Numerical Results

The algorithm has been implemented using Matlab 5.2 on Pentium-IV computer. Numerical results for the following three examples are reported in this section, in each of which the actual minimum is explicitly known. The symbols p, e, x*, x(0), x denote the entropy factor, the precision of minimum value, the solution of equations (0.1),the initial iteration point, the approximate solution of equations (0.1). Example 1.

maxjAOr^ f2(x), f3(x)} = 0

where /1(x) = x(x — 2)(x — 8)(x — 4)(x — 6), /2(x) = x(2 — x)(x — 8)(x — 4)(x — 6), /3(x)

(x — 2)(x — 7), D = e = 1e — 4, p = 104,

as follows:

let x(0) = 1.90, let x(0) = 4.20, let x(0) = 5.70, Example 2.

[0, 6]. The solutions of above equation xi = 2.0, x* = 4.0, x* = 6.0. Let /(x) = max{/1(x), /2(x), /3(x)}, with the above algorithm, the result is

then then then

x

x x

2.000251, 3.998995, 5.999987,

/ (x) = —0.001255, / (x) = 0.064320, / (x) = 0.001248.

max{/i(x), f2(x), f3(x)} = 0

51

where /1(x) = x + 3, /2(x) = -x + 1, /3(x) = — (x + 4), D = [—6, 0]. The solutions of equation 5 2 2

x1 = —4.0, x2 = —. Let e = 1e — 4, p = 104, /(x) = maxxeD{/1(x), /2(x), /3(x)}, with the 6

above algorithm, the result is as follows:

let x(0) = —1.90, then x = —1.2.000, / (x) = 0.0000, let x(0) = —4.20, then x = —4.0000, / (x) = 0.0000. Example 3.

f11 (x) , f12(x)}\ /0' maX{/21(x),/22(x)W I0

where /11 (x) = + (x - 1)2 + - 1, /12 (x) = -x1 - - 1)2 + x2 + 1, /21 (x) = sin(xi + X2), /22 (x) = xi(xi + 3x2) + X2(X2 - xi) - 2, D =

" n n" " n n"

- 2 , 2. , - 2 , 2.

T

. The solution of equations

x* = (0, 0)T .With the above algorithm (e = 1e-4, p = 104) and let initial point x0 = (0.1, 0.1)T, the approximate solution is x = (-0.001861, 0.00027)T.

The examples above and more numerical results indicate that our algorithm works reliably and efficiently.

Conclusions

With the preceding study, to eliminate the defect of nondifferentiable problem with proper smoothing method is a kind of feasible method. Though the convergence speed of algorithm only reach linear convergence, the format of algorithm is simple, not need to compute derivative of nonsmooth functions F(x), require low memory capacity and be easy to implement in computer. Another point in the paper is that, Vx(0) £ D, the iterative sequence (2.3) is convergent with suitable Yk (k = 0,1,...), and so on. In articles [1, 2], algorithm requires to compute set of F(x(k)) and dbF(x(k)) which is hard to do in general (where the definition of F(x(k)), F(x(k)) is given in [1]. In articles [4, 6], one also requires to compute V/1j1 (x(k)),... , /njn(x(k)), where j £ Jj(x), defined in [4, 6]. In the paper, we improve the above defects.

A direction for future research is to investigate the existence and uniqueness theorems for solution of nonsmooth equations (0.1) within a given region of n-space, and give sufficient condition which guarantees that a given multidimensional interval contains exact solution of the problem (0.1). Another area for future research is to find additional theoretical convergence results for the new iteration formula and to find a way to produce the {x(k)} sequence converging to x* without a prior knowledge.

Acknowledgements: We thank both referees for their valuable suggestions and remarks which improve this paper.

References

[1] Ql L. Convergence analysis of some algorithms for solving nonsmooth equations // Mathmatics of Operations Research. 1993. Vol. 18. P. 227-224.

[2] Sun D., Han J. Newton and quasi-Newton for a class of nonsmooth and related problems // SIAM. J. Optimization. 1997. Vol. 7. P. 463-480.

[3] Nashed M.Z., Chen X. Convergence for Newton-like methods for singular operator equations using outer inverses // Numer. Math. 1993. Vol. 66. P. 235-257.

[4] Gao Y. Newton and quasi-Newton methods for nonsmooth equations of max-type function // Nature Magazine. 2001. Vol. 22. P. 61-63.

[5] Shen Z., Huang Z., Wolfe M.A. An interval maximum entropy method for a discrete minimax problem // Appl. Math. and Comput. 1997. Vol. 87. P. 49-68.

[6] Gao Y. Newton method and inexact-Newton method for solving singular nonsmooth equations // Math. Appl. 2002. Vol. 15. P. 57-61.

[7] Gao Y. Newton methods for solving nonsmooth equations via a new subdiffererntial // Mathmatics of Operations Research. 2001. Vol. 54.

[8] Ortega J., Rheinboldt W. Iterative solution of nonlinear equation in several variables. N.Y.: Acad. Press, 1970.

[9] Jaynes E.T. Information theory and statistical mechanics // Phys. Rev. 1957. Vol. 106. P. 620-630.

[10] Boltzmann L. Weitere studien über das warmegleichgewicht unter gasmolekulen // Wien. Akad. Sitz. 1972. Bd. 66. P. 275-370.

Received for publication April 18, 2006