CC BY
75
Вычислительные технологии
Том 8, № 2, 2003
SOLVING THE FINITE MIN-MAX PROBLEM VIA AN EXPONENTIAL PENALTY METHOD*
X. Li
Department of Engineering Mechanics, Dalian University of Technology
Dalian, China e-mail: Lixs@dlut.edu.cn
S. Pan
Department of Applied Mathematics, Dalian University of Technology
Dalian, China e-mail: pshcoral12@sina.com
В работе рассмотрен метод сглаживания для конечной задачи минимакса, которая возникает при применении метода экспоненциального штрафа к его эквивалентной нелинейной программе. В результате исходная задача может быть решена с помощью безусловной минимизации гладкой функции. Для полноты приведен формальный вывод для экспоненциальных функций штрафа и описаны некоторые важные приложения результирующих гладких функций. Предложенные результаты демонстрируют простоту и эффективность данного подхода.
Introduction
Consider the finite min-max problem (P) defined as
where
min ¿>(x), (1)
xeRn
0(x) := max {gi(x)}. (2)
1 <i<m
In this work, we assume that all component functions gl(x) : Rn ^ R, i = 1, 2,...,m are continuously differentiable. Nevertheless, this is a typical non-smooth optimization problem due to the nondifferentiability of the max function 0(x).
The problem defined in (1) is one frequently arisen in scientific and engineering computing [1-3], due to its wide applications such as data fitting, game theory and economic equilibria. Therefore, the study on its robust and efficient algorithms becomes a prolonged research subject.
One way to deal with this problem is to transform it into an equivalent nonlinear programming problem (NLP)
min w
w,x (3)
s.t. gl(x) — w < 0, i = 1, 2,...,m,
*This work is supported by special fund for basic research under grant G1999032805. © X. Li, S. Pan, 2003.
which is then solved by certain minimization algorithms [4-8]. In contrast, smoothing techniques [9-11] try to find a smooth function to approximate the non-smooth max-type function. The latter approach has apparent advantages over the former one because it is not only applicable to solving the min-max problem (P) itself, but also applicable to a variety of contexts, especially in the recent hot smoothing algorithms for the solution of complementarity, variational inequality and MPEC (mathematical programming with equilibrium constraints) problems.
The main purpose for this paper is to show that a smooth function with a uniform approximation property could be derived by means of an ordinary penalty method. This approach is surprisingly simple, but is not appeared in the literature from our search. At the same time, this paper is to show that a penalty function could be "derived", not just understood with "naive" sense. Such an approach could provide a new insight into penalty function methods. To this end, we add a formal derivation for the exponential penalty functions before applying it to the min-max problem.
The paper is organized as follows. Section 1 is devoted to deriving exponential penalty functions for the inequality constrained optimization problem by Lagrangian perturbation. In Section 2, we exploit the exponential penalty methods to solve the finite min-max problem (P) and discuss some properties of resulted smooth functions. In Section 3, several important applications of the smooth functions are described. In Section 4, we give a very simple algorithm and report the numerical results for some test problems from the literature. Finally, brief conclusions are drawn in Section 5.
1. Derivation of exponential penalty functions
The exponential penalty approach is one of the important optimization methods both in theoretical and algorithmic developments as an instrument for converting constrained problems into unconstrained ones. Nevertheless, it is rarely to make inquires about the origin of exponential penalty functions and their relations to other means that play the same role, for example the Lagrangian function. By deriving the exponential penalty functions through a Lagrangian perturbation procedure in this section, we are intended to establish a formal link of exponential penalty functions with Lagrangian function and give a new insight into exponential penalty methods.
Consider a general inequality constrained optimization problem (GNLP)
min f (x)
x€Rn (4)
s.t. gi(x) < 0, i = 1, 2,..., m,
where f (x) and gi(x), i = 1, 2,... , m are assumed to be continuously differentiable functions from Rn to R.
As a matter of fact, the above problem (GNLP) can be equivalently written as an unconstrained optimization problem in the following form:
min ^(x) := sup L(x, A) (5)
xeR" A>o
due to the following fact
' f (x) x e X,
sup L(x, A) = ^ , , v
A>o I x e X
where L(x, A) is the ordinary Lagrangian
L(x, A) := f (x) + ]T Ajgi(x), vx g Rn, A g R^
i=1
and X := {x gRn|gi(x) < 0, i = 1, 2,... , to} represents the feasible set of (GNLP). Although the formulation of (5) is very attractive, unfortunately, the linear form of L(x, A) in dual variable A prevents from doing this to get a continuous and analytical solution A(x).
To circumvent this drawback, we introduce an entropy function £ (A) = m=1 Aj ln Aj as a perturbing term into the Lagrangian L(x, A) to formulate a perturbed problem
m
sup {Lp(x, A) := f (x) + j] Ajgj(x) - p-1£(A)}. (6)
A>o ^ ¿=i
For this perturbation problem, we have the following theorem:
Theorem 1. The perturbation problem (6) yields a smooth function ^p(x) that pointwisely approximates to ^(x) in Rn as p^ so that the original problem (GNLP) is transformed into a smooth unconstrained optimization problem
min^„(x). (7)
xeRn
Proof. As the entropy function £(A) is strictly convex in the closed convex set Rm and coercive, i.e., for every sequence {Ak} g Rm such that ||Ak|| ^ it always holds
lim £(Ak) =
fc^TO
the problem (6) must have a unique solution A(x,p) for any x g Rn and the objective function Lp(x, A) attains a finite maximum at this point, thereby defines a function of variable x
%(x) := L(x, A(x,p)) = f (x) + p-1£*(pg(x)), (8)
where the last equality comes from the definition of conjugate function, £ * is the convex
(x) = ^gi(x),g2(x),... ,gm(x)) . Since £(A) is a function, i.e., £ g L(Rm), it follows from [12] that £* g L(int(dom£*)) is also a Legendre type convex function. Thus, considering ^p(x) is a real-valued function on Rn, we can infer that ^p(x) is smooth on the whole space Rn.
Furthermore, £(A) is low-bounded in Rm and £(0) = 0, so there holds
-1
conjugate of £ and g(x) = (gi(x), g2(x),... ,gm(x))T. Since £(A) is a Legendre type convex
0 < sup{£ Aigi(x) - p-1£(A)} < sup{£ A^(x)} + (9)
A>o ¿=i A>o ¿=i pe
In view of (5), this demonstrates
lim tfp(x) = tf(x), vx g X.
However, for xg X, i.e., there at least exists an index j such that gj(x) > 0,
^p(x) = f (x) + p-1 V" sup { A*(pgi(x)) - Ai ln A*} + +p-1 sup { Aj (pgj (x)) - Aj ln j ,
X, >0 I.
where as p ^ the last term in the right hand of last equation approaches to due to the boundedness of £(A), and the second term to 0 in terms of (9). Hence, ^p(x) approximates pointwise to ^(x) as p ^
This completes the proof. □
By a simple computation, £*(A*) = exp(A/ — 1). So, following the equation (8), we have the smooth function ^p(x) in the following form:
%(x) = f (x) + p-1 J] exp [pgi(x) — 1], (10)
-i
i=1
which is just the familiar exponential penalty function. As the unique solution of perturbed problem (6),
Ai(x,p) = exp [pgi(x) — 1, i = 1, 2,... ,m, (11)
which provides an estimate for the Lagrange multipliers of the original problem (GNLP). Other exponential penalty functions, up to a constant difference with that given by (10), are often encountered in the literature and could be similarly derived through slight changes of £ (A).
If Kullback — Leibler cross entropy £(A, y) := ^™1 Ai ln A is adopted as the perturbing
Vi
term, the exponential multiplier penalty function
m
tfp(x, y) = f (x) + p-1 J] ^ exp [pgi(x) — 1 (12)
i=1
can be derived in a very similar manner. The vector y (> 0) of Lagrange multipliers has to be updated as do the other methods of multipliers.
The above derivation shows that the traditional exponential penalty functions are conjugate to some specific perturbed Lagrangian functions in nature, and incorporate some excellent properties such as strict convexity, coercive and barrier role, which play a central role in the convergence analysis of exponential penalty methods [13, 14].
2. Smoothing methods for the finite min-max problem (P)
Now we are going to the main scheme of this paper, using the exponential penalty functions to solve the finite min-max problem (P). Here, we apply the exponential penalty function ^p(x) given by (10) to its equivalent formulation (NLP), whereby transform the constrained problem into the following unconstrained one:
min^p(w, x) = w + p 1\ exp p(gi(x) — w) — 1
(13)
i=1
Since ^p(w, x) is a convex function of variable w for any x g the optimality condition for w to be a solution of (NLP) is
S^p(w, x)/dw =1 — ^^ exp p(gi(x) — w) — 1
i=1
0
which means the following relationship
m
w* = p-1 ln exp — p-1 (14)
i=1
holds between the optimal variables w* and x. Substituting it into (w, x) to eliminate the variable w yields
m
0p(x) := ^p(w*, x) = p-1 lnj^exp [pgj(x)]. (15)
i=1
Accordingly, the problem (13), hence the min-max problem (P), reduces to
min 0p(x) (16)
xeRn
due to the equivalence between (NLP) and (P). Thus, the original non-smooth problem (P) is converted into an unconstrained minimization problem with a smooth objective function 0p(x). This greatly facilitates the numerical solution.
Concerning the difference between 0(x) and 0p(x), we give the following properties [11]:
1) error bound: 0(x) < 0p(x) < 0(x) + ln(m)/p ;
2) monotonicity: 0p(x) > (x), p < q;
3) approximation: lim 0p(x) = 0(x);
4) convexity: 0(x) is convex if all gj(x), i = 1, 2,... , m are convex.
The above property of uniform approximation ensures that an e—optimal solution of the minmax problem (P) can be found by solving the problem (16) as long as p > ln(m)/e. Meanwhile, the following formula
Ai(x) ^mXP ^ (x)) o , i = 1, 2,---,m
Z^i=1 exP [pgi (x)J
provides an adaptive estimate for Lagrange multipliers of the problem (P) during iterations.
In addition, when applying the exponential multiplier penalty function ^p(x, y) defined in (12) to the problem (NLP), we can obtain, by a similar procedure, another smooth function
-1 (x)
0p(x, ß) = p 1 In Y^ exp [pgi(x)] .
i=1
where m=1 ^ = 1, ^ > 0, i = 1, 2,... , m.
Though the smooth functions 0p(x) and 0p(x, y) could be derived by some other procedures (see [9-11]), the derivation given in this paper is very elegant and does not resort to any complex concept.
Compared with 0p(x), the smooth function 0p(x, y) has an advantage of avoiding the ill-conditioning caused when p ^ with extra cost of multiplier updates. To our experiences, however, there exists a "zero" pitfall for using 0p(x, y), which may result in the instability even premature termination of algorithms at a non-optimal solution. Therefore, use of this smooth function needs very sophisticated care for algorithmic implementation.
3. Some applications of smoothing function 0p(x)
As aforementioned, the smoothing method has wide applications other than min-max problem. In this section we describe some applications which are directly or indirectly related to the non-smooth max function 0(x). By the word "indirectly", we mean that there are certain problems where the max function stems from some reformulations to meet theoretical or computational requirements. In most of circumstances, the smooth functions 0p(x) is more tractable than 0p(x, as commented in the end of last section and its appealing features have made it popular in many recent smoothing algorithms.
3.1. Linear programming
At first, we consider a linear programming problem in Karmarkar's standard form (KLP):
min{cTx : Ax = 0, eTx =1, x > 0},
x
where c g Rn, e = (1,1,..., 1)T g Rn, and A g Rmxn is a full row rank matrix with elements a,, i =1, 2,... , m, j = 1, 2,... , n. It is easy to show that its dual problem (KLD) is
max z z,y
s.t. ATy + z < c,
which is in fact the following linear min-max problem
m
min max < > a,yi — c,
y 1<j<n I
i=1
and therefore could be solved by the following unconstrained minimization
n m
ln^exp yi — c,
min p 1 y
j=1 i=1
Both dual variable y(p) and primal variable x(p), as Lagrange multipliers of the dual problem, recovered from the formula
(P)
exp p V^m A 2-ii=1 aij yi — cjy
n E,=1 exp p( E™1 ai, yi— c,)
j = 1, 2,...,n
converge, respectively, to the optimal solutions of dual problem (KLD) and primal problem (KLP), when the penalty factor p ^ (see [14]). For the standard linear programming problem (LP):
min{cTx : Ax = b, x > 0}
x
and its dual problem (LD):
max{bTy : ATy + s = c, s > 0},
y,s
where b g matrix A and vector c are the same as above. The K — K — T conditions for both primal and dual problems are
Ax = b,
ATy + s = c, (17)
xs = 0, x > 0, s > 0,
where xs denotes the coordinate-wise product of vectors x and s. With resort to NCP function min(a,b}, the above complementary conditions can be replaced equivalently by the following equation
minjxi, s} = — max{—x, — s} = 0, i =1, 2,..., n, which is further smoothened to
p-1 ln [exp(— pxj) + exp(— ps)] =0, i =1, 2,..., n
through the smooth function 0p(x). The resulted smooth system of equations can be solved by the classical Newton's method. With such a reformulation, we have developed a primal-dual non-interior point algorithm.
3.2. Nonlinear programming
Consider the inequality constrained nonlinear programming problem (ICP):
min fo(x)
x J0V ; (18)
s.t. /¿(x) < 0, i = 1, 2,..., m,
where /¿(x) : Rn ^ R, i = 0,1,...,m are assumed to be continuously differentiable. The presence of inequality constraints brings the main difficulty to its solution.
Contrast with general penalty methods that normally require to solve a sequence of unconstrained problems, the exact penalty methods need only one unconstrained minimization provided that the penalty factor exceeds a threshold. This means a lot of savings for computational efforts. However, smooth exact penalty functions either involve second derivatives or are very complicated whereas two simple exact penalty functions, L1 and L^ exact penalty functions, are not differentiable.
For the L1 exact penalty method, (ICP) is transformed into the following non-smooth unconstrained optimization problem:
m
min /o(x) + a^ max {0, /¿(x)},
¿=i
whose solution becomes exactly optimal for the original problem as long as the penalty parameter a > max A*, where A*, i = 1, 2,... , m, denote the optimal Lagrange multipliers of original
1<i<m
problem (18). For the L^ exact penalty method, (ICP) is converted to
min /o(x) + a max {0, /i(x), ■ ■ ■ , /m(x)}
X
and the penalty factor a > m=1 A* is required in this case. Replacing the above max-valued functions with their smooth substitutes through 0p(x), we can get an approximate solution of
(ICP) by solving the smooth optimization problem
i=1 or
m
min /o(x) + - V" ln 1 + exp (p/¿(x)) (19)
x p
110
min fo(x) + a ln {l + J>xp[p/t(x)]}. (20)
x p i=i
Thus, the smooth function 0p(x) makes it possible to solve the nonlinear programming problem with these non-differentiable exact penalty functions in practice. Following this line, we have proposed a quasi-exact penalty function method for solving (ICP) in [15].
Another way to resolve the difficulty of inequality constraints is that all constraints of (ICP) are replaced by a single constraint, through the maximum operator, to formulate an equivalent problem
min /o (x),
si 0(x) := max {/¿(x)> < 0 (21)
and then the non-smooth 0(x) is replaced by its uniform approximation 0p(x). As such, the problem (ICP) with multiple constraints can be approximately solved by solving the following singly-constrained smooth one:
min f0(x),
x J0K h (22)
s.t. 0p(x) < 0.
In [16] such an approach is termed as the aggregate function method.
3.3. Complementarity problem
Consider the vertical nonlinear complementarity problem (VNCP):
m
x > 0, Fi(x) > 0, ■ ■ ■ , Fm(x) > 0, x? JJ Fj (x) = 0,
i=1
where Fj(x) : Rn ^ R, i = 1, 2,... ,m are vector-valued functions and Fj(x) denotes the jth component of Fj(x). Obviously, in a special case of m = 1, the problem (VNCP) reduces to a general nonlinear complementarity problem (NCP):
x > 0, F1(x) > 0, xTF1(x) = 0.
It is easy to verify that the problem (VNCP) is equivalent to the following non-smooth equations:
min(xj,Fj (x),---,Fm(x)} =
= - max{-x?,-Fj (x),..., -Fm (x)} = 0, j = 1, 2,...
Once again, one can replace the maximum operator here by smoothing approximation 0p(x) and convert it into a smooth system of equations. A non-interior continuation method for generalized linear complementarity problems is developed in [17].
3.4. Box constrained variational inequality problem (BVIP)
This problem is to find an x g [l, u] such that
(y - x)TF(x) > 0 vy g [l, u],
where [l, u] is a box constraint in Rn with l < u. It is easy to show that the problem (BVIP) is equivalent to the system of equations:
x — mid{l, u, x — F(x)} = mid{x — l, x — u, F(x)} = 0,
where the mid operator mid{a, b, c} can be represented by
mid{a, b, c} = a + b + c — min{a, b, c} — max{a, b, c}.
Here the max and min operators could be replaced by smoothing approximations with 0p(x) in appropriate forms.
Sample applications given above are all hot topics of recent research. In fact, the smooth function 0p(x) has been extensively used in other areas such as multi-objective optimization, bi-level programming, random programming, etc.
4. Algorithmic implementation and numerical results
It should be stressed that, with our methods, the solution of min-max problem (P) becomes extremely simple because it is reduced to unconstrained minimization problem (16) with smooth function 0p(x) and 0p(x,y). Thus, any efficient unconstrained optimization algorithms (see BFGS) can be applied to this solution. This will greatly facilitate practical usage for engineers. The computations of given examples serves as an illustration for the simplicity, efficiency, stability, high accuracy and easy implementation of our algorithm. In our computations, the parameter p is deliberately set at a fixed constant for three reasons. At first, it is to show that a high accurate solution can be found by a single minimization with a sufficiently large p. Secondly, it is to demonstrate that the ill-conditioning of 0p(x) with a quite large p is not as severe as the theory claims. Third is to clarify a misunderstanding on exponential manipulations involved in the function 0p(x) and its gradient v0p(x). In fact, the computer overflow could be completely eliminated from computing through the following conversions:
0p(x) = 0(x) + p 1 lnj^exp p(gi(x) - 0(x))
¿=i
v0p(x) = £ Aivgi(x),
¿=i
where 0(x) denotes the maximum among all the component functions at current iteration and
Ai
exp pMx) - 0(x))
Ei=i exp pMx) - 0(x))
At this point, it should be noted that our smoothing algorithms have a fundamental distinction with nonlinear re-scaling algorithms proposed by Polyak [18], where the finite min-max problem (P) is replaced by unconstrained minimization of
which is defined as Lagrangian function for an equivalent problem obtained by certain exponential transformations on original constraint functions. Although both kinds of algorithms have mimic objective functions and same optimal solution x*, the overflow of exponential operations is hard to be avoided without our logarithmic operator in L(x, ^,p). Furthermore, the "zero" pitfall on multiplier updates will also occur when using L(x, ^,p), leading to premature termination of algorithms. In the computation of all examples below, we only use the smooth function 0p(x) which has favorable properties and is the most tractable one from our experiences.
Based on the above fact, we establish a simple algorithm for solving the min-max problem (P) as follows:
Algorithm 1
(5.0): Given any initial point x0 and set p = ln(m) • (1.0e + 5).
(5.1): Apply BFGS algorithm to performing a unconstrained minimization of 0p(x) and let x denote the minimizer.
The examples given below are all taken from some renowned authors' work, where they are calculated by some sophisticated algorithms. Preliminary numerical results of Algorithm 1 are reported in sequel, where x* denotes the optimal point of problem and x is the minimizer yielded by the above algorithm.
Example 1[19]
gi(x) = x^ + x2, g2(x) = (2 - xi)2 + (2 - X2)2, g3(x) = 2exp[-xi + X2], x* = (1.13904, 0.89956), 0(x*) = 1.95222, x0 = (1, -0.1), x = (1.139037261, 0.89955975), 0p(x) = 1.95223071. Example 2[19]
gi(x) = x^ + x2, g2(x) = (2 - xi)2 + (2 - x2)2, g3(x) = 2exp[-xi + x2], x* = (1,1), 0(x*) = 2, x0 = (2, 2), x = (1.00000105,0.99999750)T, 0p(x) = 2.0. Example 3[19]
gi(x) = 5xi + x2, g2(x) = -5xi + x2, g3(x) = xi + x2 + 4x2, x* = (0, -3), 0(x*) = -3, x0 = (1,1), x = (0.00000000,-3.00000000), 0p(x) = -2.99999000. Example 4[19]
gi(x) = xi + x2, g2(x) = gi(x) + 10(-4xi - x2 + 4), g3(x) = gi(x) + 10(-xi - 2x2 + 6), x* = (1.2, 2.4), 0(x*) = 7.2, x0 = (-1, 5), x = (1.20000021, 2.40000042), 0p(x) = 7.20000502. Example 5[19]
m
gi(x)
xi + x2^i
eyi (1 < i < 21),
1 + x3yi + x4y2 + x5y3
, xi + x2 yi
yl = — 1 + 0.1 (i - 1),
x* = (0.999878,0.253588, —0.746608, 0.245202, —0.037490), 0(x*) = 0, x0 = (0.5, 0, 0, 0,0), x = (0.99987755,0.25362535, —0.74656767, 0.24515748,
— 0.03747228), ¿p(x) = 0.00012714. Example 6[19]
0(x) = —xi + 2(x^ + x2 — 1) + 1.75 max {g^(x)},
gi(x) = xi + x2 — 1, g2(x) = — (xi + x2 — 1),
x* = (1, 0), 0(x*) = —1, x0 = (—1, —1),
x = (0.99999471,0.00000000), 0p(x) = —0,99999000.
Example
gi(x) = xi + x2 + xix2, g2 (x) = —gi(x), g3(x) = Sin(xi),
g4(x) = —g3(x), g5(x) = COs(x2) , g6(x) = —g5(x),
x* = (0.453296, —0,906592), 0(x*) = 0.61643246, x0 = (3,1), x = (0.45329553, —0.90659105), 0p(x) = 0.61643610. Example 8[2]
gi (x) = (1/2)[xi + 10xi/(xi + 0.1) + 2x2], g2 (x) = (1/2)[—xi + 10xi/(xi + 0.1) + 2x2], g3(x) = (1/2)[xi — 10xi/(xi + 0.1) — 2x2], x* = (0, 0), 0(x*) = 0, x0 = (3,1), x = (—0.00073970, 0.19207034), ¿(x) = 0.00000631. Example 9[2]
gi(x) = [xi — \Jx2 + x2 cos {\Jx2 + x2)]2 + 0.005(x2 + x2), g2(x) = [x2 — \Jx2 + x2 sin {-\Jx2 + x2)]2 + 0.005(x2 + x2), x* = (0, 0), 0(x*) = 0, x0 = (1.41831, —4.79462), x = (0.00000152,0.00000000), ^p(x) = 0.00001000. Example 10[8]
gi(x) = exp[x2/1000 + (x2 — 1)2], g2 (x) = exp[x2/1000 + (x2 + 1)2], x* = (0, 0), 0(x*) = e, x0 = (1.5, 0.05), x = (0.00000002,0.00000000), 0p(x) = 2.71829183. Example 11[8] gi(x) = g(x + 2e2), g2(x) = g(x — 2e2),
g(x) = exp[0.0001xi + x2 + x2 + 2x4 + x5 +-----+ xi0 ],
e2 = (0,1, 0, 0, 0,0, 0, 0, 0, 0),
x* = (0,..., 0), 0(x*) = 54.591846, x0 = (100, 0.1,..., 0.1),
x = (0.00000000, —0.00000000, —0.00000000, —0.00000000, 0.00000000, 0.00000000, 0.00000000, —0.00000000,
— 0.00000000, 0.00000000), 0p(x) = 54.59816003.
Example 12[8]
ii 1
gi(x) = V ——— exp[xt — sin(i — 1 + 2(j — 1))]2, (1 < i < 10), j — 1 + *
x* = (0.012427,0.290377, —0.33466, —0.126514,0.233137,
— 0.276568, —0.166612, 0.229147, —0.185807, —0.170438, 0.240165), 0(x*) = 3.703483, x0 = (1,1,..., 1),
x = (0.01244922,0.29069248, —0.33460669, —0.12644597, 0.23281000, —0.27653988, —0.16658122, 0.22909022, —0.18291420,
- 0.17044129, 0.24017750), 0p(x) = 3.70348527.
Example 13[8] t = xi - (x4 + 1)4,
gi(x) = t2 + (x2 - t4)2 + 2x3 - 5t - 5(x2 - t4) - 21x3 + 7x4,
g2(x) = gi(x) + 10[t2 + (x2 - t4)2 + x3 + x2 + t - (x2 - t4) + x3 + x4 - 8 ,
g3(x) = gi(x) + 10 =t2 + 2(x2 - t4)2 + x3 + 2x4 - t - x4 - 10 ,
g4(x) = gi(x) + 10 ft2 + (x2 - t4)2 + x2 + 2t - (x2 - t4) - x4 - 5 ,
x* = (0,1, 2, -1), 0(x*) = -44, x0 = (0, 0, 0, 0),
x = (-0.00000004, 0.99999996,1.99999986, -0.99999980),
0p(x) = -43.9999942.
Conclusion
In this paper we deals with a long-last research topic, the min-max problem. Due to its wide applications, many algorithms have been proposed. From previous sections, however, it is clear that the present algorithm is the simplest one compared with others and has other features favorable in numerical computation. Contributions of this work may be summarized in several aspects:
1. The derivation of exponential penalty function ^p(x) and the exponential multiplier penalty function ^p(x, through our Lagrangian perturbation approach helps with establishing a formal link between two different kinds of important functions, Lagrangian and penalty functions. This does not only give a new insight into the penalty function methods, but also suggests a way to construct useful penalty functions.
2. The derivation of smooth functions 0p(x) and 0p(x, for the finite min-max problem (P) through directly applying ordinary exponential penalty functions to its equivalent problem (NLP) is very simple and clean.
3. The algorithm based upon the smooth function 0p(x) makes it very simple to solve the non-differentiable min-max problem. It is superior to other algorithms, for instance nonlinear re-scaling.
4. The description for some applications of smooth functions may have effect on widening reader's fields of vision.
References
[1] POLAK E. On the mathematical foundations of nondifferentiable optimization // SIAM Rev. 1987. Vol. 29. P. 21-89.
[2] POLAK E., HlGGINS J.E., MAYNES D.Q. A barrier function method for minimax problems // Math. Program. 1992. Vol. 54. P. 155-176.
[3] ROCKAFELLAR R.T. Computational schemes for large-scale problems in extended linear-quadratic programming // Math. Program. 1990. Vol. 48. P. 447-474.
[4] CHARALAMBOUS C., CONN A.R. An efficient method to solve the minimax problem directly // SIAM J. Numer. Anal. 1978. Vol. 15. P. 162-187.
[5] Fletcher R. A model algorithm for composite nondifferentiable optimization problem // Math. Programming Study. 1982. Vol. 17. P. 67-76.
[6] HAN S. Variable metric methods for minimizing a class of non-differentiable functions // Math. Program. 1981. Vol. 20. P. 1-13.
[7] MuRRY W., Overton M.L. A projected Lagrangian algorithm for nonlinear minimax optimization // SIAM J. Sci. Statist. Comput. 1980. Vol. 1. P. 345-370.
[8] Polak E., Maynes D.Q., Higgins J.E. Superlinear convergent algorithm for minmax problem //J. Optim. Theory Appl. 1991. Vol. 69. P. 407-439.
[9] Ben T.A., Teboulle M. A smoothing technique for non-differentiable optimization problems // Lecture Notes in Math. 1405. B.: Springer-Verl. 1989. P. 1-11.
[10] Bertsekas D.P. Approximation procedures based on the method of multipliers //J. Optim. Theory Appl. 1977. Vol. 23. P. 487-510.
[11] Li X. An entropy-based aggregate method for minimax optimization // Eng. Optim. 1992. Vol. 18. P. 277-285.
[12] Rockafellar R.T. Convex Analysis. Princeton: Princeton Univ. Press, 1970.
[13] Comimetti R., DuSSAult J.P. Stable exponential-penalty algorithm with superlinear convergence //J. Optim. Theory Appl. 1994. Vol. 2. P. 285-309.
[14] Comimetti R., Martin J.S. Asymptotic analysis of the exponential penalty trajectory in linear programming // Math. Program. 1994. Vol. 67. P. 169-187.
[15] Li X. A smooth quasi-exact penalty function for nonlinear programming // Chin. Sci. Bull. 1991. Vol. 37. P. 806-809.
[16] Li X. An aggregate function method for nonlinear programming // Sci. China Ser. A. 1991. Vol. 34. P. 1467-1473.
[17] Peng J., Lin Z. A non-interior-point continuation method for generalized linear complementarity problems // Math. Program. 1999. Vol. 86. P. 533-563.
[18] POLYAK R., GrivA I., SOBIESzczANSKI-SOBIESKI J. Nonlinear rescaling in discrete minimax. Manuscript, Department of SEOR and Math. Sci. Jorge Mason Univ., 2001.
[19] PlLLO G.D., GrippO L. A smooth method for the finite minimax problem // Math. Program. 1993. Vol. 60. P. 187-214.
Received for publication November 4, 2002
