CC BY
10
1. Управляемые системы и методы оптимизации
УДК 519.853.32
© Т. Баяртугс, А. Энхболор, Р. Энхбат КВАДРАТИЧНАЯ МИНИМИЗАЦИЯ И МАКСИМИЗАЦИЯ
В статье мы рассматриваем квадратичное программирование, которое состоит из квадратичной максимизации и квадратичной минимизации. Основываясь на условиях оптимальности, мы предлагаем алгоритмы для решения этих задач.
Ключевые слова: квадратичная максимизация, квадратичная минимизация, алгоритм, сходимость.
T. Bayartugs, A. Enkhbolor, R. Enkhbat QUADRATIC MINIMIZATION AND MAXIMIZATION
In this paper we consider the quadratic programming which consists of convex quadratic maximization and convex quadratic minimization. Based on optimality conditions, we propose algorithms for solving these problems.
Keywords: quadratic maximization, quadratic minimization, algorithm, convergence.
1. Introduction
Consider an extremum problem of a quadratic function over a box D e Rn: f (x) = {Cx , x} + (d, x} + q ^ max(min), x e D, (1.1)
where C is an n x n matrix, d, x e Rn, and D is a box set of Rn. Here (•, •)
denotes the scalar product of two vectors.
Quadratic programming plays an important role in mathematical programming. For example quadratic programming surve as auxiliary problems for nonlinear programming in its linearized problems or in optimization problems approximated by quadratic functions. Also this has many applications in science, technology, statistics and economics. Moreover, many combinatorial optimization problems are formulated as quadratic programming including integer programming, quadratic assignment problems, linear complementary problems and network flow problems [7]. There are a number of methods for solving problem (1.1) as convex problem such as the interior point methods, the projected gradient method, the conditional gradient method, the proximal algorithm, penalty methods, finite step algorithm and so on [1, 9, 13]. Then well known optimality condition for problem (1.1) is in Rockafellar [10]. Also, the quadratic maximization problem is known as «NP» problem. There are many methods [3, 11, 12] and algorithms devobed to solution of the quadratic maximization over convex sets.
The paper is organized as follows. In section 2 we consider quadratic convex maximization problem and apply global optimality condition [11] to
this. We propose some finite algorithms by approximation of the level sets of the objective function with a finite number of points and solving linear programming as auxiliary problems. In section 3 we consider the quadratic minimization problem over box constraints and recall the gradient projection method for solving this problem. In the last section we present numerical solutions obtained by the proposed algorithms for quadratic maximization and minimization problems.
2. Quadratic Convex Maximization Problem
2.1. Problem Statement and Global Maximality Condition
Consider the quadratic maximization problem.
f (x) = (Cx , x} + {d, x} + q ^ max, x e D, (2.1)
where C is a positive semidefinite (n x n) matrix, and D e Rn is a box set of Rn. A vector d e Rn and a number q e R are given. Then optimality conditions [12] can be also applied by proving the following theorem.
Theorem 2.1 [12] Let z e D be such that f'(z) ^ 0. Then z is a solution of problem (2.1) if and only if
(f '(y),x - y) < 0 for all y e Ef (z)(f) and x e D, (2.2)
where Ec (f) = {y e Rn |f (y) = c}.
Proof. Necessity. Assume that z is a global maximizer of problem (2.1) and let y e Ef (z)(f) and x e D. It is clear f '(y) ^ 0. Then the convexity of
f implies that
0 > f (x) - f (z) = f (x) - f (y) >{ f'(y), x - y)
Suffciency. Suppose, on the contrary, that z is not a solution to problem (2.1); i.e., there exists an u e D such that f (u) > f (z). It is clear that the
closed set Lf {z)(f) = {x e Rn | f (x) < f (z)} is convex. Since int Lf {z)(f) = 0
then there is the projection y of u on Lf (z }(f) such that
lly - ull = min llx - Ul
11 11 xeEf (z)(f)" 11
Clearly,
||y - HI > 0 (2.3)
holds because u i Lf (z) (f). Moreover, this y can be considered as a solution of the following convex minimization problem:
1 II ||2
g(x) = -||x - u|| ^ min, x e Ef(2)(f) Applying the Lagrange method to this latter problem, we obtain the
following optimality condition at the point y:
4 > 0,4> O,^,, + 4> 0 <4 g (y) + 4f (y) = 0 (2.4)
4(f (y) - f (z)) = 0 If 40 = 0, then (2.4) implies that 4> 0, f (y) = f (z) and f'(y) = 0. Hence we conclude that z must be a global minimizer of f over Rn, which contradicts the assumption in the theorem. If 4 = 0, then we have 4 > 0 and g'(y) = y - u = 0, which also contradicts (2.3). So, without loss of generality, we can set 40 = 1 and 4 > 0 in (2.4) to obtain
y - u + 4 f (y) = 0, 4 > 0,
f (y) = f (z)
From this we conclude that (f'(y),u - yj > 0, which contradicts (2.2).
This last contradiction implies that the assumption that z is not a solution of problem (2.1) must be false. This completes the proof.
Sometimes it is also useful reformulate Theorem 2.1 in the following form. Theorem 2.2 Let z e D and rank(C) ^ rank(C | d'). Then z is a solution of problem (2.1) if and only if
(f '(y),X - y) < 0 for all y e Ef (z)(f) and x e D, (2.5)
where (C | d') is the extended matrix of the matrix C by the column dt.
2.2. Approximation of the Level Set
Furthermore, to construct a numerical method for solving problem (2.1) based on optimality conditions (2.2) we assume that C is a symmetric positive defined n x n matrix. Then problem (2.1) can be written as follows.
f (x) = {Cx ,x} + (d,x) + q ^ max, x e D , (2.6)
where D = {x e Rn | a < x <&},and a, b, d e Rn, q e R . Now introduce the definitions.
Definition 2.1. The set Ef (z)(f) defined by Ef (z)(f) = {y e Rn | f (y) = f (z)} is called the level set of f at z.
Definition 2.2. The set Am defined by Amz ={y1,y2,...,ym | y e Ef (z)(f)j is called the approximation set to the level set Ef {z)(f) at the point z.
Note that a checking the optimality conditions (2.2) requires to solve linear programming problems:
(f'(y), x - y) ^ max, x e D for each y e Ef {z)(f). This is a hard problem. So we need to find an
appropriate approximation set such that one could check the optimality conditions at a finite number of points.
The following lemma shows that finding a point on the level set of f (x) is computationally possible.
Lemma 2.1. Let a point z e D and a vector h e Rn satisfy (f' (z),h) < 0. Then there exists a positive number a such that z + ah e Ef (z)(f).
Proof. Note that ^Ch,h) > 0, and
(2Cz + d, h) < 0 (28) Construct a point ya for a > 0 defined by
ya _ z + ah .
Solve the equation f (ya) = f (z) with respect to a. In fact, we have
{^a y a + (d, ya) + q _ f (z)
or equivalently,
(C (z + ah), z + ah) + {d, z + ah) + q _ (Cz, z) + {d, z) + q hich yields
__ ( 2Cz + d, h) a= (Ch, h)
By (2.8), we have a > 0 and consequently, ya e Ef (z)(f). For each y' e Am, i _ 1,2,...,m solve the problem
(f'(y'), ^ ^ max, x e D . (2.9)
Let u], j _ 1,2,...,m be solutions of those problems which always exist due to their compact set D:
(f'(y'),u') _ maxXeD(f'(y'),^ . (2.10)
Refer to the problems generated by (2.9) as auxiliary problems of the A'^. Define 0m as follows:
dm _ max If '(yj),u' -y)
j_1,2,...,m \ '
The value of dm is said to be the approximate global condition value. There are some properties of A^ and dm.
Lemma 2.2. If for z e D there is a point yk e A^ such that (f (yk),uk - yk) > 0 Then
f (uk) > f (z)
holds, where uk e D satisfies (f'(yk),uk) _ niax(f'(yk),^ Proof. By the definition of uk, we have
max(f'(yk),x-yk) _(f'(yk),uk -yk)
Since f is convex, we have
f (u) - f (v) >(f '(v),u -v)
for all u,ve Rn [8, 13]. Therefore, the assumption in the lemma implies that
f (uk) - f (z) = f (uk) - f (/) > ( f'(yk),uk - yk) > 0. Let z = (Zl,z2,...,Zn) be a local maximizer of problem (2.6). Then due to [10], z = a. vb, i = 1,2,...,n. In order to construct an approximation set take the following steps.
Define points v1,v2,...,vn+1 by formulas
zt if i = k
akif zk = ak i,k = 1,2,3,...,n , (2.12)
bkif zk = bk
vk =
and
Clearly,
1 lb if z = bt
v,"+1 =\ ' ' i = 1,2,3,..., n.
\aif zi = a
\\vn+1 - z > \\vk - A, k = 1,2,...,n,
(2.13)
Z(a, -b,)2 =1
n+1
v - z
i=1
Denote by h' vectors h' = V - z, i = 1,2,...,n +1. Note that
(hk,hj) = 0, k * j, k, j = 1,2,...,n .
Define the approximation set Д^1 by
Am ={/, y у ,..., ym | y e Ef ( z )( f ), у' = z, i = 1,2,..., n} (2.14)
(2Cz + d, hi)
-Ц-—T-i-, i = 1,2,..., n +1.
where a = -
{Chi, hi)
Then an algorithm for solving (2.6) is described in the following. Algorithm MAX
Input: A convex quadratic function f and a box set D.
Output: An approximate solution x to problem (2.6); i.e., an approximate global maximize of f over d .
Step 1. Choose a point x0 e D . Set k := 0.
Step 2. Find a local maximize zk e D the projected gradient method starting with an initial approximation point xk.
Step 3. Construct an approximation set at the point zk by
zk
formulas (2.12), (2.13) and (2.14).
Step 4. For each y e A"k+1,i = 1,2,...,n +1, solve the problems
У = z'-
( f'(y'), x) ^ max, x e D which have analytical solutions u',' _ 1,2,...,n +1 found as
\bs'f (2Cy' + d) > 0, u' _ < *
4 \as'f (2Cy' + d) < 0.
where ' _ 1,2,...,n +1 and s _ 1,2,...,n . Step 5. Find a number j e{1,2,..., n +1} such that
dm _(f'(y'), u' - y') _ max lf'(yj), u' - yj)
» ' j _1,2,...,n+1 » '
Step 6. If f (uj) > f (zk) then xk+1 :_ uj, k:_ k +1 and go to step 1. Step 7. Find a y e Ef{ }(f) such that
(2Czk + d,uj - zk) . , ■y^—-r-^(u - zk )
{2C(uj - zk),uj - zky '
Step 8. Solve the problem (f'(y), x) ^ max, x e D . Let v be solution, i.e.,
Step 9. If (f' (y),v - y) > 0 then xk+1:_ v, k:_ k +1 and go to step 1. Otherwise, zk is an approximate maximizer and terminate.
Theorem 2.2. If dnk+1 > 0 for k _ 1,2,... then Algorithm MAX converges to a global solution in a finite number of steps.
Proof immediate from lemma 2.2 and the fact that convex function reaches its local and global solutions at vertices of the box set D.
3. Quadratic Convex Minimization Problem
3.1. Problem Statement and Global Minimality condition
Consider the quadratic minimization problem over a box constraint.
f (x) _ (Cx, x) + (d, x) + q ^ min, x e D,
(3 1)
D _{x e Rn | at < xt < bt,' _ 1,2,...,n}.
where C is a symmetric positive semidefinite n x n matrix and and
a,b,d e Rn, q e R .
Theorem 3.1. [1] Let z e D . Then z is a solution of problem (3.1) if and only if
(f'(z), x - z) > 0 for al x e D (3.2)
We propose the gradient projection method for solving problem. Before describing this algorithm denote by PD (y) projection of a point
y e Rn on the box set D which is a solution to the following quadratic programming problem
||x - y|| ^ min, x e D ,
We can solve this problem analytically to obtain its solution as follows.
'arif y < a
(Pd (y) )i =|ji if a, < yi < b i = 1,2,..., n (3.3)
b,ifyi ^ b
It is well known that for the projection PD the following conditions hold [1].
(Pd(y) - y,x -Pd(j)} > 0 Vx e D (3.4)
We show that how to apply the gradient projection method for solving problem (3.1). It can be easily checked that the function f (x) defined by (3.1) is strictly convex quadratic function. Its gradient is computed as:
f'(x) = 2Cx + d
Lemma 3.1. The gradient f'(x) satisfies the Lipshitz condition with a constant L = ^|C||.
Proof. Compute ||f (u) - f (v)|| for arbitrary points u, v e D . Then we have ||f '(u) - f (v)|| = C(u - v)|| < 2||C||||u - v|| which completes the proof.
The gradient projection algorithm generates a sequence xk in the following way.
xk+1 = Pd(xk -aj(xk)), k = 0,1,2,...,x0 e D
where f (x*+') < f(xk).
Theorem 3.2. Assume that a sequence {xkjeD is constructed by the gradient projection algorithm, in which ak = a, k = 0,1,2,..., 0 <aL<L. Then a sequence ixk} is a minimizing sequence, i.e., lim f (xk) = min f (x).
n^w x*eD
Proof. By the algorithm, we have
xa = Pd (xk -af (xk)), a> 0,
xk+1 = xa, 0 <a<-a L
By (3.5), we can write (xka - xk +af (xk),x - x^ > 0, Vx e D, a > 0 . If we set x = xk in the above, then it becomes
(xka - xk +af (xk),x -xka) > 0, a > 0,
(xka - xk, x - xka) + a(f'(xk ), xk - xka) > 0, a > 0, (3.6)
a(f '( xk ), xk - xk )>-|| xk - xk| |2.
By virtue of Lipshits condition and the property of the remainder bounedness,
f (x^)- f (xk) <(f'(xk),xk -xk) + L^ -xk Taking into account the inequality (3.6), we have
,-k „k112
f(xk) - f (xk) kJl^OI- + L\xka - xk||2 ={~a + L j(|| xa - xk||2) (3.7)
Set in (37) ak =a~ 0 <a < L-, a = ak . In this case
C = -! + L <0, f (xa) -f (xk) < dlxf - xk112 <0 a 11 11
It is clear that the sequence {f (xk)j is decreasing. On the other hand, as a strict convex quadratic function, f (xk) is bounded from below, then
lim( f (xk+1) - f (xk))
Consequently,
kirn | f (xk+1) - f(xk )| = 0 (3.8)
for a given x0, the set M(x0) defined by M(x0) = {x e D | f (x) < f (x0)j is bounded. Since
f (x0) > f (x1) >... > f (xk) > f (xk+1) > ...,
Then
{xk jc M (x0), k = 0,1,2,... By the Weierstrass theorem the set S* ={x e D | f (x) = f (x*)j is non empty, where f (x) = min f (x). Since f (x) is convex, for any
xeD
x* e S, xk e D we have
0 < f (xk) - f (x') <(f'(xk),xk -x') = (f'(xk),xk -xa + xa -x') =
\ / \ / (3.9)
(f'(xk),xk - xka) + (f'(xk),xa -x^, a > 0 From here, we obtain
\f\xk), xa - x) xka - xk, x - xka), a > 0
a
Setting a = a, 0 <a <L in (3.9), we obtain
0<f(xk)-f(x*)<(a(x -xk+1)-f'(xk),xk+1 -xk^:
r^l|x* - xkf'(xk)| Цxk+1 - xkl
Since M (x0) is bounded, then
x* - xk+M < sup ||x - j|| = K < +<x>
x, yeM (x0)
Moreover,
г'( xk )|| f'(xk) - f'( x0) + f'(xk )|| < || f'(xk) - f'(x 0)|| f'(x 0)|| < Lx - x0|| +1 f'(x0)|| < L • K + N,
where N = f'(x0) , x eM(x0). Then
K
0 < f(xk) - f (x ) <| — + L ■ K + N xk+1 - xk||. a
Taking into account (3.8) that
we have
lim xk+1 -xk\\ = 0
k ^даИ II
lim f (xk) = f (x)
which proves the assertion.
Numerical Experiments
The proposed algorithms for quadratic maximization and minimization problems have been tested on the following type problems. The algorithms are coded in Matlab. Dimensions of the problems were ranged from 200 up to 5000. Computational time, and global solutions are given in the following tables.
Problem 1.
subject to where
max I
xeD
:(( Ax, x) + ( B, x) ) D = {-(n -i +1) < x,. < n + 0.5/, i = 1,2,...,n}
A =
n n
- 1 ... 1 ^
n -1 n ... 2
, B = (1,1,...,1)
Problem 2.
subject to Problem 3.
max V (n -1 - 0.1 • i )x2
xeD ¿i^ ' 1
i =1
D = {x e Rn |-1 -i < xt < 1 + 5i, i = 1,2,...,n}
imin ^ Ax,x) B,x) ^
subject to where
Problem 4.
subject to
D = {10 < x < 30, i = 1,2,...,n}
A =
n n
- 1 ... 1 ^
n -1 n ... 2
, B = (1,1,...,1)
max X ( -1)2
i=1
D = {x e Rn | i +1 < xt < i +10, i = 1,2,...,n}
Table
proble m n Initial value Global value Computing time (sec)
1 200 2.66690000000000e+006 335.337841669956e+009 4.166710
1 500 41.6672500000000e+006 32.7084036979206e+012 29.082187
1 1000 333.334500000000e+006 1.04625056270796e+015 202.615012
1 2000 2.66666900000000e+009 33.4733378341612e+015 1579.706486
1 5000 41.6666725000000e+009 3.26848965365074e+018 22248.485127
2 200 37.7900000000000e+003 12.3936575899980e+009 4.766626
2 500 236.975000000000e+003 482.716617724994e+009 30.115697
2 1000 948.950000000000e+003 7.71590814447147e+012 196.977634
2 2000 3.79790000000000e+006 123.393965944640e+012 1786.474928
2 5000 23.7447500000000e+006 3.5660855482763485e+18 27342.423215
3 200 600.004500000000e+006 266.668000000000e+006 9.625021
3 500 9.37501125000000e+009 4.16667000000000e+009 69.149144
3 1000 75.0000225000000e+009 33.3333400000000e+009 408.735783
3 2000 600.000045000000e+009 266.666680000000e+009 3035.512940
3 5000 937.500011250000e+012 416.6666800000000e+12 21432.421674
4 200 500 200.00 9.227003
4 500 12500 500.00 56.679240
4 1000 25000 1000.00 365.62471
4 2000 50000 2000.00 2904.275652
4 5000 125000 5000.00 22134.532145
Conclusion
To provide a unified view, we considered the quadratic programming problem consisting of convex quadratic maximization and convex quadratic minimization. Based on global optimizality conditions by Strekalovsky [11, 12] and classical local optimality conditions [1, 7], we proposed some algorithms for solving the above problem. Under appropriate conditions we have shown that the proposed algorithms converges to a global solution in a finite number of steps. The Algorithm MAX generates a sequence of local maximizers and and uses linear programming at each iteration which makes algorithm easy to implement numerically.
References
1. Bertsekas D.P. Nonlinear Programming, 2nd edition Athena Scientific. -Belmont, MA, 1999.
2. Bomze I., Danninger G. A Finite Algorithm for Solving General Quadratic Problem, Journal of Global Optimization, 4. - 1994. - Р. 1-16.
3. Enkhbat R.. An Algorithm for Maximizing a Convex Function over a Simple. Set // Journal of Global Optimization, 8. 1996. - Р. 379-391.
4. Horst R. On the Global Minimization of a Concave Function: Introduction and Servey // Operations Research Spectrum, 6. - 1984. - Р. 195-200.
5. Horst R. A General Class of Branch and Bound Methods in Global Optimization with some New Approaches for Concave Minimization // Journal of Optimization Theory and Applications, 51. - 1986. - Р. 271-291.
6. Horst R., Tuy H. Global Optimization, Springer-Verlag. - New-York, London, Tokyo, 1990.
7. Horst R., Pardalos P.M., Nguyen V. Thoai. Introduction to Global Optimization, Kluwer Academic, Dordrecht. - Boston, 2000.
8. Karmanov V.G., Mathematical Programming // Mir Publisher. - Moscow, 1989.
9. Pshenichnyi B.N., Danilin Yu.M. Numerical Methods in Extremal Problems. - Moscow: Nauka, 1975.
10. Rockafellar R.T. Convex Analysis // Princeton University Press, Princeton, 1970.
11. Strekalovsky A.S. On the Global Extremum Problem // Soviet Math.Doklady, 292(5). 1987. - Р . 1062-1066.
12. Strekalovsky A.S. Global Optimality Conditions for Nonconvex Optimiza tion // Journal of Global Optimization, 12. - 1998. - Р. 415-434,
13. Vasiliev O.V. Optimization Methods. - Atlanta: World Federation Publishers, 1996.
T. Bayartugs, Mongolian University of Science and Technology, Ulaanbaatar, Mongolia, e -mail: bayart1969@yahoo.com
R. Enkhbat, professor, National University of Mongolia, Ulaanbaatar, Mongolia, 210646, e-mail: renhkbat46@yahoo.com
A. Enkhbolor, Inje University, Korea.
Т. Баяртугс, преподаватель, Монгольский университет науки и технологий, г. Улан-Батор, Монголия, e -mail: bayart1969@yahoo.com
Р. Энхбат, профессор, Монгольский национальный университет, Улаанбаа-тар, Монголия, 210646, e -mail: renhkbat46@yahoo.com
А. Энхболор, Инчже университет, Корея.
