The quasidifferential calculus, separation of convex sets and the Demyanov difference Текст научной статьи по специальности «Математика»

UDC 519.7 Вестник СПбГУ. Прикладная математика... 2018. Т. 14. Вып. 1

J. Grzybowski1, D. Pallaschke2, R. Urbanski1

THE QUASIDIFFERENTIAL CALCULUS, SEPARATION OF CONVEX SETS AND THE DEMYANOV DIFFERENCE

1 Adam Mickiewicz University, 87, ul. Umultowska, Pozna, ZIP: 61-614, Poland

2 Karlsruhe Institute of Technology University, 89, Kaiserstrasse, Karlsruhe, ZIP: 76-133, Germany

In this paper we give a survey of the influence of the quasidifferential calculus of V. F. Demyanov and A. M. Rubinov to the field of generalized convexity. In particular, we will show the strong relations between the order cancellation property of bounded closed convex set and the separation property of bounded closed convex sets by sets. Moreover, a generalization of the Demyanov difference of compact convex sets infinite dimension and its role in the context of convex sets by sets is discussed. Refs 15. Figs 2.

Keywords: quasidifferential calculus, convex sets, subdifferential calculus, generaliz convexity.

Дж. Гжибовски1, Д. Паллашке2, Р. Урбански1

КВАЗИДИФФЕРЕНЦИАЛЬНОЕ ИСЧИСЛЕНИЕ, РАЗДЕЛЕНИЕ ВЫПУКЛЫХ МНОЖЕСТВ И РАЗНОСТЬ ДЕМЬЯНОВА

1 Университет Адама Мицкевича, Польша, Зип: 61-614, Познань, ул. Ульмитовска, 87

2 Технологический институт Карлсбруэльского университета, Германия, Зип: 76-133, Карлсбруэ, Кайзерштрассе, 89

Приведен обзор влияния квазидифференциального исчисления B. Ф. Демьянова и А. М. Рубинова на поле обобщенной выпуклости. В частности, показаны сильные соотношения между свойством отмены порядка ограниченного замкнутого выпуклого множества и свойством разделения ограниченных замкнутых выпуклых множеств множествами. Кроме того, обсуждены в деталях теория отделимости ограниченных замкнутых выпуклых множеств и разность Демьянова в банаховых пространствах. Библиогр. 15 назв. Ил. 2.

Ключевые слова: квазидифференциальное исчисление, выпуклые множества, субдифференциальные исчисления, обобщенная выпуклость.

1. Introduction. The quasidifferential calculus which was developed by V. F. Demyanov and A. M. Rubinov around 1980 (see [1, 2]) looks at the first glance as an exact subdifferential calculus for pairs of subdifferentials which satisfies the classical rules of additivity and the Leibniz rules. However, a closer view shows that the quasidifferential calculus had far-reaching consequences on generalized convexity which did lead to a fun-

Grzybowski Jerzy — doctor of physical and mathematical sciences, professor; jgrz@amu.edu.pl Pallaschke Diethard — doctor of physical and mathematical sciences, professor; diethard.pallaschke@kit.edu

Urbanski Ryszard — doctor of physical and mathematical sciences, professor; rich@amu.edu.pl

Гжибовски Иржи — доктор физико-математических наук, профессор; jgrz@amu.edu.pl Паллашке Дитхард — доктор физико-математических наук, профессор; diethard.pallaschke@kit.edu

Урбански Ричард — доктор физико-математических наук, профессор; rich@amu.edu.pl

© Санкт-Петербургский государственный университет, 2018

damental reconsideration of the algebraic and geometric properties of the semigroup of bounded closed convex sets. It created a new theory on minimal representation of the elements of the Minkowski—Radstrom—Hormander lattice which consist of equivalence classes of pairs of bounded closed convex sets. For details we refer to the book [1].

We begin our paper with a short description of the basics of quasidifferential calculus and point out in particular its close analogy to the classical Frechet differential calculus. Then we switch to generalized convexity and continue with the Minkowski duality. Within this context we discuss in details a separation theory for bounded closed convex sets by convex sets [3] and present a generalization of the Demyanov difference for closed bounded convex sets in arbitrary Banach spaces [4]. Here we point out in particular the very natural connection between the algebraic lattice rules and their geometric counterparts for convex sets. Exemplarly we discuss this point in detail for the order cancellation law which has the separation law for convex sets by sets as a geometric counterpart and which gives rise to a new research direction that could be called geometric algebra for convex sets.

At the end of the paper we show that in a locally convex vector space the generalized Demyanov difference of two closed bounded convex sets gives is a separating set.

2. The quasidifferential calculus. We begin with a short survey on the fundamental concept of quasidifferentiation.

Definition 1. Let X be a normed linear space and U C X an open set. A function f : U ^ R is quasidifferentiable at x0 e U if f is continuous at x0 and a) for every v e X \ {0} the directional derivative

й.

dv

= lim exists;

t

X

0

b) the mapping

Df\я

X

v

R M.

dv

e DCH(X) with

DCH(X) = {^ = p — q | p,q : R" ^ R, sublinear and continuous}.

There exists an obvious similarity between quasidifferentiation and the Frechet differential calculus in finite dimensional spaces, by extending the notion of a derivative from a linear functional to a difference of sublinear functions. The next theorem is taken from [5, theorem 10.2.6].

2.1. The Frechet property.

Theorem 1. Let (X, || • ||) be a finite-dimensional normed vector space, U C X an open subset and f : U —► R a locally Lipschitz function. Then f is quasidifferentiable at x0 e U if and only if there exists an element df \ e DCH(X) such that for every e > 0 there exists a 6 > 0, such that for all h e U with ||h|| ^ 6 and x0 + h e U the following inequality: f f f

|f (xo + h) — f (xo) — df \xo (h)| < eH (FP)

holds.

Proof. The sufficiency is obvious. Now let f be a locally Lipschitz function which is quasidifferentiable. Then we have to show that the condition (FP) is satisfied. Assume that this is not true. Then there exists a locally Lipschitz quasidifferentiable function,

f : U ^ R, which does not satisfy condition (FP). Hence there exists an e > 0, such that for every A; G N there exists an element hk G X \ {0} with ||/ifc|| < | and

f (xo + hk) - f (xo) - df Ц (hk)

> 4hk\\-

Since f : U ^ R is locally Lipschitz, there exists a ball B(x0,r) C U and a real number M > 0, such that for all x,y e B(x0,r) we have If(x) — f(y)\ < M\\x — y\\. Now define:

gk = and choose a convergent subsequence, also denoted by (gk)ken, with \ivngk = g and limafc = 0, where a.k = Then for all k G N we have

f (xo + akg) - f (xo) - df Ц (akg)

f (xo + ak gk) + f (xo + ak g) - f (xo + ak gk) - f (xo) -

- df L(akg) +df L(akgk) - df L(akgk)

(f (xo + ak gk) - f (xo) - df Ц (ak gk)) -

- ((f (xo + akgk) - f (xo + akg)) - (df I (akg) - df L (akgk))

f (xo + ak gk) - f (xo) - df Ц (ak gk)

(f (xo + akgk) - f (xo + akgk)) - (df I (akgk) - df I (akg))

x0 x0

f (xo + ak gk) - f (xo) - df \ (ak gk)

- (\f (xo + akgk) - f (xo + akg)\ +

df L (akgk) - dfL(akg)

> e0a.kr - (Ma.k\\gk - g\\ + Lak\\gk - f ||) = = (e0r - M\\gk - g\\-L\\gk - f \\)ak. There exists an index k0 G N such that for all k > k0

eor - M\\gk - g\\-L\\gk - f \\ >

£q r 2 '

where L denotes the Lipschitz constant of df \ . Hence for all к > ko we have

f (xo + ak g) - f (xo)

ak

- df \

£q r 2 '

which implies that the directional derivative of the function f in direction g at the point x0 G U does not exist. But this is a contradiction to the assumption, that f : U ^ R is quasidifferentiable at xq G U. q

From the Frechet property follows for quasidifferentiable real valued functions in finite dimensional spaces immediately:

d(af + !3g)\Xo = a df X0 + ¡3 dgX0 , a, 3 e R,

d(fg)\x0 = f (xo) dgg\x0 + g(xo) df\x

d[ -

g

Xo J \ 0 1 0 J Ixo '

дЫ) df\x0 - /(so) d3\xо ,, wn

-"2-, /(^oJ^U.

/ (x0 )

Let us state, that V. F. Demyanov and A. M. Rubinov proved this rules for arbitrary Banach spaces.

2.2. A Banach space norm for DCH(X). A further important property of the Frechet differential calculus in a Banach space (X, || • ||) is the basic fact that the space of gradients coincides with the dual space (X', || • ||) and is hence complete in the dual norm || • This property, transformed to the quasidifferential calculus, would request the existence of a norm for the space DCH(X), under which it becomes a Banach space. This can indeed be satisfied as the following result shows, which reflects a further important analogy between the Frechet and the quasidifferential calculus.

Theorem 2 ([6, theorem 8.1.26]). Let (X, || • ||) be a Banach space. Then the space

DCH(X) = {p = p — q | p,q are sublinear and continuous} endowed with the norm || • ||a given by

||p||a = inf < max < sup p(x), sup q(x) > > ,

V='p-q ^ IjMl^l IMI^l ) )

where the infimum is taken over all continuous sublinear functions p, q such that p = p — q, is a Banach space.

Proof. First we show that ||p||A is well defined for p e DCH and that || • ||a is a norm.

Let p = p — q e DCH(X) be given. From p(0) = q(0) =0 it follows that ||p||a > 0. Since p,q are Lipschitz continuous, ||p||A < From the definition of || • ||A it follows that for all p e DCH(X) and t e R the homogeneity condition ||tp||A = |t|||p||A holds. Next we prove the triangle inequality.

Let e > 0 be given and let pi,p2,qi, q2 be continuous sublinear functions with pi = pi — qi, P2 = p2 — q2 e DCH(X) and

||pi||A < ma^ sup pi(x), sup qi(x) > < ||pi|a + e,

IMICi IM|<i

\\p2\U < ma^ sup p2(x), sup 92 (x) > < \\p2\U + £•

IjHK1 IHI<1 J

Now we have

+ Р2||д < max ^ sup [pi(x) + p2(x)], sup[gx(x)+ q2{x)]\ <

||s||<1 I|z|K1

< max < sup p1 (x) + sup p2 (x), sup q1 (x) + sup q2 (x) > <

I y^y^i I|z|K1 IWK1 I

< max < sup pi (x), sup qi (x) > + max < sup p2 (x), sup q2 (x) > ^

I ||z||^i IMKi I I ||s||<i IWKi I

< ||^i||a + || || a + 2e.

The triangle inequality follows from the arbitrariness of e > 0. In this way we have proved that DCH(X) is a normed vector space.

Now we show that the space DCH(X) is complete. Let (^n)neN be a sequence of elements of DCH(X) such that

IKIlA < —•

We show that the series ^=1 ¥n is convergent in the space DCH(X). This convergence implies obviously the completeness of the space DCH(X).

Let e be an arbitrary positive number and let p£n, q(n be continuous sublinear functions with ^n = pn — qn and

1Ы|л < max { sup pen(x), sup qen(x) ¡> < ||^„||д +

|Ы|<1 IM|<1

It follows from the definition of the norm \\ • \ that the series J2'^==1 Pn and J2'^==1 Qn are uniformly convergent on the unit ball of X, thus they have limits pE,qE which are continuous sublinear functions.

Moreover, the sequences (p™ ) and (q™ ) have limits p and q which are also

V J nen V J neN continuous sublinear functions. We write ^(x) = p(x) — q(x). Clearly, ^ e DCH(X). It remains to show that ^ is a limit of the series C^==1 ¥n.

From the above assumptions and a simple calculation it follows that there exists an index me such that for all m ^ me

- (pe - qe)

< 2e.

A

Since pn, qn have the limits p, q, there exists an me such that for m > me

E

<Pn - (p - q)

< 2e.

A

The arbitrariness of e > 0 implies that the series ¥n is norm-convergent in the space

DCH(X). n

Further metrics for DCH(X) are the Demyanov and the Hausdorff metric which are studied in detail in [7].

2.3. The approach of V. F. Demyanov and A. M. Rubinov. Different from the above presentation of the quasidifferential as a difference of two sublinear functions, V. F. Demyanov and A. M. Rubinov used an other approach via the Minkowski duality. More precisely:

In the case of the quasidifferential calculus we have to deal with a normed vector space (X, \\ • \\) and we endow the dual space X with the weak-*-topology a(X, X ). The dual norm of X will be denoted by \\ • \\ and (•, •) : X x X ^ R stands for the dual pairing.

Moreover, let us denote by K(X ) the set of all compact convex subsets of the dual space X' endowed with the weak-*-topology a(X,X ). It follows from the theorem of Alaoglu—Bourbaki that the elements of K(X ) are bounded in the dual norm.

Now L. Hormander [8] showed that an arbitrary sublinear function p : X ^ R is continuous in the norm topology if and only if its subdifferential at the origin

dp\o = {v e X \ (v,x) < p(x), x e X}

is a compact convex subset of the dual space X' with respect to the weak-*-topology a(X,X ) of X' and that p : X ^ R has the representation p(x) = pa(x) = max (a,x)

with A = dp\o eK(X').

To formalize this assignment more precisely, let us consider the set K2(X') j of all

equivalence class of pairs (A,B) e K2(X'). For two pairs of nonempty compact convex subsets the following relation (A, B) ~ (C, D) ^^ pA + pD = pB + pc. Two pairs (A, B) and (C,D) which satisfy this relation are called equivalent, and K2(X')j is the linear

space of all classes of equivalent pairs. The element of K2(X')j which contains the pair

(A, B) is denoted by [A, B] e K2(X') / . In 1966 A. G. Pinsker [9] introduced the following

ordering on K2(X')j , namely: [A, B] < [C, D] ^^ pA + pD < pB + pc, which is independent of the special choice of representatives, because of the order cancellation law. The space ^K2 (X')j , ^ is called the Minkowski—Radstrom—Hormander space of

classes of pairs of nonempty compact convex sets. It is a complete vector lattice and a direct calculation shows that the assignment:

DCH(X) —► K 2(X')j

with p ^ [f\ = {(dp\0, dq\0) \ with p = p — q, p,q continuous and sublinear

is a lattice isomorphism, called Minkowski duality (see [5, theorem 3.4.3]). Note that it is not a Banach lattice with respect to the norm || • ||a given in theorem 2. V. F. Demyanov and A. M. Rubinov assigned to p e DCH(X) the set

[p] = {(cp|o, dqjo) | with p = p — q, p,q continuous and sublinear} e K (X')

L

and considered the set K2 (X')j of all equivalence class of pairs (A, B) e K2(X') instead

of differences of sublinear functions. They reformulated condition b) of Definition 1 in the following way:

b'). There exist two sets df\x0, df\Xo G K,(X ) called sub- and superdifferential such

that

^ dJ-

dg

= max (v, g) + min (w,g)= max {v,g)~ max (w, g) G DCH(X).

vEdf |x0 wEdf |X[1 vEdf |x0 wE-df |X[1

The pair D f\x = {df\ ,df\ ) consisting of a sub- and superdifferential is called a quasidifferential o? / at xq G U and for the class of all equivalent pairs holds:

[df\xo,-dfU] €/C2(X')/ which is an element of the the Minkowski—Radstrom—Hormander lattice. Вестник СПбГУ. Прикладная математика. Информатика... 2018. Т. 14. Вып. 1 25

In the Section 3 we one step further towards generalised convexity and geometric algebra for arbitrary topological vector spaces (see also [10]).

3. The semigroup of closed bounded convex sets. For a Hausdorff topological real vector space (X,t) let us denote by A(X) the set of all nonempty subsets of X, by B*(X) the set of all nonempty bounded subsets of X, by C (X) the set of all nonempty closed convex subsets of X, by B(X) = B*(X) nC(X) the set of all bounded closed convex sets of X and by K(X) the set of all nonempty compact convex subsets of X. For A,B e A(X) the algebraic sum is defined by A + B = {x = a + b | a e A and b e B} and for A e R and A e A(X) the multiplication is defined by AA = {x = Xa | a e A}.

The Minkowski sum for A, B e A(X) is defined by

A + B = cl({x = a + b I a e A and b e B}),

where cl(A) = A denotes the closure of A c X with respect to t.

For a Hausdorff topological real vector space X two pairs (A, B), (C, D) e B2(X) =

B(X) x B(X) are called equivalent if B + C = A + D holds and [A, B] denotes the equivalence class represented by the pair (A, B) e B2(X). An ordering among equivalence classes is given by [A, B] < [C, D] if and only if A + D c B + C. This is the ordering on the Minkowski—Radstrom—Hormander space and is independent of the choice of the representatives. Note that this definition is a natural extension of the former definition of "equivalence" and "ordering" for compact convex subsets of the dual space X' from a Banach space X endowed with the weak-*-topology a(X, X ).

For A e B(X) we denote by ext(A) the set of its extreme points and by exp(A) the set of its exposed points (see [5]). Next, for A,B e A(X) we define: A V B = cl conv(A U B), where conv(AuB) denotes the convex hull of AUB. We will use the abbreviation A +BVC for A + (B V C) and C + d instead of C + {d} for all bounded closed convex sets A,B,C e A(X) and a point d e X.

A distributivity relation between the Minkowski sum and the maximum operation is expressed by the Pinsker Formula (see [9]), which is stated in a more general form in [5].

Proposition 1. Let (X,t) be a Hausdorff topological vector space, A,B,C e A(X) and C be a convex set. Then

(A + C) V (B + C) = C + (A V B). (PF)

The following generalization of the Minkowski—Radstrom—Hormander Theorem is due to R. Urbanski [11].

Theorem 3. Let X be a Hausdorff topological vector space. Then for any A e A(X), B e B*(X) and C e C(X) the inclusion

A + B C C + B implies A C C. (olc)

This implies that B(X) endowed with the Minkowski sum " +" and the ordering induced by inclusion is a commutative ordered semigroup (i. e. a ordered set endowed with a group operation, without having inverse elements), which satisfies the order cancellation law and contains K(X) as a sub-semigroup.

4. The separation law for convex sets. In this section we study a generalized separation property for convex sets.

Definition 2. Let X be a topological vector space, I be a finite index set and S,Ai e B(X), i e I. We say that the set S separates the sets Ai, i e I, if and only if (conv {ai | i e I}) fl S = 0 for every collection ai e A, i e I.

The definition is illustrated in Figure 1 for the convex sets A1,A2,A3,S C R2. We will

Figure 1. The definition for the convex set Ai, A2 ,A3 , S C R2

k

use the notation Viei {ai} for conv {ai | i G I} and write ^ Ai = A1 + A2 + ••• + Ak.

i=1

Now we have the following algebraic characterization of the separation property.

Theorem 4. Let X be a topological vector space, I be a finite index set and S, Ai G B(X), i G I. Then S separates the sets Ai, i G I, if and only if

E Ai c WE Ak) + S.

iei iei ^kei\{i} '

Proof. Necessity. Let ai G Ai, i G I, be given. Then there exist ai > 0, = 1

such that ^ aiai G S. Therefore,

iei

iei

= E ( ^2 a4 ai + ^2 aai =

iei ^kei\{i} ' iei

iei

iei x kei\{i}

= E ai[ a4 + E aiaie V ( J2 A

iei

iei xkei\{i}

which proves the necessity.

Sufficiency. Now fix any ai G Ai, i G I. Then it follows from the assumption:

that for every i G I

EAi с V E Ak) + 5

iei iei 4 kei\{i}

ai + ^ Ak С \J ( Ak

kei\{i} iei ^ kei\{i}

which means

E Ak c V ( E Ak} + (S - ai), i G I. kei\{i} iei ^ kei\{i}

From the Pinsker formula we get

k

V Е

iei Kkei\{i}

С

V V

iei

Liei

E Ak

kei\W

+ - ai)

lei

V( E Ak

kei\{i}

) + V (s

iei

- ai)

and gives by the order cancellation law that 0 £\J (S — ai).

iei

Now again by the Pinsker formula we get 0 £ \J (S — ai) = S + \J {—ai} , which implies that (conv {ai | i £ I}) H S = 0

iei

iei

□

Parallel to the notation of separation the notation of shadowing is also used (see [5, p. 67 and 77] ) to express the same property. Namely the physical interpretation of the separation by sets is as follows: if the sets A, B, S are considered as celestial and A shines, then S separates A and B if and only if B lies in the shadow of S. Of special interest is the separation of two sets A and B by a convex set S, the case A,B,S £ B(X) which is despicted on the left hand site of in Figure 2. Then S separates the sets A, B if and only if

Figure 2. The case A,B,S e B(X)

A + B a A V B + S.

(SL)

The formula (SL) is called the separation law. Now it turns out that the separation law is equivalent to the order cancellation law, as we will see in the following section.

5. The separation and the order cancellation law.

Theorem 5. Let A, B be nonempty subsets of a topological vector space X and assume that A V B £ B(X). Moreover, let S be a closed convex subset of X. Then S separates the sets A and B if and only if

A + B c A V B + S.

Proof. Necessity. Fix any a £ A and b £ B. Then it follows from the assumption A + B c A V B + S that a + B c A V B + S and b + A c A V B + S. Hence B c A V B + S — a and A c A V B + S — b. By this inclusions and the Pinsker formula (PF) we have A V B c A V B + (S — a) V (S — b) = A V B + (S + [—a, —b]). Now we obtain from the order cancellation law (olc) that 0 £ S + [—a, —b]. Hence [a, b] H S = 0.

Sufficiency. Now assume that formula (SL) is satisfied. Let A,B,C £ A(X) and assume that B is bounded and C is closed and convex. Now take any a £ A and b £ B. From A + B c B + C it follows that a +(B — b) c (B — b) + C. Therefore, {0} + (B — b) c (B — b) V {0} + (C — a). Now from the separation law (SL) it follows that the set C — a separates the sets {0} and B — b. Hence 0 = a0 + ft0 £ C — a for some a, ft ^ 0, a + /3 = 1. So, we have 0 £ C — a. Hence A c C. □

6. The Demyanov difference. Demyanov original subtraction A—B (see [12]) of compact convex subsets in finite dimensional space is defined with the help of the Clarke subdifferential (see [13]) of the difference of support functions, i. e.

A-B = dci(pa - pb)

0

where pA and pB are the support functions of A and B, i. e. pA(x) = max (a, x). This can

aeA

be equivalently formulated by

A-B = conv{a -b\a e A,b e B,a + b e exp(A + B)},

here exp(A + B) are the exposed points of A + B. For the proof see Proposition 2 in [12] and note that every exposed point of A + B is the unique sum of an exposed point of A with an exposed point of B.

To extend the definition of the difference A—B to locally convex topological vector spaces, the set of exposed points will be replaced by the set of extremal points of A + B. This leads to the following generalization of the Demyanov difference.

Definition 3. Let (X,t) be a locally convex vector space and K(X) the family of all nonempty compact convex subsets of X. Then for A, B e K(X) the set

is called the Demyanov difference of A and B.

This is a canonical generalization of the above definition, because for every A,B G K(X) every extremal point z g ext(A + B) has a unique decomposition z = x + y into the sum of two extreme points x G ext(A) and y G ext(B) (see [14, Proposition 1]). Since in the finite dimensional case the exposed points are dense in the set of extreme points of a compact convex set, this definition coincides with the original definition of the Demyanov difference in finite dimensional spaces. Note that in [4] a generalization of the Demyanov difference to arbitrary topological vector spaces is also given.

The Demyanov difference possesses many important properties and in [15] the following statement is proved.

Proposition 2. Let X be a locally convex vector space and A,B,C G K(X). The Demyanov difference has the following properties: (D1). If A = B + C, then C = A - B. (D2) (A -B) + B D A. (D3). If B C A, then 0 G A-B. (D4) (A-B) = -(B-A). (d5) A-C c (A-B) + (B-C).

7. Demyanov difference and separating sets. From property (D2) of the above proposition a theohem follows immediately.

Theorem 6. Let X be a locally convex vector space, I be a finite index set and S, Ai G K(X), i G I. Then the Demyanov difference

separates the sets Ai, i G I.

Corollary. Let ai,a2, ...,Ak G K(Rn) be given. Then for the Demyanov difference holds

where dclP|0 is the Clarke subdifferential of P = min{pAl,pAl, ...,pAk} at 0 G R", i. e. the minimum of the support functions of the sets Ai.

A-B = œnv{a - b\a e A,b e B,a+b e ext(A + B)} G ЦХ)

Proof. This follows immediately from the definition of the Demyanov difference for the finite dimensional case (see [12]) and the formula

min {pAi ? PAi ,--,PAk },

which completes the proof. References

1. Demyanov V. F., Rubinov A. M. Kvazidifferentsial'nie funktsionali [Quasidifferentiable functionals]. Dokl. Akad. Nauk SSSR [Papers of Academy of Sciences USSR], 1980, vol. 250, no. 1, pp. 21—25. (In Russian)

2. Demyanov V. F., Rubinov A. M. Quasidifferential calculus. New York, Optimization Software Inc. Publ. Division, 1986, 288 p.

3. Grzybowski J., Pallaschke D., Urbanski R. A pre-classification and the separation law for closed bounded convex sets. Optimization Methods and Software, 2005, vol. 20, pp. 219—229.

4. Grzybowski J., Pallaschke D., Urbanski R. Demyanov difference in infinite dimensional spaces. Proceedings of CNSA, Saint Petersburg, Springer Optim. Appl., 2014, vol. 87, pp. 13—24.

5. Pallaschke D., Urbanski R. Pairs of compact convex sets — Fractional arithmetic with convex sets. Mathematics and its Applications, 2002, vol. 548, pp. 487—496.

6. Pallaschke D., Rolewicz S. Foundations of mathematical optimization. Mathematics and its Applications, 1997, vol. 426, pp. 417-429.

7. Diamond P., Kloeden P., Rubinov A. M., Vladimirov A. Comperative properties of three metrics in the space of compact convex sets. Set-Valued Analysis, 1997, vol. 5, no. 3, pp. 267-289.

8. Hormander L. Sur la fonction d' appui des ensembles convexes dans un espace localement convexe [On the function of support sets convex in a space that is locally convex]. Arkiv for Matematik [Archives for Mathematics], 1954, vol. 3, pp. 181-186.

9. Pinsker A. G. The space of convex sets of a locally convex space. Trudy Leningrad Engineering-Economic Institute, 1966, vol. 63, pp. 13-17.

10. Rubinov A. M. Abstract convexity, global optimization and data classification. Opsearch, 2001, vol. 38, pp. 247-265.

11. Urbanski R. A generalization of the Minkowski—Radstrom—Hormander theorem. Bull. Acad. Polon. Sci. Series Sci. Math. Astr. Phys., 1976, vol. 24, pp. 709-715.

12. Rubinov A. M., Akhundov I. S. Differences of compact sets in the sense of Demyanov and its application to non-smooth-analysis. Optimization, 1992, vol. 23, pp. 179-189.

13. Clarke F. H. Optimization and Nonsmooth Analysis. New York, J. Wiley Comp. Publ., 1983, 306 p.

14. Husain T., Tweddle I. On the extreme points of the sum of two compact convex sets. Math. Ann., 1970, vol. 188, pp. 113-122.

15. Gaudioso M., Gorgone E., Pallaschke D. Separation of convex sets by Clarke subdifferential. Optimization, 2011, vol. 59, pp. 1199-1210.

For citation: Grzybowski J., Pallaschke D., Urbanski R. The quasidifferential calculus, separation of convex sets and the Demyanov difference. Vestnik of Saint Petersburg University. Applied Mathematics. Computer Science. Control Processes, 2018, vol. 14, iss. 1, pp. 20-30. https://doi.org/10.21638/11701/spbu10.2018.103

Статья рекомендована к печати проф. Л. А. Петросяном. Статья поступила в редакцию 15 октября 2017 г. Статья принята к печати 11 января 2018 г.