Approximate optimal control synthesis for nonuniform discrete Systems with linear-quadratic state Текст научной статьи по специальности «Математика»

issn 2079-3316 PROGRAM SYSTEMS: THEORY AND APPLICATIONS udc 517.977

a

Irina V. Rasina, Oles V. Fesko

Approximate optimal control synthesis for nonuniform discrete systems with linear-quadratic state

Abstract. Nonuniform discrete systems linear-quadratic over its state are the subject of intense study in optimal control theory. This work presents an approximate optimal control synthesis method in this class based on Krotov's sufficient optimality conditions and illustrates it with a simple example.

Key words and phrases: nonuniform discrete systems, sufficient optimality conditions, approximate optimal control synthesis.

2010 Mathematics Subject Classification: 49M30; 49N10,

Introduction

Nonuniform control systems are the subject of intense study in optimal control theory. Their state in terms of controlled differential and discrete systems depends on time. Approaches to the development of their mathematical models and investigations, as well as the terminology, e.g., systems with variable structure [?1], discrete-continuous [?2], logic-dynamic [?3, ?4], impulsive [?5], and hybrid systems [?6, ?7], are very diverse. One possible approach to study optimal control problems for such systems is the generalization of Krotov's sufficient optimality conditions for them [?8, ?9]. In [?2, ?10, ?11, ?12], the authors propose a two-level mathematical model of a discrete-continuous system (DCS). The lower level describes the continuous controllable processes at the individual stages. The upper level integrates system descriptions into a unique process and controls the functioning of the entire system as a whole to ensure the minimum of the functional. The work [?12] establish sufficient optimality conditions for the model and presents control improvement methods.

© I. V. Rasina, O. V. Fesko, 2019

© Ailamazyan Program Systems Institute of RAS, 2019

© Program Systems: Theory and Applications (design), 2019

DO lY&Jj1

Work [?13] extends this model to the case where the lower level contains discrete systems. This case is called the nonuniform discrete systems (NDS). In [?14], the authors propose an iterative control improvement method for this model with the resolving Krotov functions linearly depend on the states at both levels. In the case of linear-quadratic nonuniform discrete systems, if we specify the Krotov functions as linear-quadratic, then we get the solution as an approximate synthesis of optimal control. The purpose of this work is to derive the method and illustrate it by example.

1. Nonuniform discrete systems with linear-quadratic state

We consider the NDS model representing a two-level controlled system of the form

x0(k + 1) = x0(k) + 1 a(k)lxl2 + b(k, u), a > 0,

x(k + 1)= A(k)x(k) + B(k,u), x0 € R, x € Rm(k),

k € K = [kr, kI + 1,..., kF}, u € U(k, x) C Rr(k),

where k is the step (stage) number, U(k, x) is the set given for each k and x, A(k) is an m(k) x m(k) matrix, B(k,u) is an m(k) x 1 matrix, a(k), b(k, u) are given functions.

On some subset K' C K, kF € K', there is a discrete system of the lower level

xd0(k,t + 1) = xd0(k,t) + 1 ad(k,t)lxdl2 + bd(k,t,ud), xd0 € R,

xd(k,t + 1) = Ad(k,t)xd + Bd(k,t,ud), xd € Xd(z,t) C Rn(k), t € T (z) = [tr (z), ti (z) + 1,..., tF (z)}, ad > 0, ud € Ud (z,t,xd) C Rp(k),

where Ad(k,t) is an n(k) x n(k) matrix, Bd(k,t,ud) is an n(k) x 1 matrix, ad(k,t), bd(k,t,ud) are given functions, the right-hand side operator is given by

x0(k + 1) = x0(k) + 1 jS(k)lxdF |2, x(k + 1) = x(k) + Z (k)xdF, xd(k,tr)= £(k)x, x0d(k,tr) = 1 k)|x|2, k € K',

where £, £1, Z, p are matrices of the corresponding dimensions. Here

z = (k, x) is a set of the upper-level variables playing the role of lower-level parameters.

The solution of this two-level system is the set

m = (x0(k), x(k), u(k)) ,

where for k e K': u(k) = (ud(k), md(k)) , md(k) e Dd (z(k)) (called a nonuniform discrete process with linear-quadratic state), where md(k) is a discrete process (xd0, xd(k,t),ud(k,t)), t e T(z(k)), and Dd(z) is the set of admissible processes md satisfying with the specified discrete system. Let us denote the set of elements m satisfying all the above conditions by D and call it a set of admissible nonuniform discrete processes with linear-quadratic state.

We consider the problem of finding the minimum on D of the functional I = lTx (kF) + 2n\x(kp)|2 + d, where l is a vector, k is a matrix, d is a constant, for fixed kI = 0, kF = K, x (kI) and additional constraints x(k) e X(k), xd e Xd (z,t), where X(k), Xd (z,t) are given sets.

2. Basic constructions and sufficient optimality conditions

The sufficient optimality conditions for this model are derived by analogy with Krotov's sufficient conditions for discrete systems [?9] as follows. The discrete chains from D and Dd are excluded and scalar functions (functionals) < (k,x), <c (z,t,xd) are introduced. Then we construct a generalized Lagrangian by analogy with the Krotov Lagrangian for discrete systems:

L = G (x (kF)) R(k,x(k),u(k))+ K\K'\kF

+ ^ (yGd(z) - ^ Rd(z, t, xd(k, t), ud(k, t))^,

k' t(z)\tF

G (x) = F (x) + < (kF, x) — < (ki, x (ki)), R (k,x,u) = < (k + 1, f (k,x,u)) — < (k, x), Gd (k, z, Yd) = — < (k +1,0 (k, z, Yd)) + < (k, x (k)) +

+ <d (k,z,tF,xdF) — <d (k,z,tI,xd) , Rd (k,z,t,xd,ud) = <d (k,z,t + 1,fd (k,z,t,xd,ud)) — <d(k,z,t,xd), d (k, z, t) = sup {Rd (k, z, t, xd, ud) : xd e Xd(k, z, t),

ud e Ud (k,z,t,xd)}, ld (k, z) = inf {Gd (k, z, Yd) : (yd) e rd(k, z), xd e Xd(k, z,tF)}.

_ Jsup{R (k, x,u) : x e X(k), u e U (k, x)}, t e K \ K', — inf{ld (z) : x e X (k), uv e Uv (k,x)}, k e K',

M (k) = \ ■ r,tnd ,

l = inf[G (x) : x € r n X (kF)}.

Here f (k, x) is an arbitrary functional, fd (k, z, t, xd) is an arbitrary parametric family of functionals (with parameters k, z), and f, 9, fd denote the right-hand sides of disctete systems at the lower and upper-levels on K\K', K', T, respectively.

Theorem 1. For any element m € D and any f, fd the estimate is I(m) - inf I < A = I(m) - l.

Let there be two processes m1 € D and m11 € E and functionals f and fd such that L (m11) < L (m1) = I (m1) , and m11 € D. Then I(m11) < I(ml).

Theorem 2. Consider a sequence of processes [ms} C D and functionals f, fd such that

(1) R (k,xs (k) ,us (k)) ^ ^ (k), k € K;

(2) Rd (zs,t,xd (t) ,ud (t)) - nd (zs,t) ^ 0, k € K', t € T (zs);

(3) Gd (zs,id) - ld (zs) ^ 0, k € K';

(4) G (xs (tF)) ^ l.

Then [ms} is a minimizing sequence for I on D.

The proofs of these theorems are given in [?13].

3. Approximate optimal control synthesis and algorithm

To construct an approximate optimal control synthesis method, here we use Krotov's sufficient optimality conditions, the extension [?gurman] and localization priciples [?gurman_rasina]. Suppose that U(k,x) = Rr(k), Ud (z,t,xd) = Rp(k) and the used constructions of sufficient optimality conditions are such that all the following operations are valid. The method for solving this problem is generated via some improvement operator n(m) : D ^ D such that I(n(m)) < I(m) [?15].

We define an auxiliary functional [?gurman_rasina]

Ia = La = aI +2(1 - a) I £ |Au (k) |2 + £ £ IAud (k,t) |2

\k\k'\kF k' t(z)\tF

where a G [0,1] and its increment:

1

« GjAx - ^ (rTAX + R^Au + ^Au1 RddAuj -k\k' '

- Y.^AX + Gil AXF )-

k'

- V (RfAx + RdlAxd + RdlAud + 1AudlRdd d Audj)

/ j v d ' dd 1 dd 1 2 udud >j

d 1 xd 1 ud 1 2 udw

T(z)\tF

where R, G, Rd, Gd, L are the constructions of sufficient optimality conditions and Au = u - uI, Aud = ud - udI, Ax = x - x1, Axd = xd - xdI, mI = (udI,xI,udI,xdI) is the given element from D.

Suppose that matrices Rdd and Rc^ddd are negative definite (we can always make it so by varying a [?gurman_rasina]) and find the minimum of ALa with respect to Au, Aud, Ax, Axdp, Axd. We specify the functions P, Pd

p = $l (k) x (k) + 1 xl (k) a (k) x (k) + x0,

pd = $dl (k, t) xd (k, t) + 1 xdl (k, t) ad (k, t) xd (k, t) + xd0,

where are the vector functions, a, ad are matrices of the corresponding dimensions.

Taking into account these requirements, we get

(1) Au = -R-dRu, Aud = -Rd-Xd Rdd,

(2) GdF =0, Rd = 0, Gd = 0, Gddp = const, Rd = 0, Rdd = 0.

The condition (??) may be detailed by

(3) GdF = a(l + kxf ) + $(kF) + a(kp )xf = 0,

Rd = Al$(k +1) + 1(Ax + B)la(k + 1)A+ 1

+ -Ala(k + 1)(Ax + B) + ax - $(k) - a(k)x = 0,

Gd = -$(k + 1) - 1 a(k + 1)(x + Z(k)xdp)-1

(4)

(5) - 2(x + Z(k)xdp)la(k + 1) + $(k) + a(k)x -

- 1 (lad(tI)tx -\(tx)lad(ti)Z - Z^x = 0,

Gdxd = —0(k)T^(k + 1) — 1Z (k)Ta(k + 1)(x + Z (k)x<F ) —

F 1 2

(6) — ^(x + Z (k)xFp )Ta(k + 1)Z (k) —

— pxF + ^d(tF) + °d(tF )xF = o,

RXd = AdT^d(k,t + 1) + \(Adxd+

2 1

(7) + Bd)Tad(k,t + 1)Ad + - AdTad(k,t + 1)(Adx + B d)+

+ adxd — 4>d(k, t) — ud(k, t)xd = 0. The equalities (??)-(??) are valid if

^(k) = A(k)T^(k + 1) + 1 A(k)T a(k + 1)B(k)+

(8) 21

+1 B(k)Ta(k + 1)A(k), ^(kF ) = —al,

(9) a(k)= A(k)Ta(k + 1)A(k) + a(k), k e K \ K', a(kF) = —an,

(10) ^(k) = ^(k + 1) + 2*(k + 1) S(k)xF + 1 (S(k)xF)Ta(k + 1)+

+ £T^d(k,ti),a(k) = a(k + 1) + £Tad(ti)Z + Z1, k e K', 4>d(k,t) = A(k,t)dT'^d (k,t + 1)+

+ 1 Ad(k,t)Tad(k,t +1)Bd(k,t)+

(11) 21

+ 1 Bd(k,t)Tad(k,t + 1)Ad(k,t),

d(k,tF ) = Z T^(k + 1) = HxdF, (12) (tF) = ZTv(k + 1)Z + S, k e K',

( ) ad(k,t)= Ad(k, t)Tad(k, t +1)Ad(k,t) + ad(k,t).

Note that system (??)-(??) is linear, i.e., it is certainly feasible. Denote

H (k,x,u,^(k + 1)) =

= ^T(k + 1)(A(k)x(k) + B(k,u)) — b(k) — 1(1 — a)\Au\2,

k e K \ K \ kF,

H d(k,t,x,xd,ud,^d(k,t)) =

= V>cT(k,t)(Ad(k,t)xd + Bd(k, t, ud)) —

— bd(k,t,u) — 1(1 — a)\Aud (k,t) \2.

Then

Ru = Hu + 1 B^Aax + 1 xTATaBu + 1 BTaBu + 1 B^aB + bu,

Ruu = Huu + ^ B^uAax + 1 xTATaBuu + B^aBu + BTaBuu + bu

1

2 Buu^ 1

2 Bud

Rid = Hdd + 1 BdJ Adadxd + 1 xdT AdTadBdd +

+ 1 BdTadBtd + 2 BdJadBd + b?,d,

tid _ tjd , ^dt Ad^d^d , 1 „dT Adt d nd ,

R,,d„d = Hudud + 2 BududA a x + -x A a Budud +

2 ud ud

dT d d dT d d d

+ Bud a Bud + B a Budud + budud

Note that from the equality RX = 0 we can't obtain equations. Obviously, these formulas for first and second derivatives of R, Rd with respect to control variables linearly depend on state variables of the upper and lower levels, respectively. Therefore, the obtained solution represents approximate optimal control synthesis.

4. Iterative procedure

As a whole, we get the following iterative procedure on a step s.

1. «Left to right» we compute NDS for u = us(k), ud = uds(k,t) with given initial conditions, getting the corresponding trajectories

xs(k), xds(k,t).

2. «Right to left» we resolve NDS with respect to ^ (k), ^d (k,t), a(k), ad(k, t) according to (??)-(??).

3. We find Au, Aud and new controls us+1(k) = us(k)+Au, uds+1(k,t) = uS(k,t) + Aud according to (??).

4. «Left to right» we calculate the initial NDS with the controls found and given initial conditions.

This iterative process is over when |/s+i — Isl ~ 0 with a given precision.

Theorem 3. Suppose that the formulated iterative procedure is constructed for a given NDS, and functional I is bounded from below. Then it generates the improving sequence {ms} G D that converges with respect to the functional, i.e., there is a number I * such that I * < I (ms), I (ms) ^ I *.

Proof. The proof immediately follows from the monotone property of the improvement operator under consideration. Thus, we get a monotone

sequence of numbers

{Is} = {I(ms)}, Is+1 < Is, bounded from below, that converges to a certain limit Is ^ I*.

5. Example

□

Consider the following NDS

„d0f

1

1

d\3

(t +1)= xd0(t) + -(xd(t))2 + -(uf)

2K 3

xd(t + 1) = —2xd(t) + (ud — 1)2,

l(0) = 1,

t = 0,1, 2, 3,

„d0f

1

(t +1)= xd0(t) + -(xd)2 +

u

xd(t + 1) = (t — ud)2, t = 4, 5, 6, I = xd(7) ^ min .

It is easy to see that K = 0,1, 2. Since xd is a linking variable in the two periods under consideration, we can write a process of the upper level in terms of this variable x(0) = xd(0, 0), x(1) = xd(0,4), x(2) = xd(1, 7), x(1) = xd(0,4), and xd(1,4) = x(1). Then I = x(2). From this it follows ad(0,t) = 1, bd(0,t) = 3(ud)3, Ad(0,t) = 1, Bd(0,t) = (ud — 1)2, ad(1,t) = 1, Ad(1,t) = 0, Bd(1,t) = (t — ud)2, Z = 1, Z = 1, C1 =0, S = 0.

The equations of the method have the form

4d(0,t) = —2^}d(0, t + 1) — 2ad(0,t +1)(ud — 1)2,

ad(0,t) =4ad(0,t + 1) + 1, ad(1,t) = 1,

4>d(1,t)=0, 4d(1,tF ) = kF ) = —a, ad(1,tF ) = 0, 4(2) = —a, v(2)=0,

4(1) = №) + \ °(2),

4d(0,tF ) = 4(1),

a(1)= a(2) + ad(1,ti), ad(0,tF ) = a(1),

Rd(0, t)ud =24d(0,t +1)(ud — 1)+

+ 2ad(0, t + 1)((ud — 1)2 — 2xd)(ud — 1) + (ud)2 — (1 — a)Auf, Rd(0, t)udud = 24d(0,t + 1) + 6ad(0,t + 1)(ud — 1)2 —

— 4ad(0, t +1xd + 2ud — (1 — a), Rd(1, t)ud = —24d(1, t + 1)(t — ud) — 2ad(1, t + 1)(t — ud2)3+

+ 1 — (1 — a)Au2,

x

Rd(1,t)uiui = 2^(1,t + 1) + 6a2(1,t + 1)(t - u2)2 - (1 - a).

The initial approximation for the improvement procedure is u2(0) = u2(1) = 1, u2(2) = u2(3) = 2, u2(4) = -1, u2(5) = -1, u2(6) = -1. The results are given by Fig. ?? and Table ??.

1

i

( t

H

< »

0 2 4 6

8 0 t

4

D

d

x

u

1,2 2

0

2

2

4

6

Figure 1. The initial (blue) and final (red) iterations of the method Table 1. Changes of the functional on iterations

Iteration Functional

0 49

1 16

2 4

3 0

Conclusions

In this paper, we propose an approximate optimal control synthesis method for nonuniform discrete systems linear quadratic with respect to state. It is derived from the well-known scheme when specifying Krotov function. We develop an algorithm that implements this method and demonstrates its quality through a model example.

References

S.V. Emelyanov. Theory of Systems with Variable Structures, Nauka, Moscow, 1970 (Russian). t

V.I. Gurman. "Theory of Optimum Discrete Processes", Autom. Remote Control, 34:7 (1973), pp. 1082-1087 (English). t

S.N. Vassilyev. "Theory and Application of Logic-Based Controlled Systems", Proceedings of the International Conference Identification and Control Problems, Moscow, Institute of control sciences, 2003, pp. 53-58 (Russian).

A.S. Bortakovskii. "Sufficient Optimality Conditions for Control of Deterministic Logical-Dynamic Systems", Informatika, Ser. Computer Aided Design, 1992, no.2-3, pp. 72-79 (Russian). t

B.M. Miller, E.Ya. Rubinovich. Optimization of the Dynamic Systems with Pulse Controls, Nauka, Moscow, 2005 (Russian). t

J. Lygeros. Lecture Notes on Hybrid Systems, University of Cambridge, Cambridge, 2003 (English). t

A.J. Van der Shaft, H. Schumacher. An Introduction to Hybrid Dynamical Systems, Springer-Verlag, London, 2000 (English). t V.F. Krotov, V.I. Gurman. Methods and Problems of Optimal Control, Nauka, Moscow, 1973 (Russian). t

V.F. Krotov. "Sufficient Optimality Conditions for the Discrete Controllable Systems", Dokl. Akad. Nauk SSSR, 172:1 (1967), pp. 18-21 (Russian). t V.I. Gurman, I.V. Rasina. "Discrete-Continuous Representations of Impulsive Processes in the Controllable Systems", Autom. Remote Control, 73:8 (2012), pp. 1290-1300 (English).t

I.V. Rasina. "Discrete-Continuous Models and Optimization of Controllable

Processes", Program Systems: Theory and Applications, 5:9 (2011), pp. 49-72 (Russian), (url

I.V. Rasina. "Iterative Optimization Algorithms for Discrete-Continuous Processes", Autom. Remote Control, 73:10 (2012), pp. 1591-1603 (English). t

I.V. Rasina. Hierarchical Control Models for Systems with Inhomogeneous Structures, Fizmatlit, Moscow, 2014 (Russian). t

I.V. Rasina, I.S. Guseva. "Control Improvement Method for Non-Homogeneous Discrete Systems with Intermediate Criterions", Program Systems: Theory and Applications, 9:2 (2018), pp. 23-38 (Russian). uRiDoi: 10.25209/ 2079-3316-2018-9-2-23-38 t

V.I. Gurman. "Abstract Problems of Optimization and Improvement", Program Systems: Theory and Applications, 5:9 (2011), pp. 14-20 (Russian).

V.I. Gurman. The Extension Principle in Control Problems, Nauka, Moscow, 1997 (Russian). t

V.I. Gurman, I.V.Rasina. "Practical Applications of Conditions Sufficient for a Strong Relative Minimum", Autom. Remote Control, 40:10 (1979), pp. 1410-1415 (English). t

Received

Revised

Published

24.02.2019 17.06.2019 27.06.2019

Recommended by

dr. Ni Min Kan

Sample citation of this publication:

Irina V. Rasina, Oles V. Fesko. "Approximate optimal control synthesis for nonuniform discrete systems with linear-quadratic state". Program Systems: Theory and Applications, 2019, 10:2(41), pp. 67-77.

10.25209/2079-3316-2019-10-2-67-77 url http://psta.psiras.ru/read/psta2019_2_67-77.pdf

The same article in Russian: 10.25209/2079-3316-2019-10-2-79-91

About the authors:

Irina Viktorovna Rasina

Ailamazyan Program Systems Institute of RAS

[ha 0000-0001-8939-2968 e-mail: irinarasina@gmail.com

Oles Vladimirovich Fesko

Ailamazyan Program Systems Institute of RAS

[ha 0000-0002-9329-5754 e-mail: oles.fesko@hotmail.com

Эта же статья по-русски:

10.25209/2079-3316-2019-10-2-79-91