A numerical method for solving linear-quadratic control problems with constraints Текст научной статьи по специальности «Математика»

URAL MATHEMATICAL JOURNAL, Vol. 2, No. 2, 2016

A NUMERICAL METHOD FOR SOLVING LINEAR-QUADRATIC CONTROL PROBLEMS WITH

CONSTRAINTS1

Mikhail I. Gusev

N.N. Krasovskii Institute of Mathematics and Mechanics, Ural Branch of the Russian Academy of Sciences, Ekaterinburg, Russia,

gmi@imm.uran.ru

Igor V. Zykov

N.N. Krasovskii Institute of Mathematics and Mechanics, Ural Branch of the Russian Academy of Sciences, Ekaterinburg, Russia,

zykoviustu@mail.ru

Abstract: The paper is devoted to the optimal control problem for a linear system with integrally constrained control function. We study the problem of minimization of a linear terminal cost with terminal constraints given by a set of linear inequalities. For the solution of this problem we propose two-stage numerical algorithm, which is based on construction of the reachable set of the system. At the first stage we find a solution to finite-dimensional optimization problem with a linear objective function and linear and quadratic constraints. At the second stage we solve a standard linear-quadratic control problem, which admits a simple and effective solution.

Key words: Optimal control, Reachable set, Integral constraints, Convex programming, Semi-infinite linear programming.

Introduction

The optimal control problems under integrally constrained controls were studied in many papers (see, for example, [1,5,7-9,12,13]). In [2,3] the authors considered the linear control system with integrally constrained control. They studied the problem of minimization of a linear terminal cost under linear terminal constraints and proposed a saddle-point method to solve it.

We propose here a two-stage numerical algorithm for the solution of above optimal control problem. It is based on constructing the reachable set of the control system, and we use here the well-known result that this set for a linear control system with integral quadratic constraints on controls is an ellipsoid in the state space. Then, at the first stage, we find a minimum of a linear function on the intersection of the polyhedron and ellipsoid. This problem may be solved numerically in different ways. At the second stage, we solve the standard linear-quadratic control problem with fixed endpoints of the trajectory, this problem has a simple and effective solution in the linear case. The typical unpleasant feature of the optimal control problem with terminal cost is a nonuniqueness of solutions, which always takes place if the endpoint of the optimal trajectory belongs to the interior of the reachable set. This leads to additional difficulties in the construction and implementation of numerical algorithms. The method proposed here avoids these problems.

*The research is supported by Russian Science Foundation, project no. 16-11-10146.

1. Notation and problem statement

We use the following notations. By AT we denote the transpose of a real matrix A, 0 stands for a zero vector of appropriate dimension, I is an identity matrix. For x, y € Rn let (x, y) = xTy be the inner product, xT = (xi,..., xn), ||x|| = (x,x)2 be the Euclidean norm, and Br(x): Br(x) = {x € Rn : ||x — x|| < r} be a ball of radius r > 0 centered at x. For a set S C Rn let dS, intS, clS, coS be a boundary, an interior, a closure, and a convex hull of S respectively; Vg(x) is the gradient of a function g(x) at the point x, is the Jacobi matrix of a vector-valued function

f(x). For a real k x m matrix A a matrix norm is denoted as ||A||kxm. The symbols Li, L2 and C stand for the spaces of summable, square summable and continuous functions respectively. The norms in these spaces are denoted as || ■ ||Ll, || ■ ||L2, || ■ ||C.

We consider a linear control system with integral constraints on a controls

£c(t) = A(t)x(t) + B(t)u(t), t € [to, ti], x(to) = x0, (1.1)

where x € Rn, u(t) € Rr, A(t),B(t) are integrable on [to,ti] matrix functions. Let the control constraints are defined by the integral quadratic inequality

u(-) € U = ju(-) € L2 : J(u(-)) = ||u(-)||L2 = ji ' uT(t)u(t)dt < ^^, (1.2)

where ^ > 0 is a given number. For any u(-) € L2 there exists a unique absolutely continuous solution x(t) of system (1.1), which is defined on interval [t0,t1].

Assume that m x n-matrix D, vectors d € Rm and c € Rn are given. We consider here the following optimal control problem for system (1.1):

I(u(-)) = cTx(ti) ^ min, (1.3)

under constraints

u(-) € U, Dx(ti) < d. (1.4)

Definition 1. The reachable set G(ti) of system (1.1) at time instant ti is a set of all states x(ti) that can be reached by the trajectories of (1.1) corresponding to controls u(-) € U :

G(ti) = {x € Rn : 3u(-) € U, x = x(ti)},

where x(t) is a solution of (1.1).

The considered optimization problem may be split into two following subproblems.

Problem 1 (the first stage): to find x* € Rn that solves the finite-dimensional optimization problem

cTx ^ min,

under constraints

x € G(ti), Dx < d.

Here the reachable set G(ti) is an ellipsoid in Rn those parameters are calculated effectively. Thus, the optimization Problem 1 may be solved by employing the methods of linear or convex programming.

Problem 2 (the second stage): to find a control u(-) € U that steers the trajectory x(t) of (1.1) to point x(ti) = x* and minimizes functional J(u(-)).

2. Description of reachable sets

Let X(i,r) = $(i)$-1(r), where $(t) is a Cauchy matrix, satisfying the equation

$(t) = A(i)$(i), $(îq)= I. A solution of (1.1) at time instant t1 has the form

x(t1) = x + f1 X (i1,r )B (t )u(t )dr, (2.1)

Jtn

where x = X(t1, t0)x0. Take an arbitrary vector l € Rn, l = 0 and calculate the maximum of inner product (l,x(t1)) over all x(t1) € G(t1):

rti

max (Z.x(ii)) = lTX + max / lTX(ti,r)B(t)u(t)dr = lTX + max (v(-),u(-)) =

i)ec(ti)v Jto M),U(0><^2

fii

x(ti)eG(ti)............U()& Jt0 ............«),<)><M2

= /Tx + /x|K)||L2 = ß^lTW(t!)l + /Tx.

Here v(t) = BT(t)X T(ti,t)l,

/• ti

(v(-),u(-)) = vT (t)u(t)dt

Jtn

'to

is an inner product of functions v(-),u(-) in the Hilbert space L2, and symmetric matrix W(t) is defined by the equality

W(t) = / X(i,r)B(t)Bt(t)X t(i,r)dr.

to

Differentiating the last equality in t we get the following matrix differential equation for W:

W(t) = A(t)W (t) + W (t)AT (t) + B (t)BT (t), W (to) = 0. (2.2)

It is known (see, for example, [10]), that system (1.1) is completely controllable on [t0,t1], if and only if W(t1) is positive definite. In this case G(t1) is a nondegenerate ellipsoid

G(t1) = {(x - x) 1 W-1(t1)(x - x) < ^2}.

If the system is not completely controllable, the reachable set is a degenerate ellipsoid (ellipsoid, lying in a subspace of dimension less than n). It is obvious, that x € dG(t1) if and only if there exists a vector l = 0 such that (l,x) = maxyeG(tl)(l,y), and hence, a control u(-) steering the system trajectory to point x satisfies the relation

r lTX(t,t)B(t)u(t)dr = max 2«),u(-)), (2.3)

./to («(•)>«(')><M2

where v(t) is defined above. If the system is completely controllable, v(-) = 0 and the equality (2.3) uniquely determines u(t) = av(t), where a = ^/||v(-)||l2 . Denote p(t) = aX T(t1, t)l, assuming that p(t) is a nontrivial solution of the adjoint differential equation p(t) = —AT(t)p(t) and the relations

u(t) = BT (t)p(t), J (u(-)) =

hold. Define a Hamiltonian H(p, t, x, u) by the expression

ii(p, t, x,«) = + pT(A(t)x +

Equating to zero the gradient of H(p(t), t, x(t), u) in u, from the concavity of the Hamiltonian in u we get that control u(t) = BT(t)p(t) satisfies the maximum principle

H (p(t),t,x(t),u(t)) = max H (p(t), t, x(t), v),

where p(t) is a nontrivial solution of the adjoint differential equation

dH

p{t) = - — {p(t),t,x(t)Mt))-

This form of the maximum principle corresponds to the problem of minimization of a convex functional J(u(-)) on solutions of linear system (1.1)

J(u(-)) — min, x(t0) = x0, x(ti) = x. (2.4)

Since for a linear-convex optimal control problem the maximum principle provides necessary and sufficient optimality conditions, a control u(t) = BT(t)p(t), found from the maximum principle, solves problem (2.4). Inversely, let u(-) be the solution to problem (2.4) and let J(u(-)) = Then we will have x € G(ti). Indeed, if we assume that x / dG(ti), then

(x — x)TW-i(ti)(x — x) = v2 <

Hence, the control system can be transferred to the point x by the control v(-), for this control we have J(v(-)) < v2 < which contradicts to the optimality of u(t). Thus, we come to the following statement.

Assertion 1. Let system (1.1)4 be completely controllable. In order to control u(-) steers a trajectory of system (1.1) to point x, lying on the boundary of the reachable set G(ti), it is necessary and sufficient that this control solves the extremal problem (2.4) and the minimum of functional J equals to

3. The algorithm of solving the optimal control problem

Having the parameters of the reachable set given, let us describe the algorithm of the solution of the problem. Consider system (1.1) and assume that it is completely controllable. Then, at the first stage, we should solve the Problem 1

cTx - min,

Dx < d, (3.1)

(x — x)TW-i(ti)(x — x) <

The problem (3.1) is a linear programming problem with an additional constraint, which is defined by an inequality with a positive definite quadratic form. This problem may be solved by various algorithms. For example, in [11] the authors proposed a finite convergent algorithm for solution of the problem of type (3.1).

The second way of finding the solution concerns with description of the ellipsoidal reachable set via its support function:

G(ti) = {x € Rn : (1,x) < ^(l), Vl € S}, where the support function has the form

with S = {1 : ||1|| = 1}. Then, the problem (3.1) may be written as follows

cTx — min,

(3.2)

Dx < d, 1Tx — ^(1) < 0,1 € S.

The problem (3.2) is a semi-infinite linear programming problem [6], which can be solved by a number of effective numerical algorithms.

Choosing a finite grid of N vectors k € S, we can approximate the problem (3.2) by the following linear programming problem with a finite number of constraints

cTx — min,

T (3.3)

Dx < d, 1,Tx — ^(1,) < 0, i = 1,..., N.

Let a solution x* of problem (3.1) ((3.2), (3.3)) be obtained (this solution is unique as a rule). Then we come to the next stage: to find the solution of the next problem

J (u(-)) — min, x(t0) = x0, x(ti) = x*. (3.4)

For completely controllable system this solution does exist and is unique. It may be obtained from the maximum principle:

u(t) = B T(t)p(t), p = —AT(t)p(t), p1 = p(ti). (3.5)

Represent p(t) as follows

p(t) = X T(ti,t)pi,

then we have

f ti

x(ti) = x+ / X (ti,T )B(t )Bt(t )X T(ti,T )pidr = x*. (3.6)

J t0

Thus, to find an optimal control it is sufficient:

1) to find a vector p1 from the linear equation

W (ti)pi + x = x*, (3.7)

2) to integrate adjoint equation (3.5) with boundary condition p(ti) = p1 and substitute p(t) into the formula for u(t).

We can put from the very beginning x(t1) = x + W(t1)p1 and solve a semi-infinite linear programming problem in the dual variables p1

cTW (t1)p1 — min

,__(3.8)

DW(ti)pl < d - Dx, FWttdp1 - ii^/lTW(ti)l < 0, leS.

The last form is more convenient, it may be used even in the case of degenerate matrix W(t1). Really, it is known [8] that if there exists a control from L2 that steers the control system from x0 to x*, than control u(-) solving the problem (3.4) is a linear combination of columns of matrix Bt(t)XT(t1,T). That is

u(t ) = Bt(t )X t (ti,r )p1

for some p1 € Rn. Substituting the control u(t) into equality (3.6) we get (3.7). Thus, the following theorem holds.

Theorem 1. Assume that the system of constraints (1.1), (1.4) is consistent. Then the optimal control in the problem (1.3)-(1.4) is given by formulas (3.5), where p1 is a solution to the linear semi-infinite optimization problem (3.8).

4. Example

Consider an illustrative example of optimal control problem for a linear control system described by the equations

x 1

X2,

X 2 = Xi + X3,

(4.1)

XX 3

u

Figure 1. Graphs of optimal controls

with to = 0, ti = 2n and x0 = (1,0,0)T. We consider the integral quadratic constraints on controls given by the inequality (1.2) with = 1.

Assume a matrix D and a vector d that determine the terminal constraints are

'0 1

0

D =10 -1 0 ,0 0 -1,

d=(0

Integrating a differential equations (2.2) for matrix W and calculating vector x we get

'9.4264 0.0005 6.2831N W(2n) = I 0.0005 3.1399 0.0000 6.2831 0.0000 6.2832

x

Let the terminal cost is determined by vector c = (0,2,2)T. Solving the problem (3.1) we get the solution

x* = (0.9993,-1.0000, 0.0000)T, p1 = (-0.0002, -0.3185, 0.0002)T.

The graph of the optimal control is shown in Figure 1 by the black line. Here x* is the interior point of the reachable set, so there are infinitely many admissible control inputs that steer the trajectories to point x*. Among these inputs the considered control input has a minimal value of the integral functional J(u(-)) which equals to 0.3185.

Consider another case and put c = (1,0,0)T In this case we have the following solution

x* = (-0.7729, -0.0003, 0.0000)T, p1 = (-0.5640, -0.0000, 0.5640) 1 .

T

Here (x* — x) 1 W 1(2n)(x* — x) = 1, hence x* belongs to the boundary of the reachable set. In this case there exists a unique optimal control input which is shown in Figure 1 by the blue line.

2

3

4

5

6

7

5. Further generalizations

Consider here the generalized statement for the previous problem. We assume that integral constraints restrict simultaneously a control and a trajectory of system (1.1) as follows

J(u(-)) = i J*1 (xT(t)Q(t)x(t) + i,T(t)R(t)u(t)^j di < y, (5.1)

where Q(t) is a nonnegative definite and R(t) is a positive definite matrix for every t € [to,til-Suppose Q(t) and R(t) to be the measurable and bounded matrix functions.

Assume that a pair (A(t),B(t)) is completely controllable on [t0,t1 ]. Let x(t0) = x0 be fixed and let x € G(t1). The set of trajectories satisfying inequality (5.1) is a compact set in C. Hence, there exists the solution to the problem

J (u(-)) — min, x(t0) = x0, x(t1) = x,

this solution is unique due to strict convexity of the functional J(u(-)). Consider a Hamiltonian

H(t,x,u,(p0,p)) = -P07; (xTQ(t)x + uT R(t)uj + pT(A(t)x + B(t)u).

According to the maximum principle (see, for example, [4,10]) there exist (p0,p(-)) = 0 such that u(t) maximizes a Hamiltonian, hence we have ^ = 0 and

dH

p(t) = = -AT(t)p(t) +p0Q(t)x.

Assuming p0 = 0, we get p(-) = 0. Then p(t) is a nonzero solution of the equation

p(t) = —AT(t)p(t),

and the condition dH/du = 0 implies the equality pT(t)B = 0, this contradict to the controllability conditions. Thus, p0 = 0, so we can take p0 = 1. In this case we have

u(t) = R-1(t)B T(t)p(t). (5.2)

Substituting control (5.2) into equations of the control system we get

x = A(t)x + B (t)R-1(t)B T(t)p,

p = -AT (t)p + Q(t)x. Thus, (5.3) is a linear homogeneous system of differential equations

(5.3)

x \ = /A(t) B (t)R-i(t)B (t)\/x\ (54)

pj = VQ(t) -AT(t) JW' (5.4)

The solution of (5.4) with the initial state x(t0) = x0, p(t0) = p0 has the form

Xit'O=Y (tKx0

where Y (t) is the fundamental matrix of system (5.4) satisfying the initial condition Y (to) = I. Representing Y(t) as a block matrix

Y (t)= ^Yn(t) Y12(t)\

Y (t)= VY2i(t) Y22(t^ '

we will have

x(t) = Yn(t)x0 + Y12 (t)p0,

p(t)= Yi2(t)x0 + Y22(t)p0, (55)

x = x(ti)= Yii(ti)x0 + Yi2(ti )p0, .

u(t) = R-i(t)BT(t) (Yi2(t)x0 + Y22(t)p0) .

Substituting x(t) and u(t) into J(u) we get

1 rti

J(u(-)) = i ji 1 (x°TYiiT(i) + p°TYi2T(t)) Q(t) (Fn(i)x° + F12(i)p°) + (x°TYi2T(t) + p°TY22T(t^ B(i)R-lT(i)BT(t) (Yi2(t)x0 + Y22(t)p0)

, /2

or

where

x°TSiix° + x°TSi2P° + P°TS22P° < /2, (5.6)

511 = J^1 (VnT(i)Q(i)Yii(i) + Yi2T(t)B(t)R-i(t)BT(t)Yi2T(t)) dt,

512 = 2 jT (YnT(i)Q(i)Yi2(t) + Yi2T(t)B(i)R-i(i)BT(i)Y22T(t)) dt, S22 = J^ (Yi2T(t)Q(t)Yi2(t) + Y>2T(t)B(i)R-i(i)£T(i)l22T(t)) dt.

I to

Matrices Sii, S22 are, obviously, nonnegative definite.

Assertion 2. //the pair (A(t),B(t)) is completely controllable on [t°,ti] then S22 is a positive definite matrix.

Proof. Suppose to the contrary that there exists p° = 0 : S22p° = 0. Let us take x° = 0 and a € R. Denote

p(t) = Y22(t)p°, u(t) = R-iBT(t)p(t), x(t) = Yi2(t)p°. Multiply p° on a, then au(t) satisfies (5.1), that is

fti /,2

ti

aM uT(t)R(t)u(t)dt <

to

for any a, this implies u(t) = 0, that is BT(t)p(t) = 0. To zero controller u(t) and state x° = 0 there corresponds the trajectory x(t) = 0. Hence, from the equations of adjoint system we get

p(t) = -AT(t)p(t) + Q(t)x(t) = -AT(t)p(t).

Thus, p(t) is a nonzero solution of the adjoint homogenous system such that pT(t)B(t) = 0. This contradicts to the controllability of the system.

In the last case the problem (3.1) may be written as follows

cTp° — min

T T T (5.7)

Dp° < d, p° ' S22p° + x° ' Si2p° < /2 - x° ' Siix°,

where

cT = cTYi2(ti), D = DYi2(ti), d = d - DYii(ti)x°.

This is also a problem of the above type with the linear constraints and one quadratic constraint. Solving this problem we can obtain an optimal control by the explicit formulas (5.5).

The reduction of the integral constraints to quadratic constraint (5.6) allows us to easily generalize the considered problem to the case, when x° is not fixed but belongs to some polyhedron.

2

6. Conclusion

We consider an optimal control problem for a linear system with integrally constrained control, with a linear terminal cost and with terminal constraints given by a set of linear inequalities. This problem is, in fact, the ill-posed problem because of nonuniqueness of optimal control, which always takes place if the end point of the optimal trajectory belongs to the interior of the reachable set of the control system. We propose here a simple numerical algorithm for solving the optimal control problem, which uses a known explicit description of the reachable sets for linear systems with integral quadratic constraints on control functions. The algorithm is based on the reduction of considered problem to the solution of a finite-dimensional convex programming problem in primal or dual variables. This method allows to avoid difficulties related to nonuniqueness of optimal control.

REFERENCES

1. Anan'ev B.I. Motion correction of a statistically uncertain system under communication constraints // Automation and Remote Control, 2010. Vol. 71, no. 3. P. 367-378.

2. Antipin A.S., Khoroshilova E.V. Linear programming and dynamics // Trudy Inst. Mat. i Mekh. UrO RAN, 2013. Vol. 19, no. 2. P. 7-25.

3. Antipin A.S., Khoroshilova E.V. Linear programming and dynamics // Ural Mathematical Journal, 2015. Vol. 1, no. 1. P. 3-19. DOI: http://dx.doi.org/10.15826/umj.2015.L001

4. Arutyunov A.V., Magaril-Il'yaev G.G. and Tikhomirov V.M. Pontryagin maximum principle. Proof and applications. Moscow: Factorial press,(in Russian), 2006. 124 p.

5. Dar'in A.N., Kurzhanski A.B. Control under indeterminacy and double constraints // Differential Equations, 2003. Vol. 39, no. 11. P. 1554-1567.

6. Goberna M.A.,Lopez M.A.Linear semi-infinite programming theory: An updated survey //European Journal of Operational Research, 2002. Vol. 143, Issue 2. P. 390-405.

7. Gusev M.I. On optimal control problem for the bundle of trajectories of uncertain system // Lecture Notes in Computer Sciences. Springer,2010. Vol. 5910. P. 286-293.

8. Krasovskii N.N. Theory of Control of Motion. Moscow: Nauka, 1968. 476 p. [in Russian]

9. Kurzhanski A.B. Control and Observation under Conditions of Uncertainty. Moscow: Nauka, 1977. 392 p. [in Russian]

10. Lee E.B., Marcus L. Foundations of Optimal Control Theory. Jhon Willey 'S' Sons, Inc., 1967. 124 p.

11. Martein L., Schaible S. On solving a linear program with one quadratic constraint // Rivista di matematica per le scienze economiche e sociali, 1988. P. 75-90.

12. Ushakov V.N. Extremal strategies in differential games with integral constraints // J. of Applied Mathematics and Mechanics, 1972. Vol. 36, no. 1. P. 15-23.

13. Ukhobotov V.I. On a class of differential games with an integral constraint // J. of Applied Mathematics and Mechanics, 1977. Vol. 41, no. 5. P. 838-844.