On the nonparametric identification and dual adaptive control of dynamic processes Текст научной статьи по специальности «Физика»

УДК 62-503.57

On the Nonparametric Identification and Dual Adaptive Control of Dynamic Processes

Alexander V. Medvedev*

Siberian State Aerospace University Krasnoyarskiy Rabochiy, 31, Krasnoyarsk, 660014

Russia

Anastasia V. Raskina^

Institute of Space and Information Technology Siberian Federal University Kirenskogo, 26, Krasnoyarsk, 660041

Russia

Received 10.05.2016, received in revised form 24.07.2016, accepted 20.11.2016 A problem of nonparametric control of dynamic objects in discrete-continuous process is considered. The emphasis is placed on development of dual control non-parametric algorithms. Results of simulation experiment are described in details. These results demonstrate that the quality of control can be considerably improved in comparison with standard controllers by using the non-parametric dual control algorithm.

Keywords: dynamic process, non-parametric dual control, adaptive system, standard controller. DOI: 10.17516/1997-1397-2017-10-1-96-107.

Introduction

The modern control theory falls largely into category of parametric theories. It means that some parametric framework or an equation with free parameters [1] is assumed to describe the process. This approach has been developed within the framework of the theory of adaptive control systems [2] and it assumes that object parameters are known. However, a priori information is often not sufficient to reasonably choose the parametric model class. There are some things that complicate the problem solution: random noise with unknown distribution law, uncertainty and incompleteness of data. It is natural to use the non-parametric control system theory, when a priori information is not sufficient to choose a parametric model of the process [3,4]. Unlike the parametric control system theory, the non-parametric theory assumes that only qualitative indicators of a system are known. This means that the use of non-parametric theory allow us to completely exclude the definition of the parametric structure of an object model.

In what follows we consider control problems in the context of non-parametric uncertainty and also the case when a priori information complies with both parametric and non-parametric types. This paper is devoted to the study of the non-parametric algorithm of dual control of dynamic processes.

* saor.medvedev@sibsau.ru tstasy144@yandex.ru © Siberian Federal University. All rights reserved

1. The Level of a priori information

Various levels of a priori information are considered by A. Feldbaum [5]. In this paper the following levels of prior information are analysed [3].

The level of parametric uncertainty is the first level of a priori information which is conceded below. The parametric level of a priori information means that the parametric structure of the model and some characteristics of random noises with zero mean and limited variance are known. The iterative probabilistic procedures are used for estimating various parameters. Under these conditions the problem of identification is solved in "narrow sense".

The next level of a priori information is the level of nonparametric uncertainty. Nonparametric level of a priori information doesn't imply knowledge of the parametric model but it implies some information on qualitative character of dynamic processes. For example, information on whether the dynamic process is linear or nonlinear. If the process is nonlinear then the nature of its nonlinearity is required. The methods of nonparametric statistics are applied to the solution of the identification tasks (identification in "all-inclusive sense" [2]).

The third level is the level of parametric and nonparametric uncertainty. Here a priori information is neither parametric nor nonparametric. Then the problem of identification is formulated with the use of both parametric and nonparametric a priori information. The solution of such problems is very important from the practical point of view.

2. Non-parametric identification

Fig. 1 shows the block diagram of the dynamic process simulation.

Fig. 1. Scheme of the dynamic process simulation

The following notations are used in Fig. 1: u(t) is input of the process, x(t) is output of the process, t is time, xs(t) is output of the model, hf and hy are random noises of measurements corresponding to the process variables, £(t) is the vector of random disturbance.

The process belongs to the class of discrete-continuous processes. It means that the dynamic process is continuous but the "input-output" variables of the process are controlled within inter-

val A(t). By this means the sample of "input-output" variables [xi,ui, i = 1,s} is formed, where s is the sample size. The index h is omitted for reasons of simplicity.

Let us consider a dynamic object in terms of various levels of a priori information.

2.1. Nonparametric identification for the level of nonparametric uncertainty

The level of nonparametric uncertainty implies that dynamic object is linear but the structure of the parametric model is unknown. The order of the equation can not be determined from a priori information. It is known that a linear dynamic object with zero initial conditions can be described in term of the Duhamel integral. It has the form [6]

x(t) = i h(t — t)u(t)dr, (1)

J 0

where h(t — t) is the impulse response of the system which is the derivative of the transfer function k(t), i. e. h(t) = k'(t). It is known that the inverse operator to (1) is the operator

u(t) = i v(t — t)x(t)dT, (2)

0

where v(t — t) is the impulse response of the object in the direction of the "output-input" and v(t) — w'(t), where u(t) is the transfer function of the system in the same direction. Therefore, the problem is now to determine impulse response (h(t), v(t)). One of the possible ways to solve this problem is to use the Heaviside function 1(t) as input and to determine the impulse response of the object on the basis of measurements [xi = ki, ti, i = 1,s}. Non-parametric model (1) is

xs(t) = I hs(t — T,ks,ts)u(T)dT, (3)

0

where hs(t — t, ks,ts) is the non-parametric estimation of the impulse response, ks = (k1,..., ks), ts = (t1,...,ts). The non-parametric impulse response hs(t — T,ks,ts) can be obtained in the following way. We use the Heaviside function 1(t) as input. The values of the output variable xt form the system transition function kt = xt. Now one can obtain the sample ks = (k\, ...,ks),ts = (ti, ...,ts). The obtained sample can be used for the non-parametric evaluation of the regression function with the use of the Nadaraya-Watson estimation [13]

where *(•) is the kernel function, cs is the bandwidth. Function *(•) should satisfy the following conditions [5]:

$(z) < to, -/ *(dx =1, lim1^= S(t — ti) (5)

zen(z) cs Jn(x) \ cs ) ^^ cs \ cs J

cs > 0, s = 1, 2,...; lim cs = 0; lim scs = to. (6)

Let us multiply the numerator and the denominator in (1) by -1

SCs

£I=i W -A -£r

ks(t) =-/ Cy s • (7)

¿^i=1 ^ Cs J scs

The denominator in (7) is estimate of the probability density Ps. Because the time interval At is constant then Ps is also constant: Ps = const. Let us insert this constant in function $(•) and obtain new kernel function H(•)

H » = ta ■ (8)

Then the estimate of regression ks takes the form

k*(t) = — E k^t—). (9)

i=1 \ Cs J

SC

Since the impulse response h(t) is the derivative of the transfer function k(t) with respect to time

h(t) = ^, w dt '

then we have

hs(t) = kS(t) = S- EkiH'(. (10)

SCs ^ V Cs

i=1 x y

Thus, upon substituting non-parametric estimate of the impulse response (10) into (3), we obtain the non-parametric model of linear dynamical system

Xs(t)= i — E kiH'( u(T )dT. (11)

J0 SCS i=1 \ cs )

The integration of expression (11) is performed numerically.

2.2. Nonparametric identification for the level of nonparametric and parametric uncertainty

For the level of nonparametric and parametric uncertainty the structure of the dynamic process can be partially parameterized. Therefore, one can assume that the process is described by the following equation

Xt = f (Xt-1,Xt-2, ...,Xt-m,Ut), (12)

where f (•) is unknown functional, Xt and ut are the measured values of input and output variables of the process at discrete time moments, m is the system order [5]. It is determined on the basis of available a priori information.

In this case, the parametric structure of the process is partially unknown. The form of the functional f is not completely defined. The model of the process can be defined by the following nonparametric estimation of a regression function with discrete observation {Xi, uiy i = 1,s}

esu ^ nm=i * ( jj

j 1 v ' (13)

s ei=I *(^sui) nm=i $( jj '

The value of bandwidth cs follows from the condition of minimum of quadratic criterion:

R(cs) = y^ixs(uk) - Xk)2 = min, k = i. (14)

Z-' Cs

k = l

To calculate the value of bandwidth the method of sliding test is used.

3. Non-parametric dual control

The deficiency of a priori information results in the necessity to study and control an object at the same time. This type of control is called the dual control. The dual control was discovered by A. A. Feldbaum in 1960 and it was further developed on the basis of the statistical decision theory [5]. One should note that trainable control systems are the systems that have "memory". In other words, they are able not only to examine the object performance data but also to develop reasonable controlling actions. The dual control theory was further developed by various scientists, particularly by B. Wittenmark [7]. It was found that the optimal control algorithm is very cumbersome, and substitution of optimal control algorithms for nonoptimal dual controllers was proposed. The noise probability distribution and the parametric object model are assumed to be known. Such approach is only applicable to control the well-characterized processes with known parametric model structure. Another example of application of adaptive dual control algorithm under condition of parametric uncertainty is the development of dual control algorithm for unstable object [8]. The bicriteria approach is used for dual regulator synthesis. This approach uses two criteria that are consistent with two control objectives. They are control of the system output and the system excitation for speeding up the parameter estimation. As in the previous case, the control algorithm is applicable when the structure of object or regulator model is accurate to set of parameters. The adaptive dual control theory for stochastic systems with constant but unknown parameters has been developed [9,10]. Control methods for linear discrete system with dynamic random parameters has been also studied in the case of interference [11]. The control problems in the above studies were discussed in the context of parametric uncertainty. In case when the dynamic process structure is not completely specified, one can use neural networks to solve control problems [12]. This paper is devoted to the development of dual control with the use of the nonparametric dual control theory [5].

Fig. 2 shows a block diagram of the process control.

The following notation is used in Fig. 2: x*(t) is the reference signal, CU is the control unit.

Let us introduce operator A that defines the process, i.e., x(t) = A < u(t) >, where u(t) is the control input and x(t) is the output variable of the process. If there exists an inverse operator A-1, i.e., AA-1 = 1 (1 is the unit operator) then

A-1x(t) = A-1 A < u(t) >, u(t) = A-1x(t). (15)

Upon setting the trajectory x(t) = x*(t) , the ideal value u*(t) is found from (15).

Non-parametric dual control algorithm described in detail in [6] has the following discrete form

us+1 = u* + Aus+1, (16)

x(t) =k(t)

Fig. 2. Block diagram of dynamic object control

where variable u* accumulates information about the object for the subsequent formation of the adaptive control action, and Aus+i = e(x*s+1 — xs) is the probing step. The parameter e is found from the minimum of the quadratic criterion

R(e)

k=i

(xs+1 xs)

mm.

£

(17)

This is the dualism of algorithm (16). The form of the first term u*s in equation (16) depends on the level of a priori information about the object.

3.1. Nonparametric control for the level of nonparametric uncertainty

In this case, the parametric model of the object is unknown, and the object is linear. Operator A from (15) can be represented by expression (1), and inverse operator A-1 is of the form (2).

Let us consider a linear dynamic system with non-parametric level of a priori information. First term u* in equation (16) is the non-parametric evaluation of the inverse operator A-1. To define this operator it is necessary to specify the dynamic system impulse characteristics v(t) in the opposite direction (output-input). It is impossible to carry out such experiment on the real object. Then the impulse characteristics v(t) can be obtained with the use of the object model. Let us assume that the object output is the unit step function 1(t). Then using relation

xs(t) = 1(t) = hs(t — r,ks,ts )u(t )dr, J 0

(18)

one can obtain samples {ui = —i,ti, i = 1, s}.

Thus, estimate of the inverse impulse response v(t) is derived from the estimate of the reverse transfer function, that is, v(t) = ~~jtj~). Non-parametric estimation of the inverse impulse response v(t) has the form

(t) =

(19)

Substituting (19) in (2), we obtain estimate of the inverse operator of the linear dynamic system

■it>=i -L

J 0 scs

3=1

(t )dT,

(20)

v

u

where x*(r) is the reference signal. The integration of expression (20) is performed numerically. Let us note that unknown operators A and A-1 are estimated for the initial transfer function of the process (the equation of the process is unknown) in the class of non-parametric statistics.

3.2. Nonparametric control for the level of parametric and nonparametric uncertainty

In this case the structure of the dynamic process can be partially parameterised, i.e., an object representing a dynamical system is described by the equation xt = f (xt-1 ,xt-2, ■■■,xt-m,ut), where m is determined on the basis of available a priori information. The term u*(t) from equation (16) can be represented by non-parametric estimate of the regression function for discrete observations {xi,i = 1,s} in the form

Let us analyse the dualism of algorithm (16). In the initial stage of control the term Aus+1 plays the main role (accumulation of information). When information is accumulated, the term u*s is more important in forming the control signal. Thus, in dual control the object is studied in one stage, and it is adjusted in another.

4. Numeral Experiment

The methods of statistical modelling are used to test functionality of non-parametric identification and dual control algorithms. The object is represented by the equation of the form: xt = 0.2xt-1 — 0.3xt-2 + 0.4xt-3 + ut . This equation is used in computational experiment to simulate a real process. The transfer function of the object is presented in Fig. 3.

The control variable is defined by the equation: u(t) = sin(0.5t). In the first experiment variable m is unknown. The model of the object is constructed with the use of non-parametric model (11). Simulation results are shown in Fig. 4:

The following notations are used in Fig. 4: x(t) is output of the process, xs(t) is output of the model. The square error of the simulation for this experiment is 0.015.

In the following experiment the structure of the dynamic process can be partially parameterized, because value of m is known (m = 3). The model of the object is constructed with the use of non-parametric model (13). The random disturbance £(t) = 5%. Simulation results are presented in Fig. 5

(21)

MO

Fig. 3. Transfer function of the process

15 ■■..)..■ i | ■ ... | ■ ... | .... | .... t

! 10 IS M ii 30

Fig. 4. Results of the identification process using model (11)

Fig. 5. The results of the identification process using model (13)

The square error of the simulation is 0.023. One can see from Figs. 3-4 that non-parametric models (11) and (13) successfully solve the identification problem.

In the next experiment the input controlled variable is defined by the equation: u(t) = sin(0.2t). The model is constructed with the use of non-parametric model (13). The random disturbance acting on the object £(t) = 7. The simulation results are shown in Fig. 6.

The square error of the simulation is 0.056. Models (11) and (13) may be used to control a dynamic process.

The most effective algorithms in the context of non-parametric uncertainty are algorithms that form the control action using feedback based on error (residual error). Examples are the conventional regulating algorithms (P-regulators, Proportional plus reset regulators and PIDregulators). Let us compare the results of non-parametric dual control algorithm (16) and the typical algorithm with proportional plus reset regulator. The selection volume (ui, xi) is 100. The results are shown in Fig. 7 with the stepwise control action.

The following designations are adopted in Fig. 7: x(t) is the output of the object when the control unit is a nonparametric dual control regulator, x(t) is the output of the object when the control unit is the PI regulator, x*(t) is the reference signal. The first term of the dual control

0,4 0.2 0.0 -0,2 -0,4 -0,6 -0,E

Fig. 6. The results of the identification process using model (13)

yr- x(t) x(t) iC ,

v

V /

0 10 20 30

Fig. 7. The results for the stepwise reference signal

algorithm (16) is defined in (20). The square error of the control for the nonparametric regulator is 0.07, for the PI regulator is 0.34.

Let us consider the function of nonparametric dual control algorithm in detail. Control training begins with the first observation of x1 and u1. In the initial stage the control is required some time to set the object in a given state (accumulation of sample). In the next stages the algorithm almost instantly reaches the required state. The behaviour of two terms of nonparametric dual control algorithm is shown in Fig. 8.

Let us consider the case when the noise is equal to 7% (see Fig. 9).

As can be seen in Fig. 9 the nonparametric control algorithm is more tolerant to interference in comparison to the PI control algorithm.

In the next experiment (Fig. 10) the reference signal is defined by the equation x*(t) = 1.25 + sin(0.5t). In this case, the first term of dual control algorithm (16) is described by equation (21). The value of the system order is equal to 3.

As can be seen in Fig. 10 the proportional plus reset regulator needs more time to adjust to the trajectory. On the contrary, the non-parametric regulator leads object to the trajectory after the "first-step-training". The control error of non-parametric regulator is 0.12, and the control error of proportional plus reset regulator is 0.46.

Let us consider the situation when x*(t) is prescribed in a random way (Fig. 11).

Fig. 8. The behaviour of two terms of nonparametric dual control algorithm

Fig. 9. The control results with the stepwise reference signal when the noise is equal to 7%

Fig. 11 shows the algorithm performance in the case of random reference signal. As can be seen the non-parametric dual control algorithm outperforms a typical proportional plus reset regulator for accuracy. The results presented in Fig. 11 show that the non-parametric regulator provides high control quality even in such "exotic" situation. None of the known regulators can handle such problem. The presented results are of considerable interest from the theoretical point of view.

Conclusion

The dual control models and algorithms are intended for use in the case when a priori information about the object is not sufficient, and we can not choose the right parametric model for the object. In this case, the integral convolution, namely, the Duhamel integral is used to describe the object. Further, the Wiener-Hopf equation is not solved but the non-parametric evaluation of the system weighting function is used on the basis of the object "input-output"

X(t)

■ / x(0

x(t)

J

0 10 20 30

Fig. 10. The control results when the reference signal is the specified trajectory

x'(X) x(t) x(t)

2 " 1 \

0 f i ■ ■ . | . ---—|—■—■—■—■—j—m

0 10 20 30

Fig. 11. The results of control when the reference signal is random

observation results.

The non-parametric control algorithm development is connected with the object inverse operator evaluation in respect to the initial Duhamel integral. The next step is to obtain system weighting function by means of the non-parametric evaluation. The presented results of simulations show that such method is fully justified.

Besides, the non-parametric algorithms also demonstrate a quite high control quality in the case of partial non-parametric uncertainty.

References

[1] E.Asarin, A.Donze, O.Maler, D.Nickovic, Parametric identification of temporal properties, In Khurshid,S., Sen, K. (eds.) RV 2011. LNCS, Springer, Heidelberg, 2012, no. 7186, 147-160.

[2] Ja.Z.Cypkin, Adaptation and learning in automatic systems, Moscow, Nauka, 1968 (in Russian).

[3] A.V.Medvedev, The theory of non-parametric systems. Control-I, Vestnik SibGAU, 48 (2013), no. 2, 57-63 (in Russian).

[4] A.V.Medvedev, The theory of non-parametric systems. Control-II, Vestnik SibGAU, 49 (2013), no. 3, 85-90 (in Russian).

[5] A.A.Fel'dbaum, Fundamentals of the theory of optimal automatic systems, Moscow, Fiz-matgiz, 1963 (Russian).

[6] A.V.Medvedev, Fundamentals of the theory of Adaptive Systems, Krasnoyarsk, SibGAU, 2015 (in Russian).

[7] B.Wittenmark, Adaptive dual control methods: An overview., In 5th IFAC Symposium on Adaptive Systems in Control and Signal Processing, Budapest, Hungary, 1995, 67-72.

[8] N.M.Filatov, U.Keuchel, H.Unbehauen, Dual control for an unstable mechanical plant, Control Systems, IEEE, 16(1996), no. 4, 31-37.

[9] C.J.Wenk, Y.Bar-Shalom, A multiple model adaptive dual control algorithm for stochastic systems with unknown parameters, Automatic Control, IEEE, 25(2003), no. 4, 703-710.

[10] Duan Lia, Fucai Qianb, Peilin Fuc, Optimal nominal dual control for discrete-time linear-quadratic Gaussian problems with unknown parameters, Automatica, 44(2008), no. 1, 119-127.

[11] E.Tse, Y.Bar-Shalom, An actively adaptive control for linear systems with random parameters via the dual control approach, Automatic Control, IEEE, 18(2003), no. 2, 109-117.

[12] S.Fabrit, V.Kadirkamanathant, Dual Adaptive Control of Nonlinear Stochastic Systems using Neural Networks, Automatica, 34(1998), no. 2, 245-253.

[13] E.A.Nadaraya, Non-parametric estimation of the probability density and the regression curve, Tbilisi, Izd. Tbil. un-ta, 1983.

О непараметрической идентификации и адаптивном дуальном управлении динамическими процессами

Александр В. Медведев

Сибирский государственный аэрокосмический университет Красноярский рабочий, 31, Красноярск, 660037

Россия

Анастасия В. Раскина

Институт космических и информационных технологий Сибирский федеральный университет Киренского, 26, Красноярск, 660041

Россия

Рассматривается задача непараметрического управления динамическими объектами с дискретно-непрерывным характером процесса. Основное внимание уделяется построению непараметрических алгоритмов дуального управления. Достаточно подробно изложены результаты вычислительного эксперимента, которые показывают, что при использовании непараметрического дуального алгоритма управления существенно повышается качество управления по сравнению с типовыми законами регулирования.

Ключевые слова: динамический процесс, непараметрическое дуальное управление, адаптивные системы, типовые законы регулирования.