CC BY
47
AUTOREGRESSION METHODS OF THE PRESET SIGNALS RECOGNITION IN THE PRESENCE OF THE UNKNOWN SIGNALS’ CLASS
BESRUK V.M.
Kharkov National University of Radio Electronics, Ukraine E-mail: bezruk@kture.kharkov.ua
Abstract. Some practical peculiarities ofrandom signals recognition methods based on the autoregression model are considered. Recognition algorithms both for the classical statement of the recognition problem and for non-traditional cases when the signals with unknown probabilistic characteristics are presented for recognition along with the signals preset in a probabilistic sense. Through statistical simulation the analysis of indices ofrecognition quality and speed of action for the considered algorithms of signal recognition was performed.
1. Introduction
When solving many problems of image recognition (obj ects, phenomena, states) their initial description is presented in the form of random signals realization [ 1 -5 ]. Methods of signals ’ recognition in many respects are defined by the type of the probabilistic model used for model used for the signals’ description and defined by the properties of a certain class of random processes [6-9]. The choice of the specific probabilistic model of signals is defined by practical peculiarities of the recognition problem being solved.
In a number of applied problems in particular in technical medical diagnostics, vocoder telephony, honeycomb communications, speech signal recognition with the adequate probabilistic model of signals can serve a model in the form ofthe autoregressionprocesses [6-15]. A number of methods of random signals’ recognition based on the signals’ description with an autoregression model is known [10-15]. These methods were obtained with classical statement of the random signals’ recognition under the assumptionthat random signals preset only in the probabilistic sense are presented for recognition. But in many applied problems, besides the preset signals for recognition, the unknown signals, their statistical characteristics being unknown and their learning sampling impossible to be obtained, can be presented for recognition [16-22]. Under such conditions special signal recognition methods should be used at an increased a priori uncertainty when one must take into account the presence of the unknown signals’ classes while recognizing along with the preset signals’ classes. In [ 16,17,20] some general methods of solving such recognition problems based on building own regions in the signals’ space or some of their statistics. In [18,19] peculiarities of the preset signals recognition spectral algorithms’ building are considered in the presence of unknown signals based on the probabilistic model in the form of the orthoganally decomposed signals.
The distinctive peculiarities of the present work is that it considers the solution of the preset random signals recognition problem in the presence of the unknown signals’
R&I, 2003, Ns 3
class on the basis of the autoregression model [21,22]. Signals’ recognition algorithms different in difficulty are given. Peculiarities of the recognition algorithms’ practical implementation and the results of some signal recognition algorithms’ investigation results are considered. Investigations are performed with the method of statistical simulation on the samples of signals with different type of the correlation function. Some results of vector optimization of autoregression algorithms for signals’ recognition by the totality of the indices of the recognition quality and fast action are considered [23]. In particular the estimates of the recognition algorithm operation characteristics in the form of the diagrams of signal recognition indices’ change and fast of action defined by the required signal recognition duration.
2. Statement of signal recognition problem
Let M Gauss random signals Xi(t), i = 1,M preset at the final time interval (0, T) by the totality of equally spaced references Xl, l = 1,L be subject to recognition. The signals
M
are presented with the probabilities Pi, with £ Pi = 1. The
i=1
signals differ at the level of the correlation functions which are a priori unknown, but the classified learning signals’ sampling (X^J = 1,L; r = 1,ni; i = 1,M} is preset. In this way the classical statement of the random signals’ recognition problems under conditions of a priori uncertainty is formulated. But in practice the necessity is widely met to perform recognition under conditions differing from the formulated problem statement is. In particular, besides the preset M signals the unknown signals, whose probabilistic characteristics are unknown and there is no possibility to obtain the learning samplings of such signals, canbe presented for recognition. Let us consider the peculiarities of the methods for recognizing signals whose probabilistic description is defined by the autoregression (AR) model. We will present some description algorithms for signals’ recognition based on the AR model obtained both for the classical statement of the problem and non-traditional conditions of signals recognition in the presence of the unknown signals’ class.
3. Signals’ recognition algorithms
With the classical statement of the signals’ recognition problem the general view ofthe recognition algorithm, optimal by the mean probability criterion of faulty recognition, is defined by the relation [3-5]
i = arg max(Pk W (X / ji k,Rk)} (1)
k=1,M ’ V ’
where W(X/|Ik,Rk) - L-dimensional signals’ distribution densities; X = (X1,... ,Xi,...,Xl)-
With a priori uncertainty the adaptive Bayes approach can be used for the stated problem solution. In the particular case of Gauss densities of signals’ distribution it results in asymptotically optimal recognition algorithm defined by the square classifier
i = argmin[(X-^^RU^X-|ak)-ln-^H, k = 1M
k |Rkl
(2)
187
where pk, Rk, | Rk I are estimates of the mean vector, correlation matrix and its determinant, respectively, obtained using the learning signals’ sampling.
It is seen that the main difficulty in recognition algorithm realization (2) is defined by square forms calculation. W ith a great dimensionality of the initial presentation of signals L their calculation will require significant expenditures; with the computational resources limitation will hamper practical realization of signals ’ recognition algorithms in the real time. Moreover, in this case the necessity to accumulate the learning signals’ samplings of a great volume as when calculating inverse correlation matrices the condition n > L should be met. Thus the stage of reducing dimensionality of the signals presentation at the cost of a small number of informative features choice in the image recognition problems is introduced.
Let us demonstrate that when using the AR model for signals’ description it is possible to receive recognition algorithms, which do not demonstrate the above mentioned limitations in the process of practical realization. Let us consider the peculiarities of choice of the informative features and building of the solving rules defining the signals’ recognition algorithms in the framework of the AR model.
The linear AR model of the random signal is described with the recurrent equation [6]
Xi = lj T^^aj, j = 1L, (3)
J l=1 J
where ^i, l = 1, p - are coefficients of the AR model of i-th signal; a j - are independent random values with a null mathematical expectation and unit dispersion; p - is the order
of the AR model of the i-th signal; (c a)2 -is the error prediction dispersion.
Construction of the AR model of the i-th random signal reduces to finding ofthe order and parameters of p, ^i,(c^)2 model. There are different means of finding them [6]. In particular, coefficients of AR l = 1,p can be found from
solution of Ull-W alker equation defined by the relation ^ = R_1p . Here p - are, respectively, the AR coefficients
vector and signal correlation coefficients vector (dimensionality p); R _1- is the inverse correlation matrix (dimensionality: p x p). In this case the required estimates of correlation coefficients are found using the learning samplings of the preset signals realizations. The order p of the AR model is chosen, firstly, from the required quality of the signals’ recognition when solving the recognition problems.
It is an easy matter to see that the AR process is a special case of the linear random process with a discrete time formed on the output of the linear AR filter with generative process in the form of discrete white noise [8,9]. The current references of the AR process are obtained through the weight summing p previous references of the process with adding the white noise references. The weighting coefficients are equal to the AR coefficients and a number of the filter units is equal to the AR model order.
The search of efficient algorithms for the AR coefficients calculation resulted in the application of the grid filters [7].
Parameters of the grid filters (GF) are reflection coefficients uniquely connected with the AR coefficients. The quantity of GF units is equal to the AR model order. When using GF the signals are presented with the model similar to the AR model
Xl =-! Klji+ajaj, (4)
l =1 J J
where dl~ 1 - is an error of the inverse prediction in the l -
th unit; Kl are reflection coefficients of GF.
Difference between the models (3) and (4) consists in that the model (4) uses regression on the previous errors of the
inverse prediction dj~1. In this case the regression coefficients are the GF reflection coefficients. In spite of the fact that one and the same probabilistic model makes the basis of the autoregression and grid filters there is a number of differences between them and they are essential from the practical point ofview. In particular, the GF are less sensitive to the round-of noise when realizing them using computing technique. The quality of GF operation is less influenced by the quantification errors as compared to the AR filters.
The analytical expression for the multidimensional density of Gauss random processes probability obtained with the AR model parameters [6]
W(X)
___________1_________
(2tc)L/2cL_p |Rp |1/2
exp(-S(X))
(5)
Here
S(X) = (Xp -mp)trR"1a2(Xp -mp) +
1 L p 2
+---2 Z [Xl-p-IMXl -p)]2, (6)
2c l=p+1 j=1
where Xp = (x1,...,xp) , mp = (m,...,pp) , Rp,
p - are, respectively, the observation vector, the mean vector, the correlation matrix, the signal mathematical expectation.
When substituting the given expression for L -dimensional signals’ probability density (5) into the general recognition algorithm (1) it is possible to obtain the recognition algorithm which takes into account the peculiarities of the preset signals’ AR model. In this case the distribution density is completely defined with the correlation matrix of p x p dimensionality, and also the prediction error received through the parameters of the AR model of the signal. For many practical problems when the AR model order p is a small (units, tens) with L signals realizations duration (hundreds, thousands), an approximate expression for multidimensional density of signals’ distribution can be use. Thus in the expression (6) the first summand canbe neglected i.e. the square form. In this case the distribution density will be defined by the second summand i.e. by the prediction error calculated for the signal realization presented for recognition. It is not difficult to see that as compared to the recognition algorithm (2) for the Gauss densities of signals’ distribution in this case practical realization of the signals’ recognition algorithm simplifies essentially. In the given case the recognition algorithm is defined using the solving rule
188
R&I, 2003, N 3
Kk(X) - Ki(X) + ln
(2 nc1)L_Pi (2^ak)L-pk
* 4
k = 1,M, 1 * l. (7)
Here
— 1 L pk k 2
Kk(X) = -y Z [Xi -Fk-Z^-(Xi.j -^k)]2(8) 2o k l=p+1 j=1j (8)
In this recognition algorithm the decision is actually made by the minimum ofthe prediction error (8) on the AR filter output.
Other simple algorithms for the classical statement of the preset random signals ’ recognition problem are given in [1315]. Recognition algorithms are based on application of the informative features in the form of the AR coefficient or GF reflection coefficients. The decision rules take into account distribution of these features (the AR coefficients or GF reflection coefficients) estimates obtained in the process of realization of the signals presented for recognition.
Thus, from the cited AR model peculiarities’ analysis it follows that it is expedient to choose the AR coefficients or GF reflection coefficients as the informative features of the signals being recognized. With the preset form of the generating process these parameters along with the prediction error dispersion define completely probabilistic description of the preset signals being recognized. Obtaining of such features amounts to finding the AR coefficients estimates (or GF reflection coefficients) by the observed realization ofthe signal Xl, l = 1,L. With p << L the application of these informative features makes it possible to shorten dimensionality of the signals’ description essentially. It should be noted that signals’ description with a small number ofthe AR coefficient is also convenient to use for storing large volumes of the signals’ realization samplings. If necessary the signals’ realizations can be easily restored with the relation (3).
defined from the condition of ensuring the preset probabilities ofthe preset random signals’ right recognition;pk, ak - are the order and parameters ofthe AR model for the k -th signal.
According to this rule the decision in favor of the i-th preset signal is made in two stages: when meeting at least one of inequalities (9), and when meeting the inequality system (10). When inequalities (11) are met the decision is made in favor of the signals from M + 1-th class.
In a number of cases to make a decision one can use the simpler decision rule based on the use of the AR model parameters as the recognition informative features for every preset signal. Having regard to the fact that the AR parameters estimate distribution canbe approximated with the Gauss law and on the assumption non-correlation and proximity of the dispersion of the AR parameters‘ estimates the following simplified decision rule can be derived [22]
i pk k 2 ----
H1: £(aj - mk)2 < Xk, k = 1,M (U)
j=1
pi - i 2 pk „ k 2 ---
Z(aj -mj)2 <£(aj -mp2, k = 1,M, i *k.(13) j=1 j=1
HM+1: £(aj - mk)2 > Xk, k = 1,M (14)
j=1
Here a j - are estimates of the AR model parameters found
with the sequence X observed on the stage of recognition;
k
m j - is the mathematical expectation of the AR parameters estimates obtained by the learning sampling of the k -th preset signal realization; Xk - are some threshold values defined from the condition of ensuring the preset probabilities of the preset signals’ right recognition.
It should be noted that the unknown signals in the cited recognition algorithms will be related to one ofthe preset signals, this corresponds to the faulty recognition. Special recognition methods shouldbe used forthe unknown signals class recognition for registration. The general solution of the preset signals’ recognition problem in the presence of the unknown signals’ class is considered in [19,20]. Recognition algorithms were obtained fromthe conditionofmaximizationofthe preset signals’ right recognition probability with a fixed volume of the preset signals proper regions built in the signals’ space or some their statistics. Letus render concrete the general recognitionalgorithm taking into account the considered expressions forthe probability density (5,6). In particular, the decision rule can be defined with the relations [21,22]
H1: Kk(x) < Ak, k = 1M, (9)
Kt® - Ki(x) + ln (2ngi>p' ^ > lni (1o)
(2no k)pk _L Pi V '
HM+1: Kk(x) > Ak, k = 1M, (11)
where expression for K'(x) is defined by the relation (8);
L
(2n)2 o J:-pk X k
A k = ln---------------- - are some threshold values
k Pk
4. Some results of studying recognition algorithms with the statistical simulation method
Taking into account the complexity ofthe analytical methods for studying the signals’ recognition algorithms, the estimates of signals’ recognition quality were obtained with the statistical simulation method. With this aim in view the considered signal recognition algorithms are realized in the form ofthe special program complex [21]. Statistical tests were performed with samplings of three Gauss random sequences (M = 3) realizations with different type of correlation functions. The signals’ realization samplings with L =1024 were obtained with the AR processes generator based on the relation (3) and realized with a computer software. The samplings with a volume of 400 realizations for every signal were formed. The samplings were divided into halves, 200 realizations for control and learning. The learning samplings were used for calculation of the decision rules informative features and parameters. By the control samplings the estimates of the mean probability of Pe signals’ faulty recognition were evaluated under different conditions of recognition.
In particular, the dependence Pe on the number of the AR coefficients used in the framework of the AR model chosen order was investigated. It was established that a sufficiently good quality of signals recognition is ensured with a small
R&I, 2003, N 3
189
number of the used AR coefficients (p=4, 5, 6). This demonstrates that the signals being recognized are well described with the small order AR model. Investigations of Pe dependence on the length of the recognized signals’ realizations L , which practically defines speed ofthe signals’ recognition, were performed. By way of example Fig. 1 shows dependence of Pe on L with different orders of the AR
M preset signals were presented; Pm/m+1 - is the probability of the faulty decision making in favor of M preset signals under condition that unknown signals from (M +1) -th class were presented. Fig.2 shows the diagram of recognition quality indices exchange in the form of dependence of the estimates Pm+i/m on Pm/m+1 with different recognized signals’ realizations length (L =256 and
model (p =2,3,4) obtained for the decisive rule (7). It is seen that with the increase in the recognized signals’ realization duration the quality of signals’ recognition improves essentially. The obtained dependence represent the signal recognition algorithm performance characteristics in the form of the diagram of recognition and speed quality indices.
Fig. 1. Dependence of the mean probability of Pe signals’ faulty recognition on the length of realization L with different order of AR of the model p
Peculiarities ofM preset random signals’ recognition problem solution were considered in the presence of the unknown signals’ class. Besides the preset signals the realizations of other signals with differing correlation functions representing (M + 1)-th class of unknown signals were presented for recognition. For this recognition problem the following signal recognition quality indices were estimated: Pm+1/m - is the probability of the faulty decision making in favor of the (M +1)-th class under condition that realizations of one of
PH/3
Fig. 2. Diagram of the preset signals recognition quality indices exchange in the presence of unknown signals
L =512). This dependence was obtained for the decision rule (9)-(11).
5. Conclusions
Methods for preset signals’ recognition in the presence of unknown signals were considered. The methods are based on the autoregression model signals’ description, this model gives sufficiently complete description of the real signals with a small order of the model. This makes it possible to use a small number of the signals’ informative features and build the signals’ recognition decision rules simple in terms of the realization expenditures. Some results of studying the quality indices of signals’ recognition with the statistical simulation method are given.
It should be noted that the considered autoregression model and the corresponding recognition algorithms take into account the difference in signals only on the correlation functions’ level. If it is necessary to take into account differences on the level of higher moments of signals’ distribution one should use the generalized autoregression model [24]. Recognition methods taking into account nonGauss nature of signals can be obtained on its basis. It is necessary to perform synthesis and analysis of the corresponding algorithms of signals’ recognition both for the classical statement of the preset signals’ recognition problem and for the case of the unknown signals’ class existence. Moreover, practical peculiarities ofrealization and rational fields ofapplication ofnon-Gauss signals recognition algorithms based on the generalized autoregression model should be studied.
References: 1. Fu K.S. Pattern recognition and machine learning / / Transl. from Engl. M.:Nauka, 1971. 156 p. [In Russian] 2. Milen’ky L. V. Classification of signals under conditions of a priori uncertainty.
M.: Sov. Radio, 1975. 328 p. [In Russian] 3. Duda R., Hart R. Pattern recognition // Transl. from Engl. M.: — Mir, 1976. 541 p. [In Russian] 4. Tou J.T., Gonzalez
R.C. Pattern recognition principles // Transl. from Engl. M.: Mir, 1978. 411 p. [In Russian] 5. Fukunaga K. Introduction to statistical pattern recognition // Transl. from Engl. M.: Nauka, 1978. 367 p. [In Russian]6. Box G.E., Jenkins G.M. Time series analysis, forecasting and control // Transl. from Engl. - M.: Mir, 1974. 406p. [In Russian]. 7. FridlanderB. Grid filters for adaptive data procession // Proc. ofIEEE. 1982, v.70, N°6. P.54-97. 8. MarchenkoB.G., Omelchenko V.A. Probabilistic models of random signals and fields in the applied statistical radio physics. K.: YMK BO, 1988. 176 p. [In Russian] 9. Applied theory of random processes and fields/ Monograph edited by K.K. Vasilieva, V.A. Omelchenko - Ulianovsk: UlSTU, 1995. 256 p. [In Russian]. 10. Tel ’ksnis L. Definition ofthe most probable variation of properties of multidimensional dynamic systems with P3/h unknown parameters //Statistical problems of control -—I Vilnius: IM Lit.A. Sci., 1977. N 24. P.9-26. [In Russian]
11. ShpilevskyE. Principles ofthe dynamic classification of stochastic processes and systems //Statistical
190
R&I, 2003, N 3
problems ofcontrol. Vilnius. IM Lit.A.Sci., 1978. N 28. 139 p. [In Russian] 12. Lipejko A.M. Classification ofautoregression sequences with jump-like varying parameters //Statistical problems of control. Vilnius: IM Lit.A.Sci., 1978. N 30. P.9-27. [In Russian] 13. Kravchenko N.I., Bezruk V.M., Tikhonov V.A. Recognition of random signals in the framework of autoregression model // Probabilistic models and random signals and fields procession. K: YMK BO, 1991. P. 138-142. [In Russian]. 14. Kravchenko N.I., Bezruk V.M., Tikhonov V.A. Structures of devices for recognition of Gauss signals with their description by the autoregression model / /Radio Electronics and Informatics. 2001. N 4. P.49-54. [In Russian] 15. Bezruk V.M., Tikhonov V.A., Tikhonov V. V. Recognition by the grid filter reflection coefficients // ACS and automation devices. 2001. N° P.36-39. [In Russian] 16. Senin A.G. Random signals recognition. - Novosibirsc: Nauka, 1974. 76p.[In Russian] 17. Libenson M.N. Non-linear statistical method of many classes recognition //Problems od random search. Riga: 1978. N 6. P. 299317. [In Russian] 18. Omelchenko V.A. Signals’ recognition by the power spectrum in the optimal Karunev -Loev basis // Izv. vuzov. Radioelectronics. 1980. N°12. P.11-17. [InRussian] 19. Omelchenko V.A. Foundations of spectral theory of signal recognition. Kharkov: Vyshcha shkola, 1983. 156p. [In Russian]. 20. Omelchenko V.A., Balabanov V.V., Bezruk V.M., Omelchenko A.V., Fefelov N.A.
Recognition of non-completely described random signals in the presence of unknown signals class //Information choice and procession. Kiev: A.Sci. of Ukraine, 1992. N 8. P.71-80. [In Russian]. 21. Bezruk V.M., Kovalenko N.P. Synthesis and analysis of Gauss random signals algorithms in the presence of unknown signals class on the basis autoregression model //ACS and automation devices. 2000. N111. P.115-120. [In Russian]. 22. Omelchenko V.A., Bezruk V.M., Kovalenko N.P. Recognition of preset radio signals based on autoregression model //Radiotekhnika: All-Ukr. Sci. Interdep Mag. 2001.N123. P.195-199. [In Russian]. 23. Bezruk V.M. Optimization ofautoregression algorithms signals recognition by the totality of quality indices //Information-control systems in railway transport. 2001. N° 2. P. 10-13. [In Russian]. 24. Tikhonov V.A. Generalized model of non-Gauss processes autoregrression / / Radiotekhnika: All-Ukr. Sci. Interdep.Mag.- 2003. N° 132. P.7882. [In Russian]
Bezruk Valery Mikhailovich, Candidate of Techn. Sci. Ass. Prof., Net Communications Department, Kharkov National University ofRadio Electronics. Scientific interests: simulation and multicriterial optimization of signals’ recognition systems. Address: Lenin Ave., 14, KNURE, Kharkov, 61166, Ukraine. Telephone for contacts: 7021426. E-mail: bezruk@kture.kharkov.ua.
OBJECTIVE-ORIENTED META-LANGUAGE OF SIMULATION IN DESIGNING
KUZEMIN A.Ya.
Professor, Cand. Techn. Sci.
Kharkov National University of Radio Electronics, Faculty of Applied Mathematics and Management, Information Science Department, Danilevsky Str.,15, App.8, Kharkov,61058, Ukraine kuzy@.foss.kharkov.ua,_kuzy@kture.kharkov.ua
Problems of designing operate with the system-complex objects. When considering these problems it is necessary first of all to introduce some terminological stipulations. An object is something opposing to a subject or is the aim of its activity in its subject-practical activity in the frameworks of designing. At the stage of identification to reach the aims of designing a subject (or “designer”) should as if “pass ” into the state of an object (“to foresee behavior of the system being designed”). In this case the object simulates the response activity of the source i.e. subject. In other words, the subject must become the object for rightness perception of the context understanding of its knowledge, abilities, experience in object-oriented presentation of the solution versions in designing. Without such an organization of the subject and object communication we can’t reach not only the preset global aims but structurally dependent sub-aims on designing as well. T o plan each next stage in designing one should be, as a minimum, sure that the object understands right the previous stage context. Such an intelligence of the interaction process of the subject-object couple results in self-organization. The chosen intelligence can result in constructing intelligent systems for designing.
The object as the aim of designing represents the system of the connected objects (subsystems) of a lower level. Investigations into the psychology field showed that the human brain percepts information in portions and due to the
hyper geometrical growth of the latter copes with it only aligning the objects in the logically linked hierarchy. Thus, the surrounding world is a hierarchic one for a modern man and a man in a general case conceives with patterns-objects and data and knowledge (properties, actions, concepts etc.) connected with them.
The main element of the used methodology for solving the preset problems is the object-oriented analysis [1,2], where the requirements to the designing system are perceived in terms of classes and objects chosen in the subject field. The analysis process is tightly connected with classification process:
classical categorization (the classified objects are subdivided into non-crossing sets by some feature inherent only to the given set);
conceptual clusterization (judgements are based on the best “agreement” between the criteria by the function of belonging to the odd set);
prototype theory (the object is classified by the “degree of similarity” with some standard).
The process of development of the designing system joins in itself the process of the objective decomposition and presentation methods of the logical, physical, static and dynamic models of the systembeing designed. It is a common practice to link such an approach with the concepts of the obj ect-oriented designing (OOD) [1,2]. Further we will proceed from the fact that classes and objects were separated in the course of the process of the system and object-oriented analysis. But accepting the problem of the optimal identification of all object-oriented concepts of the subject field (complete formalization) unsolvable to the full extent it is possible to introduce the identification quality criteria (for example, linkage, connectivity, sufficiency, completeness) of both the system being designed as a whole, and its parts [1,2,3]. In this case some standard for the set of objects with a general structure and common behavior should be understood as a class. We will consider that the object is characterized by the state, behavior and identity. The
R&I, 2003, Ns 3
191
