CC BY
12
UDC 612.8 + 57.089 Articles
doi:10.31799/1684-8853-2018-4-39-44
MATHEMATICAL MODEL OF PATTERN SELECTION FOR COMPLEX MULTICHANNEL DATA IN EEG PROCESSING
A. E. Runnovaa, PhD, Phys.-Math., Associate Professor, orcid.org/0000-0002-2102-164X, anefila@gmail.com aYuri Gagarin State Technical University of Saratov, 77, Politechnicheskaia St., 410054, Saratov, Russian Federation
Introduction: Research on real systems relies now on processing big experimental data volumes. Recognition of short oscillatory patterns corresponding to different states of complex non-stationary systems requires new processing methods. Purpose: Design of a mathematical model for objective and expertise-independent recognition of patterns corresponding to various states of real systems. Results: We propose a new method of modeling short oscillatory events (patterns) for complex non-stationary multichannel data. A mathematical realization of the model is described in terms of continuous wavelet transformation. Human brain activity states can be recognized automatically for the analysis of long EEG registrations. The proposed mathematical model application is demonstrated by the example of processing human EEG signals non-invasively recorded in the occipital scalp region. We demonstrate successful recognition of various human states based on the analysis of EEG from the visual analyzer area. We discuss the analysis of various patterns in experimental data corresponding to the state of active visual recognition of objects. Practical relevance: This modelling method can be recommended for neurophysiological data processing.
Keywords—Mathematical Modeling, Non-Stationary Data, Continuous Wavelet Transformation, Electroencephalography, Mathematical EEG Processing.
Citation: Runnova A. E. Mathematical Model of Pattern Selection for Complex Multichannel Data in EEG Processing. Informatsionno-upravliaiushchie sistemy [Information and Control Systems], 2018, no. 4, pp. 39-44. doi:10.31799/1684-8853-2018-4-39-44
Introduction
The modern development of natural sciences and the practical application of their results poses the task of investigating complex objects, described, in general, by a large number of recorded noisy signals coming through many channels. The study of such signals is of considerable interest, in particular, for neurophysiology, and some other scientific fields [1-6]. At the same time, special attention is drawn to the problems of selection of short time intervals ("patterns") from the structure of these signals. A large number of publications are devoted to the problems of selection and analysis of patterns [1-3, 5]. Today continuous wavelet analysis is one of the tools recognized for such processing. However, most modern techniques require expert evaluations at different stages of processing real signals, which makes this processing more difficult and expensive and it's capable of introducing subjective distortions in the analyze results.
Pattern Selection and Classification Model
Let the signal {X(i)} consist of n components: {X(i)}= = {x1(i), ..., xi(t), ..., xn(t)}. Each component xi(i) of the signal {X(i)} is the recording of a strongly non-stationary process of a real system in one of the n channels.
For mathematical modeling, we introduce the following assumption. Let each one-dimensional signal xi(t), registered in channel i of the multivar-iate signal {X(i)} in the time period [Ts, Ts+Ai], be a linear superposition written in the following form:
(f) a(yl (t) + № + (1)
7=0
where y! (t) — signal component having a closing to
the stationary frequency f ; for time registration t
. y
in a certain interval tj, tj, < At for Vj, a[ is the scale
factor of the level of the presence of the component y! (t) in the signal xi(t). Further, let us assume that the number of components y! (t) significant for the processing and analysis problems is finite and takes the value of np, and the remainder terms of the sum
^ a[ y[ (t) can be referred to regular disturbances
i=nP
£,y(t). Then expression (1) takes the form:
np
x, (f) = £ a!y! (t) + m + (f) + T!(f). (2)
7=0
Performing continuous wavelet transform (CWT) for each one-dimensional signal xi(i), we obtain:
Wt (f,t0) =£ a{wi (fyi ,t0) +
7=0
+(/, f0) + (/,f0) + % (/, f0). (3)
The CWT for an arbitrary signal z(t) in the general form is defined as follows [7]:
1 ......11 - tQ
Щ (s, t0 ) = -¡= z(t)xV *
Vs V s
di,
(4)
where ySjtQ(t) is the maternal wavelet, complex function; s is the time scale defining the width of the wavelet; the symbol "*" denotes complex conjugation. Note that the time scales s of the CWT allow a transition to the classical frequencies f of the Fourier spectrum, therefore, for convenience and simplicity of the interpretation of the results, we will consider the results in the traditional plane (f, 10).
In considering expression (3), it is easy to see
that each term of the form wj | fyi, t0 j is close to
the stationary value fyi of the skeleton sc of the
CWT at time intervals t[, and, in this case, expression (3) takes the form:
Wi (f,t0) =£ o[fyi + (f,t0) +
7=0
+w%„ (f,t0) + Wn (f,t0). (5)
Thus, for selection of the desired patterns necessary for the study of experimental signals, it is sufficient to introduce the skeleton characteristics of the CWP of the original signals in the manner described below. For the initial one-dimensional signal Xj(t) for each instant t0, we introduce the instantaneous spectral slice El. (f) of the CWT W/ (f,t0 ): ^
< (/) = K (f,t0 f . (6)
f e (Fmin; Fmax) Hz, for each instant t0, we define
the np extrema maxp
which are the local
Over the whole significant frequency range f, (< )•
maxima of the dependence El. (f) (6) which sc?,
to \ / I
where p = 1, 2, ..., np corresponds to the number of the extremum in the order of their decrease; i — as well as the registration channel number in the multivariate signal {X(t)}. Thus, the skeleton of the wavelet transform takes the form:
VfAeAfp, E\o (fp)>E\o (4):
>s.
(7)
For each channel register with a sampling frequency time exceeding 250 Hz, the multidimensional signal {X(t)} described above can be calculat-
ed origin minimum quantity np = 5 skeletons sc?, further calculation is not always possible, leading to the situation.
n„+1 n„+2 SCi P » SC;
ÍSC; » const.
In practice, for the signals {X(t)} of the experimental nature, it is often sufficient for an investigation to analyze np = 2...3 skeletons sc?.
So, based on the model (5) for each channel xi(t) of the multivariate signal {X(t)}, we calculate the discrete set of skeletons {sc}, where p = 1... np and i = 1...n. At each instant t we introduce the Heaviside function of the following form:
н[ m J $ .
[o, scf г (//,// )
(8)
Here, the frequencies f-f, can be chosen both on the basis of exclusively a priori representations of the desired pattern, i.e., the component y! (t) close to stationarity has the frequency fp and
ff2 = fp + A/, so and by means of automated search over the entire frequency range (Fmin; Fmax) Hz. Next, we take into account a certain stationarity of the frequencies f^ of the desired patterns in
the experimental signal xi(t), introducing the time
analysis of the function h[ (t) (8):
H?\At =¡AtHf (í)dí
(9)
where the selection of the parameter At is carried out by means of a sliding window with respect to the time duration of the recorded signal Xj(t) with the following condition:
—Hf > 0.9.
At 1
(10)
For a multidimensional signal {X(t)}, one can proceed from an analysis of the multidimensional
function Hp\At ={h(\At, ..., h[ |Ai, ...,HP\At}
to a one-dimensional resulting function of the following form:
HP I Ai
1 n
=1Z H[
n • n
i=0
Ai'
(11)
where n is the dimension of the original multivariate signal {X(t)} (n — the number of registered channels). This integral function HP At is capable of taking a maximum value as 1 and a minimum value 0.
We analyze the dynamics of function Hsc(t) for the objective separation of experimental signal pat-
terns. Further classification of patterns is based on the analysis of frequencies fp (8), time intervals At (9) and their possible dynamics over time of the total recording time T of the signal {X(i)}. The mathematical modeling carried out is designed to fill the lack of automated techniques that would be capable of identifying and classifying patterns without involving experts in processing multicomponent multidimensional experimental {X(i)} signals.
Practical Relevance
We illustrate the practical using of the developed model on the neurophysiological data — the human electroencephalogram. The electroencephalogram (EEG), which is the sum of electrical currents that are generated by a small group of neurons and recorded using an electrode, is one of the most widespread brain signals that are used in research [8-11]. Since the neural network of the brain is a very complicated oscillatory system, the EEG signal also has a very complex spectral structure with a few frequency ranges (delta, alpha, beta, gamma, etc.), different characteristic rhythms, and oscillatory patterns that attract interest of researchers both during the study of different pathologies (e. g., epilepsy) and during analysis of different functional tests and cognitive processes [3, 6, 9-11].
Experimental Data
In our physiological experiment with EEG activity registration we used a set of images based on a well-known bistable object, the Necker cube [12] as a visual stimulus. Ambiguous (bistable) stimuli are a very useful tool for studying the decision-making process [13-16]. This is a cube Necker with transparent faces and visible ribs; an observer without any perception abnormalities treats the Necker cube as a 3D-object thanks to the specific position of the cube ribs. Bistability in perception consists in the interpretation of this 3D-object as to be oriented in two different ways, in particular, if the different ribs of the Necker cube are drawn with different intensity. In our experimental works we have used the Necker cube images with varying parameter I to be the brightness of the cube wires converging in the right upper inner corner (Fig. 1, a). The brightness of the wires converging in the left lower inner corner is defined as (1 - I).
The experimental studies were performed in accordance with the ethical standards of the World Medical Association [17]. Six healthy subjects from a group of unpaid male volunteers, between the ages of 20 and 25 with a normal visual acuity participated in the experiments. The purpose of this
a)
Right
Necker cube
Left
Necker cube
■v.
b) P4 Pz P3 POz O2 Oz O1
" 'NwA^iiaiiii.^fti...........
Left i s
Necker cube
Right *> s
Necker cube
■ Fig. 1. Examples of Necker cube images (a) and typical segments of the EEG recording from O1, Oz, O2, POz P3, Pz, P4 channels (b). Vertical lines show the time moments of presentation of various Necker cubes images to the volunteer
experiment is the study of multichannel EEG data registration in the unconscious decision on ambiguous image interpretation. We demonstrated the Necker cube images with different wireframe contrasts for a short time, each lasting between 1.0 and 1.5 s, interrupted by a background abstract picture for 5.0-5.5 s. The using of the background abstract images allows the neutralization of possible negative secondary effect of the previous Necker cube image. The whole experiment lasted about 40 min for each patient. During the experiment we exhibited the pictures of the Necker cube randomly, all for about 150 times, and recorded brain activity with multi-channel EEG. As a tool for EEG recording we used the electroencephalograph- recorder Encephalan-EEGR-19/26 (Russia) with multiple EEG channels and the two-button input device. To study EEGs the monopolar registration method and the classical ten-twenty electrode system were used [18].
Figure 1, b shows an example of a typical EEG data set from EEG registration channels of occipital area, corresponded the visual analyzer area. It seems occipital region associated with cognitive processes of perception of complex spatial objects, which include the Necker cube.
Processing of Experimental Data
In our experimental studies we recorded EEG data during the sufficiently regular ambiguous perception of complex objects. Between the two moments of the perception of ambiguous object associated with a spatial imagination Necker cube, a pause is enough
to restore the background activity is not associated with the cognitive process of spatial modeling. At the moment, we are not interested in causes, for example, the phenomenon of cognitive noise, forcing Necker cube with intensity I perceived as the "left" and "right" [11, 19-21]. However, we directly focused on the processes that occur in the perception and the internal filling volume of a two-dimensional object Necker ambiguous image.
We consider the main characteristic of the frequency range of [Fmin; Fmax]: [Fmin; Fmax]<x = [8-12] Hz
for alpha-rhythms, [Fmin; Fmax]p = [30-40] Hz for betta-rhythms, [Fmin; Fmax]A = [4-7] Hz for delta/ theta-rhythms [22].
Next, for each presentation of the cube, we calculate the function (11) for alpha-, beta- and delta/ theta-activity on the EEG data in time intervals corresponding to different stages of the visual stimulus perception. We distinguish three stages in this experimental data: (i) the state before the presentation of the cube, the passive waiting, (ii) the state when the cube is presented, the active recognition, (iii) the rest state after the recognition, passive state. To determine the time duration of these stages we used the following design of preliminary experiments. After each presentation of the cube, the person had to tell the character of his recognition of this bistable object (the left or right cube of Necker) by pressing the remote control. Preliminary studies were conducted on 15 volunteers and showed that the duration of the active stage is about 0.6 s. Function Hp | At (11) was calculated for a given time interval At = 0.6.
Results
The result of the selection three types of EEG patterns shown in Fig. 2 for different states of brain visual analyzer. For the Fig. 2 first and last states "before/after perception of ambiguous image (Necker cube)" are characterized by the predominance of low frequencies (alpha- and delta-rhythms). To obtain this statistic, the calculated function Hp |Ai (11) was averaged over all experimental events (about 150).
Active state characterized a pronounced high-frequency activity is observed. It is clear that visual analysis is not enough, and we propose a method for an estimation of level and nature of beta-activity for occipital region in the human brain. For this, well traced throughout the duration of the processes associated with the recognition of ambiguous images for the majority of volunteers. Note that one volunteer with displacement of the pattern frequency to 25-32 Hz was found in the processing data. However, it is clear that this fact is easily detected and can be corrected adaptation algorithm. Now we are limiting our processing of multi-channel EEG
<a7>
<P7>
<a7>
<a7>
?7> <S7>
<S7>
Before perception
Perception
After perception
■ Fig. 2. Averaged criteria <87>, <a7>, <P7> during different phases of bistable stimulus perception
data exceptionally occipital area of a brain electrical activity registration, in particular O1, Oz, O2, POz, P3, Pz, P4 channels of the classical ten-twenty electrode system [12, 23-25].
According to modern concepts of betta and alpha, EEG activity in the occipital region of the scalp is related to the processes of human visual analyzers [24-27]. With alpha activity, the processes of human relaxation are traditionally associated, occurring either with closed eyes, or in a calm environment without pronounced external stimuli. Betta-processes are associated, most likely, with the activity of the visual analyzer when recognizing complex objects and/or with processes of focusing on visual objects.
In Fig. 2, the recognition of the stage of human recognition of a visual bistable object (stage "perception"), associated with a fall in the level of alpha activity and an increase in betta. At the same time, the delta activity sharply disappears in the active stage and remains at a single level for the passive stages. However, the stage after recognition shows a somewhat higher level than in the process of waiting for the next visual stimulus.
All the results obtained, as shown, lie in the mainstream of modern science, allowing demonstration of the activation of visual analyzer in the process of periodic recognition of a complex visual object. At the same time, it was possible to reveal a nontrivial effect of increasing the delta/theta-ac-tivity at the end of stimulus recognition, than with its expectation.
Conclusion
We propose a new method of short episodes of activity (patterns) modeling for complex nonstation-ary multichannel data. A mathematical realization of the model is described in the terms of a continu-
ous wavelet transform. The application of the proposed mathematical model is demonstrated by the example of the processing of human EEG signals non-invasive recorded in the scalp occipital region.
We demonstrate the success of recognition of various human states based on analysis of the EEG from visual analyzer area. The results of these studies appear promising for further study of the dynamics and the activity of the cerebral cortex in
1. Maksimenko V. A., Luttjohann Annika, Makarov V. V., Goremyko M. V., Koronovskii A. A., Nedaivozov V., Runnova A. E., van Luijtelaar Gilles, Hramov A. E., Boccaletti S. Macroscopic and Microscopic Spectral Properties of Brain Networks During Local and Global Synchronization. Physical Review E, 2017, vol. 96, 012316. doi:10.1103/PhysRevE.96.012316
2. Hramov A. E., Koronovskii A. A., Ponomarenko V. I., and Prokhorov M. D. Detecting Synchronization of Self-sustained Oscillators by External Driving with Varying Frequency. Phys. Rev. E, 2006, vol. 73, iss. 2, 026208.
3. Sitnikova E., Hramov A. E., Grubov V., and Kor-onovsky A. A. Time-frequency Characteristics and Dynamics of Sleep Spindles in WAG/Rij Rats with Absence Epilepsy. Brain Research, 2014, vol. 1543, pp.290-299.
4. Kobayashi K., James C. J., Nakahori T., Akiyama T., Gotman J. Isolation of Epileptiform Discharges from Unaveraged EEG by Independent Component Analysis. Journal of Clinical Neurophysiology, 1999, no. 10 (110), pp. 1755-1763.
5. Hramov A. E., Kharchenko A. A., Makarov V. V., Khramova M. V., Koronovskii A. A., Pavlov A. N., and Dana S. K. Analysis of the Characteristics of the Synchronous Clusters in the Adaptive Kuramoto Network and Neural Network of the Epileptic Brain. Proc. SPIE, 2016, vol. 9917, 991725.
6. Gotman J., Skuce D. R., Thompson C. J., Gloor P., Ives J. R., Ray W. F. Clinical Applications of Spectral Analysis and Extraction of Features from Electroencephalograms with Slow Waves in Adult Patients. Electroencephalography and Clinical Neurophysiolo-gy, 1973, no. 35, pp. 225-235.
7. Hramov A. E., Koronovskii A. A., Makarov V. A., Pavlov A. N., and Sitnikova E. Wavelets in Neuroscience. Springer Series in Synergetics. Springer-Verlag Berlin Heidelberg, 2015. 318 p.
8. Nunez P. L. and Srinivasan R. Electric Fields of the Brain: the Neurophysics of EEG. Oxford Univ. Press, 2006.
9. Ebersole J. S., Husain A. M., and Nordli D. R. Current Practice of Clinical Electroencephalography. Wolters Kluwer, 2014. 661 p.
cognitive processes of various kinds. The technique is based on the calculation of the wavelet skeleton, it is universal for the study of various processes. Furthermore, this approach is highly customizable to individual features volunteers that allows the theoretical possibility of that using in the biofeedback systems.
This work has been supported by Russian Science Foundation (grant No. 16-12-10100).
10. Luders H. and Noachtar S. Atlas and Classification of Electroencephalography. W. B. Saunders, Philadelphia, 2000.
11. Blake R., Logothetis N. K. Visual Competition. Nature Reviews Neuroscience, 2002, no. 3(1), pp. 13-21.
12. Necker L. A. Observations on Some Remarkable Phenomena Seen in Switzerland; and an Optical Phenomenon which Occurs on Viewing of a Crystal or Geometrical Solid. Philos. Mag., 1832, no. 3, pp. 329343.
13. Schwartz J.-L., Grimault N., Hupe J.-M., Moore B. C. J., Pressnitzer D. Multistability in Perception: Binding Sensory Modalities, an Overview. Philos. Trans. R Soc. B, 2012, vol. 367, pp. 896-905.
14. Cao R., Braun J., Mattia M. Stochastic Accumulation by Cortical Columns May Explain the Scalar Property of Multistable Perception. Phys. Rev. Lett., 2014, vol. 113, 098103-1-5.
15. Leopold D. A., Logothetis N. K. Multistable Phenomena: Changing Views in Perception. Trends Cognit. Sci., 1999, no. 3(7), pp. 254-264.
16. Zeki S. The Neurology of Ambiguity. Conscious. Cogn., 2004, vol. 13(1), pp. 173-196.
17. World Medical Association (2000) Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects. The Journal of the American Medical Association, 2000, vol. 284, no. 23, pp. 3043-3045.
18. Jasper H. H. The Ten-twenty Electrode System of the International Federation. Electroencephalog-raphy and Clinical Neurophysiology, 1958, no. 10, pp. 371-375.
19. Sterzer P., Kleinschmidt A., Rees G. The Neural bases of Multistable Perception. Trends in Cognitive Sciences, 2009, no. 13, pp. 310-318.
20. Maksimenko V. A., Runnova A. E., Zhuravlev M. O., Makarov V. V., Neday vozov V. O., Grubov V. V., Pche-lintceva S. V., Hramov A. E., Pisarchik A. N. Visual Perception Affected by Motivation and Alertness Controlled by a Noninvasive Brain-Computer Interface. PLOS ONE, 2017, vol. 12, 12 Article 0188700.
21. Hramov A. E., Maksimenko V. A., Pchelintseva S. V., Runnova A. E., Grubov V. V., Musatov V. Yu., Zhuravlev M. O., Koronovskii A. A., Pisarchik A. N. Classifying the Perceptual Interpretations of a Bistable Image using EEG and Artificial Neural Networks.
Frontiers in Neuroscience, 2017, vol. 11, no. 674, pp. 1-18.
Principles of Perceptual Organization. Curr. Biol., 2006, vol. 16, pp. 1351-1357.
22. Klimesch W. Alpha-band Oscillations, Attention, and Controlled Access to Stored Information. Trends
25. Ikkai A., Dandekar S., Curtis C. E. Lateralization in Alpha-band Oscillations Predicts the Locus and Spatial Distribution of Attention. PLOS ONE, 2016, vol. 11(5), p. e0154796.
Cogn Sci, 2012, vol. 16(12), pp. 606-617.
23. Sacchet M. D., et al. Attention Drives Synchronization of Alpha and Beta Rhythms between Right Inferior Frontal and Primary Sensory Neocortex. J Neu-rosci, 2015, vol. 35(5), pp. 2074-20 82.
26. Gola M., et al. EEG Beta Band Activity is Related to Attention and Attentional Deficits in the Visual Performance of Elderly Subjects. International Journal of Psychophysiology, 2013, vol. 89(3), pp. 334-341.
24. Pressnitzer D., Hupe' J. M. Temporal Dynamics of Auditory and Visual Bistability Reveal Common
УДК 612.8 + 57.089 doi:10.31799/1684-8853-2018-4-39-44
Математическая модель выделения паттернов сложных многоканальных сигналов в применении к обработке электроэнцефалографических данных
Руннова А. Е.а, канд. физ.-мат. наук, доцент, orcid.org/0000-0002-2102-164X, anefila@gmail.com Саратовский государственный технический университет им. Гагарина Ю. А., Политехническая ул., 77, Саратов, 410054, РФ
Введение: в настоящее время исследование реальных систем опирается на обработку больших объемов экспериментальных данных. Распознавание коротких колебательных паттернов, соответствующих различным состояниям сложных нестационарных систем, требует новых методов обработки. Цель: разработка математической модели для объективного и независимого от экспертных оценок распознавания паттернов, соответствующих различным состояниям реальных систем. Результаты: предложен новый метод моделирования коротких колебательных событий (паттернов) для сложных нестационарных многоканальных данных. Описано применение модели на основе подхода непрерывного вейвлет-преобразования. В автоматическом режиме может осуществляться поиск искомых состояний активности мозга человека для анализа длительных ЭЭГ-регистраций. Применение представленной математической модели демонстрируется на примере обработки человеческих ЭЭГ-сигналов, регистрируемых неинвазивным методом в затылочной зоне скальпа. Показано успешное распознавание различных состояний человека, основанное на анализе электроэнцефалографических данных проекции зрительного анализатора. Описан анализ различных паттернов в экспериментальных данных, соответствующих активному состоянию зрительного распознавания объектов. Практическое применение: использование описанной математической модели может быть рекомендовано для обработки нейрофизиологических данных.
Ключевые слова — математическое моделирование, нестационарные сигналы, непрерывное вейвлет-преобразование, электроэнцефалография, математическая обработка ЭЭГ.
Цитирование: Runnova А. Е. Mathematical Model of Pattern Selection for Complex Multichannel Data in EEG Processing // Информационно-управляющие системы. 2018. № 4. С. 39-44. doi:10.31799/1684-8853-2018-4-39-44
Citation: Runnova А. Е. Mathematical Model of Pattern Selection for Complex Multichannel Data in EEG Processing. Informatsionno-upravliaiushchie sistemy [Information and Control Systems], 2018, no. 4, pp. 39-44. doi:10.31799/1684-8853-2018-4-39-44
