Научная статья на тему 'Development of a neurodevice with a biological feedback for compensating for lost motor functions'

Development of a neurodevice with a biological feedback for compensating for lost motor functions Текст научной статьи по специальности «Медицина и здравоохранение»

CC BY
69
10
Поделиться
Ключевые слова
neurodevice / exoskeleton / brain-computer interface / electroencephalogram / electromyogram / electrooculogram / biological feedback

Аннотация научной статьи по медицине и здравоохранению, автор научной работы — Evgeny Bogdanov, Petrov V.A., Botman S.A., Sapunov V.V., Stupin V.A., Silina E.V., Sinelnikova T.G., Patrushev M.V., Shusharina N.N.

Concurrent use of electrophysiological signals of various types, such as obtained from electroencephalogram (EEG), electromyogram (EMG), electrooculogram (EOG), and others, increases the effectiveness of systems for external device control, namely, neural prostheses, exoskeletons, robotic wheelchairs and teleoperated robots. This article presents the results of the first tests of a multifunctional neurodevice capable of detecting EEG, EMG and EOG signals simultaneously (with EOG signals, photoplethysmogram, SpO2 and temperature modules of the neurodevice were used). Measurement results were then compared to the data obtained from KARDi3 device (Medical Computer Systems, Russia) and Fluke 17b multimeter with a plug-in thermistor (Fluke Corporation, USA). The informative value and accuracy of both datasets were comparable. We also studied the effectiveness of EEG and EMG signal hybridization on the basis of the neurodevice of interest; it allowed for an increase of classification accuracy in all subjects by an average of 12.5 % up to the mean of 86.8 % (from 75 to 97 %).

Разработка нейроустройства с биологической обратной связью для восполнения утраченных двигательных функций

Одновременное использование электрофизиологических сигналов нескольких типов (данных электроэнцефалограммы (ЭЭГ), электромиограммы (ЭМГ), электроокулограммы (ЭОГ) и др.) обеспечивает более высокую эффективность систем управления внешними устройствами — нейропротезами, экзоскелетами, роботизированными инвалидными креслами и телеуправляемыми роботами. В статье представлены результаты первых испытаний многофункционального нейроустройства, способного распознавать одновременно ЭЭГ-, ЭМГи ЭОГ-сигналы (последние — с подключением модулей фотоплетизмограммы, SpO2 и температуры). Результаты измерений сигналов с помощью разработки сравнивали с данными прибора KARDi3 («Медицинские компьютерные системы», Россия) и мультиметра Fluke 17b с подключаемым термистором (Fluke Corporation, США). По информативности и точности данные были сопоставимы. Также исследовали эффективность гибридизации ЭЭГи ЭМГ-сигналов с помощью нейроустройства: она позволила увеличить точность классификации у всех испытуемых в среднем на 12,5 % — до среднего значения 86,8 % (от 75 до 97 %).

Похожие темы научных работ по медицине и здравоохранению , автор научной работы — Evgeny Bogdanov, Petrov V.A., Botman S.A., Sapunov V.V., Stupin V.A., Silina E.V., Sinelnikova T.G., Patrushev M.V., Shusharina N.N.,

Текст научной работы на тему «Development of a neurodevice with a biological feedback for compensating for lost motor functions»

development of a neurodevice with a biological feedback for compensating for lost motor functions

Bogdanov EA1E3, Petrov VA1, Botman SA1, Sapunov VV1, Stupin VA2, Silina EV3, Sinelnikova TG3, Patrushev MV1, Shusharina NN1

1 Institute of Chemistry and Biology,

Immanuel Kant Baltic Federal University, Kaliningrad, Russia

2 Department of Hospital Surgery no.1, Faculty of General Medicine, Pirogov Russian National Research Medical University, Moscow, Russia

3 Department of Human Pathology, Faculty of Postgraduate Professional Training of Physicians, The First Sechenov Moscow State Medical University, Moscow

Concurrent use of electrophysiological signals of various types, such as obtained from electroencephalogram (EEG), electromyogram (EMG), electrooculogram (EOG), and others, increases the effectiveness of systems for external device control, namely, neural prostheses, exoskeletons, robotic wheelchairs and teleoperated robots. This article presents the results of the first tests of a multifunctional neurodevice capable of detecting EEG, EMG and EOG signals simultaneously (with EOG signals, photoplethysmogram, SpO2 and temperature modules of the neurodevice were used). Measurement results were then compared to the data obtained from KARDi3 device (Medical Computer Systems, Russia) and Fluke 17b multimeter with a plug-in thermistor (Fluke Corporation, USA). The informative value and accuracy of both datasets were comparable. We also studied the effectiveness of EEG and EMG signal hybridization on the basis of the neurodevice of interest; it allowed for an increase of classification accuracy in all subjects by an average of 12.5 % up to the mean of 86.8 % (from 75 to 97 %).

Keywords: neurodevice, exoskeleton, brain-computer interface, electroencephalogram, electromyogram, electrooculogram, biological feedback

Funding: the work was supported by the Ministry of Education and Science of the Russian Federation (Grant Agreement no. RFMEFI57815X0140 dated October 27, 2015).

Acknowledgements: the authors thank Alexandr Romanov of the Rehabilitation Center of the Administrative Department of the President of the Russian Federation, Moscow; Raphael Oganov of State Research Center for Preventive Medicine, Moscow; Daniil Borchevkin, Alexey Belousov, Vladimir Savinov, Sergey Sokolov, and Alexey Medvedev for their scientific contribution and productive collaboration.

[><] Correspondence should be addressed: Evgeny Bogdanov

ul. A. Nevskogo, d. 14, Kaliningrad, Russia, 236041; eubogdanov@gmail.com

Received: 31.03.2016 Accepted: 07.04.2016

разработка нейроустройства с биологической обратной связью для восполнения утраченных двигательных функций

Е. А. Богданов1 В. А. Петров1, С. А. Ботман1, В. В. Сапунов1, В. А. Ступин2, Е. В. Силина3, Т. Г. Синельникова3, М. В. Патрушев1, Н. Н. Шушарина1

1 Химико-биологический институт,

Балтийский федеральный университет имени Иммануила Канта, Калининград

2 Кафедра госпитальной хирургии № 1, лечебный факультет,

Российский национальный исследовательский медицинский университет имени Н. И. Пирогова, Москва

3 Кафедра патологии человека, Институт профессионального образования,

Первый Московский государственный медицинский университет имени И. М. Сеченова, Москва

Одновременное использование электрофизиологических сигналов нескольких типов (данных электроэнцефалограммы (ЭЭГ), электромиограммы (ЭМГ), электроокулограммы (ЭОГ) и др.) обеспечивает более высокую эффективность систем управления внешними устройствами — нейропротезами, экзоскелетами, роботизированными инвалидными креслами и телеуправляемыми роботами. В статье представлены результаты первых испытаний многофункционального нейроустройства, способного распознавать одновременно ЭЭГ-, ЭМГ- и ЭОГ-сигналы (последние — с подключением модулей фотоплетизмограммы, SpO2 и температуры). Результаты измерений сигналов с помощью разработки сравнивали с данными прибора KARDi3 («Медицинские компьютерные системы», Россия) и мультиметра Fluke 17b с подключаемым термистором (Fluke Corporation, США). По информативности и точности данные были сопоставимы. Также исследовали эффективность гибридизации ЭЭГ- и ЭМГ-сигналов с помощью нейроустройства: она позволила увеличить точность классификации у всех испытуемых в среднем на 12,5 % — до среднего значения 86,8 % (от 75 до 97 %).

Ключевые слова: нейроустройство, экзоскелет, интерфейс мозг-компьютер, электроэнцефалограмма, электромио-грамма, электроокулограмма, биологическая обратная связь

Финансирование: работа выполнена при поддержке Министерства образования и науки Российской Федерации (Соглашение о предоставлении субсидии от 27.10.2015 № RFMEFI57815X0140).

Благодарности: авторы благодарят Александра Романова из Центра реабилитации Управления делами Президента РФ (Москва), Рафаэля Оганова из Государственного научно-исследовательского центра профилактической медицины (Москва), Даниила Борчевкина, Алексей Белоусова, Владимира Савинова, Сергея Соколова и Алексея Медведева за плодотворное научное сотрудничество и вклад в работу.

Для корреспонденции: Евгений Анатольевич Богданов

236041, г Калининград, ул. А. Невского, д. 14; eubogdanov@gmail.com

Статья получена: 31.03.2016 Статья принята к печати: 07.04.2016

Applied biorobotics improves the quality of life in patients with neurological disorders and traumas. Neuroprostheses, exoskeletons, robotic wheelchairs and telecontrol robots contribute to rehabilitation of patients, substitute for lost functions and enhance physical abilities of healthy people.

Choosing the right control scheme is very important for the development of such devices. It must ensure the accuracy, stability and safety of the device performance, given that the device will be used continuously. The majority of existing solutions are based on recording human body biopotentials using electromyography (EMG), electroencephalography (EEG), electrooculography (EOG) and electrocardiography (ECG) [1-6].

Robotic wheelchairs, prostheses and exoskeletons are good examples of the effectiveness of EMG-based schemes [7-9]. However, EMG alone is not enough if an individual who had a stroke or a spinal cord injury cannot generate a muscle signal of the required intensity. In such cases we turn to brain-computer interfaces (BCIs) that transform signals from damaged brain areas into commands for external devices. One of the recent works [10] has demonstrated a high effectiveness of a BCI for neuroprosthesis control tested by a tetraplegic patient with intact sensory and cognitive functions.

Among various methods of brain signals recording, EEG is the most convenient due to its availability, safety, cost-effectiveness and portability. The brain cortex consists of multiple areas of functional specialization in which waves of different frequency are observed [11]. The EEG spectrum is unique for every individual and changes constantly depending on a person's physiological condition and the activity performed, as long-term measurements have proved [12] By decoding EEG signals, we can discriminate between limb movements quite accurately. For example, the algorithm proposed for the reconstruction of the trajectory of finger joint angles during reach to grasp movements ensured 76 % accuracy of EEG signal [13]. Another work showed that it was possible to correctly identify one out of five actual or imaginary movements of the wrist and fingers with 65-71 % accuracy [14].

For better classification accuracy,a large number of EEG channels is thought to be necessary. However, Yang et al. [15] were able to remove irrelevant noise and improve the EEG signal classification technique that can be applied to a neural network or used for robotic device control. EEG was recorded with only 6 channels out of 32; still, the classification accuracy reached 86 % in some motor tasks. However, EEG-based BCIs have certain drawbacks resulting from incorrect electrode placement, shifting of electrodes, noises, artifacts, imperfect algorithms of filtration and signal processing.

Some researchers suggested that EMG and EEG methods should be fused [16-18]. For example, in case of paresis or limb loss, EEG signals can be used to compensate for weak EMG signals, ensuring that a prosthesis or exoskeleton is moved by mental effort. If EMG signals are of normal intensity, EEG signals can help reduce the impact of tremor, fatigue or artifacts.

Leeb et al. [19] proposed a hybrid EEG-EMG-based control system; it was tested on 6 healthy individuals. The subjects moved their left or right arm for 5 seconds (there were 60 trials in total). Brain activity was recorded by 16 sensors placed in accordance with the international 10-20 system. Muscular activity was recorded over left and right forearm flexors and extensors. The obtained EMG signals were rectified and averaged (0.3 s) to get the envelopes. The data from two classifiers were fused together to get one control signal. The hybrid system showed high classification accuracy in

all subjects. Despite the fact that EMG signals were quite informative (classification accuracy was 83 % in average), the hybrid approach was more effective (classification accuracy was 91 %), especially in case of increasing muscle fatigue.

Xie et al. also developed a hybrid EEG-EMG-based BCI (visualization of movement intention) [20]. Their study enrolled 10 post stroke patients with non-severe hemiparesis, 10 patients with peripheral nerve injury and 10 healthy individuals. All patients were between 20 and 58 years of age. For calibration, subjects were asked to lie on the bed and perform knee flexion and extension tasks. The sensor measured the angle and the force of movements; the obtained data were later used as target levels. Then EEG/EMG sensors were attached, and the experiment was carried out. The aim of the experiment was to establish the correlation between EEG/EMG signals and leg movements and to measure the accuracy of potential control commands for the external device. First, EEG data were processed followed by EMG data processing; in the third experiment the hybrid approach was applied. The results showed that the hybrid approach led to increased classification accuracy in all groups of subjects, compared to single modality approach. In healthy individuals, classification accuracy was 98 %, in post stroke patients — 84 %, for patients with peripheral nerve injury — 85 %.

Kiguchi et al. [21] carried out a study of a hybrid EEG-EMG system for controlling arm movements using SUEFUL-7 robotic device, and assessed its effectiveness [22]. The robot was equipped with a video camera and could detect arm position by rotational angle and force sensors. A 16-channel EMG interface for recording signals coming from arms and shoulders was used as a control system. The experiment enrolled four 23-year-old healthy individuals. Some of them wore an exoskeleton and a device for EEG recording and response monitoring. In the first experiment, the subjects performed arm flexion/extension tasks, and the robot did the opposite impeding the movement. In the second experiment, one full and two empty cups were put on the table. When the subject grasped the empty cup, the robot used the assistance algorithm that estimated the position of empty cups using the video camera, and randomly selected one of them; after that, the robot assisted the subject in pouring the liquid. The accuracy of choice was assessed using EEG and EMG signals. If the subject did not resist, the robot inferred that the target had been chosen correctly. In that experiment, the flexibility of the robot and its ability to correctly interpret the intentions of the subject were tested. The results showed the increased accuracy of interpretation of human actions by the robot.

BCI performance can be improved by oculography data recorded parallel to EEG. A group of scientists designed a hybrid EEG-EOG-based BCI to enhance the reliability of hand exoskeleton for continuous grasping movements [23]. EEG signals were recorded at 5 EEG sites in accordance with the international 10-20 system. The experiment consisted of two parts. In the first part, the subjects controlled the exoskeleton through EEG signals only; they made grasping movements when the visual indicator appeared (green for the movement onset, red for rest). The robotic hand opened automatically if the commands issued by the operator's brain were not intense enough to get over a preset threshold. In the second part of the experiment, EOG signals were used as a switch. When the subject looked to the left or to the right, the exoskeleton hand opened regardless of EEG signals. The hybrid model increased system safety. When only EEG signals were used, the motion of the robotic hand exceeded 25 % of a full hand closing in half of subjects at rest. The hybrid system showed the increase in

the threshold value in 10.4 % of subjects, with maximal grasp being less than 28 % of a full hand closing (in a single modality system it was 60 %) .

Cardiovascular system performance is usually assessed by monitoring arterial blood pressure and heart rate (HR); it correlates to brain activity, including that, during motor tasks [24-26]. Studies of the effect of changing mental activity on heart activity assessed by EEG show that hybrid EEG-ECG systems are a promising practical tool [27-29]. In the experiment involving 6 healthy right-handed men (mean age was 28 years), who imagined movements of their left leg or left arm, researchers assessed classification accuracy of EEG signals and ECG signals separately; then a fused EEG-ECG recording was processed [29]. For every subject, 180 sessions were held (60 sessions for each assessment method). They consisted of three parts; the subject rested for the first 6 seconds (while, data from previous sessions were processed); then the subject was presented with an indicator that randomly indicated the action that the subject had to perform (arm or leg movement visualization or rest); that part of the experiment lasted for 6 s; finally, there was a pause of unfixed length (up to several seconds). Three EEG channels were recorded (C3, C4, Cz, according to the international 10-20 system) along with ECG, R-R intervals were calculated as a difference between QRS complexes, which show heart rate, filtered at 5-10 Hz frequencies. The obtained data allowed for a few interesting conclusions. First, active visualization of limb movements induced heart rate change. Second, ECG classification accuracy was very high in almost all subjects: in many subjects the use of ECG modality was more effective than EEG. Third, the hybrid approach increased classification accuracy in almost all subjects, especially in those, whose results in a single modality mode were low.

Thus, a hybrid approach to the implementation of systems for external device control is very promising. Considering how fast these technologies are developing, we believe that such high-accuracy neurodevices will appear in the market in the nearest future. The laboratory of Neurobiology and Medical Physics of the Institute of Chemistry and Biology of Immanuel Kant Baltic Federal University is working on a multifunctional neurodevice capable of detecting different electrophysiological signals simultaneously (EEG, EMG, EOG supported by the use of photoplethysmogram, SpO2 and temperature modules), ensuring a biological feedback and transmitting the processed data to exoskeletons and robotic devices in real time. This article presents the results of the first tests of the prototype model of such a neurodevice and assesses the possibility of fused EEG and EMG signal recording based on it.

METHODS

We have implemented a prototype model of electrophysiological and biometrical recorder capable of converting biosignals into commands for an electromechanical device; we have also tested our model in a two-stage experiment. At the first stage, the neurodevice was used to study the motor activity of the subjects by recording electrophysiological signals. For the unbiased assessment of the device performance, the resulting data were compared to the data obtained with analytical devices that had proved to be reliable and are now successfully applied in medical practice. At that stage, 2 healthy men participated in the experiment, (22 and 23 years of age, height of 175 and 177 cm, respectively, weight of 70 and 75 kg). At the second stage, a possibility of fused EEG and EMG recording using the neurodevice was assessed. The experiment enrolled 10 healthy

right-handed men aged 22-29 years (mean age was 25 years).

Brain electrical activity was measured by encephalogram via scalp leads; bioelectrical potentials in skeletal muscles were measured by electromyography; bioelectrical potentials related to eye ball movements were measured by oculography; body temperature was measured by thermometry; pulse rate was measured using photoplethysmography. The obtained data were recorded digitally and graphically.

For EEG recording, silver cup electrodes (Ag/AgCl) were used; EEG caps were used to place the electrodes on subjects' heads. For EMG and EOG recording, silver plate electrodes were used. In the experiments aimed at the assessment of physiological signal parameters, the most common artifacts were detected, such as artifacts resulting from bad electrode attachment or electrical noise caused by subject's movements, artifacts caused by upper body muscle tension and forehead wrinkling, muscle potentials, skin potentials, eye blinking, pulse waves.

To study motor activity, the subjects were asked to do physical exercises, including bending and turning the head to the right and left, tilting it down and back, with the prototype model of the neurodevice attached to it. Before the experiment, we had written a program for real-time visual representation of Euler angles rotation.

Results of EEG, EMG and EOG signal recording and pulse rate data were compared to those obtained with KARDi3 device (Medical Computer Systems, Russia), intended for recording and analyzing ECG, EOG, EEG and some other parameters. Measurements were first done with KARDi3, then with the neurodevice of interest. The subject remained in the same position throughout the experiment. The electrodes attached to the body were not moved when switching from KARDi3 to the neurodevice prototype model. To reduce the amount of artifacts, electrode cables were bundled and twisted.

During EEG recording, we focused on alpha-rhythm, which is normally the most stable electrophysiological signal. To record the alpha rhythm, a bipolar lead system was used. Electrodes were attached to the back of the subject's neck, reference electrodes were attached to ear lobes. To achieve the maximal relaxation of neck and head muscles and to reduce myographic artifacts, the subject was seated in the reclined position. In total, 100 trials were conducted. For both the neurodevice prototype model and KARDi3, the same recording mode was used, with a 30 Hz low-pass filter, a 0.5 Hz highpass filter, a 50 Hz band-reject filter, speed of 30 mm/s (X-axis), sensitivity of 50 mcV/mm (Y-axis).

EOG signals were recorded during the eye movement task. The total number of trials was 100. The subjects were asked to do the following exercises: look at the yellow dot in the center of the board - then up (red circle) - center (yellow dot) - down (blue circle) - center (yellow dot) - left (red cross) - center (yellow dot) - right (blue cross) - center (yellow dot). The subject was seated in front of the board with graphic symbols. To register the signal, a bipolar montage scheme was used. Electrodes were attached to the temples, close to the right eye and on the forehead. For both the neurodevice prototype model and KARDi3, the same recording mode was used, with a 40 Hz low-pass filter, a 1 Hz high-pass filter, a 50 Hz band-reject filter, speed of 15 mm/s (X-axis), sensitivity of 50 mcV/mm (Y-axis).

To record EMG signals during thigh muscles contraction, the subject was asked to move the right leg forward for a steplike movement. There were 100 trials in total. The left leg did not move, the subject did not lean on the right leg on which electrodes were placed. The subject was standing, using his left leg and right arm as points of support; his right leg

with electrodes on it was relaxed. Electrodes were placed 5 cm apart from each other. To record the signal, a referential montage scheme was used. Electrodes were placed over the femoral muscle using adhesive rings. For both the neurodevice prototype model and KARDi3, the same recording mode was used, with a 100 Hz low-pass filter, a 1 Hz high-pass filter, a 50 Hz band-reject filter, speed of 120 mm/s (X-axis), sensitivity of 10 mcV/mm (Y-axis).

To record EEG, EMG and EOG signals using our neurodevice prototype, neurodevice, we have developed original software. To record EEG, EMG and EOG by KARDi3, Neurocortex software by Neurobotics, Russia was used.

To measure the pulse rate the subject was seated. For ECG recording (100 trials in total), KARDi3 electrodes were attached to the wrists by adhesive rings. A referential montage scheme was used. Then, the subject put his finger on the photoplethysmogram module of the neurodevice, the output being the pulse-related signal. Pulse measurement was supported by pulse oximetry (SpO2). The following recording mode was chosen for both the neurodevice and KARDi3: a 0.1 Hz low-pass filter, a 50 Hz high-pass filter, a 50 Hz band-reject filter, speed of 60 mm/s (X-axis), sensitivity of 20 mcV/mm (Y-axis). To process data obtained with KARDi3, Neocortex software was used; to process data obtained with the neurodevice, Heart Rate Monitor Demo software was used (Silicon Labs, USA).

To compare temperature measurement accuracy, Fluke 17b multimeter with a plug-in thermistor was used. The temperature sensor was attached to the subject's forehead by the adhesive ring. The total number of trials was 136. Temperature data were transmitted to the PC via Blootooth protocol every second.

At the second stage of the experiment, a possibility of EEG and EMG fused recording by the neurodevice of interest was studied to ensure its good performance in a complex with robotic devices, such as an exoskeleton.

The subjects were instructed to imagine their left leg movements and then to flex and extend the thigh (10 sessions for every participant). Classification accuracy was first assessed for EMG signals only and then for a hybrid EMG-EEG system.

Physiological parameters were continuously monitored during motor tasks and idle periods (5 s long). Fisher linear discriminant analysis was used for classification.

STUDY RESULTS

We studied motor activity involved in performing such tasks as turning and bending the head to the right or left, tilting it down and back. The results demonstrate high accuracy and precision of the data obtained with the motor sensor of the studied neurodevice. The diagram in fig. 1 shows Euler angles rotation (X-axis represents time, Y-axis represents angle): 1) pitch is rotation around the transverse axis (green line); 2) roll is rotation around the longitudinal axis (blue line); 3) yaw is rotation around the vertical axis (red line).

EEG data obtained with the studied neurodevice showed the same artifacts as EEG data obtained with KARDi3, and their amounts were comparable. It indicates that the studied neurodevice could compete with similar tools for EEG recording. Muscular activity artifacts were associated with small neck and head movements resulting from subject's fatigue. Quite a few encephalograms showed traces of cardiogram artifacts, which is possibly related to the individual specifics of the subject's cardiovascular system and the placement of electrodes over subcutaneous blood arteries.

EOG data obtained with the neurodevice were comparable to the data obtained with KARDi3 in their informative value; in case of our neurodevice, the amount of artifacts was lower. The most common artifact was eye blinking, which appeared on EOG as a sharp amplitude increase; artifacts of mimic muscles that accompanied the subject's growing fatigue were also present.

During EMG signal quality assessment, it was found that the data obtained with our neurodevuce were as informative as the data obtained with KARDi3. No artifacts were detected.

The results of temperature measurements obtained with the neurodevice were comparable to the data from Fluke 17b reference device. In average, temperature variance was 0.3%.

Pulse signal was obtained from the electrocardiogram recorded with KARDi3 and the neurodevice photoplethysmogram module. ECG R-R interval data from KARDi3 showed the same pulse values as data from the photoplethysmogram module. The mean HR in the first subject was 78 and 77 beats per minute (measured with KARDi3 and the neurodevice, respectively). The mean HR in the second subject was 72 and 71 beats per minute (measured with KARDi3 and the neurodevice, respectively). No artifacts that could affect the result were observed (fig. 2).

It is worth mentioning that the module for the assessment of cardiovascular system performance estimates blood oxygen saturation, thus providing some valuable data that can be used for exoskeleton control.

Our study of EEG and MG signal hybridization yielded results that support the idea electrophysiological signal fusion approach. The experiment showed that mean classification accuracy of EMG signals was 74.3 %. EMG-EEG hybridization led to the increased classification accuracy by an average of 12.5 % with a mean of 86.8 % (75-97 %) in all subjects. The results are presented in the table below.

DISCUSSION

It is obvious that development of a high-accuracy multifunctional neurodevice that allows for continuous recording of physiological signals and transmits data to the external device (exoskeleton) can yield very inspiring results. We have carried out a truly multidisciplinary study, at the first stage of which a prototype model of such a neurodevice was created and tested. It was demonstrated that the signals obtained with our device were identical to those obtained with reliable analytical tools.

During some motor activity measurement procedures, gyroscope drift was observed associated with a changing magnetic field generated by the accumulator battery. It was

A B

Fig. 1. Euler angles rotation during head exercises. (A) Head Is bent to the left. (B) Head is back to the initial position (the yaw and roll angles change, the pitch angle remains unchanged, gyroscope returns to the initial position)

the result of the relatively weak attachment of the accumulator to the model; rigid fixation of the accumulator helped to solve the issue. With weak attachment of the model to the subject's body, a simultaneous change of two angles was observed occasionally when the subject was performing a task. It can be explained by the "multilayered" scheme used in the experiment: gyroscope components were placed on the motor module, and the motor module was placed on the subject's head. Steady movements of the subject's head also made their contribution. We will consider it in the fabrication of the experimental sample and will use software and hardware automatic calibration of the device position with respect to the subject's position The main motor activity parameters measured by the experimental sample that we plan to fabricate will be linear acceleration of the accelerometer, angular acceleration of the gyroscope and a magnetic field vector of a magnetometer. EMG and EOG muscular artifacts will be removed using additional band-

Fig. 2. Studying heart activity. Electrode placement and ECG data obtained with KARDi3. (B) Electrode placement for photoplethysmogram and SpO2 analysis using the prototype model of the neurodevice

reject filters or special software. Cardiogram artifacts can be removed by changing electrode attachment mode from stationary to dynamic, with a possibility to shift electrodes by no less than 10 mm. Thus, the electrode can be moved if it has been placed over an artery. Besides, improving accessories for electrode attachment will also reduce the amount of artifacts.

The results of EEG-EMG fusion experiment showed the considerable advantage of hybrid BCIs over single-modality BCIs and confirmed the feasibility of simultaneous recording of various physiological signals [19-21, 23, 29]. Due to the increased classification accuracy and flexibility, a hybrid system is more reliable and exhibits higher performance. The obtained results lead us to conclude that fused EEG-EMG recording improves the interpretation of intended and actual physical activity. EEG signals unrelated to muscular activity are an additional identification tool that can be used in robotics. We speculate that improvements to the system and simultaneous use of various physiological signals will result in almost 100 % classification accuracy.

By now, very few works describing such experiments have been published. All of them are non-representative with respect to the number of participants. To increase signal classification accuracy and safety of robotic devices, further research is necessary. Still, certain difficulties remain. First, electrode shifting is a problem, because the correct placement of electrodes is what defines intensity, quality and reproducibility of signals. With respect to that, non-contact technologies can be a solution. Second, complex movements involving several muscles (hand, forearm, shoulder girdle and trunk muscles) are generated by a large number of motor cortex areas, and the size of each area is unique for every person, which impedes reconstruction of complex movements. To solve this problem, new technologies capable of isolating target movements from unrelated ones are necessary. Some solutions have been proposed so far, including invasive interfaces based on electrocorticography [30, 31].

CONCLUSIONS

The tests of the neurodevice prototype capable of simultaneous detection of different electrophysiological signals confirmed the feasibility of hybrid approach to the development of systems for external device control. Fusion of several modalities or switching from one to another to select the one that best interprets human intention increases signal classification accuracy and can possibly improve robotic device performance. Further research is necessary with a larger number of participants involved, including those with different pathologies.

EMG and EEG signal classification accuracy in single modality and hybrid approaches, %

Subject EMG EMG + EEG Dynamics

1 84.0 93.0 + 9.0

2 72.0 84.0 + 12.0

3 77.0 88.0 + 11.0

4 92.0 97.0 + 5.0

5 70.0 93.0 + 23.0

6 63.0 79.0 + 16.0

7 69.0 81.0 + 12.0

8 75.0 90.0 + 15.0

9 61.0 75.0 + 14.0

10 80.0 88.0 + 8.0

Mean 74.3 86.8 + 15.0

References

1. Singh RM, Chatterji S, Kumar A. Trends and challenges in EMG based control scheme of exoskeleton robots — a review. Int J Sci Eng Res. 2012; 3 (9): 933-40.

2. Artemiadis P. EMG-based robot control interfaces: past, present and future. Adv Robot Autom. 2012, 1 (2): e107.

3. Li G. Electromyography pattern-recognition-based control of powered multifunctional upper-limb prostheses. In: Mizrahi J, editor. Advances in applied electromyography. InTech; 2011. 224 p.

4. Kim Y, Doh NL, Youm Y, Chung WK. Robust discrimination method of the electrooculogram signals for human-computer interaction controlling mobile robot. Intell Autom Soft Comp. 2007; 13 (3): 319-36.

5. Oonishi Y, Oh S, Hori Y. New control method for power-assisted wheelchair based on upper extremity movement using surface myoelectric signal. 2008 10th IEEE International Workshop on Advanced Motion Control. 2008; 498-503.

6. Gu Y, Dremstrup K, Farina D. Single-trial discrimination of type and speed of wrist movements from EEG recordings. Clin Neurophysiol. 2009 Aug; 120 (8): 1596-600.

7. Shenoy P, Miller KJ, Crawford B, Rao RP. Online electromyographic control of a robotic prosthesis. IEEE Trans Biom Eng. 2008 Mar; 55 (3): 1128-35.

8. Kiguchi K, Hayashi Y. An EMG-based control for an upper-limb power-assist exoskeleton robot. IEEE Trans Syst Man Cybern B Cybern. 2012 Aug; 42 (4): 1064-71.

9. Rosen J, Brand M, Fuchs MB, Arcan М. A myosignal-based powered exoskeleton system. IEEE Trans Syst Man Cybern A Syst Hum. 2001; 31 (3): 210-22.

10. Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet. 2013 Feb 16; 381 (9866): 557-64.

11. Nunez PL, Srinivasan R. Electric fields of the brain: the neurophysics of EEG. Oxford, UK: Oxford university press; 2006. 611 р.

12. Arai M, Nishinaka Y, Miki N. Long-term electroencephalogram measurement using polymer-based dry microneedle electrode. In: 2015 Transducers - 2015 18th International Conference on SolidState Sensors, Actuators and Microsystems (TRANSDUCERS); 2015 June 21-25; Anchorage, AK. IEEE; 2015. p. 81-4.

13. Agashe H, Contreras-Vidal JL. Reconstructing hand kinematics during reach to grasp movements from electroencephalographic signals. Conf Proc IEEE Eng Med Biol Soc. 2011; 2011: 5444-7. doi: 10.1109/IEMBS.2011.6091389.

14. Mohamed AK, Marwala T, John LR. Single-trial EEG discrimination between wrist and finger movement imagery and execution in a sensorimotor BCI. Conf Proc IEEE Eng Med Biol Soc. 2011; 2011: 6289-93.

15. Yang J, Singh H, Hines EL, Schlaghecken F, Iliescu DD, Leesonc MS, Stocks NG. Channel selection and classification of electroencephalogram signals: an artificial neural network and genetic algorithm-based approach. Artif Intell Med. 2012; 55 (2):117-26.

16. Feldman Y, Ishai PB, Raicu V. Electrode polarization. Dielectric relaxation in biological systems: physical principles, methods, and applications. 2015 Jul 23: 140.

17. Allison BZ, Leeb R, Brunner C, Müller-Putz GR, Bauernfeind G, Kelly JW, Neuper C. Toward smarter BCIs: extending BCIs through hybridization and intelligent control. J Neural Eng. 2012

Литература

1. Singh RM, Chatterji S, Kumar A. Trends and challenges in EMG based control scheme of exoskeleton robots — a review. Int J Sci Eng Res. 2012; 3 (9): 933-40.

2. Artemiadis P. EMG-based robot control interfaces: past, present and future. Adv Robot Autom. 2012, 1 (2): e107.

3. Li G. Electromyography pattern-recognition-based control of powered multifunctional upper-limb prostheses. In: Mizrahi J, editor. Advances in applied electromyography. InTech; 2011. 224 p.

4. Kim Y, Doh NL, Youm Y, Chung WK. Robust discrimination method

ARTICLE I NEUROINTERFACES

Feb; 9 (1): 013001.

18. Pfurtscheller G, Allison BZ, Brunner С, Bauernfeind G, Solis-Escalante T, Scherer R, et al. The hybrid BCI. Front Neurosci.

2010 Apr 21; 4: 30.

19. Leeb R, Sagha H, Chavarriaga R, Mill J. Multimodal fusion of muscle and brain signals for a hybrid-BCI. Conf Proc IEEE Eng Med Biol Soc. 2010; 2010: 4343-6.

20. Xie P, Chen X, Ma P, Li X, Su P. Identification Method of Human Movement Intention based on the Fusion Feature of EEG and EMG. In: Proceedings of the World Congress on Engineering 2013, vol. 2; 2013 Jul 3-5; London, UK. IAENG; 2013.

21. Kiguchi K, Hayashi Y. A study of EMG and EEG during perception-assist with an upper-limb power-assist robot. In: 2012 IEEE International Conference on Robotics and Automation (ICRA); 2012 May 14-18; Saint Paul, MN. IEEE; 2012. p. 2711-6.

22. Gopura RARC, Kiguchi K, Li Y. SUEFUL-7: A 7DOF upper-limb exoskeleton robot with muscle-model-oriented EMG-based control. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2009 Oct 11-19; St. Louis, USA. IEEE; 2009. p. 1126-31.

23. Witkowski M, Cortese M, Cempini M, Mellinger J, Vitiello N, Soekadar SR. Enhancing brain-machine interface (BMI) control of a hand exoskeleton using electrooculography (EOG). J Neuroeng Rehabil. 2014 Dec 16; 11: 165.

24. Rumyantseva SA, Stupin VA, Oganov RG, Silina EV, Petrov VA, Kasymov VA, et al. The ways of long-term blood pressure control systems development. Cardiovascular Therapy and Prevention. 2015; (14) 1: 78-82.

25. Stupin VA, Silina EV, Oganov RG, Bogdanov YA, Shusharina NN. Development of an invasive device for long-term remote monitoring of cardiovascular system parameters, including blood pressure, in patients with comorbid conditions. Biosciences Biotechnology Research Asia, Aug 2015. Vol. 12(2), р. 1255-1263.

26. Silina EV., Stupin VA., Kolesnikova EA., Rumyantseva SA., Oganov RG. New devices for invasive blood pressure detection and the prototype for a long-term invasive distance control of cardiovascular system. Biology and medicine. 2014; 6 (3): BM-

045-14, 5 p.

27. Pfurtscheller G, Leeb R, Slater M. Cardiac responses induced during thought-based control of a virtual environment. Int J Psychophysiol. 2006 Oct; 62 (1): 134-40.

28. Pfurtscheller G, Ortner R, Bauernfeind G, Linortner P, Neuper C. Does conscious intention to perform a motor act depend on slow cardiovascular rhythms? Neurosci Lett. 2010 Jan 1; 468 (1):

46-50.

29. Shahid S, Prasad G, Sinha RK. On fusion of heart and brain signals for hybrid BCI. In 2011 5th International IEEE/EMBS Conference Neural Engineering (NER); 2011 Apr 27 - May 11; Cancun, Mexico. IEEE; 2011. р. 48-52.

30. Judy JW. Neural interfaces for upper-limb prosthesis control: opportunities to improve long-term reliability. IEEE Pulse. 2012 Mar; 3 (2): 57-60.

31. Yanagisawa T, Hirata M, Saitoh Y, Goto T, Kishima H, Fukuma R, et al. Real-time control of a prosthetic hand using human electrocorticography signals: technical note. J Neurosurg.

2011 Jun; 114 (6): 1715-22.

of the electrooculogram signals for human-computer interaction controlling mobile robot. Intell Autom Soft Comp. 2007; 13 (3): 319-36.

5. Oonishi Y, Oh S, Hori Y. New control method for power-assisted wheelchair based on upper extremity movement using surface myoelectric signal. 2008 10th IEEE International Workshop on Advanced Motion Control. 2008; 498-503.

6. Gu Y, Dremstrup K, Farina D. Single-trial discrimination of type and speed of wrist movements from EEG recordings. Clin

СТАТЬЯ I НЕЙРОИНТЕРФЕЙСЫ

Neurophysiol. 2009 Aug; 120 (8): 1596-600.

7. Shenoy P, Miller KJ, Crawford B, Rao RP. Online electromyographic control of a robotic prosthesis. IEEE Trans Biom Eng. 2008 Mar;

55 (3): 1128-35. 21.

8. Kiguchi K, Hayashi Y. An EMG-based control for an upper-limb power-assist exoskeleton robot. IEEE Trans Syst Man Cybern B Cybern. 2012 Aug; 42 (4): 1064-71.

9. Rosen J, Brand M, Fuchs MB, Arcan М. A myosignal-based 22. powered exoskeleton system. IEEE Trans Syst Man Cybern

A Syst Hum. 2001; 31 (3): 210-22.

10. Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, et al. High-performance neuroprosthetic control by

an individual with tetraplegia. Lancet. 2013 Feb 16; 381 (9866): 23. 557-64.

11. Nunez PL, Srinivasan R. Electric fields of the brain: the neurophysics of EEG. Oxford, UK: Oxford university press; 2006.

611 р. 24.

12. Arai M, Nishinaka Y, Miki N. Long-term electroencephalogram measurement using polymer-based dry microneedle electrode. In: 2015 Transducers - 2015 18th International Conference on SolidState Sensors, Actuators and Microsystems (TRANSDUCERS); 25. 2015 June 21-25; Anchorage, AK. IEEE; 2015. p. 81-4.

13. Agashe H, Contreras-Vidal JL. Reconstructing hand kinematics during reach to grasp movements from electroencephalographic signals. Conf Proc IEEE Eng Med Biol Soc. 2011; 2011: 5444-7. doi: 10.1109/IEMBS.2011.6091389.

14. Mohamed AK, Marwala T, John LR. Single-trial EEG discrimination 26. between wrist and finger movement imagery and execution in

a sensorimotor BCI. Conf Proc IEEE Eng Med Biol Soc. 2011; 2011: 6289-93.

15. Yang J, Singh H, Hines EL, Schlaghecken F, Iliescu DD, Leesonc MS, Stocks NG. Channel selection and classification 27. of electroencephalogram signals: an artificial neural network and genetic algorithm-based approach. Artif Intell Med. 2012; 55 (2):117-26. 28.

16. Feldman Y, Ishai PB, Raicu V. Electrode polarization. Dielectric relaxation in biological systems: physical principles, methods, and applications. 2015 Jul 23: 140.

17. Allison BZ, Leeb R, Brunner C, Müller-Putz GR, Bauernfeind G, 29. Kelly JW, Neuper C. Toward smarter BCIs: extending BCIs through hybridization and intelligent control. J Neural Eng. 2012

Feb; 9 (1): 013001.

18. Pfurtscheller G, Allison BZ, Brunner a Bauernfeind G, Solis- 30. Escalante T, Scherer R, et al. The hybrid BCI. Front Neurosci.

2010 Apr 21; 4: 30.

19. Leeb R, Sagha H, Chavarriaga R, Mill J. Multimodal fusion of 31. muscle and brain signals for a hybrid-BCI. Conf Proc IEEE Eng

Med Biol Soc. 2010; 2010: 4343-6.

20. Xie P, Chen X, Ma P, Li X, Su P. Identification Method of Human

Movement Intention based on the Fusion Feature of EEG and EMG. In: Proceedings of the World Congress on Engineering 2013, vol. 2; 2013 Jul 3-5; London, UK. IAENG; 2013. Kiguchi K, Hayashi Y. A study of EMG and EEG during perception-assist with an upper-limb power-assist robot. In: 2012 IEEE International Conference on Robotics and Automation (ICRA); 2012 May 14-18; Saint Paul, MN. IEEE; 2012. p. 2711-6. Gopura RARC, Kiguchi K, Li Y. SUEFUL-7: A 7DOF upper-limb exoskeleton robot with muscle-model-oriented EMG-based control. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2009 Oct 11-19; St. Louis, USA. IEEE; 2009. p. 1126-31.

Witkowski M, Cortese M, Cempini M, Mellinger J, Vitiello N, Soekadar SR. Enhancing brain-machine interface (BMI) control of a hand exoskeleton using electrooculography (EOG). J Neuroeng Rehabil. 2014 Dec 16; 11: 165.

Румянцева С. А., Ступин В. А., Оганов Р. Г., Силина Е. В., Петров В. А., Касымов В. А. и др. Пути развития систем длительного контроля артериального давления. Кардиоваскулярная терапия и профилактика. 2015; (14) 1: 78-82. Stupin VА, Silina EV, Oganov RG, Bogdanov YA, Shusharina NN. Development of an invasive device for long-term remote monitoring of cardiovascular system parameters, including blood pressure, in patients with comorbid conditions. Biosciences Biotechnology Research Asia, Аug 2015. Vol. 12(2), р. 12551263.

Silina EV., Stupin VA., Kolesnikova EA., Rumyantseva SA., Oganov RG. New devices for invasive blood pressure detection and the prototype for a long-term invasive distance control of cardiovascular system. Biology and medicine. 2014; 6 (3): BM-

045-14, 5 p.

Pfurtscheller G, Leeb R, Slater M. Cardiac responses induced during thought-based control of a virtual environment. Int J Psychophysiol. 2006 Oct; 62 (1): 134-40. Pfurtscheller G, Ortner R, Bauernfeind G, Linortner P, Neuper C. Does conscious intention to perform a motor act depend on slow cardiovascular rhythms? Neurosci Lett. 2010 Jan 1; 468 (1):

46-50.

Shahid S, Prasad G, Sinha RK. On fusion of heart and brain signals for hybrid BCI. In 2011 5th International IEEE/EMBS Conference Neural Engineering (NER); 2011 Apr 27 - May 11; Cancun, Mexico. IEEE; 2011. р. 48-52. Judy JW. Neural interfaces for upper-limb prosthesis control: opportunities to improve long-term reliability. IEEE Pulse. 2012 Mar; 3 (2): 57-60.

Yanagisawa T, Hirata M, Saitoh Y, Goto T, Kishima H, Fukuma R, et al. Real-time control of a prosthetic hand using human electrocorticography signals: technical note. J Neurosurg. 2011 Jun; 114 (6): 1715-22.