CC BY
127
Event-Related Telemetry of for Personalized Remote Monitorinj and Stress under Conditions of Everyday
DOI: 10.17691/stm2019.11.1.13 Received January 11, 2019
S.A. Polevaya, DSc, Head of the Department of Neurophysiology, Central Scientific I Head of the Department of Psychophysiology2;
E.V. Eremin, Engineer, Department of Neurophysiology, Central Scientific Research Laborati
N.A. Bulanov, MSc Student3;
А.V. Bakhchina, PhD, Researcher, Laboratory of Psychophysiology named after V.B. Shvyrkov4;
Leading Engineer, Laboratory for Behavior and Behavioral Ecology of Mammals5;
A.V. Kovalchuk, Junior Researcher, Laboratory of Autowave Processes6;
S.B. Parin, DSc, Head of the Interdepartmental Laboratory of Cognitive Psychophysiology2
1Privolzhsky Research Medical University, 10/1 Minin and Pozharsky Square, Nizhny Novgorod, 603005, Russia;
2National Research Lobachevsky State University of Nizhny Novgorod, 23 Prospekt Gagarina, Nizhny Novgorod,
603950, Russia;
3National Research University Higher School of Economics, 20 Myasnitskaya, Moscow, 101000, Russia;
"Institute of Psychology, Russian Academy of Sciences, 13, Yaroslavskaya St., Moscow, 129366; Russia;
5A.N. Severtsov Institute of Ecology and Evolution, Russian Academy of Sciences, 33 Leninsky Prospect,
Moscow, 119071, Russia;
6Federal Research Center Institute of Applied Physics, Russian Academy of Sciences, 46 Ulyanova St.,
Nizhny Novgorod, 603950, Russia
The aim of the study is to characterize a new information and telecommunication technology — event-related of the heart rate telemetry — for remote monitoring of cognitive functions and stress conditions in the context of everyday activity.
Results. A new technology of event-related telemetry of the heart rate has been developed and characterized in terms of the cognitive function monitoring. The optimal architecture of this system provides for continuous monitoring of the patient's functional state in the context of sensorimotor activity while controlling the features of information images in the virtual computer environment. Errors in this control function are considered to objectively reflect the primary cognitive functions and sensorimotor coordination. The possibility of using such multilevel psychophysiological monitoring throughout the everyday life activity is discussed.
Conclusion. A novel automated expert system — event-related telemetry — has been developed; this includes hardware, algorithmic and software Internet resources for detecting the early signs of stress and cognitive function impairment in the real time mode, without restricting person's mobility and without attracting his/her attention to the measurement process.
Key words: cognitive functions; event-related telemetry; information technology; stress diagnosis; cognitive rehabilitation.
Introduction
One of the important milestones of today's neurology, psychiatry, neurophysiology, and cognitive science is the automated assessment of cognitive processes under various clinical, experimental and natural conditions whereby the investigator or doctor can monitor the results from a distance. Until recently, electrical signals from brain structures were considered as the only physiological correlates of consciousness. At the same time, no less significant is the autonomic maintenance of cognitive functions; therefore, the search for autonomic components of cognitive functional systems
Corresponding author: Sofia A. Polevaya, e-mail: [email protected]
is promising [1]. The contextual nature of consciousness dictates special requirements regarding the tools for extraction of knowledge: namely, the measurement process should not distort the obtained information.
In this study, we propose a Web platform technology for the controlled activation of cognitive modules in the physiological system based on the event-related telemetry of the heart rate (ERT HR), which allows for monitoring the autonomic regulation in the context of everyday activity without involving the monitored subject in the measurement process. The novelty of this approach lies in that the activity paradigm of systemic psychophysiology is transferred to the known telemetric methods [2, 3].
The proposed solution to the problem is based on the multidisciplinary approach to functional diagnostics and the new forms of personalized monitoring, remote diagnosis and controlled presentation of testing stimuli using the telemetric information systems.
Event-related telemetry: heart rate monitoring
Clearly, a state-of-art measurement must meet the following criteria:
safety and convenience for use in everyday life; mobility, i.e. no restrictions on movements and remoteness of the signal source from the signal receiving unit;
continuity of the signal recording, i.e. the lowest possible power consumption by the sensors;
autonomous signal measurement mode with the automatic interruption processing;
a set of sensors integrated in the sensor platform, providing information sufficient for an objective assessment of the patient's functional condition, as well as providing an opportunity to predict changes in the functional state;
sensors to measure the physiological parameters that must be validated with certified and accepted diagnostic methods, i.e. produce correct information of high quality;
a signal recording system that is resistant to external interference as much as possible, i.e. not susceptible to external context factors not related to the actual signal source;
the possibility of obtaining data online. Based on these criteria, a schematic setup was developed and a prototype of the event-related telemetry of physiological signals and behavioral data was created. Optimal in size and power consumption, sensors of physiological signals, microprocessors, radio receiving and transmitting devices were selected. Those were Bluetooth Heart Rate/Speed & Distance Monitor (HxM) and ZephyrTM HxMTM Smart — Zephyr BIO PACH BH3-M1 (Zephyr Technology, USA), capable of continuous EEG recording during the usual everyday activity. The ZephyrTM HxMTM Smart — Zephyr BIO PACH BH3-M1 (HxM) is a specialized miniature sensor platform that combines a microprocessor, a radio receiving/transmitting unit and low-power miniature sensors for ECG, body acceleration and distance.
Data transfer to a smartphone or personal computer is mediated by Bluetooth. Packets of raw data are transmitted at 1 s intervals. Each packet contains a unique device identifier, the last 15 R-R intervals records, and the time past from the start of the recording. Without recharging, the sensor platform can function for 150 h. The maximum distance from the signal transmitter to the storage-relay device is 10 m; this device performs temporary data storage and preprocessing. This function can be performed by a personal smartphone with the Android operating system or a personal computer with the Microsoft Windows operating system.
Then, the processed data is transmitted via GSM channels to the Internet and a specialized server of the system.
Thanks to the developed application for smartphones with the Android platform [4], the system is enabled to perform the following operations:
simultaneous signal recording from 7 sensor platforms; the user can select the type of data he/she prefers to receive from the sensor;
continuous collection and storage of data from the sensor (sensor platform) for at least 24 h;
when the connection is broken, the device repeatedly attempts to connect to the sensor and continues to collect data while maintaining information about the time of the failure and the connection recovery;
data is available for transmission to external programs, and the user is enabled to set the server address;
there is an interface for marking the start and the end of the selected events;
the screen of the smartphone displays the physiological data and indicators of the autonomic function;
the system supports the data exchange with the StressMonitor Internet application, which is capable of data storage, processing, visualization, and interpretation.
The StressMonitor Internet application developed to collect telemetry data and detect stress [5], performs the following functions:
collect measurements (R-R intervals, GPS signals) from a smartphone via HTTP; register users in your account; display and filter metadata of experiments; automatically calculate the spectral characteristics of the R-R signal and stress detection;
graphic display of the R-R intervals, heart rate, spectral characteristics and stress, and GPS signal maps for each experiment;
export the experimental data for further processing in TXT and CSV formats.
The site database is a relational SQL database supported by the MySQL 5.5 software.
In the pseudo real time mode, the preprocessing and spectral analysis of rhythmograms is implemented as follows:
the received R-R signal is "sliced" into time windows of 100 s with a time shift of 10 s;
the spectrum of non-uniform signals is calculated for the obtained windows by the method of discrete Fourier transform;
the spectrum is divided into ranges, and the power of each range is calculated: VLF (0.003-0.040 Hz), LF (0.04-0.15 Hz), HF (0.15-0.4 Hz);
the derivative characteristics are calculated, namely, the total spectrum power TP=VLF+LF+HF and the sympathovagal balance LF/HF.
As a result, the researcher obtains a spectrogram,
//////////////////////^^^^
110 CTM J 2019 J v0l. 11 J N0.1 S.A. Polevaya, E.V. Eremin, N.A. Bulanov, A.V. Bakhchina, A.V. Kovalchuk, S.B. Parin
Figure 1. User screen of the StressMonitor Internet application [5]
From top to bottom: online recording of a native cardiac rhythmogram; calculated values of the heart rate variability spectrum (the total power of the variability spectrum of the R-R intervals TP — graph in black; sympathovagal balance LF/HF — graph in red); online localization of a person who is the source of the signals
Figure 2. Typical manifestations of the onset of acute stress in the native cardiac rhythmogram (a) and the dynamics of the relevant parameters (b)
TR is the total power of the variability spectrum of the R-R intervals; LF/HF — sympathovagal balance
which automatically determines the place, time and events associated with stress in an individual during his/ her everyday activity (Figure 1).
The moment the stress begins is determined automatically from characteristic changes in the native cardiac rhythmogram (Figure 2 (a)) and the dynamics of its derivatives: the drop in the total power of the heart rate variability spectrum (TR) against the background of a sharp increase in the sympathovagal balance index (LF/HF) (Figure 2 (b)). Detecting stress by these indices is based on the principles of the three-component neurochemical mechanism of stress [6, 7].
Web-based platform for the controlled activation of cognitive modules of the physiological system
To study the cognitive functions, it is crucial to model the information charges, which provide controlled
activation and measurement of primary cognitive functions and sensorimotor activity. For this purpose, a Web platform has been developed; the ApWay.ru Web platform provides for a universal infrastructure for designing and conducting various tests. The key to this architecture was the assertion of Norbert Wiener's — the founder of cybernetics: the internal properties of the information system are manifested in the distortions that this system introduces into the original signal [8]. As a measuring circuit, we selected a closed system, in which the computer is both a signal source and a recorder. Distortions and errors introduced by a human into a controlled feature of an information image characterize an individual cognitive system as follows:
errors in signal detection are absolute thresholds of perception;
errors in distinguishing signal levels are differential thresholds;
signal identification errors associated with time intervals or spatial gradients correspond to temporal and spatial thresholds.
The general measuring circuit is able to perform the following measurement cycle: generating an information image — a subjective sensory image — the motoric control of most significant signal parameters — recording of changes in the virtual object under control. Manipulations of the operator when executing instructions are converted into changes in the virtual object characteristics; they represent the only feedback signal in the closed "computer-human-computer" system. This system includes a module for generating stimuli in a wide range of amplitude-time parameters, a virtual control panel of measurements, a module for recording the operator motor reactions, a database and a module for generating a report in the form of tables and graphs. The architecture and functionality of the ApWay.ru Web platform meet the following requirements:
Color tone (H)
Time (ms)
Figure 3. Basic psychophysiological tests using the ApWay cognitive platform:
(a) computer campimetry; (b) sensorimotor activity test; (c) Stroop test
1) ensuring the collection and accumulation of measurement results;
2) providing the API for connecting any diagnostic systems, including software diagnostic tools on mobile devices;
3) saving patient records with their data, test results and other measurements that contribute to rehabilitation and the tasks of experts and researchers;
4) preliminary statistical processing of the measurements to identify basic quantitative indicators for each measurement;
5) creating and storing templates for quick launch of tests, for combining them into a "battery of tests" and for conducting the tests practically without involving an expert;
6) providing feedback to the subject and the expert both on the results of a specific measurement, and on the changes in basic indicators for a given period.
The MySQL database management system is used to provide data storage. Since each test results in a specific set of indicators, it was decided to store them in table cells as a row of the JSON format. This provides, on the one hand, the preservation of the structure, on the other — rapid access to the data using a clue.
A separate analytical module performs a primary analysis of "raw" testing data and converts them into a set of basic indicators, which are used to assess the effectiveness of cognitive functions and to display its dynamics when the test is repeated.
A template library has been created on this platform, providing access to a unified stimulus bank for all tests. The main function of the library is the storage and display-on-demand of vector images in the SVG format as required for the measurements. This format allowed us to use the vector graphics in the Web and mobile applications as well and, if necessary, instantly change the image characteristics (for example, the colors of the entire stimulus or part of it). This technical option is in demand for computed campimetry tests, for sensorimotor reaction tests, and many other tests using graphic stimuli. Program logic is fully implemented in JavaScript added with jQuery libraries and Raphael.JS.
In accordance with the described architecture, three test paradigms with specific tools for determining the thresholds for cognitive processing of information signals are currently implemented; those are computer color campimetry, sensorimotor activity tests, and the Stroop test. Each test ensures that the user is involved in a specific activity on controlling the object characteristics in the virtual environment and measuring errors of this control function (Figure 3).
All tests are accessible in the form of Web applications and, therefore, available for use with any computer connected to the Internet.
The test of sensorimotor activity is based on the classic method of measuring the sensory-motor response. However, in the proposed system, the
а
b
c
////////////////M^^
112 CTM J 2019 J VOl. 11 J N0.1 S.A. Polevaya, E.V. Eremin, N.A. Bulanov, A.V. Bakhchina, A.V. Kovalchuk, S.B. Parin
Tests
analysis of the relationship between sensory and motor events is implemented according to the activity paradigm, which allows actions associated not only with the past but also with the predicted future events. The module capabilities allow one to set various test parameters such as the type of a stimulus (pictures, text, vector images); the stimulus target, location of stimuli on the screen, the exposure time for each stimulus, interstimulus interval, background of the working area, and the latency period.
In aggregate, the chosen test parameters allow us to determine the degree of brain functional preservation, the level of sensorimotor integration, the resources of spatial and selective attention, and the ability to learn and predict.
Computer color campimetry provides a measure for the color discern ability and an assessment of the subject's emotional state. In this technique, a stimulus is presented against the background of the very same color. Then, the subject changes the color of the stimulus by pressing buttons until he/she can see an object or text that has been "hidden". If successful, the subject is then given the opposite task.
In the proposed setup, new options are available: one can choose the stimuli from the library of the vector stimuli, create textual stimuli, choose the stimulus location on the screen, specify the stimulus sizes (relative and absolute) and color coordinates, set the step of color tone change, quickly create a multitude of stimuli presentations, while combining the chosen shades, stimuli, locations and sizes.
After testing the system we obtained the threshold values of color tones for each presentation, which are then transmitted to the analytical system for the primary statistical processing.
In aggregate, the results of this test allow us to conclude on the color and shape perception, on the level of operational memory and attention, and on the level of emotional tension; the results also provide for the detection of brain function changes caused by psychotropic drugs.
The Stroop test allows one to simulate a situation of cognitive conflict. A novel feature of the test, when implemented on the ApWay.ru Web platform, is the ability to set color names in various languages or add one's own color options for experimentation purposes. According to the measurement results, the errors and reaction time in each task are noted. After a primary data analysis, the system provides information on the number of errors made and the average time needed to solve the problem in each context. Based on these indicators, it is possible to draw a conclusion about the level of cognitive control, the effectiveness of processing the verbal and color information, as well as the stress tolerance under condition of mental distress.
Time-test_ApWay
Test-event
DATA PACKAGE
I
Context
ID password
NAME Platform ApWay
R-R interval
NAME StressMonitor Android
Figure 4. Architecture of event-related telemetry technology
Conclusion
An automated expert system of a fundamentally new type has been developed — event-related telemetry, possessing hardware, algorithmic and software resources for detecting early signs of extreme states in real time, without mobility restrictions and without attracting the subject's attention to the measurement process. To collect the telemetry data and detect the state of stress, we developed an Internet resource, namely the StressMonitor application based on cogni-nn.ru. To activate and control the primary cognitive functions in contexts of sensorimotor activity, a Web platform (platform.apway.ru) has been created. At present, all the modules described above are integrated into a single information and telecommunication technology (Figure 4).
The technology provides continuous long-term collection, transmission, accumulation and pre-processing of the time-synchronized cardiac rhythmographic records, the navigator data on the trajectory of a person moving indoors and in open space, the video and audio surveillance data, and the psycho-physiological test results.
Further development of this technology will be aimed at improving the expert module and enhancing the capabilities of the system for research, diagnostics, and predictions related to physiological support of cognitive functions in various contexts of natural activity.
Research funding. This work was partially supported by the Russian Foundation for Basic Research (grants No.18-013-01225_a, 18-413-520006_p_a, and 19-013-00095_a).
Conflict of interest. The authors have no conflicts of interest.
References
1. Thayer J.F., Ahs F., Fredrikson M., Sollers J.J., Wager T.D. A meta-analysis of heart rate variability and
neuroimaging studies: implications for heart rate variability as a marker of stress and health. Neurosci Biobehav Rev 2012; 36(2): 747-756, https://doi.Org/10.1016/j. neubiorev.2011.11.009.
2. Alexandrov Yu.I. Psychophysiological regularities of learning and methods of training. Psikhologicheskiy zhurnal 2012; 33(6): 5-19.
3. Krylov A.K., Aleksandrov Yu.I. Metody eksperimental'nogo issledovaniya v paradigmakh aktivnosti i reaktivnosti. V kn.: Sovremennaya eksperimental'naya psikhologiya. Tom 1 [Methods of experimental research in the paradigms of activity and reactivity. In: Modern experimental psychology. Vol. 1]. Pod red. Barabanshchikova V.A. [Barabanshchikov V.A. (editor)]. Moscow: Izd-vo "Institut psikhologii RAN" 2011; p. 463-479.
4. Kozhevnikov V.V., Polevaya S.A., Shishalov I.S., Bakhchina A.V. Mobile HR-meter (HR-meter). Certificate of
state registration of computer programs 2014618634. 2014.
5. Eremin E.V., Kozhevnikov V.V., Polevaya S.A., Bakhchina A.V. Web service for visualization and storing of heart rate measurements. Certificate of state registration of databases 2014621202. 2014.
6. Parin S.B. Humans and animals in extreme situations: neurochemistry mechanisms, evolutionary aspect. Vestnik Novosibirskogo gosudarstvennogo universiteta. Seriya: Psikhologiya 2008; 2(2): 118-135.
7. Parin S.B., Vetyugov V.V., Bakhchina A.V., Polevaya S.A. The role of the endogenous opioid system in the control of heart rate variability under cognitive loads of various levels. Sovremennye tehnologii v medicine 2014; 6(4): 116-126.
8. Wiener N. Kibernetika, ili upravlenie i svyaz' vzhivotnom i mashine [Cybernetics, or control and communication in the animal and the machine]. Moscow: Nauka; 1983; 344 p.
/////////////////////^^^^
114 CTM J 2019 J V0l. 11 J No.1 S.A. Polevaya, E.V. Eremin, N.A. Bulanov, A.V. Bakhchina, A.V. Kovalchuk, S.B. Parin
