CC BY
30
Section BRAIN-COMPUTER INTERFACES, COGNITIVE NAVIGATION WORKSHOP AND NEUROENGINEERING
R2 = 0.379
In (CSA)
Fig.2. Dependence of classification error,ln(E/Em), (E is the error and Emis its median) on the body fat, BF, (in %) and the coefficient of synergists-antagonists, CSA, and linear regression
References
Lobov S.A., Mironov V.I., Kastalskiy I.A., Kazantsev V.B. Combined Use of Command-Proportional Control of External Robotic Devices Based on Electromyography Signals .Sovremennyetehnologii v medicine 2015; 7(4): 30-38, http://dx.doi.org/10.17691/stm2015.7A04
Fixation-Based Eye-Brain-Computer Interfaces: Approaching a Better Human-Computer Symbiosis
S.L. Shishkin1 *, Y.O. Nuzhdin1, A.G. Trofimov2, E.P. Svirin1, A.A. Fedorova1, LA. Dubynin1 and B.M. Velichkovsky1
1 NRC «Kurchatov Institute», Moscow, Russia;
2 NRNU MEPhI, Moscow, Russia.
* Presenting e-mail: sergshishkin@mail.ru
Computers are powerful tools to augment many of our intellectual abilities. However, the effectiveness of our interaction with computers depends on interfaces between them and our brains (Engelbart, 1962).
The graphical user interfaces (GUIs) and the input devices compatible with them, such as computer mice, have made the way we send commands to computers relatively fast and fluent. Can we further improve the interaction? When we fixate a GUI button or a web link with gaze and decide that we should click on them, is it really necessary to approach them, e.g., with a cursor by manually moving a mouse, and then to click the mouse button with a finger? Our gaze already indicates the position on the screen and the existing eye trackers are able to report this position. Can we design such a brain-computer interface (BCI) that could reveal our intention to click so promptly and reliably that our interaction with computers would become more effective than in the case of using conventional input devices?
Mental imagery based BCIs were already applied for supplementing the gaze based interaction with a "mental click", but the click in their operation required additional time of the order of seconds, evidently contradicting the idea of fluent control. It is much more desirable to recognize the intention to act on a certain screen position directly from brain activity patterns that accompany intention-specific fixations (Velichkovsky and Hansen, 1996).
The first attempts to implement this approach were made by Zander and colleagues (Proteak et al., 2013). They differentiated the spontaneous fixations and the fixations used to control a computer using the electroencephalogram (EEG) features. However, the fixation threshold for issuing a command in these studies was too long (1 s), so that again the interaction could hardly be considered as fluent. Moreover, the participant task was too simple compared to real-life task.
To study the issue in more complex settings, we developed a gaze controlled computer game EyeLines and recorded
Section BRAIN-COMPUTER INTERFACES, COGNITIVE NAVIGATION WORKSHOP AND NEUROENGINEERING
EEG when 8 participants played it with their gaze only. Moves in the game were made in "control-on" mode of the game with fixations exceeding 500 ms threshold. In the "control-off" mode, fixations did not led to actions, and 500 ms or longer spontaneous fixations were collected. A special procedure was developed to make sure that the analyzed EEG intervals were not contaminated by the artifacts related to eye movement.
The EEG during controlling but not spontaneous fixations showed pronounced negativity in the posterior cortical areas starting early in the course of fixations. Using a simple feature extraction algorithm, greedy feature selection strategy and a linear classifier committee, we obtained, on average, a better than 35% true positive rate for controlling fixations while keeping the false alarm rate at about 10% on the test data with 5-fold cross-validation, much above the random level. More elaborated feature extraction algorithms are currently being tested.
Moreover, a two-threshold strategy was developed to enable smooth interaction even with the current relatively low true positive rate. When a fixation exceeds the first, short (e.g., 500 ms) threshold, the BCI is applied to detect the intention to act on the fixated location. If the BCI misses the intention, the user still may issue the command by continuing fixating the same position until the second (e.g., 1000 ms) threshold is exceeded. Since spontaneous fixations of this length are rare, it is safe to execute a command at this time even without confirmation from the BCI; alternatively, a confirmation from the BCI can be required again but with a low BCI threshold. With such a strategy, the users may develop a more stable EEG pattern associated with controlling fixations, because this will lead to faster move execution.
Our results imply that the "eye-brain-computer" interfaces (EBCIs) can not only helping neurorehabilitation, as the typical BCIs (Kaplan, 2016), but also can be used by healthy persons. Fast converting of intentions into computer actions without using any supplemental tasks (such as computer mouse manipulation, as well as special mental imagery or attention to external stimulation for activating a BCI) may make certain tasks involving interaction with computers especially fluent. This will open new perspectives for unfolding the full scale of benefits from augmenting brain function with the power of computers.
At the conference, these prospects will be discussed within a more general framework of our current and planned studies, including the search for new intention markers both in the EEG and the magnetoencephalogram (MEG), investigating the factor of the feeling of agency and free will in the use of BCIs, and online interaction of the users with different types of BCIs and EBCIs.
Acknowledgements
Parts of this work related to the specific methods of intention marker detection, their use in the EBCIs and the studies of feeling of agency and free will were supported by the Russian Science Foundation, grant 14-28-00234.
References
1. A. One, B. Two, and C. Three, Phys. Rev., 1972, 8(3), 555-566.
2. D. C. Engelbart, Augmenting Human Intellect: A Conceptual Framework, 1962.
3. B. M. Velichkovsky, and J. P. Hansen, SIGCHI Conf. Human Factors in Comp. Systems, 1996, 496-503.
4. J. Proteak, K. Ihme, and T. O. Zander, UAHCI. Design Methods, Tools, and Interaction Techniques for eInclusion, 2013, 662-671.
5. A. Y. Kaplan, Fiziologiya Cheloveka [Human Physiol.], 2016, 42(1), 118-127 (in Russian).
BCI Matrix Speller Based on Coded Visual Evoked Potentials
R.K. Grigoryan1 *, D.B. Flatov1, A. Ya. Kaplan1,2,3
1 Lomonosov Moscow State University, Moscow, Russian Federation;
2 Lobachevsky State University of Nizhni Novgorod, Nizhni Novgorod, Russian Federation;
3 Pirogov National Russian Medical University, Moscow, Russian Federation. * Presenting e-mail: grraph.bio@gmail.com
Aims of the study
Brain-computer interface (BCI) is a system that utilizes neurophysiological correlates of attention to establish communication with computer. The most popular type of BCIs is BCI based on visual evoked potentials (VEP BCI), for example P300 BCI or steady-state VEP BCI. Here we examine another paradigm - BCI based on code-modulated visual evoked potentials (C-VEP BCI). Within this approach, m-sequence [1] is used as pattern of visual stimulation. The crucial feature of such sequence is its autocorrelation function that has only one peak, and equals zero in all posi-
OM&P
