CC BY
67
A comprehensive study of EEG-based control of artificial arms
Ihab Abdulrahman Satam
Obuda University, Doctoral School of Safety and Security Sciences, Budapest, Hungary;
Northern Technical University, Technical Institute / Al-hawija, Electronic Techniques Department, Republic of Iraq, e-mail: ihab.satam@uni-obuda.hu, ORCID iD: https://orcid.org/0000-0002-9749-0944
DOI: 10.5937/vojtehg71-41366; https://doi.org/10.5937/vojtehg71-41366
FIELD: mechanics, electronics ARTICLE TYPE: original scientific paper
Abstract:
Introduction/purpose: The electroencephalography (EEG) signal has a great impact on the development of prosthetic arm control technology. EEG signals are used as the main tool in functional investigations of human motion. The study of controlling prosthetic arms using brain signals is still in its early stages. Brain wave-controlled prosthetic arms have attracted researchers' attention in the last few years.
Methods: Several studies have been carried out to systematically review published articles as a means of offering researchers and experts a comprehensive summary of the present, state-of-the-art EEG-based control techniques used in the prosthetic arm and other technologies.
Results: 175 articles were studied, compared, and filtered to only include the articles that have strong connections to the study. Conclusion: This study has three goals. The first one is to gather, summarize, and evaluate information from the studies published between 2011 and 2022. The second goal is to extensively report on the holistic, experimental outcomes of this domain in relation to current research. It is systematically performed to provide a wealthy image and grounded evidence of the current state of research covering EEG-based control of prosthetic arms to all experts and scientists. The third goal is to recognize the gap in knowledge that demands further investigation and to recommend directions for future research in this area.
Keywords: EEG, BCI, comprehensive study, prosthetic arms, controllers.
Introduction
The absence of the upper limb results in severe impairment in everyday life, which can further influence both the social and mental state
ф
Ô >
со CM о CM
of
Ш Cd
О О
_J
с о 2: х о ш
I—
>-
а: <
н
(Л <
О
>о 2:
X ш I—
О
О >
Ф
(Abdulrahman Satam, 2021). For these reasons, developments in cosmetic and body-driven prostheses date from some centuries ago, and they have been evolving ever since. Research showed that the estimated percentage of impaired people is rising up due to wars, conflicts, diseases, accidents, and forgotten minefields from previous battles and wars.
A prosthesis is much more than a device; it also completes a wearer's sense of wholeness. It gives emotional comfort. The history of prosthetics is not just about the advancement of medical science, it is a history of human beings who miss an essential part of themselves. The earliest known prosthetic was not an eye, leg, or arm. It was a toe, first made by Egyptians around 3000 years ago. Then development continued with the Roman Empire to the end of the Middle Ages and finally to the civil war in the United States of America.
A decade ago, prosthetic limbs were developed as a practical complementary system for impaired people. Prosthetics, or artificial limbs, are used to replace limbs that were lost or absent limbs from birth. They enable those with congenital limb differences and amputees alike to improve function and mobility. Due to advances in medical science, prosthetics have improved and are capable of remarkable things (Osama & Allauddin, 2022).
In addition to the development of the prosthetic arm (Figure 1) design, scientists are focusing on improving the control of the techniques for the purpose of accuracy, performance enhancement, and the comfort of the prosthesis.
Ensuring the smoothness and effective control techniques of the prosthetic limb is an important factor in the interface between the wearable prosthesis and the human since the prosthetic limb is donned by a human.
Figure 1 - Prosthetic arm Рис. 1 - Протез руки Слика 1 - Простетичка рука
Therefore, those control strategies can be classified according to the human-robot interaction method. The control of the prosthetic arm is influenced by electrophysiological signals. These signals have been well-known tools to examine the capacity and conduct of the human movement in ongoing research.
Electroencephalography (EEG) has been one of frequently used physiological signals in the control techniques of prosthetic limbs, especially in the upper limbs. EEG is considered a non-invasive and convenient method that may be appropriate for realistic application. Recently, it was found that fewer endeavors have been made to efficiently audit these reviews, as a way of offering analysts and specialists a synopsis of the current, best-in-class EEG-based control systems utilized for assistive innovation. Hence, this research has three primary objectives.
The primary aim is to deliberately assemble, abridge, assess, and organize data with respect to accuracy and estimations of the past research distributed in the publications between 2011 and 2018.
The second objective is to broadly report on all the trial results of this domain's present research. It is methodically performed to give a clear picture and grounded proof of the momentum conditions of research covering EEG-based control uses and benefits for controlling assistive robotics to every specialist and researcher. The third objective is to perceive the whole of information that requests in-depth examination and to suggest ways for future research in this domain (Mandekar et al, 2022). To achieve these objectives, the following research questions (RQs) have been put forward:
(Q1) What are the types of EEG signals that are used to control the prosthetic arm?
(Q2) How do these signals translate to control commands?
The solutions to these questions will guide the reader and enhance their knowledge of the recent development of prosthetic arms based on EEG signals. A more extensive image of various emergent topics/themes, experiments, and concepts will be offered. This paper is structured into six sections.
The following section provides a background of EEG signals and prosthetic limbs. The third section describes the methodology through which the review processes were conducted. The fourth section presents the SLR results, followed by the fifth section which reports on the results of the research questions as organized according to their sequences. Finally, the sixth section presents a discussion of the review and its conclusion.
4-9.
p. p
,s sm
ro «
o t
a f o
o
o c d e s a b
ci>
E E f
o y
d
s e
> tn
n e h
e rp
m o c
A
m a t a S
a m h a
d b A
Table 1 - Symbols Таблица 1 - Обозначения Табела 1 - Симболи
Symbol Meaning
EEG Electroencephalography
ECoGs electrocorticograms
MEGs magnetoencephalograms
fMRI functional magnetic resonance imaging
fNIRS Functional near-infrared spectroscopy
BCI Brain-Computer Interface
SVM Support Vector Method
FFT Fast Fourier transform
CSP Common spatial Pattern
LDA Linear discriminant analysis
PSD Power Spectral Density
LSTM Long*short term Memory
BPNN Back Propagation Neural Network
BMI Brain Machine Interface
PET Positron Emission Tomography
BFN Brain Functional Network
ANN Artificial Neural Network
ERP Event-related Potential
WT Wavelet transform
MLP Multi-layer Perceptron
K-NN K-Nearest Neighbor
NB Naive Bayes
FES Functional Electrical Simulation
Preliminaries and literature review
In the BCI system, EEG signals are most commonly used not only for prosthetics but also for any controllable devices such as robotics arms, Exoskeletons, Wheelchairs, drones, etc. Bridges et al. (Bridges et al, 2011) and his team provide an overview of human-machine interface architecture. The article contains good information about the control system. Yanagisawa et al. (Yanagisawa et al, 2011) shows a new method of controlling a prosthetic arm using ECoG signals. The system proved its effectiveness in decoding the hand movement of a patient who suffered from a stroke and used that signal to control a prosthetic hand. Another research implemented by Taha and his team (Beyrouthy et al, 2017) is about a system that extracts the EEG signals from the brain and uses them to control a smart 3D prosthetic arm. The system showed great results and presented a reliable alternative for an invasive system. Researchers in (Bright et al, 2016) succeeded in developing an EEG-based brain control system for the prosthetic arm using a BCI Neurosky mind wave set. The system reached an accuracy of 80 %. The team of researchers in (Elstob & Secco, 2016) controlled a 5 DOF robotic and prosthetic hand. They used two software frameworks. The method showed good results both technically and economically. A study of experimenting how transradial amputees could control grasp preshaping in a prosthetic arm using an EEG-based closed Loop BMI system is done by Agashe et al. (Agashe et al, 2016). The results showed that the EEG-based BMI system is a feasible solution. Healthy participants involved in a study implemented by Vidaurre et al. (Vidaurre et al, 2016) were able to use non-invasive Motor Imagery BCI to achieve linear control of an Upper Limb FES controlled Neuro Prosthesis. An embedded system was designed by (Rashid et al, 2018) in order to control the finger movement of the prosthetic arm using EEG signals. The signal classification accuracy of this study reached an acceptable percent of 79 %. Faiman et al. (Faiman et al, 2018) investigated whether spontaneous resting-state functional connectivity could predict the degree of motor adaptation of the right (dominant) upper limb reaching in response to a robot-mediated force field. Spontaneous neural activity was measured using resting-state electroencephalography (EEG) in healthy adults before a single session of motor adaptation. Noel & Snider (Noel & Snider, 2019) used the Deep Neural Network to control the prosthetic arm. The Neural Network was used to classify the signal to detect person's intention of extending the right index finger. The model achieved an accuracy of 63.3%. Gannouni et al. (Gannouni et al, 2020) presented a study that uses machine learning in order to anticipate the
4-9.
p. p
,s sm
ro «
o t
a f o
o
o c d e s a b
ci>
E E f
o y
d
s e
> tn
n e h
e rp
m o c
A
m a t a S
a m h a
d b A
<u
O >
co CM o CM
of
UJ
a:
ZD O o
_J
<
O 2:
X
o
LU
I—
>-
a: <
1—
< -j
CD
■O 2: x
LU I—
O
o >
0
movement of all five fingers. The proposed system achieved a signal classification accuracy of 81%. A 62% accuracy was achieved for an inexpensive mind-controlled prosthetic arm based on EEG signals. The system was implemented by (Chinta et al, 2020). Fuentes-Gonzalez et al. (Fuentes-Gonzalez et al, 2021) designed a prosthetic arm using blender software. The control of the prosthetic arm was done using EEG signals. The prosthetic arm was fitted to a 64-year-old man who had suffered from an electric shock. Ali et al. (Ali et al, 2021) build an inexpensive smart functional prosthesis arm in accordance with functional and non-functional requirements to meet users' goals and requirements. Setiawan et al. (Setiawan et al, 2021) designed a system to control a prosthetic hand using EEG signals to execute flexion and extension of fingers. Chaudhry et all (Chaudhry et al, 2022) discussed EEG control algorithms for prosthetic arms. They developed a cheap three-dimensional prosthetic arm; however, it was only a prototype and could not be applied for amputees. An EEG-based control system is not restricted to prosthetic arms only since exoskeleton and robotic arms can also be included in that area. Xu, et al. (Xu et al, 2011) developed a rehabilitation system for an upper limb stroke patient where the assistive device was based on motor imaginary EEG. The system proved feasibile and is fully capable of exploring patient's motor initiatives and guiding stroke patients to perform rehabilitation training effectively. The teams in (Ramos-Murguialday et al, 2012) developed a robotic hand exoskeleton based BCI to move fingers in flexion and extension movements. The results suggest that feedback contingency (proprioceptive stimulation paired with EEG SMR desynchronization) influences the motor network enhancing significantly SMR down-regulation. Formaggio et al. (Formaggio et al, 2013) present a study to perform a robot assisted task using a Bi-Manu track robot assisted arm trainer. Eight subjects participated in the study. The results suggest new perspectives for the assessment of patients with neurological disease. Tung et al. (Tung et al, 2013) performed a study of EEG to track the effect of a BCI based therapy on brain plasticity. The results suggest that motor recovery improvement comes from increasing activation in the lesion hemisphere during the BCI therapy. Krichner et al. (Krichner et al, 2014) carried out an experiment to prove that EEG and EMG can improve the adaptability of assistive devices in accordance with demands of users. The results show that both EEG and EMG predict a movement before it is physically executed. Witkowski et al. (Witkowski et al, 2014) introduced and tested a novel hybrid brain-neural computer interaction (BNCI) system fusing electroencephalography (EEG) and electrooculography (EOG) to enhance reliability and safety of continuous hand exoskeleton-driven
grasping motions. Looned et al. (Looned et al, 2014) introduced a wearable and portable system consisting of a novel lightweight Robotic Arm Orthosis (RAO), a Functional Electrical Stimulation (FES) system, and a simple wireless Brain-Computer Interface (BCI). This system is able to process electroencephalographic (EEG) signals and translate them into motions of the impaired arm. The researchers in (Hortal et al, 2015) created a system based on a hybrid upper limb exoskeleton for neurological rehabilitation. The movement was controlled by an EEG-based BMI. The system showed the combined use of a hybrid upper limb exoskeleton. Brauchle and his team in (Brauchle et al, 2015) tested the feasibility of a 3D robotic assistant to produce movements with a multi-joint exoskeleton during MI synchronization of sensorimotor oscillations in the B-band. The team of researchers in (Elnady et al, 2015) tested the feasibility of using FES. Robotic training devices facilitate motor task completion in post-stroke individuals. A robotic training device was operated to assist a pre-defined goal-directed motor task. The results showed that the participants' ability to use proprioception to control a motor output did not affect their ability to use the BCI-driven exoskeleton with FES. A novel system for the neuro-motor rehabilitation of upper limbs was presented in (Comani et al, 2015). The system was validated in three subacute post-stroke patients. The system permits synchronized cortical and kinematic measures by integrating high-resolution EEG, a passive robotic device and Virtual Reality. The brain functional re-organization was monitored in association with motor patterns replicating activities of daily living (ADL). The patients underwent 13 rehabilitation sessions. Soekadar et al. (Soekadar et al, 2015) introduced a novel brain/neural-computer interaction (BNCI) system that integrates electroencephalography (EEG) and electrooculography (EOG) to improve control of assistive robotics in daily life environments. In (Bhagat et al, 2016), researchers demonstrated the feasibility of detecting a motor intent from the brain activity of chronic stroke patients using an asynchronous electroencephalography (EEG)-based brain machine interface (BMI). Another investigation was implemented in (Tang et al, 2016). They investigated whether self-induced variations of the electroencephalogram (EEG) can be useful as control signals for a man-made upper-limb exoskeleton. A BMI based on event-related desynchronization/synchronization (ERD/ERS) is proposed. The study showed that the system is effective to control the upper limb exoskeleton. A rehabilitation approach based on BCI providing contingent sensory feedback of brain activity was presented by Alexander et al. (Frolov et al, 2017). The results proved that adding BCI control to exoskeleton assistive devices can improve the rehabilitation process for
-9.
p. p
,s sm
ro «
o t
a f o
o
o c d e s a b
ci>
E E f
o y
d
s e
> tn
n e h
e rp
m o c
A
m a t a S
a m h a
d b A
e
7
o >
,3 2 0 2
of IE IR U O C L A IC IN H C E T
Y R
A T
< -J
CD
■O z
X LU I—
O
o >
0
post stroke patients. The researchers in (Buerkle et al, 2021) presented a novel approach of how upper-limb movement intentions can be measured with a mobile electroencephalogram (EEG). The results suggested high detection accuracies and potential time gains of up to 513 ms to be achieved in a semi-online system. Thus, the time advantages included in a simulation demonstrated the potential to increase a system's reaction time and therefore improve the safety and the fluency of Human-Robot Collaboration. The EEG based control systems have application in the field of robotic arms. Steinisch et al. (Steinisch et al, 2013) proposed a system for neuro-motor rehabilitation of the upper limbs in stroke survivors. The system is composed of a passive robotic device (Trackhold) for kinematic tracking and gravity compensation, five dedicated virtual reality (VR) applications for training of distinct movement patterns, and high-resolution EEG for synchronous monitoring of cortical activity. Another study was conducted by Shedeed et al. (Shedeed et al, 2013). They presented a BMI system based on EEG signals to control three movements (open arm, close arm, and close hand). The signal classification accuracy reached up 91%. The researchers in (Bhattacharyya et al, 2014) proposed a novel approach toward EEG-driven position control of a robot arm by utilizing motor imagery. The results showed that the system is effective in the rehabilitation process. The team in (Xu et al, 2015) designed a BCI-based online robot control system. The study included 30 participants. The system proved its effectiveness and reliability. The total accuracy of the system reached up to 91 %. Meng et al. (Meng et al, 2016) designed a system to control a robotic arm to perform reach and grasp based on non-invasive BCI technology. Thirteen participants were included in this research. The system showed that the subjects can control the arm through modulation of their brain with the training. Karakoe et al. (Karakoc et al, 2017) designed a robotic arm using solidwork software. The arm can be controlled using brainwaves. The study was successful - however, although the arm was successfully controlled, it was not applicable (only a prototype). Bousseta et al. (Bousseta et al, 2018) proposed a novel BCI system that consists of controlling a robot arm based on the user's thoughts. Four subjects (1 female and 3 males) aged between 20 and 29 participated in the experiment. They were instructed to imagine the execution of movements of the right hand, the left hand, both right and left hands or the movement of the feet depending on the protocol established. A dynamical system conceptual and preliminary design together with system modeling are introduced in (Szabolcsi, 2019). Both dynamical system design and analysis tasks based on classical and modern control engineering approaches are handled in (Szabolcsi, 2020) using MATLAB.
Research methodology used in the study
An extensive literature search was carried out. The search covered studies between 2011 and 2018. Only full-text papers published in English were considered. In this research, the combination of keywords (BCI or Brain-Computer Interface or EEG or Electroencephalography) and (Prosthetic Limb, Prosthetic Arms or Robot) and ( Control Method ) is used. Figure 2 shows the process of how the chosen papers were selected in this research.
Figure 2 -Research method Рис. 2 - Метод исследования Слика 2 - Метод истражива^а
From the figure above, the number of selected papers was decreased due to the application of several filters depending on the type of the papers, e.g. full text or not. Only English language papers were chosen, also depending on the type of input signals, i.e. EEG signals. Only the articles that dealt with an upper limb (Arm, Hand) were included.
Figure 3 shows the distributions of the articles regarding the EEG-Based Control method for artificial upper limbs (as prosthetic arms or assistive devices or robotic arms). Years from 2013 till mid of 2016 witnessed a rise in research interest in this area.
Figure 3 - Publication distribution Рис. 3 - Распространение публикаций Слика 3 - Расподела публикац^а
Background
Prosthetic limbs
In this section, the focus of the prosthetic limb will be on the prosthetic arms type. The prosthetic arm consists of several components that work together to make the arm useful.
• Limb. The limbs of a prosthetic arm are formed out of lightweight, yet durable materials.
• Socket. The socket connects the prosthesis to the residual limb to ensure that it fits securely. A poor fit can cause considerable discomfort and reduce the function of the prosthetic arm. To circumvent this problem, prosthetics are made using a personalized mold to fit the exact shape of the residual limb.
• Suspension system. The suspension system is the component that secures the prosthetic to the residual limb. There are different suspension systems, including a harness, an elastic sleeve, a suction socket, or a self-suspending socket.
• Control system. While the brain controls a natural limb and nerve impulses, a prosthetic arm cannot be controlled the same way. Control systems are myoelectric, body-powered, or motor-controlled.
Electroencephalography (EEG)
Electroencephalography (EEG) is the most common brain signal that has been utilized in brain-machine interface applications. This popularity
is due to several facts: EEG signals are non-invasive, low cost, compatible, portable and have a high temporal resolution in comparison with other brainwave measurements such as electrocorticograms (ECoGs), magnetoencephalograms (MEGs), functional magnetic resonance imaging (fMRI) and near-infrared spectroscopy (fNIRS).
Electroencephalography can be defined as the measurement of the electric brain activity caused by currents induced by neurons within the brain (Murphy et al, 2017). The EEG signal can be detected in a non-invasive way by placing the electrode on the scalp. This justifies why the EEG measurement is the most widespread brain activity measurement technique. In addition, it is comparatively affordable and provides a high temporal resolution (about 1 ms). However, it has a weak signal and is prone to several artifacts and relatively poor spatial resolution.
In EEG measurement, detected waveforms reveal cortical electrical activity. The signal intensity of EEG activity is often quite small and measured in the microvolt (^V) range (Ubeyli, 2009; Acharya et al, 2019). The main EEG rhythms are classified based on the frequency range as alpha (a), beta (P), delta (5), theta (6) and gamma as shown in Table 1.
Table 1 - EEG frequencies Таблица 1 - Частоты ЭЭГ Табела 1 - ЕЕГ фреквенцще
EEG Frequencies Description
Delta 6 0.5-4 Appear in infant and deep sleep
Theta 0 4-8 Appear in partial and temporal areas in children
Alpha a 8-13 Occur in awaken adults in the parietal and frontal region of the scalp
Beta B 13-30 These waves are related to the movements and commonly appear in the frontal and central lope. The decreasing of the Beta rhythm indicates a movement, preparation of movements, planning a move, or imagining a movement. This decrease is most dominant in the contralateral motor cortex. This attenuation in Beta waves is called event-related desynchronization. The rhythms increase after the movement and are known as event-related synchronization.
Gamma у >30 These are higher rhythms that have frequencies of more than 30 Hz.
Results
During the systematic review, 39 articles were chosen for principal studies and all of them were using EEG as an input signal.
However, the output was either a prosthetic limb, an exoskeleton device, or a robotic arm.
Figure 4 shows the number of studies that dealt with each one considered in this paper.
Figure 4 - Articles dealing with EEG application
Рис. 4 - Статьи о применении ЭЭГ Слика 4 - Чланци о примени ЕЕГ апликаци^е
The systematic review results in 39 papers, chosen as principal studies and published in the field of EEG-based control of prosthetic arms, exoskeleton, and robotic arms, are shown in Tables 2,3, and 4, respectively.
Table 2 - Prosthetic arm Таблица 2 - Протез руки Табела 2 - Простетичка рука
■чг сЬ
Reference No. EEG Extraction Method Controller Used No. of movements year
(Bridges et al, 2011) Not specified Not specified Grasp 2011
(Yanagisawa et al, 2011) Feature extraction: bandpass filter (Fast Fourier Transform) Classifier : Support Vector Machine SVM Not Specified Grasp, extension of the second and third finger ( scissor shape) 2011
(Beyrouthy et al, 2017) Not specified Raspberry Pi + Arduino Close and open hand 2016
(Bright et al, 2016) Not specified Arduino UNO Flexion, extension, pinch 2016
(Elstob & Secco, 2016) CSP spatial filter for extraction Linear Discriminant analysis LDA for classifier Arduino UNO Open, close 2016
(Agashe et al, 2016) High-pass and low-pass second-order Butterworth filters Built-in controller Grasp 2016
(Vidaurre et al, 2016) Band Pass filter for Extraction, LDA as a classifier Not specified Control, right hand or left hand 2016
(Rashid et al, 2018) Data Processing: low and high pass filter Feature extraction : calculation o band power from PSD Classification: Logistic regression classifier network Arduino UNO Finger flexion and extension 2018
(Faiman et al, 2018) Data was filtered with Bandpass and Notch filter Data extraction using Fast Fourier Transform Not specified Reaching 2018
Reference No. EEG Extraction Method Controller Used No. of movements year
(Noel & Snider, 2019) Extraction and analysis : Power spectral density Classification : Support vector machine Not specified Flexion and extension of fingers 2019
(Gannouni et al, 2020) Extracting CSP Classifying: LDA Not specified Finger movements 2020
(Chinta et al, 2020) Classification : LSTM ( Long-Short Term Memory Model Not specified Upward and downward arm movement 2020
(Fuentes-Gonzalez et al, 2021) Not specified Arduino UNO Open and close hand 2020
(Ali et al, 2021) Not specified Not specified Arm movement, fingers open and close 2021
(Setiawan et al, 2021) RC Filter OP AMP for Signal Extraction Arduino UNO Flexion and extension of fingers 2021
(Chaudhry et al, 2022) Extraction : FFT Classification: SVM Arduino UNO Fingers flexion and extension 2022
Table 3 - Exoskeleton Ta6nuu,a 3 - Экзоскепет Taöena 3 - Егзоскепет
Reference No. EEG Extraction Method Controller Used No. of movements year
(Xu et al, 2011) Extraction: WT Classification: LDA Not specified Right and left arm 2011
(Ramos-Murguialday et al, 2012) Spatial filter Not specified Fingers flexion and extension 2012
(Formaggio et al, 2013) Sampling : Band Pass Filter and FFT Not specified Hand movement 2013
(Tung et al, 2013) Not specified Upper arm movement 2013
OD
Reference No. EEG Extraction Method Controller Used No. of movements year
(Krichner et al, 2014) Sampling: FFT Band Pass Filter and Spatial filter Classification: SVM Not specified Upper arm movement 2014
(Witkowski et al, 2014) Sampling Band Pass Filter Preprocessing: Laplacian filter Not specified Hand movement 2014
(Looned et al, 2014) Extraction: Spatial Filter Classifier: Linear Classifier Not specified Arm movement and grasp 2014
(Hortal et al, 2015) Sampling : Notch Filter Extracting : Band Pass Filter Classifier: SVM Not specified Elbow flexion and extension 2015
(Brauchle et al, 2015) Digitization: High Pass Filter Classification : Linear Classification Not specified Arm reaching movement 2015
(Elnady et al, 2015) Extract: common spatial pattern Algorithm Classifier: Linear Discriminant Analysis LDA Not specified Elbow flexion , extension hand open and close 2015
(Comani et al, 2015) Samplings: notch and Bandpass filter Not specified Upper arm movement 2015
(Soekadar et al, 2015) Sampling band pass filter preprocessing: Laplacian +-filter Not specified Upper arm movement 2015
(Bhagat et al, 2016) Sampling : High then Low Pass Filter Classification: SVM Not specified Elbow flexion extension 2016
(Tang et al, 2016) Sampling : Notch and Band Pass Filter Classifier: LDA, SVM, BPNN Not specified Right- and left-hand movement both feet movement 2016
(Frolov et al, 2017) Extraction: Band Pass Filter classify: Bayesian classifier Not specified Hand open and close 2017
(Buerkle et al, 2021) Extraction :FFT Classification: SVM Not specified Right and left hand 2021
CD d.
<0 E
CO
ü o
te CO
4—
o ö
o o
T3
cu
CO CO JH
d) LU LU
4—
o
T3
cu >
cu sz CD Ü. E o
o <
E
CO
-I—'
CO W c CO
E
T3
JH <
Table 4 - Robotic arm Таблица 4 - Роботизированная (бионическая) рука Табела 4 - Роботичка рука
Reference No. EEG Extractio n Method Controller Used No. of movements year
1 (Steinisch et al, 2013) Sampling : Notch and Bandpass filter Not specified Arm movement 2013
2 (Shedeed et al, 2013) Extraction: WT, FFT, PCA Classifier: SVM Not specified Close and open arm Close hand 2014
3 (Bhattacharyya et al, 2014) Extraction: FFT Classifier: SVM Not specified Arm movement, left, right and forward 2014
4 (Xu et al, 2015) Extraction: WT Classifier: LDA Not specified Arm move upward and downward 2015
5 (Meng et al, 2016) Not specified Not specified Arm movement: left, right, up, down 2016
6 (Karakoc et al, 2017) Not specified Arduino Open and close hand 2017
7 (Bousseta et al, 2018) Extraction: FFT Classifier: SVM Not specified Arm base right, left Elbow up and down 2018
Figure 5 - EEG signal control in the prosthetic arm research studies from Table 2 Рис. 5 - Управление сигналом ЭЭГ в исследованииях о протезировании руки из
Таблицы 2
Слика 5 - Управ^аше ЕЕГ сигналом у истраживачким студи^ама ще се баве простетичком руком (табела 2)
Figure 6 - EEG signal control in the exoskeleton research studies from Table 3 Рис. 6 - Управление сигналом ЭЭГ в исследованиях экзоскелета из Таблицы 3 Слика 6 -Управ^аше ЕЕГ сигналом у истраживачким студи^ама ще се баве
егзоскелетом (табела 3)
чг сЬ !±
<л
Е со
"го 'о
Ъ го
ч—
о о
о о
тз ф
(Л
го
.Q
6 ш ш
ч—
о тз
ф >
ф
.с ф
Е о
о <
Е го
-ь-'
го ю
с го Е
тз
.Q <
Figure 7 - EEG signal control in the robotic arm research studies from Table 4 Рис. 7 -Управление сигналом ЭЭГ в исследованиях роботизированной руки из
Таблицы 4
Слика 7-Управ^аше ЕЕГ сигналом у истраживачким студи^ама ще се баве роботичком руком ( табела 4)
EEG signal types
EEG is nowadays considered a successful non-invasive realistic and practical Brain-Machine Interface BMI Technique. This is due to the fact that other techniques are considered high cost, e.g. magnetoencephalography (MEG) and positron emission tomography (PET).
Three key elements characterise the EEG-based prosthetic arm: the type of EEG signals, which part of the prosthetic arm is under control, and how to translate the EEG signal to a control command to manage the prosthesis. Figure 8 shows these key elements.
Figure 8 - Research elements Рис. 8 - Элементы исследования Слика 8 - Елементи истражива^а
Endogenous and exogenous EEG signals
Depending on the movement type, the prosthetic arm can be managed by utilizing exogenous or endogenous EEG signals.
Table 5 shows the differences between these two types.
Table 5 - EEG signal types Таблица 5 - Типы сигналов ЭЭГ Табела 5 - Типови ЕЕГ сигнала
О)
p.
EEG Sign al
Description
Advantages
Disadv antage s
Types
Description
s u o n
e g
o x E
Generated by applying external stimuli like auditory or virtual clue
Minimum training for participants
1-
Require a lot of focus. 2-
Particip ants can be really exhaust ed from strong stimuli.
Steady
State
Visually
Evoked
Potential
(SSVEP)
The reaction to the stimuli is at different frequencies. If the participant looks at a flashing light with specific frequency, the EEG signal from the visual cortex would be at the same frequency
P-300 Based Interface
The same as SSVEP but the data transfer rate is lower
s u o n
e g
o d n E
Sensorimot or Rhythms (SMR)
Does not need external stimuli
Participants 1- Need
with more
neurological training
problems 2- Data
can control transfer
prosthesis rate is
automatically lower
Endure two kinds of amplitude modulations known as event-related desynchronization(E RD) and event-related
synchronization (ERS)
Slow Cortical Potentials (SCP)
Slow event-related direct-current shifts of the
electroencephalogr am. Slow cortical potential shifts in the electrical negative direction reflect the depolarization of large cortical cell assemblies, reducing their excitation threshold.
<u
O >
co CM o CM
of
UJ
a:
ZD O o
_J
<
O 2:
X
o
LU
I—
>-
a: <
1—
< -j
CD
■O 2: x
LU I—
O
o >
0
Prosthetic arm parts
The control of a prosthetic arm has different paradigms represented by whether the control includes only fingers, hand, elbow, or a full arm. Every part requires different types of signals. Besides, the time of training is dependent on the parts.
A - Fingers
Both human hands have four fingers and a thumb each. The fingers have two main moves: flexion and extension. The flex movement is mainly for grasping, while extension is for reaching things. Due to flexion, several modes can be made, i.e. bending, making a fist, gripping, grasping and folding fingers. On the other hand, the extension of fingers includes the following modes: pointing, stretching out, and spreading out.
The thumb is responsible for 50% of the hand function. The thumb has two joints at the end and middle which flex and extend, just like the fingers. The next joint down, however, is highly specialized and allows several unique movements not possible in the fingers. These are the following motions: circumduction, abduction, adduction, and retropulsion.
There are several studies regarding finger movements using EEG signals implemented over the last few years. Paek et al. (Paek et al, 2014) investigated how the finger tapping movement ca be decoded from the scalp EEG signals. The study shows that finger kinematics can be inferred from delta band filtered fluctuation of the amplitude of EEG signals across the scalp using linear decoders with memory. Ketenci & Kayikcioglu (Ketenci & Kayikcioglu, 2019) studied the effect of theta brainwaves on movement detection. Four right-handed participants performed extensions with their fingers using EEG. They proved that theta signals participate in movement execution. Mohamed & Aharonson (Mohamed & Aharonson, 2021) studied the movement of wrist and fingers together ( i.e. left finger and wrist or right finger and wrist). The results suggest that a combination of classifiers and features from different frequency bands could improve BCI performance to enable more dexterous control of a bionic hand. Rashid and his team (Rashid et al, 2018) designed a system that can be used to control the fingers of a prosthetic limbs using EEG signals. For this system, a two-staged classifier was used. The classifier was able to distinguish between three finger movements, the thumb, and the fist with an accuracy of 70%. A novel method of classification of four finger movements (thumb movement, index finger movement, middle and index finger combined movement, and fist movement) of the right hand on the basis of EEG (Electroencephalogram) data of the movements was
presented by Javed's team (Javed et al, 2017). The PSD and Linear Regression models were used to classify the signals. These classifiers had an accuracy of 65%. Liao et al. (Liao et al, 2014) investigated the discrimination of individual fingers from one hand using non-invasive EEG. The experimental results demonstrated that a movement-related spectral structure could be decoupled from EEG power spectrum density data using the Principal Component Analysis with an accuracy achieved of 77.11% .
B - Hand
The hand movements include the fingers and the wrist movements. The wrist joint flexes and extends, but also deviates radially and ulnarly (moves from side to side). The word radially means "toward the thumb side". The term ulnarly means "toward the pinky side". One might use this motions when swinging a hammer. A study of implementing an algorithm for wrist movement detection was implemented by Ghani et al. (Ghani et al, 2013) - the movements of the hand were flexion and extension. The accuracy achieved by this algorithm was up to 91.93 % using discrete cosine transformation of energy and entropy. Huong et al. (Huong et al, 2018) used Event-Related Potential (ERP) components of P300, and the advanced features combined in Artificial Neural Network (ANN) were used to classify the electroencephalogram (EEG) signals associated with the left and right-hand movements. The results of classification are quite good and promising for the application in a BCI context to mentally control a computer or a machine.
Ting Li et al. (Li et al, 2018) proposed a model of voluntary hand movement decoding based on an HLM. The original intention of this design was to identify a computing architecture that could contain and describe complex data, such as that of an EEG BFN. Ramalingam et al. (Ramalingam et al, 2016) use machine learning algorithms to extract and classify signals from the brain to execute the motion of fingers and wrist rotation. four classes of right-hand movements were considered. The descriptive statistical features were computed from EEG signals. Feature selection was carried out to reduce the classifier complexity and an accuracy of 80.55% was achieved using the C4.5 decision tree algorithm.
C - Elbow
The elbow is one of the most important parts of the arm because it allows the hand to move in almost any position so that various activities can be done. The movements that can be done with the elbow are flexion and extension. Ji-hoon et al. presented a study of the classification of
d9>
e
lo
> co
2
O
2
of IE IR
ZD
o o
_J
<
o
IN
X
o
E
I—
>-
R
< I—
< -J
CD
■O z
X LU I—
O
o >
0
forearm movements according to elaborated rotation angles using electroencephalogram (EEG) signals (Jeong et al, 2020). They used the Hierarchical Low Convolutional Neural Network (HF-CNN) model for robust classification. The experimental results demonstrate the possibility of decoding complex kinematics information using EEG signals. Ghani et al. (Ghani et al, 2012) used EEG to analyze brain activity in order to translate human elbow movements to the movements of an artificial actuator. The work achieved 73% accuracy in the classification of the elbow movements using EEG. Faizal et al. designed an orthosis control system as a rehabilitation device by using a classification method with EEG and EMG signals, so that subjects who use this tool can carry out rehabilitation in upper arm movements, especially in the elbow joint. The system reached an accuracy of 85.2% with three movements: relax, flexion, and extension (Ferdiansyah et al, 2020).
Translating the EEG signal to the control command
EEG signal decoding has several stages in order to reach the desired output. These common stages are preprocessing, feature extraction, and classification. At each stage, an algorithm is applied.
A - Preprocessing
EEG records electrical potentials generated by nerve cells. Electrodes are placed on the scalp and recorded by amplification. By this procedure, the obtained data shows a continuous graphic with the spatial distribution of the voltage changes over time. In order to translate brain activity into commands, there are three steps to be applied. First, the brain activity is recorded with an acquisition device. Then, artifacts are unwanted faulty parts of signals that are removed from signals (Gupta & Singh, 1996).
B - Feature extraction
Relevant features are extracted by methods such as Fast Fourier Transform (FFT), Wavelet Transform (WT), and Eigenvectors (Zhang et al, 2008). There are different methods for feature extraction of the signal such as FFT, WT, Eigenvectors, Time-frequency Distributions, and Autoregressive Method. Table 6 shows every method mentioned earlier with their advantages and disadvantages.
Table 6 - Feature extraction techniques Таблица 6 - Методы выявления признаков Табела 6 -Технике издва]а^а карактеристика
■чг ст>
Name Advantages Disadvantages
FFT 1- Good tool for stationary signal processing 2- It is more appropriate for narrowband signals, such as sine wave 3- It has an enhanced speed over virtually all other available methods in real-time applications 1- Not good with nonstationary signals like EEG. 2- Suffers from large noise sensitivity
WT 1- It is better suited for the analysis of sudden and transient signal changes Needs selecting a proper mother wavelet
Eigen vector Provides suitable resolution to evaluate the sinusoid from the data Lowest eigenvalue may generate false zeros when Pisarenko's method is employed
Time frequency distribution 1- It gives the feasibility of examining great continuous segments of the EEG signal 2- TFD only analyzes clean signals for good results 1- Time-frequency methods are oriented to deal with the concept of stationary; as a result, a windowing process is needed in the preprocessing module 2- It is quite slow (because of the gradient ascent computation) 3- Extracted features can be dependent on each other
Autoregressive 1- AR limits the loss of spectral problems and yields improved frequency resolution 2- Gives good frequency resolution 3- Spectral analysis based on the AR model is particularly advantageous when short data segments are analyzed, since the frequency resolution of an analytically derived AR spectrum is infinite and does not depend on the length of analyzed data 1- The model order in AR spectral estimation is difficult to select 2- The AR method will give poor spectral estimation once the estimated model is not appropriate, and the models' orders are incorrectly selected 3- It is readily susceptible to heavy biases and even large variability
OD
C - Classification
A classifier utilizes values for independent variables (features) as inputs to predict the corresponding class to which an independent variable belongs. A classifier has a number of parameters that require training from a training dataset (Wen et al, 2021). A trained classifier will model the association between classes and corresponding features and is capable of identifying new instances in an unseen testing dataset. Several techniques of classification are explained in Table 7.
Table 7 - Classification techniques Таблица 1 - Методы классификации Табела 1 -Технике класификаци]е
No Meth od Description
1 SVM The SVM is a supervised learning algorithm that uses a kernel trick to transform input data into higher dimensional space, after which it segregates the data via a hyper-plan with maximal margins. Due to its ability to manage large datasets, the algorithm is widely used for binary classification problems in machine learning.
2 MLP The MLP is a non-linear neural network based method comprising three sequential layers: input, hidden and output, respectively, where the hidden layer transmits input data to the output layer. However, the MLP model can cause over-fitting due to insufficient or excessive numbers of neurons.
3 NB The NB classifier provides simple and efficient probabilistic classification based on Bayes' theorem, which posits that extracted features are not dependent. The NB model uses (i) a maximum probability algorithm to determine the class of earlier probabilities, and (ii) a feature's probability distribution from a training dataset. Results are then employed with a maximized posteriori decision tree to find the specific class label for a new test instance.
4 K-NN The k-nearest neighbor is a supervised learning algorithm that identifies a testing sample's class according to the majority class of k-nearest training samples; i.e., a class label is allocated to a new instance of the most common class amongst KNN in the "feature" space. In this study, the k value was set to three.
5 K- fold cross valid ation All classification models in the present work were trained and tested with EEG data and then confirmed using k-fold cross validation, which is a commonly used technique that compares (i) performances of two classification algorithms, or (ii) evaluates the performance of a single classifier on a given dataset (Wong, 2015). It has the advantage of using all instances in a dataset for either training or testing, where each instance is employed for validation exactly once.
6 LDA It consists of the statistical properties of the data calculated for each class. For a single input variable (x), these are the mean and the variance of the variable for each class. For multiple variables, these are the same properties calculated over the multivariate Gaussian, namely the means and the covariance matrix. These statistical properties are estimated from the data and plug into the LDA equation to make predictions. These are the model values that would be saved to file for the model.
Conclusion
The study of EEG-based prosthetic arms includes a wide range of fields to be familiar with, such as anatomy, signal processing, control methods, and design. All these fields represent sciences in themselves. Although researchers have done good work regarding prosthetic arm control, more needs to be done, such as increasing the accuracy in EEG signal extracting and faster control.
The use of automatic prosthetic arms is already an accepted method. It is considered one of the methods that aim to be applicable for lower limbs, wheelchairs, cars, and even drones.
In future research based on a Ph.D. thesis, the five extraction methods mentioned in Table 6 and the six classification methods mentioned in Table 7 will be used to control the full motion of the arm moving an object from one place to another using LabView or MATLAB simulation. Subsequently, a method with high accuracy will be applied for an actual prosthetic arm using Raspberry pi 4 controller.
References
Abdulrahman Satam, I. 2021. Review Studying of the Latest Development of Prosthetic Limbs Technologies. International Journal of Scientific & Engineering Research, 12(12), pp.721-731 [online]. Available at: https://www.ijser.org/research-paper-publishing-december-2021.aspx [Accessed: 20 November 2022].
Acharya, U.R., Hagiwara, Y., Nitin, Deshpande, S.N., Suren, Koh, J.E.W., Oh, S.L., Arunkumar, N., Ciaccio, E.J. & Lim, C.M. 2019. Characterization of focal EEG signals: A review. Future Generation Computer Systems, 91, pp.290-299. Available at: https://doi.org/10.1016/j.future.2018.08.044.
Agashe, H.A., Paek, A.Y. & Contreras-Vidal, J.L. 2016. Chapter 4 -Multisession, noninvasive closed-loop neuroprosthetic control of grasping by upper limb amputees. Progress in Brain Research, 228, pp.107-128. Available at: https://doi.org/10.1016/bs.pbr.2016.04.016.
Ali, H.A., Goga, N., Vasilateanu, A., Ali, L.A., Abd-Almuhsen, G.S. & Naji, H.K. 2021. A Quantitative Research to Determine User's Requirements for the Mind-Controlled Prosthesis Arm Intelligent System. In: 2021 13th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), Pitesti, Romania, pp.1-8, July 01-03. Available at:
https://doi.org/10.1109/ECAI52376.2021.9515168.
Beyrouthy, T., Kork, S.A., Korbane, J.A. & Abouelela, M. 2017. EEG Mind Controlled Smart Prosthetic Arm - A Comprehensive Study. Advances in Science, Technology and Engineering Systems Journal, 2(3), pp.891-899. Available at: https://doi.org/10.25046/aj0203111.
CT> !±
oi
E
CO «
o
t ro
M—
O O
o o
T3 <1J </) ro
.Q
CD
LLI LLI
M—
O T3
<u >
tn a <u .c <D
E o
O <
E ro ro OT
ro E ro
T3
.Q <
<u
O >
co CM o CM
of
UJ
a:
ZD O o
_J
<
O 2:
X
o
LU
I—
>-
a: <
1—
< -j
CD
■O 2: x
LU I—
O
o >
0
Bhagat, N.A., Venkatakrishnan, A., Abibullaev, B., Artz, E.J., Yozbatiran, N., Blank, A.A., French, J., Karmonik, C., Grossman, R.G., O'Malley, M.K., Francisco, G.E. & Contreras-Vidal, J.L. 2016. Design and Optimization of an EEG-Based Brain Machine Interface (BMI) to an Upper-Limb Exoskeleton for Stroke Survivors. Frontiers in Neuroscience, 10(March), art.number:122, pp.1-17. Available at: https://doi.org/10.3389/fnins.2016.00122.
Bhattacharyya, S., Konar, A. & Tibarewala, D.N. 2014. Motor imagery, P300 and error-related EEG-based robot arm movement control for rehabilitation purpose. Medical & Biological Engineering & Computing, 52, pp.1007-1017. Available at: https://doi.org/10.1007/s11517-014-1204-4.
Bousseta, R., Ouakouak, I.El, Gharbi, M. & Regragui, F. 2018. EEG Based Brain Computer Interface for Controlling a Robot Arm Movement Through Thought. IRBM, 39(2), pp.129-135. Available at: https://doi.org/10.1016/j.irbm.2018.02.001.
Brauchle, D., Vukelic, M., Bauer, R. & Gharabaghi, A. 2015. Brain state-dependent robotic reaching movement with a multi-joint arm exoskeleton: Combining brain-machine interfacing and robotic rehabilitation. Frontiers in Human Neuroscience, 9(October), art.number:564, pp.1-13. Available at: https://doi.org/10.3389/fnhum.2015.00564.
Bridges, M.M., Para, M.P. & Mashner, M.J. 2011. Control system architecture for the modular prosthetic limb. Johns Hopkins Apl Technical Digest, 30(3), pp.217-222 [online]. Available at:
https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=aaefaff90b3f4 102edb9c8de565a08ff1641deb5 [Accessed: 20 November 2022].
Bright, D., Nair, A., Salvekar, D. & Bhisikar, S. 2016. EEG-based brain controlled prosthetic arm. In: 2016 Conference on Advances in Signal Processing (CASP), Pune, India, pp.479-483, June 09-11. Available at: https://doi.org/10.1109/CASP.2016.7746219.
Buerkle, A., Eaton, W., Lohse, N., Bamber, T. & Ferreira, P. 2021. EEG based arm movement intention recognition towards enhanced safety in symbiotic Human-Robot Collaboration. Robotics and Computer-Integrated Manufacturing, 70(August), art.number:102137. Available at:
https://doi.org/10.1016Zj.rcim.2021.102137.
Chaudhry, A., Khan, U., Palla, M.R. & Singh, S.B. 2022. A Prosthetic Arm Based on Electroencephalography by Signal Acquisition and Processing on MATLAB. IJRESM - International Journal of Research in Engineering, Science and Management, 5(1), pp.119-124 [online]. Available at: https://journals.resaim.com/ijresm/article/view/1691 [Accessed: 20 November 2022].
Chinta, A., Mathur, M. & Lal, A.M. 2020. Mind Wave Controlled Prosthetic ARM Without using Brain Implants. International Journal of Recent Technology and Engineering (IJRTE), 8(5), pp.1615-1618. Available at: https://doi.org/10.35940/ijrte.e4801.018520.
Comani, S., Velluto, L., Schinaia, L., Cerroni, G., Serio, A., Buzzelli, S., Sorbi, S. & Guarnieri, B. 2015. Monitoring Neuro-Motor Recovery From Stroke With High-Resolution EEG, Robotics and Virtual Reality: A Proof of Concept. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 23(6), pp.11061116. Available at: https://doi.org/10.1109/TNSRE.2015.2425474.
Elnady, A.M., Zhang, X., Xiao, Z.G., Yong, X., Randhawa, B.K., Boyd, L. & Menon, C. 2015. A single-session preliminary evaluation of an affordable BCI-controlled arm exoskeleton and motor-proprioception platform. Frontiers in Human Neuroscience, 9(March), art.number:168, pp.1-14. Available at: https://doi.org/10.3389/fnhum.2015.00168.
Elstob, D. & Secco, E.L. 2016. A Low Cost Eeg Based BCI Prosthetic Using Motor Imagery. International Journal of Information Technology Convergence and Services (IJITCS), 6(1), pp.23-36. Available at: https://doi.org/10.5121/ijitcs.2016.6103.
Faiman, I., Pizzamiglio, S. & Turner, D.L. 2018. Resting-state functional connectivity predicts the ability to adapt arm reaching in a robot-mediated force field. Neuroimage, 174, pp.494-503. Available at: https://doi.org/10.1016/j.neuroimage.2018.03.054.
Ferdiansyah, F.A., Prajitno, P. & Wijaya, S.K. 2020. EEG-EMG based biorobotics elbow orthotics control. Journal of Physics: Conference Series, 1528(art.number:012033), pp.1-6. Available at: https://doi.org/10.1088/1742-6596/1528/1/012033.
Formaggio, E., Storti, S.F., Galazzo, I.B., Gandolfi, M., Geroin, C., Smania, N., Spezia, L., Waldner, A., Fiaschi, A. & Manganotti, P. 2013. Modulation of event-related desynchronization in robot-assisted hand performance: brain oscillatory changes in active, passive and imagined movements. Journal of NeuroEngineering and Rehabilitation, 10(art.number:24). Available at: https://doi.org/10.1186/1743-0003-10-24.
Frolov, A.A., Mokienko, O., Lyukmanov, R., Biryukova, E., Kotov, S., Turbina, L., Nadareyshvily, G. & Bushkova, Y. 2017. Post-stroke Rehabilitation Training with a Motor-Imagery-Based Brain-Computer Interface (BCI)-Controlled Hand Exoskeleton: A Randomized Controlled Multicenter Trial. Frontiers in Neuroscience, 11 (July), art.number:400, pp.1-11. Available at: https://doi.org/10.3389/fnins.2017.00400.
Fuentes-Gonzalez, J., Infante-Alarcon, A., Asanza, V. & Loayza, F.R. 2021. A 3D-Printed EEG based Prosthetic Arm. In: 2020 IEEE International Conference on E-health Networking, Application & Services (HEALTHCOM), Shenzhen, China, pp.1-5, March, 01-02. Available at:
https://doi.org/10.1109/HEALTHC0M49281.2021.9399035.
Gannouni, S., Belwafi, K., Aboalsamh, H., Alebdi, B., Almassad, Y., AlSamhan, Z. & Alobaedallah, H. 2020. EEG-Based BCI System to Control Prosthesis's Finger Movements (PREPRINT). Reacerch Square. Available at: https://doi.org/10.21203/rs.3Ts-49613/v1.
CT> !±
oi
E
CO «
o
t CO
M—
O
o
o o
T3
cd tn CO .Q
CD
LU LU
M—
o
T3
</) cd
tn c CD -C CD
E o
o <
E
CO CO OT
CO
E
CO
T3
.Q <
<u
O >
co CM
0
01
c¿
UJ
a:
ZD O O
_J
<
O 2:
X
o
LU
I—
>-
a: <
1—
< -j
CD
■O 2: x
LU I—
O
O >
O
Ghani, F., Gaur, B., Varshney, S., Farooq, O. & Khan, Y.U. 2013. Detection of wrist movement using EEG signal for brain machine interface. In: 2013 International Conference on Technology, Informatics, Management, Engineering and Environment, Bandung, Indonesia, pp.5-8, June 23-26. Available at: https://doi.org/10.1109/TIME-E.2013.6611954.
Ghani, F., Jilani, M., Raghav, M., Farooq, O. & Khan, Y.U. 2012. Elbow movement detection using brain computer interface. In: 2012 8th International Conference on Computing Technology and Information Management (NCM and ICNIT), Seoul, Korea (South), pp.736-740, April 24-26 [online]. Available at: https://ieeexplore.ieee.org/document/6268597 [Accessed: 20 November 2022].
Gupta, S. & Singh, H. 1996. Preprocessing EEG signals for direct humansystem interface. Proceedings IEEE International Joint Symposia on Intelligence and System, 1996, pp.32-37. Available at:
https://doi.org/10.1109/ijsis.1996.565048.
Hortal, E., Planelles, D., Resquin, F., Climent, J.M., Azorín, J.M. & Pons, J.L. 2015. Using a brain-machine interface to control a hybrid upper limb exoskeleton during rehabilitation of patients with neurological conditions. Journal of NeuroEngineering and Rehabilitation, 12(art.number:92). Available at: https://doi.org/10.1186/s12984-015-0082-9.
Huong, N.T.M., Linh, H.Q. & Khai, L.Q. 2018. Classification of left/right hand movement EEG signals using event related potentials and advanced features. In: Vo Van, T., Nguyen Le, T., Nguyen Duc, T. (Eds.) 6th International Conference on the Development of Biomedical Engineering in Vietnam (BME6). BME 2017. IFMBE Proceedings, 63, pp.209-215. Available at: https://doi.org/10.1007/978-981-10-4361-1_35.
Javed, A., Tiwana, M.I., Tiwana, M.I., Rashid, N., Iqbal, J. & Khan, U.S. 2017. Recognition of finger movements using EEG signals for control of upper limb prosthesis using logistic regression. Biomedical Research, 28(17), pp.7361-7369 [online]. Available at: https://www.biomedres.info/biomedical-research/recognition-of-finger-movements-using-eeg-signals-for-control-of-upper-limb-prosthesis-using-logistic-regression-8334.html [Accessed: 20 November 2022].
Jeong, J-H., Lee, B-H., Lee, D-H., Yun, Y-D. & Lee, S-W. 2020. EEG Classification of Forearm Movement Imagery Using a Hierarchical Flow Convolutional Neural Network. IEEE Access, 8, pp.66941-66950. Available at: https://doi.org/10.1109/ACCESS.2020.2983182.
Karakoc, A., Dogan, D. & Akinci, T.C. 2017. Robotic Arm Control Using The Brain Waves. The Journal of Cognitive Systems, 2(2), pp.51-54 [online]. Available at: https://dergipark.org.tr/en/pub/jcs/issue/42261/530481 [Accessed: 20 November 2022].
Ketenci, S. & Kayikcioglu, T. 2019. Investigation of Theta Rhythm Effect in Detection of Finger Movement. Journal of Experimental Neuroscience, 13, pp.15. Available at: https://doi.org/10.1177/1179069519828737.
Krichner, E.A., Tabie, M. & Seeland, A. 2014. Multimodal Movement Prediction - Towards an Individual Assistance of Patients. PLOS ONE, 9(1), art.ID:e85060, pp.1-9. Available at: https://doi.org/10.1371/journal.pone.0085060.
Li, T., Xue, T., Wang, B. & Zhang, J. 2018. Decoding Voluntary Movement of Single Hand Based on Analysis of Brain Connectivity by Using EEG Signals. Frontiers in Human Neuroscience, 12(November), art.number:381, pp.1-14. Available at: https://doi.org/10.3389/fnhum.2018.00381.
Liao, K., Xiao, R., Gonzalez, J. & Ding, L. 2014. Decoding Individual Finger Movements from One Hand Using Human EEG Signals. PLOS ONE, 9(1), art.number: e85192, pp.1-12. Available at:
https://doi.org/10.1371/journal.pone.0085192.
Looned, R., Webb, J., Xiao, Z.G. & Menon, C. 2014. Assisting drinking with an affordable BCI-controlled wearable robot and electrical stimulation: a preliminary investigation. Journal of NeuroEngineering and Rehabilitation, 11 (art.number:51]. Available at: https://doi.org/10.1186/1743-0003-11 -51.
Mandekar, S., Holland, A., Thielen, M., Behbahani, M. & Melnykowycz, M. 2022. Advancing towards Ubiquitous EEG, Correlation of In-Ear EEG with Forehead EEG. Sensors, 22(4), art.number:1568. Available at: https://doi.org/10.3390/s22041568.
Meng, J., Zhang, S., Bekyo, A., Olsoe, J., Baxter, B. & He, B. 2016. Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks. Scientific Reports, 6(art.number: 38565), pp.1-15. Available at: https://doi.org/10.1038/srep38565.
Mohamed, A-K. & Aharonson, V. 2021. Four-class BCI discrimination of right and left wrist and finger movements. IFAC-PapersOnLine, 54(21), pp.91-96. Available at: https://doi.org/10.1016Zj.ifacol.2021.12.016.
Murphy, D.P., Bai, O., Gorgey, A.S., Fox, J., Lovegreen, W.T., Burkhardt, B.W., Atri, R., Marquez, J.S., Li, Q. & Fei, D-Y. 2017. Electroencephalogram-Based Brain-Computer Interface and Lower-Limb Prosthesis Control: A Case Study. Frontiers in Neurology, 8(December), art.number:696, pp.1-8. Available at: https://doi.org/10.3389/fneur.2017.00696.
Noel, T.C. & Snider, B.R. 2019. Utilizing Deep Neural Networks for Brain: Computer Interface-Based Prosthesis Control. Journal of Computing Sciences in Colleges, 35(1), pp.93-101 [online]. Available at: http://ccsc.org/publications/journals/NW2019.pdf [Accessed: 20 November 2022].
Osama, M. & Allauddin, U. 2022. Design and modelling of lower prosthetic limb for additive manufacturing. In: IMEC-2022: 11th International Mechanical Engineering Conference, Karachi, Pakistan, p.27, January 14-15 [online]. Available at:
https://imec.neduet.edu.pk/sites/default/files/IMEC_Proceedings_2022.pdf [Accessed: 20 November 2022].
Paek, A.Y., Agashe, H.A. & Contreras-Vidal, J.L. 2014. Decoding repetitive finger movements with brain activity acquired via non-invasive electroencephalography. Frontiers in Neuroengineering, 7(March), art.number:3, pp. 1-18. Available at: https://doi.org/10.3389/fneng.2014.00003.
CT> !± cp
oi
E
CO
o
t CO
M—
o Ö
o o
T3 CD </) CO .Q
CD
LU LU
M—
o
T3
</)
CD >
01 c CD -C CD
E o
o <
E
CO CO OT
CO
E
CO
T3
.Q <
<u
O >
co CM o CM
of
UJ
a:
ZD O o
_J
<
O 2:
X
o
LU
I—
>-
a: <
1—
< -j
CD
■O 2: x
LU I—
O
o >
0
Ramalingam, V.V., Pandian, A. & Parivel, R. 2016. Controlling Artificial Limb Movement System using EEG Signals. Indian Journal of Science and Technology, 9(47), pp.1-7. Available at: https://doi.org/10.17485/ijst/2016/v9i47/107945.
Ramos-Murguialday, A., Schurholz, M., Caggiano, V., Wildgruber, M., Caria, A., Hammer, E.M., Halder, S. & Birbaumer, N. 2012. Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses. PLOS ONE, 7(10), art.ID: e47048, pp.1-10. Available at:
https://doi.org/10.1371/journal.pone.0047048.
Rashid, N., Iqbal, J., Javed, A., Tiwana, M.I. & Khan, U.S. 2018. Design of Embedded System for Multivariate Classification of Finger and Thumb Movements Using EEG Signals for Control of Upper Limb Prosthesis. BioMed Research International, 2018(art.ID:2695106). Available at: https://doi.org/10.1155/2018/2695106.
Setiawan, J.D., Alwy, F., Ariyanto, M., Samudro, L. & Ismail, R. 2021. Flexion and Extension Motion for Prosthetic Hand Controlled by Single-Channel EEG. In: 2021 8th International Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), Semarang, Indonesia, pp.47-52, September 23-24. Available at: https://doi.org/10.1109/ICITACEE53184.2021.9617220.
Shedeed, H.A., Issa, M.F. & El-Sayed, S.M. 2013. Brain EEG signal processing for controlling a robotic arm. In: 2013 8th International Conference on Computer Engineering & Systems (ICCES), Cairo, Egypt, pp.152-157, November 26-28. Available at: https://doi.org/10.1109/ICCES.2013.6707191.
Soekadar, S.R., Witkowski M., Vitiello, N. & Birbaumer, N. 2015. An EEG/EOG-based hybrid brain-neural computer interaction (BNCI) system to control an exoskeleton for the paralyzed hand Biomedical Engineering / Biomedizinische Technik, 60(3), pp.199-205. Available at: https://doi.org/10.1515/bmt-2014-0126.
Steinisch, M., Tana, M.G. & Comani, S. 2013. A Post-Stroke Rehabilitation System Integrating Robotics, VR and High-Resolution EEG Imaging. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21(5), pp.849859. Available at: https://doi.org/10.1109/TNSRE.2013.2267851.
Szabolcsi, R. 2019. Modern Control Engineering. Budapest: Obuda University. ISBN: 978-963-449-1880.
Szabolcsi, R. 2020. Control System Design Using MATLAB. Budapest: Obuda University. ISBN: 978-963-449-1873.
Tang, Z., Sun, S., Zhang, S., Chen, Y., Li, C. & Chen, S. 2016. A Brain-Machine Interface Based on ERD/ERS for an Upper-Limb Exoskeleton Control. Sensors, 16(2), art.number:2050. Available at:
https://doi.org/10.3390/s16122050.
Tung, S.W., Guan, C., Ang, K.K., Phua, K.S., Wang, C., Zhao, L., Teo, W.P. & Chew, E. 2013. Motor imagery BCI for upper limb stroke rehabilitation: An evaluation of the EEG recordings using coherence analysis. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, pp.261-264, July 03-07. Available at: https://doi.org/10.1109/EMBC.2013.6609487.
Übeyli, E.D. 2009. Combined neural network model employing wavelet coefficients for EEG signals classification. Digital Signal Processing, 19(2), pp.297-308. Available at: https://doi.org/10.1016/j.dsp.2008.07.004.
Vidaurre, C., Klauer, C., Schauer, T., Ramos-Murguialday, A. & Müller, K-R. 2016. EEG-based BCI for the linear control of an upper-limb neuroprosthesis. Medical Engineering & Physics, 38(11), pp.1195-1204. Available at: https://doi.org/10.1016/j.medengphy.2016.06.010.
Wen, T., Du, Y., Pan, T., Huang, C. & Zhang, Z. 2021. A Deep Learning-Based Classification Method for Different Frequency EEG Data. Computational and Mathematical Methods in Medicine, 2021(art.ID:1972662). Available at: https://doi.org/10.1155/2021/1972662.
Witkowski, M., Cortese, M., Cempini, M., Mellinger, J., Vitiello, N. & Soekadar, S.R. 2014. Enhancing brain-machine interface (BMI) control of a hand exoskeleton using electrooculography (EOG). Journal of NeuroEngineering and Rehabilitation, 11(art.number:165). Available at: https://doi.org/10.1186/1743-0003-11-165.
Xu, B., Peng, S., Song, A., Yang, R. & Pan, L. 2011. Robot-Aided Upper-Limb Rehabilitation Based on Motor Imagery EEG. International Journal of Advanced Robotic Systems, 8(4). Available at: https://doi.org/10.5772/45703.
Xu, B., Song, A., Zhao, G., Xu, G., Pan, L., Yang, R., Li, H. & Cui, J. 2015. Design and evaluation of a motor imagery electroencephalogram-controlled robot system. Advances in Mechanical Engineering, 7(3). Available at: https://doi.org/10.1177/1687814015573607.
Yanagisawa, T., Hirata, M., Saitoh, Y., Goto, T., Kishima, H., Fukuma, R., Yokoi, H., Kamitani, Y. & Yoshimine, T. 2011. Real-time control of a prosthetic hand using human electrocorticography signals: Technical note. JNS - Journal of Neurosurgery, 114(6), pp.1715-1722. Available at:
https://doi.org/10.3171/2011.1 .JNS101421.
Zhang, A., Yang, B. & Huang, L. 2008. Feature extraction of EEG signals using power spectral entropy. In: 2008 International Conference on BioMedical Engineering and Informatics, Sanya, China, pp.435-439, May 27-30. Available at: https://doi.org/10.1109/BMEI.2008.254.
Комплексное исследование об управлении искусственной рукой с помощью электроэнцефалографии (ЭЭГ)
Ихаб Абдулрахман Сатам
Обудский университет, Докторская школа безопасности, г. Будапешт, Венгрия;
Северный технический университет, Технический институт / Аль-Хавия, департамент электронной техники, Республика Ирак
РУБРИКА ГРНТИ: 45.31.31 Электрические аппараты низкого напряжения ВИД СТАТЬИ: оригинальная научная статья
■чг ст>
<л
Е
(О 'о
ъ
со
ч— О
О
О о тз CD (Л
со
.Q
6 ш
Ш
ч— О
тз
(Л
CD >
<л с CD -С CD
Е о о
с
Е со со ю
со Е со
тз
.Q
<
Резюме:
Введение/цель: Сигнал электроэнцефалографии (ЭЭГ) оказывает большое влияние на развитие технологии управления протезом руки. Сигналы ЭЭГ используются в качестве основного инструмента в функциональных исследованиях движений человека. Изучение управления протезом руки с помощью сигналов головного мозга все еще находится на ранней стадии, так как данный вид управления начали исследовать всего несколько лет назад.
Методы: Было проведено несколько исследований с целью систематического обзора опубликованных статей, для того чтобы исследователи и специалисты могли ознакомиться с новейшими методами управления с помощью сигналов ЭЭГ, используемых не только в области протезирования конечностей, но и в других технологиях.
Результаты: В ходе исследования проанализировано 175 статей, из которых отобраны только непосредственно относящиеся к теме данного исследования. Выводы: Данное исследование преследует три цели. Первая цель заключается в сборе, систематизации исследований и оценке полученной информации из трудов, опубликованных в период с 2011 по 2022 год. Вторая цель состоит в представлении подробного отчета о целостных экспериментальных достижениях в этой области, а также о текущих исследованиях. Таким образом систематизированный материал предоставляет большое количество примеров, описанных в новейших исследованиях об управлении протезом руки с помощью ЭЭГ. Третья цель состоит в выявлении проблем, требующих дальнейшего изучения, и в рекомендации направлений для будущих исследований в данной области.
Ключевые слова: ЭЭГ, ЭМГ, комплексное исследование, протез руки, управление.
Опсежна студи]а о управ^а^у вештачком руком помогу електроенцефалографи]е (ЕЕГ)
Ихаб Абдулрахман Сатам
Универзитет Обуда, Школа докторских безбедносних студи|а, Будимпешта, Ма^арска;
Северни технички универзитет, Технички институт / Ал-хави]а, Оде^е^е за електронске технике, Република Ирак
ОБЛАСТ: механика, електроника
КАТЕГОРША (ТИП) ЧЛАНКА: оригинални научни рад
Сажетак:
41
чг ст>
Увод/цип>: Сигнал у електроенцефалограф^и (ЕЕГ) има велики утицаj на развоj технолог^е управъача простетичком руком. При функционалном испитивачу ъудског покрета као главно средство е користе се ЕЕГ сигнали. Контрола простетичке руке путем ® можданих таласа jе jош у раним фазама испитивача. Истраживачи се тек од пре неколико година баве овом врстом управъача. Методе: Неколико студ^а jе имало за циъ да систематично обради до сада об!авл>ена истраживача како би се истраживачима и о струччацима пружио свеобухватни преглед наjновиjих техника
о
о
управъача путем ЕЕГ сигнала ще се користе не само за "
простетичке руке веЬ и за друге технолог^е. ф
Резултати: Упоре^ено jе 175 чланака, а изабрани су само они щи су £
наjmешче повезани са студ^ом. ^
Закчучак: Ова студша има три циъа. Први jе да скупи,
о тз
систематизме и процени информац^е обjавл>ене у студ^ама у периоду од 2011. до 2022. године. Други jе да пружи детаънщи извештаj о холистичким, експерименталним постигнуЬима у овоj области, као и о садшчим истраживачима. Систематично ура^ена студша обезбе^е мноштво примера из савремених истраживача управъача простетичком руком путем ЕЕГ сигнала. ТреЯи цил> jесmе да се укаже на области ще захтеваjу даъа истраживача, као ^ и да се препоруче правци за чихово спрово^ече. 8
Кчучне речи: ЕЕГ, BCI, свеобухватна студша, простетичка рука, контролери.
e
с
Paper received on / Дата получения работы / Датум приема чланка: 24.11.2022. Manuscript corrections submitted on / Дата получения исправленной версии работы / Датум достав^а^а исправки рукописа: 28.01.2023.
Paper accepted for publishing on / Дата окончательного согласования работы / Датум коначног прихвата^а чланка за об]ав^ива^е: 29.01.2023.
© 2023 The Author. Published by Vojnotehnicki glasnik / Military Technical Courier (www.vtg.mod.gov.rs, втг.мо.упр.срб). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/rs/).
© 2023 Автор. Опубликовано в «Военно-технический вестник / Vojnotehnicki glasnik / Military Technical Courier» (www.vtg.mod.gov.rs, втг.мо.упр.срб). Данная статья в открытом доступе и распространяется в соответствии с лицензией «Creative Commons» (http://creativecommons.org/licenses/by/3.0/rs/).
© 2023 Аутор. Об]авио Воjнотехнички гласник / Vojnotehnicki glasnik / Military Technical Courier (www.vtg.mod.gov.rs, втг.мо.упр.срб). Ово ]е чланак отвореног приступа и дистрибуира се у складу са Creative Commons лиценцом (http://creativecommons.org/licenses/by/3.0/rs/).
m a t a
Ю
n a m
d b
С
