CC BY
3Russian Journal of Nonlinear Dynamics, 2022, vol. 18, no. 5, pp. 859-872. Full-texts are available at http://nd.ics.org.ru DOI: 10.20537/nd221221
NONLINEAR ENGINEERING AND ROBOTICS
MSC 2010: 68T10
EMG-Based Grasping Force Estimation for Robot Skill Transfer Learning
W.Ali, S. Kolyubin
In this study, we discuss a new machine learning architecture, the multilayer preceptron-random forest regressors pipeline (MLP-RF model), which stacks two ML regressors of different kinds to estimate the generated gripping forces from recorded surface electromyographic activity signals (EMG) during a gripping task. We evaluate our proposed approach on a publicly available dataset, putEMG-Force, which represents a sEMG-Force data profile. The sEMG signals were then filtered and preprocessed to get the features-target data frame that will be used to train the proposed ML model. The proposed ML model is a pipeline of stacking 2 different natural ML models; a random forest regressor model (RF regressor) and a multiple layer perceptron artificial neural network (MLP regressor). The models were stacked together, and the outputs were penalized by a Ridge regressor to get the best estimation of both models. The model was evaluated by different metrics; mean squared error and coefficient of determination, or r2 score, to improve the model prediction performance. We tuned the most significant hyperparameters of each of the MLP-RF model components using a random search algorithm followed by a grid search algorithm. Finally, we evaluated our MLP-RF model performance on the data by training a recurrent neural network consisting of 2 LSTM layers, 2 dropouts, and one dense layer on the same data (as it is the common approach for problems with sequential datasets) and comparing the prediction results with our proposed model. The results show that the MLP-RF outperforms the RNN model.
Received September 13, 2022 Accepted December 13, 2022
This work was supported by NIR-PRIKL project: development of models and algorithms for machine learning and nonlinear control for information and control systems of mobile service robots and their formations.
Waddah Ali
[email protected] ITMO University
Kronverkskiy prosp. 49, Sankt-Peterburg, 197101 Russia
Sergey Kolyubin [email protected]
Professor, Faculty of Control Systems and Robotics, ITMO University Kronverkskiy prosp. 49, Sankt-Peterburg, 197101 Russia
Keywords: sEMG signals, multilayer perceptron regressor (MLP), random forest regressor (RF), recurrent neural network (RNN), robot grasping forces, skill transfer learning
1. Introduction
The need to solve manipulation tasks in multiple applications in complex environments raises the relevance of skill transfer learning techniques to teach robotic arms from human demonstration. Different approaches have been proposed to play an essential role for controlling the robotic manipulators by human demonstrations in order to reduce the time-consuming and traditional programming complexity and to overcome the limitations of the classical control methods for robotics to be scalable and generalized for different applications in complex environments.
The most related approaches were focusing on behavioral cloning (BC) [1], generative adversarial imitation learning (GAIL) [2] and the inverse reinforcement learning (IRL) [3]. The high computational complexity and high-dimensional task spaces for the lateral two approaches affected the efficiency of applying those methodologies to complex tasks.
In behavioral cloning, a neural network (ANN) is suitable for modeling nonlinear data and is able to account for differences between different conditions. Over the past decade, several ANN-based EMG pattern recognition methods have been introduced. For example, in [4], ANN background propagation (BP) is used to perform pattern recognition with frequency responses. In [5], researchers were able to isolate four movements of the forearm (flexion, extension, pronation, and supination) using a combination of BPNN and Hopfield ANN. In [6, 7] and others, similar work was done. However, the commonly used BPNNs in the above studies do not provide an approach to estimate muscle forces from myoelectric signals (MES). They concentrate efforts on classifying the four above-mentioned arm movements, and a large amount of training data is required as well as a large number of training iterations.
In this paper, we are proposing a new behavioral cloning based approach to develop an algorithm that takes the raw recorded human arm biosignals for muscle activities as a state and outputs the appropriate grasping forces to be applied to let the robot imitate the human demonstrated grasping skill. The goal of this study is to build the first essential block of transferring the demonstrated human grasping and manipulation skills for different complex tasks in complex environments to teach robotic manipulators the human skills, which is the estimation of applied human grasping forces from sEMG records to be mapped and transferred to the robotic arm successively.
The novelty of this approach is that it provides a new model architecture based on both random forest and MLP neural networks to address the problem of estimating human applied forces during grasping tasks from recorded sEMG signals. This approach can be implemented in human skill transfer learning for teaching robotic manipulators.
The paper is organized as follows: in Section 2 we present the experimental setup of the study; we then implement the data preprocessing needed before training the proposed model; and we explain how features were selected to train the proposed model. In Section 3 we describe how we build our proposed model in detail, while results and discussions on implementing our approach and comparing it to the RNN approach are illustrated in Section 4. The conclusion is found in Section 5.
2. Experimental setup
The putEMG-Force dataset was used to train and evaluate our proposed model [8]. It was recorded by the Biomedical Engineering and Biocybernetics Team, Poznan University of Technology, Poland. A dataset of surface electromyographic activity recorded from the forearm allows for the development of algorithms for gesture recognition and grasp force recognition. An experiment was conducted on 44 participants, with two repetitions separated by a minimum of one week. sEMG was recorded using a 24-electrode matrix.
The sEMG-Force data profile consists of 24 filtered EMG signal columns as input (Fig. 1), and 10 recorded grasping forces as output to train the proposed ML model (Fig. 2).
Filtered Recorded EMG signals from 24 surface electrodes (first 12 electrodes)
EMG 1 EMG 2
2.5 0.0 -2.5
0 -5
I 2.5
„ 0.0 of —2.5 XI
^ I
a _2 <!
2.5 0.0 -2.5
2.5 0.0 -2.5
M
200000 400000 600000 800000 1000000 1200000
EMG 3
M
200000 400000 600000 800000 1000000 1200000
EMG 5
F*
200000 400000 600000 800000 1000000 1200000
EMG 7
2.5 0.0 -2.5
M
200000 400000 600000 800000 1000000 1200000
EMG 4
Ft
200000 400000 600000 800000 1000000 1200000
EMG 6
m
2.5 0.0 -2.5
200000 400000 600000 800000 1000000 1200000
EMG 8
200000 400000 600000 800000 1000000 1200000
EMG 9
♦ ■ 4
Fi
2.5 0.0 -2.5
200000 400000 600000 800000 1000000 1200000
EMG 10
Ft
200000 400000 600000 800000 1000000 1200000
EMG 11
200000 400000 600000 800000 1000000 1200000
EMG 12
Ft
200000 400000 600000 800000 1000000 1200000
200000 400000 600000 800000 1000000 1200000
Samples
Fig. 1. An example of the recorded EMG signals from 12 sEMG electrodes after filtering. The overall 24 filtered recorded EMG signals are the input data to train the proposed ML model in this article
2.1. Data preprocessing
The recorded raw EMG-Force dataset contains noise and missing values because of noticeable artifacts like displacement of sEMG electrodes during recording, the noise that was caught by those electrodes, etc., i. e., it cannot be directly used for the proposed machine learning models [8].
Data preprocessing is a required task for cleaning the data and making it suitable for the ML model, which also increases the accuracy and efficiency of the ML model.
First of all, we had to calculate the voltage values from recorded sEMG channels as in Eq. (2.1) which were stored as ADC raw values where N is an ADC value:
5 1000 r , , ,
x = N-—---mv . (2.1)
2 200 L J v 7
The calculated voltage values were then filtered using a multi-notch filter (frequencies = 30,
49.99, 90, 60, 150 Hz) and a Butterworth band-pass filter (Ip = 20 Hz, hp = 700 Hz) as suggested
by [8].
£
-ö
CD
500
500 400
300
Measured Forces applied by human fingers during grasping Force 1 Force 2
20000
40000 60000 80000 100000 120000
Force 3
0 20000 40000 60000 80000 100000 120000
Force 4
20000
40000 60000 80000 100000 120000
Force 5
0 20000 40000 60000 80000 100000 120000
Force 6
20000
40000 60000 80000 100000 120000
Force 7
0 20000 40000 60000 80000 100000 120000
Force 8
20000
40000 60000 80000 100000 120000
Force 9
0 20000 40000 60000 80000 100000 120000
Force 10
20000
40000 60000 80000 100000 120000 0 20000 40000 60000 80000 100000 120000
Samples
Fig. 2. Measured grasping forces from human fingers. These measures are the output (target) for training the proposed ML model to predict
The filtered data was cleaned of nulls, infinite values, and outliers to ensure the consistency and stability of the model prediction performance. To compensate for missing values, such as nulls and infinite values, we used the mean value of their neighbor data points, as in Eq. (2.2), whereas to remove outliers, we used the interquartile range (IQR) method to determine the quartiles of each feature column, then removed all data points that fell outside the range defined by the quartiles (±1.5 times IQR):
x® = »E-ll+sp-l]. (2.2)
In the case of the MLP model, the data were rescaled using the standard scaler to put every feature on an equal footing without any upfront significance and to make the gradient descent converge much faster.
2.2. Feature selection
Feature selection is intended to reduce the number of input variables (the EMG signals) to those that are believed to be most useful to a model in order to predict the target variable, i.e., selects from 24 EMG columns the ones most highly affecting the prediction of the target. This is relevant to reducing the data dimensionality, which leads to reducing the complexity of the model.
In the case of our proposed model, we represent a pipeline of stacked 2 different regressors: random forest and multilayer perceptron.
The random forest regressor includes ensembles of decision trees which depend on penalized regression models that select the features that are more highly related to predicting the output [9].
For the MLP regressor, we adopted a wrapper feature selection algorithm [11] as suggested by [10], as they obtained a significant improvement in the overall results with respect to learning with the whole set of variables in most of the data sets tested.
3. Building the proposed pipeline
Our approach proposes stacking two different regressors, multilayer perceptron (MLP) and random forest (RF) regressors, via a meta-regressor, the ridge regressor. The individual regressors were trained separately based on the complete training set; then, the meta-regressor was fitted based on the outputs (meta-features) of the individual regressors in the ensemble. This approach adopts the stacking regression technique to give an improved prediction accuracy [12].
3.1. Reasons behind the proposed model architecture
The proposed method adds a new approach to benchmarking in which generated gripping forces can be predicted and estimated by recording sEMG human muscle activity signals.
The choice of the random forest model was made according to the advantages behind adopting ensemble models:
• The default hyperparameters random forest uses often produce good prediction results.
• The ability to handle the nonlinearity of the sEMG signals as independent predictors of the dependent force output.
• One of the biggest problems in machine learning is overfitting, but most of the time this does not happen thanks to the random forest classifier. If there are enough trees in the forest, the classifier will not overfit the model.
We have chosen the multiple-layer perceptron neural network (MLP) due to the following advantages:
• Its applicability to complex nonlinear problems makes it appropriate for our problem statement.
• High performance with large input data. In our case, we need a large amount of input data recorders under different scenarios to guarantee the generalizability of the proposed approach.
• Rapidity: It provides a quick prediction after training, which is critically important to execute the task in real time for a robot.
• Consistency: The same accuracy can be achieved and guaranteed using a smaller amount of data.
• Scalability to different training and prediction scenarios and different shapes of datasets.
The idea behind an ensemble of models (stacking MLP with RF regressor in our approach) is to maximize our models' predictions from multiple machine learning models by assigning weights according to their performance. To guarantee giving the better performing model more say in our final prediction in an ensemble, we use a stacking algorithm that learns how to best combine each of the models in an ensemble to come up with the best performance.
• An ordinary machine learning model only tries to map input towards output by generating a relationship function.
• Stacking acts on one level above the ordinary by learning the relationship between the prediction result of each of the ensembled models on out-of-sample predictions and the actual value.
In most of the papers discussing stacked models, the meta-model used is often just a simple model such as linear regression for regression tasks and logistic regression for classification tasks. One reason why more complex meta-models are often not chosen is that there is a much higher chance that the meta-model may overfit the predictions from the base models [13].
For our problem, ridge regression [14] works much better than Linear Regression. This is because the base model's predictions are strongly correlated, as they are all trying to predict the same relationship. Hence, a linear regression fit may cause the final prediction to be highly sensitive to changes in the data. Therefore, higher variance leads to bad generalization.
Ridge Regression comes with regularization parameters and hence is able to deal with the correlation between each base model's predictions much better than Linear Regression. This has been shown empirically to be true. However, a general proof has yet to be devised in any paper.
Going back to the MLP and RF regressors, the architecture was detected according to tuning hyperparameters of both models using a random search algorithm [15] followed by a grid search algorithm [16].
The proposed model structure is illustrated in Fig. 3.
3.2. Random forest regressor
Random forest regression [17] is a supervised learning algorithm that uses the ensemble learning method for regression. The ensemble learning method is a technique that combines predictions from multiple machine learning algorithms to make a more accurate prediction than a single model.
A random forest operates by constructing several decision trees during training time and outputting the mean of the classes as the prediction of all the trees (see Fig. 3 random forest regressor).
To get the best performance of the random forest regressor, we tuned the most significant hyperparameters using the random search algorithm [18] followed by the grid search algorithm [19].
The main idea of the aforementioned search algorithms as optimization problems is to find the optimal set of model hyperparameters by randomly iterating over the predefined set of values for each hyperparameter, training the model with these values, and evaluating the approximation results, finally setting the hyperparameters' values with the best approximation results. The tuned model hyperparameters are:
bootstrap: determines whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree.
Max_depth: the tree's maximum depth. If None, nodes are expanded until all leaves are pure or contain fewer than min_samples_split samples.
max_features: the number of features to take into account when determining the best split.
min_samples_splits: the number of samples required to split an internal node. Min_samples_per_leaf: the bare minimum of samples required at a leaf node. N estimators: the number of trees in the forest.
MLP Regressor
Random Forest Regressor
Fig. 3. MLP-RF Model Structure
3.3. Multilayer perceptron neural networks
A multilayer perceptron (MLP) [20] was adopted with a ReLU activation function.
3.3.1. MLP model architecture
The MLP model has two main hyperparameters that control the architecture or topology of the network: the number of layers and the number of nodes in each hidden layer. The MLP regressor was structured as follows:
• input layer: the size of the input layer equals the number of sEMG channels, 24 units.
• hidden layers: we have four dense layers, each of a size defined by Eq. (3.1). Each of them is connected to a batch normalization and dropout layer, respectively.
• output layer: To produce the output variables, we have an output layer of size 10, the number of grasping forces to be predicted by the model.
N
a j __input, samples /„ ,1
hidden = Factor ■ (^input features + ' 1 ' j
where Factor is an integer factor, Nhidden represents the number of nodes in the hidden layer,
Ninput samples represents the number of input data sampleS and Ninput features represents the
number of input features, i.e., the number of sEMG channels.
In addition to the essential dense layers in the MLP regressor, we added two types of special layers: batch normalization layers to accelerate the training process and dropout layers to avoid overfitting and underfitting problems. More details will be explained later in the results and discussion sections during model evaluation illustrations.
4. Results and discussions
The MLP-RF model was trained on the proposed dataset by splitting it into train and test splits (80% for training and 20% for testing). The specified batch size during MLP training was set to 500 epochs with an early stopping callback condition in case the training process does not improve the model prediction into 3 consecutive epochs. The specified batch size was 256 samples to accelerate the training process. The specified learning rate for the optimizer was set to 0.001 by convention.
The model was evaluated using the K-fold cross validation algorithm [21] to guarantee that the results are stable and accurate. The model prediction performance was evaluated by slicing the data frames into 10 equally sized folds. In each iteration, an arbitrary fold was chosen to validate the prediction results, whereas the remaining 9 folds were for tuning the model weights.
The problem of overfitting and underfitting, which is very common in machine learning models in general, requires improvement of the model prediction to make a variance-bias tradeoff. In the case of the RF model, we used a random search algorithm in Section 3.2 to tune the above-mentioned model hyperparameters, and the resulting r2 score metrics show improvement in performance after finding appropriate values for the tuned hyperparameters, as shown in Table 1.
Table 1. Tuned hyperparameters for the random forest regressor
Random forest regressor tuned hyperparameters
parameters values
bootstrap False: all data is used to train each tree
max depth None: the depth of each tree until reaching the min sample splits
max features "sqrt": the root square of the number of selected features
min samples splits 2
min samples leaf 1
N estimators 200
The mean and standard deviation values for the r2 score of model prediction performance following K-fold cross validation with tuned hyperparameters versus the default hyperparameters before tuning are shown in Table 2.
Table 2. Tuned hyperparameters for random forest regressor
r2 score for RF prediction before and after hyperparameters tuning
v score Tuned hyp Training »«•parameters validation Default hyper par Training ameters Validation
0.95(0.10) 0.75(0.05) 1.00(0.0) (overfitting) 0.4(0.1)
In the case of the MLP model, the preliminary results showed overfitting of the model because of its high complexity compared to the size of the dataset that was used for training. For that reason, the model architecture described in Section 3.3 was modified by adding special layers called dropouts, in order to modulate the model complexity according to the size of input data to avoid the overfitting problem. The drop rate was set to 0.2, which means that, for each batch of the dataset, the layer arbitrarily drops out 20% of the nodes while keeping the remaining 80 % to reduce model complexity over the dataset batch.
The second necessary modification is to add a batch normalization layer after each hidden layer to automatically standardize the inputs to a hidden dense layer from the previous one. Batch normalization was implemented as it has the effect of dramatically accelerating the training process of a neural network and, in some cases, improving the performance of the model via a modest regularization effect.
Figure 4 illustrates the evaluation of model prediction after adding a dropout layer to improve the model prediction performance over the overfitting and underfitting problems.
Model loss
Model loss
0.55 $ 0.50 O 0.45 ,..0.40 Wo.35 2 0.30 S 0.25 0.20
0 10 20 30 40 50 60 70
120 160
Epochs (b)
Epochs (d)
Fig. 4. Model prediction performance evaluation where the red curve illustrates the validation result compared to the blue one which illustrates the training result: (a) model loss before adding dropouts layers; (b) model r2 score before adding dropout layers; (c) model loss after adding dropouts; (d) model r2 score after adding dropouts
It is clear from Fig. 4 that the improvement of the model prediction performance where both training and validation on an unseen dataset gives approximately similar prediction performance (high r2 score and low mean absolute error). Even though the model outperforms on the training scenario in the first case over the second, i. e., after adding the dropouts, (r2
= 90'
> r
training with dropouts
training without dropouts
= 83%), the goal is to narrow the prediction performance gap
between training and validation scenarios so that the model can be considered stable, consistent, and accurate across different unseen datasets.
Finally, we compare our proposed model results to the RNN model [22] on the represented dataset to represent the efficiency of using it in grasping force prediction from sEMG signals. Figures 5 and 6 illustrate the model prediction performance of our model and the RNN model, respectively.
We see from Fig. 5 that the results are more likely noisy, which leads us to clarify an important point of this work; as mentioned before, in each sample step our model predicts the value of 10 grasping force measurements (2 for each finger); for that reason we notice that the resulting prediction performance is more likely noisy, i.e., in each sample step, the model tries to predict the most likely correct value for each of the 10 output forces. Taking the mean of the ten predicted values from the model, we get the final most likely correct predicted value and the result will be in the middle, which is 90 % close to the real measured value.
FORCE 1 GT vs Pred: MSE = 0.32
FORCE 2 GT vs Pred: MSE = 0.29
0 50000 100000 150000 200000 250000 FORCE 3 GT vs Pred: MSE = 0.25
0 50000 100000 150000 200000 250000 FORCE 4 GT vs Pred: MSE = 0.18
rn
4)
O
H
n
n-l 8
T3 (1) fi
ni 4
CJ
C/J 2
In 0
eS
c«
4
2
0
-2
0 50000 100000 150000 200000 250000 FORCE 5 GT vs Pred: MSE = 0.23
0 50000 100000 150000 200000 250000 FORCE 6 GT vs Pred: MSE = 0.19
0 50000 100000 150000 200000 250000 FORCE 7 GT vs Pred: MSE = 0.42
0 50000 100000 150000 200000 250000 FORCE 8 GT vs Pred: MSE = 0.16
0 50000 100000 150000 200000 250000 FORCE 9 GT vs Pred: MSE = 0.42
0 50000 100000 150000 200000 250000 FORCE 10 GT vs Pred: MSE = 0.33
50000 100000 150000 200000 250000
50000 100000 150000 200000 250000
Number of data samples
Fig. 5. MLP model prediction performance results for the 10 grasping forces; ground truth data (GT) is shown as a blue curve, while the predictions (Pred) are in red scatters; MSE: the mean squared error for each force prediction
FORCE 1 GT vs Pred: MSE = 0.51
FORCE 2 GT vs Pred: MSE = 0.45
<u u
!h £ T3
t3
S -a a <s
cc
0 50000 100000 150000 200000 250000 FORCE 3 GT vs Pred: MSE = 0.34
0 50000 100000 150000 200000 250000 FORCE 4 GT vs Pred: MSE = 0.35
0 50000 100000 150000 200000 250000 FORCE 5 GT vs Pred: MSE = 0.41
0 50000 100000 150000 200000 250000 FORCE 6 GT vs Pred: MSE = 0.31
0 50000 100000 150000 200000 250000 FORCE 7 GT vs Pred: MSE = 0.69
0 50000 100000 150000 200000 250000 FORCE 8 GT vs Pred: MSE = 0.31
0 50000 100000 150000 200000 250000 FORCE 9 GT vs Pred: MSE = 0.5
0 50000 100000 150000 200000 250000 FORCE 10 GT vs Pred: MSE = 0.63
50000 100000 150000 200000 250000
50000 100000 150000 200000 250000
Number of data samples
Fig. 6. RNN model prediction performance results for the 10 grasping forces; ground truth data (GT) is shown as a blue curve, while the predictions (Pred) are in red scatters; MSE: the mean squared error for each force prediction
Recurrent neural networks (RNN) are the state-of-the-art algorithms for sequential data and are used by Apple's Siri and Google's voice search. It is the first algorithm that remembers its input due to an internal memory, which makes it perfectly suited for machine learning problems that involve sequential data. The RNN's internal memory comes from long-short term memory (LSTM) [23] that make information cycle through a loop. When it makes a decision, it considers the current input and also what it has learned from the inputs it has previously received.
After comparing the evaluation metrics, r2 score (Figs. 7 and 4b) and mean squared error illustrated in Fig. 8 for both models, we find that our proposed model outperforms the RNN model in predicting grasping forces from recorded sEMG signals scenarios and can be adopted for further development of designing a skill transfer policy to teach a robot the human skills by demonstrating the applied forces and trajectories.
5. Conclusion
In this study, we have implemented a proposed stacking regressor of two nonlinear regressors different in nature: the random forest regressor and the multilayer perceptron neural network regressor, to estimate the generated grasping forces from human arm muscle activity records.
Model loss
5 10 15 20 Epochs
Model score
5 10 15 20 Epochs
Fig. 7. RNN model prediction evaluation; loss and score
MSE results on validation dataset: MLP vs RNN
co
0.15
s 0.10
0.05
0.00
__^^^H |mmp
11
FORCE 1 FORCE 2 FORCE 3 FORCE 4 FORCE 5 FORCE 6 FORCE 7 FORCE 8 FORCE 9 FORCE 10
Fig. 8. Comparing validation results of our model and the RNN state of the art model
The estimated grasping forces can be then transferred to a robotic manipulator to reproduce the same human grasping skills. The results have showed the outperformance of our proposed model over the RNN model that treats the biological muscle activity signals as a time series of signals and has a higher complexity, memory, and computation cost than the proposed MLP-RF model. The results conclude the efficiency of adopting our model in related applications.
Conflict of interest
The authors declare that they have no conflict of interest.
References
[1] Torabi, F., Warnell, G., and Stone, P., Behavioral Cloning from Observation in Proc. of the 27th Internat. Joint Conf. on Artificial Intelligence (IJCAI'18, Stockholm, Sweden, Jul 2018), 8 pp.
[2] Ho, J. and Ermon, S., Generative Adversarial Imitation Learning, in 30th Conf. on Neural Information Processing Systems (NIPS, Barcelona, Spain, Dec 2016), 9 pp.
[3] Hadfield-Menell, D., Dragan, A., Abbeel, P., and Russell, S., Cooperative Inverse Reinforcement Learning, in 30th Conf. on Neural Information Processing Systems (NIPS, Barcelona, Spain, Dec 2016), 9 pp.
[4] Uchida, N., Hiraiwa, A., Sonehara, N., and Shimohara, K., EMG Pattern Recognition by Neural Networks for Multi Fingers Control, in Proc. of the 14th Annual Internat. Conf. of the IEEE Engineering in Medicine and Biology Society (Paris, France, Oct 1992): Vol. 3, pp. 1016-1018.
[5] Kelly, M.F., Parker, P.A., and Scott, R. N., The Application of Neural Networks to Myoelectric Signal Analysis: A Preliminary Study, IEEE Trans. Biomed. Eng., 1990, vol. 37, no. 3, pp. 221-230.
[6] Koike, Y. and Kawato, M., Estimation of Arm Posture in 3D-Space from Surface EMG Signals Using a Neural Network Model, IEICE Transactions Fundamentals, 1994, vol. E77-D, no. 4, pp. 368-375.
[7] Huang, H.-P. and Chen, Ch.-Y., Development of a Myoelectric Discrimination System for a Multi-Degree Prosthetic Hand, in Proc. of the IEEE Internat. Conf. on Robotics and Automation (Detroit, Mich., May 1999): Vol. 3, pp. 2392-2397.
[8] Kaczmarek, P., Mañkowski, T., and Tomczyñski, J., putEMG — A Surface Electromyography Hand Gesture Recognition Dataset, Sensors, 2019, vol. 19, no. 16, Art. 3548, 17 pp.
[9] Kursa, M. and Rudnicki, W., The All Relevant Feature Selection Using Random Forest, arXiv:1106.5112 (2011).
[10] Romero, E. and Sopena, J., Performing Feature Selection with Multilayer Perceptrons. IEEE Trans. Neural Netw., 2008, vol. 19, no. 3, pp. 431-441.
[11] John, G.H., Kohavi, R., and Pfleger, K., Irrelevant Features and the Subset Selection Problem, in Proc. of the 11th Internat. Conf. on Machine Learning (Rutgers University, New Brunswick, N.J., Jul 1994), pp. 121-129.
[12] Breiman, L., Stacked Regressions, Machine Learning, 1996, vol. 24, no. 1, pp. 49-64.
[13] Lim, Y., Stacked Ensembles Improving Model Performance on a Higher Level, Towards Data Science (25 Mar 2021).
[14] Hilt, D. E. and Seegrist, D.W., Ridge: A Computer Program for Calculating Ridge Regression Estimates, Research Note NE-236, Upper Darby, Pa.: U.S. Department of Agriculture, Forest Service, Northeastern Forest Experiment Station, 1977, 7 pp.
[15] Al-Muhammed, M. J. and Zitar, R. A., Probability-Directed Random Search Algorithm for Unconstrained Optimization Problem, Appl. Soft Comput., 2018, vol. 71, pp. 165-182.
[16] Menke, W., Geophysical Data Analysis: Discrete Inverse Theory, 3rd ed., Amsterdam: Elsevier, 2012.
[17] Ho, T.K., Random Decision Forests, in Proc. of the 3rd Internat. Conf. on Document Analysis and Recognition (Montreal, QC, Aug 1995): Vol. 1, pp. 278-282.
[18] Zabinsky, Z. B., Random Search Algorithms, in Wiley Encyclopedia of Operations Research and Management Science, J. J. Cochran, L. A. Cox, P. Keskinocak, J. P. Kharoufeh, J. C. Smith (Eds.), Hobo-ken, N.J.: Wiley, 2010.
[19] Liashchynskyi, P. and Liashchynskyi, P., Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS, arXiv:1912.06059 (2019).
[20] Hastie, T., Tibshirani, R., and Friedman, J., The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., New York: Springer, 2009.
[21] Stone, M., An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike's Criterion, J. R. Stat. Soc. Series B Stat. Methodol., 1977, vol. 39, no. 1, pp. 44-47.
[22] Recurrent Neural Networks: Design and Applications, L.R. Medsker, L.C.Jain (Eds.), Boca Raton, Fla.: CRC, 1999.
[23] Graves, A., Long Short-Term Memory, in Supervised Sequence Labelling with Recurrent Neural Networks, Stud. Comput. Intell., vol. 385, Berlin: Springer, 2012, pp. 37-45.
