Introduction

Brain Computer Interface (BCI) is a modern technology providing a communication between the brain and external environment. BCIs have been used for severely paralyzed patients as a communication option, an augmentative tool for rehabilitation and assistive technology [ 1 , 2 ]. These systems offer effective assistance not only for those with motor disabilities, but also for healthy users such as computer game control [ 3 ] and mobile robots [ 4 ]. For practical applications in BCIs, electroencephalogram (EEG) is well accepted due to high temporal resolution information, and a noninvasive, inexpensive, portable method. A type of BCI known as motor imagery (MI) refers to the act of imagining a particular action without actual execution.

In the last decade, several signal processing techniques from the one-channel EEG have been proposed for developing BCIs [ 5 - 7 ] and specially MI-BCIs, namely, power spectral density using Fourier transform [ 8 , 9 ], discrete wavelet transform [ 10 ], calculation of the autoregressive model coefficients [ 11 ], common spatial pattern [ 12 , 13 ], sparse representation [ 14 , 15 ], Hilbert–Huang transform [ 16 ], empirical mode decomposition [ 17 , 18 ], and Hjorth parameters [ 19 , 20 ]. Despite of significant achievements of the aforementioned methods, none of them have been proved to be adequately reliable in the practical settings because of using EEG features from individual channels and ignoring valuable information inherent between channels. Moreover, since EEG signals are non-stationary and very sensitive to noises, a single channel can hardly achieve a good accuracy and the extracted EEG patterns, based on multi-channel, must be considered to detect the dynamic characteristics of the EEG signals. MI task causes complicated neurophysiological changes and consequently, it is expected that for solving the aforementioned limitations in MI of different limbs, we consider the relationships among brain regions with different connectivity patterns.

The brain connectivity analysis has three general subdivisions, as follows [ 21 ]: (1) the structural connectivity (2), the functional connectivity, and (3) the effective connectivity. Among these fields, effective connectivity is a significant tool for the EEG analysis. A popular method for estimating effective connectivity is Granger Causality (GC) which is a data-driven approach [ 22 ]. Theoretically, GC characterizes the directed causal interaction among neural time-series as strong as causal mechanistic couplings. Actually, this method is used for analysis and visualization of time and frequency of multivariate dependent directed information flow and causality between localized EEG channels.

After the EEG feature extraction, the EEG signal is classified using machine learning methods. In the last decade, a wide variety of machine learning, including feature selection and classification methods have been used to EEG for developing MI-BCIs, such as, principal component analysis [ 23 , 24 ], independent component analysis [ 25 ], sequential floating forward search [ 26 ], an evolutionary algorithm [ 27 ] and relative entropy criteria for feature selection [ 28 ], linear discriminant analysis [ 29 , 30 ], multilayer perceptron network [ 31 ], radial basis function network [ 32 ], Support Vector Machine (SVM) [ 33 - 36 ], least squares classifier [ 37 ], Bayesian classifier [ 38 ], adaptive neuro-fuzzy classifier [ 39 , 40 ], extreme learning machine [ 41 ], Naive bayes [ 42 ], sparse bayesian extreme learning machine [ 43 ], sparse group representation model [ 44 ] and deep learning approaches [ 45 - 47 ]. Despite the different machine learning algorithms, there hasn’t been a universally superior one for this application.

The aim of this study is to find a set of discriminative effective connectivity features from EEG signals and to design a hierarchical feature selection and classification methods for discrimination of left and right hand MI task. The ability of this novel system is evaluated with 29 subjects.

Material and Methods

A. Subjects and Data Acquisition

In this analytical study, 29 healthy subjects with no reported brain-related diseases participated [ 48 ]. EEG data was recorded at 1000 Hz sampling rate with thirty electrodes according to the international 10-5 system with Fz as the ground electrode. The subjects sat on an armchair in front of a 50-inch white screen with distance of 1.6 m and without any movement of the body.

The experiment included three sessions of left and right hand MI. Each session comprised a 1 min pre and post-experiment resting period, and 20 repetitions of the given task (10 trials for each left and right hand MI). The task started with 2 s of a visual introduction of the task, followed by 10 s of a task period and 15 to 17 s resting period randomly. Figure 1 shows the diagram of the experimental paradigm. In the task period, subjects imagine the opening and closing their hands with a speed of 1 Hz. Therefore, for each subject in the whole three sessions with 30 trials for left and also 30 trials for right hand MI were performed.

Figure 1. Schematic diagram of the data acquisition (Figure reproduced from Shin J, et al. IEEE Trans Neural Syst Rehabil Eng. 2017;25(10):1735-45. [ 48 ])

B. Effective connectivity

Effective connectivity is an important aspect of modern neuroscience due to its potential to describe interactions between brain areas and in other words representation of the direction and strength of the information flow between different brain areas [ 49 ]. A popular statistical method for estimating effective connectivity is Granger causality (GC) which is a data-driven approach [ 50 ].

GC in the frequency domain results in the analysis of EEG frequencies bands [ 51 ]. This method is based on estimation of parameters of Multi-Variable Auto-Regressive (MVAR) model for an individual signal data. Quantitative spectral measures are: Generalized Partial Directed Coherence (GPDC) [ 52 , 53 ], Directed Transfer Function (DTF) [ 54 ], and Direct DTF (dDTF) [ 55 ]. These techniques characterize the direction and spectral properties between any pair of channels in EEG signals simultaneously. By these measures, it’s possible to quantify the value of each possible electrode combination per specific frequency range. We extract each frequency range for each measure by averaging theta (4 − 7 Hz), alpha or mu (8 − 12 Hz), beta1 (12 − 15 Hz), beta2 (15 − 22 Hz), beta3 (22 − 30 Hz) and gamma (30 − 45 Hz). All calculations were done using MATLAB (The Mathworks, Inc., Natick, MA, USA) via the open-source SIFT toolbox [ 56 ].

C. Statistical Analysis

Due to the limited dataset, k-fold cross validation was used. This method partitions the dataset into k equal sized subsamples (in this paper, k is set to 10). In each trial, the classification structure is constructed with 90 percent of data and evaluated with the remaining data as testing data. The process is repeated 10 times, each subsample used exactly once as the testing data, until all the dataset has been used for testing. Evaluation performance is reported by averaging the 10 results from each subsample. Moreover, the 10-fold cross validation is done for 100 consecutive runs and the average of the results is calculated. The advantage of this method compared with repeated random sub-sampling is the use of all dataset for both training and testing.

D. Preprocessing

First, the EEG data was re-referenced to common average reference, then a fourth order of Chebyshev type II with a passband of 2 - 45 Hz was applied to the signal. Finally, electrooculogram (EOG) artifact was rejected using independent component analysis (ICA) method via the automatic artifact rejection toolbox in EEGLAB [ 57 ].

E. Feature selection method

The significance of the extracted features from effective connectivity methods between right and left hand MI groups is tested by the Kruskal-Wallis method [ 58 ]. Then, the non-significant features with p > 0.01 are discarded. After that, the Minimum Redundancy Maximum Relevance (mRMR) method [ 59 , 60 ] is used to select the best features. In this method, features can be selected far away from each other while still having the highest relevance to the target classes. This selection method is more powerful than the only maximum relevance selection methods.

F. SVM classification

SVM is one of the powerful tools for classification issues. In this method, the parameters of the separator function should be designed so that the margin between the hyperplane becomes maximum [ 61 ]. SVM maps the input space to a higher dimension in order to unfold the complexity existing in the dataset using a proper kernel function and after that, a linear decision surface is designed to identify the labels with fewer complications. Appropriate function is Radial Basis Function (RBF) and the parameters are determined in the optimization process.

Results

Several measures of effective connectivity using GC methods (GPDC, DTF, dDTF) and parameters of calculated MVAR model were estimated based on each signal frequency band ranges in 29 subjects (30 left hand MI and 30 right MI task for each subject). Best parameters for MVAR model fitting were selected according to autocorrelation function and portmanteau tests. The optimized parameters are: 5 s of window length, and model order of 60. Having 30-channel EEG, 900 (30×30) directed causal connection between channels as effective connectivity features are extracted for each GC method in each frequency band range that make computation complex. As a result, a feature subset selection based on Kruskal-Wallis statistical test is done to choose most significant features for discrimination of right and left hand MI tasks. Using this test, the nonsignificant features, which result in p > 0.01, are discarded. Then, using the mRMR algorithm via 5-fold cross validation, the best features remained after the Kruskal-Wallis test are selected. This method chooses the features with the minimum redundancy among the selected features and the highest relevance to the target classes. Finally, the best selected features are fed to SVM classification structure. The classification procedure aims to accurately classify EEG data into left and right hand MI tasks in 29 subjects. A 10-fold cross-validation was performed to evaluate the classification performance. RBF is selected as the kernel function of SVM with the optimal sigma value (σ) of 0.90 by try and error methodology. The schematic diagram of the proposed MI-BCI system is shown in Figure 2.

Figure 2. The process of the proposed Motor Imagery (MI)-Brain Computer Interface (BCI)system (a) Raw Electroencephalogram (EEG) data (b) Preprocessing (c) Construction of effective connectivity matrix (d) The statistical significance of the extracted connectivity features between right and left hand Motor Imagery (MI) groups using the Kruskal-Wallis test (e) Feature selection using Minimum Redundancy Maximum Relevance (mRMR) (f) Classification using Support Vector Machine (SVM) (g) Discriminative connectivity maps.

All data processing were calculated separately over segments of 5 s. The classification accuracies obtained from the proposed method using GC (GPDC, DTF, dDTF) connectivity measures over all subjects for 0-5 s and 5-10 s for each frequency band range are given in Table 1, separately. As seen, GPDC yields the best results with high accuracy in classifying right and left hand MI task within Mu (83.87) and beta1 (83.05) frequency bands of EEG for 0-5 s. However, other GC methods (DTF, dDTF) and other frequency bands of EEG (theta, beta2, beta3, and gamma) are not able to categorize right and left hand MI tasks with significant accuracy. It should be noted that at a time interval of 0 - 5s due to its inherent fast responses of brain electrical activity, the EEG is the best separable.

Raw 900 (30×30) connectivity features for the best results of GC method with high accuracy (GPDC) for Mu, and Beta1 frequency band over all subjects in 0-5 s for left and right hand MI task are shown in Figure 3. A higher absolute value of connectivity feature is shown in warm colors. Moreover, the scalp topographies of the averages of normalized log (p-value) obtained from the best selected of GPDC connectivity features (68 connectivity) in feature selection procedure for 0-5 s in Mu and Beta1 frequency band over all subjects are shown in Figure 4. A higher absolute value of log (p-value) is shown in warm colors i.e. a better separability with higher significance between left and right hand MI task. As seen, during the MI task, EEG generally show good separation around the motor cortex and high separation around frontal and parietal areas (Figure 4).

Figure 3. Raw 900 (30×30) Generalized Partial Directed Coherence (GPDC) connectivity features for Mu, and Beta1 frequency band over all subjects for 0-5 seconds for left and right hand Motor Imagery (MI) task. A higher absolute value of connectivity feature shows with warm colors. Thirty electrodes are as follow: F7, AFF5h, F3, AFp1, AFp2, AFF6h, F4, F8, AFF1h, AFF2h, Cz, Pz, FCC5h, FCC3h, CCP5h, CCP3h, T7, P7, P3, PPO1h, POO1, POO2, PPO2h, P4, FCC4h, FCC6h, CCP4h, CCP6h, P8, T8.

Figure 4. Normalized-log (p-value) obtained from the best selected of Generalized Partial Directed Coherence (GPDC) connectivity features (68 connectivity) in feature selection procedure for Mu and Beta1 frequency band over all subjects for 0-5 seconds. A higher absolute value of log (p-value) shows with warm colors and means a better separability between left and right hand Motor Imagery (MI) task. Directional connectivity represented by the arrows.

Discussion

In this paper, we proposed a new automated algorithm for discrimination of left and right hand MI tasks from EEG signal, based on a set of discriminative features using GPDC method and two hierarchical feature selection methods and finally SVM classification structure. This algorithm could classify the EEG data in 29 subjects in 60 trials with an overall accuracy of 84% during Mu-Beta1 frequency band, effectively.

Results of directional connectivity metrics (GPDC features) in multichannel EEG in the present study infer that information flow from different parts of the brain to the others with different direct paths plays an important role in differentiation of right and left hand MI tasks. As results shown in Figure 4, differential patterns of connectivity in GPDC method is around the motor areas and frontal and parietal areas. Moreover, the accuracies achieved with our approach (84%) is higher than other method that uses EEG features from individual channels (70%) [ 48 ] in the same database.

Previous studies have shown that when a person is performing imagining the left or right hand movements, the mu and beta1 rhythms of EEG signals as a neural oscillation of brain electrophysiological activity at the sensorimotor area are (de)synchronized [ 8 - 10 , 13 , 48 , 62 , 63 ]. Thus, this study on MI hand movement pattern discrimination is in accordance with previous results.

To mention a limitation of our study, we believe that the performance of a multi-modal BCI system based on EEG and near-infrared spectroscopy (NIRS) might improve accuracy. Complementary, information measured by these methods is capable to improve the performance of either method and finally the performance of the system of discrimination of right and left hand MI tasks might be improving.

Conclusion

This study addresses a new method based on effective connectivity quantified with GPDC method and a hierarchical machine learning structure methods for discrimination of left and right hand MI tasks from EEG signals. Results indicate that exploring causal dependencies between brain regions of subjects using directed information flow plays an important role and has potential discriminative value. This new system could reach the accuracy of 84% on the MI EEG data of 29 subjects within Mu-beta1 frequency band.

Acknowledgement

We would like to thank Medicine Shahid Beheshti University of Medical Sciences and Science and Research Branch, Islamic Azad University due to their support.

Authors’ Contribution

A. Maghsoudi and A. Shalbaf conceived the idea. Introduction of the paper was written by A. Shalbaf. A. Maghsoudi and A. Shalbaf gather the images and the related literature and also help with writing of the related works. The method implementation was carried out by A. Maghsoudi. Results and Analysis was carried out by A. Maghsoudi and A. Shalbaf. The research work was proofread and supervised by A. Maghsoudi and A. Shalbaf. All the authors read, modified, and approved the final version of the manuscript.

Ethical Approval

The Ethics Committee of Shahid Beheshti University of Medical Sciences approved the protocol of the study (Ethic code: IR.SBMU.MSP.REC.1398.851).

Informed consent

Dataset was recorded in Technical University of Berlin and are conducted according to the declaration of Helsinki and informed consent was taken from each participant.

Conflict of Interest

None

References

1. McFarland DJ, Wolpaw JR. Brain-computer interface operation of robotic and prosthetic devices. Computer. 2008; 41(10):52-56. DOI
2. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002; 113(6):767-91. DOI | PubMed
3. Finke A, Lenhardt A, Ritter H. The MindGame: a P300-based brain-computer interface game. Neural Netw. 2009; 22(9):1329-33. DOI | PubMed
4. Millán Jdel R, Renkens F, Mouriño J, Gerstner W. Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Trans Biomed Eng. 2004; 51(6):1026-33. DOI | PubMed
5. Shalbaf A, Saffar M, Sleigh JW, Shalbaf R. Monitoring the Depth of Anesthesia Using a New Adaptive Neurofuzzy System. IEEE J Biomed Health Inform. 2018; 22(3):671-77. DOI | PubMed
6. Afshani F, Shalbaf A, Shalbaf R, Sleigh J. Frontal-temporal functional connectivity of EEG signal by standardized permutation mutual information during anesthesia. Cogn Neurodyn. 2019; 13(6):531-40. Publisher Full Text | DOI | PubMed
7. Shalbaf A, Shalbaf R, Saffar M, Sleigh J. Monitoring the level of hypnosis using a hierarchical SVM system. J Clin Monit Comput. 2020; 34(2):331-8. DOI | PubMed
8. Kim C, Sun J, Liu D, Wang Q, Paek S. An effective feature extraction method by power spectral density of EEG signal for 2-class motor imagery-based BCI. Med Biol Eng Comput. 2018; 56(9):1645-58. DOI | PubMed
9. Bascil MS, Tesneli AY, Temurtas F. Spectral feature extraction of EEG signals and pattern recognition during mental tasks of 2-D cursor movements for BCI using SVM and ANN. Australas Phys Eng Sci Med. 2016; 39(3):665-76. DOI | PubMed
10. Athif M, Ren H. WaveCSP: a robust motor imagery classifier for consumer EEG devices. Australas Phys Eng Sci Med. 2019; 42(1):159-168. DOI | PubMed
11. Shalbaf A, Maghsoudi A. Mental Arithmetic Task Recognition Using Effective Connectivity and Hierarchical Feature Selection from EEG Signals. Basic and Clinical Neuroscience. 2020.
12. Mahmoudi M, Shamsi M. Multi-class EEG classification of motor imagery signal by finding optimal time segments and features using SNR-based mutual information. Australas Phys Eng Sci Med. 2018; 41(4):957-72. DOI | PubMed
13. Park SH, Lee D, Lee SG. Filter Bank Regularized Common Spatial Pattern Ensemble for Small Sample Motor Imagery Classification. IEEE Trans Neural Syst Rehabil Eng. 2018; 26(2):498-505. DOI | PubMed
14. Shin Y, Lee S, Lee J, Lee HN. Sparse representation-based classification scheme for motor imagery-based brain-computer interface systems. J Neural Eng. 2012; 9(5):056002. DOI | PubMed
15. Sreeja SR, Samanta D, Sarma M. Weighted sparse representation for classification of motor imagery EEG signals. 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE: Berlin, Germany; 2019.
16. Sun HW, Fu YF, Xiong X, Yang J, Liu CW, Yu ZT. Identication of EEG induced by motor imagery based on Hilbert Huang transform. Acta Automatica Sinica. 2015; 41(9):1686-92.
17. Park C, Looney D, Naveed R, Ahrabian A, Mandic DP. Classification of motor imagery BCI using multivariate empirical mode decomposition. IEEE Trans Neural Syst Rehabil Eng. 2013; 21(1):10-22. DOI | PubMed
18. Zheng Y, Xu G. Quantifying mode mixing and leakage in multivariate empirical mode decomposition and application in motor imagery-based brain-computer interface system. Med Biol Eng Comput. 2019; 57(6):1297-311. DOI | PubMed
19. Boostani R, Moradi MH. A new approach in the BCI research based on fractal dimension as feature and Adaboost as classifier. J Neural Eng. 2004; 1(4):212-7. DOI | PubMed
20. Rodríguez-Bermúdez G, García-Laencina PJ. Automatic and adaptive classification of electroencephalographic signals for brain computer interfaces. J Med Syst. 2012; 36(1):S51-63. DOI | PubMed
21. Lang EW, Tomé AM, Keck IR, Górriz-Sáez JM, Puntonet CG. Brain connectivity analysis: a short survey. Comput Intell Neurosci. 2012;1-21. Publisher Full Text | DOI | PubMed
22. Seth AK. A MATLAB toolbox for Granger causal connectivity analysis. J Neurosci Methods. 2010; 186(2):262-73. DOI | PubMed
23. García-Laencina PJ, Rodríguez-Bermudez G, Roca-Dorda J. Exploring dimensionality reduction of EEG features in motor imagery task classification. Expert Syst Appl. 2014; 41(11):5285-95. DOI
24. Ince NF, Arica S, Tewfik A. Classification of single trial motor imagery EEG recordings with subject adapted non-dyadic arbitrary time-frequency tilings. J Neural Eng. 2006; 3(3):235-44. DOI | PubMed
25. Ruan J, Wu X, Zhou B, Guo X, Lv Z. An Automatic Channel Selection Approach for ICA-Based Motor Imagery Brain Computer Interface. J Med Syst. 2018; 42(12):253. DOI | PubMed
26. Asensio-Cubero J, Gan JQ, Palaniappan R. Multiresolution analysis over graphs for a motor imagery based online BCI game. Comput Biol Med. 2016; 68:21-6. DOI | PubMed
27. Bhattacharyya S, Sengupta A, Chakraborti T, Konar A, Tibarewala DN. Automatic feature selection of motor imagery EEG signals using differential evolution and learning automata. Med Biol Eng Comput. 2014; 52(2):131-9. DOI | PubMed
28. Miao M, Wang A, Liu F. A spatial-frequency-temporal optimized feature sparse representation-based classification method for motor imagery EEG pattern recognition. Med Biol Eng Comput. 2017; 55(9):1589-603. DOI | PubMed
29. Kirar JS, Agrawal RK. Relevant Feature Selection from a Combination of Spectral-Temporal and Spatial Features for Classification of Motor Imagery EEG. J Med Syst. 2018; 42(5):78. DOI | PubMed
30. Schlögl A, Lee F, Bischof H, Pfurtscheller G. Characterization of four-class motor imagery EEG data for the BCI-competition 2005. J Neural Eng. 2005; 2(4):L14-22. DOI | PubMed
31. Subasi A, Erçelebi E. Classification of EEG signals using neural network and logistic regression. Comput Methods Programs Biomed. 2005; 78(2):87-99. DOI | PubMed
32. Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B. A review of classification algorithms for EEG-based brain-computer interfaces. J Neural Eng. 2007; 4(2):R1-13. DOI | PubMed
33. Gupta G, Bhatnagar M, Ghosh S, Sinha R. Design of control system for motor imagery based neuro-aid application. Biomed Eng Appl Basis Commun. 2019; 31(04):1950031. DOI
34. Arvaneh M, Guan C, Ang KK, Quek C. Optimizing the channel selection and classification accuracy in EEG-based BCI. IEEE Trans Biomed Eng. 2011; 58(6):1865-73. DOI | PubMed
35. Siuly S, Li Y. Improving the separability of motor imagery EEG signals using a cross correlation-based least square support vector machine for brain-computer interface. IEEE Trans Neural Syst Rehabil Eng. 2012; 20(4):526-38. DOI | PubMed
36. Al-Faiz MZ, Al-hamadani AA. Implementation of EEG signal processing and decoding for two-class motor imagery data. Biomed Eng Appl Basis Commun. 2019; 31(04):1950028. DOI
37. Rodrigues PG, Filho CAS, Attux R, Castellano G, Soriano DC. Space-time recurrences for functional connectivity evaluation and feature extraction in motor imagery brain-computer interfaces. Med Biol Eng Comput. 2019; 57(8):1709-25. DOI | PubMed
38. He L, Hu D, Wan M, Wen Y, VonDeneen KM, Zhou M. Common Bayesian network for classification of EEG-based multiclass motor imagery BCI. IEEE Trans Syst Man Cybern Syst. 2015; 46(6):843-54. DOI
39. Sinha RK, Ghosh S. Jaya based ANFIS for monitoring of two class motor imagery task. IEEE Access. 2016; 4:9273-82. DOI
40. Jafarifarmand A, Badamchizadeh MA, Khanmohammadi S, Nazari MA, Tazehkand BM. A New Self-Regulated Neuro-Fuzzy Framework for Classification of EEG Signals in Motor Imagery BCI. IEEE Trans Fuzzy Syst. 2017; 26(3):1485-97. DOI
41. Hsu WY. Enhancing the performance of motor imagery EEG classification using phase features. Clin EEG Neurosci. 2015; 46(2):113-8. DOI | PubMed
42. Miao M, Zeng H, Wang A, Zhao C, Liu F. Discriminative spatial-frequency-temporal feature extraction and classification of motor imagery EEG: An sparse regression and Weighted Naïve Bayesian Classifier-based approach. J Neurosci Methods. 2017; 278:13-24. DOI | PubMed
43. Jin Z, Zhou G, Gao D, Zhang Y. EEG classification using sparse Bayesian extreme learning machine for brain computer interface. Neural Comput Appl. 2018; 32(11):1-9. DOI
44. Jiao Y, Zhang Y, Chen X, Yin E, Jin J, Wang X, Cichocki A. Sparse Group Representation Model for Motor Imagery EEG Classification. IEEE J Biomed Health Inform. 2019; 23(2):631-41. DOI | PubMed
45. Tabar YR, Halici U. A novel deep learning approach for classification of EEG motor imagery signals. J Neural Eng. 2017; 14(1):016003. DOI | PubMed
46. Lu N, Li T, Ren X, Miao H. A Deep Learning Scheme for Motor Imagery Classification based on Restricted Boltzmann Machines. IEEE Trans Neural Syst Rehabil Eng. 2017; 25(6):566-76. DOI | PubMed
47. Zhang Z, Duan F, Sole-Casals J, Dinares-Ferran J, Cichocki A, Yang Z, Sun Z. A novel deep learning approach with data augmentation to classify motor imagery signals. IEEE Access. 2019; 7:15945-54. DOI
48. Shin J, von Luhmann A, Blankertz B, Kim DW, Jeong J, Hwang HJ, Muller KR. Open Access Dataset for EEG+NIRS Single-Trial Classification. IEEE Trans Neural Syst Rehabil Eng. 2017; 25(10):1735-45. DOI | PubMed
49. Astolfi L, Cincotti F, Mattia D, Marciani MG, Baccala LA, De Vico Fallani F, Salinari S, Ursino M, Zavaglia M, Ding L, Edgar JC, Miller GA, He B, Babiloni F. Comparison of different cortical connectivity estimators for high-resolution EEG recordings. Hum Brain Mapp. 2007; 28(2):143-57. Publisher Full Text | DOI | PubMed
50. Granger CW. Investigating causal relations by econometric models and cross-spectral methods. Econometrica. 1969; 37:424-38.
51. Geweke JF. Measures of conditional linear dependence and feedback between time series. J Am Stat Assoc. 1984; 79(388):907-15. DOI
52. Baccalá LA, Sameshima K. Partial directed coherence: a new concept in neural structure determination. Biol Cybern. 2001; 84(6):463-74. DOI | PubMed
53. Baccala LA, Sameshima K, Takahashi DY. Generalized partial directed coherence in Digital Signal Processing. 15th International Conference on Digital Signal Processing. IEEE: Cardiff, UK; 2007. DOI
54. Korzeniewska A, Mańczak M, Kamiński M, Blinowska KJ, Kasicki S. Determination of information flow direction among brain structures by a modified directed transfer function (dDTF) method. J Neurosci Methods. 2003; 125(1-2):195-207. DOI | PubMed
55. Kamiński M, Ding M, Truccolo WA, Bressler SL. Evaluating causal relations in neural systems: granger causality, directed transfer function and statistical assessment of significance. Biol Cybern. 2001; 85(2):145-57. DOI | PubMed
56. Mullen T, Delorme A, Kothe C, Makeig S. An electrophysiological information flow toolbox for EEGLAB. University of California: San Diego; 2010.
57. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004; 134(1):9-21. DOI | PubMed
58. Spurrier JD. On the null distribution of the Kruskal–Wallis statistic. Nonparametric Statistics. 2003; 15(6):685-91. DOI
59. Ding C, Peng H. Minimum redundancy feature selection from microarray gene expression data. J Bioinform Comput Biol. 2005; 3(2):185-205. DOI | PubMed
60. Peng H, Long F, Ding C. Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell. 2005; 27(8):1226-38. DOI | PubMed
61. Bertsekas DP. Nonlinear programming. Journal of the Operational Research Society. 1997; 48(3)
62. Neuper C, Pfurtscheller G. Evidence for distinct beta resonance frequencies in human EEG related to specific sensorimotor cortical areas. Clin Neurophysiol. 2001; 112(11):2084-97. DOI | PubMed
63. Wolpaw JR, McFarland DJ, Vaughan TM. Brain-computer interface research at the Wadsworth Center. IEEE Trans Rehabil Eng. 2000; 8(2):222-6. DOI | PubMed