Journal of Biomedical Physics and Engineering

Current Issue

By Issue

By Author

By Subject

Author Index

Keyword Index

About Journal

Aims and Scope

Staff Members

Editorial Board

Publication Ethics

Indexing and Abstracting

Related Links

FAQ

Peer Review Process

News

An Unsupervised Feature Extraction Method based on CLSTM-AE for Accurate P300 Classification in Brain-Computer Interface Systems

Document Type : Original Research

Authors

1 School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran

2 Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran

10.31661/jbpe.v0i0.2207-1521

Abstract

Background: The P300 signal, an endogenous component of event-related potentials, is extracted from an electroencephalography signal and employed in Brain-computer Interface (BCI) devices.
Objective: The current study aimed to address challenges in extracting useful features from P300 components and detecting P300 through a hybrid unsupervised manner based on Convolutional Neural Network (CNN) and Long Short-term Memory (LSTM).
Material and Methods: In this cross-sectional study, CNN as a useful method for the P300 classification task emphasizes spatial characteristics of data. However, CNN and LSTM networks are combined to modify the classification system by extracting both spatial and temporal features. Then, the CNN-LSTM network was trained in an unsupervised learning method based on an autoencoder to improve Signal-to-noise Ratio (SNR) by extracting main components from latent space. To deal with imbalanced data, an Adaptive Synthetic Sampling Approach (ADASYN) is used and augmented without any duplication.
Results: The trained model, tested on the BCI competition III dataset, including two normal subjects, with an accuracy of 95% and 94% for subjects A and B in P300 detection, respectively. 
Conclusion: CNN-LSTM, was embedded into an autoencoder and introduced to simultaneously extract spatial and temporal features and manage the computational complexity of the method. Further, ADASYN as an augmentation method was proposed to deal with the imbalanced nature of data, which not only maintained feature space as before but also preserved anatomical features of P300. High-quality results highlight the suitable efficiency of the proposed method.

Highlights

Ramin Afrah (Google Scholar)

Zahra Amini (Google Scholar)

Keywords

References
  1. Jamil N, Belkacem AN, Ouhbi S, Lakas A. Noninvasive Electroencephalography Equipment for Assistive, Adaptive, and Rehabilitative Brain-Computer Interfaces: A Systematic Literature Review. Sensors (Basel). 2021;21(14):4754. doi: 10.3390/s21144754. PubMed PMID: 34300492. PubMed PMCID: PMC8309653.
  2. Amini Z, Abootalebi V, Sadeghi MT. A comparative study of feature extraction methods in P300 detection. 17th Iranian Conference of Biomedical Engineering (ICBME); Isfahan, Iran: IEEE; 2010. p. 1-4.
  3. Eastmond C, Subedi A, De S, Intes X. Deep learning in fNIRS: a review. 2022;9(4):041411. doi: 10.1117/1.NPh.9.4.041411. PubMed PMID: 35874933. PubMed PMCID: PMC9301871.
  4. Hasbulah MH, Jafar FA, Nordin MH. Fundamental of Electroencephalogram (EEG) Review for Brain-Computer Interface (BCI) System. Int Res J Eng Technol. 2019;6(5):1017-28.
  5. Kobler RJ, Hirata M, Hashimoto H, Dowaki R, Sburlea AI, Müller-Putz GR. Simultaneous decoding of velocity and speed during executed and observed tracking movements: an MEG study. 8th Graz Brain-Computer Interface Conference; Austria: Institute of Neural Engineering Graz BCI; 2019.
  6. Abiri R, Borhani S, Sellers EW, Jiang Y, Zhao X. A comprehensive review of EEG-based brain-computer interface paradigms. J Neural Eng. 2019;16(1):011001. doi: 10.1088/1741-2552/aaf12e. PubMed PMID: 30523919.
  7. Kundu S, Ari S. Brain-Computer interface speller system for alternative communication: a review. IRBM. 2022;43(4):317-24. doi: 10.1016/j.irbm.2021.07.001.
  8. Shojaedini SV, Morabbi S, Keyvanpour MR. A New Method to Improve the Performance of Deep Neural Networks in Detecting P300 Signals: Optimizing Curvature of Error Surface Using Genetic Algorithm. J Biomed Phys Eng. 2021;11(3):357-66. doi: 10.31661/jbpe.v0i0.975. PubMed PMID: 34189124. PubMed PMCID: PMC8236098.
  9. Saliasi E, Geerligs L, Lorist MM, Maurits NM. The relationship between P3 amplitude and working memory performance differs in young and older adults. PLoS One. 2013;8(5):e63701. doi: 10.1371/journal.pone.0063701. PubMed PMID: 23667658. PubMed PMCID: PMC3646823.
  10. Li L, Gratton C, Yao D, Knight RT. Role of frontal and parietal cortices in the control of bottom-up and top-down attention in humans. Brain Res. 2010;1344:173-84. doi: 10.1016/j.brainres.2010.05.016. PubMed PMID: 20470762. PubMed PMCID: PMC2900444.
  11. Oralhan Z. A new paradigm for region-based P300 speller in brain computer interface. IEEE Access. 2019;7:106618-27. doi: 10.1109/ACCESS.2019.2933049.
  12. Li Y, Nam CS, Shadden BB, Johnson SL. A P300-based brain–computer interface: Effects of interface type and screen size. Int J Hum Comput Interact. 2010;27(1):52-68. doi: 10.1080/10447318.2011.535753.
  13. Fazel-Rezai R, Ahmad W. P300-based brain-computer interface paradigm design. In: Recent advances in brain-computer interface systems. InTech; 2011.
  14. Lu Y, Bi L. EEG signals-based longitudinal control system for a brain-controlled vehicle. IEEE Trans Neural Syst Rehabil Eng. 2018;27(2):323-32. 10.1109/TNSRE.2018.2889483.
  15. Zhang Z, Yu X, Rong X, Iwata M. Spatial-temporal neural network for P300 detection. IEEE Access. 2021;9:163441-55. doi: 10.1109/ACCESS.2021.3132024.
  16. Qu J, Wang F, Xia Z, Yu T, Xiao J, Yu Z, Gu Z, Li Y. A novel three-dimensional P300 speller based on stereo visual stimuli. IEEE Trans Human-Machine Syst. 2018;48(4):392-9. doi: 10.1109/THMS.2018.2799525.
  17. Kim M, Kim J, Heo D, Choi Y, Lee T, Kim SP. Effects of Emotional Stimulations on the Online Operation of a P300-Based Brain-Computer Interface. Front Hum Neurosci. 2021;15:612777. doi: 10.3389/fnhum.2021.612777. PubMed PMID: 33767615. PubMed PMCID: PMC7987063.
  18. Bulat M, Karpman A, Samokhina A, Panov A. Playing a P300-based BCI VR game leads to changes in cognitive functions of healthy adults [Internet]. bioRxiv [Preprint]. 2020 [cited 2020 May 30]. Available from: https://www.biorxiv.org/content/10.1101/2020.05.28.118281v3.
  19. Rasheed S. A review of the role of machine learning techniques towards brain–computer interface applications. Mach Learn Knowl Extr. 2021;3(4):835-62. doi: 10.3390/make3040042.
  20. Alzahab NA, Apollonio L, Di Iorio A, Alshalak M, Iarlori S, Ferracuti F, et al. Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review. Brain Sci. 2021;11(1):75. doi: 10.3390/brainsci11010075. PubMed PMID: 33429938. PubMed PMCID: PMC7827826.
  21. Zhang X, Yao L, Wang X, Monaghan J, McAlpine D, Zhang Y. A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers. J Neural Eng. 2021;18(3):031002. doi: 10.1088/1741-2552/abc902. PubMed PMID: 33171452.
  22. Cortez SA, Flores C, Andreu-Perez J. Single-trial p300 classification using deep belief networks for a bci system. In: 2020 IEEE XXVII International Conference on Electronics, Electrical Engineering and Computing (INTERCON); Lima, Peru: IEEE: 2020. p. 1-4.
  23. Kshirsagar GB, Londhe ND. Improving Performance of Devanagari Script Input-Based P300 Speller Using Deep Learning. IEEE Trans Biomed Eng. 2019;66(11):2992-3005. doi: 10.1109/TBME.2018.2875024. PubMed PMID: 30307853.
  24. Liu M, Wu W, Gu Z, Yu Z, Qi F, Li Y. Deep learning based on batch normalization for P300 signal detection. 2018;275:288-97. doi: 10.1016/j.neucom.2017.08.039.
  25. Lu Z, Gao N, Liu Y, Li Q. The detection of p300 potential based on deep belief network. In: 2018 11th international congress on image and signal processing, biomedical engineering and informatics (CISP-BMEI); Beijing, China: IEEE; 2018. p. 1-5.
  26. Shan H, Liu Y, Stefanov TP. A Simple Convolutional Neural Network for Accurate P300 Detection and Character Spelling in Brain Computer Interface. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18); IJCAI; 2018. p. 1604-10.
  27. Delgado JMC, Achanccaray D, Villota ER, Chevallier S. Riemann-Based Algorithms Assessment for Single- and Multiple-Trial P300 Classification in Non-Optimal Environments. IEEE Trans Neural Syst Rehabil Eng. 2020;28(12):2754-61. doi: 10.1109/TNSRE.2020.3043418. PubMed PMID: 33296306.
  28. Wu Q, Zhang Y, Liu J, Sun J, Cichocki A, Gao F. Regularized Group Sparse Discriminant Analysis for P300-Based Brain-Computer Interface. Int J Neural Syst. 2019;29(6):1950002. doi: 10.1142/S0129065719500023. PubMed PMID: 30880525.
  29. Bianchi L, Liti C, Liuzzi G, Piccialli V, Salvatore C. Improving P300 Speller performance by means of optimization and machine learning. Ann Oper Res. 2022;312:1221-59. doi: 10.1007/s10479-020-03921-0.
  30. Srimaharaj W, Chaisricharoen R. A novel processing model for P300 brainwaves detection. J Web Eng. 2021;20(8):2545-70. doi: 10.13052/jwe1540-9589.20815.
  31. He H, Bai Y, Garcia EA, Li S. ADASYN: Adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence); Hong Kong: IEEE; 2008. p. 1322-8.
  32. Singari RM, Kankar PK, editors. Music Generation Using LSTM and Its Comparison with Traditional Method. In: Advanced Production and Industrial Engineering. IOS Press; 2022.
  33. Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, et al. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data. 2021;8(1):53. doi: 10.1186/s40537-021-00444-8. PubMed PMID: 33816053. PubMed PMCID: PMC8010506..
  34. Yu J, Liu X, Ye L. Convolutional long short-term memory autoencoder-based feature learning for fault detection in industrial processes. IEEE Trans Instrum Meas. 2020;70:1-5. doi: 10.1109/TIM.2020.3039614.
  35. Sarraf J, Pattnaik PK. A study of classification techniques on P300 speller dataset. Mater Today Proc. 2023;80(3):2047-50. doi: 10.1016/j.matpr.2021.06.110.
  36. Ghazikhani H, Rouhani M. A deep neural network classifier for P300 BCI speller based on Cohen’s classtime-frequency distribution. Turkish J Electr Eng Comput Sci. 2021;29(2):1226-40. doi: 10.3906/elk-2005-201.
  37. Ditthapron A, Banluesombatkul N, Ketrat S, Chuangsuwanich E, Wilaiprasitporn T. Universal joint feature extraction for P300 EEG classification using multi-task autoencoder. IEEE Access. 2019;7:68415-28. doi: 10.1109/ACCESS.2019.2919143.
  38. Lee T, Kim M, Kim SP. Improvement of P300-Based Brain-Computer Interfaces for Home Appliances Control by Data Balancing Techniques. Sensors (Basel). 2020;20(19):5576. doi: 10.3390/s20195576. PubMed PMID: 33003367. PubMed PMCID: PMC7582676.
Journal of Biomedical Physics and Engineering
Volume 14, Issue 6
68
December 2024
Pages 579-592
Files
Share
How to cite
Statistics