Introduction

The Brain-Computer Interface (BCI) enables artificial communication and control between the human brain and a computer. In BCI systems, the user’s target is detected by analyzing electrophysiological and other brain signals [ 1 ]. Various techniques, such as Electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI), are employed to record brain activity in BCI research. EEG signal acquisition from the scalp is commonly utilized in BCI studies, due to its cost-effectiveness and relative simplicity [ 2 ]. These signals contain Steady-State Visual Evoked Potential (SSVEP), Event-Related (de) Synchronization (ERD/ERS) [ 3 ], Event-Related Potential (ERP), and motion-onset Visual Evoked Potential (mVEP) [ 4 ]. SSVEP, a periodic neural response, is detected from the occipital region [ 5 ]. To record SSVEP, the user is instructed to focus their gaze on multiple flashing stimuli that alternate at different frequencies. The stimulation frequency and a few of its harmonics provide the basis for the SSVEP response.

The SSVEP has become an auspicious method for practical BCI implementations because of its high Information Transfer Rate (ITR), simplicity of the system, and short training time [ 6 ]. Nevertheless, these systems face problems, such as fatigue caused by long-term exposure to flicker stimuli [ 7 ] and different sensitivity levels of users to flicker frequencies. Therefore, frequency detection, an essential challenge commonly associated with SSVEP signals, necessitates the use of robust and effective methods to improve performance.

Some methods have been recently proposed for detecting SSVEP frequency [ 8 ], including Power Spectral Density Analysis (PSDA) [ 9 ], Least Absolute Shrinkage and Selection Operator (LASSO) [ 10 ], Canonical Correlation Analysis (CCA) [ 11 ], Multiway Canonical Correlation Analysis (MwayCCA) [ 12 ], L1-regularized Multiway Canonical Correlation Analysis (L1-MCCA) [ 13 ], Multiset Canonical Correlation Analysis (MsetCCA) [ 12 ], and Multivariate Linear Regression (MLR) [ 8 ]. The CCA method, aimed to establish connections between two sets: recorded and reference signals. While PSDA is sensitive to noise and demands a larger time window for classification, CCA proves more beneficial than traditional frequency methods [ 12 ]. MwayCCA can optimize the reference signal using the maximum correlation between the multidimensional EEG signal, including multiple channels, and the artificial reference signal [ 12 ]; in addition, it necessitates a reference signal for its processing procedure, which is artificially constructed using sine-cosine waves. The reference signal is then mixed with the recorded signal to create an ideal reference signal. Given that CCA uses artificial reference signals that do not reflect the SSVEP features in EEG [ 14 ], MsetCCA learns real reference signals with multiple EEG sets to improve recognition accuracy. The MsetCCA method employs an improved approach to generating a reference signal by leveraging common features among recorded signals for each subject and frequency, resulting in a more natural signal than artificial sine and cosine signals [ 15 ]. MLR, a recent method proposed for identifying SSVEP features, contrasts with the MsetCCA approach by demonstrating superior efficiency within a shorter time window [ 16 ].

There are two categories into which frequency detection techniques can be divided. The first point of view is the methods’ reliance on training, while the reference signal serves for the second point of view. Methods that require training are supervised, such as L1-MCCA, MsetCCA, and MLR, and methods that do not require training are unsupervised, such as PSDA, LASSO, and CCA. From the reference signal, some techniques—like PSDA and MLR—operate apart from it, whereas the remaining techniques depend on it. The efficiency of frequency detection methods is important because the system’s accuracy will increase, indicating its proper operation. Therefore, many studies seek to utilize approaches with superior performance in this field. Neghabi et al. [ 17 ] found that MLR, MsetCCA, and Common Feature Analysis (CFA) algorithms outperformed CCA, LASSO, and L1-MCCA methods. Signal recording protocols and frequency stimuli numbers significantly influence SSVEP detection accuracy and ITR. Zhang et al. [ 18 ] examined the impacts of the SSVEP-BCI stimulus number in an Augmented Reality (AR), utilizing various stimulus targets and paradigms. In their study, which compared CCA, FBCCA, and TRCA methods, the results demonstrated a decrease in the accuracy of SSVEP recognition with an increasing number of stimuli.

When employing frequency recognition methods, it’s vital to take into account the characteristics of the dataset. Factors like the number of frequencies and the presence of harmonics at the stimulation frequencies directly impact recognition accuracy. Choosing methods less influenced by dataset characteristics helps recognition accuracy regardless of these variations. This study aimed to answer the following questions: 1) whether increasing or decreasing the number of stimuli can affect output performance, 2) does the existence of frequency harmonics affect recognition accuracy?, and 3) is it possible to determine a recognition method that is less sensitive to different characteristics of the data?

Material and Methods

This analytical study employed two datasets with different characteristics in terms of the number of frequencies and frequency harmonics. Five frequency recognition methods were identified and implemented to assess the impact of the number of frequencies and frequency harmonics on these datasets. The methods have been used in this study include: 1- CCA, 2- LASSO, 3- MsetCCA, 4- MLR, 5- L1-MCCA. The results are evaluated using two criteria: Accuracy and ITR. Figure 1 shows a block diagram of the stimuli numbers and harmonics.

Figure 1. Block diagram of a process in the (a) stimuli numbers and (b) frequency harmonics. (CCA: Canonical Correlation Analysis, LASSO: Least Absolute Shrinkage and Selection Operator, MLR: Multivariate Linear Regression, MCCA: Multiway Canonical Correlation Analysis, L1-MCCA: L1-regularized Multiway Canonical Correlation Analysis, ITR: Information Transfer Rate)

Dataset

RIKEN-SSVEP-4 dataset

The RIKEN-SSVEP-4 dataset, recorded at the Riken laboratory, Institute of Physical and Chemical Research, Japan, comprises SSVEP signals from eight channels collected from ten participants. Subjects were positioned 60 cm away from a 17-inch CRT monitor in a shielded room. Stimulation frequencies of 5.75, 7.75, 8.75, and 9.75 Hz were used. Participants were instructed to focus on each stimulus frequency for 4 seconds. Each subject completed 20 experimental runs, resulting in a total of 80 trials (4 and 20 trials per run and frequency, respectively) [ 19 ].

SEMNAN-SSVEP-21 dataset

The SEMNAN-SSVEP-21 dataset was recorded in the Biological Signal Recording and Processing Laboratory at Semnan University, Semnan, Iran. This dataset included ten participants aged between 22 and 33 years, all of whom were inexperienced with BCI systems. EEG signals were acquired using the Bayamed electroencephalograph (EEG.V.16.24, Bayamed, Iran) at a sampling rate of 250 Hz, specifically from the O_z channel. Participants were instructed to sit in a comfortable position in front of a 15.6-inch LED screen with a refresh rate of 60 Hz and maintain steady fixation without any movement. The visual stimulation pattern was selected by a 10 cm-wide flashing circle in the center of the screen. Ten experimental sessions were conducted, each displaying stimuli ranging from 6 to 16 Hz with a frequency step of 0.5 Hz, totaling 21 frequencies. Each stimulus was presented for 8 seconds with a 5-second break between stimuli [ 20 ]. In this study, two subsets were extracted from this data set with 16 frequencies, with and without harmonics. The stimulation frequency of SEMNAN-SSVEP-16 without harmonic is 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10, 10.5, 11, 11.5, 12.5, 13.5, 14.5, 15.5 and the stimulation frequency of SEMNAN-SSVEP-16 with harmonic is 6, 6.5, 7, 7.5, 8, 8.5, 9, 12, 12.5, 13, 13.5,14, 14.5, 15, 15.5, 16.

CCA

Canonical correlation analysis is a statistical method that maximizes the similarity between two data sets. For the two vectors of random variables $X=(X_1,...,X_n)$ and $Y=(Y_1,...,Y_m)$ that there is a correlation between their variables, the canonical correlation analysis method will find the linear combinations between $Xi$ and $Y_i$ with the strongest correlation. X, a test data set, includes EEG signals from several channels with T time points in each channel. Y, a pre-constructed reference signal, is formed by a series of sine cosine waves as equation (1) [ 21 ]:

$Y=\left[\begin{array}{c}sin\left(\mathrm{2\pi }\right(f_i\left)t\right)\\ cos\left(\mathrm{2\pi }\right(f_i\left)t\right)\\ .\\ .\\ .\\ sin\left(\mathrm{2\pi }\right(N{f}_i\left)t\right)\\ cos\left(\mathrm{2\pi }\right(N{f}_i\left)t\right)\end{array}\right],t=\frac{1}{F_s},\frac{2}{F_s},...,\frac{T}{F_s}$ (1)

where $f_i$ denotes the $i^{\mathrm{th}}$ stimulus frequency, N is the number of harmonics, F_s is the sampling rate, and T represents the number of sampling points. CCA can find a pair of linear transformations W_y and W_x that maximize the correlation between $y=Y^T{W}_y$ and $x=X^T{W}_x$. For this purpose, by considering two conditions according to equations (2) and (3) [ 21 ], the total correlation is obtained by solving the optimization problem of equation (4) [ 21 ].

$E\left[x{x}^T\right]=E\left[x^{T}x\right]=E[W_x^{T}XX{W}_x=1]$ (2)

$E\left[y{y}^T\right]=E\left[y^{T}y\right]=E[W_y^{T}YY{W}_y=1]$ (3)

${\rho }_i={\rho }_{W_x,W_y}\mathrm{(x,y)}=\frac{E\left[x^{T}y\right]}{\sqrt{E\left[x^{T}x\right]E\left[y^{T}y\right]}}=\frac{E\left[W_x^{T}X{Y}_T{W}_y\right]}{\sqrt{E\left[W_x^{T}XX{W}_x\right]E\left[W_y^{T}YY{W}_y\right]}}$

$O=\underset{i}{\max }{\rho }_i,i=1,2,...,i$ (4)

Where ${\rho }_i$ is the correlation coefficient and i is the number of stimulation frequencies. After finding the value of ${\rho }_i$ for all stimulation frequencies, the frequency of the maximum correlation coefficient will be set as the target frequency [ 21 ].

LASSO

This method is used to detect the SSVEP response of the EEG signal, with higher output performance than the CCA method. Each EEG trial supposes that SSVEPs for the response $YϵR^{\mathrm{n}}$ are standard linear regression in equation (5) [ 10 ].

$Y=\mathrm{X\beta }+\epsilon$ (5)

Where $X=(X_1,...,X_p)$ denotes a $n\times p$ design matrix, ε shows a noise vector with the zero mean and constant variance, and y is a $n\times 1$ vector. The LASSO estimation is obtained from equation (6) [ 10 ]. In this equation, ${\Vert .\Vert }_1$ and ${\Vert .\Vert }_2$ represent the l₁-norm and the l₂-norm, respectively. λ is a penalty parameter that encourages a sparse solution to minimize the noise:

$\hat{\beta }={\mathrm{argmin}}_{\beta }(\Vert \mathrm{Y-X\beta }{\Vert }_2^2+\lambda \Vert \beta {\Vert }_1)$ (6)

In order to develop the SSVEP detection model, the X matrix is defined as a set of symmetric square wave ($S_i$) that correspond to the frequencies of the stimuli. ($S_i$) can be decomposed into the Fourier series of harmonics shown in equation (7) [ 10 ].

$Y=\left[\begin{array}{c}sin\left(\mathrm{2\pi }\right(f_i\left)t\right)\\ cos\left(\mathrm{2\pi }\right(f_i\left)t\right)\\ .\\ .\\ .\\ sin\left(\mathrm{2\pi }\right(N{f}_i\left)t\right)\\ cos\left(\mathrm{2\pi }\right(N{f}_i\left)t\right)\end{array}\right],t=\frac{1}{F_s},\frac{2}{F_s},...,\frac{T}{F_s}$ (7)

$f_i$ and $F_s$ denote the $i^{\mathrm{th}}$ stimulus frequency and the sampling rate, respectively. Also, N and T are the number of harmonics and sampling points, respecyvely. The LASSO method calculates the $\hat{\beta }$ value between the EEG sand the reference signals to determine the contribution degree. The contribution degree for all channels is calculated in equation (8) [ 10 ] to classify the SSVEP signal and determine the frequency that the user has been focused on.

${\mathrm{CD}}_i=\frac{{\sum }_{\mathrm{K=1}}^M{\sum }_{\mathrm{J=1}}^{\mathrm{2K}}\left|{\beta }_{\mathrm{i,j}}^K\right|}{M}$ (8)

K and M equal the number of harmonics and channels, and ${\mathrm{CD}}_i$ is the contribution degree of the $i^{\mathrm{th}}$ square wave in the signal. The target frequency is indicated by the highest contribution degree [ 10 ].

MLR

Previous studies have shown that the MLR method has better results than the CCA method. This section introduces the MLR technique, which improves accuracy by extracting unique SSVEP features. The label matrix is created by labeling the training data used in this method. Following the PCA-assisted dimension reduction, the training data and label matrix are subjected to the MLR technique to identify the ideal subspace. The MLR method doesn’t use Sine and Cosine as the reference signals.

Suppose the training data is represented by equation (9) [ 8 ].

$X=[X_1,...,X_N]\in R^{\mathrm{D\times N}}$ (9)

Where D is the dimension of features (D = C channel × P time points), and N is the number of training samples. The training data are recorded in M independent stimulation frequencies. In the preprocessing step, the PCA is then implemented to reduce the dimension of the data, and the data are obtained as the equation (10) [ 8 ]:

$\tilde{X}=[{\tilde{X}}_1,...,{\tilde{X}}_N]\in R^{\mathrm{S\times N}}$ (10)

The label matrix is then constructed by Y. The label matrix of the training data is ${\tilde{X}}^{\mathrm{(i)}}$, belonging to class m, Equation (11) [ 8 ]:

$y^{\mathrm{(i)}}={[y_1,...,y_M]}^T,y_i=\{\begin{array}{c}\mathrm{1\; if\; j=m}\\ \mathrm{0\; if\; j\ne m}\end{array}$ (11)

Then, the label matrix is constructed by $Y=[y^{\mathrm{(1)}},...,y^{\mathrm{(N)}}]\in R^{\mathrm{M\times N}}$. The MLR method aimed to find the subspaces with the least sum of squares, as Equation (12) [ 8 ]:

$\underset{\mathrm{W,b}}{\min }\frac{1}{2}{\sum }_{\mathrm{i=1}}^N\Vert y^{\mathrm{(i)}}-(W^T{\tilde{X}}^{\mathrm{(i)}}+b){\Vert }_2^2$ (12)

$W[w_1,...,w_c]\in R^{\mathrm{S\times C}}$ denotes the projection matrix, and b is the model intercept. b is calculated with Equation (13) [ 8 ]:

$b=\frac{1}{\mathrm{N}}{\sum }_{\mathrm{i=1}}^N(y^{\mathrm{(i)}}-W^T{\tilde{X}}^{\mathrm{(i)}})$ (13)

Then, the training data is projected on the W, including the characteristics of the training data. After learning the projection matrix via MLR between the label matrix and training samples, the training data are projected onto a lower-dimensional space. PCA is used to reduce the dimensionality of test data, which is then projected to the space learned by MLR. Finally, the sub-space features extracted by the MLR are classified using the K-Nearest-Neighbor (k-NN) algorithm [ 8 ].

MsetCCA

The MsetCCA has recently been used to optimize reference signals using common components between multiple test signals and to increase the correlation between canonical variables from several sets of random variables with several distinct linear combinations, equation (14) [ 15 ]:

$X=[X_1,...,X_N]\in R^{\mathrm{D\times N}}$ (14)

Where D is the dimension of features (D = C channel × P time points), and N is the number of the training samples. To increase the correlation between the canonical variables, the method of maximum variance is used in equation (15) [ 15 ].

$\underset{W_P...W_N}{\max }\rho ={\sum }_{\mathrm{i\ne j}}^N{w}_i^T{C}_{\mathrm{ij}}{w}_j$

$\mathrm{s.t.}\frac{1}{N}{\sum }_{\mathrm{i=1}}^N{w}_i^T{C}_{\mathrm{ii}}=1$ (15)

Accordingly, $C_{\mathrm{ij}}=X_i{X}_j^T$ is the correlation matrix of the two sets, and ρ is the correlation coefficient. By using the Lagrange multiplier method, the increase of the Equation (15) transforms to the Equation (16) [ 15 ]:

$\mathrm{(R-S)w}=\mathrm{\rho Sw}$

$R=\left[\begin{array}{ccccc}{X}_1{X}_1^T& .& .& .& X_1{X}_N^T\\ .& .& & & .\\ .& & .& & .\\ .& & & .& .\\ X_N{X}_1^T& .& .& .& X_N{X}_N^T\end{array}\right],S=\left[\begin{array}{ccccc}{X}_1{X}_1^T& .& .& .& 0\\ .& .& & & .\\ .& & .& & .\\ .& & & .& .\\ 0& .& .& .& X_N{X}_N^T\end{array}\right],w=\left[\begin{array}{c}{w}_1\\ .\\ .\\ .\\ w_N\end{array}\right]$ (16)

$X_{\mathrm{1,m}},X_{\mathrm{2,m}},...,X_{\mathrm{N,m}}\in R^{\mathrm{(C\times P)}}\mathrm{(Cchannels\times Ppoint)}$ EEG signals recorded from N experimental training trials at the m-th stimulation frequency $f_m$. The MsetCCA is implemented for finding the multiple linear transforms $w_{\mathrm{1,m}},w_{\mathrm{2,m}},...,w_{\mathrm{N,m}}$ to increase the correlation between canonical variables ${\tilde{Z}}_{\mathrm{1,m}},{\tilde{Z}}_{\mathrm{2,m}},...{\tilde{Z}}_{\mathrm{N,m}}$ with associated spatial filters ${\tilde{Z}}_{\mathrm{i,m}}=w_{\mathrm{i,m}}^T{X}_{\mathrm{i,m}}\left(\mathrm{i=1,2,\dots ,N}\right)$. These canonical variables indicate the common features of several training datasets. The canonical variables are combined to form the optimized reference signal for the $f_m$ stimulation frequency. The optimized reference signal is as equation (17) [ 15 ]:

$Y_{\mathrm{m}}={[{\tilde{Z}}_{\mathrm{1,m}},{\tilde{Z}}_{\mathrm{1,m}},...{\tilde{Z}}_{\mathrm{N,m}}]}^T$ (17)

For each stimulation frequency $f_m$, the reference signal corresponding to that $Y_m$ is considered for calculating the maximum correlation coefficient with EEG signal [ 15 ].

L1-MCCA

This method is to use a function that automatically extracts features to optimize the reference signal in the SSVEP detection. To construct the SSVEP detection model, we consider a three-dimensional $X\in R^{\mathrm{n\times K\times I}}$ (Time × experiment × Channel) the EEG signal with a specific stimulus frequency that is recorded by multiple channels and a signal $Y\in R^{\mathrm{2N\times n}}$:

$Y=X\beta +\epsilon$ (18)

The optimization of the L1-MCCA problem is done in equation (19) [ 22 ]:

$w_1,w_3,v=\arg \underset{w_1,w_3,v}{\min }\frac{1}{2}\Vert X{\times }_1{w}_1^T{\times }_3{w}_3^T-v^{T}Y{\Vert }_2^2+{\lambda }_1\Vert w_1{\Vert }_1+{\lambda }_2\Vert v{\Vert }_1+{\lambda }_3\Vert w_3{\Vert }_1$

$\mathrm{St}.\Vert w_1{\Vert }_2=\Vert w_3{\Vert }_2=\Vert v{\Vert }_2=1$ (19)

Where $w_1\in R^I,w_3\in R^k,v\in R^{\mathrm{2N}}$ are projection’s vectors, and ${\lambda }_1,{\lambda }_2,{\lambda }_3$ are the adjustment parameters [ 22 ].

Evaluation criteria

In the present study, methods, such as CCA, MLR, LASSO, MsetCCA, and L1-MCCA, were implemented to investigate the effects of frequency harmonics and the number of frequencies. In the LASSO and L1-MCCA procedures, lambda was set to 0.5 and 0.02, respectively. The average classification accuracy was determined using the leave-one-out cross-validation method. Also, BCI performance was evaluated using the accuracy (20) and ITR (21) [ 19 ] for classification evaluation:

$\mathrm{Accuracy}=\mathrm{TP}/(\mathrm{TP}+\mathrm{TN})\times 100$ (20)

Where TP is true positive, TN is true negative

$\mathrm{ITR}=({\log }_2{N}_f+P{\log }_{2}P+(\mathrm{1-P}\left){\log }_2\right[\frac{\mathrm{1-P}}{N_f-1}\left]\right)\times \left(\frac{60}{T}\right)$ (21)

P is the classification accuracy, $N_f$ is number of frequencies, and T is the time window (time window: 0.5 to 4 seconds).

Results

Latency

The visual delay among the SSVEP responses and stimulus is critical for target recognition [ 16 , 20 ]. Furthermore, latency ensures that SSVEP responses can be found in the time window. Determining the optimal latency involves balancing accuracy and the total duration of the window. Therefore, the ideal latency provides high accuracy within the shortest timeframe. In this study, accuracy and ITR are utilized to determine a suitable latency. The CCA method was applied with latencies ranging from 0.03 to 0.4 seconds and a time step of 0.02 seconds. Subsequently, the latency was selected that has the highest accuracy and ITR for each subject. According to Figure 2, the optimal latency is 0.15 seconds, maximizing both accuracy and ITR.

Figure 2. Accuracy and ITR of a four-second time window for different latencies. The highest value in ITR is denoted by a red star. (ITR: Information Transfer Rate)

Data Validation

The average accuracy and ITR for different time windows from 0.5 to 4 seconds are shown in Figure 3(a) and (b), among all subjects using the 8-channel EEG signal of the RIKEN-SSVEP-4 data. According to Figure 3(a), the MLR has higher accuracy than the CCA and LASSO in all time windows. The MLR achieves higher performance in time windows of less than 1.5 seconds. The L1-MCCA method has better results than the other four methods during the 2- to 4- second time window. According to Figure 3(b), the time window for the highest ITR in various methods is different (CCA=2s; LASSO=1.5s; MCCA=1s; L1-MCCA=1s; MLR=1s). The highest ITR obtained by the MLR is 93.81 bits/min from a 1-second data length. The MLR, L1-MCCA, MCCA, CCA, and LASSO methods performed more efficiently during the 1-second time length, respectively. The RIKEN-SSVEP-4 data contains 4 stimulation frequencies compared to the SEMNAN-SSVEP-4 data at frequencies of 6, 8, 9, and 10 in Figure 4.

Figure 3. Average (a) accuracy and (b) ITR obtained from several methods in different time windows using RIKEN-SSVEP-4 data. (CCA: Canonical Correlation Analysis, LASSO: Least Absolute Shrinkage and Selection Operator, MCCA: Multiway Canonical Correlation Analysis, MLR: Multivariate Linear Regression, L1-MCCA: L1-regularized Multiway Canonical Correlation Analysis, ITR: Information Transfer Rate)

Figure 4. Average (a) accuracy and (b) ITR obtained from several methods in different time windows using SEMNAN-SSVEP-4 data. (CCA: Canonical Correlation Analysis, MCCA: Multiway Canonical Correlation Analysis, MLR: Multivariate Linear Regression, LASSO: Least Absolute Shrinkage and Selection Operator, L1-MCCA: L1-regularized Multiway Canonical Correlation Analysis, ITR: Information Transfer Rate)

Figure 4(a) and (b) show the average accuracy and ITR with time windows from 0.5 to 4 seconds among all subjects using SEMNAN-SSVEP-4 data. According to Figure 4 (a), the MLR method has higher accuracy and ITR than the other four methods over the time window of less than 1.5 seconds. CCA and LASSO methods have better results during the 1.5- to 4-second windows than the other three methods. According to Figure 4(b), the time window for the highest ITR in various methods is different (CCA=1.5s; LASSO=1.5s; MCCA=1.5s; L1-MCCA=1s; MLR=1s). The highest ITR obtained by the MLR is 51.27 bits/min from a 1-second data length. The MLR, CCA, LASSO, L1-MCCA, and MCCA methods, performed more efficiently during the 1-second time window, respectively. When comparing the results obtained from the SEMNAN-SSVEP-4 and RIKEN-SSVEP-4 datasets, it can be concluded that the data from Semnan is reliable. This conclusion is based on the high accuracy achieved by both datasets using a 1-second time window, where the MLR method yielded the best results.

Investigation of the effects of frequency harmonics on the efficiency of frequency recognition methods

Subsets of the SEMNAN-SSVEP-21 have been used to investigate the impacts of frequency harmonics. Figure 5(a) and (b) show the accuracy and ITR obtained from several state-of-the-art methods in different time windows using the SEMNAN-SSVEP-16 data without frequency harmonics. According to Figure 5(a), the MLR has the best results in accuracy in fewer than 2 seconds. The CCA method has the best results in the 2- to 4- second time window. According to Figure 5(b), the time window for the highest ITR in various methods is different (CCA=2.5s; LASSO=2.5s; MCCA=3s; L1-MCCA=2.5s; MLR=1.5s). The highest ITR obtained by the MLR method is 40 bits/min from a 1.5-second data length. The MLR, L1-MCCA, CCA, MCCA, and LASSO methods performed more efficiently during the 1-second window, respectively.

Figure 5. Average accuracy and ITR obtained from methods in different time windows using SEMNAN-SSVEP-16 data with harmonics and without it. (a) Average accuracy by SEMNAN-SSVEP-16 data without harmonics, (b) Average ITRs by SEMNAN-SSVEP-16 data without harmonics, (c) Average accuracy by SEMNAN-SSVEP-16 data with harmonics, (d) Average ITRs by SEMNAN-SSVEP-16 data with harmonics. (CCA: Canonical Correlation Analysis, msetCCA: multiset Canonical Correlation Analysis, MLR: Multivariate Linear Regression, LASSO: Least Absolute Shrinkage and Selection Operator, L1-MCCA: L1-regularized Multiway Canonical Correlation Analysis, ITR: Information Transfer Rate)

Figure 5(c) and (d) show the accuracy and ITR obtained from several state-of-the-art methods in different time windows using the SEMNAN-SSVEP-16 data with frequency harmonics (five frequency harmonics). According to Figure 5(c), the MLR has the best performance in terms of accuracy and ITR over the whole-time window. Figure 5(d) shows the ITRs across the subjects using different time windows. According to Figure 5(d), the time window for the highest ITR in various methods is different (CCA=2.5s; LASSO=2.5s; MCCA=3.5s; L1-MCCA=2.5s; MLR=1.5s). The highest ITR obtained by the MLR method is 35.47 bits/min from a 1.5-second time length. Similar to the order of results obtained from data without harmonics, MLR, L1-MCCA, CCA, MCCA, and LASSO methods performed more efficiently during the 1-second window, respectively.

The length of the time window is a critical factor in SSVEP-based BCIs as it determines the amount of data required for frequency recognition during each interval. However, the BCI system performance requires a trade-off between recognition accuracy and time window that will show with ITR [ 23 ]. Considering that the time window of 2.5 seconds in most methods has a higher ITR, this window length is used to compare the effect of frequency harmonic. In this regard, the amplitude of the accuracy difference between the two states (with harmonic and without harmonic) was evaluated. Table 1 shows the accuracy difference between the two states.

Investigation of the effects of the frequency number on the efficiency of the frequency recognition methods

The SEMNAN-SSVEP-16 data without harmonics were utilized to scrutinize the number of frequencies on the classification performance. The MLR was selected as the best-performing method, then the number of target stimuli (4-16 frequencies) gradually increases. Figure 6(a) and (b) show the accuracy, and ITR obtained MLR in a different frequency using the SEMNAN-SSVEP-16 data without frequency harmonics.

Figure 6. Average (a) accuracy and (b) ITR obtained from MLR method using SEMNAN-SSVEP-16 data without harmonic with a different number of frequencies. (ITR: Information Transfer Rate, MLR: Multivariate Linear Regression)

Figure 6 with a negative slope shows that increasing the number of stimulation frequencies leads to a decrease in performance because increasing the number of frequencies leads to increasing the number of classes, which reduces the accuracy. Initially, when increasing the time window to durations like 0.5, 1, or 1.5 seconds, the results exhibit closer proximity. However, as the time window further extends to durations, such as 2, 2.5, 3, 3.5, or 4 seconds, the results diverge, indicating that the number of frequencies has a notable impact on the output performance.

Discussion

This study aims to examine the influence of frequency harmonics and the number of stimuli on the accuracy of frequency recognition. Various methods, including CCA, MCCA, MLR, LASSO, and L1-MCCA, were employed and evaluated using two distinct datasets. The first dataset consisted of four frequencies, while the second dataset was divided into two subsets, each comprising 16 frequencies. Notably, the subsets differed in terms of frequency harmonics, with one subset containing harmonics and the other lacking them.

The frequency recognition accuracy of MLR decreased by 9% with an increase in the number of stimuli from 4 to 16 stimuli. The decreasing accuracy is consistent with the results of [ 18 ], which investigated the effect of the number of stimuli in AR-SSVEP on accuracy. By increasing 9 to 36 stimuli, the accuracy of frequency recognition decreased by 12.39%. In the present study, the new stimulation frequency is added to the existing frequency set. However, in reference [ 18 ], the datasets with varying numbers of frequencies have different frequency compositions.

The system performance will decrease if the stimulation frequencies include the base frequencies with their harmonics across all frequency recognition methods. However, in many studies, such as [ 24 , 25 ], the base frequency with their harmonics has been used as the stimulation frequency. Optimal performance output can be achieved by employing base frequencies without harmonics. The results of this study show that the method that has the least sensitivity to the presence of frequency harmonics is MLR. This method exhibits the least decline, at 8%.

Therefore, considering that the characteristic of the data affects the output, choosing a frequency recognition method that does not depend on the characteristics gives robust results. According to the results, not only does the MLR method outperform others in short time windows, but it also experiences smaller decreases in accuracy due to the presence of frequency harmonics compared to the other methods, indicating its lower susceptibility to frequency harmonics in stimulation frequencies. It is also less sensitive to the number of stimulation frequencies Therefore, this frequency recognition method exhibits minimal sensitivity to data characteristics and it is suggested as a suitable method for frequency recognition.

As described in section 2.2, the CCA method entails a straightforward computational procedure with relatively low time complexity, and a crucial factor often required in online BCI systems [ 26 ]. Under [ 27 ] and [ 28 ], no substantial difference exists between the results obtained using the CCA and filter bank CCA (FBCCA) that work without training data [ 29 ]. Hence, given its low computational cost, only CCA was employed in this study.

In SSVEP tasks, the utilization of the methods involving training, such as multi-way CCA, L1-MCCA, MsetCCA, and Task-related Component Analysis (TRCA) [ 30 ] can yield better classification accuracy in comparison to training-free techniques. This study investigated into L1-MCCA since it is a more powerful algorithm than multi-way CCA. The TRCA method is a training approach related to a given task from EEG signals recorded across multiple channels [ 31 ]. However, a limitation of this study is the use of a dataset recorded through only one channel. As the occipital area commonly displays the strongest SSVEP response, it is frequently utilized in SSVEP tasks [ 32 ]. The electrodes placed in the occipital locations (O₁, O₂, and O_z) are typically used in SSVEP-based BCI [ 33 , 34 ]. If multiple occipital channels were available, the impact of data characteristics on the TRCA method could be explored further.

Conclusion

This study demonstrated the important effects of frequency harmonics and the number of stimuli on frequency recognition methods. Two datasets that differed with distinct characteristics in a number of frequency and frequency harmonics were investigated. The results show increasing the number of stimuli can reduce recognition accuracy and ITR. Also, to improve system accuracy, the study also recommended avoiding using stimulation frequencies with their harmonics.

The MLR method, which is less sensitive to the characteristics of the data, is advised for frequency recognition in the future to improve accuracy.

Authors’ Contribution

M. Azadi and A. Maleki conceived and designed the study and revised the draft. M. Azadi conducted the analysis and wrote the first draft and revised the draft. A. Maleki led the revision of the draft and provided help and advice for writing. All authors contributed to editorial changes in the manuscript. All authors read and approved the final manuscript. All authors had complete access to all research data and assumed complete responsibility for the data integrity and accuracy of the data analysis.

Ethical Approval

Since this paper does not record human or animal data and relies only on publicly available datasets, obtaining ethical approval or other licenses is unnecessary.

Funding

The authors declare that they have no known financial interests.

Conflict of Interest

None

References

1. Chadaga K, Prabhu S, Sampathila N, Nireshwalya S, Katta SS, Tan RS, Acharya UR. Application of Artificial Intelligence Techniques for Monkeypox: A Systematic Review. Diagnostics (Basel).. 2023; 13(5):824. Publisher Full Text | DOI | PubMed
2. Azadimoghadam M, Maleki A. Fatigue Assessment using Frequency Features in SSVEP-based Brain-Computer Interfaces. Iran J Biomed Eng. 2022; 16(3):251-60. DOI
3. Pronina MV, Ponomarev VA, Poliakov YI, Martins-Mourao A, Plotnikova IV, Müller A, Kropotov YD. Event-related EEG synchronization and desynchronization in patients with obsessive-compulsive disorder. Psychophysiology. 2023; 60(12):e14403. DOI | PubMed
4. Amini MM, Shalchyan V. Designing a motion-onset visual evoked potential-based brain-computer interface to control a computer game. IEEE TG. 2023;1-10. DOI
5. Friðriksdóttir D, Andriyash Y. Exploring Attentional Neural Differences During an Oddball Paradigm: An SSVEP Study [dissertation].. University of Iceland; 2023.
6. Sadeghi S, Maleki A. Recent Advances in Hybrid Brain-Computer Interface Systems: A Technological and Quantitative Review. Basic Clin Neurosci. 2018; 9(5):373-88. Publisher Full Text | DOI | PubMed
7. Azadi Moghadam M, Maleki A. Fatigue factors and fatigue indices in SSVEP-based brain-computer interfaces: a systematic review and meta-analysis. Front Hum Neurosci. 2023; 17:1248474. Publisher Full Text | DOI | PubMed
8. Ziafati A, Maleki A. Genetic algorithm based ensemble system using MLR and MsetCCA methods for SSVEP frequency recognition. Med Eng Phys. 2023; 111:103945. DOI | PubMed
9. Ojha MK, Tiwari P, Choubey DK, Gupta D. Detection of SSVEP Frequency component using Filter Bank Approach for EEG Based BCI System. Neuro Quantology. 2022; 20(6):3533-40. DOI
10. Vahid F, Behboodi M, Mahnam A. Bichromatic visual stimulus with subharmonic response to achieve a high-accuracy SSVEP BCI system with low eye irritation. Biomedical Signal Processing and Control. 2023; 83:104629. DOI
11. Tong C, Wang H, Wang Y. Relation of canonical correlation analysis and multivariate synchronization index in SSVEP detection. Biomedical Signal Processing and Control. 2022; 73:103345. DOI
12. Li H, Xu G, Li Z, Zhang K, Zheng X, Du C, Han C, Kuang J, Du Y, Zhang S. A Precise Frequency Recognition Method of Short-Time SSVEP Signals Based on Signal Extension. IEEE Trans Neural Syst Rehabil Eng. 2023; 31:2486-96. DOI | PubMed
13. Nakanishi M, Wang Y, Chen X, Wang YT, Gao X, Jung TP. Enhancing Detection of SSVEPs for a High-Speed Brain Speller Using Task-Related Component Analysis. IEEE Trans Biomed Eng. 2018; 65(1):104-12. Publisher Full Text | DOI | PubMed
14. Jiao Y, Zhang Y, Jin J, Wang X. IEEE: Dalian, China; 2016.
15. Zhang Y, Zhou G, Jin J, Wang X, Cichocki A. Frequency recognition in SSVEP-based BCI using multiset canonical correlation analysis. Int J Neural Syst. 2014; 24(4):1450013. DOI | PubMed
16. Ziafati A, Maleki A. Fuzzy ensemble system for SSVEP stimulation frequency detection using the MLR and MsetCCA. J Neurosci Methods. 2020; 338:108686. DOI | PubMed
17. Neghabi M, Marateb HR, Mahnam A. Comparing Steady-State Visually Evoked Potentials Frequency Estimation Methods in Brain-Computer Interface With the Minimum Number of EEG Channels. Basic Clin Neurosci. 2019; 10(3):245-6. Publisher Full Text | DOI | PubMed
18. Zhang R, Xu Z, Zhang L, Cao L, Hu Y, Lu B, Shi L, Yao D, Zhao X. The effect of stimulus number on the recognition accuracy and information transfer rate of SSVEP-BCI in augmented reality. J Neural Eng. 2022; 19(3):36010. DOI | PubMed
19. Wang H, Zhang Y, Waytowich NR, Krusienski DJ, Zhou G, Jin J, Wang X, Cichocki A. Discriminative Feature Extraction via Multivariate Linear Regression for SSVEP-Based BCI. IEEE Trans Neural Syst Rehabil Eng. 2016; 24(5):532-41. DOI | PubMed
20. Sadeghi S, Maleki A. Character encoding based on occurrence probability enhances the performance of SSVEP-based BCI spellers. Biomedical Signal Processing and Control. 2020; 58:101888. DOI
21. Sadeghi S, Maleki A. Adaptive canonical correlation analysis for harmonic stimulation frequencies recognition in SSVEP-based BCIs. Turkish J Electr Eng Comput Sci. 2019; 27(5):3729-40. DOI
22. Zhang Y, Zhou G, Jin J, Wang M, Wang X, Cichocki A. L1-regularized Multiway canonical correlation analysis for SSVEP-based BCI. IEEE Trans Neural Syst Rehabil Eng. 2013; 21(6):887-96. DOI | PubMed
23. Da Cruz JN, Wan F, Wong CM, Cao T. Adaptive time-window length based on online performance measurement in SSVEP-based BCIs. Neurocomputing. 2015; 149:93-9. DOI
24. Chen X, Wang Y, Zhang S, Xu S, Gao X. Effects of stimulation frequency and stimulation waveform on steady-state visual evoked potentials using a computer monitor. J Neural Eng. 2019; 16(6):066007. DOI | PubMed
25. Floriano A, F Diez P, Freire Bastos-Filho T. Evaluating the Influence of Chromatic and Luminance Stimuli on SSVEPs from Behind-the-Ears and Occipital Areas. Sensors (Basel).. 2018; 18(2):615. Publisher Full Text | DOI | PubMed
26. Luo TJ. A comparative survey of SSVEP recognition algorithms based on template matching of training trials. International Journal of Intelligent Computing and Cybernetics. 2023; 16(1):46-67. DOI
27. Wong CM, Wang B, Wang Z, Lao KF, Rosa A, Wan F. Spatial Filtering in SSVEP-Based BCIs: Unified Framework and New Improvements. IEEE Trans Biomed Eng. 2020; 67(11):3057-72. DOI | PubMed
28. Zhang X, Qiu S, Zhang Y, Wang K, Wang Y, He H. Bidirectional Siamese correlation analysis method for enhancing the detection of SSVEPs. J Neural Eng. 2022; 19(4):46027. DOI | PubMed
29. Chen X, Wang Y, Gao S, Jung TP, Gao X. Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain-computer interface. J Neural Eng. 2015; 12(4):046008. DOI | PubMed
30. Huang J, Yang P, Xiong B, Wan B, Su K, Zhang ZQ. Latency Aligning Task-Related Component Analysis Using Wave Propagation for Enhancing SSVEP-Based BCIs. IEEE Trans Neural Syst Rehabil Eng. 2022; 30:851-9. DOI | PubMed
31. Hong J, Qin X. Signal processing algorithms for SSVEP-based brain computer interface: State-of-the-art and recent developments. J Intell Fuzzy Syst. 2021; 40(6):10559-73. DOI
32. Vialatte FB, Maurice M, Dauwels J, Cichocki A. Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Prog Neurobiol. 2010; 90(4):418-38. DOI | PubMed
33. Müller-Putz GR, Pfurtscheller G. Control of an electrical prosthesis with an SSVEP-based BCI. IEEE Trans Biomed Eng. 2008; 55(1):361-4. DOI | PubMed
34. Diez PF, Mut VA, Avila Perona EM, Laciar Leber E. Asynchronous BCI control using high-frequency SSVEP. J Neuroeng Rehabil. 2011; 8:39. Publisher Full Text | DOI | PubMed