Document Type : Original Research

Authors

1 MSc, Department of Biomedical Engineering and Medical Physics, Faculty of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran

2 PhD, Department of Biomedical Engineering and Medical Physics, Faculty of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Abstract

Background: Hand tremor is one of the consequences of MS disease degrading quality of patient’s life. Recently DBS is used as a prominent treatment to reduce this effect. Evaluation of this approach has significant importance because of the prevalence rate of disease.
Objective: The purpose of this study was the nonlinear analysis of tremor signal in order to evaluate the quantitative effect of DBS on reducing MS tremor and differentiating between them using pattern recognition algorithms.
Material and Methods: In this analytical study, nine features were extracted from the tremor signal. Through statistical analysis, the significance level of each feature was examined. Finally, tremor signals were categorized by SVM, weighted KNN and NN classifiers. The performance of methods was compared with an ROC graph.
Results: The results have demonstrated that dominant frequency, maximum amplitude and energy of the first IMF, deviation of the direct path, sample entropy and fuzzy entropy have the potential to create a significant difference between the tremor signals. The classification accuracy rate of tremor signals in three groups for Weighted KNN, NN and SVM with Gaussian and Quadratic kernels resulted in 95.1%, 93.2%, 91.3% and 88.3%, respectively.
Conclusion: Generally, nonlinear and nonstationary analyses have a high potential for a quantitative and objective measure of MS tremor. Weighted KNN has shown the best performance of classification with the accuracy of more than 95%. It has been indicated that DBS has a positive influence on reducing the MS tremor. Therefore, DBS can be used in the objective improvement of tremor in MS patients.

Keywords

Introduction

Multiple Sclerosis (MS) is one of the diseases which has affected a wide range of people in the present century. MS is a chronic central nervous system (brain and spinal cord) disease that impairs the transmission of neural signals [ 1 ]. Movement disorder is one of the common symptoms of the disease including side effects such as movement tremor and dysmetria [ 2 ]. These symptoms affect up to 56% of the upper limb and lead to a drastic decline in patients’ quality of life [ 3 ]. Because of the movement tremor and deviation of the direct path when approaching a target, patients use eye’s feedback to correct the position of the hand; however, the weakness of motor control system performance leads to hand deviation of the target, dysmetria [ 4 ]. It is not affected by the thalamic stimulation. Therefore, estimation of tremor intensity and dysmetria discrimination of tremor is difficult [ 2 , 5 ].

There has been no effective therapy found for MS so far, but there are techniques can relieve the symptoms of the disease. Deep Brain Stimulation (DBS) is one of the most potential therapies in which the electrodes are placed in selected regions of the brain. The area under the thalamus is permanently stimulated and has an effect to remove a part of the thalamus [ 1 ].

So far, numerous studies have reported a significant reduction in Parkinsonian tremor and Essential tremor after DBS, but little research has been done on the quantitative effect of DBS on the hand tremor of MS patients during goal-directed movements.

Kristian et al. [ 6 ] examined the effect of DBS and drug therapy on tremor treatment in Parkinson’s patients. They evaluated the difference of tremor frequency and amplitude before and after treatment with FFT analysis. Rissanen et al. [ 7 ] extracted the sample entropy, correlation dimension, recurrence rate and root mean square error (RMS) from the EMG and hand tremor acceleration signals of the Parkinson’s patients. According to the change percentage of each feature, they examined the effect of DBS. Esteki et al. [ 8 , 9 ] calculated mean and standard deviation of the distance from MS patients’ finger to the target with and without considering DBS during the back and forth movement of hands. They also calculated the deviation of the direct path and cumulative power as two features of hand movement signal. Esmailpour et al. [ 10 ] extracted threshold entropy, mean, variance and dominant frequency of the recorded signal from MS patient’s hand position. Results of this study have shown that entropy was not able to differentiate patients with DBS ON and OFF, due to the presence of dysmetria in the tremor signal.

The success rate of DBS on reducing MS tremor is qualitatively performed by physicians via clinical examinations [ 11 ]. These ways don’t have enough accuracy that is necessary to differentiate and quantitatively measure the severity of the disease and assess the effect of the therapeutic methods [ 12 ]. Therefore, the detection and quantification of tremor intensity are very important for the objective evaluation of treatment.

Tremor signal is nonlinear and nonstationary in nature, so a nonlinear analysis in order to investigate the DBS effect on tremor is implemented [ 5 ]. On the other hand, movement tremor of MS patients occurring at a frequency range of less than 7 Hz [ 13 ]. Visual examination of the tremor signal before and after applying DBS showed that there is no significant difference between the two signals requiring the extraction of appropriate features and classification techniques in order to find the difference between two signals. Detection of hand tremor before and after applying DBS is a pattern recognition problem which feature extraction is one of the most important steps. The quality of the selected features is very effective in determining the diagnostic accuracy. Appropriate features not only reduce classifier calculations but also decrease the cost of diagnosis [ 14 ].

In the next Section, a description of materials, tremor signal preprocessing, feature extraction, statistical analysis and classification are presented. Section 3, indicates results. Section 4 discusses the results, in detail. Finally, the conclusions are given in Section 5.

Material and Methods

Participants

In this analytical study, raw tremor data was provided from another research in the Surgical Center for movement disorders at the University of British Columbia, Canada. Six healthy subjects and seven MS patients participated in the experiment. The patients were randomly selected from the patients referred to Vancouver General hospitals, Canada using unilaterally or bilaterally DBS. They were with the average age of 47 years old and of both genders. The healthy subjects were randomly selected from the same hospital staff with the average age of 45 years and without any history of the neuromuscular disease [ 8 ].

Measurement procedure

The OptotrakTM motion analysis system was used in the mentioned research. A marker was installed on the tip of the index finger for the left and right hands of the participants. Two other markers were installed on the desk opposite the subjects. One of them was located at a distance of approximately 10 cm and the other one at 40 cm from the patient’s body. By back and forth movement of the hand, the farther marker, and then near marker was touched for several times. This test is similar to one of the maneuvers that neurologists use to examine patients’ hand tremor (Finger to Nose Test). The test is repeated at 10-minute intervals, three times for healthy subjects, and six times for patients (three times with DBS ON and three times with DBS OFF) [ 9 ]. The horizontal position of the index finger marker perpendicular to the line between the two fixed markers was considered as the tremor signal.

Data analysis

Figure 1 shows an overall flowchart of our analysis.

Figure 1. General structure for the tremor signal analysis.

Signal Preprocessing

In order to remove the bias, signal’s mean was removed. Butterworth lowpass filter with a cut-off frequency of 7 Hz was used to remove the effect of artifacts from the tremor signal.

Empirical Mode Decomposition (EMD) method was used to separate dysmetria from the tremor. EMD is a new method for analyzing nonlinear and non-stationary signals in the time-frequency domain that decomposes a complex signal into its constructive basis functions, called Intrinsic Mode Function (IMF) [ 15 ].

The time-frequency function of each IMF is specified by Hilbert Transform (HT) [ 16 ]. HT provides information on local variations of energy with a sufficient resolution that is useful in detecting events which are obscure in analyses such as FFT [ 17 ].

The time of dysmetria occurrence is detected by the subject’s movement signal. Then the frequency of dysmetria was obtained using HT from signal IMFs. By means of filtering methods, dysmetria is differentiated from the tremor signal. Therefore, Butterworth highpass filter with a cut-off frequency of 2 Hz was applied.

Feature Extraction

Nine features were extracted from the tremor signal. All features were computed for the right and left hand of healthy subjects and patients with DBS ON and OFF states (Figure 2).

Figure 2. Features were extracted from the tremor signal

Entropy

Entropy examines the degree of randomness and the disturbance of the signal and measures the complexity of the system. Reduction of entropy leads to more regular time series increasing the information of signal [ 18 ]. Three approximate methods were used to compute the entropy of signals: Approximate entropy(AppEn) [ 19 ], Sample entropy (SampEn) [ 20 ] and Fuzzy entropy (FuzzyEn) [ 21 ], which all are defined as the negative average natural logarithm of conditional probability in a time series of length N.

The AppEn is dependent on the length of the signal and counts each sequence as matching itself. In order to resolve the above problems, SampEn can be used [ 20 ]. FuzzyEn is a proper criterion for representing fuzziness and differentiating the complicated signals [ 22 ].

Wavelet energy

Wavelet transform is a linear time-frequency technique which allows for the local signal analysis and reveals high resolution in all frequencies. This technique is commonly considered for the nonstationary signals having large variations during a time period [ 23 ].

Wavelet decomposition is a technique that decomposes signal to “n” level. The sum of square of detailed wavelet transform coefficients in each level is considered as wavelet energy. Wavelet energy is defined as Eq 1 [ 24 ]:

Eng=i=1NAji2 (1)

Where Aji, N and j identify wavelet coefficients, the length of window and decomposition level, respectively.

Statistical features

The features of distance average to the direct path and deviation of the direct path have also been calculated for the tremor signal.

Features extracted from first IMF

The first IMF is the best presentation for high frequencies of the signal which is more related to the tremor signal. For the better analysis of tremor signal, the following features of the first IMF were studied [ 17 ].

Energy

Energy was calculated by dividing the square sum of the signal in each point over signal length.

Dominant frequency and maximum amplitude

Power Spectral Density(PSD) method was used to assess the dominant frequency (dominant freq) and peaks amplitude of time series. PSD analysis was performed using welch’s algorithm with Hamming windows of length 256 and set the overlap to 1/2 second between segments.

Statistical analysis

In this research, all features extracted are subjected to statistical analysis for identifying the significance level between different groups. Features were tested using Wilcoxon Signed Rank statistical test by determining the p-value between patients with DBS ON and OFF states. Mann-Whitney U test was used for comparing between DBS OFF patients with healthy groups and between DBS ON patients with healthy groups for the computed features. Repeated measure ANOVA non-parametric test was implemented to check the repeatability of the experiment for 3 trials of each subject.

Classification

After feature extraction step, tremor signal has been classified in three groups of healthy, patients with DBS ON and OFF. In this research, support vector machine (SVM), Weighted K-nearest neighbor (Weighted KNN), neural network (NN) classifiers have been applied to the dataset. Ten-fold cross validation method was used to be assured of the performance of classifiers and avoid over-fitting [ 25 ]. After all, we compared the performance of classifiers by means of a Receiver operating characteristic (ROC) graph with sensitivity and specificity plots under the same training dataset.

Support Vector Machine

Support Vector Machine (SVM) is a technique based on statistical learning theory which is widely applied in similar classification tasks [ 14 ]. This algorithm is able to generate a variety of learning machines by means of kernel functions [ 26 ]. In this research, SVM was employed with Gaussian and Quadratic kernel functions.

Neural Network

Neural Network (NN) classifier contains input, hidden and output layers, each of which has been constituted of neurons and creates a fully connected network [ 27 ]. NN is used in the identification pattern of the tremor signal because of nonlinearity ability [ 28 ]. In this study, a two-layer feed-forward network is implemented with sigmoid hidden and softmax output neurons. The network was trained with scaled conjugate gradient back propagation algorithm based on multilayer perceptron (MLP) structure.

Weighted K-Nearest Neighbor

Weighted K-Nearest (Weighted KNN) is a supervised learning algorithm assigning new sample to the nearest sample of K neighborhood samples in the feature space so that not to let all neighbors around the sample have the same weight [ 25 ]. This method is a very common classification technique and has shown good performance accuracy [ 29 ]. In this work, City Block distance was used.

Results

Repeated measure ANOVA test results indicated the lack of statistical significant difference between the repetitions for features. So it confirmed the repeatability of experiments.

Results of statistical test and boxplot of each feature for healthy and patient groups with DBS ON and OFF states are shown in Figures 3, 4 and 5. Results were achieved for right and left hand separately to evaluate the effect of DBS on each hand. The significance level of each feature between groups has been specified by p-value. The features of energy, dominant freq and maximum amplitude (max amp) in first IMF, FuzzyEn, SampEn and deviation of the direct path among the three groups were statistically significant.

Figure 3. The boxplot of maximum amplitude (right side) and dominant frequency (left side) features in the first Intrinsic Mode Function (IMF) of tremor signal.

Figure 4. The boxplot of sample entropy (right side) and fuzzy entropy (left side) features of tremor signal.

Figure 5. The boxplot of deviation of the direct path (right side) and energy (left side) features of tremor signal.

The healthy group had the lowest value in the deviation of the direct path, energy and max amp and the highest value in the dominant freq, SampEn and FuzzyEn in contrast to patient group. The patients with DBS ON had lower values in the variables of deviation of the direct path, energy and max amp and higher values in the variables dominant freq, SampEn and FuzzyEn than the patients with DBS OFF (Figures 3, 4 and 5, Table1).

a)
Features Healthy Patient (DBS ON) Patient (DBS OFF)
Mean SD Mean SD Mean SD
Sample Entropy 0.223 0.056 0.171 0.025 0.080 0.035
Fuzzy Entropy 0.155 0.040 0.1234 0.027 0.056 0.028
Deviation from direct path 1.653 0.343 3.228 1.391 8.774 1.461
Maximum amplitude 11.576 3.068 16.923 5.037 37.877 4.182
Dominant frequency 4.421 0.307 4.005 0.466 2.987 0.475
Energy 31.763 19.205 61.925 29.431 152.267 19.870
b)
Features Healthy Patient (DBS ON) Patient (DBS OFF)
Mean SD Mean SD Mean SD
Sample Entropy 0.305 0.070 0.241 0.039 0.113 0.041
Fuzzy Entropy 0.180 0.030 0.153 0.019 0.087 0.026
Deviation from direct path 6.856 0.855 9.728 2.933 19.076 2.592
Maximum amplitude 15.582 3.423 22.076 3.924 37.028 4.163
Dominant frequency 4.291 0.272 3.874 0.426 2.794 0.265
Energy 45.278 8.498 84.072 20.816 183.153 47.758
DBS: Deep Brain Stimulation, SD: Standard Deviation
Table 1.Mean and Standard Deviation (SD) of features extracted from hand tremor signal for healthy, patient with Deep Brain Stimulation (DBS) ON and OFF groups: (a) right hand, (b) left hand.

According to the empirical analysis, the proper number of neighbors for Weighted KNN classifier was five. Due to overall performance, Weighted KNN indicated the highest accuracy rate of 95.1% in the classification of tremor signals in three groups. NN, SVM (Gaussian kernel) and SVM (Quadratic kernel) classifiers showed the accuracy of 93.2%, 91.3% and 88.3%, respectively (Figure 6).

Figure 6. Performance of weighted K-nearest neighbor (KNN), support vector machine (SVM) and neural network (NN) classifiers.

Discussion

In average, DBS had a considerable effect on increasing dominant freq, SampEn and FuzzyEn and decreasing deviation of the direct path, energy and max amp in both hands of MS patients. This means that the tremor signal of patients is accompanied by high amplitude and low complexity. So, the regularity in MS tremor is significantly reduced by DBS. Meanwhile, hand oscillation of patients during movement between two fixed markers is high. Therefore, they require more energy during directed movement and have more deviation of the direct path than the healthy group. Results revealed that the disease is not restricted to one dominant hand only and DBS could improve both hands.

These achievements showed the positive effect of DBS on reducing the MS tremor. Therefore, it can be used in the improvement of tremor in MS patients.

Features such as AppEn, wavelet energy and distance average to the direct path were not able to separate all three groups. The following probable reasons limited the rate of correct categorization of tremor signals with above features:

• The AppEn is dependent on signal length and affected by bias, as well as counts self-matches [ 19 ]. Regarding the relatively short length of data, it seems cannot differentiate the tremor signal. In addition, AppEn has a rigid boundary of similarity whose small variations in distance of two vectors result in sudden changes of entropy value. In FuzzyEn, there is no rigid boundary and it analyzes the similarity of both vectors in fuzzily [ 30 ]. The fuzzy entropy has better results than AppEn.

• The wavelet transform is a linear method of time-frequency [ 23 ]. Actually, the tremor signals are typically nonlinear, so that cannot be analyzed reliably by linear assumption [ 5 ]. EMD considers the nonlinearity nature of the tremor signal [ 15 ]. As a result, this method demonstrates feature extraction of tremor signals more desirable.

Since the difference between left and right displacement is low or approximately zero, the distance average to the direct path is not an appropriate measure for characterizing tremor signal.

SampEn and FuzzyEn are proper measures to differentiate participants. Earlier work [ 10 ] reported entropy is an ineffective feature in distinguishing different groups. There has been no mention of the dysmetria separation from tremor and the tremor signal with the interference of its other movement disorders was analyzed. Therefore, dysmetria can be the reason for previous methods failure. Since SampEn and FuzzyEn significantly indicated the positive effect of DBS on tremor signal of MS patients when dysmetria is removed.

This research was focused on MS tremor signal with nonlinear analysis; however, previous studies [ 8 - 10 ] were done on hand movement signal of participants. They evaluated the effect of DBS by analyzing linear and statistical features. Due to nonlinearity and nonstationary properties of the tremor signal, nonlinear analysis can be useful for reliably quantifying the effect of DBS treatment in MS patients.

Weighted KNN was more appropriate than other classifiers. Since this classifier is less sensitive to an imbalance of classes. In addition, its other good feature is that it is not susceptible to the number of feature and length of the dataset [ 31 ]. Therefore, this classifier was indicated the best performance accuracy as expected.

Conclusion

In this study, we have used the new EMD technique, which is proportionate to nonlinear and nonstationary nature of the tremor signal, to identify and separate dysmetria in order to evaluate the effect of stimulation on tremor.

The results from statistical analysis indicated the reduction of tremor after applying DBS in MS patients. Therefore, results implied a success rate of this therapy in decreasing the movement tremor. According to p-value between groups and boxplot of each feature for each group in both hands, it can be found that energy, dominant freq and max amp in first IMF, FuzzyEn, SampEn and deviation of the direct path were statistically significant and had more capable of quantizing MS tremor in comparison with other features. Also, SVM, NN and Weighted KNN methods have been used to classify tremor signal in three groups of healthy, patients with DBS ON and OFF. By considering the ROC graph of mentioned classifiers and comparing the specificity and accuracy of them, Weighted KNN with performance accuracy of more than 95% could effectively conduct the classification of tremor signal.

Regarding the relatively short length of data, some of the features were not able to differentiate tremor signals; therefore, we should use features that are less sensitive to the data length. Also, according to nonlinear and nonstationary nature of tremor signal, some above-mentioned features are not able to differentiate signals. It is hoped that the results of this research will get improved by using a wider database in the future.

Acknowledgement

This article has been extracted from the thesis written by Fatemeh Valipour in School of Medicine Shahid Beheshti University of Medical Sciences (Registration No:376).

The raw tremor data was provided by the Surgical Center for Movement Disorders at the University of British Columbia, Canada. Finally, we thank Amin Asgharzadeh Alvar for his comments which greatly improved the manuscript.

Authors’ Contribution

Fatemeh Valipour conceived the design of the study, performed the analyzing and interpreting of the data, and drafted the manuscript. Ali Esteki carried out the critical revision of the manuscript. As well as, he supervised all the steps of the study. All authors read and approved the initial and final draft of this manuscript.

Ethical Approval

We didn’t collect any dataset in this research. Raw tremor data was provided from another research in the Surgical Center for movement disorders at the University of British Columbia, Canada.

Conflict of Interest:

None

References

  1. Ghaffarpasand F. Neurosurgical Approaches in Demyelinating Disorders; Where are We Now?. Galen Medical Journal. 2015; 4:60-1.
  2. Jeyalan V, Eljamel S. Efficacy & Safety of Deep Brain Stimulation on Tremor in Multiple Sclerosis Patients. Scottish Universities Medical Journal. 2012; 1:42-55.
  3. Alusi SH, Worthington J, Glickman S, Bain PG. A study of tremor in multiple sclerosis. Brain. 2001; 124:720-30. DOI | PubMed
  4. Liu X, Aziz TZ, Miall RC, Rowe J, et al. Frequency analysis of involuntary movements during wrist tracking: a way to identify ms patients with tremor who benefit from thalamotomy. Stereotact Funct Neurosurg. 2000; 74:53-62. DOI | PubMed
  5. Ayache SS, Al-ani T, Farhat WH, et al. Analysis of tremor in multiple sclerosis using Hilbert-Huang Transform. Neurophysiol Clin. 2015; 45:475-84. DOI | PubMed
  6. Blahak C, Wohrle JC, Capelle HH, et al. Tremor reduction by subthalamic nucleus stimulation and medication in advanced Parkinson’s disease. J Neurol. 2007; 254:169-78. DOI | PubMed
  7. Rissanen SM, Kankaanpaa M, Tarvainen MP, et al. Analysis of EMG and acceleration signals for quantifying the effects of deep brain stimulation in Parkinson’s disease. IEEE Trans Biomed Eng. 2011; 58:2545-53. Publisher Full Text | DOI | PubMed
  8. Esteki A, Hodgson T, Honey C. The effect of deep brain stimulation on target directed movement of the hand in multiple sclerosis patients. Gait Posture. 2006; 24:S230-S1. DOI
  9. Esteki A, Hodgson T. Quantitative Measurement of Hand’s Action Tremor in Patients with Multiple Sclerosis and the Effects of Thalamic Deep Brain Stimulation. Pajoohandeh Journal. 2007; 12:345-51.
  10. Esmailpour H, Esteki A, Seddighi A. Quantitative assessment of deep brain stimulation on tremor in multiple sclerosis disease. International Clinical Neuroscience Journal. 2015; 2:87-90.
  11. Sandstrom R. Measuring ataxia: Quantification based on the standard neurological examination. Phys Ther. 1995; 75:332.
  12. Notermans NC, Van Dijk GW, Van Der Graaf Y, et al. Measuring ataxia: quantification based on the standard neurological examination. J Neurol Neurosurg Psychiatry. 1994; 57:22-6. Publisher Full Text | DOI | PubMed
  13. Erasmus LP, Sarno S, Albrecht H, et al. Measurement of ataxic symptoms with a graphic tablet: standard values in controls and validity in Multiple Sclerosis patients. J Neurosci Methods. 2001; 108:25-37. DOI | PubMed
  14. Ai L, Wang J, Yao R. Classification of parkinsonian and essential tremor using empirical mode decomposition and support vector machine. Digital Signal Processing. 2011; 21:543-50. DOI
  15. Ur Rehman N, Mandic DP. Empirical mode decomposition for trivariate signals. IEEE Transactions on signal processing. 2010; 58:1059-68. DOI
  16. Spyers-Ashby J, Bain P, Roberts S. A comparison of fast Fourier transform (FFT) and autoregressive (AR) spectral estimation techniques for the analysis of tremor data. J Neurosci Methods. 1998; 83:35-43. DOI
  17. De Lima ER, Andrade AO, Pons JL, Kyberd P, Nasuto SJ. Empirical mode decomposition: a novel technique for the study of tremor time series. Med Biol Eng Comput. 2006; 44:569-82. DOI | PubMed
  18. Pincus SM. Approximate entropy as a measure of system complexity. Proc Natl Acad Sci U S A. 1991; 88:2297-301. Publisher Full Text | DOI | PubMed
  19. Richman JS, Moorman JR. Physiological time-series analysis using approximate entropy and sample entropy. Am J Physiol Heart Circ Physiol. 2000; 278:H2039-49. DOI | PubMed
  20. Zhang X, Zhou P. Sample entropy analysis of surface EMG for improved muscle activity onset detection against spurious background spikes. J Electromyogr Kinesiol. 2012; 22:901-7. Publisher Full Text | DOI | PubMed
  21. Chen Y, Hu H, Ma C, Zhan Y, Chen N, Li L, et al. Stroke-Related Changes in the Complexity of Muscle Activation during Obstacle Crossing Using Fuzzy Approximate Entropy Analysis. Front Neurol. 2018; 9:131. Publisher Full Text | DOI | PubMed
  22. Shi J, Zhao P, Cai Y, Jia J. Classification of Hand Motions from Surface Electromyography with Rough Entropy. Journal of Medical Imaging and Health Informatics. 2015; 5:328-34.
  23. Sengur A. Wavelet transform and adaptive neuro-fuzzy inference system for color texture classification. Expert Systems with Applications. 2008; 34:2120-8. DOI
  24. Sezgin N, Emin Tagluk M. Energy based feature extraction for classification of sleep apnea syndrome. Comput Biol Med. 2009; 39:1043-50. DOI | PubMed
  25. Oung QW, Muthusamy H, Basah SN, Lee H, Vijean V. Empirical Wavelet Transform Based Features for Classification of Parkinson’s Disease Severity. J Med Syst. 2017; 42:29. DOI | PubMed
  26. Pan S, Iplikci S, Warwick K, Aziz TZ. Parkinson’s Disease tremor classification–A comparison between Support Vector Machines and neural networks. Expert Systems with Applications. 2012; 39:10764-71.
  27. Duda RO, Hart PE, Stork DG. Pattern classification. John Wiley & Sons: New Jersey; 2012.
  28. Pan S, Warwick K, Stein J, et al. Prediction of Parkinson’s disease tremor onset using artificial neural networks. Proceedings of the fifth IASTED International Conference: biomedical engineering. ACTA Press: USA; 2007.
  29. Schiaffino L, Muñoz AR, Villora JF, et al. Feature selection for KNN classifier to improve accurate detection of subthalamic nucleus during deep brain stimulation surgery in Parkinson’s patients. VII Latin American Congress on Biomedical Engineering CLAIB 2016, Bucaramanga, Santander, Colombia. 2016. DOI
  30. Chen W, Zhuang J, Yu W, Wang Z. Measuring complexity using FuzzyEn, ApEn, and SampEn. Med Eng Phys. 2009; 31:61-8. DOI | PubMed
  31. Little MA, McSharry PE, Hunter EJ, Spielman J, Ramig LO. Suitability of dysphonia measurements for telemonitoring of Parkinson’s disease. IEEE Trans Biomed Eng. 2009; 56:1015. Publisher Full Text | DOI | PubMed