Document Type : Original Research
Authors
1 Department of Neuroscience and Addiction Studies, School of Advanced Technologies in Medicine, Tehran University of Medical Sciences, Tehran, Iran
2 Department of Radiology, Faculty of Alliance Medical Sciences, Tabriz University of Medical Sciences, Tabriz, Iran
3 Department of Nuclear Medicine, Imam Khomeini Hospital Complex, Tehran University of Medical Sciences, Tehran, Iran
4 Department of Radiology and Radiotherapy, School of Medicine, Tabriz University of Medical Sciences, Tabriz, Iran
5 Medical Radiation Sciences Research Group, Tabriz University of Medical Sciences, Tabriz, Iran
Abstract
Background: Radiomics with single Region of Interest (ROI) and single-sequence Magnetic Resonance Imaging (MRI) may facilitate the segmentation reproducibility and radiomics workflow due to a time-consuming and complicated delineation of that in multi-sequence MRI images.
Objective: This study aimed to evaluate the performance of the radiomics approach in grading glioma based on a single-ROI delineation as Gross Tumor Volume (GTV) in a single – sequence as contrast-enhanced T1-weighted MRI.
Material and Methods: This retrospective study was conducted on contrast-enhanced T1 weighted (CE T1W) MRI images of 60 grade II and 60 grade III glioma patients. The GTV regions were manually delineated. Radiomics features were extracted per patient. The segmentation reproducibility of the robust features was evaluated in several repetitions of GTV delineation. Finally, a linear Support Vector Machine (SVM) assessed the classification performance of the robust features.
Results: Four significant robust features were selected for training the model (P-value<0.05). The average Intraclass Correlation Coefficient (ICC) of the four features was 0.96 in several repetitions of GTV delineation. The linear SVM model differentiated grades II and III of glioma with an Area Under the Curve (AUC) of 0.9 in the training group.
Conclusion: High predicting power for glioma grading can be achieved with radiomics analysis by a single-ROI delineated on a single-sequence MRI image (CE T1W). In addition, single-ROI segmentation can increase radiomics reproducibility.
Highlights
Yunus Soleymani (Google Scholar)
Davood Khezerloo (Google Scholar)
Keywords
Introduction
Radiomics, in which many quantitative features are extracted from medical images, has become an attractive arena of medical imaging research [ 1 , 2 ]. The fundamental hypothesis of radiomics is based on the fact that heterogeneity inside medical images is due to molecular phenotype and genotype [ 3 - 5 ]. Many studies have demonstrated the capability of radiomics in glioma grading, determination of its molecular phenotype, and genetic classification . In addition, some studies in glioma radiomics have shown a remarkable association between morphological characteristics extracted from multi-sequence Magnetic Resonance Imaging (MRI), clinical outcomes, and survival [ 11 , 12 ]. Malignant glioma is the most common primary brain tumor, resulting in heterogeneity [ 6 ]. However, a biopsy is still invasive and costly, it is considered a standard histological and genetic classification method for glioma. Pathological diagnosis may remain inconclusive in 7–15% of patients [ 7 ].
Despite rapid and remarkable progress, there are still some drawbacks and challenges in using radiomics in the clinical workflow. Region of Interest (ROI) delineation is one of the most critical challenges in radiomics reliability and reproducibility [ 8 ]. Previous studies have warned about the dependency of radiomics features on ROI and its segmentation methods [ 4 - 6 , 12 ]. Although several techniques are considered for segmentation ROI, such as manual, semi-automatic, and fully automatic methods, manual segmentation is the standard clinical routine in quantitative analysis . Previous radiomics studies in malignant glioma have often been conducted on multi-sequence MRIs with automatic and semi-automatic segmentation and delineation of several ROIs, such as enhancing, non-enhancing, necrosis, and edema regions . However, the definition of several tumor volumes is often time-consuming and complicated, and it may cause a decline inter-and intra-observer reproducibility of radiomics. In addition, it can be hard to accurately discriminate the real border of several juxtaposed ROIs. Accordingly, simple and feasible ROI delineation, such as Gross Tumor Volume (GTV) in a single MRI sequence may improve the segmentation workflow and radiomics inter-and intra-observer reproducibility. GTV as defined as the gross demonstrable extent and location of the tumor is required for cancer staging, and changes in the GTV during treatment might be predictive of treatment outcome [ 15 ].
This study aimed to evaluate the performance of a rapid and reproducible radiomics approach to grading glioma based on delineating a single-ROI as GTV on a single-sequence MRI image, such as contrast-enhanced T1 weighted (CE T1W) MRIs. CE T1W is a routine clinical MRI sequence for diagnosing glioma. The segmentation reproducibility of the extracted radiomics features is evaluated in several repetitions of GTV delineation. In addition, the classification performance of the radiomics features is assessed by machine learning methods. We hypothesized that the manual segmentation of a simple GTV may alleviate the problems associated with the complexity of several volume determinations and the challenge related to radiomics dependency on ROI.
Material and Methods
Patient selection, image preprocessing, and GTV delineation
This is a retrospective study conducted on 120 patients with diagnosed glioma (60 with grade II and 60 with grade III). CE T1W MRI images were used to extract radiomics features and train machine learning-based models. The data were collected from the Cancer Imaging Archive (TCIA) database [ 16 ] from 2015 to 2019, collected from various imaging centers worldwide. The inclusion criteria were as follows: availability of clinical and pathological information, histological confirmation of tumor grade, and availability of CE T1W images. The criteria for exclusion from the study were also as follows: low image quality due to artifacts, the existence of hemorrhagic lesions in images, and the previous history of other brain tumors and surgery. According to “The Cancer Imaging Archive / The Cancer Genome Atlas (TCIA/TCGA) data-usage guidelines”, all analyses were carried out [ 16 ].
Due to the difference and heterogeneity of imaging protocol for the TCGA cohort, all images were preprocessed using the ComBat method [ 17 , 18 ]. Each MR image was resampled onto a 256×256 matrix size, registered to 5 mm slice thickness with no slice gap, and interpolated to the isotropic voxel size of 1×1×5 mm3. Patients were randomly divided into a training group (n=80, 40 with grade II, 40 with grade III) and an independent test group (n=40, 20 with grade II, 20 with grade III) for the test evaluations. Figure 1 shows the main procedure of this study.
Figure 1. The flow chart of the current study.
For each patient, a 2D ROI inside the largest cross-sectional area of the tumor was manually delineated by a 20-year experienced neuro-oncologist as a GTV (Figure 2). The ROIs extraction and radiomics feature calculations were performed in the free open-source standardized- IBEX (S-IBEX) software (https://github.com/abettinelli/SIBEX_Source) [ 19 ].
Figure 2. Two-dimensional GTV delineation on CE T1W images; the left columns are original images and the rights are the segmented images (a) A 19-year-old female patient with grade II astrocytoma, (b) A 39-year-old male patient with grade III oligodendroglioma.(GTV: Gross tumor volume; CE T1W: Contrast-enhanced T1 weighted)
Extraction, selection, and segmentation reproducibility of features
One hundred radiomics features were extracted per patient. The highly redundant features were then removed by a two-step feature selection procedure. In the first step, a set of radiomics features with high discrimination performance between grades II and III of glioma was obtained using the nonparametric Mann-Whitney U test. A P-value<0.05 was set as a statistically significant level [ 20 ]. The second step eliminated highly correlated features using Pearson correlation analysis, with an R threshold of 0.75 [ 21 ]. To assess the effect of manual GTV delineation variations on features reproducibility, an ROI was delineated thirty times by thirty blinded physicians on a specific slice of the images of two patients (one with grade II and the other with grade III glioma). Next, Intraclass Correlation Coefficient (ICC), calculated based on a two-way mixed, consistent, average measurement model was assessed for the features (ICC>0.9 was considered significant). All statistical analyzes were performed in SPSS v.26 software.
Model training
The Support Vector Machine (SVM) classifier was implemented to evaluate the performance of selected robust radiomics features in the differentiation of grade II from grade III glioma. SVM is the most common classifier for binary classification. After normalizing all features to , 10-fold cross-validation was employed to avoid overfitting as much as possible [ 22 ]. A linear SVM model was trained and evaluated by computing accuracy, sensitivity, and specificity values. The Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC) was also calculated for the model. All analyses were performed using the machine learning toolbox of MATLAB (MathWorks, Inc, Natick, MA) R2016b software.
Results
The demographical and clinical information of the patients is summarized in Table 1.
| Characteristics | Grade II | Grade III | |
|---|---|---|---|
| Patients (N/%) | 50% (60/120) | 50% (60/120) | |
| Age (mean±SD) | 43±2.75 | 47±2 | |
| Gender (N/%) | Male | 45% (27/60) | 60% (36/60) |
| Female | 55% (33/60) | 40% (24/60) | |
| Histologic subtype (N/%) | Astrocytoma | 68.4% (41/60) | 63.4% (38/60) |
| Oligodendroglioma | 31.6% (19/60) | 36.6% (22/60) | |
| SD: Standard Deviation | |||
Extracted and selected features
One hundred radiomics features were extracted from the images of each patient in the training group, including 22 Gray-level Co-occurrence Matrix (GLCM), 11 Gray-level Run-length Matrix (GLRLM), 49 histograms, and 18 shape-based features. After analyzing the Mann-Whitney statistical test, 31 of 100 features remained, showing high discriminative performance between grades II and III of glioma. Four significant features (homogeneity, correlation, kurtosis, and surface area density) were elicited from the previous 31 obtained features by Pearson statistical test, determined as the robust radiomics features for training the model (Figure 3). The means and Standard Deviations (SDs) of the four robust radiomics features are compared in Table 2. Also, Table 3 summarizes the ICC values of the four robust features obtained from thirty times the repetition of GTV delineation. The average ICC of the features was significantly high and obtained at 0.96.
Figure 3. The Pearson correlation coefficient heat map of the selected robust features.
| Feature parameters | Mean±SD | P-value | |
|---|---|---|---|
| Grade II | Grade III | ||
| GLCM | |||
| Homogeneity | 0.525±0.105 | 0.439±0.129 | 0.024 |
| Correlation | 0.805±0.071 | 0.724±0.112 | 0.021 |
| Intensity Histogram | |||
| Kurtosis | 10.338±10.34 | 4.327±3.171 | 0.005 |
| Shape | |||
| Surface Area Density | 4.581±1.901 | 3.454±1.046 | 0.028 |
| SD: Standard Deviation, GLCM: Gray-Level Co-Occurrence Matrix | |||
| ICC for grade II glioma | ICC for grade III glioma | |
|---|---|---|
| Homogeneity | 0.99 | 0.97 |
| Correlation | 0.96 | 0.95 |
| Kurtosis | 0.99 | 0.98 |
| Surface area density | 0.94 | 0.94 |
| ICC: Intraclass Correlation Coefficient | ||
Model performance in the training and independent test groups
Training group: The linear SVM model performance was obtained by considering the average of 10-fold cross-validation results. The model could discriminate grades II and III of glioma with a high-performance accuracy (82.5%), sensitivity (80%), specificity (85%), and AUC (0.90) (Figure 4).
Figure 4.The ROC curve of the linear SVM model for classification of grades II and III glioma in the training group. (ROC: Receiver Operating Characteristic; SVM: Support Vector Machine)
Test group: All the patient selection processes, MRI image preprocessing, GTV delineation, radiomics feature extraction, and feature selection were applied to the test group, similar to the training group. Finally, the data were imported into the linear SVM model. The model could discriminate grades II and III of glioma with accuracy, sensitivity, specificity, and AUC of 85%, 80%, 85%, and 0.84 (Figure 5).
Figure 5. The ROC curve of the linear SVM model for classification of grades II and III glioma in the test group. ROC: Receiver Operating Characteristic; SVM: Support Vector Machine)
Notably, our radiomics model had high performance despite the simplicity of the approach. Table 4 compares the results of the current study with some similar recent papers.
| Reference | Main results | Limitations | The current study’s advantages |
|---|---|---|---|
| Togao O et al. 2016 [23] | Distinguishing high-grade and low-grade glioma with 96% sensitivity and 81% specificity | • Low sample size (34) | • Higher sample size (120) |
| • Using 3-tesla MRI images is not routine in glioma diagnostic workflow | • Using a 1.5-tesla routine system | ||
| • Muli-ROI & Muli-sequences MRI | • Single-ROI & single-sequence MRI | ||
| Cho H-h et al. 2018 [6] | Distinguishing disease grades with high efficiency (AUC=0.9400) using SVM and Random Forest methods, | • Manual segmentation of various areas of necrosis, and edema leads to results subjective and less reproducible | • High reproducibility of segmentations (ICC>0.90) |
| • Manual segmentation of GTV regions which is routine and simple to draw and achieves a comparable AUC of 0.90 | |||
| • Single-ROI & single-sequence MRI | |||
| Jeong J et al. 2019 [14] | Distinguishing high grade from low using the Random Forest method and achieving AUC=0.94 | • Low sample size (25) | • Higher sample size (120) |
| • Failure to validate the results on an independent test group of patients | • Manual delineation of GTV regions as a clinically reproducible approach (ICC>0.90) | ||
| • Muli-ROI & Muli-sequences MRI | • Verification of results on an independent test group (accuracy=85%) | ||
| • Single-ROI & single-sequence MRI | |||
| Zhao S-S et al. 2020 [24] | Separating grade two and three gliomas using Random Forest algorithm and T1CE sequence along with manual segmentation of tumoral areas with sensitivity of 0.77 and AUC=0.86 | • Low sample size (36) | • Higher sample size (120) |
| • Sensitivity<0.80 | • Sensitivity=0.80 | ||
| • AUC<0.90 | • AUC=0.90 | ||
| • Muli-ROI & Muli-sequences MRI | • Single-ROI & single-sequence MRI | ||
| Kobayashi K et al. 2021 [10] | Extracting a shareable set of feature vectors that encode various parts in tumor imaging phenotypes and predict the glioma grade with 90% accuracy | • Using complex deep learning algorithms for feature extraction and model building which may be clinically not feasible | • Using Manual delineation of GTV regions as a simple and clinically reproducible approach (ICC>0.90) |
| • Muli-ROI & Muli-sequences MRI | • Comparable accuracy of 85% in the test group | ||
| • Single-ROI & single-sequence MRI | |||
| ROI: Region Of Interest, MRI: Magnetic Resonance Imaging, SVM: Support Vector Machine, AUC: Area Under The Curve, GTV: Gross Tumor Volume, ICC: Intraclass Correlation Coefficient, T1CE: T1 Contrast-Enhanced | |||
Discussion
The purpose of this study was to determine the diagnostic performance of the machine-learning-based radiomics approach to differentiate grades II and III gliomas in a single-ROI delineation as well as in a single-sequence CE T1W MRI image. A high predictive power of AUC of 0.90 was achieved by the GTV areas for segmentation, which is much easier to delineate and clinically more feasible. However, the current study was conducted on a single-ROI delineated in a single-sequence MRI image, the obtained results were consistence with previous multi- ROI delineated in Multi-sequence MRI images . We also obtained four robust radiomics features, which were significantly reproducible against slight variations in manual GTVs delineation (ICC=0.96), and also had homogeneity and correlation of GLCM, kurtosis of intensity histogram, and surface area density of shape.
Based on the obtained results, grade II glioma was more homogeneous, showing grade II glioma is more biologically proliferative, which results in more voxels of similar uptake. Kurtosis measures whether the data on the image is heavy-tailed or light-tailed relative to a normal distribution. According to the results, the images of grade II glioma had higher kurtosis because tumors with multiple voxels of similar uptake are more likely biologically proliferative. Also, a homogeneously enhanced lesion (grade II glioma lesions) has a higher correlation and surface density compared to a heterogeneously enhanced lesion (grade III glioma lesions).
Previous studies were performed using multi-sequence MRI images (T1-weighted, CE T1W, T2-weighted, and Fluid-attenuated Inversion Recovery (FLAIR)), as well as delineation of multiple tumor volumes (enhancing tumors, non-enhancing tumors, necrosis, and edema) . Qin et al. [ 13 ] in a study on 66 glioma patients, showed that three radiomics features extracted from MRI images (Cluster Shade, Entropy, and Homogeneity) had the best performance in differentiating low-grade from high-grade glioma. They also used multiple-sequence MRI images, such as T2-weighted, FLAIR, CE T1W, and Apparent Diffusion Coefficient (ADC) maps to extract radiomics features. However, they performed a layer-by-layer segmentation on the images of all sequences, which was highly time-consuming and complicated, they didn’t evaluate the predictive power of these features. Further, Xie et al. [ 9 ] assessed Dynamic Contrast-enhanced (DCE)-MRI of 42 patients with glioma and reported that entropy with AUC (0.885) and Inverse Difference Moment (IDM) with AUC (0.901) of GLCM could differentiate low- from high-grade glioma. Togao et al. [ 23 ] achieved a sensitivity of 96% and a specificity of 81% by using intra-voxel incoherent motion imaging to discriminate glioma patients. Nevertheless, the clinical use of such time-consuming and complicated protocols can be challenging. Hwan-ho et al. [ 6 ] applied three classification algorithms of logistic, SVM, and random forest in a cohort study with 285 glioma patients; In their study conducted on multimodal sequences T1-weighted, CE T1 W, T2-weighted, and FLAIR, the results showed an average AUC of 0.94 for training sets and 0.903 for test sets. However, their segmentations were obtained from the database, which provides multiple sub-regions of enhancing tumors, non-enhancing tumors, necrosis, and edema. Zacharaki et al. [ 25 ] used the SVM-based Recursive Feature Elimination (SVM-RFE) method with leave-one-out cross-validation to differentiate low- and high-grade glioma and showed an accuracy of 87% with an AUC of 0.896. However, applying the SVM-RFE in combination with other classifiers may lead to performance degradation because the SVM-RFE approach is tailored for the SVM classifier. Recently, fully automated segmentation approaches and convolutional neural networks have also been considered in glioma radiomics studies, which are highly predictive and prognostic [ 24 , 26 ]. Because oncologists routinely use manual segmentation tools for ROI delineation and demand with other simple procedures during treatment planning of radiotherapy, implementation of such intricate algorithms may be clinically not feasible.
There are some limitations to the present study as follows: 1) only 2D manual segmentation was used to minimize challenges with ROI delineation, 2) CE T1W MRI sequence was only used, 3) only Support Vector Machine (SVM) was used, and 4) some radiomics feature groups were not used, such as wavelet-based features, Gray-Level Size Zone Matrix (GLSZM), and Neighboring Gray-tone Difference matrix (NGTDM).
Conclusion
High predicting power for glioma grading can be achieved (AUC=0.9) with radiomics analysis by a single-ROI delineated on a single-sequence MRI image (CE T1W). Radiomics analysis by only a single-ROI segmentation can increase radiomics reproducibility (ICC=0.96).
Authors’ Contribution
Study conception and design: Y. Soleymani, D. Khezerloo; data collection: M. Mohammadzadeh, D. Khezerloo; interpretation of results: P. Sheikhzadeh; draft manuscript preparation: D. Khezerloo, Y. Soleymani. All authors reviewed and approved the final version of the manuscript.
Ethical Approval
This study was performed at Tabriz University of Medical Sciences from 2019 to 2020 with the ethical approval code of I.R.TBZMED.REC.1399.078.
Informed Consent
All the data of the current study were collected from the Cancer Imaging Archive database (https://www.cancerimagingarchive.net) which hosts a large archive of medical images of cancer accessible for public download. According to “The Cancer Imaging Archive / The Cancer Genome Atlas (TCIA/TCGA) data-usage guidelines,” all analyses were carried out.
Funding
This research was supported by the Vice Chancellor for Research, Tabriz University of Medical Sciences (grant number: 64454).
Conflict of Interest
None
References
- Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology. 2016; 278(2):563-77. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Soleymani Y, Jahanshahi AR, Hefzi M, Fazel Ghaziani M, Pourfarshid A, Khezerloo D. Evaluation of textural-based radiomics features for differentiation of COVID-19 pneumonia from non-COVID pneumonia. Egyptian Journal of Radiology and Nuclear Medicine. 2021; 52:1-7.
- Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, Van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012; 48(4):441-6. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Haarburger C, Müller-Franzes G, Weninger L, Kuhl C, Truhn D, Merhof D. Radiomics feature reproducibility under inter-rater variability in segmentations of CT images. Sci Rep. 2020; 10(1):12688. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Soleymani Y, Jahanshahi AR, Pourfarshid A, Khezerloo D. Reproducibility assessment of radiomics features in various ultrasound scan settings and different scanner vendors. J Med Imaging Radiat Sci. 2022; 53(4):664-71. DOI | PubMed
- Cho HH, Lee SH, Kim J, Park H. Classification of the glioma grading using radiomics analysis. PeerJ. 2018; 6:e5982. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Jang K, Russo C, Di Ieva A. Radiomics in gliomas: clinical implications of computational modeling and fractal-based analysis. Neuroradiology. 2020; 62(7):771-90. DOI | PubMed
- Singh G, Manjila S, Sakla N, True A, Wardeh AH, Beig N, et al. Radiomics and radiogenomics in gliomas: a contemporary update. Br J Cancer. 2021; 125(5):641-57. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Xie T, Chen X, Fang J, Kang H, Xue W, Tong H, et al. Textural features of dynamic contrast-enhanced MRI derived model-free and model-based parameter maps in glioma grading. J Magn Reson Imaging. 2018; 47(4):1099-111. DOI | PubMed
- Kobayashi K, Miyake M, Takahashi M, Hamamoto R. Observing deep radiomics for the classification of glioma grades. Sci Rep. 2021; 11(1):10942. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Chaddad A, Sabri S, Niazi T, Abdulkarim B. Prediction of survival with multi-scale radiomic analysis in glioblastoma patients. Med Biol Eng Comput. 2018; 56(12):2287-300. DOI | PubMed
- Bae S, Choi YS, Ahn SS, Chang JH, Kang SG, Kim EH, et al. Radiomic MRI Phenotyping of Glioblastoma: Improving Survival Prediction. Radiology. 2018; 289(3):797-806. DOI | PubMed
- Qin JB, Liu Z, Zhang H, Shen C, Wang XC, Tan Y, et al. Grading of Gliomas by Using Radiomic Features on Multiple Magnetic Resonance Imaging (MRI) Sequences. Med Sci Monit. 2017; 23:2168-78. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Jeong J, Wang L, Ji B, Lei Y, Ali A, Liu T, et al. Machine-learning based classification of glioblastoma using delta-radiomic features derived from dynamic susceptibility contrast enhanced magnetic resonance images: Introduction. Quant Imaging Med Surg. 2019; 9(7):1201-13. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Das IJ, Andersen A, Chen ZJ, Dimofte A, Glatstein E, Hoisak J, et al. State of dose prescription and compliance to international standard (ICRU-83) in intensity modulated radiation therapy among academic institutions. Pract Radiat Oncol. 2017; 7(2):e145-55. DOI | PubMed
- Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging. 2013; 26(6):1045-57. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Beer JC, Tustison NJ, Cook PA, Davatzikos C, Sheline YI, Shinohara RT, et al. Longitudinal ComBat: A method for harmonizing longitudinal multi-scanner imaging data. Neuroimage. 2020; 220:117129. DOI
- Da-Ano R, Masson I, Lucia F, Doré M, Robin P, Alfieri J, et al. Performance comparison of modified ComBat for harmonization of radiomic features for multicenter studies. Sci Rep. 2020; 10(1):10248. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Bettinelli A, Branchini M, De Monte F, Scaggion A, Paiusco M. Technical Note: An IBEX adaption toward image biomarker standardization. Med Phys. 2020; 47(3):1167-73.
- Di Leo G, Sardanelli F. Statistical significance: p value, 0.05 threshold, and applications to radiomics-reasons for a conservative approach. Eur Radiol Exp. 2020; 4(1):18. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Toubiana D, Maruenda H. Guidelines for correlation coefficient threshold settings in metabolite correlation networks exemplified on a potato association panel. BMC Bioinformatics. 2021; 22(1):116. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Xiong Z, Cui Y, Liu Z, Zhao Y, Hu M, Hu J. Evaluating explorative prediction power of machine learning algorithms for materials discovery using k-fold forward cross-validation. Computational Materials Science. 2020; 171:109203. DOI
- Togao O, Hiwatashi A, Yamashita K, Kikuchi K, Mizoguchi M, Yoshimoto K, et al. Differentiation of high-grade and low-grade diffuse gliomas by intravoxel incoherent motion MR imaging. Neuro Oncol. 2016; 18(1):132-41. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Zhao SS, Feng XL, Hu YC, Han Y, Tian Q, Sun YZ, et al. Better efficacy in differentiating WHO grade II from III oligodendrogliomas with machine-learning than radiologist’s reading from conventional T1 contrast-enhanced and fluid attenuated inversion recovery images. BMC Neurol. 2020; 20(1):48. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Zacharaki EI, Wang S, Chawla S, Soo Yoo D, Wolf R, et al. Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn Reson Med. 2009; 62(6):1609-18. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
- Sun L, Zhang S, Chen H, Luo L. Brain Tumor Segmentation and Survival Prediction Using Multimodal MRI Scans With Deep Learning. Front Neurosci. 2019; 13:810. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
)
