Introduction

Neuroendocrine tumors (NETs) are considered a heterogeneous group of tumors, which make up 0.9% of all tumors affecting about 7 out of 100,000 people each year and also emanate from nerve and endocrine cells [ 1 ]. NETs are divided into functional and non-functional groups, meaning that they produce hormones and symptoms or do not [ 2 ]. Several treatments are considered for neuroendocrine tumors, such as surgery, chemotherapy, and peptide receptor radionuclide therapy [ 3 , 4 ]. Peptide receptor radionuclide therapy (PRRT) with ¹⁷⁷Lu was approved by the U.S. Food and Drug Administration (FDA) and the European Medical Agency (EMA) is currently used for NETs treatment. Over the past few years, lutetium-177 (¹⁷⁷Lu) radionuclide has been of great interest in research and clinical fields [ 5 - 8 ] and can be used for both imaging and treatment purposes due to the emission of gamma and beta particles [ 6 ]. Many studies in recent years have been performed to evaluate the treatment of ¹⁷⁷Lu-DOTATATE in patients with NETs using clinical factors [ 7 - 9 ].

Experimental findings show that using ¹⁷⁷Lu-DOTATATE therapy for advanced, metastatic gastrointestinal NETs leads to a significant improvement in progression-free survival (PFS) and overall survival (OS) compared with high dose octreotide long-acting repeatable (LAR) 60 mg [ 7 ]. In 2020, Casas et al. studied the treatment of ¹⁷⁷Lu-DOTATATE in patients with metastatic neuroendocrine tumors and prognostic factors. PRRT was used to determine the quality of life of patients and overall survival, and the median overall survival was 12.5 months (95% confidence interval range: 9.8-15.2). Overall survival was inversely proportional to previous tumor grade and the presence of bone metastasis [ 8 ].

Dromain et al. [ 10 ] evaluated tumor growth rates to predict the sequel of patients with neuroendocrine tumors and found the tumor growth rate at three months (TGR_3m) was robust, and also early radiological features could predict progression-free survival (PFS).

Medical imaging plays a fundamental role in evaluating the performance of NETs as an inseparable part of cancer care, resulting in transforming clinical decision-making in medicine [ 11 ]. Computed tomography (CT) is the most usual imaging technique used in patients with advanced cancer, assesses comprehensively tumor location, size, shape, margin, and metastasis, and plays a fundamental role in managing patients in PRRT therapy [ 12 ]. Based on some articles [ 7 - 9 ], PRRT treatment has been evaluated without considering the radiomic features or that these features have only been used to predict the degree of neuroendocrine tumors. Radiomics can extract and analyze many attributes from the medical images that characterize the tumor phenotype [ 13 ]. Fave et al. [ 14 ] used delta-radiomics features (i.e. original image features) to predict patient outcomes in non-small cell lung cancer and found that delta features had a statistically substantial effect on the overall probability of a model for overall survival compared to a model with clinical features.

No study has been conducted to predict the response to ¹⁷⁷Lu-DOTATATE radiopharmaceutical treatment using radiomic results of CT images [ 7 - 10 ]. This study aimed to show that injecting toxic radiopharmaceuticals is not necessary if the patient is not a candidate for PRRT treatment with ¹⁷⁷Lu. Accordingly, the dose delivered to the patient is reduced by saving time and cost [ 15 ].

Material and Methods

Patients and ethical approval

The present study type is experimental and analytical. A total of 34 patients with metastatic NETs who underwent PRRT treatment in the nuclear medicine section of Shohada Tajrish Hospital were included in the prospective study. Before treatment, consent was obtained from all patients.

Seventeen patients were female, and the others were male. The mean ages of men and women were 55.47 and 56.13, respectively. The “Ki-67” index was measured for patients according to immunohistochemistry (IHC) [ 2 ]. Eleven, twelve, and two patients had Ki-67<3%, 3%<Ki-67<20%, and Ki-67>20%, respectively. The condition of seven patients was unknown, and six patients underwent four cycles of treatment. Three patients stopped treatment due to inadequate response to treatment, and six patients died in the middle of treatment. Moreover, two patients used magnetic resonance imaging (MRI) for follow-up. Seventy-two CT images were obtained from these patients with NETs used for the radiomics procedure. Baseline information is listed in Table 1.

PRRT

Each patient with NETs received four to six courses of ¹⁷⁷Lu-DOTATATE radiopharmaceuticals with an interval of two to three months. Additionally, each patient was injected with 7.4 GBq of the radiopharmaceutical. In some patients (20 patients), due to their physical condition and the diagnosis of a nuclear specialist, the injected dose was less than (5.5-7.4 GBq) the mentioned amount. Four to six weeks after receiving each dose of radiopharmaceuticals for possible side effects, patients were tested based on blood tests, kidney, and liver function parameters, such as “white blood cell count (WBC)”, “red blood cell count (RBC)”, “hemoglobin test (Hb)”, “creatinine blood test (Cr)”, “platelet count (PLT)”, “blood urea nitrogen (BUN)”, “bilirubin test (Bil)”, “alkaline phosphatase level test (ALP test)”, “alanine aminotransferase (ALT, or SGPT)”, and “aspartate aminotransferase (AST, or SGOT)”.

According to the “Working Group on Definitions of Biomarkers of the National Institutes of Health”, the mentioned biomarkers are predictive [ 16 ]. Furthermore, the dose injected into each patient in each course of treatment and Ki-67 index was considered the other clinical features. The mean index of other patients was used for patients whose index was unknown.

Follow-up

After receiving the dose in each period, ten parameters were used as a questionnaire using the “Edmonton Symptom Assessment System” (ESAS-R) scale to evaluate the effect of ¹⁷⁷Lu-DOTATATE radiopharmaceutical on the quality of life of each patient. These parameters included pain assessment, fatigue, drowsiness, nausea, anorexia, shortness of breath, depression, anxiety rate, overall improvement”, and maximum problem rate. Patients were asked to select a number between zero and ten for each assessment based on their condition that zero and ten indicated the least problem and the most problem, respectively. Eastern Cooperative Oncology Group (ECOG) performance status assessment was also used for patients’ conditions. Further, CT images of each patient were taken before treatment and two stages after treatment (six months and twelve months after treatment) [ 17 ]. After receiving the second and fourth doses, the first and second, first, and fourth questionnaires were compared to determine the effect of radiopharmaceuticals on the patient’s body. The patients’ status was labeled for classification problems and machine learning algorithms. If only 25% of the patient’s condition improved compared to before receiving treatment, it was considered as non-response to treatment and also labeled “1”. If the recovery was between 25-75%, i.e. relative recovery, in which case the patient was given a label of “2”, and label “3” was awarded for a recovery above 75%. Finally, the label was zero for patients without any therapeutic dose.

Radiomics process

The radiomics process is as follows: a) image acquisition, b) segmentation of the tumor area, c) feature extraction, d) feature selection, and e) model construction and evaluation [ 13 ].

CT acquisition and segmentation

CT images were taken of patients as the first step in the radiomics process that the first CT was taken from the whole body of the patients before injecting the first dose of ¹⁷⁷Lu-DOTATATE. The second and third CTs were taken three months after receiving the second and the last dose, respectively, to follow the patient’s condition. Depending on the treatment stage, two to three images were prepared for each patient. Since patients took images from different treatment centers, the dimensions and image spacing of the images were compared, and all of those had the same dimensions of 512×512 with different voxel sizes. Therefore, resampling was performed on all images, and the size of the voxels was changed to 1×1×1 mm3. The regions of interest (ROI) in each CT image identified by a radiology specialist (primary tumor and its metastases) were segmented using 3D slicer software (version 4.11, open-source software; https: //www.slicer.org/) [ 18 ].

Feature extraction

Feature extraction from ROI was performed using the “Radiomics” plugin by 3D slicer software. For each CT image, features were extracted slice-by-slice and combined with clinical biomarkers. The class or label was determined by the patient’s condition, according to the labeling in the follow-up stage. The extracted features included morphological, first-order statistical, texture, and second-order or higher statistical features. A total of 14 morphological features, 18 first-order statistical features, and 75 tissue features were extracted [ 19 ]. Higher-order of statistical features was obtained after using filters or applying mathematical transformations to images to identify repetitive or non-repetitive patterns, suppress noise, or highlight details using statistical methods [ 20 ]. Further, 744 wavelet-based features were extracted using the 3D slicer software. The number of metastases and the size of two large metastases of each patient were added to the other features as imaging features.

Feature selection (FS)

From 72 CT images from patients, 854 and 12 quantitative and clinical features were extracted, respectively. The number of features was reduced to a smaller number of samples to avoid the probability of the model overfitting and also increase the performance speed of the model. There are generally three methods for selecting important features, such as the filter method, the wrapper method, and the embedded method. In the current study, Extreme Gradient Boosting (XGBoost) [ 21 ] and minimum redundancy maximum relevance (mRMR) methods [ 22 ] were used to reduce the number of features and select the appropriate attributes. XGBoost is an embedded method that can support classification and regression problems. XGBoost is in the category of boosted gradient decision tree algorithms with a very good performance in feature ranking. Due to the high computational speed and good performance of the model and its practice efficiency, it can be effective in academic circles [ 21 ]. mRMR is a popular filter method in medical studies, used for classification problems. In this algorithm, mutual information is used to measure the level of similarity between the two features, in which the features with the same behavior are excluded, and the few features that are most relevant to the output are selected [ 22 ].

The t-test method was used after selecting the features in different ways to obtain the relationship of each feature with the output (response to treatment) based on the P-value. The t-test compares two averages and distinguishes if they are different from each other. A P-value of <0.05 was assumed statistically significant.

Classification

Decision trees with Gini coefficient (DT), random forest (RF), and K- nearest neighbors (KNN) classifiers were used and implemented using Python (version 3.8.4) to demonstrate the effect of the selected features [ 23 - 26 ]. A total of 9-fold cross-validations with 100 iterations on the dataset were used to avoid overfitting or underfitting of the model [ 12 , 27 ].

Model validation

Since the present study is a multi-class classification problem with labels (0, 1, 2, 3), a confusion matrix was used to evaluate the performance of the model. The dimensions of this matrix were N×N (N: the number of target classes). The constituent parameters of this matrix included true positive (TP), true negative (TN), false positive (FP), and false-negative (FN). From the confusion matrix created by the model, the performance of each class was measured individually with the criteria of sensitivity (SN), specificity (SP), and accuracy (ACC). Finally, the performance of the entire model was obtained as a macro average by averaging the size of all classes in that particular criterion [ 28 ]. The critical metrics obtained from this matrix are as follows.

$\mathrm{SN}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$ (1)

$\mathrm{SP}=\frac{\mathrm{TN}}{\mathrm{TN}+\mathrm{FP}}$ (2)

$\mathrm{F1-value}=\frac{\mathrm{2TP}}{\mathrm{2TP}+\mathrm{FP}+\mathrm{FN}}$ (3)

$\mathrm{ACC}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$ (4)

Results

Clinical biomarkers were combined with the radiomic features and inputted XGBoost and mRMR feature selection algorithms. A total of 9 features were selected by the XGBoost method with a threshold of 0.03 (Figure 1).

Figure 1. Obtained features by the eXtreme gradient boosting (XGB) algorithm

In the mRMR method, 15 top features were selected according to their importance, modeled by the classifiers (RF, DT, KNN), and the accuracy of the models was obtained. In the next step, 8-14 features were modeled separately. The results showed that the top 10 features performed better in terms of accuracy as shown as a box plot in Figure 2.

Figure 2. The box plot of selected features with the minimum redundancy maximum relevance (mRMR)

As shown in Figures 1 and 2, both the XGBoost and mRMR algorithms identify only one feature as a significant attribute among the twelve clinical biomarkers, called “Dose” in the dataset, which is the amount of injected dose into each patient during each course of treatment. Other vital features were selected from the quantitative features, which in the XGBoost method, five features were related to wavelet-based features, and two and one features were selected from the morphologic and first-order statistical features, respectively. In the mRMR method, all selected image features were wavelet-based features. Other than the “LLH_glcm_DifferenceVariance” feature with (P-value=0.2), the rest of the features with the P-value<0.01 showed a significant relationship with the response to treatment function.

These features were used in modeling with DT (Gini), RF, and KNN classifiers. The confusion matrices obtained in all three classifiers are shown in Figures 3 and 4.

Figure 3. Confusion matrices of decision tree (DT), random forest (RF), K-nearest neighbors (KNN) based on the eXtreme gradient boosting (XGB) algorithm

Figure 4. Confusion matrices of decision tree (DT), random forest (RF), K-nearest neighbors (KNN) based on the minimum redundancy maximum relevance (mRMR)

In these matrices, the columns and rows show the predicted and actual values of the target variable, respectively. The actual values were compared with the values predicted by the machine learning model and measured the model’s performance.

According to the confusion matrices in Figures 3 and 4, sensitivity (SN), F1-Score, specificity (SP), and accuracy (ACC) parameters are calculated as the average of all classes in DT (Gini), RF, KNN. For each classifier based on the XGBoost and mRMR patterns, the total average of each metric was given in Tables 2 and 3.

Table 2 illustrates modeling based on the XGBoost algorithm, high accuracy of 89% was achieved by RF with SN 86%, SP 96%, and F1-score of 87%.

According to Table 3, the RF classifier based on mRMR has an accuracy of 74% with an SN of 65%, SP of 91%, and F1 score of 66%. The number of trees (n-estimator) in the RF algorithm was considered equal to 100.

The low accuracy is based on both feature selection algorithms related to the KNN classifier. The number of neighbors (K) in the KNN algorithm for the features obtained from the XGBoost and mRMR patterns were set to 5 and 7, respectively. The classifiers gained higher accuracy due to adjusting the mentioned parameters in these numbers.

Discussion

In medical imaging and oncology, tumor size and clinical biomarkers are used to assess treatment response [ 29 ]. Many researchers have examined the relationship between imaging features, such as tumor size on CT images, and predicting response to treatment. A group of international researchers developed response assessment criteria in solid tumors to simplify treatment response using World Health Organization (WHO) criteria [ 30 - 32 ].

The Response Evaluation Criteria in Solid Tumors (RECIST) are summarized based on the evaluation of target lesions on CT by one-dimensional measurement of the longest diameter of each target lesion [ 33 ]. However, these criteria alone do not describe and support all tumor information, and the RECIST criterion has its limitations [ 34 ]. In addition to morphological features, radiomics can address other features that characterize the tumor phenotype [ 35 ].

In this research, radiomic-based features were extracted from each CT image of patients and combined with clinical biomarkers (866 features). As mentioned before, because the dose injected into different patients varied according to the patient’s physical condition and test results, injection dose was considered a clinical feature. This biomarker (injection dose), identified as an important feature in both feature selection algorithms, is much more important than the quantitative features according to Figures 1 and 2 (P-value=2.29×10^-22). The present study was consistent with Coroller et al. [ 27 ] in correlations between clinical outcomes and quantitative imaging features. The relationship between features and response to treatment function was expressed based on the P-value (P<0.01). This means that the features obtained from different methods are more than 99.99% significantly related to the response to treatment.

The sum of the numbers in each row of the confusion matrices represents the number of samples available for the corresponding label. As shown in Figures 3 and 4, the number of samples is not the same for all labels (0-3). Accordingly, in the present study, imbalanced data, affecting the accuracy and performance of the model, was investigated. There are several ways to solve this problem to achieve a reliable result, the most important of which is the use of Decision Tree-based ensemble classifiers [ 36 ]. Therefore, in this study for modeling, the DT and RF classifiers were used, and the KNN classifier was also used because of the simplicity of its algorithm.

Confusion matrices were set up to represent the performance of the classifiers. In 2020, Bethanney Janney [ 37 ] analyzed skin lesions using machine learning methods and used a confusion matrix to evaluate model performance.

According to Tables 2 and 3, the KNN classifier performed weaker than the other classifiers in both feature selection methods due to imbalanced data. Therefore, this classifier is not a good predictor of response to treatment. Amongst other machine learning techniques, ensemble methods such as “RF” are premiere to solitary machine learning methods [ 21 ], and the results obtained from the classifiers used in this research indicate this (ACC_{in XGBoost FS+RF}=89%, ACC_{in mRMR FS+RF}=74%).

Some researchers have used receiver operating characteristics (ROC) curves and the area under the curve (A.U.C) to evaluate the performance of the classification model [ 12 , 38 ]. In 2020, Bian et al. [ 39 ] used a CT-based radiomics score to distinguish between grade 1 and grade 2 nonfunctioning pancreatic neuroendocrine tumors and also used multivariate logistic regression models that the score showed high accuracy (AUC=0.86 for all PNETs; AUC=0.81 for PNETs≤2 cm). Zhou et al. [ 38 ] conducted a study using CT-based radiomic signatures for a potential biomarker to predict preoperative recurrence in hepatocellular carcinoma and used the least absolute shrinkage and selection operator logistic regression model for creating radiomics signature combined with the clinical model. Further, Zhou et al. reported the area under the curve of operating characteristics (ROC) (AUCs: 0.781 (95% CI: 0.719-0.834) and (0.836 (95% CI: 0.779-0.883)) to predict their performance to discriminate early recurrence.

Based on the current study, the response to ¹⁷⁷Lu-DOTATATE treatment for patients with neuroendocrine tumors can be predicted using radiomic features of CT images. The characteristics of the XGBoost and mRMR patterns in combination with the RF classifier achieved high and moderate accuracy, respectively (Tables 2 and 3). Therefore, the XGBoost model performs better than mRMR in determining effective features. The features obtained by the XGBoost pattern can be extracted from the new patient’s CT image that these features are trained to the RF algorithm (due to their good performance), and the label of these features is predicted with high accuracy. Labels are predicted by coding in Python software. Label “1” is no response to treatment and labels “2” and “3” are a partial and complete response to treatment, respectively. The predicted label enables the specialist for a better decision about the treatment process and uses alternative therapies. Accordingly, one of the goals of radiomics, which is the personalization of treatment, is achieved [ 40 ].

One of the limitations of this study was CT images of patients taken at various medical centers. However, resampling was performed on all images and the voxels were the same size, different scans might cause potential noise in the image. The second challenge was the number of samples. Since these tumors are rare cancers, although several years have passed since the start of PRRT treatment in the hospital, the number of images obtained from patients for machine learning is less. It is suggested that in the future, more patients and images will be used to study this disease as well as treatment methods and increase the accuracy of the model. However, manual segmentation of the tumor is more accurate and robust than automatic and semi-automatic, it has less reproducibility [ 41 ]. Therefore, it is recommended to use this segmentation method and other classifiers to measure the accuracy of the model in future studies. Considering that in this study, ESAS-R and ECOG scales were used to evaluate the response to treatment and label the classification models, the RECIST scale can be used to predict response to treatment in future studies.

Conclusion

CT images of patients with NETs treated with PRRT were used to predict the response to ¹⁷⁷Lu-DOTATATE treatment based on the radiomics process, which used XGBoost and mRMR algorithms to decrease the number of features. In both methods, quantitative features in response to treatment took precedence over clinical biomarkers. Studies have shown that XGBoost is a more efficient algorithm than mRMR. The features obtained from the XGBoost method in combination with DT, RF, and KNN classifiers showed a much better and more satisfying performance in terms of the accuracy than mRMR algorithm. This study also showed that radiomics as a non-invasive and effective method could predict the response to the mentioned radiopharmaceutical treatment for patients with NETs, leading the specialist to decide easier whether or not to continue PRRT treatment for each patient.

Acknowledgment

In this research, we would like to thank the staff of the nuclear medicine imaging department of Shohada-Tajrish Hospital who provided valuable assistance. The present article is financially supported by “The Research Department of the School of Medicine Shahid Beheshti University of Medical Science” (Grant No 24938).

Authors’ Contribution

MR. Deevband as a corresponding author made substantial contributions to the design of the study. A. Abdi-Saray was responsible for the overall supervision of the work. B. Behmanesh as the first author drafted and critically reviewed the manuscript. M. Amoui as a nuclear medicine specialist treated the patients and played a key role in this project. HR. Haghighatkhah as a radiologist supervised the segmentation of tumor areas in CT images. All authors read and approved the final manuscript.

Ethical Approval

Ethical approval was obtained by the Ethics Committee of School of Medicine- Shahid Beheshti University of Medical Science (Ethics code: IR.SBMU.MSP.REC.1400.299).

Informed consent

Before treatment, consent was obtained from all patients.

Conflict of Interest

None

References

1. Mirvis E, Toumpanakis C, Mandair D, Gnanasegaran G, Caplin M, Navalkissoor S. Efficacy and tolerability of peptide receptor radionuclide therapy (PRRT) in advanced metastatic bronchial neuroendocrine tumours (NETs). Lung Cancer. 2020; 150:70-5. DOI | PubMed
2. Mittra ES. Neuroendocrine Tumor Therapy: 177Lu-DOTATATE. AJR Am J Roentgenol. 2018; 211(2):278-85. DOI | PubMed
3. Weilin M, Xu H, Yang L, Wenqi C, Huanyu W, et al. Propensity score-matched analysis of clinical outcome after enucleation versus regular pancreatectomy in patients with small non-functional pancreatic neuroendocrine tumors. Pancreatology. 2020; 20(2):169-76. DOI | PubMed
4. Chan DL, Singh S. Current Chemotherapy Use in Neuroendocrine Tumors. Endocrinol Metab Clin North Am. 2018; 47(3):603-14. DOI | PubMed
5. Chicheportiche A, Grozinsky-Glasberg S, Gross DJ, Krausz Y, et al. Predictive power of the post-treatment scans after the initial or first two courses of [177Lu]-DOTA-TATE. EJNMMI Phys. 2018; 5(1):36. Publisher Full Text | DOI | PubMed
6. Dash A, Pillai MR, Knapp FF. Production of (177)Lu for Targeted Radionuclide Therapy: Available Options. Nucl Med Mol Imaging. 2015; 49(2):85-107. Publisher Full Text | DOI | PubMed
7. Smith-Palmer J, Leeuwenkamp OR, Virk J, Reed N. Lutetium oxodotreotide (177Lu-Dotatate) for the treatment of unresectable or metastatic progressive gastroenteropancreatic neuroendocrine tumors: a cost-effectiveness analysis for Scotland. BMC Cancer. 2021; 21(1):10. Publisher Full Text | DOI | PubMed
8. Abou Jokh Casas E, Pubul Núñez V, Anido-Herranz U, et al. Evaluation of 177Lu-Dotatate treatment in patients with metastatic neuroendocrine tumors and prognostic factors. World J Gastroenterol. 2020; 26(13):1513-24. Publisher Full Text | DOI | PubMed
9. Levart D, Kalogianni E, Corcoran B, Mulholland N, Vivian G. Radiation precautions for inpatient and outpatient 177Lu-DOTATATE peptide receptor radionuclide therapy of neuroendocrine tumours. EJNMMI Phys. 2019; 6(1):7. Publisher Full Text | DOI | PubMed
10. Dromain C, Sundin A, Najran P, Vidal Trueba H, Dioguardi Burgio M, et al. Tumor Growth Rate to Predict the Outcome of Patients with Neuroendocrine Tumors: Performance and Sources of Variability. Neuroendocrinology. 2021; 111(9):831-9. DOI | PubMed
11. Liang W, Yang P, Huang R, Xu L, Wang J, Liu W, et al. A Combined Nomogram Model to Preoperatively Predict Histologic Grade in Pancreatic Neuroendocrine Tumors. Clin Cancer Res. 2019; 25(2):584-94. DOI
12. Zhao Z, Bian Y, Jiang H, Fang X, Li J, et al. CT-Radiomic Approach to Predict G1/2 Nonfunctional Pancreatic Neuroendocrine Tumor. Acad Radiol. 2020; 27(12):e272-81. DOI | PubMed
13. Horvat N, Bates DDB, Petkovska I. Novel imaging techniques of rectal cancer: what do radiomics and radiogenomics have to offer? A literature review. Abdom Radiol (NY).. 2019; 44(11):3764-74. Publisher Full Text | DOI | PubMed
14. Fave X, Zhang L, Yang J, Mackin D, Balter P, et al. Delta-radiomics features for the prediction of patient outcomes in non-small cell lung cancer. Sci Rep. 2017; 7(1):588. Publisher Full Text | DOI | PubMed
15. Meng Y, Sun J, Qu N, Zhang G, Yu T, Piao H. Application of Radiomics for Personalized Treatment of Cancer Patients. Cancer Manag Res. 2019; 11:10851-8. Publisher Full Text | DOI | PubMed
16. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther. 2001; 69(3):89-95. DOI | PubMed
17. Strosberg J, El-Haddad G, Wolin E, Hendifar A, Yao J, Chasen B, et al. Phase 3 Trial of 177Lu-Dotatate for Midgut Neuroendocrine Tumors. N Engl J Med. 2017; 376(2):125-35. Publisher Full Text | DOI | PubMed
18. Van Griethuysen JJM, Fedorov A, Parmar C, Hosny A, et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017; 77(21):e104-7. Publisher Full Text | DOI | PubMed
19. Larue RT, Defraene G, De Ruysscher D, Lambin P, van Elmpt W. Quantitative radiomics studies for tissue characterization: a review of technology and methodological procedures. Br J Radiol. 2017; 90(1070):20160665. Publisher Full Text | DOI | PubMed
20. Rizzo S, Botta F, Raimondi S, Origgi D, Fanciullo C, Morganti AG, Bellomi M. Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp. 2018; 2(1):36. Publisher Full Text | DOI | PubMed
21. Wang J, Xu J, Zhao C, Peng Y, Wang H. An ensemble feature selection method for high-dimensional data based on sort aggregation. Syst Sci Control Eng. 2019; 7(2):32-9. DOI
22. Berrendero JR, Cuevas A, Torrecilla JL. The mRMR variable selection method: a comparative study for functional data. J Stat Comput Simul. 2016; 86(5):891-907. DOI
23. Islam MR, Kamal ARM, Sultana N, Islam R, Moni MA, ulhaq A, et al. IEEE: Rajshahi, Bangladesh; 2018. DOI
24. Blanco-Justicia A, Domingo-Ferrer J. Machine learning explainability through comprehensible decision trees. International Cross-Domain Conference for Machine Learning and Knowledge Extraction; 2019: p. 15-26.
25. Forghani R, Savadjiev P, Chatterjee A, Muthukrishnan N, Reinhold C, Forghani B. Radiomics and Artificial Intelligence for Biomarker and Prediction Model Development in Oncology. Comput Struct Biotechnol J. 2019; 17:995-1008. Publisher Full Text | DOI | PubMed
26. Paul A, Mukherjee DP, Das P, Gangopadhyay A, Chintha AR, Kundu S. Improved Random Forest for Classification. IEEE Trans Image Process. 2018; 27(8):4012-24. DOI | PubMed
27. Coroller TP, Grossmann P, Hou Y, Rios Velazquez E, Leijenaar RT, et al. CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiother Oncol. 2015; 114(3):345-50. Publisher Full Text | DOI | PubMed
28. Susmaga R. Confusion matrix visualization. Intelligent information processing and web mining. Springer: Berlin, Heidelberg; 2004: p. 107-16.
29. Grimaldi S, Terroir M, Caramella C. Advances in oncological treatment: limitations of RECIST 1.1 criteria. Q J Nucl Med Mol Imaging. 2018; 62(2):129-39. DOI | PubMed
30. Green S, Weiss GR. Southwest Oncology Group standard response criteria, endpoint definitions and toxicity criteria. Invest New Drugs. 1992; 10(4):239-53. DOI | PubMed
31. Crombé A, Kind M, Ray-Coquard I, Isambert N, Chevreau C, André T, et al. Progressive Desmoid Tumor: Radiomics Compared With Conventional Response Criteria for Predicting Progression During Systemic Therapy-A Multicenter Study by the French Sarcoma Group. AJR Am J Roentgenol. 2020; 215(6):1539-48. DOI | PubMed
32. Ligero M, Garcia-Ruiz A, Viaplana C, Villacampa G, Raciti MV, Landa J, et al. A CT-based Radiomics Signature Is Associated with Response to Immune Checkpoint Inhibitors in Advanced Solid Tumors. Radiology. 2021; 299(1):109-19. DOI | PubMed
33. Vergote I, Rustin GJS, Eisenhauer EA, Kristensen GB, Pujade-Lauraine E, et al. Re: New Guidelines to Evaluate the Response to Treatment in Solid Tumors [Ovarian Cancer]. J Natl Cancer Inst. 2000; 92(18):1534-5.
34. Nishino M. Tumor Response Assessment for Precision Cancer Therapy: Response Evaluation Criteria in Solid Tumors and Beyond. Am Soc Clin Oncol Educ Book. 2018; 38:1019-29. DOI | PubMed
35. Huynh E, Coroller TP, Narayan V, Agrawal V, Hou Y, et al. CT-based radiomic analysis of stereotactic body radiation therapy patients with lung cancer. Radiother Oncol. 2016; 120(2):258-66. DOI | PubMed
36. Longadge R, Dongre S. Class Imbalance Problem in Data Mining Review. IJCSN. 2013; 2:1305-11.
37. Bethanney Janney J, Roslin SE, Kumar SK. Computational Intelligence and Its Applications in Healthcare. Elsevier Inc Academic Press; 2020.
38. Zhou Y, He L, Huang Y, Chen S, Wu P, Ye W, et al. CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma. Abdom Radiol (NY).. 2017; 42(6):1695-704. DOI | PubMed
39. Bian Y, Jiang H, Ma C, Wang L, Zheng J, Jin G, Lu J. CT-Based Radiomics Score for Distinguishing Between Grade 1 and Grade 2 Nonfunctioning Pancreatic Neuroendocrine Tumors. AJR Am J Roentgenol. 2020; 215(4):852-63. DOI | PubMed
40. Sala E, Mema E, Himoto Y, Veeraraghavan H, Brenton JD, Snyder A, Weigelt B, Vargas HA. Unravelling tumour heterogeneity using next-generation imaging: radiomics, radiogenomics, and habitat imaging. Clin Radiol. 2017; 72(1):3-10. Publisher Full Text | DOI | PubMed
41. Hunter LA, Krafft S, Stingo F, Choi H, Martel MK, Kry SF, Court LE. High quality machine-robust image features: identification in nonsmall cell lung cancer computed tomography images. Med Phys. 2013; 40(12):121916. Publisher Full Text | DOI | PubMed