Document Type : Blackboard


PhD, Department of Computer Engineering, Faculty of Electrical and Computer Engineering, University of Kashan, Kashan, Iran


Nowadays, medical image modalities are almost available everywhere. These modalities are bases of diagnosis of various diseases sensitive to specific tissue type. Usually physicians look for abnormalities in these modalities in diagnostic procedures. Count and volume of abnormalities are very important for optimal treatment of patients. Segmentation is a preliminary step for these measurements and also further analysis. Manual segmentation of abnormalities is cumbersome, error prone, and subjective. As a result, automated segmentation of abnormal tissue is a need. In this study, representative techniques for segmentation of abnormal tissues are reviewed. Main focus is on the segmentation of multiple sclerosis lesions, breast cancer masses, lung nodules, and skin lesions. As experimental results demonstrate, the methods based on deep learning techniques perform better than other methods that are usually based on handy feature engineering techniques. Finally, the most common measures to evaluate automated abnormal tissue segmentation methods are reported.


  1. Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, man, and Sybernetics. 1979;9:62-6. doi: 10.1109/TSMC.1979.4310076.
  2. Shi J, Malik J. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2000;22:888-905. doi: 10.1109/34.868688.
  3. Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2001;23:1222-39. doi: 10.1109/34.969114.
  4. Suzuki K. Pixel-based machine learning in medical imaging. Int J Biomed Imaging. 2012;2012:792079. doi: 10.1155/2012/792079. PubMed PMID: 22481907. PubMed PMCID: PMC3299341.
  5. Khastavaneh H, Haron H, editors. False Positives Reduction on Segmented Multiple Sclerosis Lesions Using Fuzzy Inference System by Incorporating Atlas Prior Anatomical Knowledge: A Conceptual Model. International Conference on Computational Collective Intelligence; Springer; 2014. p. 11-9. doi: 10.1007/978-3-319-11289-3_2.
  6. Shen S, Szameitat AJ, Sterr A. An improved lesion detection approach based on similarity measurement between fuzzy intensity segmentation and spatial probability maps. Magn Reson Imaging. 2010;28:245-54. doi: 10.1016/j.mri.2009.06.007. PubMed PMID: 19695812.
  7. Wu Y, Warfield SK, Tan IL, Wells III WM, Meier DS, Van Schijndel RA, et al. Automated segmentation of multiple sclerosis lesion subtypes with multichannel MRI. Neuroimage. 2006;32:1205-15. doi: 10.1016/j.neuroimage.2006.04.211. PubMed PMID: 16797188.
  8. Simoes R, Monninghoff C, Dlugaj M, Weimar C, Wanke I, et al. Automatic segmentation of cerebral white matter hyperintensities using only 3D FLAIR images. Magn Reson Imaging. 2013;31:1182-9. doi: 10.1016/j.mri.2012.12.004. PubMed PMID: 23684961.
  9. Prastawa M, Gerig G. Automatic MS lesion segmentation by outlier detection and information theoretic region partitioning. Grand Challenge Work: Mult Scler Lesion Segm Challenge. 2008:1-8.
  10. Ong KH, Ramachandram D, Mandava R, Shuaib IL. Automatic white matter lesion segmentation using an adaptive outlier detection method. Magn Reson Imaging. 2012;30:807-23. doi: 10.1016/j.mri.2012.01.007. PubMed PMID: 22578927.
  11. Xie Y, Tao X, editors. White matter lesion segmentation using machine learning and weakly labeled MR images. Florida, United States: SPIE Medical Imaging; 2011. doi: 10.1117/12.878237.
  12. Anbeek P, Vincken KL, Viergever MA. Automated MS-lesion segmentation by k-nearest neighbor classification. MIDAS Journal. 2008:1-8.
  13. Harmouche R, Collins L, Arnold D, Francis S, Arbel T. Bayesian MS lesion classification modeling regional and local spatial information. 18th International Conference on Pattern Recognition (ICPR’06); Hong Kong, China: IEEE; 2006. p. 984-7.
  14. Yamamoto D, Arimura H, Kakeda S, Magome T, Yamashita Y, Toyofuku F, et al. Computer-aided detection of multiple sclerosis lesions in brain magnetic resonance images: False positive reduction scheme consisted of rule-based, level set method, and support vector machine. Comput Med Imaging Graph. 2010;34:404-13. doi: 10.1016/j.compmedimag.2010.02.001. PubMed PMID: 20189353.
  15. Schmidt P, Gaser C, Arsic M, Buck D, Forschler A, Berthele A, et al. An automated tool for detection of FLAIR-hyperintense white-matter lesions in Multiple Sclerosis. Neuroimage. 2012;59:3774-83. doi: 10.1016/j.neuroimage.2011.11.032. PubMed PMID: 22119648.
  16. Abdullah BA, Younis AA, Pattany PM, Saraf-Lavi E. Textural based SVM for MS lesion segmentation in FLAIR MRIs. Open Journal of Medical Imaging. 2011;1:26-42. doi: 10.4236/ojmi.2011.12005.
  17. Cabezas M, Oliver A, Freixenet J, Lladó X, editors. A supervised approach for multiple sclerosis lesion segmentation using context features and an outlier map. Iberian conference on pattern recognition and image analysis; Springer; 2013. p. 782-9
  18. Geremia E, Clatz O, Menze BH, Konukoglu E, Criminisi A, Ayache N. Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images. Neuroimage. 2011;57:378-90. doi: 10.1016/j.neuroimage.2011.03.080. PubMed PMID: 21497655.
  19. Anbeek P, Vincken KL, Van Bochove GS, Van Osch MJ, Van Der Grond J. Probabilistic segmentation of brain tissue in MR imaging. Neuroimage. 2005;27:795-804. doi: 10.1016/j.neuroimage.2005.05.046. PubMed PMID: 16019235.
  20. Anbeek P, Vincken KL, Van Osch MJ, Bisschops RH, Van Der Grond J. Automatic segmentation of different-sized white matter lesions by voxel probability estimation. Med Image Anal. 2004;8:205-15. doi: 10.1016/ PubMed PMID: 15450216.
  21. Zacharaki EI, Kanterakis S, Bryan RN, Davatzikos C. Measuring brain lesion progression with a supervised tissue classification system. Med Image Comput Comput Assist Interv. 2008;11:620-7.doi: 10.1007/978-3-540-85988-8_74. PubMed PMID: 18979798.
  22. Ferrari RJ, Wei X, Zhang Y, Scott JN, Mitchell JR, editors. Segmentation of multiple sclerosis lesions using support vector machines. California, United States: SPIE Medical Imaging; 2003. p. 16–26. doi: 10.1117/12.481377.
  23. Akselrod-Ballin A, Galun M, Basri R, Brandt A, Gomori MJ, Filippi M, et al., editors. An integrated segmentation and classification approach applied to multiple sclerosis analysis. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06); USA: IEEE; 2006. p. 1122-9. doi: 10.1109/cvpr.2006.55.
  24. Akselrod-Ballin A, Galun M, Gomori JM, Filippi M, Valsasina P, Basri R, et al. Automatic segmentation and classification of multiple sclerosis in multichannel MRI. IEEE Trans Biomed Eng. 2009;56:2461-9. doi: 10.1109/TBME.2008.926671. PubMed PMID: 19758850.
  25. Haralick RM, Shanmugam K. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics. 1973;6:610-21. doi: 10.1109/TSMC.1973.4309314.
  26. Galloway MM. Texture analysis using grey level run lengths. Computer Graphics and Image Processing. 1975;4:172-9. doi: 10.1016/S0146-664X(75)80008-6.
  27. Khastavaneh H, Ebrahimpour-Komleh H. Neural Network-Based Learning Kernel for Automatic Segmentation of Multiple Sclerosis Lesions on Magnetic Resonance Images. J Biomed Phys Eng. 2017;7:155-62. PubMed PMID: 28580337. PubMed PMCID: PMC5447252.
  28. Brosch T, Yoo Y, Tang LY, Li DK, Traboulsee A, Tam R, editors. Deep convolutional encoder networks for multiple sclerosis lesion segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention; Springer; 2015. p. 3-11. doi: 10.1007/978-3-319-24574-4_1.
  29. Brosch T, Tang LY, Youngjin Y, Li DK, Traboulsee A, Tam R. Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation. IEEE Trans Med Imaging. 2016;35:1229-39. doi: 10.1109/TMI.2016.2528821. PubMed PMID: 26886978.
  30. Moeskops P, Viergever MA, Mendrik AM, De Vries LS, Benders MJ, Isgum I. Automatic Segmentation of MR Brain Images With a Convolutional Neural Network. IEEE Trans Med Imaging. 2016;35:1252-61. doi: 10.1109/TMI.2016.2548501. PubMed PMID: 27046893.
  31. Ghafoorian M, Karssemeijer N, Heskes T, Van Uden IWM, Sanchez CI, Litjens G, et al. Location Sensitive Deep Convolutional Neural Networks for Segmentation of White Matter Hyperintensities. Sci Rep. 2017;7:5110. doi: 10.1038/s41598-017-05300-5. PubMed PMID: 28698556. PubMed PMCID: PMC5505987.
  32. Bae MS, Moon WK, Chang JM, Koo HR, Kim WH, Cho N, et al. Breast cancer detected with screening US: reasons for nondetection at mammography. Radiology. 2014;270:369-77. doi: 10.1148/radiol.13130724. PubMed PMID: 24471386.
  33. Henriksen EL, Carlsen JF, Vejborg IM, Nielsen MB, Lauridsen CA. The efficacy of using computer-aided detection (CAD) for detection of breast cancer in mammography screening: a systematic review. Acta Radiol. 2019;60(1):13-8. doi: 10.1177/0284185118770917. PubMed PMID: 29665706.
  34. Ghongade R, Wakde D, editors. Breast Cancer Diagnosis from Digital Mammograms Using RF and RF-ELM. Proceedings of International Conference on Recent Advancement on Computer and Communication; Springer; 2018. p. 365-74. doi: 10.1007/978-981-10-8198-9_38.
  35. Sheba K, Gladston Raj S. An approach for automatic lesion detection in mammograms. Cogent Eng. 2018;5:1444320. doi: 10.1080/23311916.2018.1444320.
  36. Shi P, Zhong J, Rampun A, Wang H. A hierarchical pipeline for breast boundary segmentation and calcification detection in mammograms. Comput Biol Med. 2018;96:178-88. doi: 10.1016/j.compbiomed.2018.03.011. PubMed PMID: 29597143.
  37. Ronneberger O, Fischer P, Brox T, editors. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention; Springer; 2015. p. 234-41. doi: 10.1007/978-3-319-24574-4_28.
  38. De Moor T, Rodriguez-Ruiz A, Mann R, Teuwen J. Automated lesion detection and segmentation in digital mammography using a u-net deep learning network. 14th International Workshop on Breast Imaging (IWBI); United States: IWBI; 2018. doi: 10.1117/12.2318326.
  39. Lee SLA, Kouzani AZ, Hu EJ. Automated detection of lung nodules in computed tomography images: a review. Machine Vision and Applications. 2012;23:151-63. doi: 10.1007/s00138-010-0271-2.
  40. Suzuki K, Doi K. How can a massive training artificial neural network (MTANN) be trained with a small number of cases in the distinction between nodules and vessels in thoracic CT? Acad Radiol. 2005;12:1333-41. doi: 10.1016/j.acra.2005.06.017. PubMed PMID: 16179210.
  41. Arimura H, Katsuragawa S, Suzuki K, Li F, Shiraishi J, Sone S, et al. Computerized scheme for automated detection of lung nodules in low-dose computed tomography images for lung cancer screening. Acad Radiol. 2004;11:617-29. doi: 10.1016/j.acra.2004.02.009. PubMed PMID: 15172364.
  42. Shi Z, Zhao M, He L, Wang Y, Zhang M, Suzuki K. A computer aided pulmonary nodule detection system using multiple massive training SVMs. Applied Mathematics & Information Sciences. 2013;7:1165. doi: 10.12785/amis/070339.
  43. Hua KL, Hsu CH, Hidayati SC, Cheng WH, Chen YJ. Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther. 2015;8:2015-22. doi: 10.2147/OTT.S80733. PubMed PMID: 26346558. PubMed PMCID: PMC4531007.
  44. Setio AA, Ciompi F, Litjens G, Gerke P, Jacobs C, Van Riel SJ, et al. Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks. IEEE Trans Med Imaging. 2016;35:1160-9. doi: 10.1109/TMI.2016.2536809. PubMed PMID: 26955024.
  45. Van Tulder G, De Bruijne M. Combining Generative and Discriminative Representation Learning for Lung CT Analysis With Convolutional Restricted Boltzmann Machines. IEEE Trans Med Imaging. 2016;35:1262-72. doi: 10.1109/TMI.2016.2526687. PubMed PMID: 26886968.
  46. Pardo A, Real E, Fernandez-Barreras G, Madruga F, López-Higuera JM, Conde O, editors. Automated skin lesion segmentation with kernel density estimation. European Conference on Biomedical Optics; Germany: SPIE; 2017. p. 8. doi: 10.1117/12.2283038.
  47. Nasir M, Attique Khan M, Sharif M, Lali IU, Saba T, Iqbal T. An improved strategy for skin lesion detection and classification using uniform segmentation and feature selection based approach. Microsc Res Tech. 2018;81:528-43. doi: 10.1002/jemt.23009. PubMed PMID: 29464868.
  48. Meskini E, Helfroush MS, Kazemi K, Sepaskhah M. A New Algorithm for Skin Lesion Border Detection in Dermoscopy Images. J Biomed Phys Eng. 2018;8(1):117-26. PubMed PMID: 29732346. PubMed PMCID: PMC5928301.
  49. He Y, Xie F, editors. Automatic skin lesion segmentation based on texture analysis and supervised learning. Asian Conference on Computer Vision; Springer; 2013. p. 330-41. doi: 10.1007/978-3-642-37444-9_26.
  50. Lin BS, Michael K, Kalra S, Tizhoosh HR, editors. Skin lesion segmentation: U-nets versus clustering. IEEE Symposium Series on Computational Intelligence (SSCI); USA: IEEE; 2017. p. 1–7. doi: 10.1109/ssci.2017.8280804.
  51. Yuan Y, Chao M, Lo YC. Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks With Jaccard Distance. IEEE Trans Med Imaging. 2017;36:1876-86. doi: 10.1109/TMI.2017.2695227. PubMed PMID: 28436853.