Document Type : Original Research
Authors
- Mahmoud Bagheri 1, 2
- Alireza Ghanadan 3
- Mobin Saboohi 1
- Maryam Daneshpazhooh 3
- Fatemeh Atyabi 4
- Marjaneh Hejazi 1, 2
1 Department of Medical Physics and Biomedical Engineering, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
2 Research Center for Molecular and Cellular Imaging, Bio-Optical Imaging Group, Tehran University of Medical Sciences, Tehran, Iran
3 Department of Dermatology, Razi Hospital, Tehran University of Medical Sciences, Tehran, Iran
4 Department of Pharmaceutical Nanotechnology, Faculty of Pharmacy, Tehran University of Medical Sciences, Tehran, Iran
Abstract
Background: The use of Hematoxylin-and-Eosin (H&E) staining is widely accepted as the most reliable method for diagnosing pathological tissues. However, the conventional H&E staining process for tissue sections is time-consuming and requires significant labor. In contrast, Confocal Microscopy (CM) enables quick and high-resolution imaging with minimal tissue preparation by fluorescence detection. However, it seems harder to interpret images from CM than H&E-stained images.
Objective: This study aimed to modify an unsupervised deep-learning model to generate H&E-like images from CM images.
Material and Methods: This analytical study evaluated the efficacy of CM and virtual H&E staining for skin tumor sections related to Basal Cell Carcinoma (BCC). The acridine orange staining, combined with virtual staining techniques, was used to simulate H&E dyes; accordingly, an unsupervised CycleGAN framework, trained to virtually stain CM images was implemented. The training process incorporated adversarial and cycle consistency losses to ensure a precise mapping between CM and H&E images without compromising image content. The quality of the generated images was assessed by comparing them to the original images.
Results: The CM images, specifically focusing on subtyping BCC and evaluating skin tissue characteristics, were qualitatively assessed. The H&E-like images generated from CM using the CycleGAN model exhibited both qualitative and quantitative similarities to real H&E images.
Conclusion: The integration of CM with deep learning-based virtual staining provides advantages for diagnostic applications by streamlining laboratory staining procedures.
Highlights
Mahmoud Bagheri (Google Scholar)
Marjaneh Hejazi (Google Scholar)
Keywords
- Muzic JG, Schmitt AR, Wright AC, Alniemi DT, Zubair AS, Olazagasti Lourido JM, et al. Incidence and Trends of Basal Cell Carcinoma and Cutaneous Squamous Cell Carcinoma: A Population-Based Study in Olmsted County, Minnesota, 2000 to 2010. Mayo Clin Proc. 2017;92(6):890-8. doi: 10.1016/j.mayocp.2017.02.015. PubMed PMID: 28522111. PubMed PMCID: PMC5535132.
- Boktor M, Ecclestone BR, Pekar V, Dinakaran D, Mackey JR, Fieguth P, Haji Reza P. Virtual histological staining of label-free total absorption photoacoustic remote sensing (TA-PARS). Sci Rep. 2022;12(1):10296. doi: 10.1038/s41598-022-14042-y. PubMed PMID: 35717539. PubMed PMCID: PMC9206643.
- Gao XH, Li J, Gong HF, Yu GY, Liu P, Hao LQ, et al. Comparison of Fresh Frozen Tissue With Formalin-Fixed Paraffin-Embedded Tissue for Mutation Analysis Using a Multi-Gene Panel in Patients With Colorectal Cancer. Front Oncol. 2020;10:310. doi: 10.3389/fonc.2020.00310. PubMed PMID: 32232001. PubMed PMCID: PMC7083147.
- Pradhan P, Meyer T, Vieth M, Stallmach A, Waldner M, Schmitt M, et al. Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning. Biomed Opt Express. 2021;12(4):2280-98. doi: 10.1364/BOE.415962. PubMed PMID: 33996229. PubMed PMCID: PMC8086483.
- Liu S, Zhang B, Liu Y, Han A, Shi H, Guan T, He Y. Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks. IEEE Trans Med Imaging. 2021;40(8):1977-89. doi: 10.1109/TMI.2021.3069874. PubMed PMID: 33784619.
- Hartmann D, Ruini C, Mathemeier L, Dietrich A, Ruzicka T, Von Braunmühl T. Identification of ex-vivo confocal scanning microscopic features and their histological correlates in human skin. J Biophotonics. 2016;9(4):376-87. doi: 10.1002/jbio.201500124. PubMed PMID: 25996548.
- Tsai ST, Liu CH, Chan CC, Li YH, Huang SL, Chen HH. H&E-like staining of OCT images of human skin via generative adversarial network. Appl Phys Lett. 2022;121(13):134102. doi: 10.1063/5.0122965.
- Borhani N, Bower AJ, Boppart SA, Psaltis D. Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy. Biomed Opt Express. 2019;10(3):1339-50. doi: 10.1364/BOE.10.001339. PubMed PMID: 30891350. PubMed PMCID: PMC6420275.
- Ortner VK, Sahu A, Cordova M, Kose K, Aleissa S, Alessi-Fox C, et al. Exploring the utility of Deep Red Anthraquinone 5 for digital staining of ex vivo confocal micrographs of optically sectioned skin. J Biophotonics. 2021;14(4):e202000207. doi: 10.1002/jbio.202000207. PubMed PMID: 33314673. PubMed PMCID: PMC8274380.
- Guida S, Arginelli F, Farnetani F, Ciardo S, Bertoni L, Manfredini M, et al. Clinical applications of in vivo and ex vivo confocal microscopy. Appl Sci. 2021;11(5):1979. soi: 10.3390/app11051979.
- Bini J, Spain J, Nehal K, Hazelwood V, DiMarzio C, Rajadhyaksha M. Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance. J Biomed Opt. 2011;16(7):076008. doi: 10.1117/1.3596742. PubMed PMID: 21806269. PubMed PMCID: PMC3154052.
- Xu Z, Huang X, Moro CF, Bozóky B, Zhang Q. GAN-based virtual re-staining: a promising solution for whole slide image analysis [Internet]. arXiv [Preprint]. 2019 [cited 2019 Jan 13]. Available from: https://arxiv.org/abs/1901.04059.
- Lee G, Oh JW, Her NG, Jeong WK. DeepHCS++: Bright-field to fluorescence microscopy image conversion using multi-task learning with adversarial losses for label-free high-content screening. Med Image Anal. 2021;70:101995. doi: 10.1016/j.media.2021.101995. PubMed PMID: 33640720.
- Han S, Lee S, Chen A, Yang C, Salama P, Dunn KW, Delp EJ. Three dimensional nuclei segmentation and classification of fluorescence microscopy images. 17th International Symposium on Biomedical Imaging (ISBI); Iowa City, IA, USA: IEEE; 2020.
- Winetraub Y, Yuan E, Terem I, Yu C, Chan W, Do H, et al. OCT2Hist: Non-invasive virtual biopsy using optical coherence tomography [Internet]. medRxiv [Preprint]. 2021 [cited 2021 Apr 6]. Available from: https://www.medrxiv.org/content/10.1101/2021.03.31.21254733v1.
- Ruini C, Vladimirova G, Kendziora B, Salzer S, Ergun E, Sattler E, et al. Ex-vivo fluorescence confocal microscopy with digital staining for characterizing basal cell carcinoma on frozen sections: A comparison with histology. J Biophotonics. 2021;14(8):e202100094. doi: 10.1002/jbio.202100094. PubMed PMID: 33991061.
- Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (ICCV); Venice, Italy: ICCV; 2017. p. 2223-32.
- Van Der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9(11):2579-605.
- Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: A review. Med Image Anal. 2019;58:101552. doi: 10.1016/j.media.2019.101552. PubMed PMID: 31521965.
- Nečasová T, Burgos N, Svoboda D. Validation and evaluation metrics for medical and biomedical image synthesis. In Biomedical Image Synthesis and Simulation; Academic Press; 2022. p. 573-600.
- Gareau DS. Feasibility of digitally stained multimodal confocal mosaics to simulate histopathology. J Biomed Opt. 2009;14(3):034050. doi: 10.1117/1.3149853. PubMed PMID: 19566342. PubMed PMCID: PMC2929174.
- Vladimirova G, Ruini C, Kapp F, Kendziora B, Ergün EZ, Bağcı IS, et al. Ex vivo confocal laser scanning microscopy: A diagnostic technique for easy real-time evaluation of benign and malignant skin tumours. J Biophotonics. 2022;15(6):e202100372. doi: 10.1002/jbio.202100372. PubMed PMID: 35233962.
- Bai B, Yang X, Li Y, Zhang Y, Pillar N, Ozcan A. Deep learning-enabled virtual histological staining of biological samples. Light Sci Appl. 2023;12(1):57. doi: 10.1038/s41377-023-01104-7. PubMed PMID: 36864032. PubMed PMCID: PMC9981740.
- Rivenson Y, Wang H, Wei Z, De Haan K, Zhang Y, Wu Y, et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng. 2019;3(6):466-77. doi: 10.1038/s41551-019-0362-y. PubMed PMID: 31142829.
- Lo YC, Chung IF, Guo SN, Wen MC, Juang CF. Cycle-consistent GAN-based stain translation of renal pathology images with glomerulus detection application. Appl Soft Comput. 2021;98:106822. doi: 10.1016/j.asoc.2020.106822.
- Runz M, Rusche D, Schmidt S, Weihrauch MR, Hesser J, Weis CA. Normalization of HE-stained histological images using cycle consistent generative adversarial networks. Diagn Pathol. 2021;16(1):71. doi: 10.1186/s13000-021-01126-y. PubMed PMID: 34362386. PubMed PMCID: PMC8349020.
)