Neural Networks for Medical Data Processing: Segmentation of Fluorescence Lifetime Imaging Microscopy

Ilya Shchechkin (Login required)
Privolzhsky Research Medical University, Nizhny Novgorod, Russian Federation
N.I. Lobachevsky Nizhny Novgorod National Research State University, Russian Federation

Svetlana Rodimova
Privolzhsky Research Medical University, Nizhny Novgorod, Russian Federation

Artem Mozherov
I.M. Sechenov First Moscow State Medical University, Russian Federation

Vadim Elagin
Privolzhsky Research Medical University, Nizhny Novgorod, Russian Federation

Nikolai Bobrov
The Volga District Medical Centre of Federal Medical and Biological Agency, Nizhny Novgorod, Russian Federation

Vladislav Shcheslavsky
Privolzhsky Research Medical University, Nizhny Novgorod, Russian Federation

Vladimir Zagainov
The Volga District Medical Centre of Federal Medical and Biological Agency, Nizhny Novgorod, Russian Federation
Nizhny Novgorod Regional Clinical Oncologic Dispensary, Russian Federation

Elena Zagaynova
Privolzhsky Research Medical University, Nizhny Novgorod, Russian Federation
Lopukhin Federal Research and Clinical Center of Physical-Chemical Medicine of Federal Medical Biological Agency, Moscow, Russian Federation

Daria Kuznetsova
Privolzhsky Research Medical University, Nizhny Novgorod, Russian Federation
I.M. Sechenov First Moscow State Medical University, Russian Federation




DOI: 10.18287/JBPE25.11.030304

Abstract

Fluorescence lifetime imaging microscopy (FLIM) is a powerful tool for visualizing the distribution of fluorescence lifetimes of fluorophores within a tissue, and is used for optical metabolic imaging. However, processing the large volumes of data generated by FLIM can be time-consuming and challenging for an untrained operator. Convolutional neural networks (CNNs) have demonstrated its effectiveness for tissue image and FLIM segmentation, making them a promising tool for automated analysis of FLIM images. The article presents a method for segmenting FLIM images of liver tissue during the regeneration using a CNN based on the UNet++ architecture. The CNN was trained to detect cell boundaries and nuclei, and the resulting masks were used to approximate the fluorescence decay curves for the FLIM-image data. The results show that the CNN is effective in segmenting FLIM images of cells and tissues, and obtaining fluorescence decay curves. The use of CNNs and other machine learning methods in medical image segmentation and analysis will improve and speed up diagnosis and reduce diagnostic errors.

Keywords

deep learning; segmentation; convolutional neural networks; FLIM; liver regeneration; multiphoton microscopy

Full Text:

PDF SUPPL.FILE

References


1. W. Becker, “Fluorescence lifetime imaging – techniques and applications,” Journal of Microscopy 247(2), 119–136 (2012).

2. V. Mannam, Y. Zhang, X. Yuan, T. Hato, P. C. Dagher, E. L. Nichols, C. J. Smith, K. W. Dunn, and S. Howard, “Convolutional neural network denoising in fluorescence lifetime imaging microscopy (FLIM),” in Multiphoton Microscopy in the Biomedical Sciences XXI, A. Periasamy, P. T. So, K. König (Eds.), SPIE Proceedings 11648, 116481C (2021).

3. P. Kumar, P. Nagar, C. Arora, and A. Gupta, “U-Segnet: Fully convolutional neural network based automated brain tissue segmentation tool,” 25th IEEE International Conference on Image Processing (ICIP) (IEEE, 2018), 3503–3507 (2018).

4. F. Ouhmich, V. Agnus, V. Noblet, F. Heitz, and P. Pessaux, “Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks,” International Journal of Computer Assisted Radiology and Surgery 14(8), 1275–1284 (2019).

5. E. A. Shirshin, M. V. Shirmanova, A. V. Gayer, M. M. Lukina, E. E. Nikonova, B. P. Yakimov, G. S. Budylin, V. V. Dudenkova, N. I. Ignatova, D. V. Komarov, V. V. Yakovlev, W. Becker, E. V. Zagaynova, V. I. Shcheslavskiy, and M. O. Scully, “Label-free sensing of cells with fluorescence lifetime imaging: The quest for metabolic heterogeneity,” Proceedings of the National Academy of Sciences of the United States of America 119(9), e2118241119 (2022).

6. S. Rodimova, N. Bobrov, A. Mozherov, V. Elagin, M. Karabut, I. Shchechkin, D. Kozlov, D. Krylov, A. Gavrina, V. Zagainov, E. Zagaynova, and D. Kuznetsova, “Optical biomedical imaging reveals criteria for violated liver regenerative potential,” Cells 12(3), 479 (2023).

7. S. Rodimova, D. Kuznetsova, N. Bobrov, V. Elagin, V. Shcheslavskiy, V. Zagainov, and E. Zagaynova, “Mapping metabolism of liver tissue using two-photon FLIM,” Biomedical Optics Express 11(8), 4458 (2020).

8. F. Nassir, R. S. Rector, G. M. Hammoud, and J. A. Ibdah, “Pathogenesis and prevention of hepatic steatosis,” Gastroenterlogy and Hepatology 11, 167–175 (2015).

9. C. Wu, Z. Feng, H. Zhang, and H. Yan, “Graph neural network and superpixel based brain tissue segmentation,” Proceedings of International Joint Conference on Neural Networks (IJCNN) (2022).

10. H. Rahman, T. F. N. Bukht, A. Imran, J. Tariq, S. Tu, and A. Alzahrani, “A deep learning approach for liver and tumor segmentation in CT images using ResUNet,” Bioengineering 9(8), 368 (2022).

11. H. Zhang, K. Luo, R. Deng, S. Li, and S. Duan, “Deep learning-based CT imaging for the diagnosis of liver tumor,” Computational Intelligence and Neuroscience 2022, 3045370 (2022).

12. D. Ramachandram, J. L. Ramirez-GarciaLuna, R. D. J. Fraser, M. A. Martínez-Jiménez, J. E. Arriaga-Caballero, and J. Allport, “Fully automated wound tissue segmentation using deep learning on mobile devices: Cohort study,” Journal of Medical Internet Research Mhealth Uhealth 10(4), e36977 (2022).

13. E. M. Darling, F. Guilak, “A neural network model for cell classification based on single-cell biomechanical properties,” Tissue Engineering Part A 14(9), 1507–1515 (2008).

14. C. Laiton-Bonadiez, G. Sanchez-Torres, and J. Branch-Bedoya, “Deep 3D neural network for brain structures segmentation using self-attention modules in MRI images,” Sensors 22(7), 2559 (2022).

15. M. A. K. Sagar, K. P. Cheng, J. N. Ouellette, J. C. Williams, J. J. Watters, and K. W. Eliceiri, “Machine learning methods for fluorescence lifetime imaging (FLIM) based label-free detection of microglia,” Frontiers in Neuroscience 14, 931 (2020).

16. Y. Ding, R. Acosta, V. Enguix, S. Suffren, J. Ortmann, D. Luck, J. Dolz, and G. A. Lodygensky, “Using deep convolutional neural networks for neonatal brain image segmentation,” Frontiers in Neuroscience 14, 207 (2020).

17. D. Hu, P. Sarder, P. Ronhovde, S. Orthaus, S. Achilefu, and Z. Nussinov, “Automatic segmentation of fluorescence lifetime microscopy images of cells using multiresolution community detection—a first study,” Journal of Microscopy 253(1), 54–64 (2014).

18. Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “UNet++: A nested U-Net architecture for medical image segmentation,” in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support 11045, 3–11 (2018).

19. M. T. Duong, J. D. Rudie, J. Wang, L. Xie, S. Mohan, J. C. Gee, and A. M. Rauschecker, “Convolutional neural network for automated FLAIR lesion segmentation on clinical brain MR imaging,” American Journal of Neuroradiology 40(8), 1282–1290 (2019).

20. E. V. Potapova, V. V. Shupletsov, V. V. Dremin, E. A. Zherebtsov, A. V. Mamoshin, and A. V. Dunaev, “In Vivo time-resolved fluorescence detection of liver cancer supported by machine learning,” Lasers in Surgery and Medicine 56(10), 836–844 (2024).

21. M. V. Shirmanova, V. I. Shcheslavskiy, M. M. Lukina, W. Becker, and E. V. Zagaynova, “Exploring tumor metabolism with time-resolved fluorescence methods: from single cells to a whole tumor,” Chapter 3 in Multimodal Optical Diagnostics of Cancer, V. V. Tuchin, J. Popp, and V. Zakharov (Eds.), Springer Cham, 133–155 (2020). eBook ISBN: 978-3-030-44594-2.

22. A. V. Meleshina, V. V. Dudenkova, A. S. Bystrova, D. S. Kuznetsova, M. V. Shirmanova, and E. V. Zagaynova, “Two-photon FLIM of NAD(P)H and FAD in mesenchymal stem cells undergoing either osteogenic or chondrogenic differentiation,” Stem Cell Research & Therapy 8(1), 15 (2017).

23. S. A. Rodimova, A. V. Meleshina, E. P. Kalabusheva, E. B. Dashinimaev, D. G. Reunov, H. G. Torgomyan, E. A. Vorotelyak, and E. V. Zagaynova, “Metabolic activity and intracellular pH in induced pluripotent stem cells differentiating in dermal and epidermal directions,” Methods and Applications in Fluorescence 7(4), 044002 (2019).

24. N. Shahid, T. Rappon, and W. Berta, “Applications of artificial neural networks in health care organizational decision-making: A scoping review,” PLoS ONE 14(2), e0212356 (2019).




Bookmark and Share


Сontact

34 Moskovskoe shosse, Samara, 443086, Russian Federation
Email: j-bpe@ssau.ru
Phone: +7-846-267-4550
© 2014-2025 J-BPE