Multimodal neural network analysis of Raman spectra and dermoscopic images of skin tumors
I. Matveeva1
1- Samara National Research University, Moskovskoye shosse 34, Samara, 443086, Russia
m-irene-a@yandex. ru
Accurate diagnosis of the cancer type and early diagnosis are key factors in the successful treatment of neoplasms [1]. Diagnosis of neoplasms by visual examination is difficult, since malignant and benign neoplasms may have similar external signs [2]. In addition to visual examination, there are instrumental methods of skin analysis such as dermatoscopy [3] and Raman spectroscopy [4,5]. However, the accuracy of these methods does not approach that of histological examination [6]. The aim of the research is to develop a method for identifying skin tumors based on multimodal j oint analysis of Raman scattering data and dermatoscopic images.
An in vivo study of skin tumors was carried out at the Samara Regional Clinical Oncology Center. Experimental skin Raman spectra were recorded using a portable setup that includes a laser source with a central wavelength of 785 nm. The spectra were recorded with a spectral resolution of 0.2 nm in the range from 837 to 920 nm, which corresponds to 792-1874 cm-1. Dermatoscopic images of skin neoplasms were obtained using a digital dermatoscope.
Machine learning methods, in particular, convolutional neural networks, were used to analyze the registered data. The classification model for malignant melanoma and pigmented nevus has shown an increase in classification accuracy compared to the analysis of Raman spectra or dermatoscopic images alone.
As a result, combined multimodal method for diagnosing skin cancer, which simultaneously takes into account both specific spectral features of tumors and spatial inhomogeneities in the distribution of optical density, has been proposed. The studied approaches to the analysis of optical biopsy data can be further used as part of the software for automated screening diagnostics of skin pathologies in order to detect tumors at an early stage of development.
[1] J. Ferlay, M. Colombet, I. Soerjomataram, D.M. Parkin, M. Pineros, A. Znaor, F. Bray, Cancer statistics for the year 2020: An overview, International journal of cancer, vol. 149(4), pp. 778-789, 2021.
[2] H.A. Haenssle, C. Fink, R. Schneiderbauer, F. Toberer, T. Buhl, A. Blum, A. Kalloo, A.B.H. Hassen, L. Thomas, A. Enk, L. Uhlmann, Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists, Annals of oncology, vol. 29(8), pp. 1836-1842, 2018.
[3] I.A. Matveeva, A.I. Komlev, O.I. Kaganov, A.A. Moryatov, V.P. Zakharov, Multidimensional Analysis of Dermoscopic Images and Spectral Information for the Diagnosis of Skin Tumors, Journal of Biomedical Photonics & Engineering, vol. 10(1), pp. 010307, 2024.
[4] I.A. Bratchenko, L.A. Bratchenko, A.A. Moryatov, Y.A. Khristoforova, D.N. Artemyev, O.O. Myakinin, A.E. Orlov, S.V. Kozlov, V.P. Zakharov, In vivo diagnosis of skin cancer with a portable Raman spectroscopic device, Experimental Dermatology, vol. 30(5), pp. 652663, 2021.
[5] I. Matveeva, I. Bratchenko, Y. Khristoforova, L. Bratchenko, A. Moryatov, S. Kozlov, O. Kaganov, V. Zakharov, Multivariate curve resolution alternating least squares analysis of in vivo skin Raman spectra, Sensors, vol. 22(24), p. 9588, 2022.
[6] G.V. Long, S.M. Swetter, A.M. Menzies, J.E. Gershenwald, R.A. Scolyer, Cutaneous melanoma, The Lancet, vol. 402(10400), pp. 485502, 2023.