CC BY
4Cellular Therapy and Transplantation (CTT). Vol. 10, No. 3-4, 2021 doi: 10.18620/ctt-1866-8836-2021-10-3-4-78-83 Submitted: 06 December 2021, accepted: 17 December 2021
Experience in the use of a radio-oriented augmented reality navigation system for biopsy of a neoplasm of the lower jaw
Anna V. Lysenko ', Andrey I. Yaremenko 2, Vladimir M. Ivanov 3, Sergey V. Strelkov 3, Elizaveta A. Ivanova 2
1 Department of Maxillofacial Surgery, Research Institute of Dentistry and Maxillofacial Surgery, St. Petersburg, Russia
2 Department of Maxillofacial Surgery, Pavlov University, St. Petersburg, Russia
3 Peter the Great St. Petersburg Polytechnic University, St. Petersburg, Russia
Dr. Anna V. Lysenko, Department of Maxillofacial Surgery, Phone: +7 (812) 429-03-33
Research Institute of Dentistry and Maxillofacial Surgery, E-mail: lysenko.anna@mail.ru
Pavlov University, 44 Petrogradskaya Emb., 197101, St. Petersburg, Russia
Citation: Lysenko AV, Yaremenko AI, Ivanov VM et al. Experience in the use of a radio-oriented augmented reality navigation system for biopsy of a neoplasm of the lower jaw. Cell Ther Transplant 2021; 10(3-4): 78-83.
Summary
Radiomics is a quantitative approach to medical imaging that applies advanced mathematical analysis in order to improve the existing data available to clinicians. Radiomics quantifies texture information by mathematical extraction of spatial distribution of the signal intensities and pixel relationships. Quantitative evaluation of the texture 2-D information employs analytic techniques from the field of artificial intelligence. The data derived from radiographic images, when compared with clinical data, may potentially provide additional information
aiming for support of decision-making in clinical medicine. In this study, a preliminary radiomic analysis of a lower jaw neoplasm was performed. Based on the data obtained, the optimal site for tissue biopsy was chosen. During diagnostic intervention, an augmented reality navigation system was used which took into account the results of the mentioned mathematical analysis.
Keywords
Radiomics, augmented reality, dynamic navigation systems, jaw neoplasms.
Introduction
Medical imaging is one of the main diagnostic methods that defines the treatment strategy. The main advantage of X-ray data examination is its ability for digitilization. The data obtained by the digitization provide multidimensional data which, due to appropriate standardization and analysis, enable additional information about specific features of the diseases [1].
Radiomics is a quantitative method for the analysis of digitized X-ray images, which is based on mathematical analysis [2]. Radiomics allows quantifying texture information by mathematically extracting the spatial distribution of signal intensities and pixel relationships [2, 3]. The concept of ra-diomics is based on the search for imaging biomarkers that are specific to certain pathological processes undetectable by
standard visual inspection of the generated images. Visually noticeable differences in the intensity, shape and texture of the image can be quantified using radiomics, which allows the more objective process of interpreting X-ray images [3]. The key purpose of radiomics applied for medical image analysis is to reveal objective non-invasive prognostic bio-markers of the disease, presuming its further transition to personalized medicine [4].
Four main steps for extraction of radiomic data from medical images are as follows:
1. Acquisition of images using methods of radiological diagnostics;
2. Segmentation of the region of interest (ROI or volume of interest - VOI);
3. Extraction of radiomic features;
4. Obtaining biomarkers of imaging [5].
However, in order to improve the accuracy of diagnostic procedures, such as biopsy of the neoplasm, the obtained data require integration into the clinical picture. This can be done by directly overlaying the region of the location of the imaging biomarker on the region of neoplasm through augmented reality.
Augmented reality is the projection of any digital information (images, video, text, graphics, etc.) over a real image [6]. As a result, the real world is supplemented with artificial elements and new information. This technology is becoming increasingly popular in various fields of medicine, including maxillofacial surgery [6]. The following is required with respect to ensurance of the technology functioning:
1. Acquisition of images using methods of conventional radiology diagnostics (obtaining data on a three-dimensional object);
2. Segmentation of the region of interest (e.g., neoplasms);
3. Loading the region of interest into augmented reality glasses;
4. Conducting surgery in augmented reality, focusing on the location of the region of interest.
As seen from the below algorithms, working in radiomics and augmented reality has several identical stages. Moreover, integration of the latter technique contributes to creation of a radio-oriented navigation system (Fig. 1).
The purpose hereof is to present an algorithm for radiomic image analysis using the example of a neoplasm of the lower
jaw and to demonstrate practical applicability of this technology, i.e., to conduct a radiomically targeted biopsy of a neoplasm directed by the augmented reality technology.
Materials and methods
The authors used a comprehensive open source platform PyRadiomics, built into 3D Slicer, the interface of the software was designed for analyzing and working with medical images, to handle the radiological data. The software is intended for processing and extraction of radiomic characteristics from medical images using a large panel of hard-coded function algorithms (Fig. 2).
Stage 1. Acquisition of images using methods of radiology diagnostics
The first stage is to identify the clinical problem and obtain a digital image, excluding low quality studies. When planning to perform an intervention in augmented reality, it is necessary to perform an X-ray examination using X-ray contrast markers, according to which the augmented reality marker will be localized and calibrated for the prospective surgery.
Taking into account the mobility of lower jaw, an individual occlusal splint with X-ray contrast marks and an integrated holder for the augmented reality marker was pre-fabricated. Cone beam computed tomography (CBCT) was performed with a mouthguard containing X-ray contrast marks fixed in the patient's mouth (Fig. 3).
Figure 1. Algorithm for radiome analysis of Cone Beam Computed Tomography images and the algorithm for the operation of augmented reality technology (Original picture)
Figure 2. Appearance of the software 3D Slicer
A
1
Figure 3. Appearance of the occlusal splint with X-ray contrast marks (A); Patient positioning at the cone-beam computed tomography with occlusal splint (B)
Stage 2. Segmentation of the region of interest
Upon receipt of the radiological data in DICOM format, one should define the region of interest (ROI) in two-dimensional projection (2D), or volume of interest (VOI) in three-dimensional (3D) projection begins. ROI/VOI determine the region in which the radiomic features are calculated. The image segmentation may be performed manually, semi-automatically or fully automatically (Fig. 4).
Stage 3. Extraction of radiomic features
After image segmentation and processing, the extraction of radiomic features can be performed, which is performed automatically after activation of the PyRadiomics module. The features extracted from images could be divided into morphological parameters (volume and shape), histogram features (description of gray tone intensities, texture analysis) (Fig. 5).
Stage 4. Obtaining imaging biomarkers
After receiving the radiomic data, selection or reduction of specific features is carried out. After that, the multivariate data analysis is started: a connection is established between the texture features of the gray level coincidence matrix and the morphological type of neoplasms. Isolated radiomic features that closely correlate with clinical findings may be assigned to the imaging biomarkers (Fig. 6).
Stage 5. Creation of a radiomally oriented augmented reality navigation system
To perform the surgery, two augmented reality markers were used. Marker 1 was attached to a holder embedded into the occlusal splint (as in cone-beam computer tomography?? CTCBCT). Marker 1 was associated with an image of a segmented neoplasm of the lower jaw with a highlighted visualization biomarker. Marker 2 was attached to the handle of the dental handpiece. Projection of the bone trephine was attached to marker 2 (Fig. 7 a, b).
Figure 4. Segmentation of the area of interest in 3D Slicer
Figure 5. Processing of radiomic feature extraction (stage 3)
Figure 6. Imaging biomarkers of neoplasm
A
Figure 7. Marker 1 fixed in the oral cavity (A); marker 2, at the dental handpiece (B)
B
Stage 6. Radiomic-guided biopsy of mandibular neoplasm in augmented reality
During the surgical intervention after fixation of the augmented reality markers, we checked calibration of the marker positions, projection of the lower jaw neoplasm, and the tip of the biopsy instrument. The surgery was performed under local anesthesia. Skeletonization of the outer plate of the lower jaw in the area 3.3-4.3 was performed. Bone tissue in the region under inspection had no visible abnormal changes (Fig. 8).
The neoplasm imaging was made in augmented reality. Using a bone trephine, a fragment of the formation was sampled in projection of the imaging biomarker (Fig. 9).
The material was sent for histological examination. The postoperative period was uneventful.
Histological report was as follows: an area of fibrous dysplasia of the lower jaw.
Discussion
In the presented clinical case, according to the results of the radiomic analysis of the segmented neoplasm of the lower jaw, its distinctive radiomic features were revealed - imaging biomarkers, that were transferred to the surgical field using augmented reality technology.
Figure 8. The body of the lower jaw in the projection of the neoplasm. Pathological changes are not visualized
The object of a lower jaw neoplasm obtained as a result of segmentation with a highlighted area of region biomarkers is combined with real anatomical structures in the oral cavity at the time of surgery by means of augmented reality technology. The combined model of this neoplasm demonstrates its anatomical and topographic position and structure, thus making it possible to determine the optimal operative access and most informative biopsy area for histological examination. Usage of this approach to tracking the instrument tip (bone trephine) in mixed reality enables control of its position, immersion depth, and reduced risk of traumas to adjacent anatomical structures. Real volume and topography of the object and adjacent anatomical structures fully corresponded to the data obtained in the course of virtual planning.
Conclusion
Analysis of medical images allows non-invasive assessment of the characteristics of human tissues. However, the interpretation of research results is often quite subjective. Recent advances in acquisition and analysis of medical imaging allow high-precision digital data extraction to quantify the difference between healthy and diseased tissue.
Radiomics applies mathematical analysis and advanced computational methodologies to medical imaging data to provide quantitative descriptors of pathological tissues. This is especially true for oncology.
A
B
Figure 9. Location of the neoplasm in virtual imaging (A), and actual location of the object (B)
Radiomics analysis of X-ray images has the potential to promote development of personalized medicine.
Clinical application of radiomic data is of the greatest interest for targeted biopsy of neoplasms in maxillofacial area. In order to improve accuracy of the diagnostic procedure, we recommend usage of augmented reality technology, which makes it possible to accurately visualize the most informative biopsy area for histological examination.
On the basis of advances in modern digital technologies such as radiomics and augmented reality, one can personalize and adapt established methods of treating diseases in the maxillofacial region for each patient, to improve the results of diagnosis and treatment, as well as reducing the number of complications.
References
1. Van Griethuysen JJM, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V. Computational radiomics system to decode the radiographic phenotype. Cancer Research 2017; 77(21) :104-107. doi: 10.1158/0008-5472.CAN-17-0339
2. Aerts HJWL, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nature Commun. 2014; 5(1): 1-9. doi: 10.1038/ncomms5006
3. Caudell JJ, Torres-Roca JF, Gillies RJ, Enderling H, Kim S, Rishi A, et al. The future of personalised radiotherapy for head and neck cancer. The Lancet Oncology 2017; 18( 5):266-273. doi: 10.1016/S1470-2045(17)30252-8
4. van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B. Radiomics in medical imaging - "How-to" guide and critical reflection. Insights Imag. 2020; 11(1): 1-16. doi: 10.1186/s13244-020-00887-2
5. Yaremenko AI, Lysenko AV, Ivanova EA, Galibin OV. Augmented reality technology for auricular reconstruction in the treatment of microtia. Cell Ther Transplant. 2020; 9(2): 78-82. doi: 10.18620/ctt-1866-8836-2020-9-2-78-82
6. Lysenko AV, Razumova AYa, Yaremenko AI, Mirzakhme-dov RR. Augmented reality in the treatment of sialolith-iasis. Stomatology. 2020; 99(4): 64-66. doi: 10.17116/sto-mat20209904164
Резюме
Радиомика - это количественный подход к медицинской визуализации, который направлен на улучшение существующих данных, доступных клиницистам, с помощью передового математического анализа. Посредством математического извлечения пространственного распределения интенсивностей сигналов и взаимосвязей пикселей, радиомика количественно определяет текстурную информацию, используя методы анализа из области искусственного интеллекта. Данные, извлеченные из рентгенологических изображений, при их сопоставлении с клиническими данными, потенциально могут предоставить дополнительную информацию для поддержки принятия решений в клинической медицине.
В данном исследовании выполнен предварительный радиомический анализ новообразования нижней челюсти. На основании полученных данных произведен выбор оптимального участка для биопсии. Во время проведения диагностического вмешательства использовалась навигационная система дополненной реальности, которая учитывала результаты данного математического анализа.
Ключевые слова
Радиомика, дополненная реальность, динамические навигационные системы, новообразования челюстей.
Опыт применения радиомически-ориентированной навигационной системы дополненной реальности при биопсии новообразования нижней челюсти
Анна В. Лысенко ', Андрей И. Яременко 2, Владимир М. Иванов 3, Сергей В. Стрелков 3, Елизавета А. Иванова 2
1 Отдел челюстно-лицевой хирургии НИИ стоматологии и челюстно-лицевой хирургии, Санкт-Петербург, Россия
2 Кафедра челюстно-лицевой хирургии, Первый Санкт-Петербургский государственный медицинский университет им. акад. И. П. Павлова, Санкт-Петербург, Россия
3 Санкт-Петербургский политехнический университет Петра Великого, Санкт-Петербург, Россия
