Inf-Seg: Automatic segmentation and quantification method for CT-based COVID-19 diagnosis Текст научной статьи по специальности «Компьютерные и информационные науки»

Information Technologies Информационные технологии

Research article

DOI: https://doi.org/10.18721/JCSTCS.15301 UDC 004.932.72

INF-SEG: AUTOMATIC SEGMENTATION AND QUANTIFICATION METHOD FOR CT-BASED COVID-19 DIAGNOSIS

F. Shariaty1 H , S.V. Zavjalov2, V.A. Pavlov3, T.M. Pervunina4, M. Orooji5

1,2,3 Peter the Great St. Petersburg Polytechnic University, St. Petersburg, Russian Federation;

4 Almazov National Medical Research Centre, St. Petersburg, Russian Federation;

5 University of California, Los Angeles, USA H shariaty3@gmail.com

Abstract. The global spread of the COVID-19 has increased the need for physicians and accurate and efficient diagnostic tools. The best way to control the spread of COVID-19 is through public vaccination as well as early intervention to prevent the spread of the disease. According to the World Health Organization, chest CT scans in the early stages of COVID-19 disease have good accuracy, which leads to the widespread use of these images in the diagnostics and evaluation of COVID-19 disease. Lung CT scan segmentation is an essential first step for lung image analysis. The purpose of this article is to evaluate the existing computer systems and to present a more efficient computer system for CT scan image segmentation. For this propose, a novel artificial intelligence (AI)-based COVID-19 Lung Infection Segmentation (Inf-Seg) method is proposed to automatically identify infected regions from chest CT scan. In Inf-Seg, after pre-processing of medical image and improving the image quality, texture feature extraction methods are used to collect high-level features and generate a global map. In the next step, we used YOLACT, which consists of a backbone part of a network of feature pyramids for creating multi-scale feature maps and efficient classification and localization of objects of various sizes (with better information than a regular feature pyramid for object detection), a Protonet part and prediction.

Keywords: automated segmentation, COVID-19, artificial intelligence, computed tomography scans, machine learning, deep learning

Funding: The reported study was funded by Russian Foundation for Basic Research and INSF, project number 20-57-56018.

Citation: Shariaty F., Zavjalov S.V., Pavlov V.A., Pervunina T.M., Orooji M. Inf-Seg: Automatic segmentation and quantification method for CT-based COVID-19 diagnosis. Computing, Telecommuni-cations and Control, 2022, Vol. 15, No. 3, Pp. 7-21. DOI: 10.18721/JCSTCS.15301

© Shariaty F., Zavjalov S.V., Pavlov V.A., Pervunina T.M., Orooji M., 2022. Published by Peter the Great St. Petersburg Polytechnic University

Научная статья

DOI: https://doi.org/10.18721/JCSTCS.15301 УДК 004.932.72

INF-SEG: АВТОМАТИЧЕСКИЙ МЕТОД СЕГМЕНТАЦИИ И КОЛИЧЕСТВЕННОГО ОПРЕДЕЛЕНИЯ ДЛЯ ДИАГНОСТИКИ COVID-19 НА ОСНОВЕ КТ

Ф. Шариати1 е , С.В. Завьялов2, В.А. Павлов3, Т.М. Первунина4, М. Оруджий5

1,2,3 Санкт-Петербургский политехнический университет Петра Великого,

Санкт-Петербург, Российская Федерация;

4 Национальный медицинский исследовательский центр им. В.А. Алмазова,

Санкт-Петербург, Российская Федерация;

5 Калифорнийский университет, Лос-Анджелес, США

н shariaty3@gmail.com

Аннотация. Глобальное распространение COVID-19 увеличило потребность во врачах и точных и эффективных диагностических инструментах. Лучший способ контролировать распространение COVID-19 — вакцинация населения, а также раннее вмешательство для предотвращения распространения болезни. По данным Всемирной организации здравоохранения, КТ грудной клетки на ранних стадиях заболевания COVID-19 имеет хорошую точность, что приводит к широкому использованию этих изображений в диагностике и оценке заболевания COVID-19. Сегментация КТ легких является важным первым шагом для анализа изображений легких. В статье рассмотрены существующие компьютерные системы и представлена эффективная компьютерная система для сегментации изображений КТ. Предложен новый метод сегментации легочной инфекции COVID-19 (Inf-Seg) на основе искусственного интеллекта (ИИ) для автоматического выявления инфицированных областей при компьютерной томографии грудной клетки. В Inf-Seg после предварительной обработки медицинского изображения и улучшения качества изображения используются методы извлечения признаков текстуры для сбора признаков высокого уровня и создания глобальной карты. На следующем этапе используется YOLACT, состоящий из базовой части сети пирамид функций для создания многомасштабных карт объектов и эффективной классификации и локализации объектов различных размеров (с лучшей информацией, чем обычная пирамида функций для обнаружения объектов), а также часть Protonet и предсказание.

Ключевые слова: автоматическая сегментация, COVID-19, искусственный интеллект, компьютерная томография, машинное обучение, глубокое обучение

Финансирование: Работа выполнена при финансовой поддержке РФФИ и INSF, проект № 20-57-56018.

Для цитирования: Shariaty F., Zavjalov S.V., Pavlov V.A., Pervunina T.M., Orooji M. Inf-Seg: automatic segmentation and quantification method for CT-based COVID-19 diagnosis // Computing, Telecommunications and Control. 2022. Т. 15, № 3. С. 7-21. DOI: 10.18721/JCSTCS.15301

Introduction

Coronavirus, or COVID-19, is an epidemic disease caused by SARS-CoV2 that has spread worldwide in a short period, according to the global records from the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU) [1] as of October 24, 2022, it has resulted in 627 million cases and 6 million confirmed deaths. As a result, the World Health Organization (WHO) declared a COVID-19 pandemic. To control the spread of the virus, the most important tool after general vaccination (which is not currently available to most people) is the screening of a large number of suspected cases for quarantine and appropriate treatment. The main tool for the current diagnosis of COVID-19 is

© Шариати Ф., Завьялов С.В., Павлов В.А., Первунина Т.М., Оруджи М., 2022. Издатель: Санкт-Петербургский политехнический университет Петра Великого

Информационные технологии -►

RT-PCR, while, in addition to the time-consuming nature of this test, the lack of equipment, and strict conditions for testing environment, the sensitivity of this test is not high enough and cannot effectively prevent epidemics. Thus, false negatives of RT-PCR are a potential threat to the general health of the community, which not only causes other people to become infected with the virus, but the virus also progresses in the patient's lungs and reduces the chances of survival.

Imaging equipment such as chest X-rays and chest CT scans are readily available and can help physicians for early detection of COVID-19 [2]. However, CT scan screening is preferred and more accurate than X-ray imaging due to such merits as three-dimensional view and better contrast. Recent studies have reported [3, 4] symptoms of lung infection in CT scan images as ground-glass opacity (GGO). Qualitative evaluation of the infection and its volumetric changes in CT scan contain useful information, such as quantitative measurement of disease progression and evaluation of the effect of drugs used to treat a patient with COVID-19. However, manual isolation of these infections is a tedious and time-consuming task, while the annotation of the infection by the radiologist is also a mental task and can be affected by individual bias and clinical experiences.

Despite the importance of automatic segmentation of lesions and infections caused by COVID-19 in lung CT scans, this is still a challenge. Lesions and infections caused by COVID-19 have a complex appearance (GGO) and on the other hand, the size and location of these lesions are very different in different stages of the disease and in different patients. In addition, these lesions have irregular shapes and blurred borders, and in some cases, no clear borders at all, and have a very similar contrast to the surrounding area of the lungs. Given these challenges, there is an urgent need for an automated computer system of high accuracy and speed for the segmentation of lung CT scan.

Segmentation of CT scans allows to extract areas of interest, such as lung lobes, infected areas, or lesions, for further analysis and diagnosis [5]. Deep learning techniques are widely used to segment regions of interest in CT [6]. In terms of target ROIs in the segmentation approaches of COVID-19 applications can be divided into two categories: lung-region-oriented methods and lung-lesion-oriented methods [7]. The lung-region-oriented approaches attempt to differentiate lung regions, such as the total lung and lung lobes, from other (background) regions in CT or X-ray, which is a required step in COVID-19 applications [8-10]. For example, Jin et al. [11] suggest a two-stage pipeline for screening COVID-19 in CT images, in which an efficient segmentation network based on UNet++ detects the entire lung region first. The lung-lesion-oriented approaches [12, 13] strive to differentiate lesions (or metal and motion artifacts) in the lung regions. As lesions or nodules might be small and have a range of shapes and textures, detecting the regions of the lesions or nodules is necessary and has traditionally been seen as a difficult detection task. Apart from segmentation, the attention mechanism has been reported as an effective localization strategy in screening, which can be used in COVID-19 applications.

In COVID-19 applications, the U-Net is a widely utilized technique for segmenting both lung regions and lung lesions [9, 8, 14]. Ronneberger [15] developed a U-shape design with symmetric encoding and decoding signal routes for the U-Net, a sort of a fully convolutional network. The layers of the coding network and the corresponding layers of the decoding network are connected directly to each other. As a result, the network can learn superior visual semantics and detailed context in this scenario, which is useful for medical image segmentation. In addition, many other U-Nets and their variants were developed, with reasonable segmentation results. Milletari et al. [16] propose the V-Net, which uses residual blocks as the basic convolutional block and a Dice loss to improve the network. Shan et al. [17] use a VB-Net for more effective segmentation by supplying the convolutional blocks with the so-called bottleneck blocks. The UNet++, proposed by Zhou et al. [18] is a significantly more complicated network than U-Net, as it inserts a nested convolutional structure between the encodables.

This work presents an efficient computer system for CT scan image segmentation. For this propose, a novel artificial intelligence (AI) based COVID-19 Lung Infection Segmentation (Inf-Seg) method is proposed to automatically identify infected regions by means of chest CT scan. In our Inf-Seg, after

pre-processing medical image and improving the quality, texture feature extraction methods are used to collect high-level features and generate a global map [19]. In the next step, YOLACT (You Only Look At CoefficienTs) [20] structure was used which consists of a backbone part of a network of feature pyramids for creating multi-scale feature maps and efficient classification and localization of objects of various sizes (with better information than a regular feature pyramid for object detection), a Protonet part and prediction.

Patients and Methods

Patients

Patients with flu-like symptoms and an initial diagnosis of COVID-19 were chosen for the study, independent of their age or gender. High-resolution CT (HRCT) scans of the patients were taken. Inclusion criteria for each patient included a confirmation of COVID-19 by RT-PCR test. From early 2020 to April 2020, imaging was performed on COVID-19 patients between 3 and 6 days after the onset of the disease. Exclusion criteria for normal group (patients without COVID-19):

♦ Negative RT-PCR test;

♦ No fever;

♦ Ground glass opacities;

♦ Interlobular septal thickening;

♦ Bilateral bronchovascular bundle thickening.

Database acquisition

254 cases were acquired from Shariati Hospital (associated with Tehran University of Medical Sciences) and Taleghani Hospital of Tehran(affiliated with the Shahid Beheshti University of the Medical Sciences) in order to reach a reliable and extensible result. Every case study that was considered had a histopathologic confirmation to ensure that the patient had COVID-19. The CT scans' slice thickness ranged from 1 to 6 mm, and the number of slices per scan ranged from 226 to 389. The CT scan images were taken using a Siemens scanner with a kilovoltage peak distribution of 120—140 kVp and currents ranging from 25 to 40 mA, depending on the patient's health. Each slice featured a 512 x 512 pixel XY planar resolution and a 16-bit grayscale resolution in Hounsfield Units (HU).

Pre-Processing

After creating gray-scale images, an experienced radiologist reviewed the CT scan images. Following that, slices from the CT scan image that showed illness symptoms were chosen (Fig. 1).

On lung CT scans, another pre-processing procedure was used to improve the quality of the images for better diagnostic outcomes [21]. This stage is critical because the lungs include various features that can make a precise diagnosis difficult. Linear interpolation [22], middle filter [23], morphological operation [24], Gaussian filter [25], and weight addition filter [26] are some of the pre-processing methods available. In this work, a Wiener filter was utilized to improve the quality of CT scan images. Filtering entails the creation of a neighborhood (typically a small rectangle) and the application of a predefined operation to the pixels of the image inside that neighborhood.

The filtering operation creates a new pixel with coordinates equal to the neighborhood center's coordinates and a value equal to the filtering operation's result. As the center of the filter is positioned on each of the pixels of the input image, the processed (filtered) image is executed concurrently [27—29].

Wiener filtering is used to reduce the noise that has contaminated an image, resulting in an output that is identical to the first image. The goal is to have the smallest mean square blunder possible. Wiener filtering investigates prior noise information in an image. There is a compelling case for utilizing the Wiener filter for image improvement, and when compared to the median filter and adaptive min-max, the Wiener filter achieves a higher PSNR [30].

To reduce signal noise, a Wiener filter (a form of linear filter) is utilized to replace the FIR filter [31]. Inverse filtering can be used to recover an image that has been blurred by a known low pass filter. Inverse

Fig. 1. Slices showing disease symptoms selected by radiologist: a — acceptable slices; b — non-acceptable slices

filtering, on the other hand, is extremely sensitive to additive noise. Wiener filtering achieves the best balance of inverse filtering and noise smoothing. It simultaneously removes additive noise and inverts blurring [32, 33]. In the inverse filtering and noise smoothing process, it reduces the overall mean square error. A linear estimation of the original image based on a stochastic framework is used in Wiener filtering. Because of the orthogonality principle, the Wiener filter in the Fourier domain can be written as follows:

<•) = H'(f\, f2) Sxx (f\, f2)

2 \H (fl, f2 )|2 S„ (fl, f2 ) + Snn(fi, f2 )'

where Sjfvf2) and S (fj,f2) are the original image's and additive noise's power spectra, respectively, and Hfj, f2) is the blurring filter [31].

Study of Deep Learning

YOLACT (You Only Look At CoefficienTs) solves the problem of segmentation of instances by dividing the task into two smaller subtasks that are executed in parallel: prototype masks and mask coefficients for each individual instance in the image, thereby remaining a one-stage detector. Thus, the method implicitly learns to localize the instance masks and therefore can skip the localization step that is very common in instance segmentation methods. The network architecture is shown in Fig. 2.

The results of the work of parallel branches on the generation of prototype masks and mask coefficients are followed by an assembly step — a linear combination. Next, the operation of trimming according to the predicted coordinates of the bounding rectangles of the instances is carried out for the threshold truncation of detection with predicted probabilities below a given threshold.

YOLACT uses FPN (Feature Pyramid Networks), which consists of upstream (C1—C5) and downstream (P3—P7) paths (Fig. 2). The input of this FPN is a 3D tensor which consists of original image, Gray (entropy) texture feature and Absolute Gradient (Kurtosis) texture feature.

Haralik et al. [34] introduced the gray texture feature, which is a second-order structural feature based on Gray-Level Co-occurrence Matrices (GLCMs) according to the grayscale image's target are-

W ( f

as. Contrast, correlation, inverse difference moment, angular second moment, and entropy are five gray texture features recovered in this study. The following mathematical formula is used to achieve entropy characteristic:

f-i^ih-";)- ©

From each determined gradient matrix CT scan, a set of five texture features can be generated: Mean, Variance, Skewness, Kurtosis, and Moment. After generating the histogram (His) of a gradient matrix, gradient features are generated. This histogram is calculated for gradient values that fall within the range [-255, 255]. The following is the gradient (Kurtosis) feature determined [35, 36]:

f

H;s (v + 255)( v Total number of p;xels

(3)

The output values of the functions are represented by f and the indices of the GLCM retrieved from the image are represented by i and j in these equations. Furthermore, v is the gradient value that is extended between -255 and 255, P stands for probability, and ^ for mean.

As stated, the used FPN consists of upstream (C1-C5) and downstream (P3-P7) paths (Fig. 3), in which upward path is a conventional ResNet-101 feature extraction convolutional network, where the spatial resolution decreases as one moves upward. The output from the last layer of each stage is used as a reference set of feature maps to enrich the downstream path through lateral join. Each side join combines feature maps of the same spatial size with upward and downward paths. FPN provides a top-down path for building higher resolution layers from a semantically rich layer.

The Protonet block (Fig. 4) generates the final output feature map "138 x 138 x k" by folding and deconvolution of the P3 feature map. The oversampling operation is performed once on the P3 layer. The size of the final output feature map is 1/4 of the original image. The number of output channels is k, where k is 32 by default. The value of k is very robust. Even choosing other values will not have much of an impact on the results.

Fig. 2. YOLACT Architecture

4

Информационные технологии

Fig. 3. Feature Pyramid Networks

This is where the main feature map can create a more robust mask. High-level feature maps can produce higher quality features, but are better suited for small targets. The final pin is processed using a rectified linear unit (ReLU) activation function to ensure that the final pin segmentation results are positive.

In the Prediction Head block (Fig. 5, where c is the number of classification categories, a is the number of anchors, and k is the number of divided output channels equal to the number of channels output by the Protonet block), the mask correlation coefficient branch is combined with the RetinaNet multitasking branch. Finally, based on the two branches of traditional classification and regression, an additional split branch is added. This is followed by a block of non-maximum suppression (NMS), which selects one of the many overlapping objects. Blocks with objects are sorted according to their confidence and all those with a lower degree of confidence that have an IoU overlap (Intersection over Union) exceeding a certain threshold are removed. The IoU is the ratio of the overlapping area of ground truth and predicted area to the total area:

TP

IoU =-. (4)

TP + FP + FN

A nonlinear combination, represented as a sigmoid, is needed to combine the Protonet and Prediction Head branches.

M = o( PCT ), (5)

whereMis selected object; P is the h x w x k prototype mask matrix; C — coefficients of the mask nxk; n is the number of predicted objects after passing non-maximum suppression. This is followed by a trim operation with predicted bounding boxes for each instance. The clipping operation reduces the load on the network to suppress noise outside the bounding box. After all the steps described, the network outputs the final segmentation result. Statistical analysis

The statistical values of the segmented COVID-19 lesion are compared with the results of other proposed methods to evaluate and determine the performance of the proposed segmentation approach. The probability of a lesion absence in the image might be given, resulting in patients being considered normal and healthy and so no segmentation is necessary. In addition, we used statistics to evaluate the suggested Inf-Seg quality by selecting: Accuracy [37], Sensitivity [38], F-Measure [38], Precision [39],

Fig. 4. Feature Pyramid Networks

Fig. 5. Prediction Head block

Dice [40], and Specificity [38]. Higher values on these indexes, by definition, indicate better segmentation quality.

Accuracy =

TP + TN P + N

(6)

TP

Sensitivity =

(7)

TN

Specificity =

(8)

Информационные технологии -►

TP

Precision =--(9)

TP + FP

„ 2 x Precision x Recall

F _ score =--(10)

Precision + Recall

Experimental Results and Discussion

Demographic characteristics

A total of 254 COVID-19 patients were studied, with 113 women and 141 men ranging in age from 50.22 ± 10.85 years (mean age ± standard deviation). In the COVID-19 patients, gender distribution did not differ significantly (P > 0.05).

Table 1

Demographic characteristics of data

Characteristics COVID-19 P-value*

Age (Mean ± SD) 50.22 ± 10.85 < 0.001

Number of Females (Percentage) 113 (44.44 %) 0.600

Number of Males (Percentage) 141 (55.56 %) 0.600

A professional radiologist used CT imaging to retrieve 5759 imaging patches including COVID-19. For training, testing, and validation, the COVID-19 dataset was separated randomly into three sections (without duplications): 3455 (training), 1152 (testing), and 1152 (validation).

The performance of the deep learning method

Fig. 6 shows the visual performance of the algorithm presented in the paper. In this figure, the greyer pixels, in the results of segmentation, represent the areas that are more possible to be infected with COVID-19, and the less-gray areas indicate the least likely to be infected. Patients with COVID-19 can be acute or non-acute, as shown in Fig. 6, where first row shows that the infection fills most of the lung volume and according to results, it can be said that the Inf-Seg algorithm works very well in both cases.

To verify the proposed algorithm, a radiologist segmented the areas infected with COVID-19 manually. Finally, the results of the lung and infection segmentation algorithm are compared with the results of manual segmentation by a physician. As can be seen in Table 2, Inf-Seg has high performance with Accuracy of 0.99.

In this work, the sensitivity and positive predictive value (PPV) criteria are determined in addition to the Accuracy. According to TP and FN, the suggested algorithm has a sensitivity of 0.96, indicating that it is a powerful tool for diagnosing unhealthy regions in COVID-19 patients.

Table 2

Results of the proposed Inf-Seg and lung segmentation methods

Accuracy Sensitivity F-measure Precision Dice Specificity

Inf-Seg 0.99 0.96 0.97 0.98 0.97 0.99

Lung-seg 0.99 0.97 0.98 0.99 0.98 0.99

4

)

■ m- m *

v

Original Lung Segmentation Inf-Seg Fig. 6. Visual segmentation results

Comparing the performance of deep methods

YOLACT is a more optimal model for segmentation, because it has a good reputation for trade-offs in speed and accuracy. It is capable of achieving Medium Accuracy much faster than other competing platforms. YOLACT has a number of advantages over existing architectures, one of the most significant being forecast speed. It is also the only architecture in existence that can provide inference in real time.

The results clearly show that the Better Instance Segmentation (YOLACT) and Better Semantic Segmentation (U-Net) models provide very similar metrics in terms of average pixel precision and average frequency weighted IoU. The chosen architecture has many practical advantages:

1. Ease of assembly due to parallel design.

2. A negligible amount of computational cost for single-stage detectors such as ResNet101.

3. The quality of the masks is high.

4. The general concept of adding prototype generation and mask coefficients, which can be added to almost any modern object detector.

5. Work in real time.

We compared our method to other current COVID-19 segmentation algorithms based on CT images for further assessment. Table 3 presents the overview of related work for COVID-19 segmentation and comparison of segmentation performance results. We implemented our Inf-Seg method without using pre-processing step to compare it with the Inf-Seg method. Considering the results presented in Table 3, the performance of the algorithm decreases without the pre-processing step.

State of current limitation

However, it is vital to remember that the majority of current segmentation algorithms in research are not suitable for clinical use. The most of existing models are biased in that they are only trained using

Информационные технологии -►

Table 3

Overview of related work for COVID-19 infection segmentation and comparison of segmentation performance results using Dice criteria

Author Model Architecture Sample Size Dice of COVID-19

Vivek et al. [41] LungINFseg 1800 0.803

Saood et al. [42] SegNet 80 CT 0.749

Wang et al. [43] U-Net (Standard) 211 0.704

He et al. [44] M2UNet (Segmentation only) 666 0.759

Ma et al. [45] U-Net (Standard) 20 CT 0.608

Pei et al. [46] MPS-Net (Supervision U-Net) 300 0.833

Fan et al. [47] Inf-Net (Attention U-Net) 1650 0.764

Ma et al. [48] nnU-Net 20 CT 0.673

Wang et al. [49] Attention Mechanism 20 CT 0.847

Yan et al. [50] COVID-SegNet 731 0.726

Zheng et al. [51] MSD-Net 3824 0.785

Qiu et al. [52] MiniSeg (Attention U-Net) 3558 0.773

Saood et al. [42] U-Net (Standard) 80 CT 0.733

Müller et al. [53] 3D U-Net 20 CT 0.804

Implemented U-net U-net 254 CT 0.834

Inf-Seg without pre-processing YOLACT 254 CT 0.915

Proposed Inf-Seg YOLACT 254 CT 0.97

COVID-19-related photos. As a result, it is unclear how well the models distinguish between COVID-19 lesions and other pneumonias, let alone completely unrelated disease disorders like cancer.

Although neural networks can provide reliable decision assistance, their resilience is heavily dependent on the amount of the training dataset. Various medical situations, such as uncommon or new diseases, have insufficient data for model training, reducing generalizability and increasing the danger of overfitting.

Furthermore, the RT-PCR test used to distinguish COVID-19 in this study may be inaccurate. RT-PCR testing, on the other hand, is a molecular test with a number of drawbacks, including the fact that it is time-consuming, expensive, and requires a specialized kit and well-equipped laboratories. For low-income countries, these limitations are even more important, with dire and dangerous consequences.

Potential extensions for future research

As for the further research, we are planning to expand our dataset and evaluation by adding cases with non-COVID-19 conditions like pulmonary Edema or lung cancer. In this study, the ML model will detect and segment the infected region or the nodule and determine the type of disease by three categories: Cancer, COVID-19, and Edema.

Conclusions

Because of the relatively high infection rate, the ongoing COVID-19 pandemic has been declared a global health emergency. As of today, no clinically approved therapeutic medicine for COVID-19 is available. COVID-19 must be diagnosed early since it can be lethal. An automatic approach for segmenting COVID-19 infected areas (Inf-Seg) in CT images is proposed in this research. After applying a Wiener filter to the CT images, Inf-Seg used the YOLACT DL network to segment the COVID-19

infected area. We demonstrated that our medical image segmentation technique can successfully train accurate and robust models on minimal data without overfitting. Moreover, for COVID-19-infected areas, we were able to surpass existing state-of-the-art semantic segmentation techniques. According to the results, Inf-Seg performs well in segmenting infected areas, with an accuracy of 0.99. Our model has a high potential for use as a clinical decision support system for COVID-19 quantitative evaluation and disease tracking in a clinical setting.

REFERENCES

1. Shariaty F., Mousavi M., Moradi A., Oshnari M.N., Navvabi S., Orooji M., Novikov B. Semi-automatic segmentation of COVID-19 infection in lung CT scans. International Youth Conference on Electronics, Telecommunications and Information Technologies, Springer, Cham, 2022, Pp. 67—76. DOI: 10.1007/978-3-030-81119-8_7

2. Velichko E.N., Shariaty F., Orooji M., Pavlov V.A., Pervunina T.M., Zavyalov S.V., Khazaei R., Rad-

mard A.R. Development of computer-aided model to differentiate COVID-19 from pulmonary edema in lung CT scan: EDECOVID-net. Computers in Biology and Medicine, 2022, Vol. 141, P. 105172. DOI: 10.1016/j. compbiomed.2021.105172

3. Shariaty F., Shariati S., Navvabi S., Najafi M., Novikov B. Application of artificial intelligence for rapid prevention of epidemic diseases (COVID-19), 2022. DOI: 10.21203/rs.3.rs-1309509/v3

4. Faridoddin Sh., Pavlov V.A., Zavyalov S.V., Makhdi O., Pervunina T.M. Primeneniye modeli vneshnego vida tekstury dlya segmentatsii legochnykh uzlov pri kompyuternoy tomografii grudnoy kletki. Izvestiya Vys-shikh Uchebnykh Zavedeniy Rossii. Radioelektronika, 2022, Vol. 25 (3), Pp. 96—117. (rus)

5. Shariaty F., Orooji M., Velichko E.N., Zavyalov S.V. Texture appearance model, a new model-based segmentation paradigm, application on the segmentation of lung nodule in the CT scan of the chest. Computers in Biology and Medicine, 2021, P. 105086. DOI: 10.1016/j.compbiomed.2021.105086

6. Shariaty F., Pavlov V.A., Velichko E.N., Pervunina T.M., Orooji M. Severity and progression quantification of COVID-19 in CT images: A new deep-learning approach. 2021 International Conference on Electrical Engineering and Photonics (EExPolytech). IEEE, 2021, Pp. 72—76. DOI: 10.1109/EExPoly-tech53083.2021.9614774

7. Shi F., Wang J., Shi J., Wu Z., Wang Q., Tang Z., He K., Shi Y., Shen D. Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for COVID-19. IEEE Reviews in Biomedical Engineering, 2020.

8. Shariaty F., Pavlov V.A., Zavyalov S.V., Orooji M., Pervunina T.M. Application of a texture appearance model for segmentation of lung nodules on computed tomography of the chest. Journal of the Russian Universities. 1998;25(3):97.

9. Pavlov V.A., Shariaty F., Orooji M., Velichko E.N. Application of deep learning techniques for detection of COVID-19 using lung CT Scans: Model development and validation. International Youth Conference on Electronics, Telecommunications and Information Technologies, Springer, Cham, 2022, Pp. 85—96. DOI: 10.1007/978-3-030-81119-8_9

10. Huang L., Han R., Ai T., Yu P., Kang H., Tao Q., Xia L. Serial quantitative chest ct assessment of COVID-19: A deep learning approach. Radiology: Cardiothoracic Imaging, 2020, Vol. 2, no. 2, P. e200075.

11. Jin S., Wang B., Xu H., Luo C., Wei L., Zhao W., Hou X., Ma W., Xu Z., Zheng Z. et al. AI-assisted CT imaging analysis for COVID-19 screening: Building and deploying a medical ai system in four weeks. MedRxiv, 2020.

12. Tang L., Zhang X., Wang Y., Zeng X. Severe COVID-19 pneumonia: Assessing inflammation burden with volume-rendered chest CT. Radiology: Cardiothoracic Imaging, 2020, Vol. 2, no. 2, P. e200044. DOI: 10.1148/ryct.2020200044

Информационные технологии -►

13. Shen C., Yu N., Cai S., Zhou J., Sheng J., Liu K., Zhou H., Guo Y., Niu G. Quantitative computed tomography analysis for stratifying the severity of coronavirus disease 2019. Journal of Pharmaceutical Analysis, 2020, Vol. 10, no. 2, Pp. 123-129.

14. Shariati F. Vyyavleniye priznakov zabolevaniy legkikh kompyuternymi metodami: vypusknaya kvali-fikatsionnaya rabota magistra: napravleniye 11.04. 02 «Infokommunikatsionnyye tekhnologii i sistemy svy-azi»; obrazovatelnaya programma 11.04. 02_07 «Lazernyye i optovolokonnyye sistemy (mezhdunarodnaya obrazovatelnaya programma)».

15. Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, Pp. 234-241.

16. Milletari F., Navab N., Ahmadi S.-A. V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 4th International Conference on 3D Vision (3DV). IEEE, 2016, Pp. 565-571. DOI: 10.1109/3DV.2016.79

17. Shan F., Gao Y., Wang J., Shi W., Shi N., Han M., Xue Z., Shen D., Shi Y. Lung infection quantification of COVID-19 in CTimages with deep learning. arXiv preprint arXiv:2003.04655, 2020.

18. Zhou Z., Siddiquee M.M.R., Tajbakhsh N., Liang J. Unet++: A nested U-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, 2018, Pp. 3-11.

19. Shariaty F., Baranov M., Velichko E.N., Galeeva M., Pavlov V.A. Radiomics: extracting more features using endoscopic imaging. 2019 IEEE International Conference on Electrical Engineering and Photonics (EEx-Polytech). IEEE, 2019, Pp. 181-194.

20. Bolya D., Zhou C., Xiao F., Lee Y.J. Yblact: Real-time instance segmentation. Proceedings of the IEEE/CVFInternational Conference on Computer Vision, 2019, Pp. 9157-9166.

21. Shariaty F., Mousavi M. Application of cad systems for the automatic detection of lung nodules. Informatics in Medicine Unlocked, 2019, Vol. 15, P. 100173.

22. Cascio D., Magro R., Fauci F., Iacomi M., Raso G. Automatic detection of lung nodules in CT datasets based on stable 3D mass-spring models. Computers in Biology and Medicine, 2012, Vol. 42, no. 11, Pp. 1098-1109.

23. Chen B., Kitasaka T., Honma H., Takabatake H., Mori M., Natori H., Mori K. Automatic segmentation of pulmonary blood vessels and nodules based on local intensity structure analysis and surface propagation in 3D chest CT images. International Journal of Computer Assisted Radiology and Surgery, 2012, Vol. 7, no. 3, Pp. 465-482.

24. Soltaninejad S., Keshani M., Tajeripour F. Lung nodule detection by KNN classifier and active contour modelling and 3D visualization. The 16th CSIInternational Symposium on Artificial Intelligence and Signal Processing (AISP 2012). IEEE, 2012, Pp. 440-445.

25. Namin S.T., Moghaddam H.A., Jafari R., Esmaeil-Zadeh M., Gity M. Automated detection and classification of pulmonary nodules in 3D thoracic CT images. 2010 IEEE International Conference on Systems, Man and Cybernetics. IEEE, 2010, Pp. 3774-3779.

26. Mekada Y., Kusanagi T., Hayase Y., Mori K., Hasegawa J.-i., Toriwaki J.-i., Mori M., Natori H. Detection of small nodules from 3D chest X-ray CT images based on shape features. International Congress Series, Elsevier, 2003, Vol. 1256, Pp. 971-976.

27. Shariaty F., Orooji M., Mousavi M., Baranov M., Velichko E.N. Automatic lung segmentation in computed tomography images using active shape model. 2020 IEEE International Conference on Electrical Engineering and Photonics (EExPolytech), IEEE, 2020, Pp. 156-159.

28. Shariaty F., Hosseinlou S., Rud V.Y. Automatic lung segmentation method in computed tomography scans. Journal of Physics: Conference Series, IOP Publishing, 2019, Vol. 1236, no. 1, P. 012028.

29. Mousavi M., Shariaty F., Orooji M., Velichko E.N. The performance of active-contour and region growing methods against noises in the segmentation of computed-tomography scans. Internation-

al Youth Conference on Electronics, Telecommunications and Information Technologies. Springer, 2021, Pp. 573-582.

30. Halalli B., Makandar A. Computer aided diagnosis-medical image analysis techniques. Breast Imaging, 2018, Vol. 85.

31. Naimi H., Adamou-Mitiche A.B.H., Mitiche L. Medical image denoising using dual tree complex thresholding wavelet transform and wiener filter. Journal of King Saud University-Computer and Information Sciences, 2015, Vol. 27, no. 1, Pp. 40-45.

32. Kaur J., Kaur R. Image denoising for speckle noise reduction in ultrasound images using dwt technique. International Journal of Application or Innovation in Engineering & Management (IJAIEM), 2013, Vol. 2, no. 6.

33. Ajwad A.A. et al. Noise reduction of ultrasound image using wiener filtering and haar wavelet transform techniques. Diyala Journal of Medicine, 2012, Vol. 2, no. 1, Pp. 91-100.

34. Haralick R.M., Shanmugam K., Dinstein I.H. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, 1973, Nov(6):610-21.

35. Lerski R.A., Straughan K., Schad L.R., Boyce D., Blüml S., Zuna I. VIII. MR image texture analysis — An approach to tissue characterization. Magnetic Resonance Imaging, 1993 Jan 1;11(6):873—87. DOI: 10.1016/0730-725X(93)90205-R

36. Al-Kilidar S.H., George L.E. Texture classification using gradient features with artificial neural network. Journal of Southwest Jiaotong University, 2020;55(1).

37. Fernandez-Moral E., Martins R., Wolf D., Rives P. A new metric for evaluating semantic segmentation: Leveraging global and contour accuracy. 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2018, Pp. 1051-1056.

38. Boughorbel S., Jarray F., El-Anbari M. Optimal classifier for imbalanced data using Matthews correlation coefficient metric. PloS one, 2017, Vol. 12, no. 6, P. e0177678. DOI: 10.1371/journal.pone.0177678

39. Gupta S., Girshick R., Arbel'aez P., Malik J. Learning rich features from RGB-D images for object detection and segmentation. European Conference on Computer Vision, Springer, 2014, Pp. 345—360.

40. Trongtirakul T., Oulefki A., Agaian S., Chiracharit W. Enhancement and segmentation of breast thermograms. Mobile Multimedia/Image Processing, Security, and Applications 2020, Vol. 11399. International Society for Optics and Photonics, 2020, P. 113990F.

41. Müller D., Soto-Rey I., Kramer F. Robust chest CT image segmentation of COVID-19 lung infection based on limited data. Informatics in Medicine Unlocked, 2021, Vol. 25, P. 100681. DOI: 10.1016/j. imu.2021.100681

42. Saood A., Hatem I. COVID-19 lung ct image segmentation using deep learning methods: U-net versus segnet. BMC Medical Imaging, 2021, Vol. 21, no. 1, Pp. 1-10.

43. Wang Y., Zhang Y., Liu Y., Tian J., Zhong C., Shi Z., Zhang Y., He Z. Does non-COVID-19 lung lesion help? Investigating transferability in COVID-19 CT image segmentation. Computer Methods and Programs in Biomedicine, 2021, Vol. 202, P. 106004. DOI: 10.1016/j.cmpb.2021.106004

44. He K., Zhao W., Xie X., Ji W., Liu M., Tang Z., Shi Y., Shi F., Gao Y., Liu J. et al. Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of COVID-19 in CT images. Pattern Recognition, 2021, Vol. 113, P. 107828. DOI: 10.1016/j.patcog.2021.107828

45. Ma J., Wang Y., An X., Ge C., Yu Z., Chen J., Zhu Q., Dong G., He J., He Z. et al. Towards efficient COVID-19 CT annotation: A benchmark for lung and infection segmentation. arXiv e-prints, pp. arXiv-2004, 2020.

46. Pei H.-Y., Yang D., Liu G.-R., Lu T. MPS-net: Multi-point supervised network for CT image segmentation of COVID-19. IEEE Access, 2021,Vol. 9, Pp. 47144-47153.

47. Fan D.-P., Zhou T., Ji G.-P., Zhou Y., Chen G., Fu H., Shen J., Shao L. INF-net: Automatic COV-ID-19 lung infection segmentation from CT images. IEEE Transactions on Medical Imaging, 2020, Vol. 39, no. 8, Pp. 2626-2637. DOI: 10.1109/TMI.2020.2996645

Информационные технологии

48. Isensee F., Ja'ger P.F., Kohl S.A., Petersen J., Maier-Hein K.H. Automated design of deep learning methods for biomedical image segmentation. arXiv preprint arXiv:1904.08128, 2019.

49. Wang G., Liu X., Li C., Xu Z., Ruan J., Zhu H., Meng T., Li K., Huang N., Zhang S. A noise-robust framework for automatic segmentation of COVID-19 pneumonia lesions from CT images. IEEE Transactions on Medical Imaging, 2020, Vol. 39, no. 8, Pp. 2653—2663.

50. Yan Q., Wang B., Gong D., Luo C., Zhao W., Shen J., Shi Q., Jin S., Zhang L., You Z. COVID-19 chest CT image segmentation — A deep convolutional neural network solution. arXiv preprint arXiv:2004.10987, 2020.

51. Zheng B., Liu Y., Zhu Y., Yu F., Jiang T., Yang D., Xu T. MSD-net: Multi-scale discriminative network for COVID-19 lung infection segmentation on CT. IEEE Access, 2020, Vol. 8, Pp. 185786-185795. DOI: 10.1109/ACCESS.2020.3027738

52. Qiu Y., Liu Y., Li S., Xu J. Miniseg: An extremely minimum networkfor efficient COVID-19 segmentation. arXiv preprint arXiv:2004.09750, 2020.

53. Kumar Singh V., Abdel-Nasser M., Pandey N., Puig and D. Lung INF-SEG: Segmenting COVID-19 infected regions in lung CT images based on a receptive-field-aware deep learning framework. Diagnostics, 2021, Vol. 11, no. 2, P. 158. DOI: 10.3390/diagnostics11020158

INFORMATION ABOUT AUTHORS / СВЕДЕНИЯ ОБ АВТОРАХ

Шариати Фаридоддин Faridoddin Shariaty

E-mail: shariaty3@gmail.com

Завьялов Сергей Викторович Sergey V. Zavjalov

E-mail: zavyalov_sv@spbstu.ru

Павлов Виталий Александрович Vitalii A. Pavlov

E-mail: pavlov_va@spbstu.ru

Первунина Татьяна Михайловна Tatiana M. Pervunina

E-mail: danishevskij.ns@edu.spbstu.ru

Оруджи Махди Mahdi Orooji

E-mail: morooji@gmail.com

Поступила: 23.09.2022; Одобрена: 26.10.2022; Принята: 04.12.2022. Submitted: 23.09.2022; Approved: 26.10.2022; Accepted: 04.12.2022.