CC BY
18
Meningioma Detection in MR Images Using Convolutional Neural Network and Computer Vision Methods
Yulia D. Agafonova1*, Andrey V. Gaidel1,2, Evgeniy N. Surovtsev3, and Aleksandr V. Kapishnikov4
1 Samara National Research University, 34 Moscovskoe shosse, Samara 443086, Russia
2 IPSI RAS - Branch of the FSRC "Crystallography and Photonics" RAS, 151 Molodogvardeiskaya str., Samara 443001, Russia
3 LDC MIBS - Tolyatti, 68 October str., Tolyatti 445009, Samarskaya obl., Russia
4 Samara State Medical University, 89 Chapaevskaya str., Samara 443099, Russia
* e-mail: [email protected]
Abstract. The article discusses research efficacy of different architectures of convolutional neural network and methods of computer vision. This paper presents a novel approach to pattern detection of meningioma of the human brain in MR images. MRI images of real patients were made with a help of Samara State Medical University. The result of the research is the automatic procedure of meningioma detection. As a result, post-contrast T1 weighted MRI sequence was the most appropriate for the method based on the baseline statistical segmentation and the diffusion weighted MRI sequence was the most appropriate for the method based on the convolutional neural network. © 2020 Journal of Biomedical Photonics & Engineering.
Keywords: computer vision; magnetic-resonance imaging; segmentation; convolutional neural network; meningioma.
Paper #3382 received 26 Aug 2020; revised manuscript received 18 Sep 2020; accepted for publication 20 Sep 2020; published online 30 Sep 2020. doi: 10.18287/JBPE20.06.030301.
1 Introduction
There is a problem in medicine to make a reliable diagnosis in the shortest possible time due to the growing volume of medical research. This problem is especially important in case of diagnosis of various neoplasms. The presented algorithms can help to solve this problem.
In this paper, we compare algorithms for solving the problem of automated detection of meningioma of the brain in MRI images using deep learning methods, as shown in Ref. [1], videlicet, using convolutional neural network and using computer vision methods, as shown in Ref. [2]. The first approach to the solution of this problem is based on a convolutional neural network and divided into three steps: the image preprocessing, due to the different number of pixels on each image in the sample, the creation of an optimal convolutional neural network architecture and analysis of the results for various MRI sequence. The second approach to the solution of this problem is based on computer vision methods.
The segmentation of brain MRI images for detecting tissues of a given type is an extremely urgent task, as
evidenced by the increasing of interest in it. In Ref. [3], a hybrid approach was proposed combining a convolutional neural network and an ensemble of decision trees. In Ref. [4], a modification of the method of fuzzy clustering of C-means is used for the task of tissue segmentation in images of the brain MRI. Extensive overview of actual methods of detection and localization of brain neoplasms in MRI can be found in Ref. [5]. This article presented the following results in average: precision - 86%, sensitivity - 60%.
Previously, the authors proposed an algorithm for detecting pathological changes in magnetic resonance imaging images of the brain [6], however, the pathological changes themselves were not detected in the work, but the entire image was considered. Later, a heuristic algorithm for automatically highlighting a region of interest in CT images of the lungs was examined for the reason of improving of the quality of further diagnostics [7]. There are also learning methods based on statistical theory. A popular learning method based on statistics is discussed in Ref. [8].
The purpose of this research work was to identify the most appropriate type of MRI sequence for the tumor detection problem. When conducting magnetic
resonance imaging, several pictures are taken in the different types of weighting for the same patient. Some MRI sequences are used depending on the medical case and some of them are always performed. Based on the results of the MRI, the doctor made a conclusion for later use. Therefore, when solving the problem of segmentation and detection of meningioma, we conducted a study to determine the optimal type of weighting for the tumor detection problem. The identification of the optimal type of weighting is an important step for the further analysis of such medical images. In total, the sample consisted of 227 source MRI images and 227 manually marked MRI images. The images had several types of weighting: the FLAIR (T2, T1, F), the diffusion (D) and the contrasting T1 (C). Images were made with a help of MRI of real patients at Samara State Medical University. Examples of input data can be seen in Fig. 1 (a, b). Examples of output manually marked MRI images are shown in Fig. 2 (a, b), respectively.
Fig. 1 Examples of input data for patient (a) and patient (b).
Fig. 2 Examples of output image for patient (a)and patient (b).
2 Materials and Methods
2.1 Baseline statistical segmentation
The first method is based on baseline statistical segmentation. Let x(m, n) be the digital images which map pixel coordinates (m, n) to the gray level q. Actually, we used the whole image as a region of interest. The problem is to detect a tumor in a given image, i.e. to construct a predictor ®[x(m, n)], which
maps an input image x(m, n) to the output image y(m, n) of the same shape such that y(m, n) = 1 for (m, n) from the tumor region and y(m, n) = 0 for the rest of the image.
We considered a simple statistical prediction model as a baseline solution of the segmentation problem. Let U be the training sample containing a set of images x(m, n). We computed an occurrence rate of each gray level in the tumor region:
K ! =
I! I!
I
I
x(m, n),
where |A| is a number of elements in the finite set A, and Dl{x | q) is a set of tumor pixels of gray level q.
We used h(x(m, n)) as a prediction score of a pixel (m, n) because Eq. (1) is a histogram feature representing the best estimation of the conditional probability of the event that pixel (m, n) belongs to the tumor region if it has a gray level q. Of cause, this feature does not know anything about spatial neighborhood of a pixel (m, n), it takes into account only the gray level of the pixel.
To show the effectiveness of the algorithm based on a baseline statistical segmentation, this study will use the following quality metrics:
• sensitivity
Jr = Ntp/(Ntp + Nfn),
• specificity
Spec = Ntn/(! TN + Nfp) ,
• precision (2) Jp = NTP/(NTP + Nfp) ,
• F1 -score
! = 2JpJr/(Jp + Jr) ,
• accuracy
J a = (!tp + /Wtp + ntn + Nfp + !FN),
where is NP - number of correctly classified images with pathological changes, Nm - number of correctly classified images without pathological changes, N -number of images without any pathological changes classified as they have one, and Nfn - number of images with pathological changes classified as they have no pathological changes.
As a result of the study, five MRI sequences were considered. We worked with: MRI sequence C -55 images from the entire sample, available for each
Fig. 3 Architecture diagram of the convolutional neural network used.
patient; MRI sequence D - 21 images from the entire sample, not available for each patient; MRI sequence F - 45 images from the entire sample; MRI sequence T1 - 52 images from the entire sample, available for each patient; MRI sequence T2 - 52 images from the entire sample, available for each patient. These data were obtained from 20 patients, including 8 healthy and 12 patients with the disease. The median age of the patients was 41 years.
2.2 Convolutional neural network
Before input images in a convolutional neural network, we perform image preprocessing. Image pre-processing is included bringing the images to a single format and size, since at different sizes we can get accuracy indicators that are different from reality.
Further, for working with images, the convolutional neural network was used [9]. The original architecture of the neural network is shown in Fig. 3. This neural network consisted of eleven convolutional neural layers. The values of convolution kernels ranged from three to nine samples, the value of filters from 32 to 256 samples.
Initially, the images ranged in size from 256 x 256 to 512 x 512 pixels. When fed into a convolutional neural network, we compress the images to a size of 64 x 64 pixels. The neural network training took place over 7 epochs. In this research work, as an activation function, we used the ReLU activation function.
To estimate the losses during training, we used a binary cross-entropy (3) as a loss function, where N is the number of images in the sample, yi is the class of the i-th image, pi is the value generated by the neural network for the i-th image:
Hp -
1 !
Oi • l°g(Pi) + С1 - yd ■ !°g(1 - Pi))
(3)
The sample for training was about 75% of the main sample, the rest of the images fell into the control sample. The neural network was trained and tested on each MRI sequence separately.
3 Results and Discussion
Table 1 shows the metrics of algorithm based on the baseline statistical segmentation. To evaluate the performance indicators, we also used the ROC curve [10].
As a result, MRI sequence C was the most appropriate subtype of weighting. The results of the algorithm can be seen in Fig. 4, where there is the reference mask on the left, and there is the result obtained by the program on the right. The curves for MRI sequence C are shown in Fig. 5, respectively.
Fig. 4 Examples of output reference mask (left) and obtained by the program mask (right).
The prediction score (1) can be used for ROC-analysis and so on. The final decision rule is h(x(m, n)) > t, where t denotes to a variable threshold. We picked up an optimal threshold t that maximizes the F-score of tumor detection on the training sample.
The results of the algorithm operation based on the convolutional neural network can be seen in Table 2. The metrics presented in (2) were used. As a result, MRI sequence D was the most appropriate subtype of
weighting. The curves are for MRI sequence D shown in Fig. 6, respectively.
Table 1 The result of the baseline statistical segmentation.
Measure -
MRI sequences
T2 T1
F
D
C
Spec Sens
Fi Ja
0.44 0.56 0.06 0.88
0.80 0.20 0.17 0.99
0.88 0.69 0.12 0.31
0.15 0.15
0.99 0.97
1.00 0.30 0.46 1.00
0.4 0.6
False Positive Rate
Fig. 5 Changes in the detection assessment parameters during the training of the baseline statistical segmentation on the ROC curve.
Fig. 6 Changes in the detection assessment parameters during the training of the convolutional neural network the ROC graph.
References
Table 2 The resulting metrics on the control sample, obtained as a result of the convolutional neural network training
Measure
MRI sequences
T2 T1
F
D
Spec 0.33 0.55 0.56 0.78 0.39
Sens 0.67 0.45 0.44 0.22 0.61
F1 0.04 0.04 0.11 0.20 0.11
J 0.84 0.90 0.95 0.99 0.93
4 Conclusion
The paper compares approaches for the automatic detection of brain meningioma in the MRI images. A comparison was made of an algorithm based on baseline statistical segmentation and an algorithm based on a convolutional neural network. All work was performed on five different subtypes of MRI sequences. The most optimal subtype of weighting was identified for each method. These approaches can be combined later into a single method that solves the detection problem even more effectively. The most informative MRI sequences for making a diagnosis automatically using proposed algorithms are the diffusion (D) and the contrasting T1 (C). The results can be used in the design of an information system for the automated diagnosis of various pathologies using MRI images of the human brain.
Disclosures
All authors declare that there is no conflict of interests in this paper.
Acknowledgements
The work was partially funded by the Russian Foundation for Basic Research under grants No. 19-2901235 and 19-29-01135 (theoretical results) and the RF Ministry of Science and Higher Education within the government project of the FSRC "Crystallography and Photonics" RAS under grant No. 007-GZ/Ch3363/26 (numerical calculations).
J. D. Agafonova, A. V. Gaidel, "Localization of the area of pathological changes in the images of braint MRIs, Proceedings of the Advanced Information Technology (AIT 2019), 362-365 (2019) [in Russian].
in
C
2. R. Szeliski, Computer vision: algorithms and applications, Springer Science & Business Media (2010). ISBN 9781-84882-934-3.
3. T. Yang, J. Song, and L. Li, "A deep learning model integrating SK-TPCNN and random forests for brain tumor segmenta-tion in MR," Biocybernetics and Biomedical Engineering 39(3), 613-623 (2019).
4. A. Haider, N. A. Talukdar, "Brain tissue segmentation using improved kernelized rough-fuzzy C-means with spatio-contextual information from MRI," Magnetic Resonance Imaging 62, 129-151 (2019).
5. A. Wadhwa, A. Bhardwaj, and V. S. Verna, "A review on brain tumor segmentation of MRI images," Magnetic Resonance Imaging 61, 247-259 (2019).
6. Yu. D. Agafonova, A. V. Gaidel, P. M. Zelter, and A. V. Kapishnikov, "Efficiency of machine learning algorithms and convolutional neural network for detection of pathological changes in MR images of the brain," Computer Optics 44(2), 266-273 (2020).
7. A. V. Gaidel, "Method of automatic ROI selection on lung CT images," Procedia Engineering 201, 258-264 (2017).
8. V. N. Vapnik, Statistical learning theory, New York, John Wiley & Sons (1998).
9. K. He, J. Sun, "Convolutional neural networks at constrained time cost," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5353-5360 (2015).
10. T. Fawcett, "An introduction to ROC analysis," Pattern Recognition Letters 27(8), 861-874 (2006).
