CC BY
0
УДК 332.85
CONVOLUTIONAL BUILDING BLOCKS ANALYSIS
M.V. Gordienko*, N. V. Shaposhnikova, Y. S. Ganzha
Reshetnev Siberian State University of Science and Technology 31, Krasnoyarskii rabochii prospekt, Krasnoyarsk, 660037, Russian Federation Е-mail: manamah24@yandex.ru
In this paper, the convolutional neural networks building block are demonstrated.
Keywords: convolutional neural networks, convolutional building blocks
АНАЛИЗ БЛОКОВ СВЕРТОЧНЫХ НЕЙРОННЫХ СЕТЕЙ
*
М.В. Гордиенко , Н. В. Шапошникова, Ю. С. Ганжа
Сибирский государственный университет науки и технологий имени академика М.Ф. Решетнева Российская Федерация, 660037, г. Красноярск, просп. им. газ. «Красноярский рабочий», 31
Е-mail: manamah24@yandex.ru
В представленной работе представлен обзор различных блоков сверточных нейронных сетей.
Ключевые слова: сверточные нейронные сети, сверточные блоки
Currently, convolutional neural networks are a very serious tool for solving many applied problems, including computer vision problems. There are many different state-of-the-art architectures of varying degrees of complexity and speed. Most of these architectures are nothing more than a sequential set collection of specific convolutional building blocks.
The Inception Block [1] fig.1 uses the idea of parallel use of convolutions with different kernel values and its subsequent concatenation into a single tensor. To reduce computational costs, a 1x1 convolution is used before each convolutional block, which reduces the dimension of the signal for its further transmission to convolutions with large core sizes.
Fig. 1. Inception Module
Секция «Математические методы моделирования, управления и анализа данных»
It is well known that deep networks are difficult to train because of the infamous vanishing gradient problem - that is, when the gradient back extends to earlier layers, repeated multiplication can make the gradient extremely small. As a result, as the network deepens, it becomes saturated and its accuracy begins to decline rapidly. The idea of the Residual Block [2] fig. 2 is to add an additional link passing by the convolutional layers. Then the output signal is summed with the input signal.
Fig. 2. Residual Block
Dense Black [3] fig. 3 is the ultimate use case of a Residual Block, where each convolutional layer gets the output of all the previous convolutional layers in the block. It is important to note that, unlike ResNet, feature maps are not summed before they are passed to the next layer, but are concatenated into a single tensor.
Squeeze-and-Excitement Block [4] fig. 4 is a simple module that can be added to any existing architecture with almost no computational cost. Each of the input channels is compressed to a single value and transmitted to a two-level neural network. Depending on the distribution of channels, this network learns to evaluate channels based on their importance. At the end, this weight is multiplied by the resulting convolutions.
С'
С
Fig. 4. Squeeze-and-Excitation Block
Each channel is compressed into a single value and fed into a two-layer neural network. Depending on the channel distribution this network will learn to weight the channels based on their importance. In the end this weight is multiplied with the convolutional activations.
Thus, some of the block types have been considered.
References
1. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1-9, 2015.
2. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, "Deep residual learning for image recognition," in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
3. Huang, Gao, et al. "Densely connected convolutional networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017
4. Jie Hu, Li Shen, and Gang Sun: "Squeezeand-excitation networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132-7141.
© Gordienko M.V., Shaposhnikova N. V., Ganzha Y. S., 2021
