CC BY
11
УДК 004.93
АКТУАЛЬНЫЕ НАПРАВЛЕНИЯ В РАЗВИТИИ НЕЙРОННЫХ СЕТЕЙ И СПОСОБЫ ИХ УЛУЧШЕНИЯ
Д. Ю. Манылов
Сибирский государственный университет науки и технологий имени академика М. Ф. Решетнева
Российская Федерация, 660037, г. Красноярск, просп. им. газ. «Красноярский рабочий», 31
E-mail:diyyydn@gmail.com
Данная статья освещает проблему эффективности нейронных сетей и их применения в различных областях человеческого знания. Рассматриваются текущие типы нейросетей, их алгоритмы, достоинства и недостатки применения в той или иной ситуации. Предлагается улучшение работы одного из алгоритмов путем применения транспортной задачи. В результате теоретически обосновывается ожидаемое изменение работы алгоритма и повышение эффективности работы нейронной сети.
Ключевые слова:нейронные сети, машинное обучение, сверточные сети, рекуррентные сети.
CURRENT DIRECTIONS IN THE DEVELOPMENT OF NEURAL NETWORKS AND METHODS OF THEIR IMPROVEMENT
D. Y. Manylov
Reshetnev Siberian State University of Science and Technology 31, Krasnoyarsky Rabochy Av., Krasnoyarsk, 660037, Russian Federation E-mail: diyyydn@gmail.com
The article covers the problem of the effectiveness of neural networks and their applications in various areas of human knowledge. We consider the current types of neural networks, their algorithms, advantages and disadvantages of application in a given situation. Further, we propose to improve the operation of one of the algorithms by applying the transport problem. As a result, the expected change in the operation of the algorithm and the increase in the efficiency of the neural network are theoretically justified.
Keywords: neural networks, machine learning, convolutional networks, recurrent networks.
Neural networks, as one of the methods of machine learning, are now riveted by the attention of many people, both experts in the field of data analysis and people who are not connected with such professions. This is largely due to the fact that problems solved with the help of neural networks often have a connection with speech recognition, as well as with the analysis of text and images.
Although neural networks have been experiencing a rise of popularity over the past five years, this technology itself already has a half-century history. Formally, the first version of a neuron or cell of a neural network was proposed in 1943 by scientists Walter Pitts and Warren McCulloch [2]. After some time, the scientist Frank Rosenblatt proposed the first simple neural network, which distinguished objects in two-dimensional space. The rise of the popularity of neural networks over half a century has happened three times: in the 50s, when the first impressive results were obtained, in the 90s and in the 10s of the 21st century. Decrease of popularity occurred in those time periods when other machine learning algorithms turned out to be more efficient at the given time. For present time, the popularity of neural networks is due to three factors:
- the appearance of new algorithms;
- availability of large computing resources;
Секция «Программные средства и информационные технологии»
- growing amount of data for training.
Next will be considered the current directions in the development of neural networks. One of the methods that have proven to be effective is convolutional neural network. The example in Figure 1 demonstrates the work principle of this network.
Figure 1. Network operation principle
It is assumed that all the pixels in the image can be tied to a single neuron (as in the situation in Figure 1 on the left), but this option is limited to the result that a large number of weights are obtained, which will lead to a long calculation time for the sum of each neuron. This method will also be unstable to retrain, which is why the neural network will work ineffectively with examples that are not the part of training set [1]. To solve these problems, an architecture was developed in which each neuron is associated with only a small area of the image, and all neurons have the same weights. This process, shown in Figure 2, is a convolution of the image.
(4x0) (0*0) (0X0) (0*0) (OKI)
Over the past year another neural network model, the recurrent model, is also gaining popularity. This network, in contrast to convolutional networks for images, was developed primarily for operating with texts and originates from direct propagation networks. The difference is that a regular network of direct distribution does not store the memory of previous actions. If it is required to analyze any text, it is necessary to make sure that each analyzed word is taken into account when further searching the text. For this, recurrent networks have been developed, since in addition to the presence of paths from the input to the output from the network, they take signals from previous time steps into account, which concludes in some kind of flow of information back to the input.
This method is also applicable for text generation. To do this, initialization of the text with a single word is enough, and then as the training continues, the network will be able to reproduce meaningful
words. Another use for such networks is the generation of source code for programs. One example is a network that is trained on the Linux kernel, which is already capable to generate meaningful functions, conditional expressions and loops. Further this may lead to partial automation of programming. Another application of this network is an acquiring of brief extract of the article submitted to the input. Finally, with this method you can train the system to communicate with users in technical support, for which numerous logs of people communicating with the technical support team are used.
The previously presented types of neural networks are only a small part of the innovations in the sphere of machine learning. There is also a combination of using convolutional and recurrent networks, and their application in generating an image description or vice versa, generating an image as described. Next, the article will describe own ideas for improving the performance of some neural network algorithms.
It was decided to use Rosenblatt's one-dimensional perceptron [3] and build a neural network based on it, and therefore to develop it with the help of a transportation problem. Further, it is intended to expand the network to multidimensional, which will greatly complicate the calculations and network development. For comparative results, this neural network will be written in the C# and Python programming languages, which will provide a practical result of the neural network (for example, working with text or images) and draw conclusions about the success of improving the efficiency of the selected algorithm.
Thus, current trends in the development of neural networks were considered and a proprietary method for improving the performance of the perceptron algorithm was proposed. It is likely that the current popularity of neural networks in machine learning will soon subside, as has already happened several times in the last century. This will most likely be dictated by the effectiveness of the use of neural networks for current tasks in image recognition.
References
1. Fausett, Laurene. Fundamentals of neural networks. London: Pearson, 1993. p. 125
2. Gonzalez, Rafael. Pattern Recognition Principles. Boston: Addison-Wesley, 1977. p. 56
3. Rutkowski Leszek, Rutkowska Danuta. Neural networks, genetic algorithms and fuzzy systems. Warsaw: PNW, 1997. p. 350
© MaHMTOB fl. ro., 2019
