CC BY
5Scientific Journal Impact Factor (SJIF 2022=4.63) Passport: http://sjifactor.com/passport.php?id=22230
DEVELOPMENT OF AN ADAPTIVE LEARNING METHOD ALGORITHM FOR ARTIFICIAL NEURAL NETWORKS
This article presents the main factors and methods of artificial intelligence. A universal algorithm of artificial intelligence has also been developed. Using a universal algorithm, gradients for solving a given problem are derived. A general analysis of gradient study methods for neural networks is presented, each of which is presented as a separate case of the proposed algorithm.
Keywords: information system (IS), information technology, intelligent information system (IIS), neural network, artificial neural network (ANN), adaptive method, universal algorithm.
Advances in the field of mechatronics, micro(nano)processor technology and information technology lead to the need to develop and create a new type of information processing and control systems - intelligent. Intellectual problems are those related to the development of algorithms for solving previously unsolved problems of a certain type. Intelligence is a universal algorithm capable of developing algorithms for solving specific problems.
Any information system (IS) performs the following functions:
- perceives the information requests entered by the user and the necessary initial
data;
- processes the data entered and stored in the system in accordance with a known algorithm and generates the required output information.
An intelligent information system (IIS) is an information system based on the concept of using a knowledge base to generate algorithms for solving problems of various classes depending on the specific information needs of users.
The task of training an artificial neural network (ANN) can be considered as an optimization problem, while the main problem is to choose the most suitable among a variety of optimization methods.[2,3]
Basically, all optimization algorithms can be divided into three categories:
• random search methods;
• methods of stochastic gradient descent;
• gradient methods.
Sobitjon Ilhomiddin ogli Ibrohimov
sobitj onibrohimov7 5 @gmail .com
ABSTRACT
UZBEKISTAN
www.caajsr.uz
Scientific Journal Impact Factor (SJIF 2022=4.63) Passport: http://sjifactor.com/passport.php?id=22230
The group of gradient methods includes the following methods:
• gradient descent method;
• heavy ball method;
• conjugate gradient methods.
The choice in favor of gradient methods is justified by the fact that, as a rule, in learning problems, the learning criterion can be expressed as a differentiable function of the neural network weights. However, the uncertainty of the choice of teaching method remains. In automated systems of neural network programming, one should strive to reduce the uncertainty that is inherent in these technologies. The uncertainty in the choice of learning algorithm is eliminated to some extent in the adaptive learning method proposed below.[1,3]
A general analysis of gradient learning methods for neural networks suggests that any of these methods can be represented as a special case of the proposed algorithm.
We express the general formula for changing the weights by formula (1):
Wk+1 = Wk + r]kPk (1)
where the vector pk specifies the direction of movement, and 77 kis the step size at the k-th iteration.
formula for calculating the vector p kas follows:
imin(fe-l,m)
Choosing a starting point
^^^^ criterion ^^^^
Compute the antgradient at a point
1
Pushing the direction onto the stack
1
We consider the direction vector by the formula (2)
1
Moving towards a new point
-» -> f) I II1IIII/V A., I IL I . 1
Pk=9k P + gl i=1 ' Ji,k-
K (2)
where vector sets the direction of movement; - the direction of the antigradient at the j -th iteration; P¿-coefficient that determines the weight of the i -th gradient; m determines the number of memorized gradients; k is the serial number of the current iteration.
The gradient learning method from formula 2 is obtained with . And the methods of conjugate gradients[1], which are most often used in training neural networks, are obtained by summing all the previous directions ( at ) . Thus, the
CENTRAL ASIAN ACADEMIC JOURNAL ISSN: 2181-2489
OF SCIENTIFIC RESEARCH VOLUME 2 I ISSUE 5 I 2022
Scientific Journal Impact Factor (SJIF 2022=4.63) Passport: http://sjifactor.com/passport.php?id=22230
proposed adaptive algorithm is a more flexible solution for training neural networks.
The general algorithm of the adaptive method:
1. Start
2. We choose a starting point with some coordinates (x gj(at the first iteration, the starting point).
3. Push the current direction onto the direction stack.
4. We consider the direction vector according to formula 2.
5. We move along the calculated vector to a new point.
6. We return to step 2. If the stopping criterion is positive, then we end the algorithm, if not, we go to step 3.
7. End of the algorithm. We have a point close to the minimum of the function.
Fig. 1. General algorithm of the adaptive method.
Consider an example of using this approach when optimizing the Rosenbrock function [2]:
f(x, y) = 100 ■ (y - x2)2 + (1 - x)2 (3)
Calculating the minimum of the Rosenbrock function is considered a difficult problem for iterative methods of finding the minimum.
Thus, for the complex Rosenbrock function (3), the adaptive algorithm turned out to be better than the classical methods: gradient descent and heavy ball.
The application of an adaptive algorithm for training neural networks requires tuning the parameters: m, p and h. In this regard, the following adaptive learning method for ANNs is proposed, based on an adaptive algorithm. [2,4]
Denote by ANN 1 - the main neural network that will be trained on real data -training set 1 using a customized adaptive algorithm. To adjust the parameters of the learning algorithm, ANN 2 is used, topologically nested in ANN 1, and training sample 2, obtained by random selection of some examples of training sample 1. To reduce the number of adjustable parameters, we use the representation fyj = a1, where 0 < a < 1.
The controller that trains ANN 2 in automatic mode using the example of training sample 2 selects the optimal parameters for the adaptive algorithm. The criterion in this case is the minimum number of iterations required to achieve the minimum.
After that, the optimal training parameters are transferred to ANN 1 for training on real data.
The proposed training method minimizes human intervention in ANN training, which makes it attractive. Since not every user of ANN technologies has knowledge in
Scientific Journal Impact Factor (SJIF 2022=4.63) Passport: http://sjifactor.com/passport.php?id=22230
the field of optimization methods. In addition, the method is a flexible and customizable learning method for the training sample.
REFERENCES.
1. Yusupbekov N.R, Igamberdiev H.Z, Mamirov U.F: Adaptive control system with a multilayer neural network under parametric uncertainty condition. CEUR Workshop Proc . 2782, 228-234 (2020)
2. Yusupbekov N.R, Igamberdiev H.Z, Zaripov O.O, Mamirov U.F.: Stable iterative neural network training algorithms based on the extreme method. In: Aliev , RA, Kacprzyk , J., Pedrycz , W., Jamshidi , M., Babanli , M., Sadikoglu , FM (eds.) ICAFS 2020. AISC, vol . 1306, pp . 246-253. Springer , Cham (2021). https://doi.org/10.1007/978-3-030-64058-3 30
3. Sharovin I.M., Smirnov N.I., Repin A.I. The use of artificial neural networks for the adaptation of automatic control systems during their operation // Industrial ACS and controllers. 2012. №4
4. Zueva V. N., Stasevich V. P., Shumkov E. A. Construction of adaptive ABS // Intelligent systems: Proceedings of the VII International Symposium / Ed. K. A. Pupkova. - M .: RUSAKI, 2006. - C. 519 - 522.
UZBEKISTAN
www.caajsr.uz
