CC BY
11
УДК 517.937, 517.928.2, 519.711.2
Васильев С.А., Канзитдинов С.К.
Российский университет дружбы народов, г. Москва, Россия
МОДЕЛЬ НЕЙРОННЫХ СЕТЕЙ С БЕСКОНЕЧНЫМ ЧИСЛОМ ЭЛЕМЕНТОВ И
НАЛИЧИЕМ МАЛОГО ПАРАМЕТРА
АННОТАЦИЯ
В статье исследована динамика сложных систем с помощью нейронных сетей с бесконечным количеством элементов (клеток). Сформулирована задача Коши для систем дифференциальных уравнений бесконечного порядка, которая описывает нейронные сети с бесконечным числом клеток, и рассмотрен вопрос о существовании и единственности решении такой задачи.
КЛЮЧЕВЫЕ СЛОВА
Бесконечные нейронные сети; системы дифференциальных уравнений бесконечного порядка; сингулярно возмущенные системы дифференциальных уравнений; малый параметр; прогнозирование динамики сложных систем.
Vasilyev S.A., Kanzitdinov S.K.
Peoples' Friendship University of Russia, Moscow, Russia
MODEL OF NEURAL NETWORKS WITH AN INFINITE NUMBER OF CELLS AND SMALL
PARAMETER
ABSTRACT
A method of analysis the dynamics of complex systems using neural networks with an infinite number of cells was investigated. For the Cauchy problem for systems of differential equations of countable order, which describes the neural network with infinite number of cells, considered the question of the existence and uniqueness of its solution.
KEYWORDS
Infinite neural networks; systems of differential equations of an infinite order; singularly perturbed systems of differential equations; a small parameter; predictions of the dynamics of complex systems.
INTRODUCTION
The recent research of the dynamics of complex systems using neural networks with an infinite number of cells in faced with the problem of the solutions analysis of certain infinite systems of ordinary differential equations to a time-independent solution. A model for a large network of "neurons" with a graded response (or sigmoid input-output relation) was studied [4]. The idea was used in biological systems was given added credence by the continued presence of such properties for more nearly biological "neurons". In the paper [1] was given existence and uniqueness results for the equations describing the dynamics of some neural networks for which there were infinitely many cells. Such system was considered and neural nets which were modelled were described by the infinite system of ordinary differential equations.
Bruce D. Calvert and Armen H. Zemanian [2] investigated a nonlinear infinite resistive network, an operating point could be determined by approximating the network by finite networks obtained by shorting together various infinite sets of nodes, and then taking a limit of the nodal potential functions of the finite networks. By taking a completion of the node set of the infinite network under a metric given by the resistances, limit points were obtained that represent generalized ends, which it be called "terminals," of the infinite network. These terminals could be shorted together to obtain a generalized kind of node, a special case of a 1-node. An operating point will involve Kirchhoff's current law holding at 1-nodes, and so the flow of current into these terminals was studied. They gave existence and bounds for an operating point that also had a nodal potential function, which was continuous at the 1-nodes. The existence was derived from the said approximations.
Haiying Huang, Qiaosheng Du and Xibing Kang [5] studied a class of neutral high-order stochastic Hopfield neural networks with Markovian jump parameters and mixed time delays. The jumping parameters was modeled as a continuous-time finite-state Markov chain. The existence of equilibrium point for the addressed neural networks was studied. By utilizing the Lyapunov stability theory, stochastic analysis theory and linear matrix inequality (LMI) technique, new delay-dependent stability criteria were presented in terms of linear matrix inequalities to guarantee the neural networks to be globally exponentially stable in the mean square.
In paper [15] Xiao Liang, Linshan Wang, Yangfan Wang and Ruili Wang focused on the long time behavior of the mild solution to delayed reaction-diffusion Hopfield neural networks (DRDHNNs) driven by infinite dimensional Wiener processes. They analyzed the existence, uniqueness, and stability of this system under the local Lipschitz function by constructing an appropriate Lyapunov-Krasovskii function and utilizing the semigroup theory. Some easy-to-test criteria affecting the well-posedness and stability of the networks, such as infinite dimensional noise and diffusion effect, were obtained. The criteria could be used as theoretic guidance to stabilize DRDHNNs in practical applications when infinite dimensional noise was taken into consideration. Considering the fact that the standard Brownian motion is a special case of infinite dimensional Wiener process, they undertake an analysis of the local Lipschitz condition, which had a wider range than the global Lipschitz condition.
In 1984 Hopfield investigated a neural network which was described using system of ordinary differential equations [4]
cdU = fr/g U (t))+d - ''M,
i dt P J ' R (1)
i, J = 1,...,N,t > 0
or
du.
—•- = f (u, g, t) + r, ,i, J = 1,..., N,
dt
f (u, g, t) = fT/g(Uj )/ Ct,rt = (d, - ut (t) / R)/ C,, (2)
J=1
where Ui (t) are the voltage changes on the neuron, which determines the state of the system, C, >0, R > 0 , Tv, di are sets of real numbers, and a function g e [-1;1], (5 e R) increasing function. In this work, we study the following system of singularly perturbed differential equations, which is a generalization of system (1):
du.
= f (u,g,t) + mt, i = 1,...,N,
(3)
dt
/u-—U- = F(U,g,t) + , i = N +1,..., dt
f, (u, g, t ) = fT/g (Uj )/ C,,
j=1
mt =(dt - ut (t)/ R)/ Ct, i = 1,..., N,
F, (u, g, t ) = fT/g (Uj )/ Ci,
j=1
Mi =(d, - ut (t)/ R)/ C, i = 1,2,.,
(4)
(5)
where u, (t), i = 1,2,. is a functional sequence, Ct >0, R >0, TJ, dt <z ll are numerical sequence, and p. > 0 is a small parameter.
For system (3) we can formulate the following Cauchy problem:
—'- = f (u, g, t) + mt ,i = 1,..., N, dt
p — = F (u, g, t) + Mt ,i = N +1,...,
dt
u (0) = u°,i = 1,2,.,
where u0 c ll numerical sequence.
Cauchy problems for the systems of ordinary differential equations of infinite order was investigated A.N.Tihonov [10], K.P.Persidsky [9], O.A.Zhautykov [13], [14], Ju.Korobeinik [6] other researchers.
It was studied the singular perturbated systems of ordinary differential equations by A.N. Tihonov [11], A.B.Vasil'eva [12], S.A. Lomov [8] other researchers.
In this paper the Cauchy problem (6) is considered the existence and uniqueness of its solution, an algorithm for constructing asymptotic solutions using approximate methods of solutions of differential equations with a small parameter at the highest derivative and analyzed the possibility of applying it solutions to predict the dynamics of complex systems in conditions of uncertainty.
For the Cauchy problem for systems of differential equations of countable order, which describes the neural network with infinite number of cells, considered the question of the existence and uniqueness of its solution, an algorithm for constructing asymptotic solutions using approximate methods of solutions of differential equations with a small parameter at the highest derivative and analyzed the possibility of applying it solutions to predict the dynamics of complex systems in conditions of uncertainty.
In this paper we apply methods from [12] for the singular perturbated systems of ordinary differential equations of infinite order of Tikhonov-type.
TIKHONOV-TYPE CAUCHY PROBLEMS FOR SYSTEMS OF ORDINARY DIFFERENTIAL EQUATIONS OF INFINITE ORDER WITH A SMALL PARAMETER
Let's consider Tikhonov-type Cauchy problems for systems of ordinary differential equations of infinite order with a small parameter p and initial conditions (6):
x = f (x(t), y(t), t) + m,
where x, f, m e X , X e Rn are n-dimensional functions; y, F,M e Y , Y c lx are infinite-dimensional functions and t e \t0, tl ] (t0 < tl <x>), t e T, T e R ; u0 e X and u0y e Y are given vectors, p > 0 is a small real parameter; x(t) = (xl,...,xN)T and y(t) = (y1,y2,---)T are solutions of (7). Given functions f(x(t),y(t),t) = (f1,., fN)T and F(x(t),y(t),t) = (F1,F2,...)T are continuous functions for all variables.
Let S is an integral manifold of the system (7) in X x Y x T . If any point t* e\t0,tl] (x(t'),y(t'),t*) e S of trajectory of this system has at least one common point on S this trajectory (x(t), y(t), t) e S belongs the integral manifold S totally. If we assume in (7) that p = 0 than we have a degenerate system of the ordinary differential equations and a problem of singular perturbations
where the dimension of this system is less than the dimension of the system (7), since the relations F(x(t),y(t),t) = 0 in the system (8) are the algebraic equations (not differential equations). Thus for the system (8) we can use limited number of the initial conditions then for system (7). Most natural for this case we can use the initial conditions x(t0) = u°° for the system (8) and the initial conditions y(t0) = u0y disregard
py = F (x(t), y (t), t) + M ;
x(L ) = u° v(L ) = u°
(7)
y
x = f ( x(t ), y(t ), t ) + m, 0 = F ( x(t ), y(t ), t); x(t0 ) = u0x,
(8)
otherwise we get the overdefined system. We can solve the system (8) if the equation F(x(t),y(t),t) = 0 could be solved. If it is possible to solve we can find a finite set or countable set of the roots yq (t) = wq (x(t), t) where q e N .
If the implicit function F(x(t),y(t),t) = 0 has not simple structure we must investigate the question about the choice of roots. Hence we can use the roots yq (t) = wq (x(t), t) (q e N) in (8) and solve the degenerate system
= f (xd (t), wq (xd (t), t), t) + m;
yd (t0) = ul
(9)
Since it is not assumed that the roots yq (t) = wq (x(t), t) satisfy the initial conditions of the Cauchy problem (7) (yq(t0) ф wx, q e N), the solutions y(t) (7) and yq(t) do not close to each other at the initial moments of time t >0 . Also there is a very interesting question about behaviors of the solutions x(t) of the singular perturbated problem (7) and the solutions xd (t) of the degenerate problem (9). When t = 0 we have x(t0) = xd(t0). Do these solutions close to each other when t e(t0,tl] ? The answer to this question depends on using roots yq (t) = wq (x(t), t) and the initial conditions which we apply for the systems (7) and (8).
LOCAL EXISTENCE THEOREM FOR CAUCHY PROBLEMS FOR SYSTEMS OF ORDINARY DIFFERENTIAL EQUATIONS OF INFINITE ORDER
Let Tikhonov-type Cauchy problems for systems of ordinary differential equations of infinite order with a small parameter p > 0 and initial conditions (7) has a form:
Г z = P( z(t, G, p), t, p) + Q;
I к, (10)
z(t0, P) = G,
where
Z = (xi, X2 ,..., XN , yi, y2,...) ,
P(z(t, G, p), t, p) = f f2,.., fN, p-1 Fx, p-lF2,...)T,
Q = (m1, m2,..., mN, Mn+1, MN+ 2,...)t ,
/->/00 0 00 \T
G = (Uol, Ux2 ,UN , uyl, uy2,...) ,
P (z(t, G, p), t, p) is the infinite-dimensional function; G is the given vector; t e[t0, ] (t0 < <x>).
Let z(t,G, p) be a continuously differentiable solution of the Cauchy problems (10) then there are
®(t,G,p) = dz(t,G,p)/dG, T(t,G, p) = dz(t,G,p) /dp where ®(t,G,p) and T(t,G,p) satisfy of the system
of ordinary differential equations in variations:
z = P(z(t, G, p), t, p),
Ф (t, G, p) = Jz (t, G, p)(^{t, G, p), T (t, G, p) = Jz (t, G, p)T (t, G, p) + Ap(t, G, p); z(t0, G, p) = G,
Ф(t0,G,p) = I, T(t0,G,p) = 0,
(11)
where Jz (t, G, p) = (dp / dzj)°°J=1 is Jacobi matrix, I is an identity operator and Ap(t, G, p) = (dp / dp)"=l is a vector.
Theorem 1 (local existence theorem). Let P(z(t,G,p),t,p), Jz(t,G,p), Ap(t,G,p) be continuous and meet Gelder's local condition with z e Us (G) then the system (11) has only one solution, which meet the conditions z(t0,G,p) = G, z(t,G,p) e US(G). Thus z(t,G,p) continuously differentiable with respect to the initial condition, and its derivative meet the equation (11).
A = 0
Proof. This statement is following from [3] (theorem 3.4.4) when the unlimited operator be . End proof.
z (t Cj )
The behavior of the solution ^ ' ' (10) and the nonnegative condition for the off-diagonal
elements of the matrix Jz (t, G, p) is demonstrated by the following theorem.
Theorem 2. Let the solution z of (10) be z(t, G, p) e ll for any t > 0, G e ll and p. The following claims are equal: (i) the off-diagonal elements Jz (t, G, p) are non-negative for any G; (ii) for any G and any vector h e ll, h > 0,z(t,G + h, p) > z(t,G, p) ..
Proof. Let us examine a convex set Z , and z(t, G, p) e Z for any G e Z, derivative 0(t,G, p) of function z(t,G,p) can be specify by simultaneous equations (11). In that case the following formula is fair for any G0, G1 e Z :
z(t,G\ p) - z(t,G0, p)= f ® (t, y(s), p)(Gl - G0)ds
j0 (12)
where ^(s) = (1 - s)G0 + sG1,0 < s < 1.
In fact the function z(t,G,p) transfer the segment y(s) into the curve z(t,y(s),p) in (12). The following formula is fair because of the continuous differentiability of function z(t, G, p)
z(t,y(r), p) = z(t, G0, p) + f dz(t,r(s),p) ds.
J0 ds
By the formula of complex derivative
dz(t, y(s), p) _ dz
ds dG
(7(s))r'(s)
t = 1
Recalling that dz / dG = ® and y'(s) = G1 - G0, with we get (12). Let us suppose that statement (i) is
fair. So because of (12)
z(t, G + h, p) - z (t, G, p)= \ O(t, y( s), p)hds
J 0
y(s) = G + sh 0 < s < 1 Ji(t, G, P)
where ' v ' ' . Because of non-negativeness of function outside of diagonal
from (12) we get 0(t,y(s),p) > 0 , so O(t,y(s),p)h > 0 whence we get statement (ii).
Let us suppose that (ii) is fair. Under the conditions of Theorem 1 P, Jz with z e Us (G) be continuous and meet Gelder's local condition. Let Gelder's local condition be P P P< M0, P J P< M1, and there are numbers S >0, S = min(e /M0,1/M1). Let z(t,G,p) = G + z'(t,G,p) be a solution of (12), where
z'(t,G,p) is a fixed point of Picard's mapping (^f9)(t) = \ P(G + d(r))dT under conditions
t e [t0-8X,t0 + 5j], 8X< S . Mapping ^ is contraction with coefficient X = 81M1<\. Consider the approximation to solution z(t, G, p) = G + z'(t, G, p) = G + (t -10) P ( z (t, G, p), t, p). We can see that
P z (t, G, p) - z(t, G, p)P==P z* (t, G, p) - z* (t, G, p) P<
<7^P US(t, G, p) - z(t, G, P)P,
1 - A
G, p) - z(t, G, p) =
= \'P(G + (t-t0)P)dz - \tPdz =
Jt0 Jt0
= \t (P(G + (t -10)P) -P)dT = D.
Jt0
Because of the derivative of the function P is limited and P meet Gelder's local condition with the constant M1, where
P P(G + (T -10 )P(G)) - P(G)P< M,P{T- t0 ) P(G) P< M 0 M, | T -10\, so P D P< M0 Ml(t -10)2/2(1 - A), or P z(t, G, p) - z(t, G, p)P< M0 Ml(t -10)2 / 2(1 - A). Using this estimation and for all small Ç > 0 we have that
0 < z (t, G + Çej, p) - z (t, G, p) = Çej + (t -10)[ P (G + Çej ) - P (G )] + y (G, t), where Py(G, t)P< M0 Ml(t -10)2/2(1 - A) and es is a vector, which has all coordinates equal to 0 but j -th coordinate equal to 1. Component i ^ j of this inequality is given by 0 < (t - t0)[Pi (G + t,ej ) - Pt (G)] + yt (G, t)
. Dividing by t -10>0 and directing t ^ t0 on the right, considering yt (G, t)/(t -10) ^ 0 we get 0 < P(G + C,ej) - P(G). Let us divide last expression by С and direct С ^ 0
0 P(G + Cej) - P(G) = dP, = т
0 < й+-С--" Jj
what is mean the fairing of statement (i). End proof.
Ф G°,G1 e X t > 0 ll> 0
Theorem 3. Let be Markovian mapping and , , ^ than
Pz(t,G\p) -z(t,G0,p)P<PG1 -G0 P.
Proof. Using (6) from the proofing of theorem 4 we have
Pz(t,G\p) -z(t,G0, p)P< £P<&(t, y(s))(G1 -G0)Pds (13)
Let function ^(s)) is Markovian mapping for any
t > 0, s e [0,1], PФ(t,y(s))(G1 -G0)P<PG1 -G0P. Estimating the integral, considering this inequality, we get required. End proof.
This theorem shows us the following sufficient condition for the boundedness of the norm-solution
z(t, G, p).
Corollary fact from theorem 3. Let 3G* e X : z(t,G*,p) = G*. Then Pz(t,G,p) - G* ||<PG - G*P with t > 0,G e X .
This fact we can use for solutions analysis of the systems (10).
CONCLUSIONS
For the Cauchy problem for systems of differential equations of countable order, which describes the neural network with infinite number of cells, considered the question of the existence and uniqueness of its solution. Next step for investigation is constuting an algorithm for asymptotic solutions using approximate methods of solutions of differential equations with a small parameter at the highest derivative and analyzed the possibility of applying it solutions to predict the dynamics of complex systems in conditions of uncertainty.
References
Calvert B.D. Neural networks with an infinite number of cells, Journal of Differential Equations. Vol. 186, Issue 1, 2002. — pp. 31 - 51.
Calvert B.D., Zemanian A.H. Operating points in infinite nonlinear networks approximated by finite networks, Trans. Amer. Math. Soc. Vol. 352, No 2 , 2000. — 753 - 780.
3. Henry D. Geometric theory of semilinear parabolic equations. Lecture Notes in Mathematics. Springer-Verlag, Berlin, 1981.
4. Hopfield J.J. Neurons with graded response have collective computational properties like those of two state neurons, Proc. Natl. Acad. Sci. USA. Vol. 81, 1984. — pp. 3088 - 3092.
Haiying Huang, Qiaosheng Du, Xibing Kang. Global exponential stability of neutral high-order stochastic Hopfield neural networks with Markovian jump parameters and mixed time delays. ISA Transactions. Vol. 52, Issue 6, 2013. — pp. 759 - 767. Korobeinik Ju. Differential equations of infinite order and infinite systems of differential equations. Izv. Akad. Nauk SSSR Ser. Mat. Vol. 34, 1970. — pp. 881 - 922.
Krasnoselsky M.A., Zabreyko P.P. Geometrical methods of nonlinear analysis. Springer-Verlag, Berlin, 1984. Lomov S. A. The construction of asymptotic solutions of certain problems with parameters. Izv. Akad. Nauk SSSR Ser. Mat. Vol. 32, 1968. — pp. 884 - 913.
9. Persidsky K.P. Izv. AN KazSSR, Ser. Mat. Mach., Issue 2, 1948. — pp. 3 - 34.
10. Tihonov A. N. Uber unendliche Systeme von Differentialgleichungen. Rec. Math. Vol. 41, Issue 4, 1934. — pp. 551 - 555.
11. Tihonov A. N. Systems of differential equations containing small parameters in the derivatives. Mat. Sbornik N. S. Vol. 31, Issue 73, 1952. — pp. 575 - 586.
12. Vasil'eva A. B. Asymptotic behaviour of solutions of certain problems for ordinary non-linear differential equations with a small parameter multiplying the highest derivatives. Uspehi Mat. Nauk. Vol. 18, Issie 111, no. 3 , 1963. — 15 - 86.
13. Zhautykov O. A. On a countable system of differential equations with variable parameters. Mat. Sb. (N.S.). Vol. 49, Issue 91, 1959. — pp. 317 - 330.
14. Zhautykov O. A. Extension of the Hamilton-Jacobi theorems to an infinite canonical system of equations. Mat. Sb. (N.S.). Vol. 53, Issue 95, 1961. — pp. 313 - 328.
15. Xiao Liang, Linshan Wang, Yangfan Wang, Ruili Wang. Dynamical Behavior of Delayed Reaction-Diffusion Hopfield Neural Networks Driven by Infinite Dimensional Wiener Processes. IEEE Transactions on Neural Networks. Vol. 27, No 9, 2016. — pp. 1816 - 1826.
Поступила 21.10.2016
Об авторах:
Васильев Сергей Анатольевич, доцент кафедры прикладной информатики и теории вероятностей Российского университета дружбы народов, кандидат физико-математических наук, svasilyev@sci.pfu.edu.ru;
Канзитдинов Солтан Каанзитдинович, аспирант кафедры прикладнои информатики и теории вероятностеи Россииского университета дружбы народов, кандидат физико-математических наук, skanz@yandex.ru.
