CC BY
30
UDC 004.8
A.T. Nguyen, A.M. Korikov
Neural network model with fuzzy activation functions for time series predictions
This study develops neural models using fuzzy activation functions to solve the problems of time series predictions. Several fuzzy neural networks with different types of activation function are created. The paper shows the comparison result between the feasibilities of these networks for solving time series prediction problems. Keywords: fuzzy neural network, fuzzy activation function, membership function, time series predictions. doi: 10.21293/1818-0442-2016-19-4-49-51
In recent years, methods based on artificial intelligence have been widely used to solve prediction problems for data provided in the time series form. Several neural network models have been developed, such as the feed-forward network and the Elman network models [1]. In this paper, we propose fuzzy neural networks that utilize membership functions as activation functions (AFs) to obtain time series solutions.
Modeling of membership functions as the activation functions
According to literature [2], the fuzzy number is a convex, normalized fuzzy set, whose membership function |(x) is at least segmentally continuous. Moreover,
l(x) must have a functional value at only one value of
x. This value of x is regarded as the mean of the fuzzy number.
There are various different types of AF used in neural networks. In this paper, we propose a new form of membership function using the triangular fuzzy number N = (A, B, C), where A, B and C are three specific points.
The support of fuzzy number N is defined as follow [2]:
supp(N) = [A,C]: |N(x)>0, Vxe[A,C].
The LR-type membership function, which is defined by expression (1), is used in our paper [3].
|n (x,A,B,C) =
fL (x), xe[AB);
1 x=B; fR (x), xe(B,C]; 0, xg[A,C].
(1)
where fL (x), fR (x), which have the form of second-
order polynomial, denote the left and right parts of the membership function.
If fL (x), fR (x) are second-order polynomials and
their derivatives, equal zero at specific points, the membership function given by equation (1) could be given by one of these expressions [3]:
|fL (A)=0;
f (C )=0.
IfL '(A) = 0; f (C )=0.
IfL '(A) = 0; f (C) = 0.
|fL '(A) = 0; Vr (C) = 0.
(2)
(3)
(4)
(5)
The shapes of membership functions utilizing triangular fuzzy numbers with the conditions (2)-(5) are shown in Fig. 1.
c d
Fig. 1. Membership functions with additional conditions: a - conditions by equation (2); b - conditions by equation (3); c - conditions by equation (4); d - condition by equation (5)
ffomadu TYCYPa, moM 19, № 4, 2016
50
ELECTRONICS, RADIO ENGINEERING, COMMUNICATIONS, AND COMPUTER SCIENCE
A fuzzy neural network is represented in a multilayer feed-forward network with AND - OR neurons [4]. More details about the definitions of AND - OR neurons can be found in [5]. The AND and OR fuzzy neurons realize pure logic operations on the membership values. The role of the connections is to differentiate between particular levels of impact that the individual inputs might have on the result of aggregation.
The fuzzy neural network, which is mentioned above, is termed the first-type fuzzy neural networks. The present paper proposes the second-type fuzzy neural network model, whose AFs are given by Fig. 1. The first-type fuzzy neural networks utilize fuzzy inference systems by neural network methods. For the second-type fuzzy neural network model, fuzziness is an attribute of neurons. Moreover, it is necessary to study the third type of fuzzy neural network, which is formed by combining the first and second types.
Application of the second-type fuzzy neural network to solve the problem of time series predictions
Time series are sequences of numbers having time-dependent characteristics. A time series is usually represented by a vector x(t), t = 0,1,....., where t - elapsed
time.
Theoretically, the value of x changes continuously with time t. However, for numerical analyses, signals of physical systems are represented by series of discrete data. The windowing method [6] is used to divide time series into segments (windows), and then a training data set is obtained. After using the windowing method with the segments of size d, we can create the mapping set of input-output data. The methods of so-called «theory of experiment» [7] are used to process the data of the experiment.
In this paper, we investigate the feasibility of the second-type fuzzy neural network model for the problem of time series predictions.
The experiment uses the set of time series data, which provides daily measured pollution for the last one and half years. The fuzzy neural network is built based
on a NAR network [8]. The hidden layer comprises 200 neurons with fuzzy AFs. A linear AF is employed for the output layer.
Table 1
Input-output data set _
Input Output
x(1), x(2), x(3) x(d) x(2),x(3),x(4), ...,x(d+1) x(n-d-1), x(n-d), x(n-d+1), ..x(n-1) x(d+1) x(d+2) x(n)
Hidden layer Output layer
Fig. 2. Network structure
The training process is based on LevenbergMarquardt backpropagation procedure [8]. This training function is the fastest backpropagation-type algorithm. The Levenberg-Marquardt backpropagation algorithm was designed to approximate the second-order derivative with no need to compute the Hessian matrix, therefore increasing the training speed. For the training process, data are randomly divided with 70% used for training and 30% for testing.
Figure 3 shows the time-series response after training with our neural network when using ordinary.
By showing mean squared errors, Table 2 can confirms the accuracy of the prediction by each neural network model.
w
-0.1
50 100 150 200 250 300 350 400 450 500
Time
Fig. 3. Time-series response by neural networks using ordinary AF
1.5
u 1.4
£0
E-
CL
O 1.2
1.1 0.1
Momadbi TYCYPa, moM 19, № 4, 2016
Table 2
Mean squared errors by neural networks using ordinary and fuzzy AFs
Ordinary AF AF type a AF type b AF type c AF type d
26х10"5 28х10"5 19 х10"5 27 х10"5 25х10"5
From the test results it follows that the mean squared errors of predicted data are less than 5%. Therefore, the neural networks with fuzzy activation functions can be used for the problems of time series predictions with an acceptable level of accuracy.
The results also showed that neural networks with the fuzzy activation functions type b and type d can provide more precise results than those by neural network models with an ordinary activation function.
Conclusions
The paper shows that the model of second- type fuzzy neural neuron and fuzzy neural network can well solve the problems of time series predictions. Therefore, neural networks with fuzzy impulse activation functions may be commonly used in various fields, such as business, medicine, science, etc. for some problems of classification, prediction, approximation, etc. For the next studies, we will analyze the advantages of each fuzzy activation function and when to use them.
References
1. Puchkov E.V. Methodology of training recurrent artificial neural network with dynamic stack memory / E.V. Puchkov, V.B. Lila // Software products and systems. -2014. - № 4 (108). - PP. 132-135.
2. Beer M. Fuzzy Randomness: Uncertainty in Civil Engineering and Computational Mechanics / M. Beer, B. Moller. -New York: Springer-Verlag, 2004. - 307 p.
3. Yefremov A. A. On application of stepwise-continuous functions to setting the membership functions of (L-R)-type fuzzy numbers / A.A. Yefremov, A.M. Korikov // Siberian Journal of Science. - 2011. - № 1 (1). - PP. 340-343.
4. Yarushkina N.G. Lectures on Neuroinfomatics. - M.: Moscow Engineering Physics Institute, 2004. - PP. 151-199.
5. Fuller R. Neural fuzzy systems. - Abo Akademi University. - 1995. - 346 p.
6. Latypova R.R. Prediction regional dynamics with spatial relationships based on neural networks / R.R. Latypova, A.P. Kirpichnikov, A.S. Semeenko // Kazan Technological University Journal. - 2014. - № 15. - PP. 320-325.
7. Korikov A.M. Experiment in scientific research // Doklady Tomskogo gosudarstvennogo universiteta system upravleniya i radioelectroniki [Proceedings of Tomsk State University of Control Systems and Radioelectronics]. -2015. - № 2(36). - PP. 148-154.
8. An Application of Non-Linear Autoregressive Neural Networks to Predict Energy Consumption in Public Buildings / L.G.B. Ruiz, M.P. Cuellar, M.D. Calvo-Flores, M.D.C.P. Jimenez // Energies. - 2016. - № 9 (9). - 684 p.
Nguyen Anh Tu
Department of Automatics and Computer Systems, National Research Tomsk Polytechnic University Phone.: +7-952-157-48-78 E-mail: nguyenanhtu@tpu.ru
Korikov Anatoliy Mihailovich
Head of the Department of Automatic Control Systems, Tomsk State University of Control Systems and Radioelectronics;
Professor of the Department of Automatics and Computer Systems, National Research Tomsk Polytechnic University Phone.: +7 (382-2) 41-42-79 E-mail: korikov@asu.tusur.ru
Нгуен А.Т., Кориков А.М.
Модель нейронной сети с нечеткими функциями активации для прогнозирования временного ряда
Разрабатываются модели нейронов с нечеткими функциями активации для решения проблемы прогнозирования временных рядов. На основе моделей нейронов с нечеткими функциями активации создаются модели нечетких нейронных сетей (ННС) и проводится анализ возможностей ННС для решения задач прогнозирования временных рядов.
Ключевые слова: модель нейрона, функция принадлежности, функция активации, прогнозирование временных рядов.
Доклады ТУСУРа, том 19, № 4, 2016
