CC BY
3Prognostic Maps of Climatic Indicators Based on a Multivariate Neural Network
VM. Berdnikov* N.V. Abasov Melentiev Energy Systems Institute of Siberian Branch of Russian Academy of Sciences, Irkutsk, Russia
Abstract — The paper is concerned with the construction of maps of climatic indicators using a multivariate neural network. The application of the k-means clustering method for processing input data entering a neural network is described. An alternative neural model is considered within the framework of the multivariate neural network evolution.
Index Terms: neural networks, predictive maps, clustering, MNN, k-means, forecasting.
I. Introduction
Predictive estimates of water inflow significantly affect the operating conditions of hydroelectric power plants associated with the alternation of extreme high-water and low-water years, which is especially important to factor in for such objects as Lake Baikal. The inflow of water into the reservoir is a determining factor for the operation of hydropower plants (HPPs). Hydropower accounts for 80% of the total electricity generated in the Irkutsk region. Efficient operation and planning of the power system requires long-term (from 3 months to several years) forecasting of the inflow into the reservoir [1-12].
A long-term forecast however cannot be guaranteed, only probabilistic estimates can be considered. One of the objectives of this research is to increase the probability of predictive estimates.
Over the long history of climate observations, various patterns of building the interrelated climatic indicators have been identified, and various methods of forecasting natural processes have been developed on their basis. Melentiev Energy Systems Institute of Siberian Branch of the Russian Academy of Sciences is engaged in developing an area of long-term forecasting associated
* Corresponding author. E-mail: berdVM98@gmail.com
http://dx.doi.org/10.38028/esr.2022.04.0003
Received November 16, 2022. Revised November 21, 2022.
Accepted December 03, 2022. Available online January 30, 2022.
This is an open access article under a Creative Commons Attribution-NonCommercial 4.0 International License.
© 2022 ESI SB RAS and authors. All rights reserved.
with the construction of powerful hydroelectric power plants on the Angara and Yenisei rivers. However, due to the global climate changes observed over the past century, the results obtained with the methods relying on patterns may differ from actual indicators. The very task of long-term forecasting is difficult by definition and the developed global models cannot give a guaranteed result due to the incompleteness of available information and insufficient knowledge of natural processes. The Institute has developed the GeoGIPSAR system for the long-term predictive estimation of water inflow and the climatic situation, which relies on probabilistic, approximative, and other methods for long-term estimation. Recently, neural network approaches have been used to make a long-term forecast [13-15] considering the global climate changes. A multivariate neural network (MNN) with a variety of settings (changing the number of hidden layers and a set of neurons in a layer, the type of activation function in a neuron, etc.) has been developed as a separate component of the GeoGIPSAR system to build various models in different forms (from numerical indicators to interval estimates) [13]. The MNN factors in the climate change by searching for the most correlated predictors and generating interval estimates for a forecast period. The interval method involves dividing the entire observation range into intervals and calculating the probability that the value will fall into each interval when calculating the predictive estimate. Interval estimates can increase the probability of predictive indicators, but are not sufficient to obtain guaranteed results. To increase the probability, the paper presents an approach to the creation of predictive climate maps for precipitation, temperature and other conditions to eliminate disagreement with interval estimates and other methods, for example, forecasting based on global climate models (CFSv2) [11, 12, 15, 16].
II. MNN-Based Prediction of the Climatic Situation
The diagram of MNN use to generate predictive estimates of the climatic situation is shown in Fig. 1.
The input data for this approach is a set of predictors in the form of a matrix of daily indicators of geoclimatic data for different periods. The output is prognostic maps of the studied region in the form of spatial distributions of
DB temporary ranks
Fig. 1. The diagram of the MNN use for the generation of predictive estimates.
Calculate the
Input data matrices Calculate initial centroids of clusters measure of proximity of clusters to Form new Complete
cluster groups clustering
centroids
i
Fig. 2. A diagram of the k-means clustering algorithm.
meteorological indicators. A neural network with training and verification is an internal mechanism of the presented diagram. The criterion for the acceptability of the result in the verification sample is the following condition
2
XXj yj [t ] - rj [t ])
t =tj i, j
( t2 - t + 1)-X c
< e
,(1)
speed. The principle of this method is to minimize the total quadratic deviation of clusters points from their centers
m
= argmin x( x- )2 vi =
(2)
where c, £ [0,1] is weighting coefficients of the weights
of the significance of various sections of the studied area,
which are set by the expert;
t is a discrete time period from t1 to t2;
y, is a predictive indicator for coordinates i, j;
r- is an actual indicator for coordinates i, j;
e is a permissible average error of the prognostic values
relative to the actual ones.
At each point, the forecast is compared with the fact. The formula is invariant with respect to the sample length. It allows building various neural network models that meet this criterion.
In fact, high or low water content is determined by a very few standard maps whose prediction can give a better result than numerical prediction of all indicators of grid data. One can distinguish N-types of maps from the set of all accumulated data on the selected meteorological indicator and reduce forecasting to the calculation of probability of belonging to these maps. To do this, it is necessary to perform cluster analysis by partitioning the accumulated data matrices into N clusters. To this end, the k-means method is used. [18-24].
The k-means method is a fairly popular clustering method due to its ease of implementation and high processing
where k is a cluster index; Vj is a cluster centroid; x, is a data matrix element.
The algorithm divides the complete set of data matrices into clusters (Fig. 2), which are the closest to the cluster's center, and clustering itself occurs due to the rearrangement of clusters, which involves:
1. Selecting a multidimensional space matrix for initial clustering;
2. Calculating the initial position of the centroid for each cluster;
3. Calculating a measure of proximity of the data matrices to the cluster centroids and assigning the cluster number to the matrix;
4. Repeating the cycle after new clusters are formed by changing the centroid;
5. Completing the clustering if the clusters do not change during the next cycle.
The performance of the method depends on the high sensitivity to noise and the dependence on the initial choice of cluster centers.
III. MNN-Based Methodology for Predicting Cluster Weights
In contrast to the generation of interval estimates for predicting a scalar value, for example, the inflow of a reservoir for a selected period of time, the use of cluster partitioning allows obtaining the probabilities of belonging to clusters.
h
Fig. 3. The scheme of the convolutional neural network.
The use of MNN with the cluster method of distribution of weights involves assigning the value of a specific cluster to the forecast, thereby increasing the forecast accuracy. Whereas on the training sample, the values of the cluster are assigned to the probability of belonging to a particular cluster: 1 - belongs and 0 - does not belong, on the verification sample, the probabilities of the trained network are compared to the actual values of clusters. Acceptable results are considered to be the minimum deviation of prognostic indicators from actual ones according to the following criteria:
1 T N
- lEu, - fkA£ s ,(3)
T t=1 k=1
where T is the length of the verification sample; 5 is an average error across the entire verification sample; ykJ is the weight of belonging to predictive clusters; fkJ is weights of the actual data belonging to clusters.
The very process of the MNN operation to find out the membership to clusters is similar to the method of generating interval estimates. The only difference is that in the cluster method, the final value belongs to a specific cluster, whereas in the interval method, the value is only a set of probabilities of falling into a specific interval.
IV. Description of Software Tools for the implementation of the MNN-Based Cluster Assessment
The software components of the presented technology are implemented in the LuaISEM language (the standard Lua language with a set of libraries implemented at ESI). All the necessary input data and settings of the neural model are recorded in a single configuration file (SCF), which is subsequently input into the neural network model.
The MNN core is a multilayer perceptron with an inverse error propagation method. This model is chosen because the identification of all explicit and implicit patterns is a laborious process and sometimes impossible. This model solves this problem at a basic level and shows
decent results. The multilayer perceptron is the simplest model of a neural network capable of performing tasks of small complexity. The inverse method of error propagation in the neural model identifies erroneous results of neurons and then decides on the reliability of intermediate results, while training the neural model "without a teacher" allows one to avoid interfering in the course of the MNN operation and is due to the unknown "correct answer."
The output matrices can be transformed into predictive maps by both using the Gnuplot graphical editor and superimposing the received data in the form of various layers in GIS systems (for example, Google Earth).
V. Prospects for the MNN Development
The process of the MNN operation is related to the use of a multilayer perceptron model. It has proven itself to be simple in the neural network model implementation, its setting, and showed the efficiency of calculations. In order to develop the MNN, the possibility of switching from the multilayer perceptron model to the convolutional neural network (CNN) model is considered [13, 14, 25-27].
The idea of convolutional neural networks comes from biology, namely from the visual cortex of the brain. The principle behind the model operation is the use of special (convolution) layers, using customizable filters, and pooling layers to identify the key features of the object. The CNN algorithm is a sequential application of convolution layers, where data is convoluted, and pooling layers, in which data is compressed. Further, the important features are identified with the algorithm and the feature vector is subsequently fed into a fully connected classification module, where it is assigned a class of the object under study. The larger the number of the consecutive convolution and pooling layers, the larger the output vector. With an excessively large number of convolution layers, the process of "retraining" the network occurs, and the resulting vector becomes incorrect. The process of CNN operation is schematically
shown in Fig. 3.
The convolution operation itself is a sequential multiplication of an element of the original input values by an element of the receptive field filter matrix, which results in a matrix of new values, but of a smaller dimension. Each convolution layer generates an output matrix. If 4 filters are used in a layer, then the output matrix will consist of 4 layers. Each convolution layer forms more and more detailed features.
The pooling layer works on the principle of a convolutional layer, but with a view to reducing the size of the collapsed object in space. This must be done to reduce the computing power used in data processing. This process is useful for extracting dominant features. Two types of associations are distinguished: maximum and average. The first type returns the maximum value of the values covered by the receptive field, and the second type returns the average value. The maximum pooling also performs the function of noise reduction by discarding noisy activation values and eliminates noise by reducing dimensionality. The average pooling, in turn, simply reduces the dimension to suppress noise.
The input data for the convolutional neural network was a large database of accumulated data on the water inflow into Lake Baikal, as well as databases of various geoclimatic indicators. Further, the data for a particular year were synthesized into a single vector, which was transmitted to the convolutional neural network.
VI. Conclusion
The presented method of predicting spatial distributions of meteorological indicators (precipitation, temperatures in the catchment basins of hydroelectric power plant reservoirs) increases the reliability of predictive scenarios of water inflows into hydroelectric power plant reservoirs.
The effective refinement of the methodology requires a lot of research on the choice of predictors and acceptable estimates on verification samples, which is planned in the future.
The individual methods of convolutional neural networks are also considered promising to factor in the specific features of the influence of various areas of the data matrices used to increase the reliability of predictive estimates.
Acknowledgement
The research was carried out under State Assignment Project no. FWEU-2021-0003. reg. number AAAA-A21-121012090014-5 of the Fundamental Research Program of the Russian Federation 2021-2025.
References
[1] N. V. Abasov, T. V. Berezhnykh, V. M. Nikitin, E. N. Osipchuk, "Methodology for the formation of long-term prognostic scenarios of water content and temperature regimes," in Proceedings of
the international conference «Energy of the 21st century: Sustainable Development and Intelligent Management» (September 7-11, 2020), Irkutsk, Russia, 2020, pp. 100-101.
[2] N. V. Abasov, V M. Nikitin, E. N. Osipchuk, "A System of Models to Study Long-Term Operation of Hydropower Plants in the Angara Cascade," Energy Systems Research, vol. 2, no. 2(6), pp. 5-18, 2019.
[3] A. P. Reznikov, Prediction of natural processes by the learning system (physical, informational, methodological aspects). Novosibirsk, Russia: Nauka, 1982. (In Russian)
[4] N. V Abasov, T. V. Berezhnykh, A. P. Reznikov, "Long-term forecasting of environmental factors in the energy sector," in Systematic studies of energy problems. Novosibirsk, Russia: Nauka, 2000, pp. 415-429.
[5] N. V. Abasov, T. V. Berezhnykh, A. P. Reznikov, "Long-term forecast of environmental factors of energy in the information and prognostic system GIPSAR," Proceedings of RAS. Power Engineering, no. 6, pp. 22-30, 2000. (In Russian)
[6] I. P. Druzhinin, A. P. Reznikov, T. V. Berezhnykh, N. V. Abasov, "The state and prospects of the problem of long-term forecasting of natural processes," Problems of Regional Ecology, no. 2, pp. 7-15, 2001. (In Russian)
[7] N. V. Abasov, "Fundamentals of the universal programming environment ZIRUS," Bulletin of INRTU, no. 26, pp. 62-68, 2006. (In Russian)
[8] N. V. Abasov, V. V. Vetrova, "Climate data processing technology," in Proceedings of the XII Baikal All-Russian Conference "Information and mathematical technologies in science and management. " Part II, Irkutsk, Russia, 2007, pp. 85-91. (In Russian)
[9] N. V. Abasov, "From programming languages to program models," in Proceedings of theXIIBaikal All-Russian Conference "Information and mathematical technologies in science and management." Part III, Irkutsk, Russia, 2007, pp. 140-147. (In Russian)
[10] N. V. Abasov, "On maintaining data integrity in long-term forecasting tasks," in Proceedings of the XIII Baikal All-Russian Conference "Information and mathematical technologies in science and management." Vol. 1. Irkutsk, Russia, 2008, pp. 184190. (In Russian)
[11] N. V. Abasov, About the development of the predictive system GIPSAR based on Lua components
// Proceedings of XVI Baikal All-Russian Conference "Information and mathematical technologies in science and management." Vol. 3. Irkutsk, Russia, 2011, pp. 212-217. (In Russian)
[12] N. V Abasov, T. V Berezhnykh, V. V. Vetrova, "Long-term forecasting of the hydropower potential of the HPP cascade in the conditions of climate change," Proceedings of RAS. Power Engineering, no.1, pp. 49-57, 2012. (In Russian)
[13] K. Benidis, S. S. Rangapuram, V. Flunkert, B. Wang,
D. Maddix, C. Turkmen, J. Gasthaus, M. Bohlke-Schneider, D. Salinas, L. Stella L., et al, "Neural Forecasting: Introduction and Literature Overview," arXivpreprint arXiv: 2004.10240, 2020.
[14] A. Ghosh, A. Sufian, F. Sultana, A. Chakrabarti, D. De, "Fundamental Concepts of Convolutional Neural Network," in Recent Trends and Advances in Artificial Intelligence and Internet of Things. Springer: Cham, Switzerland, 2020, pp. 519-567.
[15] N. V Abasov, V M. Nikitin, T. V Berezhnykh, E. N. Osipchuk, "Monitoring and Predictive Estimations of Atmospheric Parameters in the Catchment Area of Lake Baikal," Atmosphere, vol. 13, no. 1, Art. 49, 2022. DOI: 10.3390/atmos13010049.
[16] S. Saha, S. Moorthi, Pan Hu, "The NCEP Climate Forecast System Reanalysis," American Meteorological Society, vol. 8, pp. 1015-1057, 2010.
[17] V. Berdnikov, "Generation of prognostic interval estimates of water inflows to hydroelectric reservoirs using multiparametric neural network," E3S Web Conf., vol. 289, Art. 01003, 2021.
[18] L. H. Gitis, Cluster analysis in classification, optimization and forecasting tasks. Moscow, Russia: Publishing House of Moscow State Mining University, 2001, 157 p.
[19] K-Means Algorithm. Available: http://www. datascientist.one/k-means-algorithm/. Accessed on May 06, 2022. (In Russian)
[20] 2.3 Clustering. Available: http://www.scikit-learn.ru/ clustering/. Accessed on May 06, 2022. (In Russian)
[21] K-means algorithm. Available: http://www.algowiki-project.org/ru/Algorythm_k-means. Accessed on May 06, 2022.
[22] K-means algorithm. Available: http://www. helenkapatsa.ru/mietod-k-sriednikh/. Accessed on May 06, 2022. (In Russian)
[23] Lecture 14: Methods of cluster analysis. Iterative methods. Available: http://www.intuit.ru/studies/ courses/6/6/lecture/184?page=1. Accessed on May 06, 2022. (In Russian)
[24] K-means algorithm (1). Available: http://www. russianblogs.com/article/3114832336/. Accessed on May 06, 2022. (In Russian)
[25] Convolutional Neural Networks, Explained. Available: http://www.towardsdatascience.com/convolutional-neural-networks-explained-9cc5188c4939. Accessed on May 08, 2022.
[26] Convolutional neural networks. Available: http://www. neerc.ifmo.ru/wiki/index.php?title=Convolutional neural network. Accessed on May 08, 2022.
[27] Convolutional Neural Network. Available: http:// www.towardsdatascience.com/covolutional-neural-network-cb0883dd6529. Accessed on May 08, 2022.
Vladislav Berdnikov graduated from Irkutsk State Transport University with a degree in Information Systems and Technologies. Currently, he is a postgraduate student majoring in energy systems and complexes and a parttime research engineer at the Melentiev Energy Systems Institute of SB RAS. His research interests include the introduction of neural network processes into the methods and methodologies for improving the efficiency of hydroelectric power plants in the context of a changing climate.
t Nikolai Abasov graduated from Moscow Physical Technical Institute in 1978. Currently he is a senior engineer at the Melentiev Energy Systems Institute SB RAS. His research interests include the development of new techniques and methodologies to improve the efficiency of hydroelectric system in the context of a changing climate.
