CC BY
59
UDC 519.766.4
Karayuz I. V.1, Bidyuk P. I.2
Postgraduate student, National Technical University of Ukraine «Kyiv Polytechnic Institute», Institute for Applied System Analysis,
E-mail: [email protected], Ukraine
2Doctor of Science, Professor, National Technical University of Ukraine «Kyiv Polytechnic Institute», Institute for Applied System
Analysis, Ukraine
SHORT-TERM FORECASTING OF COAL AND OIL PRODUCTION _IN UKRAINE_
In this study the problem of short-term forecasting for coal and crude oil production in Ukraine within the period of 2008-2012 is considered. Linear autoregressive and autoregressive moving average models as well as optimal filtering algorithm for linear systems (Kalman filter) based upon autoregressive model of second order were constructed for short term forecasting. The Kalman filter was successfully applied for generating optimal estimates of states and short-term forecasts. The state noise covariances were estimated recursively with new data coming what corresponds to the general ideology of adaptation. The best forecasting results for coal and crude oil production were received for autoregressive models with optimal filter and for ARMA models.
Keywords: Autoregressive model, autoregressive moving average model, Kalman filter, coal and crude oil production, short-term forecasting.
INTRODUCTION
Safety of Ukraine in the sphere of energy production highly depends on import of gas and oil. Production of coal, oil and gas in Ukraine itself, their transporting and use results in intensive contamination of environment. One of the first priority tasks is development and implementation of energy saving policy and maximum possible transition to non-traditional and renewable sources of energy. To solve effectively such problems it is necessary to develop and forecast fuel-and energy balance for Ukraine with taking into consideration production of coal and crude oil on Ukrainian territory. To increase quality of managerial decisions and risk management, quality of automatic control for engineering systems and technology it is necessary to develop and apply to solving practical problems new forecasting techniques directed towards further improvement of short- and medium term forecasting. Existing today forecasting methods that are based on various analytical procedures, logical rules and rational expert reasoning cannot provide in many cases desirable quality of forecasting results what requires from researchers new efforts to enhance quality of forecast estimates [1, 2]. Quality of the forecast estimates depends highly on quality of data itself, preliminary data processing directed on improvement of their statistical characteristics, correct application of structure and parameters estimation procedures as well as techniques of generating forecasts themselves. Very often substantial improvement of forecasts can be reached with optimal Kalman filter (KF) application that takes into consideration state and measurement noise covariances. KF algorithm also generates one-step ahead forecasts that are usually of acceptable quality.
Besides there exists a possibility for combining forecasts estimates generated by ideologically different techniques, what results very often in better forecasts than quality of
© Karayuz I. V., Bidyuk P. I., 2014
separate estimates generated by each technique [3]. At any rate such approach to forecasting may lead to substantial decreasing of forecasting errors variance if applied correctly.
PROBLEM STATEMENT
The purposes of the study are as follows: - to develop adaptive model development and forecasting system based on the modern system analysis approach; - to construct linear autoregression models with moving average (ARMA) for the process of coal and crude oil production and to compute short-term forecasts on their basis; - to apply Kalman filter (KF) for optimal estimation of the coal and crude oil production state and for short-term forecasting of the variables; - to perform comparative analysis of forecasts estimates computed with the models constructed and Kalman filter.
SOME MODERN FORECASTING TECHNIQUES
Adaptive Regression Analysis Approach. Correct application of modern modeling and adaptive estimation techniques, probabilistic and statistical data analysis provide a possibility for organizing computing process in such a way so that to get higher quality of forecast estimates in conditions of structural, parametric and statistical uncertainties. Such uncertainties arise due to availability of nonstationary and nonlinear process under study, incomplete data records, noisy measurements, extreme values and short samples. One of the possibilities for adaptation provide Kalman filtering techniques that generate optimal state estimates together with short term forecasts in conditions of influence of external stochastic disturbances and measurement noise. However, such techniques require estimates of statistical parameters for stochastic disturbances and measurement noise in real time what creates extra burden and errors for forecasting procedures.
DOI 10.15588/1607-3274-2014-1-9
61
We propose a concept of a model adaptation for dynamic processes forecasting based on modern system analysis ideas that supposes hierarchical approach to modeling and forecasting procedures, taking into consideration of possible structural, parametric and statistical uncertainties, adaptation of mathematical models to possible changes in the processes under study and the use of alternative parameter estimation techniques aiming to model and forecast estimates improvement. The functional layout of adaptive forecasting system is given in Fig. 1. Here each step of data processing is controlled by appropriate set of statistical parameters each of which characterizes specific features of data, model as a whole, model parameters and finally quality of the forecast estimates generated.
We propose the new adaptive scheme that is distinguished with several possibilities for adaptation using a complex quality criterion. The data collected should be correctly prepared for model structure and parameter estimation. The model structure estimation is a key element for reaching necessary quality of forecasts. It is proposed to define a model structure as follows:
S = {r, p, m, n, d, z, l},
where r is model dimensionality (number of equations); p is model order (maximum order of differential or difference equation in a model); m is a number of independent variables in the right hand side; n is a nonlinearity and its type; d is a lag or output reaction delay time; z is external disturbance and its type; l are possible restrictions for parameters and variables. For automatic search of the «best» model it is proposed to use the following criteria:
VN (6,Dn ) = e|1-R2 + ln(1 + SSE) + e|2-DW| +
+ ln(1 + MSE) + ln(MAPE) + eU , (1)
where 6 is a vector of model parameters; N is a power of time series (sample) used; R2 is a determination coefficient; DW is Durbin-Watson statistic; MSE is mean square error; MAPE is mean absolute percentage error; U is Theil coefficient. The power of the criterion was tested experimentally and proved with a wide set of models and statistical data. The criteria helps to search automatically for the best model in particular application.
There are several possibilities for adaptive model structure estimation (Fig. 1): 1) automatic analysis of partial autocorrelation for determining order of autoregression; 2) automatic search for the lags estimates of exogeneous variables (detection of leading indicators); 3) automatic analysis of residual properties (value of autocorrelation, type of distribution); 4) analysis of data distribution type and its use for selecting correct model estimation method; 5) adaptive model parameter estimation with hiring extra data;
7) optimal selection of weighting coefficients for exponential smoothing, nearest neighbor and some other techniques;
8) the use of adaptive approach to model type selection.
Data and knowledge base
Preliminary data processing: normalization filterings smoothing data imputation
Model structure estimation: correlation analysis^ order and lag estimation, description of nonlinearitv
a
Model parameters estimation: OLS= ML. RLS. RML. MCMC
IE
Derivation of forecasting function and Computing of short-term forecasts
V
Fig. 1. Adaptive model estimation and process forecasting procedure
The use of a specific adaptation scheme depends on the volume and quality of data, specific problem statement, requirements to forecast estimates, etc. In some cases we used successfully logistic regression together with linear regression to describe the data mathematically. These models as well as classification trees and Bayesian networks have been used successfully to forecast direction of stock price movement and some macroeconomic processes.
Application of the concept described provides the following advantages: 1) take into consideration some statistical and parametric uncertainties; 2) automatic search for the «best» model reduces the search time for many times; 3) it is possible to analyze much wider set of candidate models than manually; 4) the search is optimized thanks to the use of complex quality criterion; 5) derivation of forecasting functions on the basis of estimated AR and ARIMA models; 6) in the frames of computer system developed it is possible to integrate ideologically different methods of modeling and forecasting and compute combined forecasts estimates that are distinguished with better quality. Testing of the system with stock price and macroeconomic data showed that it is possible to reach a value of absolute percentage error of about 3-4 % for short term forecasting.
Kalman filtering. Kalman filtering algorithms could be easily hired for solving short term forecasting problems in the frames of adaptation procedure given above. The models constructed according to the adaptation scheme considered should be transformed into state space representation form that makes it possible further application of the Kalman type optimal filtering algorithm. An advantage of the approach is in the possibility of model adjusting to random external disturbances and taking into consideration possible measurement errors. In most cases of application such approach provides for high quality of short term forecasts thanks to availability of optimal state estimates computed by the filtering algorithm.
Bayesian networks. Bayesian networks (BN) or Bayesian belief networks are probabilistic models in the form of a directed graph the vertices of which represent selected variables, and arcs reflect existing cause and effect relations between the variables [4]. BN find quickly expanding applications in various areas of human activities such as computer based medical and engineering diagnostic systems, process forecasting, classification problems, risk management, and many others. BN provides a possibility for discovering existing dependences between variables and for determining new conditional probabilities for states and situations after receiving new information by any node of a graph. Success of application of the approach depends on correctness of a problem statement, appropriate variables selection, availability of necessary data and/or expert estimates for the structure and parameter learning. General problem statement touching upon application of Bayesian networks includes the following steps: 1) thorough studying of a process being modeled; 2) data and expert estimates collecting; 3) selection of known or development of a new method for model structure estimation (learning); 4) BN parameter learning (construction of conditional probability tables); 5) development of a new or selection of known inference method (final result); 6) testing the BN constructed using actual and generated data; 7) application of the model to practical problem solving, i.e. state forecasting, classification etc. In spite of the fact that general theory of BN has been developed quite well as of today, usually many questions arise when a specific practical problem is solved. This is especially true regarding the problems of forecasting because the quality requirements to forecasts estimates are continuously increasing what results in further refinement of computing methods and algorithms.
Group Method for Data Handling. The group method for data handling (GMDH) is a powerful modern instrument for process modeling and forecasting developed at the Ukrainian National Academy of Sciences (NAS) in the second half of last century by O. G. Ivakhnenko [5]. It generates the forecasting model in the form of the Kolmogorov-Gabor polynomial that could be used for describing linear and nonlinear systems. The main positive feature of the method is that it selects automatically the best model structure in the class of preselected linear or nonlinear structures. The latest versions of the fuzzy GMDH
techniques provide better possibilities for increasing the quality of forecasts estimates.
The problem statement for application of the technique should include the following elements: 1) selection of partial descriptions that create a basis for the possible final model;
2) selection and adaptation of the model parameters membership functions for a particular application;
3) development of a new or application of known model parameter estimation technique; 4) selection of a model quality criteria for the use at intermediate computation steps and for the final model selection. The models constructed with appropriately developed and tuned GMDH approach usually provide medium or high quality of short term forecasts.
Generalized Linear Models. Generalized linear models (GLM) is a class of models that extend the idea of linear modeling and forecasting to the cases when pure linear approach to establishing relations between process variables cannot be applied [6]. The GLM approach also extends the possibilities for mathematical modeling in cases when statistical data exhibit distribution different from normal. GLM constructing can be considered from classical statistics or a Bayesian perspective. Usually the problem statement regarding such type of model construction is touching upon the following elements: type of prior distribution for model parameters; a method for parameters estimation using appropriate simulations techniques; necessity for hierarchical modeling, posterior simulation etc. GLM could be successfully applied to solving the problems of classification and nonlinear process prediction. For example, they are used widely in scoring systems for predicting solvency of bank clients.
Combination of the forecasts. The problem of forecasts combination arises in the cases when one selected technique is not enough for achieving desirable quality of forecasting. In such cases it is necessary to select two or more ideologically different forecasting techniques and to compute combined estimate using appropriately selected weights. In a simple case equal weights are assigned to the individual forecasts. Other approaches to computing these weights are based on previously found prediction errors for each method or on optimization procedures. Especially good results of combination are achieved in the cases when the error variances for individual forecasting techniques do not differ substantially from each other.
THE MODELS CONSTRUCTED
To describe coal and oil production in Ukraine several models had been constructed and tested. The first one was
linear AP(p) model of the form:
p
y(k) = a0 aiy(k -i)+e(k), k=0,1,2,... i=l
where p is autoregression order; y(k) is a measurement at k-th moment of (discrete) time; e(k) - is a normal disturbance;
a, is i -th parameter of the model. The autoregressive moving average model (ARMA(^, q)) and ARIMA(^, d, q)) is defined as follows:
y(k) = a0 + £ aty(k - i) + £b]e{k - j) + e(k). i=1 j=0
where q is moving average order; ai, bj - model parameters.
The models developed have been used to construct forecasting functions allowing to generate multistep forecasts. As an example below is given forecasting function (for three steps) constructed for the process ARMA(2, 1):
y(k + 3) = Ek [ y(k + 3)] = ao + aE [ y(k + 2)] +
2 3
+ a2 Ek [ y(k +1)] = ao(1 + a1 + a1 + a2) + (a1 + 2aa )y(k) +
2 2 2 + (a1 a2 + a2 )y(k -1) + a2)e(k).
And recursive expression for arbitrary 5 steps forecasting can be written in the form:
y (k + 5) = Ek [ y(k + 5)] = ao + aE [ y(k + 5 -1)] + + a2 Ek [ y(k + 5 - 2)].
Kalmanfilter application. The linear Kalman filter was applied for computing optimal estimates of states and short-term forecasting based on ARMA-type models. The state filtering and forecasting process includes the steps given below.
Step 1. Formulation of mathematical model of the process under study:
x (k ) = F (k) x(k-1) + w (k -1);
:(k )= H (k )x(k )+v (k ),
y(k)
where x(k) = l ^'W is 2-dimensional state vector;
w Iy((k - 1)J
F (k ) =
ay a2 1 0
is state transition matrix for AR(2) model;
w (k) ~ N (o, Q(k)) is random disturbance that is supposed to be normal; H(k) is measurements matrix; v (k)~ N(o,R (k)) is a measurement (sensor) noise; initial state and respective covariances are as in the standard problem statement for optimal filtering:
E{xo}= xo, E{xo,xoT}= po = Po;
E {w(k), vT (k) }= o, E {w (k), xT (o) }= o, E {v (k), xT }= o.
Step 2. State extrapolation (one-step ahead projection of a state):
Step 3. Extrapolation for the estimation errors covariance matrix:
P'(k) = F (k )P(k -1) FT (k) + Q(( -1).
Step 4. Compute matrix filter gain:
K (k) = P' (k )H T (k ){h (k) P' (k) H T(k) + R (k )}r1.
Step 5. Optimal state estimation with taking into consideration the last measurement z(k):
x(k )= F (k) x(k -1) + K (k)[:?(() - H (k )x(k)].
Step 6. Compute errors covariance matrix for the next iteration:
P (k )= [I - K (k )H (k )]P' (k).
Step 7. Go to step 2.
MODELS CONSTRUCTION AND THEIR APPLICATIONS
To select the best models constructed the following
statistical criteria were used: determination coefficient (R ) with ideal value equal 1; Durbin-Watson statistic (DW) with ideal value equal 2; Fisher F-statistic; and Akaike information criterion (AIC). The forecasts quality was estimated with making use of the following criteria: mean squared error (MSE); mean absolute percentage error (MAPE), and Theil inequality coefficient (U) with its ideal value approaching zero.
The data characterizing coal and crude oil production in Ukraine were taken from the site of State Statistical Service of Ukraine [7]. The time graphs for the process are given in Fig. 2.
Thus, the best results of coal production forecasting (Table 1) were received for the models with Kalman filter (one- and two-step predictions), and three-step prediction was the best for ARMA(2,1). The graphic forms of the results are shown in Fig. 3.
Thus, the best result of one-step and two-step forecasting for crude oil production (Table 2) was received with Kalman filter + AR(2) model (with o,2% and 2,43% errors). The Kalman filter also showed the best value of MAPE = 2,97%. The graphic forms of results are shown in Fig. 4.
Further improvements of the forecasts were achieved with application of the adaption scheme given in Fig. 1 and complex quality criterion (1). An average improvement of the forecasts was in the range between o,5-1,2 %, what justifies advantages of the approach proposed. Combination of forecasts generated with different forecasting techniques helped to further decrease mean absolute percentage forecasting error for about o,3-o,8 % in this particular case.
X (k ) = F (k )X (k -1).
Fig. 2. Production of coal (left fig.) and crude oil (right fig.) in Ukraine, thousand tons
Fig. 3. Coal forecasting results in graphic form, thousand tons
Table 1. The model for coal production
Ready for use coal
Model q uality
Model type R2 2 e2(k ) DW
AR(2) 0,56 4346371 2,17
ARMA(2,1) 0,61 3955251 1,96
AR(2) + KF 0,56 4346371 2,17
Quality of predictions
Model type MSE MAPE U
AR(2) 4276086 4,85 0,039
ARMA(2,1) 3955251 4,58 0,037
AR(2) + KF 3959098 4,82 0,038
Three-steps forecasts (January, February, March 2013)
Model type 1 % 2 % 3 %
AR(2) 5502 9,23 5446 8,24 5480 0,94
ARMA(2,1) 5476 8,70 5480 8,92 5489 0,78
AR(2) + KF 5473 8,64 5423 7,79 5394 2,49
Data sample 5037 5031 5532
Table 2. Models for crude oil production
Crude oil
Model quality
Model type R2 2e2(k) DW
AR(2) 0,89 4243 1,94
ARM A( 2,1) 0,89 4220 1,96
AR(2) + KF 0,89 4243 1,94
Prediction quality
Model type MSE MAPE U
AR(2) 4058 2,92 0,028
ARMA(2,1) 4220 2,88 0,027
AR(2) + KF 4348 2,97 0,029
Three-steps forecasts (January, February, March 2013)
Model type 1 % 2 % 3 %
AR(2) 180 2,80 178 7,11 177 5,13
ARMA(2,1) 180 2,70 178 7,25 178 4,91
AR(2) + KF 185 0,20 183 10,24 182 2,43
Data sample 185 166 187
30
■26
-30
-•-Res ids AR{2)+FK —.......-.....
n « » ci I*. A A A £ Qi M
9S 99 3S fi
Fig. 4. Results for the crude oil production forecasting, thousand tons
CONCLUSIONS
The forecasting methodology based on application of adaptation scheme proposed, including structural and parametric adaption, proved to be useful for forecasting crude oil and coal production using the data for Ukraine. The methodology has also been applied successfully to forecasting macroeconomic processes. The basic positive features of the approach proposed are as follows: testing data quality with a set of statistical parameters; continuous analysis of data directed towards identification of forecasting model structure and its parameters; generation of candidate models and selection of the best one with another set of model quality parameters; and generation of a set of forecasts estimates on the basis of candidate models. The best forecast estimate is selected with a set of statistical forecasts quality parameters. Thus, the whole computational process is controlled with the three sets of quality parameters mentioned what guaranties acceptable quality of final result.
The linear Kalman filter was successfully applied for generating optimal estimated of states and short-term forecasts for the model selected. The state noise covariances were estimated recursively with new data coming what corresponds to the general ideology of adaptation. The best results of coal production forecasting were received for the models with Kalman filter (one- and two-step predictions), and three-step prediction was the best with ARMA(2,1). The best results for one-step and two-step forecasting for crude oil production was received with Kalman filter+AR(2) model (with 0,2 % and 2,43 % errors). The linear Kalman filter also showed the best value of MAPE = 2,97 %. Further improvements of the forecast estimates were achieved with application of the adaption scheme given in Fig. 1. An
average improvement of the forecasts was in the range between 0,5-1,2 %, what justifies advantages of the approach proposed. Combination of forecasts generated with different forecasting techniques helped to further decrease mean absolute percentage forecasting error for about 0,3-0,8 % in this particular case.
The future research should be directed towards expanding of the adaptive forecasting scheme with new methods for adaptive model parameters estimation, and alternative forecasting techniques based on intellectual data processing schemes.
SPISOK LTTERATURY
1. Shumway, R. H. Time Series Analysis and its Applications / R. H. Shumway, D. S. Staffer- New York: Springer Verlag, 2006. - 588 p.
2. Бiдюк, П. I. Аналiз часових рядiв / П. I. Бщюк, В. Д. Романенко, О. Л. Тимощук. - К. : Полтехнжа, НТУУ «КШ», 2013. - 601 с.
3. Harris, R. Applied Time Series Modelling and Forecasting / R. Harris, R. Sollis- West Sussex: John Wiley & Sons Ltd., 2005. - 313 p.
4. Jensen F. V., Bayesian Networks and Decision Graphs / F. V Jensen, Th. Nielsen. - New York: Spinger-Verlag, 2009. -457 p.
5. Згуровский, М. З. Основы вычислительного интеллекта: монография / М. З. Згуровский, Ю. П. Зайченко ; НАН Украины, ИПСА НТУУ «КПИ». - К. : Наукова думка, 2013. - 406 с.
6. Dobson, A. An Introduction to Generalized Linear Models / A. Dobson. - New York : CRC Press Company, 2013. - 407 p.
7. Державна служба статистики Украши [Електронний ресурс]. - Електрон. даш. - Режим доступу: http:// www.ukrstat.gov.ua/, вшьний. - Заголовок з екрану. - мова укр., рос., англ.
Стаття надшшла до редакцй 1.04.2014.
Караюз I. В.1, Бщюк П. I.2
'Астрантка, 1нститут прикладного системного аналiзу НТУУ «КШ», Укра!на
2 Д-р техн. наук, професор, 1нститут прикладного системного аналiзу НТУУ «КШ», Украша
КОРОТКОСТРОКОВЕ ПРОГНОЗУВАННЯ ПРОЦЕС1В ВИРОБНИЦТВА ВУГ1ЛЛЯ I НАФТИ В УКРА1Н1
У статп розглядаеться задача короткострокового прогнозування виробництва вугшля та нафти в Укра!ш у часовому дiапа-зош 2008-2012 рр. Для розв'язання задачi короткострокового прогнозування побудовано моделi авторегресй, авторегресй з ковзним середшм, а також використано алгоритм фшьтрацй лшшних систем (фшьтр Калмана), що грунтуеться на моделi авторегресй другого порядку. Фшьтр Калмана устшно використано для оптимального ощнювання сташв та короткострокового прогнозування. При цьому коварiащ'! збурень сташв ощнювались рекурсивно з надходженням нових даних, що узгоджуеться iз загальною щеолопею адаптацй. Кращi результати прогнозування виробництва вугшля i сиро! нафти отримано за допомогою оптимального фшьтра (на основi авторегресй) i моделi авторегресй з ковзним середшм.
Ключовi слова: авторегреая, авторегреая з ковзним середшм, фшьтр Калмана, видобуток вугшля i сиро! нафти, коротко-строкове прогнозування.
Караюз И. В.1, Бидюк П. И.2
'Аспирантка, Институт прикладного системного анализа НТУУ «КПИ», Украина
2Д-р техн. наук, профессор, Институт прикладного системного анализа НТУУ «КПИ», Украина
КРАТКОСРОЧНОЕ ПРОГНОЗИРОВАНИЕ ПРОЦЕСОВ ПРОИЗВОДСТВА УГЛЯ И НЕФТИ В УКРАИНЕ
В статье рассматривается задача краткосрочного прогнозирования производства угля и нефти в Украине во временном диапазоне 2008-2012 гг. Для решения задачи краткосрочного прогнозирования построены модели авторегрессии, авторегрессии со скользящим средним, а также использован алгоритм фильтрации линейных систем (фильтр Калмана), основанный на модели авторегрессии второго порядка. Фильтр Калмана успешно использован для оптимального оценивания состояний и
краткосрочного прогнозирования. При этом ковариации возмущений состояний оценивались рекурсивно с поступлением новых данных, что согласуется с общей идеологией адаптации. Лучшие результаты прогнозирования производства угля и сырой нефти получены с помощью оптимального фильтра (на основе авторегрессии) и модели авторегрессии со скользящим средним.
Ключевые слова: авторегрессия, авторегрессия со скользящим средним, фильтр Калмана, добыча угля и сырой нефти, краткосрочное прогнозирование.
REFERENCES
1. Shumway R. H., Stoffer D. S. Time Series Analysis and its Applications. New York, Springer Verlag, 2006, 588 p.
2. Bidyuk P. I., Romanenko V. D., Timoshchuk O. L. Time Series Analysis. Kyiv, Polytechnika, NTUU «KPI», 2013, 601 p.
3. Harris R., Sollis R. Applied Time Series Modelling and Forecasting, West Sussex, John Wiley & Sons Ltd., 2005, 313 p.
4. Jensen F. V., Nielsen Th. Bayesian Networks and Decision Graphs. New York, Spinger-Verlag, 2009, 457 p.
5. Zgurovsky M. Z., Zaichenko Yu. P. Basics of Computing Intelligence. Kyiv, Naukova Dumka, 2013, 406 p.
6. Dobson A. An Introduction to Generalized Linear Models. New York, CRC Press Company, 2013, 407 p.
7. The State Statistics Service of Ukraine. - Electronic data. -Available from: http://www.ukrstat.gov.ua/, free, Title from the screen, Lang. ukr, rus, eng.
