ROBUST ESTIMATION OF THE AREA OF ADEQUACY OF FORECASTING ONEPARAMETER MODEL OF EXPONENTIAL SMOOTHING Текст научной статьи по специальности «Компьютерные и информационные науки»

-□ □-

Розглянуто задачу параметричного синтезу прогноз-ног однопараметричног моделi експоненщального згладжу-вання для предиктивного ощнювання значень показнитв органюацшно-техтчног системи. Для видыення ттерва-лiв заданог якостi на областi допустимих значень внутргш-нього параметра обраний критерт абсолютног похибки множинного прогнозу. Його використання дозволило сфор-мувати аналгтичну ретроспективну модель з «м'якими» обмеженнями. В результатi розроблений метод робаст-ного оцтювання областi адекватностi прогнозног однопараметричног моделi експоненщального згладжування, який дозволяв аналгтично ощнювати межi областi адек-ватностi прогнозног моделi в залежностг вгд вимог до гг ретроспективног точностi. Запропонований метод дав можливкть користувачевi задавати набiр допустимих ретроспективних похибок в залежностi вiд вимог техтчно-го завдання на прогнозування. Запропонований метод може бути використаний для параметричного налаштування однопараметричних прогнозних моделей i служить инструментом пгдтримки прийняття ршень в процеп прогнозу-вання. Результата моделювання являють собою ттер-вальт ощнки, використання яких в процеп параметричного синтезу краще точкових. На вiдмiну вiд пошукових методiв, аналгтична форма ретроспективних залежностей дозволяв отримувати ршення з високою точтстю i при необхiд-ностi надав аналгтику можливостi для графгчного аналюу областi адекватностi моделi. На прикладi показаний фрагмент ощнювання динамгки часового ряду при ретроспективному аналШ глибиною в три значення i заданих гранич-них вгдносних похибках в 1-4 %. За таких умов область для обгрунтованого вибору настроювального параметра визна-чавться об'вднаними гнтервалами шириною близько 20 % вiд початковог областi допустимих значень

Ключовi слова: експоненщальне згладжування, тверс-на верифжащя, адекваттсть прогнозног моделi, робастне

ттервальне ощнювання -□ □-

UDC 658.5:004.94

|DOI: 10.15587/1729-4061.2020.205843|

ROBUST ESTIMATION OF THE AREA OF ADEQUACY OF FORECASTING ONE-PARAMETER MODEL OF EXPONENTIAL SMOOTHING

Y. Romanenkov

Doctor of Technical Sciences, Professor* Е-mail: [email protected] Y. Pronchakov PhD, Dean

Faculty of Software Engineering and Business** Е-mail: [email protected] T. Z i e i n i i e v

PhD, Associate Professor* Е-mail: [email protected] *Department of Management** **National Aerospace University Kharkiv Aviation Institute Chkalov8 str., 17, Kharkiv, Ukraine, 61070

Received date 20.05.2020 Copyright © 2020, Y. Romanenkov, Y. Pronchakov, T. Zieiniiev

Accepted date 17.06.2020 This is an open access article under the CC BY license

Published date 30.06.2020 (http://creativecommons.org/licenses/by/4.0)

1. Introduction

The degree of informatization of the management decision-making process inevitably increases with the growth of the volume of various data that directly or indirectly characterize the state of modern organizational and technical systems. They are accumulated, as a rule, in the form of short time series of indicators with rheonomic constraints. Therefore, to estimate the dynamics of such parameters, simple, one- or two-parameter forecasting models are most often used.

The desire to use this kind of information is due to high requirements for the quality and efficiency of management decisions. Management has actually become a proactive tool, rather than a response to factors and trends. Under these conditions, the role of short-term forecasting or evaluating critical parameters increases dramatically, since the correct and flexible application of forecasting methods becomes a competitive advantage in the commercial struggle.

The current level of development of forecasting support [1] of the management decision-making process provides the analyst with a large range of forecasting models [2]

for solving practical problems. At the same time, the role of the analyst or other decision-maker remains crucial, since after choosing a model, it should be configured and verified. Therefore, each analyst chooses forecasting tools based on his experience, professional preferences, and sometimes even corporate traditions. At the stage of selecting and adjusting the forecasting model, he can and must evaluate its adequacy, guided by the classical postulates of mathematical modeling and statistics in particular.

To ensure the accuracy and reliability of forecasting results, it is necessary to check the adequacy or verify the forecasting model used [3]. When checking the adequacy of the forecasting model, as for any mathematical model, it is enough to make sure that two properties are met: accuracy and consistency [4]. In the case of verification by single parameter values, the requirement of consistency no longer exists, and adequacy becomes equivalent to correctness, which is the only one that should be evaluated [5].

Thus, if regular and repeated short-term forecasting is needed, models and methods that allow parameter adjustment of forecasting models are required. Requirements for

such models and methods can be adaptability, parameter invariance, robustness.

2. Literature review and problem statement

The number of practical problems that use the one-parameter model of exponential smoothing is extensive [6]. Even when this model is used to solve behavioral problems [7], the question of adequacy assessment and parameter adjustment remains relevant.

To adjust the forecasting one-parameter model of exponential smoothing, there are many recommendations in the literature regarding the selection of an internal parameter [8] (Fig. 1).

Table 1

Verification methods for forecasting models [14]

a

0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

classical admissible set Kc = {a: 0<a< 1}

out-of-limit admissible set

Kout = {a : 1 <a< 2}

extended admissible set

Kext = Kc u Kout = {«: 0<a< 2}

Fig. 1. Recommendations for selecting smoothing parameter values (based on [8])

Given the fact that the modern forecasting process provides for the implementation in automatic mode [9], including as part of decision support systems [10], implementation of the interactive modeling mode requires special attention. This aspect becomes especially important if forecasting procedures are applied to samples from Big Data storages [11]. In such cases, hybrid methods are widely used, which, however, also require parameter adjustment [12].

To assess the adequacy of statistical forecasting methods, inverse verification is most often used (Table 1), that is, checking the adequacy of the forecasting model and object in the retrospective period. It is based on the following rule: the proposed model can be applied for long-term forecasting if it gives adequate results in a retrospective evaluation of the characteristics of an already occurred event. In this case, the absolute retrospective verification of the already occurred event serves as a confirmation of the correctness of the chosen model, its parameters and a method of relative verification for predicting future events [13].

Inverse verification of the forecasting one-parameter exponential smoothing model can be implemented in two basic ways. The direct type of problem solving includes search procedures. They allow obtaining a bundle of retrospective forecasting values by changing the value of the internal parameter with a certain step [15]. The closest one to the real value is chosen from them [16].

Verification method Verification technology

Direct verification Development of a model of the same object using a different forecasting method

Indirect verification Comparison of results from this model with data from other sources

Consequent verification Verification of simulation results by analytical or logical derivation of the forecast from previous forecasts

Opponent verification Verification by refuting opponent's criticisms on the forecast

Expert verification Comparison of forecasting results with expert opinion

Inverse verification Checking the adequacy of the forecasting model and object in the retrospective period

Partial target verification Construction of conditional submodels equivalent to the full model in typical situations for the designed system

Structural verification Comparison of structures without experimental verification of the comparison in general

An analytical approach to solving the forecasting problem is probably more informative. Various aspects of it are described, for example, in [1]. The main idea is to form algebraic retrospective equations, whose real roots determine the optimal values of the smoothing parameter.

However, the issues related to the analysis of a system of several retrospective equations rather than a single one remained unresolved. The practical interest in such a problem is due to the fact that in most cases, the coefficients of pair correlation between sequences of forecasting and actual values are used to measure forecasting accuracy [17, 18]. Moreover, real time series may contain data of different reliability or significance [19].

Therefore, it is necessary to develop methods that analytically solve the inverse forecasting problems taking into account the requirements for a sequence of retrospective errors. When solving such problems, interval analysis [20], which has proven itself well in robust estimation problems [21], is a promising tool. At the same time, robustness is considered in two aspects. The first is the resistance of statistical procedures to outliers [22], the second is parameter robustness, that is, resistance to changes in internal parameters [23]. The advantage over statistical adaptation methods [24] in this case is the absolute accuracy of analytical retrospective dependencies.

3. The aim and objectives of the study

The aim of the study is to identify intervals of a given quality in the range of admissible values of the internal parameter for the parameter synthesis of the forecasting model.

To achieve this aim, the following objectives should be accomplished:

- to select quality criteria for the forecasting model;

- to form a retrospective model of one-parameter exponential smoothing in accordance with the inverse verification technology;

- to develop a method for robust estimation of the adequacy area of the one-parameter exponential smoothing model;

- to illustrate the application of the developed method on test data.

4. Selection of quality criteria for the forecasting model

The literature mentions that «at present there is no sufficiently complete study of all possible accuracy criteria, which makes it difficult to assess the capabilities of various models and experience of their application in applied works on forecasting specific processes» [17]. However, the use of a set of statistical quality criteria for forecasting models is considered classic [25] (Table 2).

Table 2

Classification of quality criteria for forecasting models

Criterion Expression Characteristic

1 2 3

Absolute error e, = y,- V, Internal, absolute

Relative error 8 = y' - y' .100%% , y, Internal, absolute

Maximum absolute error emax = max .....»! ' Internal, absolute

Sum of squared errors SSE = £e,2 '=i Internal, absolute

Root mean square error c = \n ti Internal, absolute

Mean squared error MSE =1 Ye2 nJk Internal, absolute

Mean absolute deviation 1 » MAD =1Y N Internal, absolute

Percentage mean absolute deviation 1 » MAPE=» IN Internal, absolute

General form of comparative accuracy criterion K = i Y(( - y ')2 =1 Z (( - y ' )2 =1 Internal, comparative

Divergence coefficient KH = 1 Z(( - y')2 '=1 ±y'2 =1 Internal, comparative

Modification of divergence coefficient KH, = 1 Z(( - y ')2 =1 t(( - y ')2 Internal, comparative

Sample correlation coefficient t (( - y )( y - y 2 f = '=1 Internal, comparative

Jt (y, -y )2 -Jy ( y- v )2

Continuation of the Table 2

1 2 3

Determination coefficient Y(( - y)2 t(y.- y 22 =1 Internal, comparative

Theil's inequality coefficient t ((V.U = JlnÎL f» Y V 2 - V 22 1 ±yt 2 » 1=1 Internal, comparative

Modulo mean relative error A = i Y » =1 y ' - y ' y -100% Internal, comparative

Displacement share VM = (( - y 2 »Y ( - y'2 Internal, qualitative

Variation share VS (SV - Sv 22 1Y (( - y'22 Internal, qualitative

Covariance share VC 2(1 - f2- Sf ■ SV » Y(( - V 22 Internal, qualitative

Regularity criterion t(( - y .22 a-(b2=- y y, ieS External

Minimum displacement criterion t (vA - VB 22 »2 = ^- tV'2 =1 External

Short-term forecasting accuracy criterion t(( - y'22 ä'(C 2=" t y- teC External

Step-by-step integration accuracy (convergence) criterion Z(( - y'22 12 (N 2= « ZV' 2 =1 External

Variable balance criterion B2 = Z =1 - 12 I2 y' -Zy'j _ j=1 _ External

It should be noted that the number of combined criteria, as well as variations of their sequential application, exceeds the number of classical ones [18].

The chosen criterion should take into account the features of a specific forecasting problem, such as the forecasting horizon, available sample size, degree of data «purity», etc.

Despite the variety of statistical characteristics and the corresponding criteria, most of them are in some form derived from the magnitude of the forecasting error:

ei = y,- y,. (1)

This suggests that if the short-term forecasting technology uses a criterion of the form (1), then its methodological and algorithmic support can be reconfigured to use any other criterion from Table 1.

5. Retrospective model of one-parameter exponential smoothing according to inverse verification technology

Consider the one-parameter exponential smoothing model (Brown model) [26]:

yt = ayt_1 + a (1 -a)yt-2 +... + a ( - a)"-1 yt_n =

n

= £a(1 -a)'-1 yt -„ (2)

i=1

where yt is the estimate (forecast) of the observed indicator for time point t; yt-1, yt-2, •, yt-n are the values of the series at the corresponding time points; n is the sample length of the time series; a is the smoothing parameter.

Using the model (2) to solve the short-term forecasting problem requires a reasonable selection of the smoothing adjustment parameter a.

The classic range of acceptable values of a is the interval ae[0,1, extended - ae[0, 2} [26].

The technology of inverse verification or retrospective analysis [1] consists in solving retrospective equations written for the occurred time points (t-1), (t-2) and earlier. For example, for a posteriori calculated errors at time points (t-1), (t-2), •.., (t-n+1) we can write:

et-i (a)=y-i (a) - yt-i = È«(i - «r yt- yt

i=1

n-2

et-2 (a) = yt-2 (a) - y t-2 = Xa (1 - a)i-1 yt-i-2 - yt^ (3)

et-n+1 («) = yt-n+1 («) - yt-n+1 = «yt-n - yt-

Obviously, by solving the equation:

et-1 («) = 0,

(4)

Consider a system of m retrospective equations formed for time points (t-1), (t-2), „., (t-m):

e -1 («)=0, et-2 («) = 0,

(5)

(a) = 0,

where the integer m e [1, n -1 is the depth of inverse verification or retrospective analysis.

Obviously, the equations included in the system (5) generally are not required to have common roots, including real ones. This analytically confirms the authors' opinion that the one-parameter exponential smoothing model, being adaptive in nature [27], still needs parameter adjustment at each forecasting step.

Since the system (5) generally has no solutions on a valid set of the parameter, we reformulate the parameter synthesis problem in the optimization statement:

SSE (a) = ^ e2ti (a) ^ min, a e A,

(6)

where A is the classical or extended admissible set of values of the internal parameter (determined in accordance with the analyst's preferences or opinion).

Problem (6) is easily solved analytically, because the function SSE(a) is a polynomial of order 2(t-1) with real coefficients. Its solution, however, is local in nature and does not allow the analyst to fully assess the adequacy of the forecasting model on the entire allowable set A, including in the vicinity of the obtained solution.

Thus, the robust formulation of the parameter synthesis problem seems promising and useful from a practical point of view.

6. Method of robust estimation of the adequacy area of the one-parameter exponential smoothing model

We soften the conditions for the absolute accuracy of retrospective forecasts (5) and rewrite them as inequalities:

among its real roots, those can be found that lie in the allowable range (whether in classical or extended). By guaranteeing retrospective (a posteriori) accuracy at time point (t-1), the found values of a can be reasonably selected to calculate the forecast at time point t.

The case when only one «last» retrospective equation (4) is considered is described in detail in [27]. However, a point estimate of accuracy cannot fully characterize the quality of the forecasting model [18]. Therefore, different coefficients of pair correlation between the sequences of forecasting and actual values are commonly used as an efficiency criterion (Table 2).

Given the fact that the values of these coefficients (for example, MAPE [18]) are uniquely determined by the values of several «last» errors, it makes sense to consider the system of several first equations from (3).

|et-1 (a))^t-P |et-2 (a)|^t^

et-m (a)^t-m,

(7)

where £,t-1, £,t-2, „., £,t-m are the limit values of permissible errors for the corresponding time points.

The set £,t-1, £,t-2, „., £,t-m is determined by the analyst, based on technical specifications or subjectively. Its presence in the forecasting model characterizes it as robust [27, 28] in the sense that the values of the smoothing parameter a found with its help guarantee the accuracy of retrospective forecasts at the corresponding time points no worse than the given one.

The system of inequalities (7) defines the set A* cA (Fig. 2), which can be considered the adequacy area of the one-parameter exponential smoothing model with accuracy

up to ^ £t-2, £t-m.

e

0

Fig. 2. Analytical interval estimation of the adequacy area of the one-parameter exponential smoothing model, obtained by solving a system of retrospective inequalities

The roots of the equation (8) are found using the Maple symbolic mathematics package:

a1 = 0.2494763881, a2 = 1.547120170,

a3 = 1.664281724,

a4,5 = 1.472360072 ± 0.5323217608Î,

a6,7 = 1.124656056 ± 0.5790963556Î,

a89 = 0.8004009037 ± 0.6997716119Î,

a1011 = 0.4129263204 ± 0.414307274Î (9)

The coefficients and real roots of the retrospective equations for time points t = 12, t = 11 and t = 10 are summarized in Table 3.

Thus, the analyst gets the opportunity to assess the adequacy area of the forecasting model not pointwise, but at intervals, which is preferable for multiple forecasting. It becomes possible to carry out optimal parameter synthesis on an adequate, rather than on an allowable, area of the internal parameter. The efficiency of parameter synthesis procedures can be estimated as the ratio of the initial and found intervals.

7. Using the proposed method in the process of predictive assessment of the dynamics of the logistics hub indicators

The analyst has data on the dynamics of the logistics hub indicator (requests for loading equipment) in the form of a time series (Fig. 3).

Let us write down a retrospective equation of the form (4) for time point t = 12:

e12 (a) = 15.08a11 - 167.11a10 + 841.83a9 -

- 2,544.56a8 + 1,854.76a7 -3,272.25a6 +

+ 3,956.32a5 -3,317.02a4 + 1,901.86a3 -

- 712.65a2 + 157.05a-14.71 = 0. (8)

Table 3

Coefficients and real roots of retrospective equations (RE) of the forecasting model

Coefficients and real Time points

roots of RE t — 12 t— 11 t— 10

a0 15.08 -15.08 15.08

a1 -167.11 152.03 -136.95

a2 841.83 -689.80 552.85

a3 -2544.56 1854.76 -1301.91

a4 5127.01 -3272.25 1970.34

Coefficients a5 -7228.57 3956.32 -1985.98

of RE a6 7273.34 -3317.02 1331.04

a7 -5218.88 1901.86 -570.82

as 2614.51 -712.65 141.83

ag -869.70 157.05 -15.22

a10 171.96 -14.91 -

an -14.71 - -

Real roots of RE a1 0.24948 0.27846 0.34017

a2 1.54712 1.73612 1.41307

a3 1.66428 - 1.65978

16.5 16.0 15.5 15.0 14.5 14.0 13.5

Indicator value

1 1 1

1 1 1

23456789 10 Fig. 3. Primary data in the form of a time series

11

Specify the set £12 = 0.2, ^ = 0.3, £10 = 0.4. It determines valid values of relative errors:

S12 = 1.36%, S41 = 2.01%,

S10 = 2.63%.

(10)

The solution to the system of inequalities (7) for m=3 is given (Fig. 4):

.A. — A.12 aA-11 A10 — — [1.041,1.385] u[1.665,1.726].

(11)

12 Time

We determine the optimal point estimate of the smoothing parameter in accordance with (6) (Fig. 5):

a —1.694.

(12)

1

Eastern-European Journal of Enterprise Technologies ISSN 1729-3774

0.4

(a)/---- A2 = [0.229,0.278] u [1,1.726] /

/1 II 1 1 / 1 / 1 i ^ ' i a

/ 0.4 0.6 0.8 1.0 1.2 1.4 1.6

= [0.264,0.337] u [1.041,1.385] u [1.665,1.769]

0.5

-0.5

"ei0 («) A0 = [0.282,1.734]

1 1 1 1 / 1 / --—^^ /1 / 1 ! a

: /0.4 0.6 0.8 1.0 1.2 1.4

c

Fig. 4. Graphical solution of three retrospective inequalities for time points: a - t = 12, b - t = 11, c - t = 10

Fig. 5. Function SSE(a)

As a result of the simulation, in addition to the optimal estimation of the adjustment parameter (12), we obtain the configuration of the adequacy area of the forecasting model. This allows you to reasonably choose the values of the smoothing parameter, depending on the profile of the retrospective accuracy.

8. Discussion of the results of implementing the robust estimation method of the adequacy area

The context diagram of the robust estimation method for the adequacy area of the one-parameter exponential smoothing model is shown in Fig. 6.

Note that any quality indicator from Table 1 can be a criterion. The proposed form (1) is selected because it is primary in relation to the other criteria from Table 1. Algorithmically it is also possible to use convolutions of criteria. Moreover, the analytical form of their presentation ensures the invariance of the criteria.

The system of retrospective equations of the form (5) describes a reference forecasting model that provides absolute retrospective accuracy to a depth of m values. To ensure its analytical solvability, the formulation of the equivalent problem with «soft» constraints is proposed (7). The user has the opportunity to determine the profile and depth of the retrospective accuracy of the model by specifying the set _, in (7).

The considered example shows the fragment of estimating the dynamics of the time series (Fig. 3) in retrospective analysis with a depth of three values and specified limit relative errors of 1-4 % (10). Under such conditions, the area for a reasonable selection of the adjustment parameter is determined by the intervals (11), which is about 20 % of the initial range of acceptable values.

Thus, the proposed method enables parameter synthesis of the forecasting one-parameter model according to the criterion of retrospective accuracy of multiple forecasts.

tools analyst

Fig. 6. Context diagram of the robust estimation method for the adequacy area of the one-parameter exponential smoothing model

0

b

0

The form of the criterion allows for independent restrictions on the accuracy of each individual retrospective forecast.

The advantage of this study over subjective approaches to the parameter synthesis of the exponential smoothing model [3, 8] is its objectivity, that is, the dependence of the result not on the user's will, but on the value of the selected quality indicator. Unlike search methods [16], the analytical form of retrospective dependencies (7) allows you to obtain a solution with high accuracy and, if necessary, provides the analyst with the opportunity for graphical analysis of the adequacy area. The results can be interpreted as a development of the results obtained in [1, 26, 27] regarding the depth of the retrospective analysis. Their reliability is confirmed by the coincidence of the results for the case with a unit depth (m = 1, £,t -1 = 0).

The limitation of the proposed method is the length of the time series. To solve retrospective equations and inequalities of high order (n > 30), the capabilities of standard mathematical packages may not be enough. The accuracy of finding the roots in this case drops sharply and may be unsatisfactory. However, for a sample of 10-20 values, i. e. within the range when the one-parameter exponential smoothing model is most often used [26], the proposed method is efficient. Another natural limitation is the depth of retrospective analysis, i. e., the number of inequalities in the system (7). The user should take into account that the deeper the analysis and the stricter the requirements for retrospective accuracy, the closer the desired adequacy area to degeneration. Therefore, the proposed method should not be considered as optimal, but as methods for ensuring a given quality.

The following research areas within the framework of the considered problem are promising:

- estimation of the computational complexity and accuracy of solutions to retrospective equations and inequalities of high order (n > 30);

- estimation of the optimal sample length (and, consequently, the order of retrospective equations) in case of data redundancy.

9. Conclusions

1. The selection of the target quality criterion to assess the adequacy of the one-parameter exponential smoothing model is made. As such, the profile of retrospective absolute forecasting errors with adjustable depth is selected. This option has a high degree of clarity and is invariant with respect to other quality indicators.

2. A retrospective model of one-parameter exponential smoothing is formed in accordance with the inverse verification technology. It describes a reference forecasting model that is characterized by absolute retrospective accuracy of multiple forecasts. The algebraic form of retrospective dependencies makes it possible to abandon search procedures in favor of an analytical solution of the inverse forecasting problem. The robust formulation of the optimal parameter synthesis problem, mitigating accuracy constraints is proposed.

3. A method for robust estimation of the adequacy area of the forecasting one-parameter exponential smoothing model, based on inverse verification technology is developed, which allows one to analytically evaluate the limits of the adequacy area of the forecasting model depending on the requirements for its retrospective accuracy profile. The proposed method can be used for parameter adjustment of one-parameter forecasting models in an interactive mode and serves as a decision support tool in the process of forecasting or evaluating parameters critical for a researcher.

4. As an example, the process of analyzing the adequacy of the exponential smoothing model for assessing the dynamics of a time series with a length of 12 values is considered. Given the restrictions on three retrospective forecasts of 1-4 %, the area of the given model quality is determined as a combination of two intervals with a width of 20 % of permissible. It is from the found area that a reasonable value of the adjustment parameter for a subsequent forecast can be selected.

References

1. Romanenkov, Y., Vartanian, V. (2016). Formation of prognostic software support for strategic decision-making in an organization. Eastern-European Journal of Enterprise Technologies, 2 (9 (80)), 25-34. doi: https://doi.org/10.15587/1729-4061.2016.66306

2. De Baets, S., Harvey, N. (2020). Using judgment to select and adjust forecasts from statistical models. European Journal of Operational Research, 284 (3), 882-895. doi: https://doi.org/10.1016/j.ejor.2020.01.028

3. Shorikov, A. F., Butsenko, E. V. (2006). Problema vybora metoda prognozirovaniya rezul'tatov investitsionnogo proektirovaniya. Journal of new economy, 5 (17), 183-191. Available at: https://cyberleninka.ru/article/n/problema-vybora-metoda-prognoziro-vaniya-rezultatov-investitsionnogo-proektirovaniya

4. Samarskiy, A. A., Mihaylov, A. P. (2005). Matematicheskoe modelirovanie: idei, metody, primery. Moscow: Fizmatlit, 320. Available at: https://biblioclub.ru/index.php7page=book_red&id = 68976

5. Kublanov, M. S. (2015). Check of the mathematical model adequacy. Civil Aviation High Technologies, 211, 29-36. Available at: https://avia.mstuca.ru/jour/article/view/373

6. Hyndman, R., Koehler, A., Ord, K., Snyder, R. (2008). Forecasting with Exponential Smoothing: The State Space Approach. Springer, 362. doi: https://doi.org/10.1007/978-3-540-71918-2

7. De Oca, M. A. M., Ferrante, E., Scheidler, A., Rossi, L. F. (2013). Binary Consensus via Exponential Smoothing. Complex Sciences, 244-255. doi: https://doi.org/10.1007/978-3-319-03473-7_22

8. Vasiliev, A. A. (2013). The method of selecting smoothing constant in the forecasting model Brown. Vestnik TvGU. Seriya: Ekonomika i upravlenie, 17, 183-196. Available at: https://core.ac.uk/download/pdf/74270258.pdf

9. De Livera, A. (2010). Automatic Forecasting with a Modified Exponential Smoothing State Space Framework. Monash University. Available at: https://core.ac.uk/download/pdf/6340705.pdf

10. Corberiin-Vallet, A., Bermudez, J. D., Segura, J. V., Vercher, E. (2010). A Forecasting Support System Based on Exponential Smoothing. Intelligent Systems Reference Library, 181-204. doi: https://doi.org/10.1007/978-3-642-13639-9_8

11. Shastri, S., Sharma, A., Mansotra, V., Sharma, A., Bhadwal, A. S., Kumari, M. (2018). A Study on Exponential Smoothing Method for Forecasting. International Journal of Computer Sciences and Engineering, 6 (4), 482-485. doi: https://doi.org/10.26438/ ijcse/v6i4.482485

12. Wan Ahmad, W. K. A., Ahmad, S. (2013). Arima model and exponential smoothing method: A comparison. AIP Conference Proceedings. doi: https://doi.org/10.1063/1.4801282

13. Butakova, M. M. (2008). Ekonomicheskoe prognozirovanie: metody i priemy prakticheskih raschetov. Moscow: KNORUS, 168.

14. Lipatova, N. G. (2015). Imitatsionnoe modelirovanie protsessov tamozhennogo kontrolya. Moscow: Izd-vo Rossiyskoy tamozhen-noy akademii, 164. Available at: http://rta.customs.ru/nrta/attachments/3756_978-5-9590-0846-8.pdf

15. Billah, B., King, M. L., Snyder, R. D., Koehler, A. B. (2006). Exponential smoothing model selection for forecasting. International Journal of Forecasting, 22 (2), 239-247. doi: https://doi.org/10.1016/j.ijforecast.2005.08.002

16. Budaev, P. V. (2009). Prakticheskoe primenenie kolichestvennyh metodov prognozirovaniya. System Research & Information Technologies, 2, 92-106. Available at: https://ela.kpi.ua/bitstream/123456789/7771/V08_Budae.pdf

17. Litvinov, V. V., Zadorozhniy, A. A. (2014). Instrumental'nye sredstva sozdaniya modeley v usloviyah nepolnoty dannyh. Mate-maticheskie mashiny i sistemy, 4, 60-71. Available at: https://cyberleninka.ru/article/n/instrumentalnye-sredstva-sozdaniya-modeley-v-usloviyah-nepolnoty-dannyh

18. Botchkarev, A. (2019). A New Typology Design of Performance Metrics to Measure Errors in Machine Learning Regression Algorithms. Interdisciplinary Journal of Information, Knowledge, and Management, 14, 045-076. doi: https://doi.org/10.28945/4184

19. Yager, R. R. (2013). Exponential smoothing with credibility weighted observations. Information Sciences, 252, 96-105. doi: https:// doi.org/10.1016/j.ins.2013.07.008

20. Mayer, G. (2017). Interval Analysis. De Gruyter, 518. doi: https://doi.org/10.1515/9783110499469

21. Olive, D. J. (2017). Robust Multivariate Analysis. Springer. doi: https://doi.org/10.1007/978-3-319-68253-2

22. Cipra, T. (1992). Robust exponential smoothing. Journal of Forecasting, 11 (1), 57-69. doi: https://doi.org/10.1002/for.3980110106

23. Crevits, R., Croux, C. (2017). Forecasting Using Robust Exponential Smoothing with Damped Trend and Seasonal Components. SSRN Electronic Journal. doi: https://doi.org/10.2139/ssrn.3068634

24. Van den Broeke, M., De Baets, S., Vereecke, A., Baecke, P., Vanderheyden, K. (2019). Judgmental forecast adjustments over different time horizons. Omega, 87, 34-45. doi: https://doi.org/10.1016/j.omega.2018.09.008

25. Vasiliev, A. A. (2012). The selection criteria of forecast models (review). Vestnik TvGU. Seriya: Ekonomika i upravlenie, 13, 133-148. Available at: https://docplayer.ru/65174367-Kriterii-selekcii-modeley-prognoza-obzor-a-a-vasilev.html

26. Romanenkov, Yu. (2015). Analysis of the predictive properties of Brown's model in the extended domain of the internal parameter. MOTROL. Commission of Motorization and Energetics in Agriculture, 17 (8), 27-34. Available at: https://www.researchgate.net/ publication/315692053_Analysis_of_the_predictive_properties_of_Brown%27s_model_in_the_extended_domain_of_the_inter-nal_parameter

27. Romanenkov, Yu. A., Zieiniiev, T. G. (2014). Method of parametric synthesis of brown's model based on retrospective multi-objective optimization. Zbirnyk naukovykh prats. Seriya: haluzeve mashynobuduvannia, budivnytstvo, 2 (41), 48-56. Available at: https:// www.researchgate.net/publication/315693687_Metod_parametriceskogo_sinteza_modeli_Brauna_na_osnove_retrospektivnoj_ mnogokriterialnoj_optimizacii

28. Valeev, S. G., Kuvaiskova, Yu. E., Yudkova, M. V. (2010). Robust methods of estimation: software, effectiveness. Vestnik Ul'yanov-skogo gosudarstvennogo tehnicheskogo universiteta, 1 (49), 29-33. Available at: https://cyberleninka.ru/article/n/robastnye-metody-otsenivaniya-programmnoe-obespechenie-effektivnost