ESTIMATION OF PARAMETERS OF AUTOREGRESSIVE MODELS WITH FRACTIONAL DIFFERENCES IN THE PRESENCE OF ADDITIVE NOISE Текст научной статьи по специальности «Математика»

Scientific article

DOI: 10.18287/2541-7525-2023-29-3-103-109

Submited: 11.07.2023 Revised: 15.08.2023 Accepted: 30.10.2023

D.V. Ivanov

Samara National Research University, Samara, Russian Federation Samara State University of Transport, Samara, Russian Federation E-mail: dvi85@list.ru. ORCID: https://orcid.org/0000-0002-5021-5259

ESTIMATION OF PARAMETERS OF AUTOREGRESSIVE MODELS WITH FRACTIONAL DIFFERENCES IN THE PRESENCE OF ADDITIVE NOISE1

ABSTRACT

For modeling in time series, models with fractional differences are widely used. The best known model is the ARFIMA (autoregressive fractionally integrated moving average) model. It is known that for integer-order autoregressive models, autoregressive models with additive noise can outperform ARMA and autoregressive models in terms of accuracy. This article considers a class of autoregressive models with fractional order differences. The article presents a new method for estimating parameters autoregressive models with fractional differences in the presence of additive noise with an unknown variance of additive noise. The propose algorithm was realized in Matlab. The simulation results show the high efficiency of the propose algorithm.

Key words: Fractional difference; autoregressive model; total least squares; additive noise; unknown ratio of variances; generalized instrumental variables; long run memory.

Citation. Ivanov D.V. Estimation of parameters of autoregressive models with fractional differences in the presence of additive noise. Vestnik Samarskogo universiteta. Estestvennonauchnaya seriya / Vestnik of Samara University. Natural Science Series, 2023, vol. 29, no. 3, pp. 103-109. DOI: http://doi.org/10.18287/2541-7525-2023-29-3-103-109. (In Russ.)

Information about the conflict of interests: authors and reviewers declare no conflict of interests.

© Ivanov D.V., 2023

Dmitriy V. Ivanov — associate professor, Candidate of Physical and Mathematical Sciences, Department of Information Security, Samara National Research University, 34, Moskovskoye shosse, 443086, Russian Federation; associate professor, Department of Information Technologies, Samara State University of Railway Transport, 2B, Ulica Svobody, 443066, Russian Federation.

Introduction

To describe processes of various nature, equations with derivatives are increasingly used. and differences of fractional order. Despite the lack of a simple interpretation, which give derivatives, integrals and differences of integers, models described by fractional-order equations, make it possible to accurately simulate many processes in physics and technology [1-4]. In connection with the active development and application of equations with differences and fractional derivatives for modeling and forecasting problems, methods for estimating systems have also begun to actively develop, describing fractional-order equations and differences.

Autoregressions with fractional differences are widely used in the analysis of time series with long memory [5; 6]. There are a large number of different models with generalizations of fractional differences,

1The work was carried out as part of the development program of the Scientific and Educational Mathematical Center of the Volga Federal District, agreement No. 075-02-2023-931.

such as Gegenbauer autoregressive moving average (GARMA) [7; 8], fractional ARUMA [9], seasonal autoregressive fractionally integrated moving average (SARFIMA) [10; 11], and autoregressive tempered fractionally integrated moving average (ARTFIMA) [12; 13]. Various aspects of using fractional differences for time series analysis have been considered [14; 15].

It is known that for autoregressive models of an integer order, autoregressive models with additive noise can exceed the accuracy of ARMA models and autoregressive models [16]. An overview of methods for estimating integer-order autoregressions in the presence of noise is presented in [17]. In the articles [8; 18; 19], the author considered the estimation of autoregressions with fractional-order differences in the presence of noise with a known noise ratio.

The article presents a new method for estimating parameters autoregressive models with fractional differences in the presence of additive noise with an unknown variance of additive noise.

1. Basic results

Time series, is described by linear stochastic equations with fractional order differences:

r

Zi =J2 b(m)Aamz— + Zi, yi = Zi + (1.1)

m=1

oo

where b(m) are constant coefficients; 0 < a1 ... < ar; r(a) = f e-tta-1dt ;

0

Aamzi = ¿ ( — 1)j ( a'm ) zi-j is fractional difference;

j=0 \ j J

ajm ^ = r(j+i)r(a+—j+1) is generalized binomial coefficients.

It is required to estimate the unknown coefficients of the dynamic system described by (1.1) from the observed sequence {yi} with noise for the known orders r , am.

If r and, am are unknown, it is necessary to apply algorithms based on global optimization, such as genetic algorithms [8].

The following assumptions are introduced:

A1. The dynamic system (1) is asymptotically stable.

A2. Noises {&} and {Zi} are statistically independent sequences with E{&} = 0, E^} =0, E{ £?} = = < rn, E{ Z2} = < ro a.s., where E is the expectation operator.

A3. The output sequence {zi}, is independent of noise sequence {^}. The noise sequences {&} ,{Zi} are mutually independent.

In [18], the following objective function was proposed for estimating the parameters:

\\Y — Cb\\2 , N

min—--, (1.2)

b 1+ y + bT H£ b v '

where

H(mk) = limi^oo N S1 ( a;)( a;)N,m=i;-r,k = 1,7,

C = {^T ■■■vTN1 e RrxN, Y = (y1 ...yN) e RN, b = (b(1)... b(r))T e Rr, Pi = (Aai yi-1,..., Aar y-1) e R1xr ,y = a2c/al

Theorem 2.1. [18] Let the dynamic system described by Equation (1.1)with initial zero conditions and assumptions A1-A3 be introduced. Then, the estimate of the coefficients determined by expression (1.2) exists, is unique, and converges to the true value of the coefficients with probability 1, i.e.:

b(N)—. bo (1.3)

Proof. The proof of the theorem is similar to the proof given in [20]. The minimum of function (1.2) can be found as a solution to the biased normal system of equations

(CTC — a\H£) b = CTY. (1.4)

If the noise variance is unknown a|, then it is necessary to use the estimate of the additive noise variance The variance estimate can be found as the minimal generalized singular value

= ^min (C,Lz), (1.5)

where amin (C, L^) is the minimal generalized singular number of matrices C and Lz, C= ( Y C ).

H = LJ Le, Hz = ( 7 .

In [17] a review of methods for parametr estimation integer-order autoregressions with additive noise is presented. One of the most accurate was the approach proposed in the article [21]. This article uses a generalization of this approach to the case of autoregressions with fractional order differences. The maximum value of the variance a|max is if the variance a2 =0 is defined as

a2 max = amin (C, Lmax)

where amin (C,Lmax) is minimal generalised singular values of matrices C and Lmax,

H = T T T H = I 1 0

¡max LmaxLmax, ¡max I 0 ¡^

The true value of the variance belongs to the interval a| e ( 0 a|max ).

In [21], high-order Yule-Walker equations are used to determine the variance. However, this approach cannot be applied directly, since it is impossible to obtain a vector of instrumental shifts for equation (1.1).

Minimization (1.2) can be written as an eigenvector problem:

(CTС - <r|H6) b = 0, (1.6)

where b = ( ^

Equation (1.6) requires knowing not only the variance of the additive noise a|, but also the variance ai. In order to eliminate the need to evaluate a| and al simultaneously for fractional order autoregressions, we use generalized instrumental variables [22], the application of generalized instrumental variables for fractional order systems is considered in the article [23].

The vector of instrumental variables ^i satisfies the equality

lim {CTС - а\Щ) b = 0, (1.7)

where

Ct = (ФТ ...ФТ) e RrxN

i = (Aai yi-2,.., Aary-2) e R1xr, H^ = ( 0 H^ ),

<k)=n zNtO^ a;)N- m=k=

For a finite sample, equality (1.7) will not be strict, the problem of determining the variance estimate a| can be described as a quadratic function minimization problem

min J (at), (1.8)

rj( 6(0,0"; max)

J (ae) = bT С С - а|Нт)Т С С - аЩ) Ъ.

where T

H) C - a

Based on equations (4), (6) and (8), an iterative algorithm is proposed for estimating the parameters b and the variance a| .

Step 1. Determine the maximum value of the variance a|max is defined as

a2 max = amin (C, Lmax) .

Step 2. Start from a generic value a| e ( 0 a|max ) . Step 3. Compute the parameter vector from equation (1.4)

e CT C - a|H5) b = CT Y,

Step 4. Compute the cost function (1.8)

-T

J (at) = C C - alH^f C C - a\H^j b.

Step 5. Choose a new value a| . The choice can be made using one of the methods of one-dimensional optimization.

Step 6. Repeat steps 3-5 until the value associated with the minimum of is found.

2. Simulation results

The proposed algorithm has been compared with ordinary least squares and the algorithm based on objective function (1.2) with a known noise variance ratio. The minimum (1.2) of the objective function can be found from the solution of the equation (1.4) or the augmented system of equations [24].

Test cases were compared by the following characteristics:the normalized root mean square error (NRMSE) of parameter estimation, defined as

sb = ^ |b - bo||21

2 , 2

•100%,

and normalized root mean square error of modelling (NRMSEM), defined as

Sz P - z||2 / ||z||2.100%.

The results were based on 50 independent Monte-Carlo simulations. Example 1. The AR model is described by the equation

* = 0.45Д-°'Ч_1 + Zi, Vi = zi + Zi, (2.1)

Noise standard deviation ratio

a? ¡az = 0.5,7 = 2.605 The number of data points N in each simulation was 10000.

Table 2.1 shows the mean values of tNRMSE and NRMSEM and their standard deviations.

Table 2.1

Mean values of NRMSE and NRMSEM and their standard deviations

Таблица 2.1

Средние значения NRMSE и NRMSEM и их стандартные отклонения

Ordinary least squares, % Algorithm with known ratio, % Proposed algorithm with unknown ratio, %

Sb 8.95 ± 5.74 1.05 ± 1.20 1.44 ± 1.93

Sz 43.50 ± 15.42 12.88 ± 9.88 13.55 ± 10.98

Example 2. The AR model is described by the equation

Zi = 0.5A°'7Zj_i + Zi, Vi = zi + Zi,

Noise standard deviation ratio

a ^ ¡az = 0.5,7 = 2.11

The number of data points N in each simulation was 2000.

Table 2.2 shows the mean values of tNRMSE and NRMSEM and their standard deviations.

(2.2)

Mean values of NRMSE and NRMSEM and their standard deviation

Table 2.2

Таблица 2.2

Средние значения NRMSE и NRMSEM и их стандартное отклонение

Ordinary least squares, % Algorithm with known ratio, % Proposed algorithm with unknown ratio, %

Sb 15.73 ± 2.62 2.16 ± 1.46 3.00 ± 2.74

Sz 26.70 ± 2.36 5.82 ± 4.47 7.13 ± 5.39

Conclusion

This paper proposed an estimation method of the parameters of fractional AR models with additive noise. The simulation results showed that the parameter estimates obtained using the proposed algorithm are highly accurate.

Further development of the proposed approach is the study of the best choice of instrumental variables and the choice of the weighting matrix.

References

[1] Stiassnie M. On the application of fractional calculus for the formulation of viscoelastic models // Applied Mathematical Modelling. 1979. Vol. 3. Issue 4. Pp. 300-302. DOI: https://doi.org/10.1016/S0307-904X(79)80063-3.

[2] Bagley R.L. Fractional calculus - a different approach to the analysis of viscoelastically damped structures // AIAA Journal. 1983. Vol. 21. Number 5. Pp. 741-748. DOI: https://doi.org/10.2514/3.8142.

[3] Reyes-Melo M.E., Martinez-Vega J.J., Guerrero-Salazar C.A., Ortiz-Mendez U. Application of fractional calculus to modeling of relaxation phenomena of organic dielectric materials // Proceedings International Conference Solid Dielectrics (ICSD'04). Toulouse, 2004. Vol. 2. P. 530-533. DOI: http://doi.org/10.1109/ICSD.2004.1350485.

[4] Vinagre B.M., Feliu V. Modeling and control of dynamic system using fractional calculus: Application to electrochemical processes and flexible structures // Proceedings of 41-st IEEE Conference Decision Control. Las Vegas, 2002. Pp. 214-239. Available at: https://www.researchgate.net/profile/Vicente-Feliu-2/publication/252440774_Modeling_and_control_of_dynamic system_using_fractional_calculus_Application_to_electrochemical_processes_and_flexible_structures/links/ 54aa71290cf2eecc56e6f22e/Modeling-and-control-of-dynamic-system-using-fractional-calculus-Application-to-electrochemical-processes-and-flexible-structures.pdf.

[5] Granger C.W., Joyeux R. An introduction to long-memory time series models and fractional differencing // Journal of Time Series Analysis. 1980. Vol. 1, No. 1. Pp. 15-29. DOI: http://doi.org/10.1111/ j.1467-9892.1980.tb00297.x.

[6] Hosking J.R.M. Fractional differencing // Biometrika. 1981. Vol. 68, No. 1. Pp. 165-176. DOI: http://doi.org/ 10.1093/biomet/68.1.165.

[7] Woodward W.A., Cheng Q.; Gray H.L. A k-factor GARMA long-memory model // Journal of Time Series Analysis. 1998. Vol. 19, Issue 4. Pp. 485-504. DOI: https://doi.org/10.1111/j.1467-9892.1998.00105.x.

[8] Ivanov D.V., Engelgardt V.V., Sandler I.L. Genetic Algorithm of Structural and Parametric Identification of Gegenbauer Autoregressive with Noise on Output // Procedia Computer Science. 2018. Vol. 131. Pp. 619-625. DOI: http://dx.doi.org/10.1016/j.procs.2018.04.304.

[9] Giraitis L., Leipus R. A generalized fractionally differencing approach in long memory modelling // Lithuanian Mathematical Journal. 1995. Vol. 35. Pp. 53-65. DOI: https://doi.org/10.1007/BF02337754.

[10] Reisen V.A., Rodrigues A.L., Palma W. Estimation of Seasonal Fractionally Integrated Processes // Computational Statistics & Data Analysis. 2006. Vol. 50, Issue 2. Pp. 568-582. DOI: https://doi.org/10.1016/j.csda.2004.08.004.

[11] Reisen V.A., Rodrigues A.L., Palma W. Estimating seasonal long-memory processes: A Monte Carlo study // Journal of Statistical Computation and Simulation. 2006. Vol. 76, Issue 4. Pp. 305-316. DOI: http://dx.doi.org/10.1080/10629360500107790.

[12] Meerschaert M.M., Sabzikar F., Phanikumar M.S., Zeleke A. Tempered fractional time series model for turbulence in geophysical flows // Journal of Statistical Mechanics Theory and Experiment. 2014. P09023. DOI: http://doi.org/10.1088/1742-5468/2014/09/P09023.

[13] Sabzikar F., Meerschaert M.M., Chen J. Tempered Fractional Calculus // Journal of Computational Physics. 2015. Vol. 293. Pp. 14-28. DOI: https://doi.org/10.1016/jjcp.2014.04.024.

[14] Ram M. Recent Advances in Time Series Forecasting, 1st ed.; Bisht D.C.S. (Ed.) Boca Raton, FL, USA: CRC Press, 2021. 238 p. DOI: https://doi.org/10.1201/9781003102281.

[15] Palma W. Long-memory time series: Theory and Methods. Hoboken, NJ, USA: Wiley, 2006. 304 p. Available at: https://books.google.ru/books?id=NtSbmQyQcSMC&printsec=frontcover&hl=ru#v=onepage&q&f=false.

[16] Guidorzi R., Diversi R., Vincenzi L., Simioli V. AR+ noise versus AR and ARMA models in SHM-oriented identification // Proceedings of the 23rd Mediterranean Conference on Control and Automation (MED), Torremolinos, Spain, 16-19 June 2015. Pp. 809-814. DOI: https://doi.org/10.1109/med.2015.7158845.

[17] Ivanov D., Yakoub Z. Overview of Identification Methods of Autoregressive Model in Presence of Additive Noise // Mathematics 2023. Vol. 11, Issue 3. P. 607. DOI: https://doi.org/10.3390/math11030607.

[18] Ivanov D.V. Identification autoregression non-integer order with noise in output signal. In: Interdisciplinary research in the area of mathematical modelling and informatics: Materials of the research and practical internet-conference. June 18-19, 2019. Togliatti, June 18-19, 2013. Togliatti: SIMJET, 2013, pp. 64-67. Available at: https://www.elibrary.ru/item.asp?id=20742187. EDN: https://www.elibrary.ru/rlxlcf. (In Russ.)

[19] Ivanov D.V., Serebryakov A.Yu., Ivanov A.V. Identification of autoregression described by fractional difference equations with 1/f noise in the output signal. Vestnik transporta Povolzhya, 2016, no. 5 (59), pp. 93-99. Available at: https://www.elibrary.ru/item.asp?id=27468166. EDN: https://www.elibrary.ru/xdcfvf. (In Russ.)

[20] Ivanov D.V. Estimation of parameters of linear fractional order ARX systems with noise in the input signal // Tomsk State University Journal of Control and Computer Science. 2014. No. 2(27), pp. 30-38. Available at: https://www.elibrary.ru/item.asp?id=21649106. EDN: https://www.elibrary.ru/sftlbf. (In Russ.)

[21] Diversi R., Guidorzi R., Soverini U. Identification of autoregressive models in the presence of additive noise // International Journal of Adaptive Control and Signal Processing. 2007. Vol. 22. P. 465-481. DOI: http://doi.org/10.1002/acs.989.

[22] Soderstrom T. A generalized instrumental variable estimation method for errors-in-variables identification problems // Automatica. 2011. Vol. 47, Issue 8, Pp. 1656-1666. DOI: https://doi.org/10.1016/ j.automatica.2011.05.010.

[23] Ivanov D.V., Sandler I.L., Kozlov E.V. Identification of Fractional Linear Dynamical Systems with Autocorrelated Errors in Variables by Generalized Instrumental Variables // IFAC-PapersOnLine, 2018, Vol. 51, Issue 32. Pp. 580-584. DOI: http://dx.doi.org/10.1016/j.ifacol.2018.11.485.

[24] Ivanov D., Zhdanov A. Symmetrical Augmented System of Equations for the Parameter Identification of Discrete Fractional Systems by Generalized Total Least Squares // Mathematics. 2021. Vol. 9, Issue 24. Article number: 3250. DOI: https://doi.org/10.3390/math9243250.

Научная статья DOI: 10.18287/2541-7525-2023-29-3-103-109

УДК 519.254.1 Дата: поступления статьи: 11.07.2023

после рецензирования: 15.08.2023 принятия статьи: 30.10.2023

Д.В. Иванов

Самарский национальный исследовательский университет имени академика С.П. Королева, г. Самара, Российская Федерация Самарский государственный университет путей сообщения,

г. Самара, Российская Федерация E-mail: dvi85@list.ru. ORCID: https://orcid.org/0000-0002-5021-5259

ОЦЕНИВАНИЕ ПАРАМЕТРОВ АВТОРЕГРЕССИИ С РАЗНОСТЯМИ ДРОБНОГО ПОРЯДКА ПРИ НАЛИЧИИ АДДИТИВНОГО ШУМА2

АННОТАЦИЯ

Для моделирования во временных рядах широко используются модели с дробными разностями. Наиболее известной моделью является модель ARFIMA (авторегрессионная частично интегрированная скользящая средняя). Известно, что для авторегрессионных моделей целого порядка авторегрессионные модели с аддитивным шумом могут превосходить по точности ARMA и авторегрессионные модели. В данной статье рассматривается класс авторегрессионных моделей с разностью дробного порядка. Представлен новый метод оценивания параметров авторегрессионных моделей с дробными разностями при наличии аддитивного шума с его неизвестной дисперсией. Предлагаемый алгоритм реализован в среде Matlab. Результаты моделирования показывают высокую эффективность предложенного алгоритма.

Ключевые слова: дробная разность; авторегрессионная модель; сумма наименьших квадратов; аддитивный шум; неизвестное отношение дисперсий; обобщенные инструментальные переменные; долговременная память.

Цитирование. Ivanov D.V. Estimation of parameters of autoregressive models with fractional differences in the presence of additive noise // Вестник Самарского университета. Естественнонаучная

2Работа выполнена в рамках реализации программы развития Научно-образовательного математического центра Приволжского федерального округа, соглашение № 075-02-2023-931.

серия / Vestnik of Samara University. Natural Science Series. 2023. Т. 29, № 3. С. 103-109. DOI: http://doi.org/10.18287/2541-7525-2023-29-3-103-109.

Информация о конфликте интересов: авторы и рецензенты заявляют об отсутствии конфликта интересов.

© Иванов Д.В., 2023

Дмитрий Владимирович Иванов — кандидат физико-математических наук, доцент кафедры безопасности информационных систем, Самарский национальный исследовательский университет имени академика С.П. Королева, 443086, Российская Федерация, г. Самара, Московское шоссе, 34; доцент кафедры цифровых технологий, Самарский государственный университет путей сообщения, 443066, Российская Федерация, г. Самара, ул. Свободы, 2B.