NON-ASYMPTOTIC CONFIDENCE ESTIMATION OF THE AUTOREGRESSIVE PARAMETER IN AR(1) PROCESS BY NOISY OBSERVATIONS Текст научной статьи по специальности «Физика»

ВЕСТНИК ТОМСКОГО ГОСУДАРСТВЕННОГО УНИВЕРСИТЕТА

2022 Управление, вычислительная техника и информатика № 59

Tomsk: State University Journal of Control and Computer Science

Original article

doi: 10.17223/19988605/59/9

Non-asymptotic Confidence Estimation of the Autoregressive Parameter in AR(1) Process by Noisy Observations

Sergey E. Vorobeychikov1, Andrey V. Pupkov2

12 Tomsk State University, Tomsk, Russian Federation 1 sev@mail. tsu. ru 2 andrewpupkov@gmail. com

Abstract. For parameter in an AR(1) process corrupted by noise, the paper proposes the construction of confidence interval for unknown parameter with a prescribed coverage probability. The noises both in observable and in unobservable processes are assumed to be Gaussian with unknown variance. The estimation procedure is non-asymptotic and uses a special stopping rule. The results of numerical simulation by Monte-Carlo method are presented. Keywords: autoregressive process; non-asymptotic estimation; confidence interval

For citation: Vorobeychikov, S.E., Pupkov, A.V. (2022) Non-asymptotic confidence estimation of the autoregressive parameter in AR(1) process by noisy observations. Vestnik Tomskogo gosudarstvennogo universiteta. Upravlenie, vychislitelnaja tehnika i informatika - Tomsk State University Journal of Control and Computer Science. 59. pp. 83-90. doi: 10.17223/19988605/59/9

Научная статья УДК 519.246.2 doi: 10.17223/19988605/59/9

Доверительное неасимптотическое оценивание параметра авторегрессии AR(1)

по зашумленным данным

Сергей Эрикович Воробейчиков1, Андрей Викторович Пупков2

12 Томский государственный университет, Томск, Россия 1 sev@mail. tsu. ru 2 andrewpupkov@gmail. com

Аннотация. Рассматривается задача построения доверительного интервала неизвестного параметра процесса авторегрессии первого порядка, зашумленного аддитивным шумом. Предполагается, что управляющий шум процесса и шум в канале наблюдений - гауссовские с неизвестными дисперсиями. Построенная процедура является неасимптотической и опирается на специальное правило остановки наблюдений. В статье приводятся результаты численного моделирования, реализованного методом Монте-Карло.

Ключевые слова: процесс авторегрессии; неасимптотическое оценивание; доверительный интервал

Для цитирования: Воробейчиков С.Э., Пупков А.В. Доверительное неасимптотическое оценивание параметра авторегрессии AR(1) по зашумленным данным // Вестник Томского государственного университета. Управление, вычислительная техника и информатика. 2022. № 59. С. 83-90. doi: 10.17223/19988605/59/9

In many applications related to signal processing the linear models specified by stochastic difference equations are widely used. To identify unknown parameters in such models the methods of least squares

© S.E. Vorobeychikov, A.V. Pupkov, 2022

(LSE), maximum likelihood and other have been developed. The properties of the estimators are studied usually in asymptotic when the number of observations tends to infinity. For an autoregressive process the asymptotic properties of the LSE have been studied in [1]. To overcome the problems of investigation the properties of the estimators obtained by a fixed number of observations the sequential methods were developed. Sequential procedures use the sampling schemes with random stopping times. It allows one to study the properties of the estimators. The sequential sampling scheme was proposed in works [2] and [3] to estimate the parameter of a first order autoregressive process

xk = 0xk-i + sk> k = 1,2v-

The estimator in [2] has the guaranteed mean square deviation.

The problem of estimation of parameters in ARMA processes and in AR processes with noise was studied in [4-7]. A sequential procedure of identification parameters of autoregressive processes by noisy observations was proposed in [8]. This procedure uses the Yule-Walker estimators with guaranteed mean square deviation.

A problem of confidence estimation of the mean in a sequence of independent identically distributed Gaussian variables with unknown variance was studied in [9]. A sequential procedure was proposed because no procedure with fixed sample size can guarantee the prescribed coverage probability.

Recently a procedure for constructing a fixed-size confidence interval with any prescribed coverage probability for parameter in AR(1) process was proposed in [10]. The interval estimation for a first-order autoregressive process was constructed in [11].

The paper is organized as follows. In Section 1 we construct a sequential point estimator of unknown parameter in AR(1) process with noises. In section 2 we construct a confidence interval with any prescribed coverage probability for the parameter in AR(1) model with unknown noise variances. In section 3 the results of numerical simulation are presented.

1. Sequential point estimator

Consider an unobservable AR(1) process described by equations

xk = 0Xk-1 + Sk > k = 1,2V> (1)

which is observed with noises

yk = xk , k =1,2•••• (2)

Here 9 is unknown parameter, 10|< 1, {sk} and {^k} are the sequences of Gaussian independent random variables with zero means Esk = E = 0 and variances Es2 = a2 ; E^k = A2 respectively, initial value x0 and processes {sk}, {-qk} are independent. It's assumed that a2 and A2 are unknown. The problem is to construct the non-asymptotic confidence interval for the parameter 9 using observations {yk}.

Note that from (1) and (2), we obtain the equation for the observed process

yk =0yk -i + %k. (3)

where ^ =Sk + % -0%-i, E^k = 0 and E^ =A2(1 + 02) + a2.

First we obtain a point estimator of an unknown parameter of AR(1) process. The estimator is used later to construct the confidence interval. The scheme of estimating the parameter 9 follows the approach proposed in [10] and includes three stages.

First we obtain the pilot Yule-Walker estimator of 0 by a fixed number of observations. On the second stage we construct an estimator of the variance of the random variable . On the third stage we construct a sequential modification of the Yule-Walker estimator of the parameter 0 .

For an integer n1 > 3 define the Yule-Walker estimator of the parameter 0 as

0(»i) = iz^-2^-il "tyk-2yk- (4)

\k=3 J k=3

To compensate an unstable behavior of estimator (4) for small sample size we use the projection of the estimator into the interval [-1,1]:

efa), ifieooKi, e = §(»,) = J i, if ec^ > > 1,

-l, ife(/i!)<-i.

Using the estimator 9(n1) define the estimator of the variance ^k as

rn1,n2 = C(n2)Sn1,n2 , (5)

where

/ N-1

n I n \ O _L n

" .2 I Sk + %

Sn,„2 (yk-9y,-1)2, C(n2) = E|Zvk | , Vk • (6)

k=1

C

n2 2

Note that vk are standard Gaussian random variables. Hence i v^ is Chi-square random variable with n2

k=1

degrees of freedom and C(n2) = (n2 - 2)-1. Denote

< = Z (£k + %)2, l > 0. (7)

k=n1 +1

The properties of the variables Vnl establishes the following theorem

Theorem 1. Let the random processes jSni 11 and [V^} are defined by (5) and (7) respectively. Then the following inequality holds true

P (< z )< PQ{vlni < z )• (8)

The proof of Theorem actually proceeds along the lines of the proof of Theorem 1 in [10]. Using Theorem 3 in [10] one has Theorem 2. The following inequality holds true

e-l- <■ 1

r^n a2 +c2'

This inequality allows one to obtain the upper bound of the variance E,k .

Now we construct a sequential point estimator of the parameter 9. The estimator is based on the esti-

2 1

mator proposed in [12] when the variances c and A in (1) and (2) are known.

To construct sequential point estimator of the parameter 9 , we divide the set of indexes T(n) = [1,2,..., n} into two subsets

T1 (n) = [2j -1: j = 1,2,2j -1 < n}, T2(n) = [2j : j = 1,2,...;2j < n}. Note that T (n) = Tx (n) + T2 (n). Next we introduce stopping times

zt(h) = inf J n > n +n2 +1: £ ^k-2.X{keT.(„)} > h, i = 1,2. (9)

[ k=n1 + n2 +1 1 n1,n2 J

and X{A} is indicator of event A. The value h is a parameter of the identification procedure and defines

the quality of the estimator.

Define sequential estimators of the parameter 9 from odd and even observations by formulas

r I-1 x ^

91 (h) = f) f) >/Ptk , t = 1,2, (10)

k=«1 + «2+1 n2 j k=«1 + «2+1 n2

where the coefficients Pik, i = 1,2, are defined as

1, if k g T (xi (A)) and k < xi (A); Pa = jaf (A), if k = T,. (A);

0, ifk g 2i(Ti(A)). Correction factors 0 < ai (A ) < 1,i = 1,2 are determined by the equalities

2 2

/'r ^{k.T„)} + a(A)^ = A.

k=„1 +„2 +1 i „j,, i n2

Consider the deviation of the estimators (10). Substituting yk from (3) into (10) we have

1

(11)

0 г (A)-0 =

Sj(h) +и2+1

Ъ(A) ,-

У*-2 (0У£-1 + = Ci (A) . = 12

/F"

where

Denote

Ci (A) = 1° s, (A) = 1°

k="i+"2 +1 -y/1 «1,«2 k="i +"2 +1 д/i «1,«2

¿=«1+«2+l ' ' Hence, the deviation 9 i ( A) - 9 takes the form

'2"+02) + <-2

(12)

(13)

0, (h )-0 = K

Ci (h) Sl(hY

i = l,2.

According to Lemma 2 in [12] the following result holds.

Lemma 1. Let {sk} and {%} in (3) be the sequences of Gaussian random variables with zero means

and variances Esk = u2, Er^ = A2, and random variables ¿j, (A), i — 1, 2 are described by the equations (13). Then for each h> 0 the random variables /T12^ (A) and A~1/2^2(A) are standard Gaussian. In the next section we construct a non-asymptotic confidence interval for parameter 9.

2. Non-asymptotic confidence interval

The main result of the paper gives the following Theorem.

Theorem 3. Let {sk} and {^} in (3) be the sequences of Gaussian random variables with zero means and variances Es2 = ct2 , E^ = A2, and the sequential point estimators 99 i (h), i = 1,2 are defined by formulas (9), (10), (11). Then for all h > 0 and z > 0

Pa

| s* (A) | ,01(A) + 0 2(A)

Л

A.

/Г"

\ n1,n

-0 |< Z

> 1 -a (a, z),

(14)

where r n, is defined by (5) and s* (A) = min ( Sj(A), s2(A) ) and

,/2-1

a(A, z) = 4jt1 - ф(z^yac") )] * /9чехр(-У^.

i(„2 / 2)

Here C (n2 ) is defined by (6), r(n2 / 2) is the Gamma-function and

ф( x) =

I

dy.

(15)

1

X

Proof: Using estimators (5) and (10) we obtain the inequality

-0

| s*(h) | 01(h) + 0 2(h)

h4r "1,"2 2

K0 | s* (h) |

2h4r "1,"2 hih)

- + -

<

<

k9 | s*(h)|

2 h

/T"

Si(h)

S2(h)

(16)

<

Ka

2 h

/T"

{^(/OI + IW)!]-

Introduce the filtration

^o ^{yo}, Tn =CT{^o,e1>...,e„>ii1>...>ii„}.

To construct the confidence interval, we obtain the upper bound for probability

( - - \ ( \

Pa

\ s* (h) | .01(h) + 02(h)

-0 |> z

< Pa

Ka

2h.

/T"

{^(/OI + IWOl]

>z

= EPa

L [Ki(h)\ + K 2(h)\]>

(17)

N/h"

Then

^ [ Ki(h)| + K 2(h) | ]> ^^

Vh"

+ Pa

Ka

= 2Pfi

V/t"

V/t"

KA

Ka

+

(18)

As a result from (17) and (18) we obtain

EPa

1 / ~ ~ \ 2z./hr«

A

Vh"

>1-2EP

Kim>zJ^

= 2

1 - EPfl

KA

-1 = 2EPfi

4h

v

Kim^WZ^

Vh k0

Ka

(19)

-1.

Let G (y) be the distribution function of random variable r / k0 . Taking into account that 101< 1

2 2 2 2 2 2 we come to inequality k0=A(1 + 0) + a <2(A + a ). Using expression (5) for r« « and inequality (8)

we

obtain the upper bound for the distribution function G (y)

Pa

r„

< z

= Pa

'1+«2 9 z K,

X (yk-0yk-1)2 <

.2 (

< Pa

k=« +1

C(«2)

J

'1 + "2 7 Z K,

X (8k + %) <

v

k=«+1

C («2)

<

<P

where v k = (a2 + A) (8k + %) • [10] using int

«+« 2 2z(a2 + A2)^ _ ( "1+"2 v2 <

(a2 + A2 ) X v k

V 'k=«! +1 C ("2)

= Pa

J

«+« ~ 2 z ^

(20)

v

X v 2 ,

k=«+1 C ("2)

Analogously to [10] using integration by parts and (20) we have

f ~ n^—

EPP

yfh

KA

= iPQ 0

yfh

dG(y) =

= 1 -J G( y)dPQ 0

(21)

yfh

>

2

2

K0

Л

> 1 "I P 0

да

= 1 Ре

0

"1 + "2 2

С2 2 2 y

2 V2 < y

Л (

v k="1+1

С(и2)у

dPa

Jh

\Ш)\ *Jh

dPa

ч+"2 7 2 y

2 V2 < y

y Л

v k=" +1

С("2)y

Taking into account that 2 vir has Chi-square distribution with n2 degrees of freedom, inequalities (17),

k=n1 +1

(19), (21) and Lemma 1 we can construct a non-asymptotic confidence interval for the parameter 9 in the form

s*(h)| 91(h) + 9 2(h) ^

P

да

> 2 J Ре

0

Ci (h) Jh

= 2^f 2ф( zjhy)-1

л/Äy 1^4

2

л /

--е |< z

>

< z.

У

(1/2)"2/2 ( 2y

Г(«2/2)

+"2 2 2 y

2 V2

V k="1+1 c ("2)

\"2/2-1 г л

y I

vС("2)y

exp <

-1 =

У

2dy

(22)

С("2) I С("2)

-1 =

= 2j[ 2Ф( z^yhC ("2) )

-1

y

"2/2-1

Г("2 / 2)

exp(-y)dy -1,

where O(x) is determined by the formula (15).

From (22) one has the equality to defining the value h of the form

a(h, z) = 4да 1 -Ф( ZyjyhC("2) )

y"2/2-1

Г("2 / 2)

exp(-y)dy.

We come to the result of Theorem 3.

Remark 1. Note that the random coefficient s* (h) / h in the confidence interval converges to constant almost surely due to Proposition 1 in [12].

3. Simulation results

In this section, we report and discuss the results of Monte Carlo experiments. Table 1 presents selected data obtained by the simulations.

Represent the interval for parameter 9

k*(h)| e 1(h)+e 2(h)

ЧГ"1,"2 2

-е

< z

in the form

e(h)-s<e<e(h)+s,

e(h) =

e 1(h )+e 2 (h ) 2 ,

(23)

(24)

where 5 = zh^Jr« ^ /1 s*(h) | is the confidence semi-interval.

The results of Monte Carlo simulation are reported in Table. All the results were obtained by 1000 replications by making use of the programming language R. The values n1 and n2 were chosen equal to 30. The averaged values of sequential estimators 9( h) as well as mean length of semi-interval 5 have been calculated for every value of the parameter 9. The confidence probability 1 -a(h, z) was equal to 0,9 for different values of h and z. The quantity T denotes the observed averages of the stopping time (x1(h) + x2(h)) / 2. The quantity p denotes the frequency count of the number of times when the confidence interval contains the true values 9 .

да

Averaged confidence estimates of parameter AR(1)

9 o2 A2 h = 779, z = 0,1 h = 3 115, z = 0,05

0(h) 5 X P 0(h) 5 X p

-0,99 1 0,25 -0,990 0,098 156,5 1 -0,991 0,05 359,6 1

-0,8 1 0,25 -0,802 0,133 954,3 1 -0,800 0,067 3504,4 0,999

-0,6 1 0,25 -0,598 0,187 1545,9 1 -0,600 0,095 6079,5 1

-0,4 1 0,25 -0,401 0,290 2171,2 0,999 -0,402 0,147 8315,1 1

-0,3 1 0,25 -0,300 0,387 2447,0 0,999 -0,300 0,198 9445,0 1

0,3 1 0,25 0,302 0,448 2420,0 1 0,303 0,212 9627,7 1

0,4 1 0,25 0,400 0,323 2202,5 1 0,402 0,156 8446,0 1

0,6 1 0,25 0,599 0,202 1611,1 1 0,600 0,098 6113,8 1

0,8 1 0,25 0,799 0,140 913,3 1 0,800 0,069 3572,4 1

0,99 1 0,25 0,991 0,103 158,4 1 0,990 0,051 340,0 1

0,99 4 0,25 0,990 0,102 144,9 1 0,990 0,051 293,0 1

0,99 4 1 0,990 0,103 159,4 1 0,990 0,051 347,0 1

0,99 1 4 0,991 0,114 426,0 1 0,990 0,056 1286,3 1

2 _

Note that as the variance a of the process increases, the sample size x decreases. Increasing the variance

2 _

A of additive noise leads to increasing the sample size x . This fact follows from Proposition 1 in [12].

Conclusions

The proposed sequential procedure allows one to construct the confidence non-asymptotic interval for autoregressive parameter 9 in the presence of additive noise in observations. The procedure is independent of variances of noises in unobservable and observable processes. It is based on a special rule of determining the needed sample size. The results can be used in identification and control problems.

References

1. Lai, T.L. & Wei, C.Z. (1983) Asymptotic properties of general autoregressive models and strong consistency of least-squares

estimates of their parameters. Journal of Multivariate Analysis. 13. pp. 1-23. DOI: 10.1016/0047-259X(83)90002-7

2. Borisov, V.Z. & Konev, V.V. (1977) On sequential parameter estimation in discrete time processes. Automation and Remote

Control. 38. pp. 1475-1480.

3. Lai, T.L. & Siegmund, D. (1983) Fixed accuracy estimation of an autoregressive parameter. Annals of Statistics. 11. pp. 478-485.

DOI: 10.1214/aos/1176346154

4. Anderson, T.W. (1971) The Statistical Analysis of Time Series. New York: Wiley.

5. Hasan, Md.K., Chowdhury, A.K.M.Z.R. & Khan, M.R. (2003) Identification of autoregressive signals in colored noise using

damped sinusoidal model. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. 50. pp. 966969.

6. Mahmoudi, A., Karimi, M. & Amindavar, H. (2012) Parameter estimation of autoregressive signals in presence of colored AR(1)

noise as a quadratic eigenvalue problem. Signal Processing. 92. pp. 1151-1156. DOI: 10.1016/j.sigpro.2011.11.015

7. Xia, Y. & Zheng, W.X. (2015) Novel parameter estimation of autoregressive signals in the presence of noise. Automatica. 62.

pp. 98-105. DOI: 10.1016/j.automatica.2015.09.008

8. Vasiliev, V.A. & Konev, V.V. (1985) Sequential identification of parameters in dynamical systems in presence of multiplicative

and additive noises. Automation and Remote Control. 46. pp. 706-716.

9. Stein, C. (1945) A Two-Sample test for a linear hypothesis' whose power is independent of the variance. Annals of Mathematical

Statistics. 16. pp. 253-258. DOI: 10.1214/aoms/1177731088

10. Vorobeychikov, S. & Burkatovskaya, Yu. (2020) Non-asymptotic confidence estimation of the autoregressive parameter in AR(1) Process with an Unknown Noise Variance. Austrian Journal of Statistics. 49. pp. 19-26. DOI: 10.17713/ajs.v49i4.1121

11. Wei, C., Hao, Y. & Ching, K. (2018) Interval estimation for a first-order positive autoregressive process. Journal of Time Series Analysis. 39. pp. 1475-1480. DOI: 10.1111/jtsa.12297

12. Konev, V. & Pupkov, A. (2021) Confidence estimation of autoregressive parameters based on noisy data. Automation and Remote Control. 82. pp. 1030-1048. DOI: 10.1134/S0005117921060059

Information about the authors:

Vorobeychikov Sergey E. (Doctor of Physics and Mathematics, National Research Tomsk State University, Tomsk, Russian Federation). E-mail: [email protected]

Pupkov Andrey V. (Post-graduate Student, National Research Tomsk State University, Tomsk, Russian Federation). E-mail: [email protected]

Contribution of the authors: the authors contributed equally to this article. The authors declare no conflicts of interests. Информация об авторах:

Воробейчиков Сергей Эрикович - доктор физико-математических наук, профессор кафедры системного анализа и математического моделирования Института прикладной математики и компьютерных наук Национального исследовательского Томского государственного университета (Томск, Россия). E-mail: [email protected]

Пупков Андрей Викторович - аспирант кафедры системного анализа и математического моделирования Института прикладной математики и компьютерных наук Национального исследовательского Томского государственного университета (Томск, Россия). E-mail: [email protected]

Вклад авторов: все авторы сделали эквивалентный вклад в подготовку публикации. Авторы заявляют об отсутствии конфликта интересов.

Received 24.03.2022; accepted for publication 30.05.2022 Поступила в редакцию 24.03.2022; принята к публикации 30.05.2022