Sequential empirical process of independence Текст научной статьи по специальности «Математика»

УДК 519.24

Sequential Empirical Process of Independence

Abdurahim A. Abdushukurov*

Dpt. Applied Mathematics and Informatics Branch of Moscow State University in Tashkent Dpt. Appl. Math. and Informatics av. Timur, 1000060, Tashkent Uzbekistan

Leyla R. Kakadjanova^

Dpt. Probability Theory and Mathematical Statistics National University of Uzbekistan VUZ Gorodok, Tashkent, 100174 Uzbekistan

Received 03.01.2018, received in revised form 05.06.2018, accepted 20.07.2018 Uniform strong laws of large numbers and the central limit theorem for special sequential empirical process of independence for a certain class of measurable functions are considered in the paper.

Keywords: sequential empirical processes, metric entropy, Glivenko-Cantelli theorem, Donsker theorem. DOI: 10.17516/1997-1397-2018-11-5-634-643.

1. Introduction and preliminaries

Let us consider a sequence of experiments in which observed data consist of independent pairs {(Xk,Ak),k > 1}, where Xk are random variables (r.v.-s) on a probability space (Q,A, P) with values in a measurable space (X; B) and Ak are events with common probability p = P(Ak) £ (0,1). Let Sk = I (Ak) be an indicator of the event Ak. At the n-th stage of experiment the observed data are S(n) = {(Xk,Sk), 1 < k < n}. Each pair (Xk,Sk) induces a statistical model with sample space X ® {0,1} with a-algebra G of sets B ® D and distribution Q* (■) on (X ® {0,1} , G):

q* (B ® D)= P (Xk £ B,Sk £ D) ,B £ B,D c {0,1} .

We consider submeasures Qm (B) = Q* (B ® {m}) ,m = 0,1 and Q (B) = Q0 (B) + (B) = = Q* (B ® {0,1}) ,B £ B. From a practical point of view, it is important to test the validity of hypothesis H for independence of r.v. Xk and event Ak for each k > 1. In order to verify this we use the signed measure A (B) = Qi (B) — pQ (B) ,B £ B, where p = Qi (X) and the validity of H is equivalent to the equality A (B) = 0 for any B £ B. We introduce the empirical estimates of the above introduced measures for B £ B from sample S(n) :

1 n 1 n

(в ) = -У2(1 - sk) I (Xk e в), Qm (B) = -Y/ ski (Xk e B).

n z—' n z—'

k=l k=l

*[email protected] 1 [email protected] © Siberian Federal University. All rights reserved

n

Qn (B) = Qon (B) + Qrn (B) = -J21 (Xk G B),

tl < J

(1)

k=i

K (B) = Qin (B) - PnQn (B) ,pn = Qin (X).

By the strong law of large numbers (SLLN) we have for a fixed set B that Qmn(B) —:—>

n—^

and An(B)

— A(B). If hypothesis H is valid

Qm(B), m = 0,1; Qn(B) ——

n—n—^^

then An(B) a:s > 0. Then we arrive at the study of limit behaviour of normalized process

n

{xn = an (An (B) — A (B)) ,B e G} , where {an, n > 1} is a (possible random) sequence of positive numbers, and G is a certain class of sets from B. The specially normalized empirical process of independence indexed by the class F of measurable functions f e F was studied [1]. Class F coincides with xn when f = I (■) is the indicator. In this paper we extend these results for the sequential analogue of that process.

2. Sequential uniform law of large numbers

For a measure G and class F of Borel measurable functions f : X ^ R we introduce the following integral

Gf = f fdG, f eF. ■Jx

Let us introduce the following F-indexed extensions of (1) for f e F:

nn

Qonf = -Y,(1 — Sk) f (Xk), Qinf =-Y/ 5k f (Xk),

T> < * :n < *

k=i

k=i

Qnf = Qonf + Qinf =-J2 f (Xk),

m < J

(2)

k=i

-

and Anf = Qinf - PnQnf, where pn = Qinl = Qin (X) = - J2 ôk. Relations (1) are special

n k=i

cases of (2) when F = {I (B) ,B G G}. We define F-indexed empirical process Gn : F — R as

f — Gnf = Vn (Qn - Q) f = n-i/2£ (f (Xk) - Qf ), f G F.

k=i

Here Gnf = G0nf + G1nf with subempirical processes

Gjnf = vn (Qjn — Qj) f, j = 0, l,f e F. For a given f by SLLN and central limit theorem (CLT) we have (a) Qnf —— Qf as Q\f \ < rc;

(3)

(4)

(5)

(6)

(b) Gnf ^ Gf = N (0, aQ (f)), n ^rc as Qf2 < rc,

where aQ (f )= Q(f — Qf )2.

There is theory for uniform variants of special classes F of measurable functions in (5) and (6) (see, for example, [2-4]). There are various extensions of the Glivenko-Cantelli theorem and the Donsker theorem for F-indexed empirical processes (3) under certain conditions on the set F of

measurable functions. These conditions ensure that n-i/2\\Gnf = sup {n-i/2 \Gnf \, f £ F} converges either in probability or almost surely to zero. These classes F are called the weak or strong Glivenko-Cantelli classes, respectively. Donsker-type theorems provide general conditions on F in order to get weak convergence

Gnf ^ Gf in l~ (F), (7)

where l(F) is the space of all bounded functions f : X ^ R with the supremum-norm \\.\\^ (see [3], p. 81). Class F with condition (7) is called the Donsker class. The limiting field {Gf, f £ F} in (7) is called Q-Brownian bridge. Let us introduce tight Borel measurable element of l(F) and Gaussian field with zero mean and covariance function

cov (Gf, Gg) = Qfg — QfQg, f,g £F. (8)

Remind that Q-Brownian bridge {Gf, f £ F} can be represented in terms of Q-Brownian sheet {W (f) ,f £ F} with zero mean and covariance

cov (W (f) ,W (g))= Qfg, f,g £F, (9)

by distribution equality

Gf = W (f) — W (1) Qf, f £ F. (10)

For a given f with the conditions Qj \f \ < x>, j =0,1 by SLLN we have

Anf a4' Afund=r H 0 (11)

Moreover, for a given f variable ^fn (An — A) f is a linear functional of subempirical processes (4) with the condition Qj f2 < to, j = 0,1. It has limiting normal distribution N (0, aQ (f)). Uniform SLLN and CLT for the specially normalized empirical F-indexed process

{Anf (Pn (1 — Pn))

i/2

= <pnrr—pn^j (A-—a)!-!£F>

Were proved [1].It was shown that the limiting distribution is Q-Brownian bridge {Gf, f £ F} with covariance (8). Let us consider the following sequential extension of {anf, f £ F}

{an (s; f) = (pn (1 — pn))-i/2n-i/2 [ns] (A[nS] — A) f, (s; f) £ d} , (12)

where V = T ® F,T = [0,1], A[ns] = Qi[ns] — p[ns]Q[ns] and [a] denotes the integer part of a. Then anf = an (1; f). Let \\$ (s)\\T = sup {\^ (s)\, 0 < s < 1} and \\an (s; f)\\D = = sup {\An (s; f )\, (s; f) £ V}. We will prove uniform strong and weak LLN's for process

[ns]

{M (A[ns] — A) f, (s; f) £V} .

Sequential SLLN is considered in the following theorem. Theorem 2.1. Let us assume that Qjf2 < to, j = 0,1, f £ F. Then

Ans]— A) f

a.s

0. (13)

Proof. It is easy to see that

i 1 (1 — ) [™s]

(AM - A) f = n Sf (Xk) - Qi f )-

k=i

P[ns]

[ns]

[ns]

]T ((1 - Sk) f (Xk) - Qof ) - - £ (4 - p) Qf. (14)

k=i

k=i

Assuming Qi 1 = p, from (14) we have

n1 (A[ns]- A) f

< QI

[ns]

-J2(Sk -Qii)

n i-'

k=i

+

[ns]

-J2(Sk f (Xk) - Qif )

n i-'

k=i

+

+

[ns]

-£ ((i - Sk) f (Xk) -

k=i

. (15)

Using sequential SLLN (Theorem 1.1 in [2]), we obtain (13) for all three terms in the right hand side of (15). Theorem 2.1 is proved. □

Remark 2.1. The assumptions in Theorem 2.1 can not be weaken. But for sequential weak LLN

p

(A[ns]- A) f

-> 0

only the validity of the assumption Qj \f \ < to, j = 0,1, f e F is required.

In order to prove that D = T ® F are uniform variants of the Glivenko-Cantelli theorem and the Donsker theorem we need some notations from bracketing entropy theory. Let Lq (Q) be the space of functions f : X ^ R with norm

. i/q

llQ,q = (Q\f\q )1/q = U \ lx

To determine the complexity or entropy of a set of Borel measurable functions F it is necessary

to define a concept of e-brackets in Lq (Q). So e-bracket in Lq

is a pairs of functions p,^ G

Lq (Q) such that Q (p (X) < ^ (X)) = 1 and H^ - p||Qjq < e, that is, Q(^ - p)q < eq. Function f e F is covered by bracket [p,^\ if Q (p (X) < f (X) < ^ (X)) = 1. Note that functions p and ^ may not belong to the set F but they must have finite norms. The bracketing number

N[] (e, F, Lq (see, [3,4]):

is the minimum number of e-brackets in q

needed to cover the set

N[] (e, F, Lq

lk

UM F

k : for some fi,..., fk G Lq

C U [fi,fj]: \\fj - fiWQqq < e.

The number Hq (e) = log N[] (e, F, Lq (Q)) is called the metric entropy of class F in Lq (Q). The metric entropies of a class F in Lq (Qj), j =0,1 is we denoted by Hjq (e) = log Nj [] (e, F, Lq (Q)). Integrals of metric entropies are

ts

j (S) = Jj[] (S, F, Lq (Qj)) = J (Hjq (e))1/2de, 0 < S < 1, j =0,1.

Let us recall the important properties of numbers N[] (.). They tend to when e I 0. However, for the Donsker theorems they should converge to not very fast. This rate of convergence

n

is measured by integrals jjq (S) (for more details, see [3,4]). Let us prove stronger properties of considered random fields and introduce following normalized empirical processes on V = T ®F:

Yn (s; f )= Yon (s; f) + Yin (s; f), Zn (s; f) ^VnYn (s; f) = Zon (s; f) + Zin (s; f),

where for j = 0, 1

[ns ]

Yjn (s; f )=[nYj[ns] (s; f), [ns] [ns]

Yon (s; f ) = n±yj((1 — Sk) f (Xk) — Qof (Xk)), n

k=i

1 [ns]

Yin (s; f ) = ~T\ (Sk f (Xk) — Qif (Xk)),

n

k=i

I [ns]

Zjn(s;f) = VnYjn(s;f) = Y —Gj[ns]f,

n

with Zjn (1; f) = Gjnf.

Let l(D) be a space of all bounded functions on D = T®F with the supremum norm . In what follows we show that the role of s £ T is negligible in the LLN theorems. Let P* be the outer probability.

Theorem 2.2. There exists a universal constant C such that for every e > 0

P* (\\Yn (s; f )\\D > 4e) < 2Cmax P* (\\Yjn (1; f )\F > e) . (16)

Proof. For (s; f) £ V we have \Yn (s; f )\ < 2max \\Yjn (s; f )\\T. Hence

j=o,i

\\Yn (s; f)\\v < 2 max sup \\Yjn (s; f)\\(17)

k

In the right hand side of (17) the parameter s may take values — with k = 1,... ,n. Because

n

Ins I Ins I

Yjn (s; f) =vnYj[ns] (s; f ) = n — Qj) f,j = 0,1, we obtain from (17) that

k

\\Yn (s; f)\\v < 2i=xi max -№ — Qj) f\F. (18)

J—o,i iXkxn rn

It follows from the Ottaviani inequality A.1.1. [3] that

P* ( k ^ fii P* (\\(Qjn — Qj) f \\f >e) . 01 (19)

P ma^ — (Qo^ — Q,) t\\t > e \ < -t,-F-), j =0,1. (19)

ViXkXn nIKVjk j llF ) — max P* (£\\(Qjk — Qj) f\y > e)' J ' V ;

iXkXn

Thus, the numerator of (19) converges to zero as n ^ to on condition that F is a weak Glivenko-Cantelli class. The term

k

\,Q' Qj) J \\f

maxp* Q\\(QJk — Qj) f\\f > e

iXkXn \n

indexed by k < n can be controlled with the help of inequality

jk — Qj) f \\F < F (Xk ) + 2noP * F, j =0,1 (20)

k = i

for an envelope function F of the class F. For sufficiently large n0 the terms indexed by k > n0 are bounded away from 1 by the uniform weak LLN for Qjn, j = 0,1. Moreover, the denominator in (19) is bounded away from zero. Using inequalities (19) and (20) twice, we obtain (16) from (17) and (18). Theorem 2.2 is proved. □

Let us introduce some definitions of uniform weak and strong LLN [2] and adapt them to our processes.

Definition 2.1. A class of measurable functions F is a sequential weak Glivenko-Cantelli class if

lY (s; f C 0.

n—^^O

Definition 2.2. A class of measurable functions F is a weak Glivenko-Cantelli class if

Yn(1; ■)* —— 0,

n

where Yn(1; ■)* is the measurable cover function of Yn (1; ■).

Definition 2.3. A class of measurable functions F is a sequential strong Glivenko-Cantelli class if

lY (s; f)llD -— 0.

n

Definition 2.4. A class of measurable functions F is a strong Glivenko-Cantelli class if

Yn(1; ■)* —— 0.

n—*x>

Because llYn (1; ■)!?: ^ llYn (s; f then by Theorem 2.2 for every e > 0 we have

P * (llYn (1; -)ll^ > 2e) < P * (lYn (s; f )Hv > 2e) < CP * (lYn (1; Ol^ > e). (21) Taking into account (21), we have

Corollary 2.1. A class F is a sequential weak (or strong) Glivenko-Cantelli class if and only if it is a weak (or strong) Glivenko-Cantelli class.

Consider singleton set of measurable functions {f}. If Q \f \ < to then by weak LLN

lY (1; OlU} = (Qn - Q) f —— 0,

and by Corollary 2.1 the singleton set {f} is a sequential weak Glivenko-Cantelli class.

Definition 2.5. A class of measurable functions F is a sequential complete Glivenko-Cantelli class if

J2P (llYn (s; f )HD > 1) < to (22)

n=1

and lY (s; f)llD 0.

Definition 2.6. A class of measurable functions F is a complete Glivenko-Cantelli class if

lY (1;^ ^ 0.

By introducing summation in each side of inequality (21) we have

Corollary 2.2. A class of measurable functions F is a sequential complete Glivenko-Cantelli class if only if it is a complete Glivenko-Cantelli class.

The sequential SLLN was proved in Theorem 2.1 in terms of the second moment condition. But such results can be established by bracketing entropy.

Theorem 2.3. Let us assume that

FcC2 (Qj) and j (1) < to, j = 0,1. (23)

Then F is a sequential strong Glivenko-Cantelli class, that is,

^ (A[H- A) f

n

*

a.s.

■> 0. (24)

Proof. Let us obtain almost sure convergence (24) in terms of the complete convergence

0. (25)

(AM- A) f

*

C

Consider Corollary 2.2. In order to prove (25) it is enough to prove

y (An - A) f -- 0. (26)

Taking into acount (14), we have

(An - A) f = (1 - Pn) Uln (f ) - PnUon (f ) - (pn - p) Qf, (27)

where Ujn(f ) = f fd(Qjn - Qj), j = 0,1. Using Proposition 3.3 [2] with the condition Qj f2 < œ,

X

j = 0,1, we obtain

Ujn (f ) -- 0,j = 0,1 (28)

Using the Berstein inequality [5],

œ / 2 \

x—>œ x—> i n£ \

X)n=1 P (Pn - p\ >£) < 2!>xp( -—J < œ, £> 0,

n=l ^ '

we obtain

Pn -- p. (29)

n^œ

Statements (26) and (25) follow from (27)-(29). This completes the proof of (24) and Theorem 2.3.

3. Sequential uniform central limit theorem

Let us consider the sequential specially normalized empirical D = T ® F — indexed random fields defined by relation (12). It was proved under the mild conditions [1] that

An (1; f ) ^ Af in lœ (F), (30)

where {Af, f G F} is a Gaussian fields with zero mean and subject to hypothesis H that it coincides with the Q-Brownian bridge with covariance (8). Here we extend convergence (30) to the sequential field (12). To begin with we prove that two-dimensional vector-field

{(Zn (s; f ), Zin (t; g)), (s; f ), (t; g) G D} (31)

weakly converges to corresponding Gaussian field uniformly with respect to semimetric of product space lœ (D) ® lœ (D) for every Donsker class of measurable functions F.

Theorem 3.1. Let us consider the class F such that

F C £2 (Qj) and j) (1) < to, j = 0,1. (32)

Then for n ^ to sequence of random vector-fields (31) weakly converge in l(D) ® l(D) to the Kiefer-Muller-type Gaussian field {(Z (s; f ), Zi (t; g)), (s; f ), (t; g) G D} with zero mean and covariance structure

cov (Z (s; f), Z (t; g)) = min (s; t) cov (Zi (s; f), Zi (t; g)) = min (s; t) {Qf g - Qf Qig} , (33)

cov (Z (s; f), Zi (t; g)) = min (s; t) {Qf g - Qf Qg .

Proof. Consider the first condition in (32). Then for the fixed f € F it follows that Qj f2 < to, j = 0,1 and hence Qf2 = Qof2 + Qif2 < to. For every such Donsker class F with the second condition in (32) the sequences Zn (s; f) and Z1n (t; g) are asymptotically tight (see, Lemma 1.3.8 in [3]). There exists a tight Borel measurable version of Gaussian processes Z (s; f) and Z1 (t; g), that is, the Kiefer-Mwller processes with zero mean and jointly covariances (32). Tightness and measurability of limiting processes Z (■, ■) and Z1 (■, ■) are equivalent to the existence of versions of all sample paths (s; f) ^ Z (s; f), (t; g) ^ Z1 (t; g) uniformly bounded and uniformly continuous with respect to the corresponding semimetrics with squares given by (see, [3], p. 226)

E(Z (s; f) - Z (t; g))2 = |s - t\ [a2 (f) I (s > t) + a2 (g) I (s < t)] + min (s; t) a2 (f - g),

E(Z1 (s; f) - Z1 (t; g))2 = |s - t\ [aQx (f) I (s > t) + aQx (g) I (s < t)] + min (s; t) aQx (f - g),

where aQ (f) = Q(f - Qf )2, aQ (f) = Q1 (f - Qf )2.

On the other hand, the considered vector-field is the normalized sequential sum of independent and identically distributed random vectors

[n(sAt)]

(Zn (s; f), Z1n (t; g))= n-1/2 £ (f (Xk) - Qf,5kg (Xk) - Q1g). (34)

k=1

Then by the multivariate CLT the marginals of the sequence of vector-fields converge to the marginals of a Gaussian vector-valued field with zero mean and covariance matrix defined by structure (33). Vector-field (34) is element of l(D)®l(D), and it also induces tight sequences of distributions in product space by Lemma 1.4.3 [3]. Covariance structure of vector (34) has the form

cov(Zn(s; f), Zn(t; g)) = min(M'[n,t]) {Qfg - Qf Qg} ,

n

cov(Zm(s; f), Zm (t; g)) = min([ns], [n,t]) {Qfg - Qf Q1g} , (35)

n

cov(Zn(s; f), Zm(t; g)) = mm([nsL [n,t]) {Qfg - Qf Q1g} ,

n

and we see that (33) is the limiting value of (35). These arguments complete the proof of Theorem 3.1. □

Remark 3.1. Consider relation (34). At g = 1 for s,t € T and f € F we have Q11 = p and hence

cov(Z (s, f), Z1 (t; 1) = min (s, t) {Qf - pQf} = min(s, t) ■ Af. (36)

Because covariance (36) is zero for any s,t e T and f e F under hypothesis H then Kiefer-Muller field {Z (s; f), (s, f) e D} and rescaled Wiener process {Zi (t; 1), t e T} with covariance min(s,t) p (1 — p) are independent. We use this fact in the following theorem. Now we consider the intermediate random field

| A: (s; f ) = M . (A[ns] — A) f, (s; f) ev} , (37)

i /

connected by An (s; f) in terms of An (s; f) = (pn (1 — pn)) '2 • An (s; f). Process (37) plays a supporting role in the study of basic process (12) which property of weak convergence to a corresponding Gaussian process is contained in the following statement.

Theorem 3.2. Under conditions of Theorem 3.1 for n ^ to we have

An (s; f) ^ A (s; f) in l~ (D), (38)

where {A (s; f), (s; f) e D} is a Gaussian field with zero mean and hypothesis H is valid. For s,t e T and f,g e F it coincides with Kiefer-Muller random field with covariance

cov (A (s, t) A (t; g)) = min (s, t) (Qfg — Qf Qg). (39)

Proof. Let us consider process (37) and represent it in the form of linear functional of sequential subempirical processes

An (s; 2- (GiM f — pG[ns]f — QfG1M1) + Rn (s; t) = A0n (s; f) + Rn (s; f), (40)

where Rn (s; f )= n 12 [ns] (pM — p) (Q[nS]f — Qf and hence

R (s; f )\\v = op (1), n ^ to. (41)

We consider only An (s; f). It is not difficult to see that An (s; f) have zero mean and for s,t e T, f,g e F its covariance is

cov (a:> (s; f), An> (t; g)) = £ Cj, (42)

j=i

where

C1 = Qifg — Qi f Qig, C2 = —p (Qfg — QfQig), C3 = — (1 — p) Qf Qg

C4 = —p (Qifg — QgQif), C5 = p2 (Qfg — Qf Qg), C6 = pQf (Qig — pQg), (43)

C7 = — (1 — p) QgQif, C8 = pQg (Qf — pQf), C9 = p (1 — p) QfQg.

Taking into account Theorem 3.1, we have

An (s; f) ^ A0 (s; f) in l~ (D), (44)

where A0 (•; •) is a mean zero Gaussian process and accordingly to (42) its covariance is

9

cov (A0 (s; f), A0 (t; g)) = min(s,t)J^ C, (45)

j=i

where Cj are defined in (43). Assuming that hypothesis H is valid and taking into account Remark 3.1, it is easy to obtain that

cov (A0(s; f), A0(t; g)) = p (1 - p) min(s, t) (Qfg - QfQg), (46)

and

(p (1 - p))-1'2 ■ A0n (s; f) ^ A (s; f) in l~ (D). (47)

Now relation (38) follows from (39)-(47). Theorem 3.2 is proved. □

References

[1] A.A.Abdushukurov, L.R.Kakadjanova, A class of special empirical processes of independence, J. Siberian Federal Univ. Math. Phys., 8(2015), no. 2, 125-133.

[2] J.Bae, S.Kim, The sequential uniform law of large numbers, Bull. Korean Math. Soc., 43(2006), no. 3, 479-486.

[3] A.W.Van der Vaart, J.A.Wellner, Weak convergence and empirical processes, Springer, 1996.

[4] A.W.Van der Vaart, Asymptotic Statistics, Cambridge University Press, 1998.

[5] Yu.V.Prokhorov, An enlarge of S. N. Bernstein's inequality to the multivariate case, Theory Probab. Appl, 13(1968), no. 3, 266-274 (in Russian).

Последовательные эмпирические процессы независимости

Абдурахим А. Абдушукуров

Кафедра прикладной математики и информатики Филиал Московского государственного университета в Ташкенте

ав. Тимура, 100060, Ташкент Узбекистан

Лейла Р. Какаджанова

Национальный университет Узбекистана им. М. Улугбека

вузгородок, Ташкент, 100174 Узбекистан

Мы доказываем равномерные усиленные законы больших чисел и центральную предельную теорему для специальных последовательных эмпирических процессов независимости для специальных классов измеримых функций.

Ключевые слова: последовательные эмпирические процессы, метрическая энтропия, теоремы Гливенко-Кантелли и Донскера.