CC BY
6
On the Use of Entropy as a Measure of Dependence of
Two Events. Part 2
Valentin Vankov Iliev •
Institute of Mathematics and Informatics Bulgarian Academy of Sciences Sofia, Bulgaria viliev@math.bas.bg
Abstract
The joint experiment J(A,B) of two binary trials A U Ac and B U Bc in a probability space can be produced not only by the ordered pair (A, B) but by a set consisting, in general, of 24 ordered pairs of events (named Yule's pairs). The probabilities £1, £2, £3, £4 of the four results of J(a,b) are linear functions in three variables a = Pr(A), f = Pr(B), 0 = Pr(A n B), and constitute a probability distribution. The symmetric group S4 of degree four has an exact representation in the affine group Aff (3, R), which is constructed by using the types of the form [a, f, 0] of those 24 Yule's pairs. The corresponding action of S4 permutes the components of the probability distribution (£1, £2, £3, £4), and, in particular, its entropy function is S4-invariant. The function of degree of dependence of two events, defined in the first part of this paper via modifying the entropy function, turns out to be a relative invariant of the dihedral group of order 8.
Keywords: probability space; experiment in a sample space; probability distribution; entropy; degree of dependence; relative invariant.
1. Introduction
The initial idea of this work was to describe all symmetries of the sequence of Yule's pairs from (1) which produce one and the same experiment [3, 4.1,(1)]. If we consider the equivalence classes of the form [(a,f, 0)] that contain the members of (1), then the naturally constructed in terms of coordinate functions a, f, 0 affine automorphisms of the linear space R3 form a group which is isomorphic to the symmetric group S4, see Section 2, Theorem 1. The components £1, £2, £3, £4 of the probability distribution [3, 4.1,(2)] are linear functions in a, f, 0. The group S4 naturally acts via above isomorphism and permutes £'s. As a consequence we obtain Theorem 2 which asserts that the entropy function Ea,f (0) = E(a, f,0) of the probability distribution (£1,£2,£3,£4) (see [3,
5.1]) is an absolute S4-invariant.
In Section 3, Theorem 3, we show that the degree of dependence function ea,f (0), defined in [3,
5.2] via "normalization" of the entropy function Ea,f (0), is a relative invariant of the dihedral group D8, see [2, Ch.1,1.]. The proof uses the embedding of D8 as one of the three Sylow 2-subgroups of S4.
We use definitions and notation from [3, 2].
2. Methods
In this paper we are using fundamentals of:
• Affine geometry and Real algebraic geometry
• Invariant Theory.
3. The Group of Symmetry of an Experiment
3.1. Yule's Pairs and Experiments
Let A, B G A. We define AQB = (AAB)c, where AAB = (Ac n B) U (A n Bc) is the symmetric difference of A and B.
Any ordered pair (A, B) G A2 produces the experiment J = J(A,B) from [3, 4.1,(1)], which is naturally identified with the partition {A n B, A n Bc, Ac n B, Ac n Bc} of Q (cf. [4,1,§5]). The proof of the next Lemma is straightforward.
Lemma 1. Yule's pairs from the sequence with members
(A, B) of type (a, ft, 9), (A, Bc) of type (a, 1 - ft, a - 9), (Ac, B) of type (1 - a, ft, ft - 9), (Ac, Bc) of type (1 - a, 1 - ft, 1 - a - ft + 9), (B, A) of type (ft, a, 9), (B, Ac) of type (ft,1 - a, ft - 9), (Bc, A) of type (1 - ft, a, a - 9), (Bc, Ac) of type (1 - ft, 1 - a, 1 - a - ft + 9), (A, AQB) of type (a, 1 - a - ft + 29,9), (AQB, A) of type (1 - a - ft + 29, a, 9), (B, AQB) of type (ft, 1 - a - ft + 29,9), (AQB, B) of type (1 - a - ft + 29, ft, 9), (Ac, AQB) of type (1 - a, 1 - a - ft + 29,1 - a - ft + 9), (AQB, Ac) of type (1 - a - ft + 29,1 - a, 1 - a - ft + 9), (Bc, AQB) of type (1 - ft, 1 - a - ft + 29,1 - a - ft + 9), (AQB, Bc) of type (1 - a - ft + 29,1 - ft, 1 - a - ft + 9), (A, A AB) of type (a, a + ft - 29, a - 9), (A AB, A) of type (a + ft - 29, a, a - 9), (B, AAB) of type (ft,a + ft - 29, ft - 9), (AAB, B) of type (a + ft - 29,ft, ft - 9), (Ac, AAB) of type (1 - a, a + ft - 29, ft - 9), (AAB, Ac) of type (a + ft - 29,1 - a, ft - 9), (Bc, AAB) of type (1 - ft, a + ft - 29, a - 9), (AAB, Bc) of type (a + ft - 29,1 - ft, a - 9),
are exactly the pairs that produce the experiment J(a,b) .
Remark 1. (i) According to [1, 2.1, 2.7.1, 2.8.4], the set of points (a, ft, 9) in R3 where the types from Lemma 1 are pair-wise different is semi-algebraic, open, and three-dimensional. Its trace U3 on the interior T3 of the classification tetrahedron T3 from [3, 4.1] is not empty because otherwise T3 would be subset of a finite union of planes. Theorem 2.2.1 from [1, 2.1] guaranties that the open two dimensional projection of U3 onto aft-plane is semi-algebraic. Note that "openness" is with respect to the standard topology in R3.
(ii) Under some "plentifulness" condition on Boolean algebra A (for example, if it is non-atomic), there exist plenty of Yule's pairs (A,B) of type (a,ft) G U2. In this case (we call it "general") the sequence from Lemma 1 consists of 24 Yule's pairs.
(1)
3.2. The Group of Symmetry
Let E be the set of all experiments in the probability space (Q, A, Pr), that is, the set of all finite partitions of Q with members from A. The rule (A, B) ^ J(a,b) defines a map J: A2 ^ E and Lemma 1 implies that the inverse image J-1(J(A,B)) coincides with the associated set of the sequence (1). Let us denote by I(a,b) the set of equivalence classes in A2 of the form [(a,ft, 9)], which contain the members of J-1 (J(A,B)). If a = Pr(A), ft = Pr(B), 9 = Pr(A n B), then (A, B) is a Yule's pair of type (a,ft, 9), (A, Bc) is a Yule's pair of type (a, 1 - ft, a - 9), (Ac, B) is a Yule's pair of type (1 - a, ft, ft - 9), etc. Considering a, ft, 9 as coordinate functions in R3, the members of I(a,b) produce the set 64 consisting of 24 affine automorphisms of R3 from the following list:
?(1)(a, ft, 9) = (a, ft, 9), <P(12)(34) K 9) = K 1 - a - 9) <P(13)(24)(a, ft, 9) = (1 - a, ft, ft - 9), <P(14)(23) (a,ft, 9) = (1 - a, 1 - ft, 1 - a - ft + 9), P(23)(a, ft, 9) = (ft, a, 9), <P(1342) (a, ft, 9) = (ft, 1 - a, ft - 9),
<P(1243) K 9) = (1 - a a - 9^
<P(14)(a,ft,9) = (1 - ft,1 - a, 1 - a - ft + 9), <P(34) (a, ft, 9) = (a, 1 - a - ft + 29,9), ^(243) (a,ft, 9) = (1 - a - ft + 29, a, 9), <P(234) (a, ft, 9) = (ft,1 - a - ft + 29,9), <P(24) (a, ft, 9) = (1 - a - ft + 29, ft, 9), ^(142) (a,ft, 9) = (1 - a, 1 - a - ft + 29,1 - a - ft + 9), <P(i423)(a, ft, 9) = (1 - a - ft + 29,1 - a,1 - a - ft + 9), <P(i43)(a,ft,9) = (1 - ft,1 - a - ft + 29,1 - a - ft + 9), <P(1432)(a,ft, 9) = (1 - a - ft + 29,1 - ft, 1 - a - ft + 9), <P(12) (a, ft, 9) = (a, a + ft - 29, a - 9), ^(123) (a,ft, 9) = (a + ft - 29, a, a - 9), P(132)(a, ft, 9) = (ft, a + ft - 29, ft - 9), <P(13) (a, ft, 9) = (a + ft - 29, ft, ft - 9), <P(1324)(a, ft, 9) = (1 - a, a + ft - 29, ft - 9), <P(134)(a, ft, 9) = (a + ft - 29,1 - a, ft - 9), <P(124) (a, ft, 9) = (1 - ft, a + ft - 29, a - 9), ^(1234)(a,ft,9) = (a + ft - 29,1 - ft,a - 9).
The above affine automorphisms of R3 are indexed by the permutations a from the symmetric group S4 because of the theorem below.
The operator of symmetry
a: H ^ H, (£1, £2, £3, £4) ^ (1), £a"1 (2), £a"1(3), £a~1(4)),
permutes the components of the probability distribution [3, 4.1,(2)] produced by the experiment J( A,B) and we have
Theorem 1. (i) One has i o fa-1 = a o i.
(ii) The map
S4 ^ Aff(3, R), a ^ fa-1, (2)
is a group anti-monomorphism with image 64.
(iii) The group 64 is the affine symmetry group of the classification tetrahedron T3.
Proof. (i) It is enough to check the equality fa-1 = i-1 o a o i for all a € S4. For example, let a = (1243), so a-1 = (1342). We have
(a ◦ l)(a, J 0) = (£a-1(1), £a-1(2), £a-1(3), £a-1 (4)) = (£3, £1, £4, £2 ), (i-1 o a o i)(a, f, 0) = i-1 (£3, £1, £4, £2) = (f,1 - a, f - 0) = f(1342)(a, J 0) = fa-1 (a J 0).
(ii) The map (2) is injective; moreover, it is a group anti-homomorphism because f^ = i-1 o
(1) o i = (1) and fT-1a-1 = f(aT)-1 = i-1 o (ax) o i = i-1 o a o t o i = i-1 o a o i o i-1 o t o i = fa-1 o vt-1.
(iii) In accord with part (i), for any a € S4 we have i( fa(T3)) = a-1 (i(T3)) = a-1 (A3) = A3, hence (T3) = i-1 (A3) = T3. On the other hand, S4 is the symmetry group of the regular tetrahedron (see, for example, [5, 8.4]). Since both tetrahedrons are isomorphic as affine spans, the proof is done.
For any a € S4 we write down the affine automorphism in terms of coordinates in
)tain that q>a maps the components oi the corresponding components of the
R3: (a,f,0) = (a(a),f(a),0(a)) and obtain that maps the components of the partition
T3 = U(af)€[01]2 {a} x {f} x I (a, f) onto the corresponding components of the partition T3 = U(af)€[01]2{a(a)} x {f(a)} x I(a(a),f(a)). Moreover, maps the components of the partition T3 = U(af)€(01)2 {a} x {f} x I (a, f) onto the corresponding components of the partition T3 =
U(a,f)e(o,i)2{a(a)}x{f(a)}x I(a(a),f(a)).
Let us set T3 = U(a j) € (0i)2 {a} x {f} x I(a, f). In particular, we obtain the following
Lemma 2. Let (a, f) € (0,1)2, a € S4. (i) The automorphism maps the set
{(a, f, i(a, f)), (a, f, r(a, f))} of endpoints of the segments {a} x {f} x I (a, f) onto the set
{(a(a), f(a), i(a(a), f(a))), (a(a), f(a), r(a(a), f(a)))}
of endpoints of their images {a(a } x {f(a)} x I (a(a), f(a)). (ii) One has fa(T3) = T3.
In accord with Theorem 1, (ii), the group S4 acts on the real functions F: R3 ^ R via the rule a ■ F = F o fa-1. Let
G: A3 ^ R, G(£1, £2, £3, £4) = -£1 ln £1 - £2 ln £2 - £3 ln £3 - £4 ln £4, E: T3 ^ R, E = G o i.
The function G is continuously differentiable on the interior A3 and can be extended under the name G as continuous on A3 = i (T3). The function E is continuously differentiable on the interior T3 and can be extended under the name E as continuous on T3 (cf. [3, 5.1,Theorem 2, (iii)]). Moreover, G = G o a (that is, G is an absolute S4-invariant) and E = G o i. Lemma 2, (ii), allows us to extend the action of the symmetric group S4 on T3 via the rule a ■ E = E o fa-1.
Throughout the end of the paper, with an abuse of the language, we designate G via G and E via E.
Theorem 2. The function E: T3 ^ R is an (absolute) invariant of the symmetric group S4.
Proof. Theorem 1, (i), yields E = G o 1 = G o a o 1 = G o 1 o = E o = a ■ E for all a G S4.
4. DEgREE OF DEPENdANcE: FURTHER Properties
4.1. The Groups of Symmetry
Let us suppose (a, ft) G (0,1)2 and set
f__e(«,ftAft)-E(a,ft,9) if p(a ft) < 9 <aa
e(aft9)= I E(«,ft,«ft)-E(aM",ft)) if £(a, ft) < 9 < a ft (3)
e(a,ft,9) = i flfft»ft if aft < 9 < r(a,ft), (3)
^ E(a,ft,aft)-E(a,ft,r(a,ft)) " — — \ ' r/'
where I(a,ft) = [¿(a,ft),r(a,ft)]. Note that in [3, 5.2] the function ea^(9) = e(a ,ft,9) is said to be the degree of dependence of events A and B with a = Pr(A), ft = Pr(B), and 9 = Pr(A n B).
Let us consider the dihedral subgroup D8 = ((1342), (14)) of S4 and let x: D8 ^ R* be its Abelian character with kernel K = ((14), (23)) and image {1, -1}.
Theorem 3. The function e from (3) is a relative invariant of weight x of the dihedral group D8. Proof. Given a G S4 we have
(a-1 ■ e)(a, ft, 9) = e( ^ (a, ft, 9)) = e(a(a), ft(a), 9(a)) =
E(a(a),ft(a),a(a)ft(a))-E(«(a),ft(a),9(a)) fJ(Ja) a(a)\ s a(a) s Ja) a(a)
_'"W'*(a>ß(a))-E(«(a),ß(a),Q(a)) if lU(a) o(a))< Q(a) < a(a) ß(a)
E(a(a),ß(a),a(a)ß(*))-E(«(*),ß(*) /(«(a),ß(a))) ^ ' ß ) - Q " ß
■(a),ß(a),a(a) ß(a))-E(a(a),ß(a) /(a (a),ß(a.
E(a (a))ß(a))-E(a(a),ß(a);Q(a)) ' if a(") ß(a) < Q(a) < r(a(") ß("))
'(a) «(a) a(") «(a))_ E(a(a) «(a) ¿>(a (a) «("^ 11 a ß - Q - ' (a >'
E(a (a) ,ft(a),a(a) ft(a))-E(a(a) ,ft(a)/(a (a) ,ft(a)))
where I(a(a),ft(a)) = [i(a(a),ft(a)),r(a(a),ft(a))]. For any a G D8 we have ^(a,ft,aft) = (a(a), ft(a), a(a)ft(a)). On the other hand, given a G K, the inequalities a , ft) < 9 < a ft are equivalent to the inequalities £(a(a),ft(a)) < 9(a) < a(a)ft(a) and the inequalities aft < 9 < r(a,ft) are equivalent to the inequalities a(a)ft(a) < 9(a) < r(a(a),ft(a)). Given a G D8\K, the inequalities ¿(a,ft) < 9 < aft are equivalent to the inequalities a(a)ft(a) < 9(a) < r(a(a),ft(a)) and the inequalities a ft < 9 < r(a, ft) are equivalent to the inequalities £(a(a), ft(a)) < 9(a) < a(a) ft(a). The corresponding equalities hold simultaneously because of Lemma 2, (i). Now, Theorem 2 yields that a ■ e = x(a)e for all permutations a G D8.
■
We obtain immediately the following Corollary 1. For any (a, ft) G (0,1)2 and for any 9 G I(a, ft) one has
e«,ft (9) = eftA(9) = e!_a,1_ft (1 - a - ft + 9) = e!_ft,1_a (1 - a - ft + 9), -e«(9) = ea,1-^(a - 9) = e^»(ft - 9) = e^-«(ft - 9) = e^^ (a - 9).
AckNOwlEdgEMENTs I would like to thank the referees for their very useful remarks.
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Deciaration of CoNFlicTiNg Interests The Author declares that there is no conflict of interest.
References
[1] Bochnak J., Coste M., Roy M-F. (1998). Real Algebraic Geometry, Springer.
[2] Dieudonne J. A., Carrell J. B. (1970). Invariant Theory, Old and New, Academic Press, Inc.
[3] Iliev V. V. (2021). On the Use of Entropy as a Measure of Dependence of Two Events. Reliability: Theory & Applications, 16(2): 237 - 248.
[4] Kolmogorov A. N. (1956). Foundations of the Theory of Probability, Chelsea Publishing Company, New Yourk.
[5] Smith J. T. (2000). Methods of Geometry, John Wiley & Sons.
