CC BY
28
UDC 530.16:51-73
TOWARDS UNIQUE PHYSICALLY MEANINGFUL DEFINITIONS OF RANDOM AND TYPICAL OBJECTS
L. Longpre, O. Kosheleva
To distinguish between random and non-random sequence, Kolmogorov and Martin-Lof proposed a new definition of randomness, according to which an object (e.g., a sequence of Os and Is) if random if it satisfies all probability laws, i.e., in more precise terms, if it does not belong to any definable set of probability measure 0. This definition reflect the usual physicists’ idea that events with probability 0 cannot happen. Physicists - especially in statistical physics - often claim a stronger statement: that events with a very small probability cannot happen either. A modification of Kolmogorov-Martin-Lof’s (KLM) definition has been proposed to capture this physicists’ claim. The problem is that, in contrast to the original KLM definition, the resulting definition of randomness is not uniquely determined by the probability measure: for the same probability measure, we can have several different definitions of randomness. In this paper, we show that while it is not possible to define, e.g., a unique set R of random objects, we can define a unique sequence Rn of such sets (unique in some reasonable sense).
1. Definitions of Random and Typical Objects: Reminder
Some sequences are random, some are not. Traditional probability theory does not allow us to distinguish between random and non-random numbers, sequences, etc., it only allows us to determine the probability that, e.g., a random sequence of Os and Is - corresponding to flipping a fair coin - belongs to a given set of sequences. However, from the physical viewpoint, some sequences of Os and Is are random - in the sense that they can appear as a result of the actual coin tosses - while other cannot. For example, it is intuitively clear that a periodic sequence 0101... cannot result from the actual coin tosses.
Kolmogorov-Martin-Lof randomness. To formalize the above intuitive notion, A. N. Kolmogorov and P. Martin-Lof proposed the following idea. How do we know that a sequence is not random? For example, a sequence 00... (consisting of all 0s) is not random because we know that:
Copyright © 2012 L. Longpre, O. Kosheleva
University of Texas at El Paso (USA)
E-mail: longpre0utep.edu, olgak0utep.edu
• for almost all sequence, the proportion of Os among the first n symbols tends to 1/2 as n —— to, while
• for the sequence 00..., the proportion of Os is equal to 1.
Similarly, a periodic sequence 0101... is not random because:
• for almost all sequence, the frequency of 11 tends to 1/4, while
• for the periodic sequence 0101..., this frequency is 0.
Frequency limit is a simple case, we may have more complex reasons why a given sequence is not random.
In general, it is reasonable to say that a sequence is random if it satisfies all the
probability laws, i.e., all the statements (defined in a certain language L) which
are true for almost all sequences. To be more precise, a probability law on the set X of all sequences is an L-definable subset S C X for which P(S) = 1 - or, equivalently, for whose (similarly definable) complement — S, we have P(—S) = 0. So, we arrive at the following definition.
Definition 1. We say that an object x e X is random in the sense of
L
0.
Remark 1. Definable simply means that there is a formula in the corresponding language that uniquely determines the object. For example, in the usual theory of real numbers, 1, 2, n, y/2, e, any number that can we thing of is definable. This does not means, of course, that every real number if definable: in each language L
L
which are not definable.
L
L
L
in particular, define the smallest integer that cannot be described by fewer than 100 words. By definition, this integer cannot be described by fewer than 100 words, but the above description “the smallest integer that cannot be described by fewer than 100 words” is a description of this integer in fewer than 100 words - a contradiction.
L
Li in which L-definability can be formally described. For each language L, the existence of such a stronger language Li can be easily proven; see, e.g., [2].
Almost all sequences are Kolmogorov-Martin-Lof random. As we have
L
L
describe all sequence which are Kolmogorov-Martin-Lof random, we delete, from
XL sets of measure 0. A union of countably many set of measure 0 is also of measure 0, so almost all sequences are random in this sense.
Towards a more physically adequate notion of randomness. Kolmogorov -Martin-Lof definition prevents sequences 00... from being called random. This prevention makes perfect physical sense: if we flip a coin and every time get tail, this clearly is not a fair coin.
However, one can easily check that for each random sequence u, a sequence
0... 0 (1,000,000 times) followed by u is also random in the sense of Kolmogorov -Martin-Lof. This is not physically meaningful: it is clearly not realistically possible to toss a fair coin million times and get tail every time. Physicists usually argue that this situation is not physical because its probability is too low - in the above example, this probability is equal to 2-1,000,000, This argument is behind the usual applications of statistical physics: e.g., from the purely mathematical viewpoint, it is possible that all the randomly moving molecules of a human body will start going into the same vertical direction, and the person will float on air - but the probability of this event is so small, then it is not physically possible.
This idea cannot be described by simply setting a small threshold p0 c 1 and claiming that no event with a probability < p0 is possible: indeed, for a sequence of n coin tosses, every sequence of 0s and Is has the same probability 2-n; so, for a sufficiently large n, for which 2-n < p0, we would arrive at a conclusion that no such sequence is possible at all. In other words, the threshold p0 is not a universal constant, it depends on the property.
In particular, if we are looking for the impossibility to have too many 0s at the beginning, then we have a sequence of sets
An = {u starts with n zeroes} = {u : ui = ... = un = 0}
for which Ai D A2 D ... D An D ... and P(An) — 0 as n — to. In general, we have a definable sequence of sets An for which An D An+i and P(An) — 0. Our claim is that for each such sequence, there is an N for which P(AN) is so small that a truly random sequence cannot belong to the set AN. In other words, we arrive at the following definition of the class R of random sequences:
Definition 2. We say that a non-empty set R C X is a set of random objects if for every definable sequence An for which An D An+i and P(An) — 0, there exists an N for which AN n R = 0.
Discussion. The existence of such a set follows if we consider a language in L
contrast to the Kolmogorov-Martin-Lof definition, we cannot have a set of random elements R for whi ch P (R) = 1. However, we can have a set of random elements which is “almost” of measure 1 [2]. To formulate this result, let us recall that
S
define:
• an inner measure P(S), defined as the supremum of all the values P(S') for all measurable sets S' C S, and
• an outer measure P(S), defined as the infimum of all the values P(S') for all measurable sets S' D S.
The existence result is that for every e > 0, we can have a set of random elements R for whi ch P (R) > 1 — e.
From random to typical. In the above examples, we assume that we know the probability distribution on the set of all possible objects. In some cases, physicists talk about “typical” objects even when no such probability distribution is known. For example, they argue that since “almost all” cosmological solutions of General Relativity theory have a certain asymptotic behavior, the actual University must exhibit the same behavior; see, e.g., [3]. The possibility of such conclusions comes from the fact that if An D An+i and nAn = 0, then P(An) — 0 no matter
P
Definition 3. We say that a non-empty set T C X is a set of typical objects if for every definable sequence An for whi ch An D An+i and nAn = 0, there exists N AN n T = 0
2. Formulation of the Problem
In contrast to the Kolmogorov-Martin-Lof definition, the new notions are not uniquely determined. The problem dealt with in this paper comes from the fact that the above physically adequate notions of random and typical objects are not uniquely determined.
Indeed, in the Kolmogorov-Martin-Lof’s definition, once we know the probabil-PL
RT
Definitions 2 and 3.
Let us show that this non-uniqueness is inevitable. Indeed, let us consider a
P
D = {(x,y) : x2 + y2 < 1}.
This probability distribution is invariant w.r.t. arbitrary rotations around the disk’s center (0,0). So, any definable rotation preserves the situation. Thus, if we had a
R
such rotations. It turns out that such an invariance is not possible:
X D P
PC X be a set of random objects. Then, there exists a definable rotation under which the set P is not invariant.
Remark 2. For readers’ convenience, all the proofs are placed in the special (last) Proofs section.
Possible reason for non-uniqueness. A possible reason for non-uniqueness is that, as we have mentioned earlier,
• while we cannot have a set of random elements R with P(R) = 1,
• for every e > 0, we can have a set of random elements R for which P(R) > 1 — e.
Thus, to capture the general intuitive meaning of randomness, it is not enough to have one set of random elements, it is more appropriate to have a sequence of such
e
This is what we will do in this paper.
3. Definitions and the Main Result
Comment about languages. As we have mentioned, we start with a language
L
Li L Li
definable sets of random elements and sets of typical elements [2]. We want to Li
L2
Li Li L
Li
definable sets of random elements Rn for whi ch Rn C Rn+i.
Li
definable sets of typical elements Tn for whi ch Tn C Tn+i.
When is a description universal? When are two descriptions equivalent? To answer these questions, we can take into account that every non-empty subset of a set of random elements is also a set of random elements, and that every non-empty subset of the set of all typical elements is also a set of typical elements:
Lemma 1. If R is a set of random elements, and S is a non-empty subset of R, then S is also a set of random elements.
T S
TS
Remark 3. The proofs follow directly from Definitions 2 and 3. Now, we can formally describe universality and equivalence.
Rn
Li R n R C Rn
Tn
Li T n T C Tn
Definition 8. We way that two descriptions of randomness Rn and R'n are equivalent if the following two conditions hold:
• for every natural number n, there exists n' for whi ch Rn C R'nr, and
• for every natural number n', there exi sts n for whi ch R'n, C Rn,
Tn Tn'
alent if the following two conditions hold:
• for every natural number n, there exists n' for whi ch Tn C 7^; anc*
• for every natural number n', there exi sts n for whi ch T'n, C Tn.
Proposition 2. There exists a universal description of randomness, and this description is unique modulo equivalence.
Proposition 3. There exists a universal description of typicality, and this description is unique modulo equivalence.
Remark 4. In other words:
• there exists a universal description of randomness, and all universal descriptions of randomness are equivalent to each other, and
• there exists a universal description of typicality, and all universal descriptions of typicality are equivalent to each other.
The proof of Propositions 2 and 3 is based on the following lemmas:
X R' R''
their union R = R' U R'' is also a set of random elements.
Lemma 4. For every set X, if T' and T'' are sets of typical elements, then their union T = T' U T'' is also a set of typical elements.
4. Proofs
PCD
d e D can be described by its polar coordinates (r, 0), where 0 < r < 1 is the distance from this point to (0,0), and 0 e [0,2n) is the angle between the direction 0d d x
An
An = {(r,0) : |0|< 2-n}.
Then, clearly, An D An+1 and nAn = 0, so P(An) — 0. Thus, by definition of the
N AN n P = 0
words, no point (r, 0) e P can have a value |0| < 2-N.
P
P
to arbitrary definable rotations, then it would be invariant with respect to rotation by 2-n radians, by 2 ■ 2-N radians, etc. Since no points with |0| < 2-N are in the set P, by rotating by 2-N, we conclude tot no poi nts with 0 e [0,2 ■ 2-N ] can be in this set, then that no points with 0 e [2-N, 3 ■ 2-N] can be in this set - etc.,
0P
an empty set. The proposition is proven.
Proof of Lemma 3. To prove that the union R = R' U R'' is a set of random
An An D
An+1 and P(An) — 0. We need to prove that there exists an integer N for which
AN n R = 0
R' N'
which An nR' = 0. Since m < n implies Am D An, we conclude that AN nR' = 0 for all N > N'.
R'' N''
which AnnnR'' = 0. Sinee m < n implies Am D An, we conclude that ANnR'' = 0 for all N > N''.
For N = max(N', N ''), we ha ve N > N 'and N > N'' and henc e, AN n R' = 0 AN n R'' = 0 AN R' R''
AN R = R' U R'' AN n R = 0
proven.
Proof of Lemma 4. To prove that the union T = T' U T'' is a set of typical
An An D
An+1 and nAn = 0. We need to prove that there exists an integer N for which
AN n T = 0
T' N'
which An/ n T' = 0. Sinee m < n implies Am D An, we conclude that AN n T' = 0 for all N > N'.
T'' N''
which An// nT'' = 0. Sinee m < n implies Am D An, we conclude that AN nT'' = 0 for all N > N''.
For N == max(N', N''), we have N > N' and N > N'' and hence, AN n T' = 0 AN n T'' = 0 AN T' T''
AN T = T' U T'' AN n T = 0
proven.
Proof of Proposition 2. Let us first prove that every two universal descrip-Rn R'n n
by definition of a description of randomness, the set Rn is a indefinable set of
R'n
a natural number n' for which Rn C R^. Similarly, we can prove that for every n', there exists a natural number n for whi ch Rn C R^. Equivalence is proven.
Let us now prove existence of a universal description of randomness. Indeed, in the language i2, we can formally describe L1-definability, so we can have a sequence R^), R(2), ..., of all indefinable sets of random elements. Now, we can take
d ef
Rn = R(i) U ... U R(n).
Each of these sets is also indefinable. Due to our construction, we have Rn C
Rn+1, so this is indeed a description of randomness.
Let us prove that this description is universal. Indeed, each indefinable description of randomness R is in the sequence R^), R(2), ..., so it has the form
R = R(n) for some n. Then, by definition of Rn, we have R = R(n) C Rn.
Universality is proven, and so it the proposition.
Proof of Proposition 3 is similar.
References
1. Li M., Vitanyi P. An Introduction to Kolmogorov Complexity and Its Applications. Berlin, Heidelberg, New York : Springer, 2008.
2. Kreinovich V., Finkelstein A.M. Towards applying computational complexity to foundations of physics // Notes of Mathematical Seminars of St. Petersburg Department of Steklov Institute of Mathematics. 2004. V. 316. P. 63-110; reprinted in Journal of Mathematical Sciences. 2006, V. 134, N. 5, P. 2358-2382.
3. Misner C.W., Thorne K.S., Wheeler J.A. Gravitation. New York : W. H. Freeman, 1973.
