Solution for a class of stochastic coalitional games Текст научной статьи по специальности «Математика»

Solution for a Class of Stochastic Coalitional Games

Xeniya Grigorieva

St.Petersburg State University,

Faculty of Applied Mathematics and Control Processes, University pr. 35, St.Petersburg, 198504, Russia E-mail: [email protected] WWW home page: http://www.apmath.spbu.ru/ru/staff/grigorieva/

Abstract The stochastic game r under consideration is repetition of the same stage game G which is played on each stage with different coalitional partitions. The probability distribution over the coalitional structures of each stage game depends on the initial stage game G and the n-tuple of strategies realized in this game. The payoffs in stage games (which is a simultaneous game with a given coalitional structure) are computed as components of the generalized PMS-vector (see (Grigorieva and Mamkina, 2009), (Petrosjan and Mamkina, 2006)). The total payoff of each player in game r is equal to the mathematical expectation of payoffs in different stage games G (mathematical expectation of the components of PMS-vector). The concept of solution for such class of stochastic game is proposed and the existence of this solution is proved. The theory is illustrated by 3-person 3-stage stochastic game with changing coalitional structure.

Keywords: stochastic games, coalitional partition, Nash equilibrium, Shap-ley value, PMS-vector.

Introduction

This paper belongs to the well investigated direction in management theory - game theory which deals with existence and finding optimal solution problems of management in a collision of parties, when each party tries to influence development of the conflict in accordance with its own interest. Problems that arise in different practical spheres are solved according to the game theory models. In particular, these are the problems of management, economics, decision theory.

In the paper a class of multistage stochastic games with different coalitional partitions is examined. A new mathematical method for solving stochastic coali-tional games, based on calculation of the generalized PMS-value introduced in (Grigorieva and Mamkina, 2009), (Petrosjan and Mamkina, 2006) for the first time, is proposed along with the proof of the solution existence. The probability distribution over the coalitional structures of each stage game depends on the initial stage coalitional game and the n-tuple of strategies realized in this game. The theory is illustrated by 3-person 3-stage stochastic game with changing coalitional structure.

Remind that coalitional game is a game where players are united in fixed coalitions in order to obtain the maximum possible payoff, and stochastic game is a multistage game with random transitions from state to state, which is played by one or more players.

1. Statement of the problem

Suppose finite graph tree Г = (Z, L) where Z is the set of vertices in the graph and L is point-to-set mapping, defined on the set Z: L (z) С Z, z G Z. Finite graph tree with the initial vertex z0 will be denoted by Г (zo).

In each vertex z G Z of the graph Г (z0) simultaneous N -person game is defined in a normal form

G (z) = (N,X1,...,Xn, Kx,..,Kn) , (1)

where

* N = {I, ..., n} is the set of players identical for all vertices z G Z;

* X, = { xj\xj = k, к = 1, rtij } is the set of pure strategies of player j G N identical for all vertices z G Z;

— m, is the number of pure strategies of player j G N in the set X,;

— xz is the pure strategy of player j G N at the vertex z G Z;

— fj,j = < fj?k> __________________G Sj is the mixed strategy of player j G N at the vertex

k — 1 ,m j

z G Z where j is a choice probability for the j-th player to pick k-th pure strategy:

. _ _ тз .

3 = 1, n, к = 1, nij , E Mi = !;

k—1

— S, is the set of all mixed strategies of j-th player;

— xz = (xf, ..., xz) G X, xz G Xj, j = 1, n, is the n-tuple of pure strategies in the game G (z);

— X = П Xi is the set of n-tuples identical for all vertices z G Z;

i= 1, П

— Ijlz = (/xf, ..., /t^) G £, /t| € Sj, j = 1, n, is the n-tuple of mixed strategies in the game G (z) in the mixed strategies at the vertex z G Z;

— S = П S, is the set of n-tuple in the mixed strategies identical for all

j = l,n

vertices z G Z;

* K,(xz) , : xz G X, is the payoff function of the player j, identical for all vertices z G Z ; it is proposed that K, (xz) > 0 V xz G X and V j G N.

Furthermore, let in each vertex z G Z of the graph Г (z0) the coalitional parti-

tion of the set N be defined

i

Sz = {Si, ...,Si} , l < n,Si n S, = 0 V i = j U Si = N,

i—1

i. e. the set of players N is divided into l coalitions each acting as one player. Coalitional partitions can be different for different vertices z, i. e. l = l (z) .

Then in each vertex z G Z we have the simultaneous l-person coalitional game in a normal form associating with game G (z)

G (z, Sz ) = (N,XzSi ,...,XzSi ,HzSi ,..,HZS) , (2)

where

* xzs. = П Xj is the set of strategies xzSi of coalition S\,i = 1, /, where the

" jeSi S

strategy xzSi G X7^ of coalitian Si is n-tuple strategies of players from coalition Si, i. e. xzSi = { x, G X, | j G S^;

— xz = (xzSi, ...,Xg ) G Xz, G * = 1, /, is n-tuple of strategies in the

game G (z, Sz)^

— Xz = n Xzg. is the set of n-tuples in the simultaneous game G (z , Sz);

i=Tj

— fiz is the mixed strategy of coalition S\ , i = 1, I, at the vertex z G Z;

— Sz is the set of all mixed strategies of coalition S\ , i = 1, I, at the vertex

zg2; ____

— fiz = (p,\, fif) G , /if G X1?, * = 1, I, is the n-tuple of mixed strategies

in the game G (z) in the mixed strategies at the vertex z G Z;

— ^z = El is the set of n-tuple in the mixed strategies at the vertex z G Z;

i=ij

* the payoff of coalition S\ is defined as a sum of payoffs of players from coalition Si, i. e.

mi(xz) = '£,Kj(x), * = 177,

jeSi

where xz = (xf, ..., xzn) and xz = {xzSi, ... , xzSl) are the same n-tuples in the games G (z) and G (z , Sz) correspondingly such that for each component xz, j =

1, n, from the n-tuple xz it follows that the component xz, j G Si, is included in the strategy xzs from the n-tuple xz.

Denote by only xz the n-tuple in the games G (z) and G (z, Sz). However it does not lead to ^z = pz.

After that for each vertex z G Z of the graph r (zo) the transition probabilities p (z, y; xz) to the next vertices y G L (z) of the graph r (z0) are defined. The probabilities p (z, y; xz) depend on n-tuple of strategies xz realized in the game G (z , Sz) with fixed coalitional partition:

p (z, y ; xz) > 0 ,

E p(z, y; xz) = 1.

yeL(z)

Definition 1. The game defined on the finite graph tree r (z0) with initial vertex zo will be called the finite step coalitional stochastic game r (zo):

r (z0)= (yN, r (z0) , {G (z, ^z )}zeZ , {p (z,y ; xZ )}zeZ,yeL(z) ,x*EXZ , k_T^(3)

where

— N = {1, ..., n} is the set of players identical for all vertices z G Z;

— r (z0) is the graph tree with initial vertex z0;

— {G (z, Sz)} z£z is the simultaneous coalitional /-person game defined in a normal form in each vertex z G Z of the graph r (z0) ;

— {p (z, y; xz)}zeZ yeL(z) xzeXZ is the realization probability of the coalitional game G (y, Sy) at the vertex y G L (z) under condition that n-tuple xz was realized at the previous step in the simultaneous game G (z, Sz);

— kp is the finite and fixed number of steps in the stochastic game r (z0); the k-th step in the vertex zk G Z is defined according to the condition of zk G (L (z0))k,

i. e. the vertex zk is reached from the vertex z0 in k steps.

States in the multistage stochastic game r are the vertices of graph tree z G Z with the defined coalitional partitions Sz in each vertex, i. e. pair (z, Sz). Game r

is stochastic, because transition from state (z , Sz) to the state (y, Sy), y G L (z), is defined by the given probability p (z, y ; xz).

Game r (z0) is realized as follows. Game r (z0) starts at the vertex z0, where the game G (z0 , Szo) with a certain coalitional partition Szo is realized. Players choose their strategies, thus n-tuple of game xz0 is formed. Then with given probabilities p (z0, zf; xz0) depending on n-tuple xz0 the transition from vertex z0 on the graph tree r (z0) to the game G (zf, Sz 1) , zf G L (z0), is realized. In the game G (zf, Sz 1) players choose their strategies again, n-tuple of game xz 1 is formed. Then from vertex zf G L (z0) the transition to the vertex z2 G (L (z0))2 is made, again n-tuple game xz2 is formed. This process continues until vertex zkf, G (L (z0))fcf , L (zkp) = 0 is reached.

Denote by r (z) the subgame of game r (z0), starting at the vertex z G Z of the graph r (z0), i. e. at coalitional game G (z , Sz). Obviously the subgame r (z) is also a stochastic game.

Denote by:

— uz (•) is the strategy of player j, j = 1, n, in the subgame r (z) which to each vertex y G Z assigns the strategy xjy of player j in each simultaneous game G (y, Sy) at all vertices y G r (z) i. e.

uz (') = {xy I yG r (z)};

— uzSi (•) is the strategy of coalition Si in the subgame r (z) which is a set of strategies uz (•) , j G Si;

— uz (•) = (uf (•) , ... , uzn (•)) = (uzSi (•) , ... , uzSn (•)) is the n-tuple in the game r (z).

It’s easy to show that the payoff Ez (uz (•)) of player j, j = 1, n, in any game r (z) is defined by the mathematical expectation of payoffs of player j in all its subgames, i. e. by the following formula (Zenkevich et al., 2009, p. 158):

EZz (uz (•)) = Kj (xz) + £ [p (z,y ; xz) Ey (uy (•))] . (4)

yeL(z)

Thus, a coalitional stochastic game r (z0) can be written as a game in normal form

r (z0) =

(n, r(z0),{G(z, Zz)}zez, {p(z, y; xz)}zeZ, , {Uz} {£??} *fV,5)

\ yeL(z) ’ ’ /

where Uz is the set of the strategies uz (•) of the player j, j = 1, n.

The payoff Hg (xz) of coalition S\ Є Sz, і = 1, /, in each coalitional game G (z , Sz) of game Г (z0) at the vertex z Є Z in all n-tuple xz is defined as the sum of payoffs of players from the coalition Si:

Hk (xz )= E Kj (xz). (6)

jeSi

The payoff Hg. (uz (•)) , Si Є Sz, і = 1, /, in the subgame Г (z) of the game Г (zo) at the vertex z Є Z is defined as the sum of payoffs of players from the coalition Si in the subgame Г (z) at the vertex z Є Z:

HI (uz (■)) = E Ez (uz (■)) = E\Kj (xz) + E [p (z,y ; xz) Ey (uy (■))] 1(7)

jeSi jeSi [ yeL(z) )

It’s clearly, that in any vertex z Є Z under the coalitional partition Sz the game Г (z) with payoffs Ez of players j = 1, n defined by (4), is a non-coalitional game between coalitions with payoffs Hg (uz (■)) defined by (7). For non-coalitional games the existence of the NE (Petrosjan et al., 1998, p. 137) in mixed strategies is proved.

Remind that the Nash equilibrium (NE) is n-tuple uz (■):

HzSi («z (')) > (uz (•) || u*Si (•)) V u*Si (•) Є Ug., У Si e Sz, і = hi,

where Ug is the set of the strategies uzs (•) of coalition S\ Є Sz, і = 1, I, and u (■) II uzS^ (■)) means that the coalition Si deviates from the n-tuple uz (■) choosing a strategy uzS. (■) instead of strategies uzS, (■) Є uz (■).

However, as the payoffs of players j, j = 1, n, are not selected from the payoff of coalition in the subgame Г (z), it may occur at the next step in the subgame Г (y) , y Є L (z) , with another coalitional partition at the vertex y, the choice of player j is not trivial and is different from the corresponding choice of entering into an equilibrium strategy uz (■) in the subgame Г (z).

So, solving the coalitional stochastic subgame Г (z) means forming the NE uz (■) in the subgame Г (z) taking into account the presence of coalition structures in the subgames included in the subgame Г (z) in particular, by calculating the PMS-vector of payoffs of players in all subgames included in the subgame Г (z).

Formulate follow problem: it’s required to solve coalitional stochastic game Г (zo), i. e. to form n-tuple of NE uz (■) in the game Г (z0) by using the generalized PMS-vector as the optimal solution of coalitional games.

2. Nash Equilibrium in a multistage stochastic game

Remind the algorithm of constructing the generalized PMS-value in a coalitional game. Calculate the values of payoff Hg. (xz) for all coalitions S\ Є Sz, і = 1, /, for each coalitional game G (z , Sz) by formula (6):

Hk (xz) = E Kj (xz) .

jeSi

In the game G (z , Sz) find n-tuple NE xz = {xzSi,..., xzSi) or /2z = {/zSl,..., /Si). In case of l = 1 the problem is the problem of finding the maximal total payoff

of players from the coalition Sf, in case of l = 2 it is the problem of finding of NE in bimatrix game, in other cases it is the problem of finding NE n-tuple in a non-coalitional game. In the case of multiple NE (Nash, 1951) the solution of the corresponding coalitional game will be not unique.

The payoff of each coalition in NE n-tuple HS. (pz) is divided according to Shapley’s value (Shapley, 1953) Sh (Si) = (Sh (Si : 1), ...,Sh(Si : s)):

^ ('s'-l') l (s-sr) l ____

Sh (Si '■ j) = ^2 -----------^---------[w(S") - w(S"\{j})] Vj = l, s, (8)

S'cSi

S'3j

where s = |Si| (s' = |S'|) is the number of elements of set S\ (S') and v (S') is the

total maximal guaranteed payoff of subcoalition S' C Si. We have

s

v (Si) = £; Sh (Si : j).

j=i

Then PMS-vector in the NE in mixed strategies in the game G (z , Sz) is defined

as

PMSj) = (PMSf (jz) ,..., PMS„ (jz)) ,

where

PMSj (pz) = Sh (Si : j) ,j G Si} * = 1,1.

We proceed to the construction of solutions in the game r (z0).

Step 1. Calculate PMS-vector in NE in the mixed strategies for all coalitions Si € Sz, i = 1, /, of each coalitional game G (z , Sz), L (z) = 0:

PMS (z) = (PMSf (z) ,..., PMS„ (z)) ,

where PMS (z) := PMS (j2z) and PMSj (z) := PMSj (j2z) are PMS-vector and components of PMS-vector accordingly in one step coalitional game G (z, Sz),

L (z) = 0.

Step 2. Consider from the end of game r (z0) all possible two stage subgames

r (z) , y G L (z) , L (y) = 0 , with payoffs of coalitions

HzSi (uz ())=E \Kj (xz)+ E [p (z,y ; xz) PMSj (y)]

jeS^ y£L(z)

V Si G Sz, i = I, I . Find the NE xz or jlz and uz (•). Calculate PMS-vector in NE for all coalitions Si G Sz, i = 1, /, of each coalitional stochastic subgame r(z) , y G L (z) , L (y) = 0 :

PMS (z) = (PMSi (z) ,PMS„ (z))

where PMS (z) \= PMS (uz (•)) and PMSj (z) := PMSj (uz (•)) are PMS-vector and its components accordingly in the coalitional stochastic game r (z), L (z) = 0.

Step k. Define the operator PMS® as PMS-vector, which for each player j =

1, n in any coalitional game G (z, Sz) of subgame P (z) , y G [L (z)]k 1 , L(y) =

0, correspondingly PMS-components in the NE uz (■) :

PMSj (z) = PMS© I Kj (xz) + E [p(z’z'’ xZ) <>')] j =T7^,(9)

[ z'eL(z) )

where PMSj (z ) := PMSj (uz (•)), PMSj (z1) := PMSj (V' (•)) etc.

3. Examples

Example 1. Let there be 3 players in the game each having 2 strategies, and let payoffs of each player in all game n-tuples be defined, see table 1. Consider all possible combinations of coalitional partition, cooperative and non-coalitional games.

Tablel.

The strategies The payoffs The payoffs of coalition

I II III I II III (I, II) (II, III) (I, III) (I, II, III)

1 1 1 4 2 1 6 3 5 7

1 1 2 1 2 2 3 4 3 5

1 2 1 3 1 5 4 6 8 9

1 2 2 5 1 3 6 4 8 9

2 1 1 5 3 1 8 4 6 9

2 1 2 1 2 2 3 4 3 5

2 2 1 0 4 3 4 7 3 7

2 2 2 0 4 2 4 6 2 6

1. Solve coalitional game G (S1), X1 = {Si = {I, II} , N\S1 = {III}}, by calculating PMS-value: PMS1 = 2f; PMS2 = 2f; PMS3 = 2± .

2. Solve coalitional game G (X2), = {S2 = {II, III}, N\S2 = {I}} , by calculating PMS-value in pure strategies: PMS-]_ = PMS2 = PMS3 = 3.

3. Solve coalitional game G (X3), = {S3 = {I, III} , N\S3 = {II}}: PMS1 =

= 2.59; PMS2 = 2.5; PMS3 = 2.91.

4. Solve cooperative game G (X4), = {N = {I, II, III}}, see table 2. Find the

maximum payoff of coalition N and divide it according to Shapley’s value:

Shi = ~ [v {I, II} + v {I, III} - v III} - v {III}] + - [v {I, II, III} - v {II, III} + v {I}] ;

6 3

Sh2 = I [v {II, 1} + V {II, III} - V {1} - V {III}]h4 [v {I, II, III} - V {I, III} + V {II}] ; 63

Sh3 = hv {III, 1} + v {III, II} - v {1} - v {11}] + ^ [v {I, II, III} - v {I, II} + v {III}] . 63

Table2.

The strategies The payoffs The payoffs of coalition Imputation of the maximal coalition payoff proportional by Shapley’s vector

I II III I II III Hs (I, II, III) AiHs \2tls XsHs

111 4 2 1 7 2.5 3.5 3

112 12 2 5

12 1 3 15 9

12 2 5 13

2 11 5 3 1

2 12 12 2 5

2 2 1 0 4 3 7

2 2 2 0 4 2 6

Find maximal guaranteed payoffs:

v {I, II} = max {4, 3} = 4; v {I, III} = max {3, 2} = 3; v {II, III} = max {3, 4} = 4;

v {I} = max {1, 0} = 1; v {II} = max {2, 1} =2; v {III} = max {1, 2} = 2 . Then

Sh{2' h 1} = Sh[1,2,2) = Sh2>1) = - + - + - [9 -4] + - = - + - + - + - = 2-,

1 1 1 3 6 3 L J 3 3 6 3 3 2

Sh%’h1} = Sh%’2’2) = Sh%’2,1) = - + - + -[9-3]+ - = - + - + - + -= 3-,

2 2 2 2 3 3 3 2 3 3 3 2

Sh<2' 1} = Shg'2'2) = Shi1'2' 1) = | + | + |[9-4] + | = | + | + | + |=3.

5. Solve the non-coalitional game G (£5), S5 = {Si = {I}, S2 = {II},

S3 = {III}} . NE does not exist in pure strategies.

Use maximal guaranteed payoffs calculated in item 4

v {I} = 1; v {II} = 2; v {III} =2 .

Find optimal strategies according to Nash arbitration scheme (Grigorieva, 2009), see table 3, where ” — ” means that strategies are not optimal according to Pareto, but ” + ” - are optimal according to Pareto. Then we have optimal n-tuples (1, 1, 2) and (2, 1, 2) which provide identical payoff (1, 2, 2) in both n-tuples.

Conclusion

* For Si = {Si = {I, II} , N\Si = {III}} we have payoff ((2.71, 2.43), 2.33).

* For S2 = {s2 = {II, III} , N\S2 = {I}} we have payoff (3, (3, 3)).

* For = {s3 = {i, III}, N\S3 = {Ii}} we have payoff (2.59, (2.5), 2.91).

* For S4 = {N = {I,II, III}} we have (2.5, 3.5, 3).

* For S5 = {Si = {I} , S2 = {II} , S3 = {III}} we have optimal payoff (1, 2, 2) in n-tuples (1, 1, 2) and (2, 1, 2).

Example 2. In example 1 we considered the game with 3 players, each having

2 strategies, see table 1. Consider the following stochastic game according to the example 1, see figure 1.

On the graph shown in the figure 1, transition probabilities of passing from one game to another are shown, moreover (pi, p2, p3) is determined by the table 4.

The strategies The payoffs

I II III I II III

1 1 1 4 2 1 (4- 1) (2 - 2) (1 - 2) < 0 -

1 1 2 1 2 2 (1- 1) (2 - 2) (2 - 2) = 0 +

1 2 1 3 1 5 (3- 1) (1 - 2) (5 - 2) < 0 -

1 2 2 5 1 3 (5- 1) (1 - 2) (3 - 2) < 0 -

2 1 1 5 3 1 (5- 1) (3 - 2) (1 - 2) < 0 -

2 1 2 1 2 2 (1- 1) (2 - 2) (2 - 2) = 0 +

2 2 1 0 4 3 (0- 1) (4 - 2) (3 - 2) < 0 -

2 2 2 0 4 2 (0- 1) (4 - 2) (2 - 2) < 0 -

'll "2.71" '2.5' '2.59'' "2 .S'! v "3" '2.5s

2 2.43 3.5 2 2.5 3.5 2 3 3.5

X 2 33 sf J 2.91 \ J UJ A A S&J

noncoal <% coop noncoal coop c* noncoal C' coop

GSl

(h2> 2)

Figurel.

Remark. Frankly speaking, there must be 5 such tables (according to the number of games G, from which stochastic game can begin), and transition probabilities from one game to another at different steps may differ. In the given example we would use data in the table 4 for all games G, and we would consider them to be constant at each step. Payoffs of players in each game G are obtained in the example

1.

Table4.

The strategies The payoffs The probabilities

I II III I II III G^noncoal GSl Gs2 Gs3 Gcoop

1 1 1 4 2 1 0.8 0.1 0 0 0.1

1 1 2 1 2 2 0.1 0 0 0.8 0.1

1 2 1 3 1 5 0.7 0 0.1 0 0.2

1 2 2 5 1 3 0 0.1 0.3 0.6 0

2 1 1 5 3 1 0.8 0.1 0 0 0.1

2 1 2 1 2 2 0.25 0 0 0.25 0.5

2 2 1 0 4 3 0.1 0 0.7 0 0.2

2 2 2 0 4 2 0 0.25 0.5 0.25 0

Algorithm for solving the problem.

1. Consider two step game J1 (GS3), shown at the very left and upper angle of the graph in the figure 1. Make up table 5 of payoffs of players: insert data about payoffs in all n-tuples of game GS3 and optimal payoffs for all coalitional games.

1.1) Values of payoffs:

Er(Os3 ) (1, 1 1) = K (1, 1 1) + Pi (GS3 II G noncoal ■> (1, 1, 1)) PMSnoncool +

+P2 (GS3 11 GS1 ■ (1, 1 1)) PMSGSl + P3 (GS3 11 Gcoop■ (1, 1 1)) PMScoop •

According to i. 3 of example 1 it is followes that K (1, 1, 1) = (4, 2, 1) (see table 5). Payoffs in n-tuple of NE of the following simultaneous games are equal to PMSnoncooi = (1, 2, 2) , PMSGsi = (2f, 2f, 2±) , PMScoop = (2.5, 3.5, 3). Then

Er(aS3) (!» !) = (4’ 2’ !) + °-8 • (!» 2’ 2) + 01 • (2h 2h +

3 +01 • (2^5, 3^5, 3) « (5^32, 4T9, 3T3) •

Similarly calculate payoff vector in the game GS3 in all other n-tuples, see table

6.

1.2) Solve coalitional game r (GS3) with payoffs from table 6 (payoffs are calculated by formula (9), data is used from table 5).

1.2.1) Find NE in mixed strategies in the bimatrix game:

n = 08 1 - n = 0T9

+1 +2

0 - (1, 1) 9) 1 5 00 (1L80, 3^40) \

0 - (2, 2) (7^88, 4^88) (7^64, 6^73)

1 - +( o 2) ^•25, 4^55) (13^60, 3^64)

(2 +( 3 CO o - 1 1) 9) 1 LO 5 •8 6 O •7 0 )

Second row is dominated by the forth one. First row is dominated by the convex linear combination of the third and forth rows. Find n-tuple of NE in the mixed strategies in the bimatrix game by the formulas

j1 = v2uB-1, j2 = v\A-1u ,

where

vi = 1/(uA-1u) , V2 = 1/(uB-1u) , u = (1, ... , 1) ,

and the matrixes A and B are

= / 8.25 13.60 \ = / 4.55 3.64 \

A = ^9.45 8.60 y , B = ^5.19 7.00 J .

Then we get NE in mixed strategies: j1 = (0 0 0.67 0.33) ; j2 = (0.81 0.19). Find mean payoffs of coalition {I, III} and player II in the n-tuple of NE

E (j1, j2) = 0.67 (0.81 • [8.25 ; 4.55] + 0.19 • [13.6 ; 3.64]) +

+0.33 (0.81 • [9.45 ; 5.19] + 0.19 • [8.6 ; 7]) = [9.29 ; 4.76] .

Table5.

/l\ /2.71\ /3\ / 2.59 \ /2.5

PMS — vectors of stage games I 2 I I 2.43 I I 3 I I 2.5 I I 3.5

\2) y 2.33 J y 2.91 J \3

The strategies The payoffs The probabilities for the stage games

I II III I II III G^noncoal GSl Gs2 Gs3 Gc oop

1 1 1 4 2 1 0.8 0.1 0 0 0.1

1 1 2 1 2 2 0.1 0 0 0.8 0.1

1 2 1 3 1 5 0.7 0 0.1 0 0.2

1 2 2 5 1 3 0 0.1 0.3 0.6 0

2 1 1 5 3 1 0.8 0.1 0 0 0.1

2 1 2 1 2 2 0.25 0 0 0.25 0.5

2 2 1 0 4 3 0.1 0 0.7 0 0.2

2 2 2 0 4 2 0 0.25 0.5 0.25 0

1.2.2) Find guaranteed payoffs v {I} and v {III} of players I and III. For this purpose fix strategy of the II-nd player j2 = (0.81 0.19).

Find mathematical expectation of payoffs of players I and III, see table 7.

Thus, v {I} = 4.26, v {III} = 4.74.

1.2.2) Divide the mean payoff of coalition {I, III} in the n-tuple of NE E (j1, j2) = 9.29 between its players according to Shapley value:

Shi = v {1} + ^ [v {I, III} - v {1} - v {III}] = 4.41,

Sh3 = v {III} + ^[v {I, III} - v {1} - v {III}] = 4.89.

Table6.

The strategies The payoffs The probabilities

1 11 III I II III (I, III) II

1 1 1 5.32 4.19 3.13 8.45 4.19

1 1 2 3.42 4.55 4.83 8.25 4.55

1 2 1 4.50 3.40 7.30 11.80 3.40

1 2 2 7.73 3.64 5.88 13.60 3.64

2 1 1 6.32 5.19 3.13 9.45 5.19

2 1 2 3.15 4.88 4.73 7.88 4.88

2 2 1 2.70 7.00 5.90 8.60 7.00

2 2 2 2.83 6.73 4.81 7.64 6.73

Table7.

Math. Expectation

5.16 3.94

4.26 5.03

5.62 3.67 0

3.09 4.74 £ = 0.67

vi V3 1 - £ = 0.33

min 1 4.26 3.67 0

min 2 3.09 4.74

max 4.26 4.74

The strategies of N\ S,

the payoffs of S

n = 0.81 1 — n = 0.19

+1 +2

(1, 1) /(5.32 , 3.13) (4.50, 7.30)

(1, 2) (3.42 , 4.83) 3 .7 (7. 5.88)

(2, 1) (6.32 , 3.13) (2.70, 5.90)

(2 , 2) 3) .7 4. 5 .1 (3. (2.83, 4.81)

Table8.

3 - tuple (i; i; 2) (i; 2; 2)

Probability of passage in the game Gj 0.1 0 0 0.8 0.1 0 0.1 0.3 0.6 0

Probability of realization 3 - tuple 0.67 • 0.81 = 0.54 0.67 • 0.19 = 0.13

3 - tuple (2; 1; 2) (2; 2; 1)

Probability of passage in the game Gj 0.8 0.1 0 0 0.1 0.1 0 0.7 0 0.2

Probability of realization 3 - tuple 0.33 • 0.81 = 0.27 0.33 • 0.19 = 0.063

Probability of realization 3 - tuple of NE 0.28 0.04 0.08 0.51 0.09

G3 G^noncoal GSl Gs2 Gs3 Gc oop

1.2.3) PMS-value of players I and III of coalition S3 gets the following values:

PMS1 = 4.41; PMS2 = 4.76; PMS3 = 4.89 .

1.3) Define transition probabilities to the games G from GS3 in the subgame r (Gs3) and get table 8.

2. Similarly solve games r (GS2) and r (GSl), see figure 1.

2.1) Solve the coalitional game r (GS2) with payoffs from table 9 (payoffs are calculated by formula (9), data is used from table 5).

n =1 1 - n = 0

+1 -2

0 - (1, 1) .7 3 CO .5 3 ) (8.33, 2) 3. 6.

0 - (2, 2) (9.52, 7.73) (11.54, 2.83)

e = 0 - (1, 2) (9.38, 3.42) .9 6 O 3.15)

1 - e = 1 + (2, 1) 0) 5. 4. 0 7. 0. (1 (12.90, 0) 7. 2.

There are n-tuple of NE in the pure strategies:

j1 = (00 0 1) ; j2 = (10) .

Guaranteed payoffs equal v {II} = 4.19, v {III} = 4.83 correspondingly. Then PMS1 = 4.5; PMS2 =5.03; PMS3 = 5.67 .

Table9.

The strategies The payoffs The probabilities

I II III I II III (II, III) I

1 1 1 5.32 4.19 3.13 7.33 5.32

1 1 2 3.42 4.55 4.83 9.38 3.42

1 2 1 4.50 3.40 7.30 10.70 4.50

1 2 2 7.73 3.64 5.88 9.52 7.73

2 1 1 6.32 5.19 3.13 8.33 6.32

2 1 2 3.15 4.88 4.73 9.60 3.15

2 2 1 2.70 7.00 5.90 12.90 2.70

2 2 2 2.83 6.73 4.81 11.29 3.07

2.2) Solve coalitional game r (GSl) with payoffs from table 10.

0

0

e = 0.53

= 0.48 +1

1 - n = 0.52

+2

- (1 , 1) 3) 1 3. 1 5. (9. 3) 8. 4. 7 9. (7.

- (2, 2) (9.70, 5.90) (9.56, 4.81)

+ (1, 2) (7.90, 7.30) (11.37, 5.88)

+ (2, 1) 3) 1 3. 1 5. 1 (1 3) 7. 4. 2 0. (8.

There are n-tuple of NE in the mixed strategies:

j1 = (0 0 0.53 0.47) ; j2 = (0.48 0.52) .

Find guaranteed payoffs v {I} and v {II} of players I and II, see table 11. Thus, v {I} = 4.34, v {II} = 4.38.

Find the mean payoffs of coalition {I, III} and player II in the n-tuple of NE

E (j1, j2) = 0.53 (0.48 • [7.9 ; 7.3] + 0.52 • [11.37 ; 5.88]) +

+0.47 (0.48 • [11.51; 3.13] + 0.52 • [8.02 ; 4.73]) = [9.7 ; 5.34] .

Then

PMS1 = 4.83; PMS2 = 4.87; PMS3 = 5.34 .

Table10.

The strategies The payoffs The probabilities

I II III I II III (I, II) III

1 1 1 5.32 4.19 3.13 9.51 3.13

1 1 2 4.22 4.02 4.57 8.23 4.57

1 2 1 4.50 3.40 7.30 7.90 7.30

1 2 2 8.32 3.24 5.68 11.56 5.68

2 1 1 6.32 5.19 3.13 11.51 3.13

2 1 2 3.40 4.71 4.65 8.10 4.65

2 2 1 2.70 7.00 5.90 9.70 5.90

2 2 2 3.07 6.57 4.73 9.64 4.73

Table11.

The strategies of N\S,

Math. Expectation s to o £ p e h t of S

4.34 4.38 n = 0.48 1- n = 0.52

6.17 3.53 +1 +2

4.67 5.03 0 - (1, 1) /(5.32 , 4.19) (3.42 , 4.55)

2.77 6.86 £ = 0.53 + (1, 2) (4.5, 3.4) (7.73, 3.64)

vi V2 1 - £ = 0.47 +(2, 1) (6.32, 5.19) (3.15 , 4.88)

min 1 4.34 4.38 0 - (2 , 2) (2.7, 7) (2.83, 6.73)

min 2 2.77 3.53

max 4.34 4.38

Define transition probabilities to the games G from GSl in the subgame r (GSl), see table 12.

3. Now solve the game r (GS3) , r(GS2) , r (GSl )j, see figure 1. Initial table of data is included in the table 13. Compose the table of payoffs used by formula (9), see table 14.

Then we have the bimatrix game:

3 - tuple (i; 2; i) (i; 2; 2)

Probability of passage in the game Gj 0.7 0 0.1 0 0.2 0 0.1 0.3 0.6 0

Probability of realization 3 - tuple 0.25 0.28

3 - tuple (2; i; i) (2; 1;2)

Probability of passage in the game Gj 0.8 0.1 0 0 0.1 0.25 0 0 0.25 0.5

Probability of realization 3 - tuple 0.23 0.24

Probability of realization 3 - tuple of NE 0.419 0.051 0.109 0.228 0.193

G3 G^noncoal GSl Gs2 Gs3 Gc oop

Table13.

4.83 \ / 4.5 \ 4.41\ /2.5

PMS — vectors of stage games 4.87 I I 5.03 I I 4.76 I I 3.5

5.34 / \ 5.67 \ 4.8^ 3

The strategies The payoffs The probabilities for the stage games

I II III I II III G^noncoal GSl Gs2 Gs3 Gc oop

1 1 1 4 2 1 0.8 0.1 0 0 0.1

1 1 2 1 2 2 0.1 0 0 0.8 0.1

1 2 1 3 1 5 0.7 0 0.1 0 0.2

1 2 2 5 1 3 0 0.1 0.3 0.6 0

2 1 1 5 3 1 0.8 0.1 0 0 0.1

2 1 2 1 2 2 0.25 0 0 0.25 0.5

2 2 1 0 4 3 0.1 0 0.7 0 0.2

2 2 2 0 4 2 0 0.25 0.5 0.25 0

Table14.

The strategies The payoffs The probabilities

I II III I II III (I, n) III

1 1 1 5.53 4.44 3.43 9.97 3.43

1 1 2 4.87 6.36 6.41 11.23 6.41

1 2 1 4.65 3.60 7.57 8.25 7.57

1 2 2 9.47 5.86 8.17 15.33 8.17

2 1 1 6.53 5.44 3.43 11.97 3.43

2 1 2 3.60 5.44 5.22 9.04 5.22

2 2 1 3.75 8.42 7.77 12.17 7.77

2 2 2 4.56 8.93 7.39 13.48 7.39

n = 0.32 8 6. O sn 1 1

+1 +2

0 - (1, 1) 3) 4. 3. 7 9. (9. 1) 4. 6. 3 2. 1 (1

(2 - +( 1 6. O 2) (12.17, 7.77) (13.48, 7.39)

1 +( 9 3. O 1 1 2) (8.25, 7.57) (15.33, 8.17)

0 - (2, 1) 3) 4. 3. 7 9. 1 (1 2) 2. 5. 4 0. (9.

The n-tuple of NE in the mixed strategies: y1 = (0 0.61 0.39 0) ; y2 = (0.32 0.68^ Find guaranteed payoffs v {I} and v {II} players I and II, see table 15. Thus, v {I} = 5.08, v {II} = 5.44.

Find mean payoffs of coalition {I, III} and player II in the n-tuple of NE:

E (y1, y2) = 0.61 (0.32 • [12.17 ; 7.77] + 0.68 • [13.48 ; 7.39]) +

+0.39 (0.32 • [8.25 ; 7.57] + 0.68 • [15.33 ; 8.17]) = [13.1; 7.69] .

Then

PMS1 = 6.35; PMS2 = 6.71; PMS3 = 7.69 . Transition probabilities to the game r (G) from GSl, see table 16.

Table15.

Math. Expectation

5.08 5.75

7.93 5.13

4.54 5.44 0

4.30 8.76 £ = 0.39

vi V2 0

min 1 5.24 5.44 1 — £ = 0.61

min 2 4.42 5.13

max 5.08 5.44

The strategies of N\S, the payoffs of S

n = 0.32 1 — n = 0.68

+1 +2

(1, 1) (5.53, 4.44) (4.87, 6.36)

(1, 2) (4.65, 3.60) (9.47, 5.86)

(2, 1) (6.53, 5.44) (3.60, 5.44)

(2 , 2) (3.75, 8.42) (4.56 , 8.93)

Since the game r {^r (Gs3), r (Gs2), r (GSl )J is a three stage game, then mean payoff of each player at one step can be calculated by formula:

(6.35, 6.71, 7.69)/3 = (2.12, 2.24, 2.56)

Moreover, in the optimized case the examined three step stochastic game is different from the graph, shown in the figure 1. Show the solved game in the figure

2.

3 - tuple (1; 2; 1) (i; 2; 2)

Probability of passage in the game Gj 0.7 0 0.1 0 0.2 0 0.1 0.3 0.6 0

Probability of realization 3 - tuple 0.12 0.27

3 - tuple (2; 2; 1) (2; 2;2)

Probability of passage in the game Gj 0.1 0 0.7 0 0.2 0 0.25 0.5 0.25 0

Probability of realization 3 - tuple 0.20 0.41

Probability of realization 3 - tuple of NE 0.104 0.13 0.438 0.265 0.064

G3 G noncoal GSl Gs2 Gs3 G coop

Figure2.

4. Conclusion

In this paper the new algorithm of solving of finite step coalitional stochastic game is

proposed. A mathematical method for solving stochastic coalitional games is based

on calculation of the generalized PMS-value. The example shows realization of the

proposed approach.

References

Grigorieva, X., Mamkina, S. (2009). Solutions of Bimatrix Coalitional Games. Contributions to game and management. Collected papers printed on the Second International Conference ”Game Theory and Management” [GTM’2008]/ Edited by Leon A. Petrosjan, Nikolay A. Zenkevich. - SPb.: Graduate School of Management, SpbSU, 2009, pp. 147-153.

Petrosjan, L., Mamkina, S. (2006). Dynamic Games with Coalitional Structures. Intersectional Game Theory Review, 8(2), 295-307.

Zenkevich, N., Petrosjan, L., Young, D. (2009). Dynamic Games and their applications in management. - SPb.: Graduate School of Management.

Petrosjan, L., Zenkevich, N., Semina, E. (1998). The Game Theory. - M.: High School.

Nash, J. (1951). Non-cooperative Games. Ann. Mathematics 54, 286-295.

Shapley, L. S. (1953). A Value for n-Person Games. In: Contributions to the Theory of Games( Kuhn, H. W. and A. W. Tucker, eds.), pp. 307-317. Princeton University Press.

Grigorieva, X. V. (2009). Nash bargaining solution for solving the coalitional bimatrix games. In: Interuniversity thematic collection of works of St. Petersburg State University of Civil Engineering (Ed. Dr., prof. B. G. Wager). Vol. 15. Pp. 56-61.