Stochastic coalitional games with constant matrix of transition probabilties Текст научной статьи по специальности «Математика»

Xeniya Grigorieva

St.Petersburg State University,

Faculty of Applied Mathematics and Control Processes, University pr. 35, St.Petersburg, 198504, Russia E-mail: kseniya196247@mail.ru WWW home page: http://www.apmath.spbu.ru/ru/staff/grigorieva/

Abstract. The stochastic game r under consideration is repetition of the same stage game G which is played on each stage with different coalitional partitions. The transition probabilities over the coalitional structures of stage game depends on the initial stage game G in game r. The payoffs in stage games (which is a simultaneous game with a given coalitional structure) are computed as components of the generalized PMS-vector (see (Grigorieva and Mamkina, 2009), (Petrosjan and Mamkina, 2006)). The total payoff of each player in game r is equal to the mathematical expectation of payoffs in different stage games G (mathematical expectation of the components of PMS-vector). The concept of solution for such class of stochastic game is proposed and the existence of this solution is proved. The theory is illustrated by 3-person 3-stage stochastic game with changing coalitional structure.

Keywords: stochastic games, coalitional partition, Nash equilibrium, Shap-ley value, PMS-vector.

1. Introduction.

In the papers (Grigorieva, 2010) a class of multistage stochastic games with different coalitional partitions where the transition probability from some coalitional game to another depends from coalitional partition in the initial game and from the n-tuple of strategies which realizes in initial game is examined. A new mathematical method for solving stochastic coalitional games, based on constructing Nash equilibrium (NE) in a stochastic game similarly scheme of constructing of absolute NE in a multistage game with perfect information ((Zenkevich et al., 2009), (Petrosjan et al., 1998)), and based on calculation of the generalized PMS-value introduced in (Grigorieva and Mamkina, 2009), (Petrosjan and Mamkina, 2006), for the first time, is proposed along with the proof of the solution existence. In this paper transition probability from some coalitional game to another depends only from coalitional partition in the initial game. So the matrix transition probabilities which is constant during of whole multistage game is form.

The theory is illustrated by 3-person 3-stage stochastic game with changing coalitional structure.

Remind that coalitional game is a game where players are united in fixed coalitions in order to obtain the maximum possible payoff, and stochastic game is a multistage game with random transitions from state to state, which is played by one or more players.

2. State of problem.

Suppose finite graph tree r = (Z, L), where Z is the set of vertices in the graph and L is point-to-set mapping, defined on the set Z: L (z) C Z, z G Z. Finite graph tree with the initial vertex z0 will be denoted by r (zo).

In each vertex z G Z of the graph r (z0) simultaneous N -person game is defined in a normal form

G (z) = {N,X1,...,Xn, Ki,...,Kn),

where

— N = {1, ..., n} is the set of players identical for all vertices z G Z;

— Xj is the set of pure strategies xz of player j G N, identical for all vertices ze Z;

— xz = (xf, ..., xz) , xz G Xj, j = 1, n, is the n-tuple of pure strategies in the game G (z) at vertex z G Z;

— /tz = (/xf, ..., /t*) , j = 1, n, is the n-tuple of mixed strategies in the game G (z) in mixed strategies at vertex z G Z;

— Kj (xz), is the payoff function of the player j identical for all vertices

z G Z; it is supposed that Kj (xz) > 0 V xz G X and V j G N.

Furthermore, let in each vertex z G Z of the graph r (z0) the coalitional partition of the set N be defined

i

Sz = {Si, ...,Si} , l < n, Si n Sj = 9 V i = j , U Si = N,

i=1

i. e. the set of players N is divided into l coalitions each acting as one player. Coalitional partitions can be different for different vertices z.

Then in each vertex z G Z we have the simultaneous l-person coalitional game in a normal form associating with the game G (z)

G (z, Sz ) = ( N,XzSi ,...,XzSl, HzSi ,...,HzSl) ,

where

— -X Si = n Xj is the set of strategies xzs. of coalition Si,i = 1, /, where the

* jeSi ^ strategy xSS. G XT S. of coalition S\ is n-tuple of strategies of players from coalition Si, i. e. xzs. = { xj G Xj| j G S'*};

—xz = (xg ,..., xzSl) , xzSi G Xg , i = 1,1, is n-tuple of strategies in the game G(z, Sz); ____

—jiz = (/if, ..., fif) ,i = l,l, is n-tuple of mixed strategies in the game G (z) in mixed strategies at the vertex z G Z; however notice that ^ ;

— the payoff of coalition Si is defined as a sum of payoffs of players from coalition S\, i. e.

HzSi(xz)='$2Kj(x) , i = l,1.

je Si

For each vertex z G Z of the graph r (z0) by matrix of transition probabilities the probabilities p (z, y) of transition to the next vertices y G L (z) of the graph r (z0) are defined:

p(z, y) > 0, E p(z, y) = 1.

yeL(S)

Definition 1. The game defined on the finite graph tree r (z0) with initial vertex z0 is called the finite step coalitional stochastic game r (z0) with constant matrix of transition probabilities:

r (z0) = (N, r (z0) , {G (z )}SeZ , {p (^ y)}SeZ , yeL(z) , kf) ,

where

— N = {1, ..., n} is the set of players identical for all vertices z G Z;

— r (z0) is the graph tree with initial vertex z0;

— {G (z, Sz)} z€Z is the simultaneous coalitional l-person game defined in a normal form in each vertex z G Z of the graph r (z0);

— {p (z, y)}zeZ yeL(z) is the realization probability of the coalitional game G (y, Sy) at the vertex y G L (z) under condition that the simultaneous game G (z, Sz) was realized at the previous step at vertex z;

— kp is the finite and fixed number of steps in the stochastic game J1 (zo); the step k, k = 0, kp at the vertex zj. € Z is defined according to the condition of zk G (L (z0))k, i. e. the vertex zk is reached from the vertex z0 in k stages.

States in the multistage stochastic game r are vertices of graph tree z G Z with the defined coalitional partitions in each vertex Sz, i. e. pair (z, Sz). Game r is stochastic, because transition from state (z, Sz) to state (y, Sy), y G L (z), is defined by the given probability p (z, y).

Multistage stochastic coalitional game r (z0) is realized as follows. At moment t0 the game r (z0) starts at the vertex z0, where the game G (z0 , Sz0) with a certain coalitional partition Sz0 is realized. Players choose their strategies, thus n-tuple of strategies xz0 is formed. Then on the next stage with given probabilities p (z0, z1) the transition from vertex z0 on the graph tree r (z0) to the game G (z1, Szi) , z1 G L (z0) is realized. In the game G (z1, Szi) players choose their strategies again, n-tuple of strategies xzi is formed. Then from vertex z1 G L (z0) the transition to the vertex z2 G (L (z0)) is made, again n-tuple of strategies xz2 is formed. This process continues until at the end of the game the vertex zkt G (L (z0)) r , L {zkp) = 0 is reached.

Denote by r (z) the subgame of game r (z0), starting at the vertex z G Z of the graph r (z0), i. e. at coalitional game G (z , Sz). Obviously the subgame r (z) is a stochastic game as well.

Denote by:

— (•) the strategy of player j, j = 1, n, in the subgame J1 (z), which to each vertex y G Z assigns the strategy xjy of player j in each simultaneous game G (y, Sy) at all vertices y G r (z), i. e.

US (y) = { xVy I y G r (z)} ;

— uSi (o the strategy of coalition Si in the subgame r (z), which is a set of strategies uz (•) , j G Si;

— uz (•) = (uf (•) , ... , U (•)) = (uzSi (•) , ... , uzSn (•)) the n-tuple in the game

r (z). 1 "

It’s easy to show that the payoff Ez (uz (•)) of player j, j = 1, n, in any game

r (z) is defined as the mathematical expectation of payoffs of player j in all its

subgames, i. e. by the following formula (Zenkevich et al., 2009, p. 158):

ES (uz (•))= Kj (xz)+ E [p (z, y) Ey (uy (•))] .

yeL(z)

(2)

Thus, a coalitional stochastic game r (z0) with constant matrix of transition probabilities can be written as a game in normal form

r (z0) =

= (n, r(zo),{G(z, Sz)}z€Z, {p(z, y)}zeZ, , {[//} {Ez} ,kP),

\ yeL(z) ’ ’ /

where Uz is the set of the strategies uz (•) of the player j, j = 1, n.

The payoff Hg (xz) of coalition Si G Sz, i = 1,1, in each coalitional game G (z, Sz) at the vertex z G Z is defined as the sum of payoffs of players from the coalition Si, see formula (1):

HSi (xz) = E Kj (xz).

jeSi

The payoff Hg. (uz (•)) , Si G Sz, i = 1, I, in the subgame t (z) of the game r (z0) at the vertex z G Z is defined as the sum of payoffs of players from the coalition Si in the subgame r (z) at the vertex z G Z:

HSi (uz (•)) = E ES (uz (•)) =

jeSi

J2\Kj (xz)+ E [p (z, y) Ey (uy (•

jeSi

yeL(z)

E k (xz) + e| E

[p (z, y) Ejy (uy (•))]

jeSi

Y,k, x)+ E

jeSi yeL(z)

p (z, y)Y^ Ej (uy (•))

jeSi

= HSi (xz)+ E p (z, y) HI (uy (•))]

yeL(z)

It’s clear, that in any vertex z G Z under the coalitional partition Sz the game r (z) with payoffs Ez of players j = 1, n defined by (2), is a non-coalitional game between coalitions with payoffs HSi (uz (•)) defined by (3). For finite non-coalitional games the existence of the NE (Petrosjan et al., 1998, p. 137) in mixed strategies is proved.

However, as the payoffs of players j, j = 1, n, are not partitioned from the payoff of coalition in the subgame r (z), it may occur at the next step in the subgame r (y) , y G L (z) , with another coalitional partition at the vertex y, the choice of player j will be not trivial and will be different from the corresponding choice of equilibrium strategy uz (•) in the subgame r (z).

3. Nash Equilibrium in a multistage stochastic game with constant matrix of transition probabilities

Remind the algorithm of constructing the generalized PMS-value in a coalitional game. Calculate the values of payoff Hg. (xz) for all coalitions S'* G Sz, i = 1, /, for each coalitional game G (z , Sz) by formula (1):

HSi (xz) = £ Kj (xz) . jeSi

In the game G (z , Sz) find n-tuple NE xz = {xzSi,..., xzSi) or jlz = {fizSi,..., ftS^. In case of l = 1 the problem is the problem of finding the maximal total payoff of players from the coalition S1, in case of l = 2 it is the problem of finding of NE in bimatrix game, in other cases it is the problem of finding NE n-tuple in a non-coalitional game. In the case of multiple NE (Nash, 1951) the solution of the corresponding coalitional game will be not unique.

The payoff of each coalition in NE n-tuple HS. (fiz) is divided according to Shapley’s value (Shapley, 1953) Sh (Si) = (Sh (Si : 1), ... ,Sh(Si : s)):

Sh{Si-.j)= E -----------------J]------lv(S') Vj = l, s, (4)

S'c Si ■

S'3j

where s = |Si| (s' = \S'|) is the number of elements of set S\ (S') and v (S') is the

total maximal guaranteed payoff of subcoalition S' C Si. We have

s

v (Si) = £ Sh (Si : j).

j=1

Then PMS-vector in the NE in mixed strategies in the game G (z , Sz) is defined

as

PMS(fiz) = (PMS1 (jlz) ,..., PMS„ (pS)) ,

where

PMSj (pz) = Sh (Si : j) ,j G Si, t= lj.

Remark. If the calculation of PMS-vector is difficult, then any other ’’optimal” solution can be proposed to be used as a PMS-solution, for example, Pareto-optimality or Nash arbitration scheme (Grigorieva, 2009).

We apply the known algorithm of constructing NE n-tuple in a stochastic coali-tional game to the stochastic coalitional game r(z0) with constant matrix of transition probabilities (Grigorieva, 2010).

4. Examples.

Example 1. Let there be 3 players in the game each having 2 strategies, and let payoffs of each player be defined, see table 1. Consider all possible combinations of coalitional partitions, cooperative and non-coalitional cases.

Table 1.

The strategies The payoffs The payoffs of coalition

I II III I II III (І, П) (II, III) (I, III) (I, II, III)

1 1 1 4 2 1 6 3 5 7

1 1 2 1 2 2 3 4 3 5

1 2 1 3 1 5 4 6 8 9

1 2 2 5 1 3 6 4 8 9

2 1 1 5 3 1 8 4 6 9

2 1 2 1 2 2 3 4 3 5

2 2 1 0 4 3 4 7 3 7

2 2 2 0 4 2 4 6 2 6

1. Solve coalitional game G (X'i), S1 = {S' = {I, II} , N\S = {III}}, by calculating PMS-value (Grigorieva and Mamkina, 2009) as follows.

1.1. Find NE in mixed strategies in the bimatrix game:

П = 3/7 1 - n =4/7 1 2

0 (1, 1) [6, 1] [3, 2]

0 (2, 2) [4, 3] [4, 2]

£ = 1/3 (1, 2) [4, 5] [6, 3]

1 - £ = 2/3 (2, 1) [8, 1] [3, 2].

First and second rows are dominated by the last and third ones correspondingly. Find n-tuple of NE in the mixed strategies in the bimatrix game

Д2 = (3/7 4/7) , Д1 = (0 0 1/3 2/3) ,

by using the theorem about complete mixed equilibrium [(Petrosjan et al., 1998), p. 135].

Realization of payoffs of coalitions S and N\S in mixed strategies take place with follows probabilities:

Пі П2

£1 00 6 00 .

& 1/7 4/21 £4 2/7 8/21

Calculate mathematical expectation of payoffs in NE in mixed strategies:

1

2

4

8

E (jj1 , jj2) — - [4, 5] + - [8, 1] + — [6, 3] + — [3, 2] —

7

21

21

'36 7' 11

У’ з = 5-, 27’ 3

1.2. Find guaranteed payoffs v {I} and v {II} of players I and II, see table 2, as follows. Fix strategy of player III

= (3/7 4/7) .

Then mathematical expectation of payoffs of players of coalition S under fix strategy of coalition N\S looks as:

Es( 1,1) (A‘2) = (f • 4 Es(i,2) (a«2) = (7 ' 3 Es(2,1) (ft2) = (r • 5 ES(2,2) (a*2) = (r • 0

Hence, guaranteed payoffs are calculated as follows:

1-1; f -2+1-2) =(2f;2) ; 4 5; 1 + |-1) = (4±; l) ;

1; f ' 3 + f ' 2) =(2|; 2f); 0; f -4+1-4) = (0; 4) .

minHi (x\ = 1, x2, ft2) minHi (xi = 2, x2, ft2) minH2 (xi, x2 = 1, ft2) minH2 (xi, x2 = 2, ft2)

min{2f;4i} = 2f;

= min {2|; 0} = 0; = min {2; 2f } = 2; = min {1; 4} = 1;

v {1} = max {2|; 0} = v {II} = max {2; 1} = 2.

ol ■

Thus, guaranted payoffs equals: v {1} = 2|, v {II} = 2.

Table 2.

" ■ ^=1

■lj

Id 'j'l

Math.Expect.

2.286 2.000

4.143 1.000

2.714 2.429

0.000 4.000

vl v2

2.286 2.000

0.000 1.000

2.286 2

M

0.00

0.33

0.67

0.00

The strategies of coalitions N\S, the payoffs of coalitions 3

0.43

0.57

1 S 2

1, 1 4 2 6 1 2

1,2 3 1 4 5 1

2, 1 5 3 8 1 2

2,2 0 4 4 0 4

vl v2 vl v2

min 1 3 2 1 2

min 2 0 1 0 1

max 3 2 1 2

1.3. Divide the payoff E\ (/t1, /t2) = 5^ according to the Shapley’s value (Shap-ley, 1953):

Shi = v {1} + i (v {I, II} - {II} - {I}) = 2f + i (5± - 2| - 2) = 2f; Sh2 = v {II} + i (v {I, II} - v {II} - v {I}) = 2|.

Then PMS-vector (Grigorieva and Mamkina, 2009) will be:

5 3 1

PMSi = 2-; PMS2 = 2-; PMS3 = 2- .

2. Solve coalitional game G (S2), S2 = {S = {II, III} , N\S = {I}}, by calcu-

lating PMS-vector (Grigorieva and Mamkina, 2009) as follows.

2.1. Find NE in mixed strategies in the bimatrix game:

n= 11 — n =0 1 2

(1, 1) [3, 4] [4, 5]

(2, 2) [4, 5] [6, 0]

(1, 2) [4, 1] [4, 1]

1 (2, 1) [6, 3] [7, 0].

First three rows are dominated by the last, second column is dominated by first. Hence,

j2 =(10) , j1 =(000 1) ,

and vector of coalitional payoffs is E (j1, j2) = [6, 3]

2.2. Find guaranteed payoffs v {II} = 2 and v {III} strategy of player I

0

0

S = 0 1 — S

2 of players II and III. Fix

(10;

Then:

min

min

min

min

H2 (f2, H2 (f2, H3 (f2, H3 (f2,

X2 X2 x2, x3 x2, x3

1, x3)

2, x3) 1) 2)

min {2; 2} min {1; 1} min {1; 5} min {2; 3}

v {II} = max {2; 1} = 2; v {III} = max {1; 2} = 2.

Thus, guaranted payoffs are: v {II} = 2, v {III} = 2.

2.3. Divide the payoff E1 (j1, j2) = 6 according to the Shapley’s value (Shapley, 1953):

Sh2 = v {II} + \(v {II, III} - v {II} - v {III});

Sh3 = v {III} + \(v {II, III} - v {II} - v {III}).

Then PMS-vector ((Grigorieva and Mamkina, 2009), (Petrosjan and Mamkina, 2006)):

PMS1 = PMS2 = PMS3 = 3 .

{II}}, by calcu-

3. Solve coalitional game G (S3), S3 = {S' = {I, III} , N\S -lating PMS-value (Grigorieva and Mamkina, 2009) as follows.

3.1. Find NE in mixed strategies in the bimatrix game:

n = 5/6 1 — n =1/6

1 2

S = 1/2 (1, 1) [5, 2] [8, 1]

0 (2, 2) [3, 2] [2, 4]

0 (1, 2) [3, 2] [8, 1]

1 — S =1/2 (2, 1) [6, 3] [3, 4].

Second and third rows are dominated by first. Find NE in the mixed strategies in the bimatrix game

j2 = (5/6 1/6) , j1 vector of coalitional payoffs is

(1/20 0 1/2)

5

1

5

E (/x1, 1?) — — [5, 2] + — [8, 1] + — [6, 3] + — [3, 4] —

12

12

12

'66 30' 11

12’ 12 = 5-, 22’ 2

3.2. Fix strategy of player Нм2 = (Б/6 1/6) and find guaranteed payoffs v {I} =

= 1.6В and v {III} = 2 of players I and III, see table 3.

Table З.

w g

■Li

■ri ij'J

Math.Expect.

3.83 1.68

1.68 2.17

4.15 1.34

0.83 2.00

vl v3

1.68 1.34

0.83 2.00

1.68 2.00

Д

0.5

0

0.5

0

The strategies of coalitions N\S, the payoffs of coalitions 3

0.5

0.5

1 3 2

1, 1 4 1 5 3 5

1,2 1 2 3 5 3

2, 1 5 1 6 0 3

2,2 1 2 3 0 2

vl v3 vl v3

min 1 1 1 3 3

min 2 1 2 0 2

max 1 2 3 3

3

3.3. Divide the payoff E1 (j1, j2) = 5.5 according to the Shapley’s value (Shap-ley, 1953):

Shi = v {1} + ^ (v {I, III} - v {1} - v {III}); Sh3 = v {III} + \ (v {I, III} - v {1} - v {III}).

Then PMS-vector in mixed strategies (Grigorieva and Mamkina, 2009):

PMS1 = 2.59; PMS2 = 2.5; PMS3 = 2.91.

4. Solve cooperative game G (£4), £4 = {N = {I, II, III}}, see table 2. Find the maximum payoff of coalition N and divide it according to Shapley’s value (Shapley, 1953):

Shi = ~ b {I, II} + v {I, III} - v {II} - v {IH}] + - k’ {I, II, HI} - v {II, III} + v {I}] ; 63

Sh2 = I [t> {II, 1} + v {II, III} - v {1} - v {III}]+4 [v {I, II, HI} - v {I, III} + v {II}] ; 63

Sh3 = - {III, 1} + v {III, II} - v {1} - v {II}] + - {I, II, III} - v {I, II} + v {III}] .

63

Find guaranteed payoffs: v{I, II} = max{4, 3} = 4, v{I, III} = max{3, 2} = 3, v{II, III} = max{3, 4} =4, v{I} = max{1, 0} = 1, v{II} = max{2, 1} = 2, v{III} = max{1, 2} = 2 .

Then

І

1 1 1 Б 1

І

S/42’X’1} = 57гі1’2’2) = 57гі1’2,1) = - + - + -[9 — 4]+ - = - + - + - + - = 2-

1 1 1 3 6 3 L J 3 3 6 3 3 2

Sh

(2, 1, 1)

Sh

(1, 2, 2)

Sh

(1, 2, 1)

І І І 2 І І 6 2 І

2 + 3 + 3[9“3] + 3~2 + 3 + 3 + 3~32’

2

Table 4.

The strategies of players The payoffs of players The payoff of coalition Shapley’s value

I II III I II III HN(l, II, III) Ai Hn A 2-ffjV A3 Hn

1 1 1 4 2 1 7

1 1 2 1 2 2 5

1 2 1 3 1 5 9 2.5 3.5 3

1 2 2 5 1 3 9 2.5 3.5 3

2 1 1 5 3 1 9 2.5 3.5 3

2 1 2 1 2 2 5

2 2 1 0 4 3 7

2 2 2 0 4 2 6

Shf x'1} = Shi1'2'2) = Shi1'2' 1) = | + | + |[9-4] + | = | + | + | + |=3.

5. Solve the non-coalitional game G (£3),

Z3 = {S1 = {I}, S2 = {II} , S3 = {III}} . NE does not exist in pure strategies.

Use maximal guaranteed payoffs calculated in item 4 v {I} = 1; v {II} =

= 2; v {III} = 2 . Find optimal strategies according to the Nash arbitration scheme (Grigorieva, 2009), see table 5, where ”—” means that strategies are not Pareto optimal, but ”+” — are Pareto optimal. Then we have optimal n-tuples (1, 1, 2) and (2, 1, 2) which provide identical payoff (1, 2, 2) in both n-tuples.

Table 5.

The strategies of players The payoffs of players Optimality by Pareto (P) and Nash arbitration scheme

I II III I II III Nash arbitration scheme P

1 1 1 4 2 1 (4 - 1) (2 - 2) (1 - 2) < 0 -

1 1 2 1 2 2 (1 - 1) (2 - 2) (2 - 2) = 0 +

1 2 1 3 1 5 (3-1) (1-2) (5-2) < 0 -

1 2 2 5 1 3 (5-1) (1-2) (3-2) < 0 -

2 1 1 5 3 1 (5-1) (3-2) (1-2) < 0 -

2 1 2 1 2 2 (1 - 1) (2 - 2) (2 - 2) = 0 +

2 2 1 0 4 3 (0 - 1) (4 - 2) (3 - 2) < 0 -

2 2 2 0 4 2 (0 - 1) (4 - 2) (2 - 2) < 0 -

Conclusion.

— For S1 = {S1 = {I, II} , N\S1 = {III}} we have payoff ((2.71, 2.43), 2.33).

— For £2 = {s2 = {II, III} , N\S2 = {I}} we have payoff (3, (3, 3)).

— For E3 = {S3 = {I, III} , N\S3 = {II}} we have payoff (2.59, (2.5), 2.91).

— For S4 = {N = {I, II, III}} we have (2.5, 3.5, 3).

— For = {S1 = {I} , S2 = {II} , S3 = {III}} we have optimal payoff (1, 2, 2)

in n-tuples (1, 1, 2) and (2, 1, 2).

Fig. 1.

Example 2. Consider the following stochastic game, see picture 1. Transition probabilities (p1, p2, p3) from one game to another game are shown on the graph shown in the picture 1.

Transition probabilities are determined by table 6. Table 6 defines the matrix of transition probabilities as follows. In the left column the games from which is done transition to the games locating by the first row of the table with probabilities in corresponding rows are located.

Table 6.

The probabilities G^noncoal GSl Gs2 Gs3 Gcoop

G^noncoal 0.1 0 0.7 0 0.2

GSl 0.6 0 0.3 0.1 0

Gs2 0.25 0 0 0.25 0.5

Gs3 0.8 0.1 0 0 0.1

Gcoop 0.1 0.1 0.5 0.1 0.2

Payoffs of players in n-tuple of NE of the following simultaneous games G are equal to (see example 1)

PMSnoncoal = (1, 2, 2), PMSGsi = (2.71, 2.43, 2.33), PMSgS2 = (3, 3, 3),

PMSGs3 = (2.59, 2.5, 2.91), PMScoop = (2^, 3^, 3 Algorithm for solving the problem.

1. Consider two step game -T(GS3), shown at the very left and upper angle of the graph in the picture 1:

PMSr(GS3 ) ( j) = PMSGs3 (j) + P1(GS3 II GnonCOal) PMSG„oncoal +

+P2 (GS3II Gsi )PMSGsi + P3(GS3II Gcoop) PMSgc

1.32N /2.59\ (1.32\ /3.91

= PMSGs3 (j) + ( 2.19 I = I 2.5 I + I 2.19 I = I 4.69

2.13 J \2.91/ y 2.13 J \5.04y

j2 = {<r(1, 1, 1) = 0.42, <r(1, 2, 1) = 0.08, a(2, 1, 1) = 0.42, a(2, 2, 1) = 0.08},

where a(x) is realization probability of NE n-tuple x in pure strategies.

2. Similarly solve games r(GS2) and -T(Gnoncoal), see picture 1.

PMSr(GS2 ) (x) = PMSGs2 (x^) + p1 (GS2 II Gnoncoal) PMSGnoncoal + +p2(GS2 II GS3 )PMSGs3 + p3(GS2 II Gcoop) PMSGcoop =

/A /2.59\ /2.5'

= PMSGs2 (;r) + 0.25 • ( 2 j +0.25 • ( 2.5 j +0.5 • ( 3.5

2.15 3 2.15 5.15

= PMSgS2 (x) + ( 2.88 I = ( 3 I + ( 2.88 I = ( 5.88 I ,

2.73 3 2.73 5.73

x = (1, 2, 1);

PMS_T(Gnoncoal) (x) = PMSGnoncoal (x) + p1 (GnoncoalII Gnoncoal) PMSGB„„,o,i +

l)(

+p2(G

noncoal II GS2 )PMSGs2 + p3 (GnoncoalII Gcoop)

3

PMS

PMSGnoncoal (x) + 0.1 • ( 2 I +0.7 • ( 3 I +0.2

3

PMSG

l(x)

2.7

3

2.9

12 +

2.7

3

2.9

Gc

2.5

3.5

3.7

5

4.9

x = (1, 1, 2) or x = (2, 1, 2).

3. Now solve the game r(r(GS,3 ),T(GS2),-T(Gnoncoal)), see picture 1.

PMS

r( r(Gs3 ),r(Gs2 ),r(Gnoncoal))

j = PMSGs1 (j +

+pi(GS1 II -T(Gnoncoal)) PMSr(Gn

)+

+p2(Gsi II T(Gs2))PMSf(G ) + p3(Gsi II r(GS3)) PMS^

PMSgSi (j) + 0.6 •

3.7

5

4.9

4.16

5.23 I =

5.16

0.3 •

2.71

2.43

2.33

5.15

5.88

5.73

+

+ 0.1 •

4.16

5.23 I =

5.16

3.91

4.69

5.04

6.87

7.66

7.49

j = {a(1, 2, 1) = 0.14, <r(1, 2, 2) = 0.19, a(2, 1, 1) = 0.29, a(2, 1, 2) = 0.38}.

Since the game r(r(GS,3), r(GS2), -T(Gnoncoal)) is three stage game, then mean payoff of each player at one step can be calculated by formula:

(6.87, 7.66, 7.49) /3 = (2.29, 2.55, 2.5).

3

References

Grigorieva, X. (2010). Solutions for a class of stochastic coalitional games. Contributions to Game Theory and Management. Collected papers presented on the Third International Conference ”Game Theory and Management” (GTM’2009) / Edited by Leon A. Petrosjan, Nikolay A. Zenkevich. - SPb.: Graduate School of Management, SPbSU, 144-161.

Grigorieva, X. V. (2010). Optimization for a class of stochastic coalitional games. Game Theory and Application. ., v., SPbSU, pp. 45-60.

Zenkevich, N., Petrosjan, L., Young, D. (2009). Dynamic Games and their applications in management. - SPb.: Graduate School of Management.

Petrosjan, L., Zenkevich, N., Semina, E. (1998). The Game Theory. - M.: High School.

Grigorieva, X., Mamkina, S. (2009). Solutions of Bimatrix Coalitional Games. Contributions to game and management. Collected papers printed on the Second International Conference ”Game Theory and Management” [GTM’2008]/ Edited by Leon A. Petrosjan, Nikolay A. Zenkevich. - SPb.: Graduate School of Management, SpbSU, pp. 147-153.

Petrosjan, L., Mamkina, S. (2006). Dynamic Games with Coalitional Structures. Intersectional Game Theory Review, 8(2), 295-307.

Nash, J. (1951). Non-cooperative Games. Ann. Mathematics 54, 286-295.

Shapley, L. S. (1953). A Value for n-Person Games. In: Contributions to the Theory of Games( Kuhn, H. W. and A. W. Tucker, eds.), pp. 307-317. Princeton University Press.

Grigorieva, X. V. (2009). Nash bargaining solution for solving the coalitional bimatrix games. In: Interuniversity thematic collection of works of St. Petersburg State University of Civil Engineering (Ed. Dr., prof. B. G. Wager). Vol. 15. Pp. 56-61.