GAME THEORETIC APPROACH TO MULTI-AGENT TRANSPORTATION PROBLEMS ON NETWORK Текст научной статьи по специальности «Математика»

Contributions to Game Theory and Management, XV, 8—17

Game Theoretic Approach to Multi-Agent Transportation Problems on Network*

Khaled Alkhaled and Leon Petrosyan

St. Petersburg State University, 7/9, Universitetskaya nab., St. Petersburg, 199034, Russia E-mail: [email protected] E-mail: [email protected]

Abstract In this paper, we consider a network game where players are multi-agent systems (we call them in this paper ''coalitions'') under the condition that the trajectories of players (coalitions) should (have no common arcs, or have no common vertices) i. e. must not intersect. In the same time the trajectories of players inside coalition can intersect (have common arcs,or have common vertices). The last condition complicates the problem, since the sets of strategies turn out to be mutually dependent. A family of Nash equilibrium is constructed and it is also shown that the minimum total time (cost) of players is achieved in a strategy profile that is a Nash equilibrium. A cooperative approach to solving the problem is proposed. Also, another cooperative mini maximal approach to solving the problem is investigated. We also consider the proportional solution and the Shapley value to allocate total minimal costs between players. Two approaches for constructing the characteristic function have been developed.

Keywords: Nash equilibrium, the Shapley value, the proportional solution.

1. Introduction

Theory of games on networks have been growing in recent years. Mazalov and Chirkova (2019) provided a comprehensive discussion of the topic. Given that most practical game situations are more dynamic (intertemporal) rather than static, dynamic network games have become a field that attracts theoretical and technical developments. One special case of network games is transportation game. This problem was considered in the articles by (Petrosyan, 2011) and by (Seryakov, 2012) about the game theoretic transportation model in the network. In (Petrosyan, 2011; Seryakov, 2012) a game theoretic approach is considered for n-player which want to reach the fixed node of the network with minimal time (cost). It is assumed that the trajectories of players should have no common arcs, i. e. must not intersect. The last condition significantly complicates the problem, since the sets of strategies turn out to be mutually dependent. A family of Nash equilibrium is constructed and it is also shown that the minimum total time (cost) of players is achieved in Nash equilibrium strategy profile. A cooperative approach for solving the problem is proposed. We consider the game theoretic approach (Petrosyan, 2011) where players are coalitions under the condition that the trajectories of players (coalitions) should have no common arcs, or have no common vertices i.e. must not intersect. The trajectories of players inside coalition can intersect (have common arcs, or have common vertices).

*This research was supported by the Russian Science Foundation grant No. 22-1100051, https://rscf.ru/en/project/22-11-00051/

https://doi.org/10.21638/11701/spbu31.2022.01

A family of Nash equilibriums is constructed and it is also shown that the minimum total time (cost) of players (coalitions) is achieved in a strategy profile that is a Nash equilibrium. A cooperative approach for solving the problem is proposed. We also suggest another cooperative mini maximal approach. The proportional solution (Barry Feldman, 1999) and The Shapley value (Harold and Albert, 2016) are proposed to allocate the costs inside each coalition. Two approaches for constructing the characteristic function have been developed. In both cases, to define the characteristic function, approaches are used based on corresponding Nash equilibrium. It is shown on example that the proposed solutions are not time consistent and the two level solution concept of the game is developed.

2. Model

The game takes place on the network G = (X, D), where X is a finite set, called the vertex set and D— set of pairs of the form (y, z), where y G X, z G X, called arcs. Points x G X will be called vertices or nodes of the network. On a set of arcs D a nonnegative symmetric real valued function is given 7 (x, y) = 7 (y, x) > 0, interpreted for each arc (x, y) G D as the time (or cost) associated with the transition from x to y by arc (x, y). As mentioned before we consider the case when players are coalitions Mi, ..., Mk,..., Mp.

Define p — player transportation game on network G. The transportation game r is system r = (G,P,M(P),a), where G— network, P = {1,...,p}— is set of players (coalitions), a G X - some fixed node of the network G. M(P) - subset of coalitions of network G, M (P) = {1(M), 2(M),..., k(M),...,p(M)}, indicating the coalitions in which players are located in M(P) at the beginning of the game process (the initial position of players (coalitions)). We will say that the paths of players (coalitions) hM and hM do not intersect, and write hM' n hM = 0, if they do not have common (arcs, or vertices). Denote this game by r.

The set Mk = ¿k,..., ikr,..., ikrk} in network G, we call coalition. The Strategies of coalition are defined as any path connecting his initial position (initial position of players from Mk) with a fixed node a. The paths of players inside coalition may intersect.

( .k -k -k *"] ( ■k -k ■k Denote by hMk = lhl 1,... ,hi r,... ,hl rk L where lhl 1,... h r,... ,hl rk ^ are

strategies of players {iki,..., ikr,..., ikrk} in coalition Mk.

hik = {(xkr, xkr), (xkr, xkr),..., (xk _i, a)} , are the strategies of player i^ (inside

coalition Mk) and xkr is initial position (node) of player i'k inside coalition Mk.

k

lr is a number of arcs of hir for player i; inside coalition Mk. The strategies of coalition Mk have the form:

hMk = ^ (xQi, xki) , (xki, x2i) , . . . , (xf1-1, a) },...,

{ (x0r, xlr) , (xlr, x2^ , . . . , (xfr-i, a } , . . . , k k k

) , (xirfc, x2rfc ) , . . . , (xfrfc -1, ^ }].

, x1rfc/ , Vx1rfc , x2rk

A bunch of all strategies of Mk we denote by HMk. The strategy profiles hM = (hMl,..., hMp), hMl G HMl,..., hMp G HMp are called admissible if the paths hMki and hMkj not intersect (not contain common arcs, or not contain common

vertices). hMki n hMkj = 0, ki = kj. The set of all admissible strategy profiles is denoted by HM.

In this section we define for each arc (xl}m,xl}+1m) the cost function Yi (xfc fm,xkf+im) equal to the cost which necessary to reach the node from

node xkm by player Mk (coalition Mk). The coalition costs are defined as

rk lm — 1

CMk (hM) = ^ ^ Yi (xkfm,xkf + im) = C (hMk) . (1)

r=1 f=0

3. Nash Equilibrium Between Coalitions in Game

In the game r the strategy profile (hM = hMl,..., hMp j is called a Nash equilibrium, if CMk ^hM || hMk j > CMk ^hM j holds for all admissible strategy profiles (W || hM^ e HM and k e P.

Let n be some permutation of numbers (1,... ,p),n = (Mkl,..., Mkp). Consider an auxiliary transportation problem on the network G for player(coalition) Mkl. Find the path in the network G, minimizing the player (coalition) Mkl cost to reach from initial position to fixed node a e X. Denote the path that solves this problem by

hMki

c(hMkA = min C (hMki) . (2)

V / hMk i EHmk l

Remind that players inside the coalition may use paths with common arcs (vertices). Denote by G\hMki a subnetwork not containing arcs (vertices) hMki . Consider an auxiliary transportation problem for player (coalition) Mk2 on network G\hMki . Find the path in subnetwork G\hMkl. Minimizing the player (coalition) Mk2 cost to reach from his initial position to fixed node a e X. Denote the path that solves this problem by hMk2

C (hMkA = min C (hMk2) . (3)

V J h"k2EH"k2

Proceeding further in a similar way, we introduce into consideration the subnetworks of the network G, that do not contain arcs (vertices) which belong to paths hMki , ..., hMkm-l . Consider the auxiliary transportation problem of the player Mkm on the network G\ W^—-1 hMki. Find the subnetwork G\ U^—11 hMki, minimizing the player (coalition) Mkm cost to reach the node a e X. Denote the path that solves this problem by hMkm

C (hMkm) = min C (hMkm) . (4)

V J hMkm^km

As a result, we get a sequence of paths hMki ,..., hMkp, minimizing players (coalitions) Mkl, Mk2,..., Mkm,..., Mkp cost on subnetworks:

G, G\hMkl ,...,G\ U—1 hMkm ,...,G\ U—1 hMki.

The sequence of bunches of paths hMki ,..., hMkm,..., hMk p by construction consist of pairwise non-intersecting arcs (vertices), and each of them hMki e HMki. There-

fore the strategy profile (hMfci,..., hMkm,..., hMkp) = hM(n) G HM is admissible in r.

Theorem 1. The strategy profile hM (n) G HM is an equilibrium strategy profile in r for any permutation n.

Proof. Consider the strategy profile. hM(n)||hMkm , where hMk™ = hMkm, hMK™

G HMfcm, hM (n)|hMfcm g HM. By construction hMkm is determined from the

condition ^

C(hMkm) = min C (hMk-

V ' hMk meG\Um-1hMfci

However, the strategy profile hM (n)||hMkm is admissible (if hMKm G G\ IX-1 hMk ) and therefore C (hMk^ < C (hMkm1 = cMfcm [hM(n)||hMk c (hMkm) = CMkm (hM(n)), and CMkm [hM(n)] < CMkm [hM(n)||hMk "

hM (n)||hMfcm G HM, which proves the theorem.

for all

This theorem indicates a rich family of pure strategy equilibrium profiles in r depending on permutation n. Thus, in r we have at lest p! equilibrium strategy profiles in pure strategies. If the initial states of players(coalitions) are different. The strategy profile hM (n) is called a best equilibrium if

p p

£ CMk (hM (n)) = min £ CMk (hM (n)) = W. (5)

k=i k=i

4. Cooperative Solution

However, there are other Nash equilibrium profiles in r. Consider the strategy profile hM, solving the minimization problem

p

min £ CMk (hM) = £ CMk (hM) = V. (6)

k=i k=i

We can simply show that hM is also a Nash equilibrium strategy profile. Because if one player changes his strategy and other players do not change their strategies his time (cost) under this condition will be more than equal of his time (cost) in case when has not changed his strategy. Consider the strategy profile

hM = hM ,...,hMK ,...,hMP '

if player Mk change his strategy, we get Ek=i Cm(hM || hMk) > Ek=i Cm^ (hM)

C(hM1) + C(hM) + ... + C(hMk) + ... + C(hMp) > C(hMl) + C(hM)+

... + C(hMk) + ... + C(hM)soC(hMk) > C(hMk).

We call the strategy profile hM a cooperative equilibrium in r. In some cases V = W, (see the example). Consider now another approach to define the cooperative

solution. For each strategy profile we define the player (coalition) Mk with the maximal time (cost) necessary to reach from the initial position to fixed node a, then from all strategy profiles we select such strategy profile for which this maximal time (cost) is minimal. This strategy profile will call cooperative mini maximal

strategy profile hM.

r 1 p

CMk (hM) = min max (cm(hM)) , Denote Y CMk (hM) = R (7)

k=i

Remind the definition of cooperative path (coalition)

= [{ (^Oi1, , (^li1, , . . . , ^if-1, a) } ,

. . . { (=xOik, xl k) , (=xlfefc, x2fck) , . . . , (^k-1, a) } , . . .

\ , x1p ^ , , x2p J , . . . , J J,

where L = max lk.

1<fc<p

Denote by X(r) cooperative trajectories corresponding to cooperative path.

X = (=M1 xM1 xM1 xM1 ) (=Mk =Mfc

x = (x01 , X11 , X21 , . . . , XI1-1, a), . . . (x0k , x1k ,

= Mk = Mk ) (= Mp XMp = Mp = Mp )

x2k ,...,= Ik-1, a), . . . (X0p , = 1p , = 2p , . . . , = Ip-1, a)

The subgame starts from the state X (r) = (x°11,..., x°°k,..., x°°p),

where X°°k = (X°kk, X°kk, X°kk,..., X°-1, a), k = 1,..., P, where r is a stage number

for players (coalitions), (Petrosyan and Karpov, 2012).

In the cooperative version of the game between coalitions we suppose that all players (coalitions) jointly minimize the total costs and this minimal total cost we denote by V(P).

The proportional solution (Barry Feldman, 1999) in cooperative subgame is defined as:

№ (X(r),r) = pV(Mk; X(r),r) V(P; X(r), r); K e P

Y V(Mk; X(r),r) (8)

k=1

^5Mk(X(r),r): is the cost player Mk starting from X(r) on cooperative trajectory. V(P; X(r),r): is a minimal joint cost for all players (cooperative solution) starting from x (r) .

V(Mk; X(r),r): is a minimal joint cost for player Mk along cooperative trajectory starting from x (r) .

The Shapley value Sh = {ShMk }keN in cooperative game r starting from X(r) is a vector with components (Harold and Albert, 2016):

ShMk (X(r),r) = Y (P S)!(S 1)! (V (S,X(r),r) - V (S\{Mk},X(r),r)) .

MkGSCP p

k (9)

Here V(S; x(r), r): is defined as minimal total cost for subset of players along cooperative trajectoriesx(r), starting from x(r).

Here V (S\{Mk}, x(r), r) : is defined as minimal total cost for subset of players (coalitions) (cooperative solution) without player Mk, starting from x(r). Example: for two players (coalitions) formula for the Shapley value will have the form:

ShMi (x(r), r) =

5. Two Stage Solution Concept in Game

We consider two different approaches First approach: consider cooperative game between players (coalitions), and find the Proportional solution ^5Mk in r. This solution shows the loses of every given coalition, then investigate the problem how to distribute this loses between members of coalition. For this reason we use the Shapley value but it is necessary to define the characteristic function for players inside the coalition. The characteristic function is defined in following way: suppose S C P then V(S) can be taken as the loses of S in some fixed Nash equilibrium (under fixed permutation) in the game played by (coalitions) S with other players as individual players [we may suppose that the strategies of players do not have common (arcs, or vertices) ]. Denote the Shapley value inside coalition as shj(Mk); Mk C S. We propose to allocate the loses as

Second approach: consider cooperative game between players (coalition), and find the Shapley value shMk in r. This solution consider loses for given coalition, then the problem how to distribute this loses between members of coalition. For this reason we compute the proportional solution. Denote the Proportional solution inside the coalition as ^5j(Mk); Mk C S. We propose to allocate the losses

V (M1,x(r),r) ShM2 (x(r), r)

V(M1, x(r), r) + V(M2, x(r), r) - V((M1, M2), x(r), r)

2

V (M2,x(r),r)

V(M2, x(r), r) + V(M1, x(r), r) - V((M1, M2), x(r), r)

2

^¿(Mk )= pkshi(Mk ) (^Mfc ; k g{1, ..^rf.

£ shi(Mk)

(10)

^(Mk )= pf(Mk ) shMk ; k G{l,...,p}.

i=1

(11)

6. Example (Time Consistency Problem):

Fig. 1. Two players (coalitions) in game

In this figure we denote nodes by capital Latin letters. The coalitions M = {M1, M2}; M1 = A, B, M2 = I, F

Two players (coalitions) want to reach the fixed node E under condition (paths have no common arcs).

The loses are written over the arcs and are equal, respectively to

Y(A, B) = 2, y(a, F) = 1, y(b, C) = 0, y(b, G) = 0, y(c, D) = 2, y(c, H) = 0, y(c, G) = 0.7, y(D, E) = 0, Y(D, H) = 1, y(1, F) = 0, y(f, G) = 0, y(F, J) = 2, Y(J, H) = 1, y(h, E) = 0.

Non-cooperative solution

For permutation: n = {M1, M2}

h^1 = [(A, F), (F, G)(G, B), (B, C)(C, H), (H, E)J, [(B, C), (C, H)(H, E)J

CM1 (hM ) = 1+0 = 1 hX2 = [(I, F), (F, J), (J, H), (H, D), (D, E)J, [(F, J), (J, H), (H, D), (D, E)]

cm2 (hM) = 4 + 4 = 8

For permutation: n = {M2, M1}

hM2 = [(I, F), (F, G), (G, B), (B, C), (C, H), (H, E)], [(F,G), (G,B), (B, C), (C, H), (H,E)J cm2 (hM) =0 + 0 = 0 hM1 = [(A, F), (F, J), (J, H), (H, D), (D, E), (H, E)], [(B, A), (A, F), (F, J)(J, H), (H, D), (D, E), (H, E)J cm1 (hM) = 5 + 7= 12.

Thus, both equilibrium hM(M2, M1) and hM(M1, M2) are cooperative equilibrium. In best Nash equilibrium in r we get W = 9.

Game Theoretic Approach to Multi-Agent Transportation Problems on Network 15 Cooperative solution

hMl = [(A, B), (B, C)(C, D), (D, E)], [(B, C), (C, D), (D, E)] Cm, (hM) =4 + 2 = 6 hM2 = [(I, F), (F, G), (G, C), (C, H), (H, E)], [(F, G), (G, C), (C, H), (H, E)] Cm2 (hM) = 0.7 + 0.7= 1.4 Cm, (hM) + Cm2 (hM) = 6 + 1.4 = 7.4 = V = R.

We get the result R = V < W, (see (5), (6), (7)).

The proportional solution in game (apply formula (8)) For r = 0, n = (Mi,M2)

№ (X(0), 0) = (1/9)7.4 = 0.822, 9m2 (x(0), 0) =(8/9)7.4 = 6.578

For r = 0, n = (M2,M1)

9Mi (X(0), 0) = (12/12)7.4 = 7.4, 9M2 (X(0), 0) =(0/12)7.4 = 0

For r = 1, n = (M1,M2)

95m, (X(1), 1) = (0/6)5.4 = 0, 95m2 (X(1), 1) =(6/6)5.4 = 5.4

For r = 1, n = (M2,M1)

95Mi (X(1), 1) = (9/9)5.4 = 5.4, 9M2 (x(0), 0) =(0/12)5.4 = 0

Compare the results

¥>Mi (x(1), 1) + 1 = 1 = <^Mi (x(0), 0) = 0.822 S5M2 (x(1), 1) + 2 = 7.4 = ^2 (x(0), 0) = 6.578

^Mi (x(1), 1) + 3 = 8.4 = ^5MI (x(0), 0) = 7.4 ^ (x(1), 1) + 0 = 0 = s5m2 (x(0), 0)

The proportional solution is not time consistent in the game.

The Shapley Value(apply formula (9)) For r = 0, n = (M1;M2)

ShM2 (x(0), 0) = 8

ShM1 (x(0), 0) = 12

12 + 8 - 7.4 2

8+12 - 7.4 2 "

1.7

5.7

For r = 0, n = (M2,M1)

ShM2 (x(0), 0) = 0

ShM1 (x(0), 0) = 1

1 + 0 - 7.4 2

0+1 - 7.4 2

4.2

3.2

For r = 1, n = (Mi,M2)

_ 9 + 6_ 5 7

ShM! (X(1), 1) = 9 - + „ • = 2.85

ShM2(X(1), 1) = 6 - 6 + 57 = 1.35

For r = 1, n = (M2,M1)

ShM!(X(1), 1) = 1 - 1+0 -5.7.4 = 3.35 ShM2(X(1), 1) = 0 - 0 + 1 - 5.7 = 2.35

Where r is a stage number for players (coalitions). Compare the results

ShMl (X(1), 1) + 1 = 2.85 + 1 = 3.85 = 5.7 = ShMl (X(0), 0) ShM2 (X (1), 1) + 2 = 1.35 + 2 = 3.35 = 1.7 = ShM2 (X (0), 0) ShMl (X(1), 1) + 3 = 3.35 + 3 = 6.35 = 4.2 = ShMl (X(0), 0) ShM2 (X(1), 1) + 0 = 2.35 + 0 = 2.35 = 3.2 = S hMl (X (0), 0) The Shapley value is not time consistent in the game.

Two stage solutions concept in game

In the case of best Nash equilibrium n = (M1, M2) we get:

The proportional solution for two players (coalitions) M1, M2 : (/5m1 = 0.822, (M2 = 6.578.

The Shapley value for the players inside coalitions M1, M2 :

sh1(M1) = 1, sh2(M1) = 0, sh1(M2) = 4, sh2(M2) = 4. Applying (10) (first approach) we get:

^1(M1) = (0.822)(1) = 0.822, ^2(M1) = (0.82)(0) = 0 ^(M2) = (6.578)(4/8) = 3.289, ^2(M2) = (6.578)(4/8) = 3.289.

Consider now the second approach: The Shapley value for two players (coalitions) M1, M2 :

shMl = 5.7, shM2 = 1.7.

The proportional solution for the players inside coalitions M1, M2:

<^1(M1) = 1,( (M1) =0, (3^2) =4, (2(^2) =4.

Applying (11) (second approach) we get:

01 (M1) = (5.7)(1/1) = 5.7, 02(M1) = (5.7)(0) = 0 01(M2) = (1.7)(4/8) = 0.85, 02(M2) = (1.7)(4/8) = 0.85.

References

Barry Feldman (1999). Scudder Kemper Investments. 222 South Riverside Plaza, Chicago, IL, 60606.

Harold, W.K. and Albert, W. T. (2016). Contributions to the Theory of Games (AM-28),

Volume II. Princeton University Press. Mazalov, V. V. and Chirkova, J. V. (2019). Networking Games Network Forming Games

and Games on Networks. Elsevier Inc. Petrosyan, L. A. (2011). One transport game-theoretic model on the network. Mat. Teor. Igr Pril, 3(4), 89-98.

Petrosyan, L. A. and Karpov, M.I. (2012). Cooperative solutions in communication networks. Vestnik of Saint Petersburg University. Series 10. Applied Mathematics. Computer Science. Control Processes, 4, 37-45. Seryakov, A. I. (2012). Game-theoretical transportation model with limited traffic capacities. Mat. Teor. Igr Pril, 4(3), 101-116.