CC BY
3Construction of Internal Time Consistent Optimality Principle
Elena Gubar
Faculty of Applied Mathematics and Control Processes,
St. Petersburg State University,
35, Universitetskiy pr., St. Petersburg, 198504, Russia fax: + 7 (812) 428 71 59 e-mail address: [email protected]
Abstract. Many real conflicting situations occur on a given time interval and can be described by dynamic cooperative games. Most important properties in dynamic cooperation are time consistency and internal time consistency. This properties allow to preserve players cooperation during the whole time period.
In this paper we investigate possibilities of construction of new optimality principles with properties of time consistency and internal time consistency in case of dynamic hierarchical game.
As a basic model consider multistage cooperative game G with hierarchical n + 1 player game r, played on each stage. We choose core as a solution concept in each stage game, with the use of characteristic function defined in multistage game as sum of stage characteristic functions. The corresponding optimality principle is defined and internal time consistent. Example of a game with such optimality principle is also considered.
Keywords: Dynamic cooperative games, optimality principle, time consistency, internal time consistency, hierarchical games, multistage games.
Introduction
Consider two-level hierarchical system. Structure of this system consists of the center A0, which carries out control and has material and working resources in system and n subordinated “productive” centers Bi, i = 1,...,n.
Suppose that the administrative center A0 on the first level of hierarchy chooses u = (ui,... ,un) from a set U, where U is the strategy set of Ao, and ui is control, which influence on subordinate center Bi, i = 1,...,n and limits the possibilities
of actions set of Bi. Center Bi is situated on the second level hierarchy. It chooses strategy vi Є V(ui), where V(ui) - is strategy set of Bi, which is predetermined by a choice of A0.
Thus the center A0 has the right of the first move and can limit opportunities of the subordinated centers. The purpose of the center A0 is to maximize the payoff K0(u, vi(u) .. .vn(u)) by choosing u, and centers Bi have their own purposes and aspire to get maximum of payoff Ki(ui, vi) choosing vi.
1. One-stage game
Consider (n + 1) person hierarchical game and denote it as Г. The players of this game are administrative center A0 and subordinated centers Bi.
Let player A0 chooses value u Є U, where:
n
U = {u = (ui,..., un), ui > 0,ui Є R1,y^'/ui < b, b > 0, i = 1,..., n}, (1)
i=1
here ui are resources, which A0 supplies to Bi.
Having the information of the choice u Є U Bi chooses xi Є Vi(ui), where:
Vi(ui) = {xi Є R1 : XiOi < ui + a.i, xH > 0, i = 1,..., n}, (2)
value xi can be interpreted as production program of Bi, xi is function of ui, ai are production coefficients, also they can be interpreted as prime cost, ai are amounts of available resources of Bi, Vi (ui) is strategy set of the player Bi.
Define the payoff functions of the players. For the player A0, the payoff function is equal to:
n
K0(u,xi(-), . . .,xn (■)) = ^ Ci xi(ui), i = 1,...,n,
i=1
where ci are utilities of the output of the player Bi for the administrative center A0.
The payoff function of Bi is equal to:
Ki(u, xi(-),..., xn()) = di xi(ui), i = 1,...,n, where di are utilities of the output of Bi for Bi.
Assume that cooperation between centers is permitted. In this case for each coalition S C N = {Ao,B1,..Bn} define its maximal guaranteed payoff v(S) as follows:
0, if S = {Ac} minmax(^ diXi) = ~~~> if Ao ^ S
u ieS ai
' (V' ( і j \ I A
maxmax mm (2^ (a + di)Xi + CjXj) = 2^ --------------------------------> Г34)
u ieS jeN\S ieS \S i^S ai ( 3
ifAo Є 5
l І і 7 \ \ v—' T di) (с^г T ^г) • -f c f ЛП
maxmax(2^ (a + di)Xi) = 2^ ----------—--------if S = {A},
u xS i^S i=l ai
v(S)
_ c* + di
where Ui is defined in the following way. Let ------------= Ki, i = 1,... ,n, then
ai
{b, if Ki = max Kj
j=i,...,n . (4)
0, Vi = j. ()
The necessity of introduction of this values can be confirmed with the follow-
(u■ ~\~ ol ■)
ing argumentation. From (2) we have x* < —----------------—. Take expression for v(N) and
Cll
substitute ^ 1 ^—— instead of x* and get v(N) = .
ai i=1 a
2. Multistage game
Describe now the multistage cooperative game G. Suppose that on the stage k — l,k = l,m a stage game rfc_1 is realized with characteristic function vk~1(S) and optimality principle Ck-1. Let Yk-1 = (yq-1, ... ,YQ-1) is the imputation from the optimality principle Ck-1, and yO)-1 is component of imputation Yk-1 for center Aq, Yi^1 are the components for centers Bj, i = 1, n.
It is assumed, that administrative center A0 can employ i’s own component yO)-1 of imputation yk-1 to purchase resources and to its own needs, and also on each stage any of centers Bi can dispose its own components Yq-1 of imputation yk-1 to improve the production coefficients, purchase personal resources and to its own needs. Hence, on the next stage k parameters of our model are changed and we get new characteristic function vk (S) and optimality principle Ck.
Suppose, that on stage k — 1 each player assigns some part of his payoff to general system evolution. Denote this part as p.
On each stage parameters of our model are changed in accordance with following formulas:
t k 1 i P^O k k— 1 , P^i ■ 1 /r'l
b — b -\-------------, q.1 — 1 -|-------, % — 1, . .., n7 (5)
so si
where si, i = 1,...,n are cost of resources for each center.
Using formulas (5) we can construct new characteristic function vk (S) for a stage game rk:
beginarrayl S = {A0}, vk(S) = 0;
es ai Si
A0 € 5, vk(S) = vk-\S) + <*±*2 (^ + X2(«+d*)ryt1
ai \ si so ) ai Si
S = {N},v\S) = vk~\s)+^ + dl) (ntl+ntl) + (6)
ai V si so J “ ai Si
Choose core as an optimality principle for a stage game and one can see that the imputation Yk = (Yo, Yi,..., Yn) with following components:
k _ (ci + di) bk = (ci + di) (^fc_1 ^ P7o ^ .
Yo =
a1
a1
so
k (Ci+di) fc_i (Ci + di) k_i py,
!i = —;------ai = —;-\ai + —
k1
i = 1,...,n
will belong to the core Ck for a stage game rk. Formulas (6) define the dynamics of the characteristic function evolution:
vk (S) = FS (vk-1 (S), Yk-1), Yk-1 e Ck-1, k =1,...,m. (7)
Since vk is characteristic function in the game rk, then for FS we should have:
FsiUs2 (vk-1 (S1 U S2), Yk-1) > Fs! (vk-1 (S1), Yk-1) + Fs2 (vk-1(S2), Yk-1),
for coalitions S1, S2 C N, S1 n S2 = $, F$(vk-1($),Yk-1) characteristic function v(S) for multistage game G as:
0. Define a new
J(S) = £ vo (S), for all S C N.
(8)
k=0
This function is also superadditive.
For the multistage game G, with hierarchical game rk, played on each stage, using expression (8) and (6) construct characteristic function. Formulas for the characteristic function v(S) are following:
S = {A0}, v(S)=J2vk(S) = 0-,A0<£S\
k = 1
j(s) = YJ[vk-1(s)+YJ
diak-1 pYk-\
k=1
Ao e S,
m
v(S) = E k=1
S = {N},
m
v(S) = E k=1
o
ieS
1
m(v°(S)) +
k=1
y- dj PT
a-
iieS
,,fc-i(5) + (ci + rfi) | P7i 1 + Plo 1 | + (ci + di)Pr,
k-1
a1
i—1
s1
so
ieS
k=1
(c1 + d1) (y° , Y°\ , sr-(ci + di) PYi
a1
ieS
vk-+ (ci + di) (Pli 1 + Plo 1 j + (ci + di)P%
k1
a1
s1
so
1
i=2
k=1
(ci + di)„ fjo_, I (ci + di)pYk
Qk + Qk + n_. Qk
P \ k 1 k a1 \ so sk
i=2
a
s
sk
a
Construct the core for the multistage game G with characteristic function v(S) and denote this core as C{7), since the core is generated by characteristic function v(S) which according to (10) is generated by 7. It can be easily seen that the
imputation 7 = (70, 7i, • • •, 7„) with components
— (Cl + ^!) !,fc — (ci + di) k ■ 1 V, 1 +. +1, 7=?
7o= y,--------------°1 li = 2,--------------a > * = 1,n, belongs to the core 6.
a 1 0/j
k=0 1 k=0 1
Definition 1. Let 7 = (70, 71,..., 7*,..., 7„) G C. Vector sequence p = po, /?i,..., /3№ (p = polpul), such that
m
Yi = ^2 pil, pil — 0, (11)
1=1
is called imputation distribution procedure (IDP) (see [Filar, 2000]).
Let Gk be the subgame of multistage game G, starting from the stage k and lasts to the end of the game G.
Definition 2. Following vector sequence p = po,..., pk,..., (Pi = Pol,..., pui) we will call time consistent imputation distribution procedure, if
m
Yi = 53 pa, 1 e N, (12)
l=1
7* = $>«, 7° = 7, (13)
l=k
7fc = (7o j 7i j j 7n) ^ Gk(j), where Ck (7) is optimality principle in subgame Gk, generated by y (see [Filar, 2000]).
One can interpret imputation distribution procedure in the following way: pik is the payment to player i on stage k in the multistage game G, or it is payment on the first stage of the subgame Gk. In the game G each player has payoff Yi as i-th component of the optimal imputation 7 G C (i = 0,1,..., n).
Definition 3. Optimality principle C(7) is time consistent in game G, if for each 7 G C{ 7) there exist IDP p = (po, Pi, , pm), Pk >0, k = 0,1,2, ... ,m, such as
mm
' = ^2pi, ik = J2P'-eC (7),70 = 7, l=1 l=k
where C (7) is optimality principle generated by 7 in the subgame G .
Y
Definition 4. Optimality principle C(7) is called internal time consistent in the game G, if it is time consistent, and for each 7 G C(7) we have
4 & —fe /
l>k
^zk ,
where C (7) is optimality principle in subgame & of the multistage game G, generated by 7 = {7k(see [Filar, 2000]).
Definition 5. Imputation 7 Є C(7) is called internal time consistent, if condition 7 Є C (7), k = 0, .. ., to is realized.
Remark. Not every imputation 7 can be presented in the following way:
m
7 = £ 7k ,7k Є Ck. (14)
k=0
In our case imputation 7 Є С(^) of the multistage game G can be presented in form
(14):
mm
7о = ХЛ(Л 7i = * = 1 • • 7 k Є Ck, k = 0,1,.. ,,m.
k=0 k=0
Denote as (7 optimality principle where all imputations can be represented in form
(14), 7 is imputation from the optimality principle C, 7fc is imputation in optimality — k л principle (7 of subgame Gk, k = /,..., m.
Each component of the imputation 7 = (70,71,..., 7^) on stage k can be represented as
70fc= (Gl+rflV, 7f= (Cl+rfl)«fc, * = 1 ,...,n.
ai ai
Hence, each imputation 7k, which can be presented in form (14), belongs to optimality principle Ck of the game rk. Let all players on each stage distribute their intermediate payoff in according with the optimality principle, chosen on the first stage.
In our case imputation distribution procedure is constructed in the following way: рік is equal to 7^ and рік for the imputation 7 is time consistent, because conditions of definition 2 are realized.
3. Construction of internal time consistent optimality principle in multistage cooperative game
Consider possibility of construction internal time consistent optimality principle. It is a difficult process, and one possible way is to construct the intersection
of optimality principles for multistage games, generated by different imputations. Describe the process.
Consider multistage game G and suppose that on each stage hierarchical game r is played. This game r has the same structure as in introduction section. Let v1 (S) is characteristic function and C1 is core of the game r1, played on the first stage. Core is multiple optimality principle, hence, on the second stage, as in the previous section, we can construct the set of characteristic functions and the set of corresponding optimality principles in the following way. Take some finite set of imputations from the core C1 and denote this set as A = {£|, £2,...,£p}. Use the set of imputations A on stage two to construct the set of characteristic functions vj(S; £j), j = 1,...,p and the set of corresponding optimality principles Cj. Then on the second stage of the multistage game G we have the set of games Gj with two stages. On the third stage of the multistage game G we get the set of characteristic functions vj (S; j and corresponding set of optimality principles Cj.Continue the process in the similar way, on stage k, for each game Gk, we constructed characteristic functions vk (S; )
and corresponding optimality principles Ck, j = 1,...,p.
After m stages we have the set of multistage games Gj. Every game has its own characteristic function
m
vj(S; ij) =X^vk(S; j ), j =1,...,p, where £jj = (£%,...,£kjn). (15)
k = 1
For each characteristic function Vj(S; £j) construct corresponding optimality principle Cj for every multistage game Gj, j = 1,,p. Consider the intersection of optimality principles Cj and denote it as M:
M=f)Cj(tj). (16)
j=1
Suppose that M = ty. Define as ^ j. The main question is, if subset A
l>k
belongs to intersection M or not. If A, then we can say, that for every imputation G A, 1 < j < p, condition ^ 6 C (£j), 1 <j < p is realized, and so subset A is internal time consistent optimality principle. But it is difficult to check condition Ac M, because we must check that imputation £j belongs to every optimality principle from the intersection M.
Consider new function w(S): w(S) = maxvj(S,£j).
j
Function w(S,ij) can be not superadditive. Construct A as a finite set of
n
{j = (j ,j ,...,j): ^ w(S), S C = w(N)}. (17)
i£S i=1
Then Ac M and condition G Ck(£j), 1 < j < p is realized (if A ^ 0 and M ^ 0).
4. Example
Consider cooperative game r with three players. Let the values of characteristic function be as following: v(1) = v(2) = v(3) = 1,v(1, 2) = v(1, 3) = v(2, 3) =
5,v(1, 2, 3) = 7.
As in the previous sections, construct multistage game G. In our example we consider four stages game. We use two types of imputations: proportional imputation Y = (71,72,73) and Shapley value $ = (^,^2,^3). Using that imputations we get two optimality principles for multistage game, denote as C($>) optimality principle, generated by Shapley value and denote as C(7) optimality principle, generated by proportional imputation 7.
Construct the set L; (see (17)) which is an intersection of optimality principles C(7) and C'($). Then we take the imputation £ G L, and construct for £ imputation distribution procedure 3 and a new optimality principle, generated by the new imputations £ such as £jk = 3ik, i = 0,1,...,n,k = 0,...,m in stages games.
As a result, we get that new optimal principle C'(£), which is practically congruent with the intersection L. To construct the graph of the intersection L we use (0-1) reduced form for each multistage game. Coordinations of vertexes of the optimality principle C(<S>) for multistage game G\ generated by Shapley value are:
(0.5000000012,0.2499999994,0.2499999994),
(0.2499999994,0.5000000012,0.2499999994),
(0.2499999994,0.2499999994,0.5000000012).
Coordinations of vertexes of the optimality principle C(7) for multistage game G2 generated by proportional imputation are:
(0.4999999924, 0.2500000038,0.2500000038),
(0.2500000038, 0.4999999950,0.2500000012),
(0.2500000012,0.2500000038,0.4999999950).
The vertexes of the intersection L have following coordinates:
(0.5000000012,0.2499999994,0.2499999994),
(0.2499999994,0.5000000012,0.2499999994),
(0.2499999994,0.2499999994,0.5000000012).
Coordinates of the vertexes of the optimality principle C(£) for the new multistage game G3 generated by imputation £ are:
(0.5000000050, 0.2499999988,0.2499999962),
(0.2499999962, 0.5000000076,0.2499999962),
(0.2499999962, 0.2499999988,0.5000000050).
In our example we get that imputation £ G L generates new optimality principle C*(£) for the multistage game. For this optimality principle C(£) we have that imputation £ G L also belongs to optimality principle C'(£), so conditions of definition 4 are realized and the intersection L is internal time consistent optimality principle.
Fig, 1 : Intersection of optimality principles of multistage games.
References
Petrosjan L.A., Zenkevich N. A., Semina E. A. 1998. Game theory. M.
Filar G. A., Petrosjan L. A. 2000. Dynamic Cooperative Games. International Game Theory Review. Vol. 2. (1): 47-66.
Korniyenko E.A. 2002. Cooperative Multistage Hierarchycal Game on a Tree. In: Proceedings of the Tenth International Symposium on Dynamic Games and Applications Vol. 1: 440-445.
Gubar E.A. 2005. Optimality principle properties in cooperative multistage hierarchical games. In: Proceedings of International Conference in memory of V.
I. Zubov, Stability And Control Processes. Ed.: D.A. Ovsyannikov, L.A. Petrosjan. Vol. 1: 533-543.
