CC BY
4Contributions to Game Theory and Management, X, 245—286
Cooperation in Bioresource Management Problems *
Anna N. Rettieva
Institute of Applied Mathematical Research Karelian Research Centre of RAS Pushkinskaya str., 11, Petrozavodsk, 185910, Russia E-mail: [email protected] WWW home page: http: //mathem.krc. karelia.ru/member.php?plang=r&id=25
Abstract This paper is devoted to overview of the previously available and the author's own results of cooperative behavior analysis in dynamic games related to bioresource management problems. The methodological schemes to maintain the cooperation are considered and modified. The incentive condition for rational behavior and characteristic function construction method are presented. The question of coalition stability is revised and extended. The cooperative behavior determination schemes for games with asymmetric players are obtained. Some analytical and numerical modelling results for particular dynamic bioresource management problems are presented.
Keywords: dynamic games, bioresource management problem, Nash equilibrium, cooperative equilibrium, incentive equilibrium, dynamic stability, imputation distribution procedure, incentive conditions for rational behavior, coalition stability, asymmetric players, different planning horizons.
1. Introduction
This paper is dedicated to overview of the results of rational behavior analysis in dynamic bioresource management problems. The primary aim of rational resource exploitation consists in sustainable development of a population. Therefore, studying the difference between cooperative and egoistic (individual) behavior in optimal bioresource management problems represents an important issue (e.g., see (Kaitala and Lindroos, 2007; Lindroos et al., 2007)).
Optimal control problems for biological objects are very popular among researches. Many papers have been dedicated to these problems. Classical bioresource dynamic models were investigated in (Gimelfarb et al., 1974; Clark, 1985; Goh, 1980). The papers (Baturin et al., 1984; Puh, 1983; Selutin et al., 1999) are dedicated to models with migration processes. Optimal control models of interacting biological species are considered in (Bazikin, 1985; Chaudhuri, 1986; Silvert and Smith, 1977). Discrete-time bioresource optimal control problems were considered in the papers (Abakumov, 1993; Il'ichev et al., 2000; Shapiro, 1979). Models with age-distributed populations are investigated in (Abakumov, 1994; Baturin et al., 1984; Gurman, 1978; Svirezhev and Elizarov, 1972).
The game-theoretic approach for bioresource management problems was pioneered by Smith M.J. (Smith, 1968). Haurie A. (Haurie and Tolwinski, 1984), Petrosyan L.A. (Petrosyan and Zakharov, 1981; Petrosyan and Zakharov, 1997), Tolwinski B. (Tolwinski et al., 1986), Levhari D. (Levhari and Mirman, 1980), Mir-man L.J. (Fisher and Mirman, 1992), Vislie J. (Vislie, 1987) and many others applied the game-theoretic approach to resource management problems. The optimal
* This work was supported by the Russian Science Foundation, project no. 17-11-01079.
noncooperative and cooperative players' behavior in harvesting problems were obtained in (Ehtamo and Hamalainen, 1993; Hamalainen et al., 1984; Lindroos et al., 2007; De Zeeuw, 2008; Kulmala et al., 2009; Lindroos et al., 2007). Levhari D. and Mirman L.J. (Levhari and Mirman, 1980) presented the "fish war" model which is convenient for analyzing bioresource exploitation processes in the discrete-time setting. This framework proceeds from the power function of population evolvement and the logarithmical functions of "instantaneous" payoffs. Then the total payoff of a player forms a finite or infinite sum of discounted instantaneous payments. Here, Nash equilibrium strategies and cooperative strategies are defined analytically.
As is well-known, cooperation leads to a sparing mode of bioresource exploitation. The special importance of cooperative behavior for "common resource" exploitation was stressed by Nobelist E. Ostrom (Ostrom, 1990). This review will focus on the results by Ehtamo H., Fisher R.D., Hamalainen R.P., Haurie A., Kaitala V., Leitmann G., Lindroos M., Mirman L.J., Tolwinski B. (Fisher and Mirman, 1992; Ehtamo and Hamalainen, 1993; Kaitala and Lindroos, 2007; Tolwinski et al., 1986); Haurie and Tolwinski, 1984; Fisher and Mirman, 1996; Hamalainen et al., 1984) in this regard.
There are several methodological schemes to maintain a cooperation. Here we focus on two of them: incentive equilibrium and time-consistent imputation distribution procedure.
The concept of cooperative incentive equilibrium was introduced by Ehtamo H. and Hamalainen R.P. (Ehtamo and Hamalainen, 1993), as a natural extension of D.K. Osborn's work (Osborn, 1976) about cartel stability. In this concept players punish each other for a deviation from cooperative behavior by changing their optimal cooperative strategies.
The question of dynamic stability in differential games has been investigated in the past three decades. Haurie A. (Haurie, 1976) raised the problem of instability of the Nash bargaining solution. The concept of time-consistency (dynamic stability) was introduced by Petrosyan L.A. (Petrosyan, 1977). Time-consistency involves the property that, as the cooperation develops, participants are guided by the same optimality principle at each time moment and hence do not have incentives to deviate from cooperation. Petrosyan L.A. (Petrosyan and Danilov, 1979) has developed the notion of time-consistent imputation distribution procedure. Petrosyan L.A. (Petrosjan, 1993; Petrosjan and Zenkevich, 1996) offered a method of regularization to construct time-consistent solutions. Petrosyan L.A. and Zaccour G. (Petrosjan and Zaccour, 2003) presented time-consistent Shapley value allocation in a differential game of pollution cost reduction. Yeung D.W.K. (Yeung, 2006) introduced the "irrational-behavior-proofness" condition that guaranties the stability of cooperative agreement against unpredictable collapse of the coalition.
The analysis of stable international environmental agreements (IEA) in game theory was pioneered by Barrett S. (Barrett, 1994), Carraro C. and Siniscalco D. (Carraro and Siniscalco, 1992), and was surveyed in (Ioannidis et al., 2000) and (Finus, 2008). IEAs typically use the concept of internal and external stability (D'Aspremont et al., 1983). In classical works (Barrett, 1994; Barrett, 1994) it is assumed that only one coalition can be formed.
The "new coalition theory" (Bloch, 1995; Yi, 1997; Carraro, 2000; Finus, 2008) does not restrict coalition formation to a single coalition but allows for the exis-
tence of multiple coalitions. Studies in this direction were published in (Ray and Vohra, 1997; Yi and Shin, 1995; Bloch, 1996; Osmani and Tol, 2010; Eyckmans and Finus, 2003). The main questions investigated were the rules of coalition formation. They can be Open Membership Game (Yi and Shin, 1995), Exclusive Membership Game (Eyckmans and Finus, 2003; Finus and Rundshage, 2003), Coalition Unanimity Game (Bloch, 1996) and Equilibrium Binding Agreements (Ray and Vohra, 1997).
Most of the papers on coalition stability concern the agreement on emission reduction, and only few of them apply these concepts to fisheries (De Zeeuw, 2008; Kulmala et al., 2009; Pintassilgo and Lindroos, 2008; Lindroos, 2008).
Traditionally, cooperative behavior analysis in bioresource management problems rests on the assumption of identical discount factors for all players. If these factors differ (players are asymmetric), standard techniques do not assist in evaluating players' payoffs under cooperation. As a matter of fact, the cooperative behavior design problem is underinvestigated in this case, even though asymmetry appears widespread in real ecological problems. For instance, countries concluding a cooperative agreement can have different rates of inflation, environmental conditions, and so on. The papers (Munro, 1979) and (Vislie, 1987) demonstrated that bioresource management conflicts often occur due to the existing difference in discount factors (time preferences). Consequently, a substantial role in cooperative behavior analysis of bioresource management problems belongs to seeking an optimal compromise in the case of heterogeneous goals pursued by players (different discount factors and fishing costs).
The publication (Breton and Keoula, 2014) suggested constructing cooperative payoff as the weighted sum of individual payoffs (in the continuous-time setting, see (Plourde and Yeung, 1989)). This approach draws criticism: a player with a higher discount factor leaves the bioresource exploitation process quite soon, but has to obtain its share of the total payoff of a coalition. The cited work demonstrated that all utility from a cooperative agreement goes to participant 1 if the weight coefficients are defined by the Nash bargaining solution. Note that this infringes upon the interests of player 2, which is inadmissible in a cooperative agreement. An alternative approach was introduced in (Sorger, 2006) via a bargaining scheme.
Cooperative and noncooperative behavior analysis in bioresource management problems with random planning horizons is an important problem, both theoretically and practically. The authors (Marin-Solano and Shevkoplyas, 2011) and (Shevkoplyas, 2011) constructed cooperative strategies and time-consistent solutions in the case of a random planning horizon obeying a given distribution.
The Nash bargaining solution was adopted in (Mazalov and Rettieva, 2014) to calculate a common discount factor; subsequently, the problem was reduced to determination of a time-consistent distribution of the total cooperative payoff. Munro G.R. (Munro, 2000) obtained cooperative strategies through maximization of the weighted sum of individual payoffs; moreover, it was noted that such solution satisfies the Nash product maximization problem. A well-known result of this paper is that cooperative payoff is equally shared in the case of side payments.
Another meaningful applied problem is to find cooperative payoffs in the case of different planning horizons. When one player exploits a bioresource for a shorter period than the other, the former joins the exploitation process (in our case, fishing) for a fixed time and is willing to enter cooperation (owing to obvious profitability).
But this player has a smaller planning horizon than its partner; and so, the player under consideration is interested in gaining more from cooperation than the player that continues harvesting individually.
The model with random planning horizons in the bioresource exploitation process is the most adequate to reality: external random factors can cause cooperative agreement breach and the participants know nothing about them in advance. For instance, fishing firms can go bankrupt, their fleet can be damaged, etc. In the case of countries, negative factors include an economic crisis, abrupt variations in the rate of inflation, international or national economic and political situations, and so on. All these processes possibly break a cooperative agreement, and cooperative behavior of participants in this case has not yet been examined.
According to the aforesaid, cooperative behavior design is very important. Here we present our results in this regard. Almost all the results are derived analytically, which allows their direct application to concrete biological populations with appropriate parameters.
Further exposition has the following structure. The types of the main investigated problems are shown in Section 2. Section 3 describes the obtained cooperation maintenance and cooperative behavior determination schemas. In Section 4 the problem of cooperative behavior determination for games with asymmetric players is considered. Different types of bioresource management problems are treated in Section 5, with cooperative behavior design, cooperation maintenance schemes and the results of numerical experiments. And finally, Section 6 provides the basic results and their discussion.
2. Main problems
2.1. Continuous-time models
The dynamics of the renewable resource is described by the equation
x'(t) = f (x(t),Ml(t), . .., un(t)), x(0) = xo , (1)
where x(t) > 0 denotes the resource size at time t, «¿(t) > 0 represents the strategy (exploitation intensity) of player i at time t, i = 1,... ,n, f (x(t),u1(t),... ,un(t)) indicates the natural growth function.
Denote u(t) = (u1(t),..., un(t)). We consider players' payoffs over the finite [0, T] or infinite time horizon in the forms:
Ji = f e-ptgi(x(t),u(t))dt + Gi(x(T)) (2)
o
and
Ji = / e-ptgi(x(t),u(t))dt, (3)
o
where gi(x(t), u(t)) denotes the "instantaneous" utility of player i at time t, p means the discount factor, 0 < p < 1.
Let uN(t) = (u^(t),..., u^(t)) be the Nash equilibrium in problem (1), (2) (or (1), (3)). Under cooperation players wish to maximize the sum of their profits:
n t n n
Jc = V Ji = e-ptY^ gi(x(t),u(t))dt + V Gi(x(T)) ^ max (4)
1=1 Jo i=i tl "(t)
or
Jc = / e^Vgi(x(t),M(t))dt ^ max . (5)
Jo ^ «(t)
. -, «(
i=i
Let the set of strategies uc(t) = (uf(i),..., «n(t)) be the solution of the problem (1), (4) (or (1), (5)) and xc(t) be the cooperative trajectory derived from the equation (1) applying the strategies uc(t).
2.2. Discrete-time models
The renewable resource evolves according to the equation
xt+i = f (xt,«t), xo = x, (6)
where ut = (u1t,..., unt), xt denotes the resource size at time t, uit represents the strategy (exploitation intensity) of player i at time t, i = 1,..., n. The players' payoffs take the forms:
n
Ji = ^igi(xt,«i) (7)
t=0
and
w
Ji = ^3^igi(xi,«i), (8)
t=0
where gi(xi,ui) denotes the "instantaneous" utility of player i at time t, S means the discount factor, 0 < S < 1.
Let = («ft,..., «2t) be the Nash equilibrium of the game (6), (7) (or (6), (8)). Under cooperation the discounted sum of players' total utilities over the finite [0, m] or infinite time horizon is maximized:
m n
Jc = E gi(xt ,«t) (9)
t=0 i=1
or
w n
JC = E gi(xt ,ut) . (10)
t=0 i=1
Let the set of strategies = («it,... , «nt) be the solution of the problem (6),
(9) (or (6), (10)) and xtc be the cooperative trajectory derived from the equation (6) applying the strategies w£.
3. Cooperation maintenance 3.1. Incentive equilibrium
One of the methodological schemes to maintain the cooperation is the cooperative incentive equilibrium. This concept was introduced in the paper (Ehtamo and Hamalainen, 1993) as a natural extension of Osborn's work (Osborn, 1976) about cartel stability. The incentive equilibrium is applied for maintaining the cooperation and punishing the player who deviates. This concept is presented for the problem with two players.
Following (Ehtamo and Hamalainen, 1993) we assume that the strategy of player
i is a causal mapping 7i
Uj ^ Ui
(uj € Uj), i,j = 1, 2, i = j, where Ui denotes
the set of admissible strategies of player i, i = 1, 2. In order to give the definitions for both the continuous and the discrete cases we will omit the time parameter in the following definitions.
Definition 1. (Ehtamo and Hamalainen, 1993). A strategy pair (71,72) is called the cooperative incentive equilibrium if
7l(u2) :
72(u1):
Ji(u1,u2) > J1 (ui, 72 (ui)) Vui € Ui, J2(u1,u2) > J2(yi(U2), U2) Vu2 € U2 .
Thus, when players use incentive equilibrium strategies it is not advantageous for them to deviate from the initial cooperative agreement. The player's profit under deviation is less than under cooperation. In the traditional statement players control their behavior, punishing for deviation by changing the cooperative strategies (see Fig. 1). In (Ehtamo and Hamalainen, 1993) players use punishment strategies which are proportional to the difference between the cooperative and deviating strategies.
(U2i(t),g2(Uii(t))
Ji(Ui,Y2(Ui))
j2(gl(Ul),U2)
resource 1 resource n
ecological system
u
u
Fig. 1. Traditional cooperative incentive equilibrium
In the papers (Mazalov and Rettieva, 2007; Mazalov and Rettieva, 2008; Mazalov and Rettieva, 2010) we presented a new scheme where the center controls the cooperation agreement by changing the harvesting territory.
Let us divide the water area into two parts, s(t) and 1 — s(t) (st and 1 — st in discrete-time models), where two players exploit the fish stock. The dynamics of the fishery and the players' payoffs have the same forms (1)-(10), but the strategies also depend on the territory sharing
ui(t) = ui(t,s(t)), i = 1, 2
or
Uit = Uitst , i = 1, 2 .
Denote by sc the territory sharing under cooperation. Assume that players deviating from the cooperative equilibrium point are punished by the center proportionally to the value of deviation. So if the first player deviates the center increases
sc, and if the second player deviates - decreases sc proportionally to the difference between cooperative and deviating strategies. The proposed concept is given in the following definition (the time parameter is omitted)
Definition 2. A strategy pair (71,72) is called the cooperative incentive equilibrium if
u1(sc)= Yi(u2(sc)), u2(sc)= Y2(u1(sc)), J1(«1(sc),M2(sc)) > J1(u1(s), 72(u1(s))) Vu1 G U1, 0 < s < 1, J2K(sc),w2(sc)) > J2(71(u2(s)),«2(s)) V«2 G U2 , 0 < s < 1 .
The application of this scheme for cooperation maintenance is presented in Fig. 2. In Section 5.1 we present the results obtained for different game-theoretic models.
Fig. 2. New cooperative incentive equilibrium
3.2. Dynamic stability and conditions for rational behavior
Let us consider the infinite time horizon problem (1), (5) or (6), (10). For finite horizon problems the following definitions are similar.
Denote the profit of coalition S G N as Js(u) = Y1 gi(x(t), u(t)) (or JS(u) =
¿es
J2 gi(xt,ut)).
¿es
For the cooperative variant of the game it is required to determine the characteristic function. There are several approaches to constructing the characteristic function (Gromova and Petrosyan, 2015). The classical one is to determine the profit of coalition S assuming that the outside players form the coalition N \ S and play against the coalition S (zero-sum game, see (Neumann and Morgenstern, 1953)).
Characteristic function construction
In the papers (Mazalov and Rettieva, 2010; Mazalov and Rettieva, 2014) we constructed the characteristic function in two unusual forms. In the first model players outside coalition K switch to their Nash strategies, which were determined for the initial noncooperative game. This approach was presented by Petrosyan L.A. and Zaccour G. (Petrosjan and Zaccour, 2003). It is the case where players have no information about the fact that the coalition was formed. In the second model we
present a new approach where players outside coalition K determine new Nash strategies in the game with N\K players. This case corresponds to the situation when players know that coalition K is formed.
Model without information (Petrosjan and Zaccour, 2003). In this case players forming coalition K don't inform others. Therefore, players outside coalition K use their Nash strategies determined for the noncooperative case.
The following definitions are given for the game (6), (8). Denote by
,N
(u
uNt) the Nash equilibrium.
To determine the cooperative payoff of coalition K it is required to solve the next problem:
œ
J =¿J s
t=0
í!Z
ieK
gi(Ut)
max ,
ui, ieK '
where
Ui, i G K , N i G N\K.
u
Model with informed players. Let's consider the case where players outside the coalition K determine new Nash strategies in the game with N\K players. This case corresponds to the situation where players know that coalition K is formed.
To determine the cooperative payoff of coalition K it is required to solve the next problem:
J
K
Z^fZ gi(Ut)
t=0
ie K
max ,
ui, ieK '
where
Ui, i G K , N i G N\K,
U:
and the individual players' strategies UN, tion problems:
i G N\K are defined from the maximiza-
Ji
œ
z
t=0
Stgi(Ut)
max , i G N\K .
ui,ien\k
In (Mazalov and Rettieva, 2010) these approaches were applied for the fish war model with many players and we present some results in Section 5.3.
Using classical or new approaches we determine the characteristic function V(S, 0) as the profit of coalition S, S C N. When the characteristic function is determined, the imputation set can be defined as
e = {£(0) = (£i(0),...,£n(0)) :
n
£ £i(0) = V(N,0),£i(0) > v(i,0), i = 1,.. .,n}.
i=i
Similarly we determine the characteristic function V(S,t) and the imputation set e(t) = (ei(t),..., en(t)) for every subgame started from the state xC (or xc(t)) at time t. Further assume that one of the cooperative optimality principles is chosen; it can be proportional solution, C-core, n-core, the Shapley value or another.
The concept of time-consistency (dynamic stability) was introduced by Petrosyan L.A. (Petrosyan, 1977). Time-consistency involves the property that, as
the cooperation develops participants are guided by the same optimality principle at each time moment and hence don't have incentives to deviate from cooperation. In the paper (Petrosyan and Danilov, 1979) the notion of time-consistent imputation distribution procedure was developed.
Definition 3. The vector ,0(t) = (^(t),..., ,0„(t)) is an imputation distribution procedure (Petrosyan and Danilov, 1979; Petrosyan and Danilov, 1985) if
/• w
&(0) = e-pt&(t) dt, i = 1,...,n,
Jo
or, for a discrete-time problem
w
£¿(0) = ^¿^¿(t), i = 1,...,n.
t=o
The main idea of this scheme is to distribute the cooperation gain along the game path. Then ^¿(t) can be interpreted as the payment to player i at time moment t.
Definition 4. The vector ,0(t) = (^i(t),... , ,0„(t)) is a time-consistent imputation distribution procedure (Petrosyan, 1977; Petrosyan and Danilov, 1979) if for all t >
o t
6(0)=/" e-pTA(r) dr + e-pi£i(t), i = 1,...,n,
o
or, for a discrete-time problem
t-i
^¿(0) = 53¿t^¿(r) + ¿iei(t), i = 1,...,n,
T=0
where £j(t) is the imputation for player i at time t.
Here, players following the cooperative trajectory are guided by the same opti-mality principle at each current time and hence do not have any reasonable motivation to deviate from the cooperation agreement.
The application of these concepts to bioresource management problems are given in Sections 5.2, 5.3.
Nonetheless, some irrational player can break out of the cooperation. To indemnify players against the loss of profits in this case Yeung D.W.K. (Yeung, 2006) introduced the following condition.
Definition 5. The imputation £ = (£i,...,£„) satisfies the irrational-behavior-proofness condition (Yeung, 2006) if
i e-pT&(t) dr + e-ptV(i, t) > V(i, 0), i = 1,. .., n ,
o
or, for a discrete-time problem t
^V A(t )+ ¿t+1V (i,t +1) > V (i, 0), i =1,...,n (11)
T=o
for all t > 0, where ,0(t) = (^i(t),..., ^„(t)) is the time-consistent imputation distribution procedure.
If this condition is satisfied, then player i is irrational-behavior-proof because irrational actions that break the cooperative agreement will not bring her payoff below initial noncooperative payoff.
In the papers (Rettieva, 2009; Mazalov and Rettieva, 2010; Mazalov and Rettieva, 2012) we introduced a new condition for discrete-time problems which is stronger than Yeung's condition and is easier to verify.
Definition 6. The imputation £ = (£i,..., £n) satisfies the each step rational behavior condition if
A(t) + ¿V (i,t + 1) > V (i,t), i = 1,...,n (12)
for all t > 0, where fl(t) = (fli(t),..., fln(t)) is the time-consistent imputation distribution procedure.
The proposed condition offers an incentive to player i to maintain cooperation because at every step she gains more from cooperation than from noncooperative behavior.
In the series of papers (Rettieva, 2010; Mazalov and Rettieva, 2010; Mazalov and Rettieva, 2011; Rettieva, 2011) we verify these conditions for different models (see Sections 5.2, 5.3).
3.3. Coalition stability
For the coalition structure not only external and internal stability (D'Aspremont et al., 1983) should be examined but also the possible moves of players from one coalition to the other. Carraro (Carraro, 1997) presented the notion of intercoalition stability for such analysis.
In the papers (Rettieva, 2011; Rettieva, 2012) we extend the intercoalition stability concept to the situation where not only one player but a set of coalition members can join the other coalition (coalitional stability). This concept is close to the strong Nash equilibrium coalition structure (Finus and Rundshage, 2003), a-and fl- core concepts (Bloch, 1996).
We consider the bioresource management problem with two types of players: N = {1,... ,n} and M = {1,... ,m}. The coalition structure where players of each type form a coalition is investigated. Hence, there can be two coalitions (K C N and L C M) and single players of each type (N\K and M\L) in the game. The sizes of the coalitions are the subject of investigation.
The most popular stability concept that is applied in game-theoretical literature on IEAs is external and internal stability (D'Aspremont et al., 1983).
Definition 7. Coalition K is internally stable if
L) = ^-Vk(K, L) > ViN(K\{i}, L), Vieif. (13)
Definition 8. Coalition K is externally stable if
V:N(K, L) > V^(KU{i},L) = -^—Vk+1(KU{i},L), Vi G N\K. (14)
k + 1
Internal stability means that no coalition member wishes to leave the coalition and become a singleton. External stability means that no singleton wishes to join the coalition.
The paper (Rettieva, 2012) extends the intercoalition stability concept to the situation where not only one player but a set of coalition members can join the other coalition. The intercoalition stability is now a special case of coalitional stability. Coalition K is coalitionally internally stable if
(K,L) > V/+p(K\P,L u P), Vi G P C K, |P| = p. (15)
Coalition K is coalitionally externally stable if
V](K,L) = ^Vl(K,L)>VJk+q(KiJQ,L\Q), V? G Q C L, \Q\ = q. (16)
Here internal stability means that no set of members of coalition K wishes to leave it and join coalition L. External stability means that no set of members of coalition L wishes to leave it and join coalition K. For coalition L conditions take the forms
Vl(K, L) > (K U Q, L\Q), Vj G Q C L,
(K, L) > V/+p(K\P, L U P), Vi G P C K,
which coincides with (15), (16).
The presented concept is given in the next definition.
Definition 9. Coalition structure (K, L) is stable if conditions (15), (16) are fulfilled.
For P = {i} and Q = { j} this definition coincides with the intercoalition stability (Carraro, 1997; Osmani and Tol, 2010).
The presented stability concept enlarges the intercoalition stability for the models with two or more coalitions and possible moves of a set of coalition members. Moreover, as it will be shown below, the coalitions with a large number of members are stable under this concept.
In Section 5.4 we give the results for the great fish war model (Fisher and Mirman, 1996) with coalition structure.
4. Cooperation for asymmetric players
Our papers (Rettieva, 2012; Rettieva, 2014; Mazalov and Rettieva, 2015) suggest designing and stimulating cooperative behavior applying the Nash bargaining solution. The presented approach removes the need for summing up the payoffs of asymmetric players (Breton and Keoula, 2014). The bargaining scheme yields an absolutely different solution (e.g., see a classical example in (Owen, 1968)). Cooperative behavior design based on maximization of the weighted sum of players' payoffs may lead to the existence of parameter domains where the cooperative payoffs of players are smaller than their noncooperative counterparts (Breton and Keoula, 2014). This is impossible in the suggested scheme with cooperative behavior defined by the bargaining solution: under some parameters, players' payoffs are greater or equal to Nash equilibrium payoffs (Section 5.5 provides numerical experiments illustrating this fact).
Another meaningful applied problem is to find cooperative payoffs in the case of different planning horizons. The model with random planning horizons in the
bioresource exploitation process is the most adequate to reality: external random factors can cause cooperative agreement breach and the participants know nothing about them in advance.
In what follows, we explored a discrete-time game-theoretic bioresource management problem. Players apply different discount factors which can be interpreted as their heterogeneous time preferences. A generalization of this model is when players' planning horizons differ due to cooperative agreement breach or other reasons. Although conclusion of an agreement implies fixed exploitation periods, external factors can force a player to leave the game. Therefore, it seems natural to consider planning horizons as random variables.
4.1. Models with different discount factors
We consider discrete-time bioresource management problems (6), (7) and (6), (8) with two players. Denote by uN = (uN, uN) the Nash equilibrium of the problem (6), (8), Vi(x,Si), i = 1, 2 denotes noncooperative payoffs, respectively. uj^ = (uN,uNt) and Vin(x,Si) , i = 1, 2 give the Nash equilibrium strategies and payoffs in n-step game (6), (7).
The papers (Rettieva, 2012; Rettieva, 2013) demonstrate how to determine the total discount factor in the case where the cooperative payoff is distributed proportionally for infinite-time problems. The schemes for determining the total discount factor in order to construct cooperative payoff are offered. Assume that players use the joint discount factor S, which should be determined. So, the players solve the following problem
J = Z St gi(uit,u2t) + 92(uit,u2t)
t=0
^ max ,
Ult,U2t >0
where 0 < S < 1 denotes the unknown total discount factor.
V(x, S) denotes the cooperative payoff in this case. We suppose that the cooperative payoff is distributed in the portions 7V(x, S) and (1 — y)V(x, S) among players.
In the paper (Rettieva, 2012) for the fish war model it was shown that the joint discount factor for the case where cooperative payoff is distributed proportionally among players exists. As a result we get the set of admissible parameters S and 7. To construct the solution we propose to adopt the Nash bargaining scheme. It is necessary to solve the problem
(7V(x,S) — Vi(x,Si))((1 — 7)V(x,S) — V2(x,S2)) ^ max .
0<^,7<i
In the papers (Rettieva, 2014; Mazalov and Rettieva, 2015) for the game (6), (7) we withdraw from total discounting factor design and determine cooperative strategies applying the Nash arbitration procedure. Two bargaining schemes are introduced, viz. the one for the whole duration of the game and the recursive arbitration procedure which applies the arbitration scheme at each shot of the game. In the first case cooperative strategies and payoffs are defined by resolving the Nash product maximization problem for the whole duration of the game
(Vinc(x, Si) — Vin(x, Si))(V2nc(x, S2) — V2n(x, S2)) —^ max ,
Ui ,U2>0
CO
and in the second case the Nash arbitration scheme gets activated at each shot of the game.
It has been established that, within the framework of the proposed scheme, the cooperative payoffs of the players are greater or equal to (under some parameters) their payoffs gained by egoistic behavior (Section 5.5 provides numerical experiments illustrating this fact).
In Section 5.5 we present the cooperative behavior determination adopting the Nash bargaining solution for the fish war model.
4.2. Models with different planning horizons 4.2.1. Fixed planning horizons
Cooperative behavior has not yet been analyzed in the statement of different planning horizons. In this context, we mention the papers (Shevkoplyas, 2011; Marin-Solano and Shevkoplyas, 2011) where the planning horizon is a random variable with a given distribution. When the harvesting time of a player is smaller than that of another, the former harvests the fish stock for a fixed time and is willing to enter cooperation (owing to obvious profitability). But this player has a smaller planning horizon than its partner; and so, the player under consideration is interested in gaining more from cooperation than the player which continues harvesting individually. The cited authors designed a dynamically stable allocation procedure for this model, but with identical discount factors and harvesting times.
The papers (Rettieva, 2015; Mazalov and Rettieva, 2015) introduced the Nash bargaining solution to construct cooperative strategies in the case of different planning horizons.
Consider the harvesting process with the dynamics (6) and different planning horizons. Players 1 and 2 harvest the fish stock during ni and n2 steps, respectively. For the sake of definiteness, suppose that n1 < n2. Therefore, in this model the players enter cooperation on the time period [0, n1] and we have to find their cooperative strategies. After step n1 till step n2 player 2 continues the harvesting process individually. Hence, the players' payoffs are defined by
ni ni n2
Ji = £ ¿2 ln(uit), J2 = £ ¿2 ln(u2t)+ E ¿2 lnKt), (17)
t=0 2=0 t=n1+1
where u? (i = 1, 2) denote the cooperative strategies and w2 indicates the strategy of player 2 during individual catch.
To construct the cooperative strategies and payoffs of the players, apply the Nash bargaining solution for the whole duration of the game. Thus, it is required to solve the following optimization problem:
(V?(x,J1)[0,n1j - (x,J1)[0,n1j) x (Vc(M2)[0,n1] + VT(xcni,¿2)^2]
- VN(x, ¿2)[0, rn] - V2aN(xNni, ¿2)[n1, n2]) ^ max, (18)
where (x, ¿i)[0,n1] represent the Nash equilibrium payoffs, V2ac(xcni , ¿2)[n;[,n2] gives the payoff of player 2 owing to its individual harvesting after n1 steps of cooperative behavior, and V2aN(xNni , ¿2)[n;[,n2] is the payoff of player 2 owing to its individual harvesting after n1 steps of noncooperative behavior.
In Section 5.6 we present the cooperative behavior determination adopting the Nash bargaining solution for the fish war model with different harvesting times.
4.2.2. Random planning horizons
The model with random planning horizons in the bioresource exploitation process is the most adequate to reality: external random factors can cause cooperative agreement breach and the participants know nothing about them in advance. Therefore, it seems natural to consider planning horizons as random variables.
In the paper (Mazalov and Rettieva, 2015) we explore the model where players possess heterogeneous discount factors and, moreover, heterogeneous planning horizons. By assumption, players stop cooperation at a random step: external stochastic processes can cause cooperative agreement breach.
Suppose that players 1 and 2 harvest the fish stock during n1 and n2 steps, respectively. Here, n1 represents a discrete random variable taking values {1,... ,n} with the corresponding probabilities {01,... ,0n}. Similarly, n2 is a discrete random variable with the value set and the probabilities {w1,..., w„j. We believe that the planning horizons are independent. Therefore, during the time period [0, n1] or [0, n2] players enter cooperation, and the problem consists in evaluating their strategies.
The players' payoffs are determined via the expectation operator:
m
J1 = ^\X^1g1(M1i,M2i)1{ni<n2} + t=1
U2 ni
+ (Z 51 g1(u1t,u2t)+ Z Ô191(ult)) 1{ni >n2}} ,
t=1 t=n2+1 n2
J2 = 92(U1t,U2t)I{n2 <ni} + t=1
ni n2
+ (Ê S292(u1t,u2t)+ Z ^Kj) !{U2 >ni}} ,
t=1 t=ni+1
where uft specifies the strategy of player i when its partner leaves the game, i = 1, 2.
To define cooperative behavior, we employ the Nash bargaining solution; the role of status quo points belongs to the noncooperative payoffs of the players.
In Section 5.7 we present the cooperative behavior determination applying the Nash bargaining solution for the fish war model with random harvesting times.
An obvious advantage of the Nash bargaining solution consists in the feasibility of treating players individually. According to the conventional approach, the joint cooperative payoff function represents the sum of players' individual payoffs, which has little to do with real systems. For instance, if the players are neighboring countries, this becomes even impossible (especially in the case of different planning horizons). Other drawbacks of the traditional cooperative design are described in the Introduction and Section 4. In a certain sense, the Nash bargaining solution resembles a Nash equilibrium (see (Mo and Walrand, 2000)). The players act individually as before, but within the boundaries of a cooperative agreement.
5. Some results
Here some results of our investigations in the fields of cooperation maintenance and asymmetric players' problems are presented.
5.1. Incentive equilibrium Continuous-time model
A dynamic game model of bioresource management problem is considered in (Mazalov and Rettieva, 2007; Mazalov and Rettieva, 2008). The center (referee) who shares a reservoir, and the players (countries or fishing firms) that harvest the fish stock on their territory are the participants of this game. The equilibria are constructed in the case where the players punish each other for a deviation from the cooperative equilibrium (Ehtamo and Hamalainen, 1993) and in the case where the center punishes them for the deviations.
Let us divide the water area into two parts, s and 1 — s, where two players exploit the fish stock during T time periods. The center (referee) shares the reservoir. The dynamics of the fishery is described by the equation
x'(t) = F(x(t)) — qiEi(t)(1 — s)x(t) — q2E2(t)sx(t), 0 < t < T, x(0) = x0 , (19)
where x(t) > 0 is the population size at time t > 0, F denotes natural growth function of the population, E1(t), E2(t) > 0 give players' fishing efforts measured as the number of vessels involved in fishing at time t and q1, q2 > 0 denote catchability coefficients related to the unit fishing effort of the player.
We assume that E1, E2 belong to decision sets D1, D2. Let D1 = D2 C C([0, to)). Assume that fish population evolves according to Verhulst (Gurman, 1978) model
F(x) = rx^l — —^j ,
where r > 0 represents the intrinsic growth rate, and K > 0 denotes maximal natural object capacity.
The players' net revenues over the fixed time period [0, T] are defined by
T
J1 = g1(x(T)) + y e-plt[q1E1(t)(1 — s)x(t)(p1 — k191E1(t)(1 — s)x(t))]dt,
0
T
J = g2(x(T)) + y e-p2t[q2E2(t)sx(t)(p2 — ^^(t)sx(t))]dt, (20)
0
where is the price, kj gives catching cost, p¿ denotes the discount factor, i = 1, 2.
Functions gj(x) describe the salvage value of the stock at time T. Following usual assumptions on utility function we suppose that gj(x) > 0, gj'(x) < 0, i = 1, 2.
The player's profit is presented as an income over the time period [0, T] that depends on the difference between the price and the catching costs with discounting. Here, catching costs have quadratic forms.
Assume that players punish each other for a deviation from the cooperative equilibrium by increasing the control on the value which is proportional to the difference between cooperative and deviating strategies (Ehtamo and Hamalainen, 1993).
Proposition 1. The cooperative incentive equilibrium in the problem (19), (20) has the form
Y1(E2(t)) = E(t)+m(t)(E2(t) — Ec(t)), Y2(E(t)) = Ec(t)+n2(t)(E1(t) — E(t)),
where
,,, q2^iXi(t)s 1
»71 (i) =-ttttt;-7 > mW -
qi№M(t)(l - s)' ni(t)'
cooperative strategies EC(t), EC(t) and conjugate variables Ai(t), i = 1, 2 are defined in (Mazalov and Rettieva, 2010).
Denote by sc the territory sharing under cooperation. We assume that players deviating from the cooperative equilibrium point are punished by the center rather than by themselves, as was in (Ehtamo and Hamalainen, 1993).
Theorem 1. The cooperative incentive equilibrium in the problem (19), (20) takes the form
7i(W))= V^mwir =
a1(1 - s|(t))x(t) a2s1(t)x(t)
where
sc 1 - sc
4{t) = sC~m){E2{t)~m))' sUt) = s° + ~Em{El{t)~m))■■
and EC(t), EC(t), x(t), A(t) are defined in (Mazalov and Rettieva, 2010).
We give an example where after the second player's deviation at time instant t = 20 there is no return to cooperative behavior.
Traditional scheme. Fig. 3-5 present the parameters of the model in the cases of cooperation and deviation (dotted line). Fig. 3 shows the population dynamics. Fig. 4 presents the players' controls (in this model parameters ni and n2 and the controls E1 and E2, respectively, are almost equal). Fig. 5 shows the players' catch (v1(t) = q1E1 (t)(1 - s(t))x(t), v2(t) = q2E2(t)s(t)x(t)), respectively.
180000 /
160000 /
140000
120000
100000
0 10 20 30 40 50 *
1.4 1.2 1 0.8
0 10 20 30 40 50 't
v (t) 2000
1800
1600
1400
0 10 20 30 40 50 't
Fig. 3. Population size
Fig. 4. Players' controls
Fig. 5. Players' catch
Our scheme of incentive equilibrium. Fig. 6-11 present the difference between the parameters in the cases of cooperation and deviation (dotted line). Fig. 6 shows the population dynamics. Fig. 7 and 8 present the players' controls. Notice, the second player increases his fishing efforts and the first player decreases it. Fig. 9 shows water area sharing (s). One can see that s decreases from 0.5 to 0.1. Fig. 10 and 11 present the players'catch (v1(t) = q1E1(t)(1 — s(t))x(t), v2(t) = q2E2(t)s(t)x(t)), respectively. Notice, the first player's catch increases slightly, while the second player's catch decreases quickly (from 1420 to 900 individuals per time instant).
According to the results of numerical modelling the center's participation in optimal resource exploitation regulation has several interesting features. If the center's
®t)
180000 /
160000
140000
120000
100000
0 10 20 30 40 50 *
E(t) 1.4
1.2
0.8
0.6
0 10 20 30 40 50 *
R(t) i
-
1.4
1 0.8
0 10 20 30 40 50 't
Fig. 6. Population size
Fig. 7. Player 1's control
Fig. 8. Player 2's control
1
0.4 0.3 0.25 0.2
0 10 20 30 40 50 t
v(t)
1420
1400
1380
1360 \
0 10 20 30 40 50 *
1400 '-^^^
1300
1200
1100
1000
900
0 10 20 30 40 50 't
Fig. 9. Territory sharing
Fig. 10. Player 1's catch
Fig. 11. Player 2's catch
strategy is to punish defaulter player until the end of the planning period, the honest player has visible advantages even in comparison with cooperative equilibrium, and his opponent incurs remarkable losses. The center's strategy here is the territory sharing. The player who breaks the agreement achieved at the beginning of the game is punished by gradually decreasing the harvesting territory. This scheme can be easily realized in practice.
The economic feasibility of cooperation maintenance by the center is an advantage for players who keep agreement achieved at the beginning of the game. Therefore, there is no need for monitoring the opponent's actions that incur additional costs, and players completely rely on the center. In the case where players control each other's behavior, when the second player deviates the first player is compelled to increase his fishing efforts too, i.e. to incur additional cost on large number of ships' operation. In the case where the center punishes deviating players, the honest player reduces his fishing efforts conversely, but his catch increases that is connected with catch territory change. Thus, he gets larger profit with smaller expenses for ships' operation.
Discrete-time model
A discrete-time dynamic game model of bioresource management problem is considered in (Mazalov and Rettieva, 2008; Mazalov and Rettieva, 2009; Mazalov and Rettieva, 2011). Let us divide the water area into two parts: s and 1 — s, where two players exploit the fish stock. The center (referee) shares the reservoir. The players (countries or fishing firms) that exploit the fish stock during infinite time on their territory are the participants of this game.
The fish population evolves according to the equation (the modified fish war model (Levhari and Mirman, 1980)):
xt+1 = (sxt)a , Xo = x,
(21)
where xt > 0 is the population size at time t > 0, 0 <e< 1 gives natural death rate, 0 < a < 1 denotes natural birth rate.
Suppose that the players' utility functions are logarithmic. We consider the problem of maximizing the infinite sum of discounted utilities for two players:
w w
J1 S\ ln((1 - s)xtuj), J2 = sl ln(sxtu?), (22)
t=0 t=0
where 0 < u\ < 1 gives player i's fishing efforts at time t, 0 < Pi < 1 denotes the discount factor for player i, i = 1, 2.
To determine the cooperative equilibrium strategies ui, u? an approach of trans-fering from finite to infinite resource management problem is applied (see (Mazalov and Rettieva, 2011)).
Denote by sc the territory sharing under cooperation. Assume that the center punishes players for a deviation from the cooperative equilibrium. If the first player deviates the center increases sc, but if the second player deviates - decreases sc.
Theorem 2. The cooperative incentive equilibrium in the problem (21), (22) takes the form
e(1 — aS) e(1 — aS)
ll{U2) = ttt,-JT j 72 (wi) =
where
2(1 — s?) ' 2s1
sc 1 — sc sc--~{u2-uc2), si = scH--—{ux-ul).
In (Mazalov and Rettieva, 2011) it was shown that in the case of a short-time second player's deviation on step k
uk2 = uf + Ak
and his returning to cooperation after, the next properties are satisfied:
1 The steady-state population size under deviation is equal to cooperative one when the number of steps tends to infinity:
if ^ (£«<5.
2 The conditions of the incentive equilibrium are satisfied
Jotk > Jc , J°tk < Jc ,
where Jf denotes the player i's profit when both players apply cooperative strategies, J°tk gives the player i's profit when the second player deviates and the center punishes her (i = 1, 2).
3 The player who deviates losses less when the number of steps grows
Dn+1 < Dn,
where Dn = Jfn - J°tkn.
The developed approach can be applied for different population dynamics. In particular, the cases where the growth rate depends on nature-conservative measures of one of the players (Mazalov and Rettieva, 2011) were investigated. Namely, the growth rule has the forms:
xt+i = (esxt)a , 0 < a < 1
and
xt+1 = (ext)as , 0 < a < 1 .
s *
o O
2
2
1
5.2. Dynamic stability and conditions for rational behavior
Here, we don't focus on time consistent IDP construction. Our aim is to underline that the each step rational behavior condition (12) is much easier to verify than Yeung's condition (11). To show it we check both conditions for a bioresource management problem with two players (Mazalov and Rettieva, 2010).
Assume that the fish population evolves according to the equation (the fish war model (Levhari and Mirman, 1980)):
Xt+1 = (xt — u\ — u2)a , xo = x, (23)
where xt > 0 is the population size at time t > 0, a denotes natural birth rate, 0 < a < 1, ul, u2 > 0 give players' catch at time t.
The players' net revenues over the infinite time horizon take the forms
Ji =E ^ ln(ut) ,i = 1, 2 , (24)
t=o
where 5 denotes the discount factor, 0 < 5 < 1. The cooperative payoff has the form
MiJi + M2J2 , (25)
where denote the weighting coefficients, 0 < < 1, + ^2 = 1.
First, we determine the Nash equilibrium. The solution of the Bellman equation
Vi(x) = max{lnui + 5Vi(x — u1 — u2)a}, i = 1, 2, (26)
Ui> 0
is sought in the next form
V (x) = Ai ln x + Bi, i = 1, 2,
and we suppose that the optimal strategies are linear ui = Yix, i = 1, 2. Hence, from
1 — a
equation (26) we get the optimal catch uF = ulX =-x and the payoffs
2 — a
V1(x) = V2(x) = —!—]nx + —!—B, 1 — a 1 — 5
where a = a5, and
B = ln( 1 - a) H--— ln a--— ln(2 - a).
1 — a 1 — a
To determine the cooperative payoff (25) we apply the Bellman principle again. Similar reasoning leads to the total payoff under cooperation
^1,2(3;) = z ~ lnx + - - 7P1 2? 1 — a 1 — 5
where
Bi 2 = Mi ln Mi + M2 In A42 + ln(l — a) H--ln a.
1—a
The dynamics under cooperation is
t
t E
xt = aj=1 . (27)
The criterion of equal partition is considered as a solution of the cooperative game (23)-(25). This solution coincides with the Shapley value in two-person game and can be extended to the principle of cooperative gains' proportional division. The imputation in the problem (23)-(25) takes the form
where xt is obtained in (27).
Theorem 3. The incentive conditions for rational behavior are fulfilled in the problem (23)-(25).
Proof. First, verify the each step rational behavior condition (12). Rewrite it in the form i i
/xi ln/xi + (1 -/xi)ln(l -/xi)-
--In xt +
2 2
2
- ln( 1 - a) + --ln(2 - a)
1 — a
> 0.
It is easy to show that the expression in square brackets is greater than ln(2 — a) — 1 > 0. This inequality follows from
((l + l)b)2>e,
where b = -r—.
1 —a
Now, verify Yeung's condition (11). For presented model it takes the form
ln XQ + _L_ ln a{ * £ a, _ }+
t j=1
+ 2(ï~-<5) I*"1 + i1 — Ml) ln(l — Ml) —
- ln( 1 - a) + jZ- ln(2 - a)] > 0 .
The first expression and the expression in square brackets as was already proved are positive. Now, we need to show that
t a5(1 — 5t)
j=i
Notice that f (1) = 0. Therefore, it is sufficient to prove that f (t) is decreasing.
f®=Sa-(1-«)(1 -6)- <0'
Denote fi(t) = ln¿(1 — aS) — a1 ln(a5)(1 — 5). This function is decreasing f1 (t) < 0. To check that f1(1) < 0 consider
f2(a,S) = f1(1) = ln 5(1 — aS) — a ln(a5)(1 — 5) = = ln(a5)(1 — a) + (aS — 1) ln a .
Function f2(a, S) is increasing with respect to S and a since
iln(a) > 1— a+a ln(a) > 0 and
dh(a, Ô)
da
df2(a,S) 85
1—a
5
+
= 5—1—ln(a5)+5lna > ln(a)(5—1) > 0.
Finally, f2(1, S) = f2(a, 1) = 0, therefore f2(a, S) < 0.
As one can see in this simple case, the each step rational behavior condition (12) is easier to verify than the irrational behavior proofness condition (11).
5.3. Characteristic function construction
In the paper (Mazalov and Rettieva, 2010) we investigate the model with many players and infinite planning horizon in contrast to the traditional fish war model with two players (Levhari and Mirman, 1980). The characteristic function for cooperative game is constructed in two unusual forms.
Let n players (countries or fishing firms) exploit the fish stock during infinite time horizon. The dynamics of the fishery is described by the equation
xt+1 = (ext — EUit)a , xo = x, i= 1
(28)
where xt > 0 is the population size at time t > 0, e G (0,1) denotes natural death rate, a G (0,1) represents natural birth rate, uit > 0 gives the catch of player i, i = 1, .. ., n.
Suppose that the player i's utility function is logarithmic. Then the players' net revenues over infinite time horizon are defined by
Ji = E 5t ln(uit), i = 1,...,í
(29)
t=o
where 0 < S < 1 denotes the common discount factor.
To construct characteristic function in the first model we suppose that the players outside coalition K switch to their Nash strategies, which were determined for the initial noncooperative game (Petrosjan and Zaccour, 2003). It is the case where players have no information about the fact that coalition was formed. In the second model players outside coalition K determine new Nash strategies in the game with N\K players. This case corresponds to the situation where players know that coalition K is formed.
Model without information. First, we determine the Nash equilibrium and get the optimal catch
1
(30)
N
1 — a
n — a(n — 1)
and the payoffs
Vi(x)
In X + --; Bi , i = 1, . . . , n ,
1 — a
1—5
(31)
where
Bi =
1
ln
1 — a n — a(n — 1)
) +ln(1 — a) +
1 — a
ln
= a5 .
n
e
a
Now, we determine the payoff of any coalition K with k players. Suppose that players outside coalition K apply their Nash strategies determined in (11). Hence, we get the optimal catch
= (32)
k(n — a(n — 1))
and the payoff of coalition K
k1
VK(x) = --Inx + ---BK, (33)
1 — a 1 — o
where
„ k , /e(k — a(k — 1))\ ,, N , ka 1
BK = --In i -±-+ k(ln(l -a) - Ink) + --In a,.
1 — a V n — a(n — 1) / 1 — a
Last, we determine the payoff and optimal strategies in the case of full cooperation (grand coalition). From (15) and (16) we get
/ (I-*») • i
u, =-ex, i = i,... ,n,
1 n
n1
VI(x) = --Inx + ---5/, (34)
1 — a 1 — o
where 1
Bj = nBi + n(-ln(n — a(n — 1)) — Inn) .
1 — a
Finally, we have determined the characteristic function for the game starting at time t from the state x
' 0, L = 0 ,
V(L x t) = ; V({i},x,t) = Vi(x) L = , (35) V (L, x, t) ^ v(K, x, t) = Vk(x), L = K, (35)
V (l,x,t) = V/ (x), L = I,
where V,(x), VK(x), V/(x) are of the forms (12), (16) and (20).
In (Mazalov and Rettieva, 2010) it was proved that the characteristic function (22) is superadditive function.
Next, the imputation set should be determined. In (Mazalov and Rettieva, 2010) it was proved that the vector ^(t) = (^i(t),..., ^„(t)), where
A(t)= £i(t) — 0£i(t + 1), i = 1,...,n (36)
is time-consistent imputation distribution procedure.
Here, the Shapley value is adopted as the cooperative optimality principle. It takes the form
&(t) = —!—]nxt + —!—(Bi + Bi), i = l,...,n, (37)
1 — a 1 — o
where
B5 = —ln(l + (n- 1)(1 - a)) - Inn > 0. 1 — a
Theorem 4. The Shapley value (23) is time-consistent and both conditions for rational behavior ((11) and (12)) are satisfied.
Proof. From (4) we get
Pi{t) = —(lnxt - Jlnxt+i) + Bi + , i=l,...,n. 1 — a
Yeung's condition (11) takes the form
1 1 — S* 1 1 — S*
--(lnx0 -Sthixt) + ---(Bi + Bz) > --(In xo — Sf' In xt) + ---Bi
1 — a 1 — S 1 — a 1 — S
and is fulfilled as B^ > 0.
The each step rational behavior condition (12) takes the form
—!— (lnxt - Jlnxt+i) + Bi + Be > —(lnxt - Jlnxt+i) + Bi 1 — a 1 — a
and is also valid as B^ > 0.
Model with informed players. Consider the case where players outside coalition K determine new Nash strategies in the game with N\K players. This case corresponds to the situation where players know that coalition K is formed. Hence, the difference from the previous case is only in determining VK. For players from coalition K we solve the Bellman equation
Vk(x) = maxln ui + SVK(ex — ^^ ui — ^^ uN)a}, Ui ieK ieK ieN\K
where uN, i G N\K, corresponds to the solution of the Bellman equation for players outside the coalition K
Vi(x) = max {ln U + SVi(ex — ui — Ui)a} , i G N\K .
Ui EN\K ^ ^
i£K iEN\K
Now, we get the optimal catch of coalition K members
~K _ I — a
v'i - k(l + (n-k)(l-a)fX'lG
and the payoff of coalition K
k 1
VK{x) = --lnx+---BK, (38)
1 — a 1 — o
where
BK = k{-r—— Inf-;—-7T7;-r) + ln(l - a) + lna-Ink).
1 — a \1 + (n — k)(1 — a) J 1 — a
Hence, the characteristic function for the game starting at time t from the state x will be of the form
{0, L = 0,
V({i},x,t)=Vi(x),L = {i} ,
V(K, x, t) = Vk(x), L = K, (39)
V (I,x,t) = Vt (x), L = I,
where V,(x), VK(x), Vt(x) are of the forms (12), (27) and (20).
Theorem 5. The characteristic function (39) has a superadditive property if k, l >
w+l 3 '
This result shows that it is profitable to merge two coalitions when both of them have sufficiently large number of participants.
Similarly to the first model we determine the Shapley value and time-consistent imputation distribution procedure. From (27) we get
&(t) = —!—]nxt + —!—(Bi + Bi), i = l(40) 1 — a 1 — o
where
Ç ^ n! I 1 — a 1 + (n — &)(! — a) '
ken
-(* - ixt^- in(11T(n71)Sl;°) o - Hk -1))
v /V1 - a v1 + (n - k + 1)(1 - a)7 v " .1 - a v1 + (n - k)(1 - a)7 '
"(A - 1T(n71)il:°) J - In(A - 1))
v /V1 - a v1 + (n - k + 1)(1 - a)7 v
1 ln(l + (n-l)(l-a))-Inn.
n 1 E1
^ n
k=1
1 - a
The proof that the Shapley value is time-consistent and both conditions for rational behavior are satisfied is similar to Theorem 4.
Some properties of the characteristic function construction's variants were proved in (Mazalov and Rettieva, 2010):
1 The second model is better for free-riding.
2 The profit of coalition K in the first model is greater than in the second model.
3 The first model in the case of coalition K formation is better for population
size.
Fig. 12 presents time-consistent imputation distribution procedure (A(t)) for player i (dark line), player ¿'s Nash profit V^y (bright line) and her Shapley value
(0) (dotted line), i = 1,... ,n.
Notice that the distribution procedure is greater than the profit in noncoopera-tive case at every time instant. Hence, figure shows how to distribute the cooperative gain (the Shapley value) along the game path.
Now, we show the difference between the two approaches of coalition K formation.
Fig. 13 presents the population dynamics in the case of non-informed players (dark line) and in the case of informed players (bright line). As one can notice the population size in the first case is larger. This result shows that for ecological systems the situation where coalition is formed and other players don't have information about it is preferable.
0 2 4 6 8 10 12 14 16 18 20 t
(t)
Fig. 12. IDP, Nash profit and Shi(0) Fig. 13. Population size
Fig. 14 presents the difference between the coalition K's profits in two considered cases. Clearly, it is profitable for coalition to be formed insensibly.
Fig. 15 illustrates the difference between the profits of player i outside the coalition K in two considered cases. Notice, the second model is better for free-riding.
TO
-30
2 4 6 8 10 12 14 16 18 20 t
V(t) - =b-
0 2 4 6 8 10 12 14 16 18 20 t
Fig. 14. Profit of coalition K Fig. 15. Profit of player i outside K
5.4. Coalition stability
In (Rettieva, 2011; Rettieva, 2012) we consider a discrete-time game model related to a bioresource management problem (fish catching). The reservoir is divided into regions where players (countries or fishing firms) of two types harvest the fish stock. We assume that there are migratory exchanges between the regions (Fisher and Mirman, 1992; Fisher and Mirman, 1996). So the stock in one region (where players of type 1 exploit the fish) depends not only on the previous stock and catch in the region, but also on the stock and catch in the other region (where players of type 2 exploit the fish).
Here, in contrast to the grand coalition formation, we consider the coalition structure where players of each type can form a coalition. Therefore, there can be two coalitions and single players of each type in the game. The sizes of stable coalitions are the subjects of investigation.
Two ways to construct the players' optimal strategies are considered: all players decide simultaneously (Nash-Cournot strategies) or members of coalitions are assumed to be the leaders and players decide sequentially (Stackelberg strategies). Furthermore, the characteristic function is constructed in an unusual form: players outside the coalition K determine new Nash strategies in the game with N\K play-
ers. This case corresponds to the situation when players know that coalition K is formed (see Section 5.3).
We divide a fishery into regions, which are exploited by two types of players: i G N = {1,...,n} and j G M = {1,...,m}. The players (countries or fishing firms) that harvest the fish stock are the participants of the game.
The fish populations evolve according to the system of equations (the great fish war model (Fisher and Mirman, 1996)):
Xt+1 = xf1 y01 , Xo = x,
a2 02
yt+1 = yt xt , yo = y,
where xt > 0 is the population size in the first region at time t > 0, yt > 0 denotes the population size in the second region at time t > 0, 0 < ai < 1 gives natural birth rate, 0 < Pi < 1 denotes coefficients of migration between the regions, i = 1, 2.
Here, ai represents the direct effect of the stock on the stock in this territory in the next period. pi represents the effect of migration between two parts of the reservoir.
Let N = {1,... ,n} players exploit the stock xt and M = {1,..., m} players harvest the stock yt.
Suppose that the utility function of players are logarithmic. Then the players' net revenues over the infinite time horizon are defined by
œ œ
Ji = ^ St ln(uit) ,i G N, Jj = ^ St ln(vjt) ,j G M, (41)
t=0 t=0
where uit > 0, vjt > 0 give players' catch at time t > 0 (i G N, j G M), 0 < S < 1 denotes the common discount factor.
Each player is interested in maximizing the sum of her discounted utility. And the dynamics become
/ n -, aw m \ Pi
xt+1 = [xxt - £ Uit) [yt - J2 Vjtj , xo = x
i=1 7 j=1 (42)
/ m \a2 / n \P2 (42)
yt+1 = [yt - £ vjt) (^xt - E uitj , yo = y.
j=1
Nash-Cournot strategies. Players outside coalition K(L) determine new Nash strategies in the game with N\K (M\L) players. Players wish to maximize the following functionals
œ
J k
J '
E^E ln(vjt )
t=o jeL
ln(ukt )
t=o ieK
œ
Jf = £ Sf ln(uNt), i G N\K, JN = £ Sf ln(vjt), j G M\L .
t=o
t=o
œ
Stackelberg strategies. Assume that members of coalitions are the leaders and players decide sequentially. Hence, at first, singletons determine the Nash optimal strategies under the assumption that cooperative strategies are known. Then, the coalition members obtain their optimal catch.
I) Coalition members' strategies uk, i G K and vj, j G L are fixed. Singletons wish to maximize their net revenues
Jf = E ^ ln(uit), i G N\K, Jf = E ? In(vjt), j e M\L
t=0
under the dynamics
t=0
(x - E uit - E M¿t) (vt - E vjt - E j > xo = x,
v jcK \k ' v jeL jeM\l y
ß2
ieK
xt+i
vt+i = (vt - E vjt - E (xt - E uit - E uit ) ' vo = v •
jeL
jeM\l
ie K
ieN \k
Denote by UN, i G N\K and vN, j G M\L, the obtained strategies. II) Coalition members maximize the joint payoff
Jk = E ^ [E ln(Uit^ , J1 = E ^ [E ln(Vjt)
t=o ieK t=o jeL
under the dynamics
ixt - E uit - E (vt - E vjt - E jO
^ ieK ¿m^r ' V jeL T '
ieN\k
jeL
jeM\L
ß1
(vt- E j - E jt) (x - E uit - E uN) > vo = v
N o'er. A/T\ T ' N -icrc' Af-AT\ IS '
jeL
jeM\L
ie K
ieN\k
xo = x,
Denote by Uk, i G K and vj., j G L, the obtained coalition members' strategies.
In (Rettieva, 2012) it was proved that the payoff of a singleton is greater under Nash-Cournot strategies than under Stackelberg strategies and for a coalition member the opposite result is valid.
The fact that the payoff of a coalition member is greater in the case were two coalitions K and L form than in the case were players join into one mixed coalition K + L also was proved.
Then we checked the internal and external stability of our coalitions (D'Aspremont et al., 1983). Unfortunately, in our model, just like in the classical papers (Barrett, 1994; Carraro and Siniscalco, 1992), only small-size coalitions are internally stable (for Nash-Cournot strategies). For Stackelberg strategies, on the other hand, coalitions are internally, but not externally stable.
We adopt a new coalition stability approach (15), (16) for presented model (41), (42) and get the coalition stability conditions in the forms
Ck I
I p
\ \ - (p + l-h- s)\{h+ s - l)!(p - 1)!/! iris+h s_1+h] , ,
JLJL (p + Z)!(p-s)!(s -!)!(/ -h)\h\ [C ~C J" '
k q
EE
s=o h=i
(fc + g _ h _ g)l (fe + s _ l)!(g _ l)lfcl _
(k + q)!(q - h)!(h - 1)!(k - s)!s! [ ] " '
(44)
where the parameters are given in (Rettieva, 2012).
CO
CO
CO
CO
We consider this model for the set of parameters which are typical for the fish species in Karelian lakes and obtain the next results:
For Nash-Cournot strategies only the coalitions of size 1 (k = 1 or I = 1) are internally stable. External stability is valid for all k if n > 2 (for all I if m > 2).
For Stackelberg strategies internal stability is valid for all k if n < 35 (for all I if m < 51). Unfortunately, there are no externally stable coalitions.
Tables 1 and 2 present the coalition structures which are stable in the sense of coalitional stability (43), (44). We use the notation + for the coalitions that are stable for all p € [1, k] and q € [1,1]. Double numbers (first for p and second for q) represent the coalitions that are stable for p and q larger or equal these parameters.
k/l 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1 + + + + 1,3 1,4 1,5 1,6 1,7 1,8 1,8 1,9 1,9 1,8 1,7
2 + + + + + + 1,3 1,4 1,5 1,6 1,7 1,7 1,7 1,7 1,5
3 + + + + + + + + + 1,4 1,5 1,5 1,6 1,6 1,4
4 + + + + + + + + + + + 1,3 1,4 1,4 1,3
5 3,1 + + + + + + + + + + + + + +
6 4,1 + + + + + + + + + + + + + +
7 4,1 + + + + + + + + + + + + + +
8 5,1 + + + + + + + + + + + + + +
9 5,1 + + + + + + + + + + + + + +
10 4,1 + + + + + + + + + + + + + +
Table 1 . Nash-Cournot strategies
k/l 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1 + + 1,2 1,2 1,3 1,3 1,4 1,4 1,5 1,5 1,6 1,6 1,7 1,7 1,7
2 + + 2,2 2,2 2,2 2,3 2,3 2,4 2,4 2,5 2,5 2,6 2,5 2,7 2,7
3 2,1 + 2,2 2,2 2,2 2,3 2,3 2,4 2,4 2,4 2,5 2,5 2,5 2,6 2,7
4 2,1 2,2 2,2 2,2 2,2 2,3 2,3 2,3 2,4 2,4 2,5 2,5 2,5 2,6 2,6
5 3,1 2,2 2,2 2,2 2,2 2,2 2,3 2,3 2,4 2,4 2,4 2,5 2,5 2,6 2,5
6 3,1 3,3 3,2 2,2 2,2 2,2 2,3 2,3 2,3 2,4 2,4 2,5 2,5 2,5 2,5
7 4,1 3,2 3,2 3,2 3,2 2,2 2,3 2,3 2,3 2,4 2,4 2,4 2,5 2,5 2,5
8 5,1 4,2 4,2 3,2 3,2 3,2 3,3 3,3 2,3 2,4 2,4 2,4 2,5 2,5 2,4
9 5,1 5,2 4,2 4,2 4,2 4,2 3,3 3,3 3,3 3,3 2,4 2,4 2,4 2,5 2,4
10 6,1 5,2 5,2 5,2 4,2 4,2 4,2 4,3 3,3 3,3 3,3 3,4 2,4 2,4 2,3
Table 2. Stackelberg strategies
For intercoalition stability (p = 1 and q = 1) even the coalition structure consisting of all players (k = 10,I = 15) is stable for Nash-Cournot strategies. For Stackelberg strategies the maximal stable coalition structure is (k = 3,1 = 2).
For the situations where a set of coalition's members can move to other coalition, one can notice that the stability concept is valid for different coalitions' sizes (as coalition size is larger the set of it's members who have an incentive to move is larger too). For example, the coalition structure k = 3, I = 12 is protected against the possible moves of more than 5 coalition L's members and it is unstable for q < 5 (Nash-Cournot strategies). For Stackelberg strategies this coalition structure is also unstable (p > 2) since it is profitable for any coalition K's member to move to coalition L.
From the results of numerical modelling it can be noticed the load on the stock is minimal when players join into one coalition (cooperative case). However, the
formation of grand coalition is not natural for asymmetric players. Furthermore, we proved that it is less profitable for players to join into one mixed coalition that to form two coalitions.
Hence, to minimize the load on the stock the coalition structure should consist of large number of players and be stable. We give some advices for ecological managers to improve populations' growth in the case of asymmetric explores.
For Nash-Cournot strategies the internal stability can't be guaranteed, but it is protected from individual moves form one coalition to another (intercoalition stability). For Stackelberg strategies the coalitions are internally stable, but intercoalition stability condition is not valid.
Therefore, the manager first should determine the coalition formation process and then:
if it is Nash-Cournot, then one should use some mechanisms to internally stabilize the coalitions: it can be fines for breaking off the cooperative agreement, punishment schemas like incentive equilibrium (Mazalov and Rettieva, 2010) or transfers schemes. If it is successfully done then it is unnecessary to worry about the possible players' moves for one coalition to another because the coalitions are intercoalition-ally stable almost for all the parameters.
if it is Stackelberg, then one should prohibit individual moves from one coalition to another (it can be done by the government laws or punishment schemes, again). Then the coalition structure will be stable for most of the parameters in the sense of internal and coalition stability. The manager should not worry about the external stability because the more players decide to enter coalitions the larger population size will be.
5.5. Different discount factors
Traditionally, cooperative behavior analysis in bioresource management problems rests on the assumption of identical discount factors for all players. In the papers (Rettieva, 2014; Mazalov and Rettieva, 2014; Mazalov and Rettieva, 2015) we seek an optimal compromise in the case of heterogeneous goals pursued by players (different discount factors).
Consider a discrete-time game-theoretic bioresource management model with an identical planning horizon of both players and their different discount factors.
Suppose that two players (countries or fishing firms) harvest a fish stock on a finite planning horizon [0,n]. The fish population evolves according to the equation
where xt > 0 is the population size at time t > 0, e € (0,1) denotes the natural survival rate, a € (0,1) indicates the growth rate, and uit > 0 gives the catch of player i, i = 1, 2.
By assumption, the players possess the logarithmical payoff functions and different discount factors. In other words, the payoff functions of the players are defined
xt+i = (ext - uu - U2t)a , xo = x ,
(45)
by
n
J = E ôi ln(u¿t),
(46)
t=o
where Si € (0,1) denotes the discount factor of player i, i = 1, 2.
Theorem 6. The Nash equilibrium strategies in the problem (45), (46) have the form
t-i
t- l
ea2 E
N t
j=0
eai £
t . t
E ai E a2 -1
j=0 j=0
N l2t
j=0
t . t
E a1 E a2-1
j=0 j=0
where ai = aSi, i = 1, 2 , t = 1, ... ,n.
The individual payoffs of the players make up
VN (x, Si) = \ (ai)j ln x + > (Si)n-jAij - (Si)n ln k, i = 1, 2 , (47)
j=0
j=
A,j =ln
ea k= l
Ea^ a!k -1
k=0 k=0
j , v a
k=0 (£ak)k=i ,l,p =1, 2,1 = p,J = 1,
k=
.,n. (48)
Multi-step game and recursive Nash bargaining solution. Define cooperative behavior in this model by a recursive bargaining procedure (Rettieva, 2014). At each step, cooperative strategies are found via a bargaining solution, where noncooper-ative payoffs play the role of status quo points.
Theorem 7. The cooperative payoffs in the problem (45), (46) possess the form
Hcin(lli,..., Yin, yCi,..., YCn; x) = £ al ln(x) - Sn ln(k) +
j=0
n-l n-j _
+ £ sn—j [ln(Y Cn—j) + £ al ln(e - yCn-j - YCn-j)
j=0 i=
HCn(YCi,.. •, Yin, YCi,.. •, Yin; x) = £ a2 ln(x) - Sn ln(1 - k) +
'i- i
j=0
x) = a ln(x)
j=0 n-j
+ £ Sn j ln(YCn-j ) + £ a*2 ln(e - Y Cn-j - YCn-j )
The cooperative strategies can be evaluated recursively using the equations
n-
Y2n
j=0 n-
n-j
Y\Sn-3 MYCn-j) + £a2 ln(e-YCn-j-YCn-j) =i
-Sj A
n-J
--Y CnY, (Sn-j \ln(Yln-j ) + £ ai ln(e - YCn-j-YCn-j)
j=0
/ i n-j ) + ai -
i=
2A2n-j
- Sj A i n-j
subject to the constraint
Y2n
e - YL E ai
n
ai2
i=0
j
j
a
a
l
2
n
n
j
n
n
where Aj are defined by (48).
In (Rettieva, 2014; Mazalov and Rettieva, 2014) we have performed numerical simulation for a 20-step game. In Fig. 16-18 the black line corresponds to cooperative behavior and grey line to the Nash equilibrium.
Fig. 16. Population size
Fig. 17. Player 1's catch
Fig. 18. Player 2's catch
Fig. 16 demonstrates the dynamics of the population size, whereas Figs. 17 and 18 show the catch of each player. Note that cooperation appears beneficial to both players and, moreover, improves the ecological situation owing to sparing bioresource exploitation.
Fig. 19. The cooperative payoffs Fig. 20. Player 2's payoffs
Compare players' payoffs under different discount factors. Fig. 19 illustrates the payoffs V_nc(x, ¿1) and V2nc(x, ¿2) for ¿1 = 0.1,..., 0.9 and ¿2 = 0.1,..., 0.9. Clearly, a player with a higher discount factor gains more utility from cooperation. And the players obtain identical payoffs in the case of coinciding discount factors.
The cooperative behavior design approach suggested in this paper leads to a player's cooperative payoff which is above or equal to (under some parameters) its Nash equilibrium counterpart. The payoffs of player 2 under cooperative and egoistic behavior are presented in Fig. 20. Hence, the introduced approach stimulates cooperation, which is not always the case within other design methods of cooperative strategies and payoffs (Breton and Keoula, 2014).
5.6. Different fixed planning horizons
The harvesting process with the dynamics (45) and different planning horizons was considered in (Mazalov and Rettieva, 2014; Rettieva, 2015). Players 1 and 2 harvest the fish stock during ni and n2 steps, respectively. For the sake of definiteness, suppose that ni < n2. Therefore, in this model players enter cooperation on the time period [0, ni] and we have to find their cooperative strategies. After step ni till step n2 player 2 continues the harvesting process individually. Hence, the players' payoffs are defined by
ni ni U2
Ji = £ S[ ln(u 1 t), J2 = £ 4 ln(u2t)+ E ô2 ln(uat), (49)
t=0 t=0 t=n1 + 1
where u\ (i = 1, 2) denote the cooperative strategies and u2 indicates the strategy of player 2 during individual catch.
To construct the cooperative strategies and payoffs of the players, we apply the Nash bargaining solution for the whole duration of the game. Thus, it is required to solve the following optimization problem:
(V1c(x,ôi)[0, n 1 ] - VN(x, Si)[0,n 1 ]) • •(V2(x,S2)[0,ni] + V2ac(xcni ,S2)[ni,n2] -V<N(x, S2)[0, ni] - V2aN(xNni, S2)[ni, n2]) =
ni
Si ln(uit) - ViN(x,Si)[0,ni])^ St ln(u2t)^ St ln(uat) -
t=0 t=0 t=n1 + i
-V2n (x,S2)[0,ni] - V2aN (xNni ,S2)[ni,n2]) ^ max
>0
ni
u
u
where Vi^(x, Sj)[0,ni] represent the Nash equilibrium payoffs defined by (47) (with n = ni), V2ac(xcni, S2)[ni,n2\ gives the payoff of player 2 owing to its individual harvesting after ni steps of cooperative behavior, and V2aW(xNni ,S2)[ni,n2] is the payoff of player 2 owing to its individual harvesting after ni steps of noncooperative behavior.
Theorem 8. The cooperative payoffs in the problem (45), (49) make up
Hicni (Yn, • • •, Y^, Y2ci, • • •, Y2c„1; x)
ni ni ni J
= X>Í lnx+E¿ni-J ln(YÍj) + EC-J'E ai ln(e-7Íj-yJ) + ¿n
j=0 j=i j=i i= i _ni + i ni n „ C[ j
ni + i ni n a (i - aJ )
^Inx+E^MY^ + E*5" ln(g-7Í-72) + ^1 :
• n 1 r ^ J r ^ — n.-\
1-a i J=i i ^ iJ' J=i i 1 - a i
j=i J=i
H2cni (Y 2 i lni > Y2c i, • • •, Y2cni ; x) =
n2 ni ni n+j
E a2 ln x + E ¿ni-J ln(Y2j + E ¿"^ E a2 ln(£ - Y j - YÍj )
j=0 j=i j=i i=i
+ ^ ¿n2-J BJ + ¿TE a2 ln(1 - k)
j=i j=0
1 _ an2 + i ni ni a (1 - an+j )
InX + E Hl2j) + E W2( _ 2 j ln(e - Yi ~~ Y2) +
1 - a2 ' ^ 2 ^ 2 1 - a2
j=i J=i
n n+
+ y S?2-jBj + SI11 --ln(l - k),
2 2 1 - a2
1 — a.
JBJ+d21
j=1 where
j j j
Bj = E a2 ln (~J~) + E a2 ln(E aa) > j = • • •> n' n = "2 - ni.
j=o £ ap '=1 p= 1
p=0
The cooperative strategies of the players are related via
n+t . t
£Yh E a2 £ — Yu E ai
j = t-1 c j = 0
Yi t = — n + t n+t t n+t — > Y2t:
£ai-1 E «2+7^ E aJ2 E ai - («- + «1) E «2) E «2
j=0 j=t-1 j=0 j=0 j=0
The strategy of player 1 at the last step (the quantity 7Ï1) follows from one of the first-order optimality conditions.
We present the simulation results for the planning horizons n1 = 10 and n2 = 20. In Fig. 21-23 the black line corresponds to cooperative behavior and grey line to the Nash equilibrium.
The dynamics of the population size on the whole planning horizon [0,n2] can be observed in Fig. 21. Clearly, cooperation improves the ecological situation.
Figs. 22 and 23 show the catch of player 1 on the time period [0, n1] and the catch of player 2 on the time periods [0, n1] and [n1,n2], respectively. Interestingly, player 2 has a smaller catch in cooperation than in the Nash equilibrium, but this is compensated by its individual harvesting at subsequent steps.
And now, compare the players' payoffs for different planning horizons in the case when player 1 leaves the game earlier. Fig. 24 illustrates the payoffs V1Ï(n1,x) and
n
n
Fig. 21. Population size
Fig. 22. Player 1's payoff Fig. 23. Player 2's payoff
V2c(n2, x) for n2 = 2,..., 10 and n1 = 1,..., n2 — 1. Obviously, the closer is n1 to n2, the smaller is the difference between the payoffs.
Finally, we underline that the suggested cooperative behavior design guarantees that the cooperative payoff of a player is greater or equal to (under some parameters) its payoff in the Nash equilibrium. Fig. 25 shows the payoffs of player 2 under cooperative and noncooperative behavior for different planning horizons. This also manifests that the suggested approach stimulates cooperative behavior.
5.7. Random planning horizons
In (Mazalov and Rettieva, 2015) we explore the model (45), (49), where players possess heterogeneous discount factors and, moreover, heterogeneous planning horizons. By assumption, players stop cooperation at random steps: external stochastic processes can cause cooperative agreement breach.
Suppose that players 1 and 2 harvest the fish stock during ni and n2 steps, respectively. Here n1 represents a discrete random variable taking values {1,..., n} with the corresponding probabilities {01, ...,9n}. Similarly, n2 is a discrete random variable with the value set and the probabilities {u1, ...,un}. We believe that the planning horizons are independent. Therefore, during the time period [0, n1 ] or
[0,n2] the players enter cooperation, and the problem consists in evaluating their strategies.
The players' payoffs are determined via the expectation operator:
ni n2 ni
Hi = ln(uit )1{ni <n2> + (Ê ln(Wli)^ Sj ln«}) /{ni>n2}}:
1=1 1=1 t=n2 + 1
n n ni n 1 — 1 n2 ni
= E ME ^E S1 In(u1t)^Ê ^(E St ln(u1t)+ £ St ln«))]^)
ni = 1 n2=ni 1=1 n2 = 1 1=1 1=n2 + 1
n2 ni n2
H = ^^S1 ln(u2l)1{n2 <ni} +(E S1 ln(u2l)+ £ S1 In(uSt)) /{n2 >ni}} =
1=1 1=1 1=ni + 1
n n n2 n2 — 1 ni n2
= E --[E S2ln(u21)^E 0ni(ES2ln(u21)+ £ S2ln(u21))](51)
n2 = 1 ni=n2 1=1 ni = 1 1=1 1=ni + 1
where ua1 specifies the strategy of player « when its partner leaves the game, i = 1, 2.
To define cooperative behavior, we employ the Nash bargaining solution; the role of status quo points belongs to the noncooperative payoffs of players. Therefore, we begin with construction of Nash equilibrium strategies.
As step t occurs in the game, the Bellman functions (t, x), i = 1, 2 of players acquire the form
n £ n ni
uN ,...,uN ^ — v^ n L — v^ .—
iT' in ni=T n2=ni —; 1=T
I = T I=T
ni —1 n2
wn rfi /, T/ac
n2=T ^ W; t=T
I=T
n n ^ n2
V2Ar(r,x)= maxi^^M^ +
uN ^ v^ L v^ /1 .
1n n2=T W; ni=n2 "i t=T
I=T i=T
ni=T ^ W; t=T
i=T
where
ni ni —t n¿ —T
Via(T, ni)=¿St ln«)= £ aj ln x+ ^ Sf—T—jDj, i=1,2,
t=T j=0 j=1
^ = Eln (—J + Eln(E«?). i = i. 2-
i=o £ i=1 p=1
p=0
are the players' payoffs provided that player i, i = 1, 2 harvests the fish stock individually.
We get a relationship between (t, x) and (t + 1, x) of the form
(t,x) = ¿T ln«) + p;+1VN(t + 1,x) + ¿1 ln«t),
ni=T+1 1=T
n n2
VN (t, x) = ¿T ln« )+ Ptt + 1VN (t + 1,x) + C2T E ¿1 ln(uai):
n2=T+1 1=T
where
nn
E w E
+ 1 _ l=T+1 i=T+l
wT 1 „ 0T 1
nifl _ ' T' ' T' f~< _ ~ !__~ f~<
rT — n n ' °lr — —r, n , L/2T
n n 1T n n 2T n n
E w E 0 E^Π01 E E w
1 = T 1 = T 1 = T 1 = T 1=T 1=T
Following the standard approach in fish war models, we search for the payoff functions VjN(t, x) = AT ln x + BT and linear players' strategies = YiTx, i = 1, 2.
Theorem 9. The Nash equilibrium strategies in the problem (45), (50), (51) with random planning horizons take the form
n= __ n = _
XT AT I XT AT I AT AT UT +1 ' XT AT I XT AT I /IT /IT ]
(5\AT, + à\A\ + aAlAlPÏ+1 ' 72r ¿[AJ + 5T2A\ + '
noncooperative payoffs make up
V4n(t,x)= At ln x + BT , i = 1, 2 , (52)
where
n ni -T n n2 -T
¿T + C1T E 0ni E a1 ¿2 + C2T E wn^ «2
,,T ni=T+1 j=0 /1T n2=T+1 j=0
BT =
= _1 ' J-'J J^T = _
1 1 — aPr+1 ' 2 1 — aPr+1
n ni - T
¿Tln«) + aA1 PT + 1 ln(e-— Y2NHCW E 0ni E ¿21-T-jDj
ni=T+1 j=1
1 1 - PTT+1
n n2 - T
¿T ln(YNT) + aA2PtT+1 ln(e — -yN2) + C2t E ^ E ¿2n2-T-Dj
DT n2 =T +1 j=1
Do =
2 1 - PJ+1
n
To construct the cooperative strategies and payoffs of the players, we adopt the Nash bargaining solution for the whole duration of the game. Consequently, it is
(E UE wn^ st in(uit) +
required to solve the problem
(Vc(1,x) - VN(1,x))(VÎ(1,x) - VN(1,x))
ni
n i = 1 n2=ni 1=1
ni —1 n2 ni
+ E -n2(E s1 lnKj+ E S1 M^))] - Vn(1,x)) •
n2=1 1=1 1=n2+1
n n n2
^E S1 ln(u21) +
n2=1 "i="2 1=1
n2 —1 "I n2
+ E 0ni (E S1 ln(u21) + E S1 ln^))] - VN(1,x)) ^ ucnux,
n i = 1 1=1 1=ni +1 Uit'U2t >
where ViN (1,x) = AN ln x + , i = 1, 2 indicate the Nash equilibrium payoffs defined by (52).
Theorem 10. The cooperative payoffs in the problem (45), (50), (51) with random planning horizons have the form
Vc(n - k,x) = S™—k ln(uCn—fc ) +
+apn—Î+1 Gn—k+1 ln(ex - u1n—k - u2n—k) +
k —1
+ E P— [Sr—1 ln(YCn—i ) + «Pnn—ln(£ - YÎn—i - YÎn—i )] +
i=2
+^—1^—1 [«A ln(£ - YÏn—i - Yln—i) + Bi] +
k
+P„n^fc1Sin—1 ln(YCn—i) + EP—Cin—(ni), (53)
i=i
where
n
Vii(ni)= ^ £ St ln«t), Vî(n2)= E St ln(u2t),
ni =n —1 t=n — i n2=n —l + i t=n— i
kk
G1 V^ rn—i k — i on—i i k 4 on /o2 \ A rn—i k —i on —i , k a on
k = 2^ S1 a Pn—k + a A1Pn —k , Gk = 2^ S2 a Pn —k + a A2P n —k . i = 1 i = 1
The cooperative strategies are related by
Sn—kSn —k r Sn —k Yc g1 J^n—k r r<2
S1 S2 £ - S2 Tin — kGk c S1 £Yln —1G1
The strategy of player 1 at the last step (the quantity ycn— 1) is evaluated through one of the first-order optimality conditions.
Our simulation has employed the Monte Carlo method, n =10 and the following probabilities = 0.1, w = 0.005i + 0.0725, i = 1,..., n.
Figs. 26 and 27 demonstrate the results of numerical simulation with 50 trials under egoistic and cooperative behavior, respectively. Here points indicate the simulation results and circles correspond to the expected payoffs obtained in (52) and (53).
n
Fig. 27. Cooperative equilibrium 6. Conclusions
Cooperation plays an important role in bioresource management problems. It leads to a sparing mode of bioresource exploitation and improves the ecological situation. The paper overviews the results in the fields of cooperation maintenance and cooperative behavior determination. Namely, the author's new schemes to obtain and maintain the cooperative exploitation are presented. We extend the idea of incentive equilibrium to the case with territory sharing and control from the center. We present the incentive condition for rational behavior that is easier to verify than the existing ones. We extend the internal and external condition to the models with coalition structure and offered the coalition stability concept. It is proposed to apply the Nash bargaining approach to obtain cooperative profits and strategies in the case where players possess different discount factors. Moreover, the models where players harvesting times are different (fixed and random) were investigated and the possible cooperative behavior determination concepts were obtained. Analytical and numerical results for particular resource dynamic rules and the players' payoff functions are given. The author continues to work in this direction and the latest results were obtained in the field of multicriteria dynamic games (Rettieva, 2017).
References
Abakumov, A. I. (1994). Optimal harvesting in populations' models. Applied and indastrial
mathematics review, 1(6), 834-849 (in Russian). Abakumov, A.I. (1993). Control and optimization in the harvesting models. Vladivostok:
Dalnauka (in Russian). Baturin, V. A., Skitnevskii, D.M. and Cherkashin, A.K. (1984). Planning and forecast for
ecological-economical systems. Novosibirsk: Nauka (in Russian). Barrett, S. (1994). Self-enforcing International Environmental Agreements. Oxford Economic Papers, 46, 878-894. Bazikin, A. D. (1985). Mathematical biophisics of interacting populations. Moscow: Nauka. Berdnikov, S.V., Vasilchenko, V. V. and Selutin, V. V. (1999). Mathematical modeling of exogenous perturbation in trophic networks. Applied and indastrial mathematics review, 6(2), 145-158 (in Russian). Bloch, F. (1995). Sequential formation of coalitions with fixed payoff division and externalities. Games Econ. Behav., 14, 90-123. Bloch, F. (1996). Noncooperative models of coalition formation in games with spillovers. In: Carraro C., Siniscalco D. eds. New Direction in Economic Theory of Environment. Cambridge: Cambridge University Press. Breton, M. and Keoula, M. Y. (2014) A great fish war model with asymmetric players. Ecological Economics, 97, 209-223.
-6 -4
Fig. 26. Nash equilibrium
Carraro, C. (1997) The structure of International Environmental Agreements. Paper presented at the FEEM/IPCC/Stanford EMF Conference on "International Environmental Agreements on Climate Change". Venice, pp. 309-328.
Carraro, C. The economics of coalition formation. In: Gupta J., Grubb M. eds. Climate Change and European Leadership. Kluwer Academic Publishers, 2000. pp. 135-156.
Carraro, C. and Siniscalco, D. (1992) The international protection of the environment. J. Publ. Econ., 52, 309-328.
Chaudhuri, K. (1986) A bioeconomic model of harvesting a multispecies fishery. Ecological Modelling, 32, 267-279.
Clark, C. W. (1985). Bioeconomic modelling and fisheries management. NY: Wiley.
D'Aspremont, C., Jacquemin, A., Gabszewicz, J.J. and Weymark, J.A. (1983). On the stability of collusive price leadership. Can. J. Econ., 16(1), 17-25.
De Zeeuw, A. (2008). Dynamic effects on stability of International Environmental Agreements. J. of Environmental Economics and Management, 55, 163-174.
Ehtamo, H. and Hamalainen, R. P. (1993). A cooperative incentive equilibrium for a resource management problem. J. of Economic Dynamics and Control, 17, 659-678.
Eyckmans, J. and Finus, M. (2003). Coalition formation in a global warming game: how the design of protocols affects the success of environmental treaty-making. Working paper 56, CLIMNEG 2.
Finus, M. (2008). Game theoretic research on the design of International Environmental Agreements: insights, critical remarks and future challenges. Int. Rev. of Environmental and Resource Economics, 2, 29-67.
Finus, M. and Rundshagen, B. (2003). Endogenous coalition formation in global pollution control: a partition function approach. In: Carraro C. ed. Endogenous formation of economic coalitions. Cheltenham: Edward Elgar, pp. 199-243.
Fisher, R. D. and Mirman, L.J. (1992). Strategic dynamic interactions: fish wars. J. of Economic Dynamics and Control, 16, 267-287.
Fisher, R. D. and Mirman, L. J. (1996). The complete fish wars: biological and dynamic interactions. J. of Env. Econ. and Manag., 30, 34-42.
Gimelfard, A. A., Ginzburg, L. R., Poluektov, R. A., Puh, Yu. A. and Ratner, V.A. (1974). Dynamic theory of biological populations. Moscow: Nauka (in Russian).
Goh, B. S. (1980). Management and analysis of biological populations. Agricultural and Managed-Forest Ecology. Amsterdam: Elsevier.
Gromova, E. V. and Petrosyan, L. A. (2015) On a approach to the construction of characteristic function for cooperative differential games. Math. Game Theory and Appl., 7(4), 19-39 (in Russian).
Gurman, V.I. and Druzhinina, I. P. (1978). Nature system models. Novosibirsk: Nauka (in Russian).
Hamalainen, R.P., Kaitala, V. and Haurie, A. (1984). Bargaining on whales: A differential game model with Pareto optimal equilibria. Oper. Res. Letters, 3(1), 5-11.
Haurie, A. (1976). A note on nonzero-sum differential games with bargaining solutions. J. of Opt. Theory and Appl. 18, 31-39.
Haurie, A. and Tolwinski, B. (1984). Acceptable equilibria in dynamic games. Large Scale Systems, 6, 73-89.
Il'ichev, V. G., Rohlin, D. B. and Ougolnitskii, G. A. (2000). About economical mechanisms of bioresource management control. Russian Academy of Sciences belluten. Theory and system control, 4, 104-110 (in Russian).
Ioannidis, A., Papandreou, A. and Sartzetakis E. (2000). International Environmental Agreements: a literature review. Working Papers, GREEN.
Kaitala, V.T. and Lindroos, M. (2007). Game-theoretic applications to fisheries. In: Handbook of operations research in natural resources, pp. 201-215. Springer.
Kulmala, S., Levontin, P., Lindroos, M. and Pintassilgo, P. (2009). Atlantic salmon fishery in the Baltic Sea - A case of trivial cooperation. In "Essays on the Bioeconomics of the Northern Baltic Fisheries". Soile Kulmala. PhD thesis, University of Helsinki.
Levhari, D. and Mirman, L.J. (1980). The great fish war: an example using a dynamic Cournot-Nash solution. The Bell J. of Economics, 11(1), 322-334.
Lindroos, M., Kaitala V. T. and Kronbak L. G. (2007). Coalition games in fishery economics. In: Advances in Fishery Economics, pp. 184-195. Blackwell Publishing.
Lindroos, M. (2008). Coalitions in International Fisheries Management. Natural Resource Modeling, 21, 366-384.
Marin-Solano, J. and Shevkoplyas E. V. (2011). Non-constant discounting and differential games with random time horizon. Automatica, 47, 2626-2638.
Mazalov, V.V. and Rettieva, A.N. (2015). Asymmetry in a cooperative bioresource management problem. In: Game-Theoretic Models in Mathematical Ecology. Nova Science Publishers, pp. 113-152.
Mazalov, V.V. and Rettieva, A.N. (2014). Game-theoretic models for cooperative biore-source management problem. In: Models and Methods in the Problem of Interaction between Atmosphere and Hydrosphere: Textbook. Tomsk, pp. 449-489.
Mazalov, V.V. and Rettieva, A.N. (2012). Cooperation maintenance in fishery problems. In: Fishery Management. Nova Science Publishers, pp. 151-198.
Mazalov, V.V. and Rettieva, A.N. (2011). The discrete-time bioresource sharing model. J. Appl. Math. Mech., 75 (2), 180-188.
Mazalov, V. V. and Rettieva, A. N. (2010). Fish wars and cooperation maintenance. Ecological Modelling, 221, 1545-1553.
Mazalov, V.V. and Rettieva, A.N. (2010). Incentive conditions for rational behavior in discrete-time bioresource management problem. Doklady Mathematics 81(3), 399-402.
Mazalov, V.V. and Rettieva, A.N. (2010). Incentive equilibrium in bioresource sharing problem. Journal of Computer and Systems Sciences International, 49(4), 598-606.
Mazalov, V. V. and Rettieva, A. N. (2010). Fish wars with many players. Int. Game Theory Rev., 12(4), 385-405.
Mazalov, V.V. and Rettieva, A.N. (2009). The compleat fish wars with changing area for fishery. IFAC Proceedings Volumes (IFAC - PapersOnLine), 7(1), 168-172.
Mazalov, V.V. and Rettieva, A.N. (2008). Incentive equilibrium in discrete-time bioresource sharing model. Doklady Mathematics, 78(3), 953-955.
Mazalov, V. V. and Rettieva, A. N. (2008). Bioresource management problem with changing area for fishery. Game Theory and Applications, 13, 101-110.
Mazalov, V.V. and Rettieva, A.N. (2008). Incentive equilibrium in bioresource management problem. In: Evolutionary and deterministic methods for design, optimization and control, P. Neittaanmaki, J. Periaux and T. Tuovinen (Eds.). CIMNE. Barcelona. Spain, 451-456.
Mazalov, V.V. and Rettieva, A.N. (2007). Cooperative incentive equilibrium for a bioresource management problem. Contributions to Game Theory and Management, 1, 316325.
Mo, J. and Walrand, J. (2000). Fair End-to-End Window-Based Congestion Control. IEE/ACM Transactions on Networking, 8(5), 556-567.
Munro, G. R. (1979). The optimal management of transboundary renewable resources. Can. J. of Econ., 12(8), 355-376.
Munro, G. R. (2000). On the Economics of Shared Fishery Resources. In: Int. Relations and the Common Fisheries Policy. Portsmouth, pp. 149-167.
Neumann, J. and Morgenstern, O. (1953). Theory of Games and Economic Behavior. Princeton: Princeton University Press.
Osborn, D.K. (1976). Cartel problems. American Economic Review, 66, 835-844.
Osmani, D. and Tol, R.S.J. (2010). The Case of Two Self-enforcing International Agreements for Environmental Protection with Asymmetric Countries. Computational Economics, 36(2), 93-119.
Ostrom, E. (1990). Governing the commons: the evolution of institutions for collective action. Cambridge University Press.
Owen, G. (1968). Game theory. NY: Academic Press.
Petrosjan, L. A. (1993). Differential game of pursuit. World Scientific, Singapore.
Petrosjan, L. A. and Zaccour, G. (2003). Time-consistent Shapley value allocation of pollution cost reduction. J. of Econ. Dyn and Contr., 27(3), 381-398.
Petrosjan, L. A. and Zenkevich, N. A. (1996). Game theory. World Scientific, Singapore.
Petrosyan, L. A. (1997). Stability of differential games with many players' solutions Vestnik of Leningrad university, 19, 46-52 (in Russian).
Petrosyan, L. A. and Danilov, N.N. (1979). Stability of nonantogonostic differential games with transferable payoffs ' solutions. Vestnik of Leningrad university, 1, 52-59 (in Russian).
Petrosyan, L. A. and Danilov, N. N. (1985). Coopeartive differential games and applications. Tomsk (in Russian).
Petrosyan, L. A. and Zakharov, V. V. (1981). Game-theoretic approach to nature protect problem. Vestnik of Leningrad university, 1, 26-32 (in Russian).
Petrosyan, L. A. and Zakharov, V. V. (1997). Mathematical models in ecology. SPb: SPb State University (in Russian).
Petrosyan, L. A. and Zakharov, V. V. (1986). Introduction to mathematical ecology. Leningrad: Lenigrad State University (in Russian).
Pintassilgo, P. and Lindroos, M. (2008). Coalition formation in straddling stock fisheries: a partition function approach. Int. Game Theory Rev., 10, 303-317.
Plourde, C. G. and Yeung, D. (1989). Harvesting of a transboundary replenishable fish stock: a noncooperative game solution. Marine Resource Economics, 6, 57-70.
Puh, Yu.A. (1983). Equilibrium and stability in populaitons dynamic models. Moscow: Nauka.
Ray, D. and Vohra, R. (1997). Equilibrium binding agreements. Jour. of Econ. Theory, 73, 30-78.
Rettieva, A. N. (2017). Equilibria in dynamic multicriteria games. Int. Game Theory Rev., 19(1), 1750002.
Rettieva, A.N. (2014). A bioresource management problem with different planning horizons. Automation and Remote Control, 76(5), 919-934.
Rettieva, A.N. (2014). A discrete-time bioresource management problem with asymmetric players. Automation and Remote Control, 75(9), 1665-1676.
Rettieva, A.N. (2013). Bioresource management problem with asymmetric players. Computational information technologies for environmental sciences: selected and reviewed papers presented at the International conference CITES-2013, 127-131.
Rettieva, A.N. (2012). Stable coalition structure in bioresource management problem. Ecological Modelling 235-236, 102-118.
Rettieva, A. N. (2012). A discrete-time bioresource management problem with asymmetric players. Math. Game Theory and Appl., 4(4), 63-72 (in Russian).
Rettieva, A.N. (2011). Fish wars with changing area for fishery. Advances in Dynamic Games. Theory, Applications, and Numerical Methods for Differential and Stochastic Games, 11, 553-563.
Rettieva, A.N. (2011). Coalition structure stability in discrete time bioresource management problem. Math. Game Theory and Appl., 3(3), 39-66 (in Russian).
Rettieva, A. N. (2010). Cooperative bioresource management regulation. Appl. and Indust. Math. Rev., 17(5), 663-672 (in Russian).
Rettieva, A.N. (2009). Cooperative incentive condition in bioresource sharing problem. Large-scale Systems Control, 26.1, 366-384 (in Russian).
Silvert, W and Smith, W. R. (1977). Optimal exploitation of multispecies community. Math. Biosci., 33, 121-134.
Shapiro, A. P. (1979). Modeling of biological systems. Vladivostok (in Russian).
Shevkoplyas, E. V. (2011). The Shapley value in cooperative differential games with random duration. Annals of the Int. Soc. of Dynamic Games, 11, 359-373.
Smith, M.J. (1968). Mathematical Ideas in Biology. Cambridge University Press.
Sorger, G. (2006). Recursive Nash bargaining over a productive assert. J. of Economic Dynamics & Control, 30, 2637-2659.
Svirezhev, Yu. M. and Elizarov, E. Ya. (1972). Mathematical modeling of biological systems. Moscow: Nauka (in Russian).
Tolwinski, B., Haurie, A. and Leitmann, G. (1986). Cooperative equilibria in differential games. J. of Math. Anal. and Appl., 119, 182-202.
Vislie, J. (1987). On the optimal management of transboundary renewable resources: a comment on Munro's paper. Can. J. of Econ., 20, 870-875.
Yeung, D.W.K. (2006). An irrational-behavior-proof condition in cooperative differential games. Int. Game Theory Review, 8(4), 739-744.
Yi, S. S. (1997). Stable coalition structures with externalities. Games and Economic Behavior, 20, 201-237.
Yi, S.S. and Shin, H. (1995). Endogenous formation of coalitions in oligopoly. Mimeo; Department of Economics, Dartmouth College.
