CC BY
4Necessary and Sufficient Conditions for Pareto Optimality in Infinite Horizon Cooperative Differential Games
Puduru Viswanadha Reddy and Jacob Engwerda
Department of Econometrics and Operations Research Tilburg University, P.O.Box: 90153, 5000LE Tilburg, The Netherlands {P.V.Reddy,engwerda}@uvt.nl
Abstract In this article we derive necessary and sufficient conditions for the existence of Pareto optimal solutions for an N player cooperative infinite horizon differential game. Firstly, we write the problem of finding Pareto solutions as solving N constrained optimal control subproblems. We derive some weak sufficient conditions which entail one to find all Pareto solutions by solving a weighted sum optimal control problem. Further, we observe that these sufficient conditions are related to transversality conditions of the associated subproblems. We consider games defined by nonautonomous and discounted autonomous systems.
1. Introduction
Cooperative differential games involve situations where several players decide to cooperate while trying to realize their individual objectives, with players acting in a dynamic environment. We consider cooperative games with no side payments and open loop information structure. The dynamic environment is modelled as:
where ui(t) is the control strategy/action of player i, (ui,u2, ■ ■ ■ ,uN) G U, with U being the set of admissible controls. Each player tries to minimize the objective function
for i = 1, 2, ■■■ ,N. For the above problem to be well-defined we assume that f : R" xU ^ R" and gi : R" xU ^ R, i = 1, 2, ■ ■■ N, are continuous and all the partial derivatives of f and gi w.r.t x and ui are continuous. Further, we assume that the integrals involved in the player’s objectives converge.
Pareto optimality plays a central role in analyzing these problems. Due to cooperation, the cost incurred by a single player cannot be minimized without increasing the cost incurred by other players. So, we consider (Pareto optimal) solutions which cannot be improved upon by all the players simultaneously. We call the controls (u\,u2, ■■■ ,u*N) G U Pareto optimal if the set of inequalities Ji(ui, u2, ■■■ , uN) < Ji(u’[, u2, ■■■ , u*N), i = 1, 2, ■■■ ,N, (with at least one of the inequalities being strict) does not allow for any solution in (ui, u2, ■ ■ ■ , uN) G U.
A well known way to find Pareto optimal controls is to solve a parameterized optimal control problem (Zadeh, 1963; Leitmann, 1974). However, it is unclear whether all Pareto optimal solutions are obtained using this procedure. The closest references we could track are (Chang, 1966),
X(t) = f (t, x(t), ui(t), u2(t), ■ ■ ■ , un(t)), x(to) = xo G R"
(1)
(Blaquiere et al., 1972) and (Vincent et al., 1970). (Vincent et al., 1970) and the affiliated papers (Stalford, 1972) and (Leitmann et al., 1972) discuss necessary conditions for Pareto solutions for dynamic systems where cost functions are just functions of the terminal state. In (Vincent et al., 1970) geometric properties of Pareto surfaces were used to derive necessary conditions which are also in the spirit of maximum principle. Some difference with our work are: they assume that the admissible controls are feedback type and the terminal state should belong to some n — 1 dimensional surface. Recently, (Engwerda, 2009) gives necessary and sufficient conditions for Pareto optimality for finite horizon cooperative differential games. In this work the necessary conditions are in the spirit of the maximum principle.
We follow the same approach as mentioned in (Engwerda, 2009). In section (2) we present a necessary and sufficient characterization of Pareto optimality. In section (3) the problem of finding all Pareto solutions of an N player cooperative differential game is transformed as solving N constrained optimal control subproblems. Further, we show that by making an assumption on the transversality conditions (Lagrange multipliers), associated with the subproblems, one can find all the Pareto solutions by solving the weighted sum optimal control problem. The transversality conditions for the optimal control problems in the finite horizon, in general, do not extend naturally to the infinite horizon case. This behavior was first analyzed by (Halkin, 1974) using a counterexample. For optimal control problems, (Seierstad et al., 1986; Aubin and Clarke, 1979; Mitchel, 1982; Aseev et al., 2004) give conditions under which the finite horizon transversality conditions extend naturally to the infinite horizon case. We give some weak sufficient conditions under which these conditions extend naturally within this setting too. These weak sufficient conditions are related to growth conditions which naturally extend the finite horizon transversality conditions to the infinite horizons. We consider the games defined by nonautonomous and discounted autonomous systems. For discounted autonomous systems, (Mitchel, 1982) derives necessary conditions for free endpoint optimal control problems. We extend these results for the constrained subproblems and derive weak sufficient conditions for the above assumption to hold true. In section (4) we derive sufficient conditions for Pareto optimality in the similar lines of Arrow’s sufficiency results in optimal control.
In section (5) we consider regular indefinite infinite planning horizon linear quadratic differential games. That is the linear quadratic case, where the cost involved for the state variable has an arbitrary sign and the use of every control is quadratically penalized. This problem was recently solved for both a finite and infinite planning horizon in (Engwerda, 2008) assuming that the problem is a convex function of the control variables and the initial state is arbitrary. In this article we concentrate on the general case and where the initial state is fixed. We provide an algorithm to compute all Pareto optimal solutions when the system is scalar. In section (6) we give conclusions.
This paper addresses just the issue of finding Pareto solutions that can be realized by a grand coalition. We do not discuss about cooperative situations that can arise while sharing the joint Pareto payoff. For a discussion of some frequently used cooperative agreements, related to bargaining, one may consult e.g., chapter 6 of (Engwerda, 2005).
Notation: We use the following notation. Let SN = {1,2, ■ ■ ■ ,N} denote
the grand coalition and let SlN = SN/ {i} denote the coalition of all players excluding player i. Let PN denote the N dimensional unit simplex. RN denotes a cone consisting of N dimensional vectors with nonnegative entries. 1N denotes a vector in Rn with all its entries equal to 1. y' represents the transpose of the vector y G Rn. \x\ represents the absolute value of x G R. ||y|| represents the Euclidian norm of the vector y G RN. [y^ represents the absolute value of the ith entry of the vector y. |A|(mn) represents the absolute value of entry (m, n) of the matrix A. A > 0 denotes matrix A is strictly positive definite. fx(.) represents the partial derivative of the function f (.) w.r.t x. j G RN denotes the vector whose entries Wi are the weights assigned to the cost function of each player. We define the weighted sum function G(.) as follows:
G(u},t,x(t),u(t))=^2 Jigi(t,x(t),u(t)).
iESN
2. Pareto Optimality
In this section we state conditions to characterize Pareto optimal controls. We borrow the following lemmas from (Engwerda, 2009).
Lemma 1. Let ai G (0, 1), with ^2N=0 ai = 1. Assume u* GU is such that
u* G argmin Ji (u)[>. (3)
Then u* is Pareto optimal.
The above lemma which involves minimizing the weighted sum is an easy way to find Pareto optimal controls. It is, however, unclear whether we obtain all Pareto optimal controls in this way. The above approach may not yield any Pareto solutions even though there exist infinite number of Pareto solutions. The following example (an infinite horizon counterpart of (see example (2.2), Engwerda, 2009)) illustrates this point.
Example 1. Consider the following game problem:
(P) min Ji(xo,ui,u2)= / e-pt/2(ui(t) — u2(t))dt and
J 0
min J2(x0,u1,u2)= e-3pt/2x2(t)(u2(t) — u1(t))dt
Jo
sub. to x(t) = ^x(t) + u\(t) — u2(t), x(0) = 0, t G [0, oo)
ui(.) gU (a bounded set), i =1, 2. (4)
By construction |x(t)| < cept/2 for some constant c > 0. Taking the transformations x(t) = e-pt/2x(t) and ■ui(t) = e-pt/2ui(t), we transform the game (P) as a new
game (P) given by:
(P) min Ji(xo,ui,U2)= / (ui(t) — u2(t))dt and
Jo
min J2(xo,Ui,U2) = x2(t)(u2(t) — ui(t))dt
o
sub. to X(t) = U1(t) — u2(t), X(0) = 0, t € [0, to) (5)
Uj(t) = e-pt/2ui(t), Ui(t) €U, i = 1, 2.
By construction |x(t)| < dept/2 for some constant 0 < d < to, so limt^TO |X(t)| exists. Furthermore, we have Ji(x0,u1,u2) = Ji(x0,U1,U2), i = 1, 2. The players’ objectives can be simplified as:
Ji(xo, ui, i/2)
J2(xo, ui, i/2)
We notice that J2 = — for all (ui,u2) and choosing different values for the control functions Ui( ), every point in the (J\, J2) plane satisfying J2 = —\Ji can be attained. Moreover, every point on this curve is Pareto optimal. Now consider the minimization of Ja = (a) Ji + (1 — a) J2 subject to (4) with a € (0,1). If we choose «1=0 and u2 = c, straightforward calculations result in Ja = — + 8(-13p°-)c . By
choosing c arbitrarily negative Ja can be made arbitrarily small, i.e., J(a) does not have a minimum.
Lemma (2) mentioned below gives a both necessary and sufficient characterization of Pareto solutions. It states that every player’s Pareto optimal solutions can be obtained as the solution of a constrained optimization problem. The proof presented below is along the lines of that for the finite dimensional case as considered, e.g., in chapter 22 of (Simon and Blume, 1994).
Lemma 2. u* €U is Pareto optimal if and only if for all i, u*(.) minimizes Ji(u) on the constrained set
Ui = {uJj (u) < Jj (u*), j = 1, ••• ,N, j = i} , for i = 1, ••• ,N. (6)
Proof. ^ Suppose u* is Pareto optimal. Then u* € Uk, Vfc, so Uk = 0. Now, if u* does not minimize Jk(u) on the constrained set Uk for some k, then there exists a u such that Jj (u) < Jj (u*) for all j = k and Jk (u) < Jk (u*). This contradicts the Pareto optimality of u.
^ Suppose u* minimizes each Jk (u) on Uk. If u does not provide a Pareto optimum, then there exists a u(.) € U and an index k such that Ji(u) = Ji(u*) for all i and Jk (u) < Jk (u*). This contradicts the minimality of u* for Jk (u) on Uk.
From lemma (2) we give a result, though intuitively simple, which will be helpful to find all Pareto solutions of the co operative game. If the players costs are modified as Ji(u) = Ji(u) — c, c € R, Vi € SN, then we have the following corollary.
/ (ui(t) — u2(t)) dt = lim x(t)
Jo t
f00 1 / \ 3
x2(t) (u2(t) — u\(t)) dt = — ( lim x(t)) . Jo 3 \t^o )
Corollary 1. The set of Pareto optimal strategies for the games with players objectives as Ji(u) and Jj,(u), i G Sn, is same.
Proof. The statement of the corollary follows easily by replacing Ji(u) with Ji(u) in lemma (2).
We observe that for a fixed player the constrained set Ui defined in (6) depends on the entries of the Pareto optimal solution that represents the loss of the other players. Therefore this result mainly serves theoretical purposes, as we will see in the proof of theorem (2) and theorem (3). Using the above lemma, we next argue that Pareto optimal controls satisfy the dynamic programming principle.
Corollary 2. If u*(t) GU, t G [0, to) is a Pareto optimal control for x(0) = xo in
(1,2), then for any t > 0, u* ([t, to)) is a Pareto optimal control for x(t) = x* (t) in (1,2), where x* (t) = x (t, 0,u*([0,t])) is the value of the state at t generated by u*([0,t]).
Proof. Let Ui(T), with x(t) = x*(t), be the constrained set defined as:
Ui(T) := {u\Jj (x(t ),u) < Jj (x(t ),u* Q- to)) , j = 1 ••• ^, j = i}
Consider a control u G U(t) and let ue ([0, to)) be a control defined on [0, to) such
that ue ([0,t)) = u* ([0,t)) and ue ([t, to)) = u, then x(t, 0,ue([0,-)) = x*(t).
Further,
po
Jj (0,xo,ue)= gj(t, x(t),ue(t))dt
Jo
/T p oo
gj(t,x*(t),u*(t)) dt +J gj(t,x(t),u(t)) dt (as u G U(t), we have)
/T p OO
gj(t,x*(t),u*(t)) dt +J gj(t,x* (t),u*(t)) dt
Jo
= gj(t,x* (t),u* (t)) dt = Jj (0,x0,u*).
o
The above inequality holds for all j = 1, ••• , N, j = i. Clearly, ue ([0, to)) g U(0) i.e., every element u G U(t) can be viewed as an element ue G Ui(0) restricted to the time interval [t, to). From the dynamic programming principle it follows directly that u* ([t, to)) has to minimize Ji(T, x* (t), u) on Ui(T).
Another result that follows directly from lemma (2) is that if the argument at which some player’s cost is minimized is unique, then this control is Pareto optimal too (for the proof see corollary (2.5) in (Engwerda, 2009)).
Corollary 3. Assume J\(u) has a minimum which is uniquely attained at u*. Then (Ji(u*), J2(u*), • •• , Jn(u*)) is a Pareto solution.
3. Necessary Conditions for the General Case
Let (P) be an N person infinite horizon cooperative differential game defined as follows:
Jo
(P) for each i G Sn min / gi(t,x(t),u(t))dt
ueu Jo
sub. to x(t) = f (t, x(t), u(t)), x(0) = x0.
Let u*(t) be a Pareto optimal strategy for the problem (P) and x* (t) be the trajectory generated by u*(t). Using lemma (2), (P) is equivalent to solving N constrained optimal control subproblems. Let (Pi) denote a constrained (w.r.t control space) optimal control subproblem for player i G Sn and defined as:
o
(Pi) min / gi(t,x(t),u(t))dt
u(t)GUi.J o
sub. to x(t) = f (t, x(t), u(t)), x(0) = x0.
Introducing the auxiliary states xj (t) as
xj(t) = f gi(t,x(t),u(t))dt, xj(0) = 0, j o j the constrained set Ui can be represented by inequalities. Then (Pi) is equivalently reformulated as:
o
(Pi) min I gi(t,x(t),u(t))dt
u(t)eU.J o
sub. to x(t) = f (t, x(t), u(t)), x(0) = xo xj(t) = gj(t,x(t),u(t)), xj(0) = 0, lim xj(t) < xj , Vj G SlN
j j t—►O j j
where xj = JO gj(t,x*(t),u*(t))dt. We make the following assumption.
Assumption 1 For each subproblem (Pi), the Lagrange multipliers associated with the objective function and the terminal state (xj) constraints are non negative with at least one of them strictly positive.
Now we derive necessary conditions for (J1 (u*), J2(u*), ••• , JN(u*)) to be a Pareto optimal solution.
Theorem 1. If (Ji(u*),J2(u*), ••• ,Jn (u*)) is a Pareto optimal solution for problem (P) and assumption (1) holds, then there exists an (a G Pn, a costate function
A(t) : [0, to) ^ R" such that with H(~a,t,x(t),u(t),A(t)) = A'(t)f (t,x(t),u(t)) +
G(~a,t,x(t),u(t)), the following conditions are satisfied.
H (~ca,t, x* (t),u* (t),A(t)) <H (a,t,x* (t),u(t),A(t)), Vu(t) gU (7a) H°(~a,t,x* (t),A(t))= min H (~a,t,x* (t),u(t),A(t))
u(t)eU
A(t) = - H°(~a,t,x*(t),A(t)) (7b)
x* (t) = H°(~a,t,x* (t),A(t)) s.t x* (0) = xo (7c)
(~a, A(t)) =0, Vt G [0, to), ~a G PN. (7d)
Proof. By assumption the objective functions Ji(u), i G Sn converge1. We define the Hamiltonian associated with (Pi) as:
Hi(t, x(t), u(t), A(t)) = Ai(t)f (t, x(t), u(t)) + Aogi(t, x(t), u(t))
+ tfj(t)gj(t,x(t),u(t)). (8)
j^SN
1 If the integrals do not converge there exist many notions of optimality
(Seierstad et al., 1986; Grass et al., 2008) and the analysis becomes complicated.
From Pontryagin’s maximum principle, if the pair (x* (t),u*(t)) is optimal for the problem (Pi) then there exist a constant Aio and (continuous, piecewise continuously differentiable) costate functions Ai(t) G R" and jj(t) G R, j G SnN such that:
(A°,Ai(t),vj(t)) = (0,° 0), j G SN, t G [0, to) (9a)
u(t) = argminHi(t, x(t),u(t),A(t)) (9b)
u<EU
Ho(t, x(t),A(t)) = min Hi(t,x(t),u(t),A(t))
u(t)eU
Ai(t) = -H°x(t,x*(t),A(t)) (9c)
jj(t) = -Hiii(t,x*(t),A(t)). (9d)
J 3
Since (Ho)s,i = 0, multipliers associated with the auxiliary variables jj(t) are constants jjj. We define the set of multipliers as a vector A i = (j\, ••• , /jii_1,A<°, ji+1, • • • , JN)' and write the first order conditions as:
Ai (t)f (t,x* (t),u*(t)) + G( A i,t,x* (t),u*(t)) < Ai(t)f (t,x* (t),u(t))
+G( A i,t,x* (t),u(t)) (10)
Ai(t) = -f'x (t, x* (t), u*(t))Ai (t) - Gx(t i, t, x* (t), u*(t)). (11)
Summing (10) and (11) for all i G Sn yields
]T (Ai(t)f (t, x*(t), u* (t)) + G(\i, t, x*(t), u* (t)))
i^Sw
< ^(A'i(t)f (t,x* (t),u(t))+ G(A i,t,x*(t),u(t))) (12)
i^Sw
]T Ai(t) = -fX (t,x*(t),u* (t)) £ Ai (t) - £ Gx(~A i ,t,x*(t),u* (t)). (13)
i£Sw i£Sw i£Sw
Let’s introduce d := ieSN (a° + Y1 j^Si jjj. By assumption (1) we have d> 0.
We define A(t) := i J2iesN Xi (t), o4 := \ + J2jes% Mi) e SN and a vector
"a := (a1, • • • , aN)'. Notice that "a G PN by assumption (1). Dividing the equation (12) with d we have:
A'(t)f (t, x*(t), u*(t)) + G(a,t, x*(t), u*(t))) < A'(t)f (t, x* (t), u(t))
+G(~a,t,x*(t),u(t)) (14)
A(t) = -fX(t,x*(t),u*(t))A(t) - Gx(~a,t,x*(t),u*(t)). (15)
Next we define the modified Hamiltonian as
H(cC, t, x(t), u(t), A(t)) = A'(t)f (t, x(t), u(t)) + G(~a, t, x(t), u(t)).
The above necessary conditions for u* (t) to be Pareto optimal control can be rewritten as (7).
Remark 1. Let K be a cone defined as K = {a G Rn s.t a' 1N > 0}. Clearly, if A i gK, Vi G Sn, then all Pareto solutions of the problem (P) can be obtained by solving the necessary conditions for optimality associated with the corresponding weighted sum infinite horizon optimal control problem.
Remark 2. Generally transversality conditions are specified in addition to equations (7) to single out optimal trajectories from the set of extremal trajectories satisfying (7). If the game problem (P) is finite horizon type, then the transversality conditions associated with the subproblem (Pi) are Ai(T) = 0,0 < T < to and jj > 0 for j G SN, i G Sn and the maximum principle holds in normal form i.e., Ao = 1,i G Sn (refer to proposition 3.16, Grass et al., 2008). The assumption (1) holds true naturally for the finite horizon case. So, for finite horizon games Ai G K, i G Sn, all Pareto solutions can be obtained by solving the necessary conditions for optimality associated with the weighted sum finite horizon optimal control problem (as shown in Engwerda, 2009). However, the transversality conditions generally do not carry over to the subproblems (Pi) in infinite horizons. Refer to (Halkin, 1974; Grass et al., 2008; Seierstad et al., 1986; Aseev et al., 2004) for counter examples to illustrate this behavior. A natural extension of transversal-ity conditions , in the infinite horizon, can be made by imposing certain restrictions on the functions f (.), gi(.), i G Sn, (see Mitchel, 1982; Aseev et al., 2004; Seierstad et al., 1986; Halkin, 1974).
It is clear, from the above remarks, that an extension of finite horizon transversality conditions to the infinite horizon case is sufficient to obtain all Pareto solutions from the necessary conditions associated with the corresponding weighted sum optimal control problem. So, for the analysis that follows from now onwards we focus on the growth conditions which allow such an extension. Towards that end, we first consider non-autonomous systems. From (theorem (3.16), Seierstad et al., 1986) or (example (10.3), Seierstad, 1999), we have the following corollary:
Corollary 4. Suppose -to < JO° gi(t,x(t),u(t))dt < to, i G Sn and there exist non-negative numbers a, b and c with c > Nb such that the following conditions are satisfied for t > 0 and all x(t):
(\gix(t, x(t), u(t)|)m < ae-ct, m = 1, • • • ,N, Vi G Sn (16a)
(\fx(t,x(t),u(t)\){l m) < b, l = 1,- • • ,N, m =1,- • • ,N. (16b)
Then assumption (1) is satisfied. Consequently, for every Pareto solution the necessary conditions given by (7) hold true and in addition limt—O A(t) =0 is satisfied.
Proof. If conditions (16) hold true, then by theorem (3.16) of (Seierstad et al., 1986), the finite horizon transversality conditions do extend to the infinite horizon case. As a result, Ao = 1, jj > 0, Vj G SN and limt—O Ai(t) = 0 are satisfied for the constrained optimal control problem (Pi) and A i gK, Vi G S"n . Clearly assumption (1) is satisfied. So, the necessary conditions given by (7) hold true and in addition limt—O A(t) = 0.
Example 1 (Necessary Conditions): We consider the game problem (P) mentioned in example (1). It is easily verified that for (P), \fs(.)\ = 0, \g1j(.)\ = 0,
\g2i( )\ < 2\X(t)\\u1(t) — u2(t)\ < c (there exists a constant c > 0). Furthermore, we see that —to < Ji(x0,u1,u2) < to. The growth conditions mentioned in corollary (4) hold true for the game problem (P). Then from theorem (1) there exists a costate function A(t), with Hamiltonian defined as
H(.) := A(t) (ui(t) — u2(t)) + (a — (1 — a)X2(t)) (u2(t) — ui(t)) .
Further, H(.) attains a minimum w.r.t ui( ), i =1, 2 only if
A(t) + ^a — (1 — a)X* (t)) = 0 for all t G [0, to). (17)
As the growth conditions are satisfied we have limt—O A(t) = 0. The adjoint variable A(t) satisfies (by differentiating (17))
A(t) = 2(1 — a)X*(t)(u2(t) — u1(t)), Vt G [0, to) lim A(t) = 0.
t—— O
We see that the necessary condition (7b) also results in the same differential equation for A(t), V t G [0, to). Using (17) we can conclude that for arbitrary choices of u1(.) and u2(.), theorem (1) holds true by choosing a such that lim^oo x* (t) =
3.1. Discounted Autonomous systems
Many economic applications/situations can be effectively modeled as discounted autonomous optimal control problems. The growth conditions given in corollary (4) ensure that assumption (1) is satisfied. However, the conditions (16) are quite strict. In this section we analyze games defined by autonomous systems with exponentially discounted player’s costs. The discount factor p is assumed to be strictly positive,
i.e., p > 0. We represent the game problem in this section as (Pp) and the related subproblem as (PP). We notice that the subproblem (PP) is a mixed endpoint constrained optimal control problem. The cost minimizing efforts of players in the
coalition SnN result in terminal endpoint constraints in (PP). In (Mitchel, 1982)
proves necessary conditions for optimality for free endpoint infinite horizon optimal control problems. We first derive the necessary conditions for optimality for the mixed endpoint constrained problem (Pf) (in similar lines of (Mitchel, 1982)). If u*(t) is a Pareto optimal strategy for the game problem (PP), then from lemma
(2.2) u*(t) is optimal for each optimal control problem (PP), i G SN.
p OO
(PP) min / e-Ptgi(x(t),u(t))dt
ueu Jo
sub. to X(t) = f(x(t),u(t)), x(0) = x0, u(t) gU
Xj(t) = e-Ptgj(x(t),u(t)), xj(0) = 0, lim xj(t) < xj , Vj G SN.
j t——O j j
Let (x*(t),u*(t)), 0 < t <to be the optimal admissible pair for problem (PP), we fix T > 0 and define hi(z), i G SN as:
O
hi(z) = / e-Ptgi (x*(t — z + T),u*(t — z + T)) dt. (18)
J z
To derive the necessary conditions for optimality of u*(t), we first consider the following truncated and augmented problem (PiP ) (associated with the subproblem
(PiP )) defined as follows:
(PPT) min hi(z(T) — T)+ fT v(t)e-Pz(t)gi(Y (t), U(t))dt
1 u^U Jo
sub. to Y(t) = v(t)f(Y(t),U(t)), Y(0) = Y0, Y(T) = x*(T), U(t) gU
Yj(t)= v(t)e-Pz(t) gj (Y (t),U(t)),
Y/(0) = 0, Y/(T) + hj (z(T) — T) < Xj, Vj G SN
z(t) = v(t), z(0) = 0, v(t) G [1/2, to).
Remark 3. In problem (PP ), the constraint v(t) G [1/2, to) can be replaced by v(t) G [v, to) for 0 < v < 1.
Lemma 3. If (x* (t),u*(t)) is an optimal admissible pair for the problem (PP) then (x* (t),t,u*(t), 1), t G [0, T ], is an optimal admissible pair for the problem (PP ).
Proof. We prove the lemma using contradiction. Suppose (x*(t),t,u*(t), 1), t G [0, T] is not optimal for the problem (PP ) then there exists an admissible pair (Y(t),z(t),U(t),v(t)), t G [0, T] such that
pT p o
hi(U(T) — T)+ / v(t)e-Pz(t'>gi(Y(t),U(t))dt< e-Ptgi(x* (t),u* (t))dt Jo Jo
pT p o
hj(v(T) — T)+ v(t)e-Pz(t)gj(Y(t),U(t))dt < I e-Ptgj (x* (t),u*(t))dt,
oo
Vj g SN.
Since v(t) G [1/2, to), U(t) is an increasing function defined on [0, T] so by the inverse function theorem U(t) is invertible on [0, U(T)]. We define X(s) = y(z-1(s)) and u(s) = U(V-1(s)) for s G [0, v(T)] and observe that X(0) = y(z-1(0)) = x0 and X(z(T)) = Y(V-1(V(T))) = Y(T) = x*(T). Further, we have X(s) defined on s G [0, U(T)] as:
X(z(t)) = x0 + f Y(t)dt = x0 + f v(t)f(Y(t),U(t))dt.
oo
Taking s = z(t) we have
X(s) = X0 + f f(Y(z-1(s)),U(z-1(s)))ds = X0 + f f(X(s),u(s))ds,
J 0 J 0
for s G [0, v(T)]. Since X(s) satisfies the above integral equation we have X(s) = f (X(s), u(s)), X(0) = x0, s G [0, v(T)]. Next, for s > U(T), we define X(s) = x* (s — U(T) + T) and u(s) = u*(s — v(T) + T). Then we observe that X(s) = f (x(s), u(s)) with x(U(T)) = x*(T). Clearly the pair (X(s),u(s)), s G [0, TO is admissible for problem (PiP) and satisfies the following conditions
p O p O
/ e-Pt gi(X(t),u(t))dt< e-Pt gi(x* (t),u*(t))dt
J 0 J 0
p O p O
/ e-Ptgi(v(t),u(t))dt < e-Ptgj(x* (t),u*(t))dt, Vj G SnN
0 0 N
which clearly violates the optimality of (x*(t),u*(t)) for the problem (PP).
Theorem 2. If (x*(t),u*(t)), t G [0, to) is an optimal pair for the problem (Pf) then there exist Ai G KN, l0 G K" and continuous functions Ai(t) G K" and Yi (t) G K respectively such that
( A i, Ai(t), Yi(t)j = (0, 0, 0), Vt > 0, ^ A i, Ai(0)) =1 (19a)
Ai(t) = —e-pt GX(~A i,x* (t),u* (t)) — fX (x* (t),u* (t))Ai (t), Ai(0) = l0° (19b)
Yi(t) = p e-PtG( A i,x* (t),u* (t)), lim Yi(t) = 0 (19c)
t——O
H (A i,t,x* (t),u(t), A(t)) = Ai(t)f (x * (t),u * (t)) + e-PtG( A i,x* (t),u* (t))
H (A i,t,x* (t),u * (t),A(t)) < H (~\i,t,x* (t),u(t),A(t)) Vu(t) G U (19d)
H (A i,t,x* (t),u * (t),A(t)) = —Yi(t). (19e)
Proof. (Pf ) is a mixed endpoint constrained finite horizon optimal control problem. We first define the Hamiltonian HiT ( A iT,z(t),Y(t),v(t),U(tn as:
HiT (.)= v(t)e-pz(t) Ia0t gi(Y (t),U(t))+ £ j (t)gj (Y (t),U(t))
\ jeSN
+v(t)Ai(t)f (Y(t), U(t)) + v(t)Yi(t). (20)
The necessary conditions are: there exists A0T G K+, (t) G K, AiT (t) G K",
YiT (t) G K such that for almost every t G [0,T] (the partial derivatives of the Hamiltonian given below are evaluated at the optimal pair ((x*(t),t), (u*(t), 1))):
Aii (t) (HiT )Y
= —f'x (x * (t),u * (t)z)AiT (t) — AiT e-Pt gix(x *(t),u *(t))
— 5Z j(t)e-ptgjx(x*(t),u*(t)), AiT(T) free
j^SN
j (t) = — (HiT) j , j (T) > 0, j (T) (Aj (T) + hj (0) — X*) = 0, Vj G SN
YiT (t) = — (HiT )z , YiT (T) = AiT hix(0) + M jT(T )hjx(0)
(a0t , {j (t), j G SN },AiT (t),YiT (t)) = (0, •••, 0).
Since (HiT) xj = 0, we have MijT (t) = MijT > 0, Vj G SN. Let A ^ = (m!t , • • • ,Mi-1T, A0T ,Mi+\T, • • • ,MNt) . Then AiT G KN. Next we show by contradiction that also from the necessary conditions ^ A iT, AiT (0)) = (0, 0). For, if ^ A iT, AiT (0)) = (0, 0) then the necessary conditions give that AiT (t) = —f'x(x* (t),u* (t))AiT (t) which results in AiT (t) = 0 for t G [0, T]. Further, A iT =0 leads to YiT (t) = 0 for t G [0, T] which violates the necessary condition ^A iT ,AiT (t),YiT (t^ = (0, 0,0) for all 0 < t < T. Since AiT (T) is free, we choose (without loss of generality) AiT (0) such that A iT ,AiT (0)) = 1. The adjoint variable AiT (t) satisfies:
AiT (t) = —f'x (x* (t),u* (t))AiT (t) — e-ptGx (A iT ,x* (t),u(t)), AiT (0) = 1%, (21)
whereas YiT (t) satisfies
YiT (t) = — (HiT )z = P G(A iT ,x* (t),u * (t)). (22)
From the definition of h(.) we have:
pO ____ p O
YiT (T ) = pAiT e-ptgi(x* (t),u * (t))dt + p V' pjl e-ptgj (x* (t),u* (t))dt
Jt iGS* Jt
JO
= p e-ptG(A iT ,x* (t),u* (t))dt. (23)
TT
The Hamiltonian is linear in v(t) and the minimum w.r.t (U(t),v(t)) on the set U x [1/2, to) is attained at (u*(t), 1). The minimum of the Hamiltonian w.r.t v(t) for U(t) = u*(t) is achieved at an interior point of [1/2, to), so we have:
e-pt G(Aa iT ,X* (t),u* (t))+ AiT (t)f (x* (t),u* (t)) + YiT (t) = 0, Vt G [0, T]. (24)
The minimum of the Hamiltonian w.r.t U(t) is independent of v(t) (positive scaling) and does not depend on the term Yi.(t)v(t). So, the minimum of HiT w.r.t U(t) at v(t) = 1 is achieved at
u * (t) = argmin HiT (A iT ,t,x* (t),z(t) = t,U (t),v(t) = 1,AiT (t),YiT (t))
U (t)eU
= arg min (e-pt G( A iT ,x* (t),u(t)) + A'iT (t)f (x* (t),u(t))) , Vt G [0,T].
u(t)eU \ 1 )
(25)
Now, consider an increasing sequence {Tk}keN such that limk—O Tk = to. We can associate an optimal control problem (PiT ) with each Tk such that the necessary conditions as discussed above hold true. Then there exists a sequence
A iTfc ,l°T ) | such that A i^ G KN and
A /0
A 1, liTk
1. We, know from the
'+
Bolzano-Weierstrass theorem that every bounded sequence has a convergent subsequence. Using the same indices for such a subsequence we infer that there exists A i G KN and l0 such that
lim A ^ = A ^ KN and lim l°L = l0 such that
k—o k k—o k
ft,0) | = 1. (26) We observe that (29c) is a linear ODE. So we can write A^ (t) as:
AiTk (t) = &-fX (t, 0)l0 — $-fX (t,s)e ps Gx( A iT ,X* (s),u* (s))ds, (27)
where @-f£ (t, s) is the fundamental matrix associated with z(t) = —f'x(x* (t), u*(t)) z(t). Since the weights of Ai^ appear linearly in Gx(.) as k ^ to, Ai(t) satisfies the differential equation (19b). A similar argument holds for YiTk (t), condition (24) and (25) resulting in (19c), (19e) and (19d) respectively.
Remark 4. Though the approach in lemma (3) and theorem (2) is similar to the one given in (Mitchel, 1982), the main differences lie in problem formulation. In
(Mitchel, 1982), the necessary conditions are obtained for free endpoint unconstrained infinite horizon optimal control problem. In our case the problem (PP) is written as N mixed endpoint constrained optimal control subproblems (due to cost minimizing efforts of players). Furthermore, these constraints have a special structure, as a result the term G( A i,x*(t),u*(t)), weighted instantaneous undiscounted cost of the players, appears in all the subproblems (the choice of weights being Lagrange multipliers associated with the corresponding subproblem).
For autonomous systems (Mitchel, 1982) gives assumptions under which limt—O Ai(t) =0 for the free endpoint case with maximization problem. We show next, in corollary (5), that these conditions (formulated as assumptions (2) below) also suffice to conclude that in our subproblem (Pf) limt—O Ai(t) = 0. The proof is similar to the one given in (Mitchel, 1982). We repeat it for the sake of completeness.
Assumption 2 gi(x(t),u(t)), Vi G Sn is non positive or can be made non positive and there exists a neighborhood V of 0 G K" which is contained in the set of possible speeds f (x* (t),u(t)) for all u(t) G U if t ^ to.
Corollary 5. Let assumption (2) hold true. Then, an optimal solution for the problem (Pf) satisfies in addition to the conditions (19), the following transversality condition: limt—O Ai(t) =0.
Proof. From the necessary conditions (19d) and (19e) of theorem (2) we have
e-ptG( A i,x*(t), u(t)) + Ai(t)f (x* (t), u(t)) > e-ptG( A i,x*(t),u*(t))
+Ai(t)f (x* (t),u*(t)) = —Yi(t)( 28) By assumption (2) we have Ai(t)f (x*(t),u(t)) > —Yi(t). Next define q(t) as follows:
=-------mmv Since - 1 we have limsuPlk(i)ll = I-
max{1,\\Ai(t)\\} t—o
If l = 0 there is nothing to prove. So assume l > 0 and consider a sequence {t"} converging to infinity such that \\q(t")\\ > l/2. Since there exists u(t) G U such that Bgyo2 C /(x*(t), u(t)), there exists an e > 0 such that ^ < S. So, there exists u"(t") G U such that f (x* (t"),u"(t")) = — (2e/l) q(t"). Since, lim—o Yi(t) = 0 we take the above sequence {t"} such that —le/2 < —j(t") < le/2. Collecting all the above we have:
Ai(t")f (x* (t"),u"(t")) = — max{1,\\Ai(t" )\\} (2e/l) \\q(t")\\2 > —Yi(t")
Yi(t") > max{1, \\Ai(t")\\} (2e/l) \\q(t")\\2 > le/2.
Clearly, this is a contradiction and thus limt—O A(t) = 0.
Remark 5. a) If the instantaneous undiscounted costs of players gi(x(t),u(t)), i G Sn are bounded above for all pairs (x(t),u(t)),t G [0, to) then by assigning a new reward Ai(x(t),u(t)) = g(x(t),u(t)) — M with M = maxieSN supte[0o) gi(x(t),u(t)) leaves g(x(t),u(t)) non positive. Now, by defining a new game (PP) with A(.) as the instantaneous undiscounted costs for player i we observe that Pareto optimal controls (if they exist) of (PP) and (PP) coincide. We will use this idea in example (2) to find the Pareto optimal controls.
2 Unit ball in Knof radius S > 0.
b) Notice that the second condition in assumption (2) is identical to the notion of state reachability when the state dynamics is described by a linear constant coefficient differential equation.
We introduce another assumption based on growth conditions
(Seierstad et al., 1986) (Aseev et al., 2004) which ensure the transversality
condition i.e., limt^TO Xi(t) = 0 and as a result A i gK for the subproblem (PP).
Assumption 3 a) There exist a s > 0 and an r > 0 such that
\\9ix(x(t), u(t))\\ < s (1 + \\x(t)\\r) for all x(t) G R", u(t) G U and i G Sn.
b) There exist nonnegative constants c\, C2, c% and A G R, such that for every
admissible pair (x(t),u(t)), one has
\\x(t)\\ < ci + c2eXt for all t > 0 \\$fx (t, 0)\\ < c$extfor all t > 0.
c) For every admissible pair (x(t), u(t)) the eigenvalues of fx(x(t), u(t)) are strictly positive.
Corollary 6. Let the assumption (3) hold true. Then, an optimal solution for the problem Pi satisfies in addition to the conditions (19), the following transversality condition limt^TO Ai(t) =0 if p > (1 + r)A.
Proof. From the assumption (3) there exists constants c4 > 0, c5 > 0, c6 > 0 and c7 > 0 such that:
e-pt\gi(x(t), u(t))\ < e-pt (c4 + c5\\x(t)\\r+1) < c6e-pt + c7e-(p-(r+1)X)t.
Since p > (1 + r)A, the player’s costs Ji(u) converge for every admissible pair (x(t),u(t)). For the subproblem (Pi) we rewrite Ai(t) (from (19b)) as follows:
Aj(t)= (t, 0) 10 - f e-ps$-f, (t,s) Gx (x*(s),u*(s))ds
JO
= $-f, (t, 0)[l°i -Jt e-ps$-f, (0, s) Gx(x*(s),u* (s))ds ^we know <P-f L (t, s) = ($-!1(t, s)) = x(s,t)
= $-f, (t, 0) (l°0 -^t e-psf (s, 0) Gx(x* (s),u* (s))ds) .
The norm of Ai(t) is bounded as:
\\Aj(t)\\< \\$-f, (t, 0)| (\\lj>\\ + ^t e-ps ||f (s, 0)|| ||Gx(x*(s),u*(s))\| ds^j (from (3b) there exist a c8 > 0, a c9 > 0 such that)
< \\$-f, (t, 0) \| (\\l°\\ t e-ps (c8eXs + c9e(1+r)Xs) ds^j
(l0 is bounded, so there exist a c1o > 0, a c11 > 0 and c12 > 0) such that
< \\$-f L (t, 0)\| (do + cue-(p-X^t + c12e-(p-(1+r)X)t) .
Let 4><0(t) and ^o(t) denote the largest and smallest eigenvalues of the Hermitian part3 of -fx(x*(t),u*(t)). By assumption -f'x(x*(t),u*(t)) is bounded and has strictly negative eigenvalues so we have -to < ^0(t) < ^0(t) < 0, Vt > 0. From lemma (4.2) of (Hartman, 1964) we have:
exp ^,0(s)ds^ < W^/' (t, 0)\| < exp fjP(s)ds
Since /J°(s) < 0 for all s > 0 we have limt—TO W^/' (t, 0)\| = 0. By assumption p > (1 + r)A so limt—TO Ai(t) = 0 follows directly.
Remark 6. a) When the state evolution dynamics is linear and players objectives are convex in the control variable, the growth conditions (a) and (b) given in assumption (3) are similar to the ones given in (Aseev et al., 2004) and (Aubin and Clarke, 1979). b) In (Aseev et al., 2004), the free endpoint infinite horizon optimal control problem is approximated with a series of free endpoint finite horizon problems whereas in the current approach (PP) is approximated with fixed endpoint problems. As a result an additional term W^/' (t, 0)\| (H^H) appears in the estimate of \\Ai(t)\\ and condition (c) has to be included in assumption (3).
Theorem 3. Let assumption (2) or (3) hold true. If (J1(u*), J2(u*), ■ ■ ■ , Jn(u*)) is a Pareto optimal solution for problem (P) then there exists an ~a G Pn and a costate function A(t) : [0, to) ^ R" such that the following conditions are satisfied.
H(~a, t, x(t), u(t), A(t)) = A'(t)f (t, x(t), u(t)) + e-ptG(~a, x(t), u(t)) (29a)
H(a,t,x*(t),u*(t),A(t)) < H(a,t,x*(t),u(t),A(t)), Vu(t) GU (29b)
H0( ~a,t, x*(t),A(t)) = min H (a,t,x* (t),u(t),A(t))
u(t)eU
A(t) = -H°(~($,t,x*(t),A(t)), lim A(t) = 0 (29c)
t—
x*(t) = H°(a,t,x*(t),A(t)), x*(0) = x0 (29d)
'Y'(t) = p G(a,x*(t),u*(t)), lim 7(t) = 0 (29e)
t—
H0(a,t,x*(t),A(t)) = -Y(t) (29f)
(a, A(t)) = 0, Vt G [0, to), ~a G Pn . (29g)
Proof. If assumption (2) holds true then for each subproblem (PP) A i G K and limt—TO Aj (t) = 0. We define d = J2iESN °-i = 2 (\9 + Ejesj, Mi) , SN
and a vector ~a = (a1, ■ ■ ■ , aN)'. We notice that ~a G Pn. Taking the summation of equation (19b) for all i G Sn and defining A(t) = ^ '}2iesN \{t) we observe that conditions (29b) and (29c) are satisfied. Taking the summation of equation (19c) for all i G SN and defining 7(t) = ^ EieSjv 7i(t), the conditions (29e) and (29f) are satisfied. Since, ~a G Pn C K and limt—TO A(t) = 0 we observe that (29g) is satisfied.
We consider the example from (Kamien and Schwartz, 1991) to illustrate usage of assumption (2) and necessary conditions given in theorem (3).
3 The Hermitian part of matrix A is defined here as AH = |(A + A').
Example 2. Consider a fishery game with two players. The evolution of the stock of fish, in a particular body of water, is governed by the differential equation
x(t) = ax(t) — bx(t) lnx(t) — ui(t) — u2(t), x(0) = x0 > 2 (30)
where x(t) refers to the stock of fish, and a > 0, b > 0. It is assumed that x(t) >
2,t e [0, to). In (30), the stock of fish x(t) depends upon ax(t) births, bx(t)lnx(t) deaths and the fishing efforts of player i, ui(t) = wi(t)x(t), at each point in time t. Each fisherman tries to maximize his utility Ji( ), given by:
ptt
Ji(xo,ui,u2) = / e-pt lnui(t) dt.
J 0
We assume 0 < e < wi(t) < to for the utility to be well defined. By taking the transformation y(t) = ln x(t) the system (30) is modified as:
y(t) = a — by(t) — wi(t) — w2(t), y(0) = lnx(0), (31)
and the player’s utility is transformed as:
p tt
Ji(xo,ui,u2)= e~pt (y(t) + lnwi(t)) dt. (32)
0
We notice that the instantaneous undiscounted reward is bounded below, by controllability of the system (31) and remark (5.a) we notice that assumption (2) is satisfied. So all the Pareto solutions can be obtained by solving the necessary conditions associated with the weighted sum optimal control problem :
min —
W1,W2 | ./0
/ e-pt (y(t) + a ln w1(t) + (1 — a) lnw2 (t)) dt
0
subject to (31). Defining the Hamiltonian as
H(a, t, y, wi, w2, A) = A (a — by(t) — wi(t) — w2(t))
—e-pt (y(t) + a ln w1 (t) + (1 — a) ln w2(t)) .
Taking HWi =0, *=1,2 gives w\(t) = and w2(t) = — The
adjoint variable is governed by
^(t) = bA(t) + e-pt, lim A(t) = 0, t—
and the solution is given as A(t) = The candidates for Pareto optimal strate-
gies in open loop form are given by:
u1 (t) = a(p + b)em(t,Xo)
u*2 (t) = (1 — a)(p + b)em(t’Xo),
where m(t,xo) = e-bt lnxo+ (l — e~bt). The candidates for Pareto solutions
are given as
p ln xo + a — (p + b) ln(a(p + b)
J\ (xo,uhu2 =----------------- ——----------1--------------
P(P + b) P
p\nx0 + a-(p + b) ln((l -a)(p + b))
J2 \xo, mi, u2) —------------ ———----------1-----------------------------------. (33)
P(P + b) P
4. Sufficient Conditions for Pareto Optimality
It is well known (Fan et al., 1957) that if the action spaces as well as the players objective functions are convex then minimization of the weighted sum of the objectives results in all Pareto solutions. We give the following theorem from (Engwerda, 2005).
Theorem 4. If U is convex and Ji(u) is convex for all i = 1, 2, • • • ,N then for all Pareto optimal u* there exist a e Pn, such that u* e argminueu^i=i Ji(u).
Recently in (Engwerda, 2008), this property was used to obtain both necessary and sufficient conditions for the existence of Pareto optimal solutions for regular convex linear quadratic differential games. In general it is a difficult task to check if the players objectives are convex functions of initial state and controls. However, under some conditions the solutions of (29) result in Pareto optimal strategies. In this section we derive sufficient conditions for a strategy to be Pareto optimal. The sufficient conditions given in the theorem below are inspired by Arrow’s sufficient conditions (Seierstad et al., 1986) in optimal control. Further, these sufficient conditions are given for non autonomous systems and they hold true for discounted autonomous systems as well.
Theorem 5. Assume that there exists ~a e Pn, a costate function A(t) : [0, to) ^ R" satisfying (29c). Introduce the Hamiltonian H(t, a,x(t),u(t),A(t)) := f (t,x(t), u(t)) + G(a,t, x(t), u(t)). Assume that the Hamiltonian has a minimum w.r.t u(t) for all x(t), denoted by
H0(~a,t,x(t),A(t)) = min H(a,t,x(t),u(t),A(t)).
u(t)eU
If H°(~a,t,x(t),A(t)) is convex in x(t) and liminft—tt A'(t) (x*(t) — x(t)) > 0, then u*(t) is Pareto optimal.
Proof. From the convexity of H°(~a,t, x(t), A(t)) we have:
H 0(a, t,x(t), A(t)) — H 0(a, t, x* (t), A(t)) > HX(a, t, x* (t), A(t)) (x(t) — x* (t))
Since, H (~a,t,x(t),u(t),A(t)) > H0(~a,t,x(t),A(t)) and
H (a, t, x*(t), u* (t), A(t)) = H 0(~a, t, x* (t), A(t)) we have:
H (~a,t, x(t), u(t), A(t)) — H (a, t, x*(t),u*(t), A(t)) >
HX'(—a,t, x* (t), A(t)) (x(t) — x*(t))
= —A'(t) (x(t) — x*(t)) (by (29c)).
Using the definition of Hamiltonian the above inequality can be written as:
A'(t) (f (t,x(t),u(t) — f (t,x*(t),u* (t)))
+G(a, t, x(t), u(t)) — G(a, t, x*(t), u* (t)) > —A'(t) (x(t) — x* (t)),
(G(~a, t, x(t), u(t)) — G(~a, t,x*(t),u* (t))) > ,\'(t) (x* (t) — x(t)) + A'(t) (x*(t) — x(t))
Taking the integrals on both sides we have
rT T
/ (G(~a , t, x(t), u(t)) — G(~a, t, x*(t), u*(t))) dt > (A'(t) (x*(t) — x(t)))
J 0
0
As x* (0) = x(0) = x0 and A(0) is bounded the above inequality is given as:
I (G(a,t,x(t),u(t)) — G(a,t,x*(t),u*(t))) dt > (A'(T)(x* (T) — x(T))).
0
Taking T
J(u) — J(u*) > lim A'(t) (x*(t) — x(t)) > liminf A'(t) (x*(t) — x(t)) > 0. t—t—
Clearly, by lemma(2.1) u* is Pareto optimal.
Example 1 (sufficient conditions): We illustrate theorem (5) by considering example (1) again. First, we notice that H0(t, x*, A) = 0 so H0(t, x*, A) is convex in x(t). Next, limt—TO x(t) exists and is finite and limt—TO A(t) = 0, so liminft—x A(t) (x* (t) — x(t)) = 0. So, by theorem (5) every control (ui, u2) is Pareto optimal. Example 2 (sufficient conditions): For example (2) the candidates for Pareto solutions are given by (33). If the model in example (2) satisfies the sufficient conditions, mentioned in theorem (5), then all Pareto solutions are indeed given by (33). The minimized Hamiltonian is given by:
H°(t,y(t),A(t)) = -^-^j-e pt + e pt (1-----------—— — In (a“(l — a)1 a(p + b))\.
p + b \ p + b J
Clearly, H°(t,y(t),A(t)) is convex (linear here) in y(t). Since wi(t), i = 1, 2, is bounded we have \y(t)\ < (ci + C2e~bt). Further, A(t) = — thus we have liminft—x A(t)(y*(t) — y(t)) = 0. The fishery model satisfies the sufficient conditions as given by theorem (5). So, all the candidates given by (33) are Pareto solutions. Figure (1(a)) illustrates trajectories of optimal fish stock x*(t) and fishing efforts of the players u1(t) and u2(t). Figure (1(b)) illustrates the Pareto surface for a e [0.1714, 0.8266] (Pareto solutions which ensure positive returns for the players are shown).
5. Conclusions
In this paper, we studied the problem of existence of Pareto solutions of infinite horizon cooperative differential games. We assumed an open loop information structure. We considered nonautonomous and discounted autonomous systems for the analysis. Firstly, we gave a necessary and sufficient characterization of Pareto optimality which enables to view the problem of finding Pareto solutions as N constrained infinite horizon optimal control subproblems. We gave weak sufficient conditions under which all Pareto solutions can be obtained by solving a weighted sum optimal control problem. These weak sufficient conditions relate to conditions which extend the finite horizon transversality conditions to infinite horizons. We gave sufficient conditions for Pareto optimality and provided examples where the necessary conditions are also sufficient to find all Pareto solutions.
0 ------------,------------j------------j-----------i------------j-----------j----1-------i-----------1
0 5 10 15 20 25 30 35 40
time units
(a) Trajectories of optimal fish stock x*(t) and fishing efforts of players u* (t) and u* (t)
Ji
(b) Pareto surface for 0.1714 < a < 0.8266
Figurel. Fishery model with parameters a = 1, b = 0.2, p = 0.05 and xo = 2.
References
Aseev, S. and V. Kryazhimskiy (2004). The Pontryagin maximum principle and transver-sality conditions for a class of optimal control problems with infinite time horizons. SIAM Journal on Control and Optimization, 43(3), 1094-1119.
Aubin, J. P. and F. H. Clarke (1979). Shadow prices and duality for a class of optimal control problems. SIAM Journal on Control and Optimization, 17(5), 567-586.
Blaquiere, A., L. Juricek and K.E. Wiese (1972). Geometry of Pareto equilibria and maximum principle in N person differential games. Journal of Optimization Theory and Applications, 38, 223-243.
Chang, S. S. L. (1966). General theory of optimal processes. SIAM Journal of Control, 4(1), 46-55.
Engwerda, J. C. (2005). LQ dynamic optimization and differential games. Wiley: Chichester.
Engwerda, J. C. (2008). The regular convex cooperative linear quadratic control problem. Automatica, 44(9), 2453-2457.
Engwerda, J. C. (2009). Necessary and sufficient conditions for Pareto optimal solutions of cooperative differential games. Submitted.
Fan, K. I. Glicksberg and A. J. Hoffman (1957). Systems of inequalities involving convex functions. American Mathematical Society Proceedings, 8(3), 617-622.
Grass, D., J. P. Caulkins, G. Feichtinger, G. Tragler and D. A. Behrens (2008). Optimal Control of Nonlinear Processes with Applications in Drugs, Corruption and, Terror. Springer.
Halkin, H. (1974). Necessary conditions for optimal control problems with infinite horizons.
Econometrica, 42, 267-272.
Hartman, P. (1964). Ordinary Differential Equations. John Wiley & Sons.
Kamien, M. I. and N. L. Schwartz (1991). Dynamic Optimization: The Calculus of Variations and, Optimal Control in Economics and Management. Elsevier Science. Leitmann, G., S. Rocklin and T. L. Vincent (1972). A note on control space properties of cooperative games. Journal of Optimization Theory and Applications, 9, 379-390 Leitmann, G. (1974). Cooperative and Non cooperative Many Player Differential Games, Springer Verlag, Berlin.
Mitchel, P. (1982). On the transversality conditions in infinite horizon optimal problems.
Econometrica, 50, 975-985.
Seierstad, A. and K. Sydsaeter (1986). Optimal control theory with economic applications . North-Holland, Amsterdam.
Seierstad, A. (1999). Necessary conditions for nonsmooth, infinite-horizon, optimal control problems. Journal of Optimization Theory and Applications, 103, 201-229.
Simon, C. P. and L. Blume (1994). Mathematics for Economists. Norton & Company: New York, 1994.
Stalford, H. L. (1972). Criteria for Pareto optimality in cooperative differential games.
Journal of Optimization Theory and Applications, 9, 391-398.
Vincent, T. L. and G. Leitmann (1970). Control space properties of cooperative games.
Journal of Optimization Theory and Applications, 6, 91-104.
Zadeh, L. A. (1963). Optimality and non-scalar-valued performance criteria. IEEE Transactions on Automatic Control, 8(1), 59-60.
