CC BY
64URAL MATHEMATICAL JOURNAL, Vol. 5, No. 1, 2019, pp. 24-30
DOI: 10.15826/umj.2019.1.003
ON NECESSARY OPTIMALITY CONDITIONS FOR RAMSEY-TYPE PROBLEMS1
Anton O. Belyakov
Moscow School of Economics, Lomonosov Moscow State University, Leninskiye Gory, Moscow, 119992, Russia;
National Research Nuclear University "MEPhl", 31 Kashirskoe shosse, Moscow, 115409, Russia;
Krasovskii Institute of Mathematics and Mechanics,
Ural Branch of the Russian Academy of Sciences, 16 S. Kovalevskaya Str., Ekaterinburg, 620990, Russia
belyakov@mse-msu.ru
Abstract: We study an optimal control problem in infinite time, where the integrand does not depend explicitly on the state variable. A special case of such problem is the Ramsey optimal capital accumulation in centralized economy. To complete the optimality conditions of Pontryagin's maximum principle, so called transversality conditions of different types are used in the literature. Here, instead of a transversality condition, an additional maximum condition is considered.
Keywords: Pontryagin maximum principle, Transversality condition, Optimal control, Ramsey problem.
1. Introduction
Economic theory has been involving optimal control theory since invention of Pontryagin's maximum principle. Based on the maximum principle [1], David Cass in [2] elaborated the problem of optimal capital accumulation first discussed by Frank Ramsey [3]. Since then, this model, sometimes referred to as Ramsey-Cass-Koopmans model, is a component of advanced books on macroeconomics and growth theory, see, e.g. [4]. This model has the infinite planning horizon as well as many other models in economics.
Optimal control with infinite horizon still has theoretical issues to investigate, see [5]. The proof of the maximum principle for the infinite time horizon one can find, e.g., in [6]. The proved theorem does not include transversality conditions. Moreover, it is known [6-8] the that usually used forms of transversality conditions
lim ^(i) = 0, (1.1)
t—y^o
lim (X(t),^(t)) =0, (1.2)
t—ro
might be not necessary, where X is the optimal state variable, ^ is the corresponding adjoint variable, and the angle brackets (■, ■) denote the scalar product of two vectors.
The condition considered by Philippe Michel in [9], under assumptions including that the objective functional takes only finite values, has the form of the Hamiltonian H converging to zero
1This work was supported by the Russian Science Foundation (project no. 17-11-01093).
lim H(x(t),u(t),t, ^(t)) = 0, (1.3)
t—>-<ro
where u is the optimal control.
Karl Shell [7, Lecture III] proposes, without proof, the condition
lim sup(^(t), x(t)) > lim inf(^(t),x(t)), (1.4)
t—ro t—ro
where x is any admissible path of the state variable. In [7, footnote 4 for Lecture III], he says that
"this conjecture is related to a conjecture made by Kenneth J. Arrow in private correspondence."
Note that Arrow's sufficiency theorem contains a condition that follows from (1.4), which one can expect for problems, where the maximum principle provides both necessary and sufficient conditions of optimality.
In [10-14], the authors determine the adjoint variable uniquely by a Cauchy-type formula, that solves the adjoint equation with transversality conditions in the following form:
lim Y(t) ^(t)=0, (1.5)
t—ro
where Y(t) is the fundamental matrix of the state equation linearized about the optimal solution, see eg. [15, 16].
Conditions (1.2)-(1.5) do hold for an optimal solution of the Ramsey problem with exponential discounting, when ^(t) — 0 as t — ro. But all (1.2)-(1.5) fail for the zero discount rate, when the objective functional diverges and ^(t) — const > 0 as t — ro. This case was considered by Ramsay in [3, p. 543] with moral motivation:
"One point should perhaps be emphasised more particularly; we do not discount later enjoyments in comparison with earlier ones, a practice which is ethically indefensible and arises merely from the weakness of the imagination; we shall, however, in Section II include such a rate of discount in some of our investigations."
Even-though condition (1.3) is proved only for converging functionals, it can hold if we modify the objective improper integral subtracting from its integrand a constant, called "bliss" in [3], such that, for the optimal solution, the integral converges, see e.g., [2, Section 7].
In this paper, a new necessary condition is obtained for Ramsey-type problems both with and without discounting, where the integrand of an objective functional does not explicitly depend on the state variable. Since, without discounting, the objective functional diverges, we involve the concept of overtaking optimality. The new condition is the maximum of the integrand that completes the Pontryagin's maximum principal conditions. In contrast to the maximum condition for the Hamiltonian, the maximum of the integrand is taken w.r.t. control within admissible trajectories that satisfy the maximum principle (the maximum of the Hamiltonian, state and adjoint equations). The example of the Ramsey problem without discounting is considered.
2. Statement of the problem
Let X be a nonempty open convex subset of R, and let U be an arbitrary nonempty set in R. Let us consider the following optimal control problem:
f ro
/ e-ptg(u(t))dt — max, (2.1)
J to u
x(t) = f (x(t),u(t)), x(to) = xo, (2.2)
where the control u(t) € U and the state variable x(t) € X for all t € (t0, We call such control
u(-) and state variable x(-) trajectories admissible. The functions f and g are differentiable w.r.t. all their arguments, and f together with its partial derivatives is continuous in (x,u). Moreover, the function g is strictly concave and p > 0.
2.1. The concept of optimality
The improper integral in (2.1) might not converge for any candidate for optimal control u(-); i.e., the limit
lim J(u(-),t0,T), (2.3)
t ^^
might fail to exist, or might be infinite, where we introduce the finite time horizon functional:
cT
J(u(-),Î0,T) = / e-ptg(u(i))di
J to
subject to state equation (2.2). Thus, the functional J may be unbounded as T ^ ro. So, we use the following more general concept of optimality.
Definition 1. An admissible control u(-) is overtaking optimal (OO) if for every admissible control u(-) :
lim sup (J (u(-),t0 ,T) - J (u(-),t0,T)) < 0.
t ^^
When the ordinary optimality holds, i.e. a finite limit exists in (2.3) and, for all admissible controls u(-),
limsup J(u(-),t0,T) < lim J(u(-), t0, T),
then u(-) is also OO.
3. Optimality conditions 3.1. Pontryagin's maximum principle
With the use of the adjoint variable we introduce current value Hamiltonian
H(x, u, A) = Ag(u) + ^ f (x, u).
Theorem 1 [1, 6, 7]. There exist A > 0 and such that (A,^0) = 0 and the maximum condition
H(x(t),û(t),^(t),A) = maxH(x(t),u,^(t), A) (3.1)
u£U
holds along with the adjoint equation
df
= + Ip{t)-^{x{t),ù{t)), tP(to) = '00. (3.2)
In this theorem, remains undetermined. Note that, for ^ = = 0, the maximum condition (3.1) might have no solution with A > 0, while A = = 0 contradicts the theorem. Additional arguments are used to refine solutions of (3.1)-(3.2) and single out a nonzero value of 00.
But it turns out that the maximum condition (3.1) with ^ = 0 and A = 1 can yield an additional necessary optimality condition if we replace the set U by the set U(x(t)) defined as follows.
Definition 2.
U7(x) = {u : (u, x) € G},
where G C U x X consists of graphs of all admissible trajectories (u(-),x(-)) satisfying maximum principle (3.1)-(3.2) and the state equation (2.2). T^e denote the set of such trajectories by D, so that
G = {(u(t), x(t)) : t € (to, to), (u('), x(*)) € D}. 3.2. An additional maximum condition
In order to use the condition in the main Proposition 1, we first need to calculate the sets U(x), thus making the synthesis of control. Due to the continuous dependence of solutions of the differential equation (3.2) on the initial value and due to the convexity of the set X, we state the following.
Assertion 1. If an admissible pair (u(-),X(-)) belongs to D, i.e., satisfies the maximum principle (3.1)-(3.2) and the state equation (2.2), then, for almost all t > t0, there exists a0 > 0 such that, for all a € (0, a0), the control
u(t), t < t — a, u(t), t € (t — a, t], u(t), t > t,
is admissible, where u(t) for t € (t — a, t] satisfies the maximum principle (3.1)-(3.2) and the state equation
X(t) = f (x(t), u(t)), x(t — a) = X(t — a), with the current state x(t) at time t = t — a as the initial condition.
The following main proposition is proved with the needle variation technique similar to the one in [11], provided that Assertion 1 holds.
Proposition 1 (Necessary optimality condition). Let there exists an admissible pair (u(-),x(-)) € D. If the control U is optimal, then
g(u) < g(U(T)) (3.3)
for almost all t € [t0, to) and all u € U(x(t)).
Proof. According to Assertion 1, we can define an admissible needle variation of optimal pair (U(-),x(-)) € D at almost any time t, i.e., excluding discontinuity time instances of control, as
ÎU(t), t < t — a, u(t), t € (t — a, t], U(t), t > t,
where u(t) satisfies the maximum principle (3.1)-(3.2) and the state equation x(t) = f(x(t),u(t)) with the condition x(t — a) = x(t — a). Thus,
rr— a rr rT
J(ua(-),t0,T)= / e—ptg(U(t))dt + / e—ptg(u(t)) dt W e—ptg(U(t)) dt. ./îq Jt—a Jt
The corresponding increment in the value of the functional can be written as follows:
AJ«(T) := J(ua(-),io- J(u(-),ic,T) = J(u«(-),r - a,T) - J(u(-),r - a,T) =
= I" e—pt (g(u(t)) - g(u(t)))dt,
Hence,
^^ > ^ f e"" (g(u(t)) - g(u(t))) dt, (3.4)
a aJT—a
We have the following approximation of the latter term in (3.4), due to the continuity of u and u at t and the continuity of g w.r.t. u:
T
I [„-*
a
Je—pt (g(u(t)) - g(U(t))) dt = e—pT (g(u(T)) - g(U(T))) + O(a),
where lim O(a) = 0. Hence, inequality (3.4) takes the form
A J (T)
—^ > e_pr (g(u(r)) - g(u(r))) + O(a). a
Definition 1 of OO for the initial problem means that, for all > 0, there exists T2(e2) > to such that, for all T' > T2(e2), we have AJa(T') < e2. Let us take e2 = ae. Then the inequality AJa(T') < ae results in
e > e—pT (g(u(T)) - g(u(T))) + O(a). (3.5)
Suppose that (3.3) is violated, i.e., there exist e > 0 and u € U(x(t)) such that the following inequality holds:
e—pT (g(u) - g(u(T))) > 2e, this contradicts to (3.5) for u = u(t), when a is small enough. Hence, (3.3) should hold. □
Corollary 1. If g is a strictly monotone function and there exists an optima! control, then the optimal trajectory (u(-),X(-)) is unique and its graph belongs to the boundary of the set G.
This is the case in the Ramsey problem of optimal capital accumulation, where the optimal trajectory is on the saddle path separating the domain of admissible pairs (c(-),k(-)) of consumption c and capital k that does not deplete capital, k(t) > 0 for all t > 0, see the domain of thin solid lines in Fig. 1.
Example 1. [The Ramsey problem with p = 0.] We maximize the aggregated constant relative risk aversion utility
r c(t)1—'9 , / -— di —> max
Jo 1 - 0 c>0
subject to the dynamics of the capital
k(t) = k(t)a - 5k(t) - c(t), k(t) > 0,
where k(0) = ko > 0, 0 = 1, 0 > 0, and a € (0,1). The Hamiltonian:
c1—9
H{k, c, 0, A) = A -—- + 0 (ka - 5k - c), A > 0, (A, ip) / 0. 1-0
T— a
T—a
60 80 capital (k)
Figure 1. The bold lines are the stationary curves, k = 0 and c = 0. The solid lines are the trajectories governed by the state and Euler equations, for which k(t) > 0 for all t > t0.
The abnormal case (A = 0) would lead to ^ = 0 and thus impossible. The stationarity condition
A c(t)-e = ^(t)
and the adjoint equation
-ip(t) = (ak(t)a-1 - 5) ^(t)
result in the Euler equation
C(t) _ ak(t)a-1 - 5
c(t) 6
Due to the Euler and state equations, any admissible pair (k(•),€(•)) not violating the constraints c(t) > 0 and k(t) > 0 converges either to the steady state (k*, c*), where k* = (5/a)1/(a-1) and c* = (1 - a) k* > 0, or to (51/(a-1), 0), where k* < 51/(a-1). The solid lines in Fig. 1 belong to D and their graphs constitute the set G, which is whole space below the saddle path and the horizontal line c = 0.2 Thus, condition (3.3)
m
1-0
„1-0
>
i - d - i - d
for all c G {c : (c,k(t)) £ G}
selects the saddle path as unique possible optimal.
2It is easy to see in Fig. 1, that Assertion 1 holds, since such a variation of the saddle path trajectory diverts the remaining part of the trajectory below the saddle path so that its graph remains in G, even though the trajectory itself might not belong to D anymore, because its remaining part does not have to satisfy the maximum principle.
*
4. Conclusion
A quite simple additional necessary optimality condition is derived in the form of maximization of the integrand of the objective functional in each state x on the subset of control values U(x) C U. Even though one should first determine the set D of admissible trajectories that solve state and adjoint equations subject to the maximum condition, it is anyway done in practice when possible. The substantial assumption is that the objective functional does not explicitly depend on the state. The functional is subject to the state equation indirectly via the form of an open admissible set X. In a more general case, the additional maximum condition is more involved, see [17].
REFERENCES
1. Pontryagin L. S., Boltyanskii V. G., Gamkrelidze R.V., Mishchenko E.F. The Mathematical Theory of Optimal Processes. NY: Pergamon, Oxford, 1964. 338 p.
2. Cass D. Optimum growth in an aggregative model of capital accumulation. Rev. Econom. Stud., 1965. Vol. 32, No. 3. P. 233-240. DOI: 10.2307/2295827
3. Ramsey F.P. A mathematical theory of saving. The Economic J., 1928, Vol. 38, No. 152. P. 543-559.
4. Romer D. Advanced Macroeconomics. 4-th ed. NY: McGraw-Hill, 2012. 736 p.
5. Carlson D.A., Haurie A.B., Leizarowitz A. Infinite Horizon Optimal Control. Berlin-Heidelberg: Springer-Verlag, 1991. 332 p. DOI: 10.1007/978-3-642-76755-5
6. Halkin H. Necessary conditions for optimal control problems with infinite horizons. Econometrica, 1974. Vol. 42, No. 2. P. 267-272. DOI: 10.2307/1911976
7. Shell K. Applications of Pontryagin's maximum principle to economics. In: Mathematical Systems Theory and Economics I/II. Lect. Notes Oper. Res. and Math. Econ, 1969. Vol. 11-12. P. 241-292.
8. Kamihigashi T. Necessity of transversality conditions for infinite horizon problems. Econometrica, 2001. Vol. 69, No. 4. P. 995-1012. DOI: 10.1111/1468-0262.00227
9. Michel P. On the transversality condition in infinite horizon optimal problems. Econometrica, 1982. Vol. 50, No. 4. P. 975-985. DOI: 10.2307/1912772
10. Aseev S., Veliov V. Maximum principle for infinite-horizon optimal control problems with dominating discount. Dynamics of Continuous, Discrete and Impulsive Systems. Series B: Applications & Algorithms, 2012. Vol. 19, No. 1-2. P. 43-63. URL: http://pure.iiasa.ac.at/9874
11. Aseev S., Veliov V. Needle variations in infinite-horizon optimal control. Contemp. Math., 2014. Vol. 619. P. 1-17. DOI: 10.1090/conm/619/12381
12. Aseev S. M., Besov K.O., Kryazhimskii A.V. Infinite-horizon optimal control problems in economics. Russian Math.. Surveys, 2012. Vol. 67, No. 2. P. 195-253.
13. Aseev S. M., Kryazhimskiy A. V. The Pontryagin maximum principle and transversality conditions for a class of optimal control problems with infinite time horizons. SIAM J. Control Optim., 2004. Vol. 43, No. 3. P. 1094-1119. DOI: 10.1137/S0363012903427518
14. Aseev S. M., Veliov V. M. Maximum principle for infinite-horizon optimal control problems under weak regularity assumptions. Proc. Steklov Inst. Math., 2015. Vol. 291, Suppl. 1. P. 22-39. DOI: 10.1134/S0081543815090023
15. Khlopin D. Necessity of vanishing shadow price in infinite horizon control problems. J. Dyn. Control Syst., 2013. Vol. 19. No. 4. P. 519-552. DOI: 10.1007/s10883-013-9192-5
16. Khlopin D. Necessity of limiting co-state arcs in Bolza-type infinite horizon problem. Optimization, 2015. Vol. 64, No. 11. P. 2417-2440. DOI: 10.1080/02331934.2014.971413
17. Belyakov A. O. Necessary Conditions for Infinite Horizon Optimal Control Problems Revisited. 2017. arXiv: 1512.01206v2 [math.OC]
