CC BY
42
University of California, School of Information and Computer Science. 2010.
9. Yu J. J. Q., Lam A. Y. S., Li V. O. K. Evolutionary Artificial Neural Network Based on Chemical Reaction Optimization. In 2011 IEEE Congress on Evolutionary Computation (CEC’2011), New Orleans, LA, 2011.
10. Dam H. H., Abbass H. A., Lokan C., Yao X. Neural-Based Learning Classifier Systems. IEEE Transactions
on Knowledge and Data Engineering. 2008, vol. 20, № 1, p. 1-14.
11. Zhi-Hua Zhou and Yuan Jiang. NeC4.5: Neural Ensemble Based C4.5. IEEE Trans. Knowl. Data Eng. 2004, № 16.
© Попов Е. А., Семенкина М. Е., 2013
УДК 005; 519.7; 303.732
ABOUT PERFORMANCE IMPROVEMENT OF EVOLUTIONARY STRATEGIES TECHNIQUE APPLIED TO OPTIMAL CONTROL PROBLEM INDIRECT METHOD
I. S. Ryzhikov
Siberian State Aerospace University named after academician M. F. Reshetnev 31, Krasnoyarsky Rabochy Av., Krasnoyarsk, 660014, Russian Federation. E-mail: [email protected]
The optimal control problem for nonlinear dynamic systems is considered. The proposed approach is based on both partially analytical and partially numerical techniques of the optimal control problem solving. Optimal control problem is reduced to unconstrained extremum problem, which is related to seeking for the initial point of the co-state variables that would satisfy the boundaries. To solve the optimization problem, well-known global optimization techniques are suggested and compared. The performance of the evolutionary strategies algorithm was increased by implementing the special restarting condition in the scheme.
Keywords: stochastic optimization, optimal control, indirect method, variation problem, dynamic system.
О ПОВЫШЕНИИ ЭФФЕКТИВНОСТИ АЛГОРИТМА ЭВОЛЮЦИОННЫХ СТРАТЕГИЙ ДЛЯ РЕШЕНИЯ ЗАДАЧИ ОПТИМАЛЬНОГО УПРАВЛЕНИЯ НЕПРЯМЫМ МЕТОДОМ
И. С. Рыжиков
Сибирский государственный аэрокосмический университет имени академика М. Ф. Решетнева Российская Федерация, 660014, Красноярск, просп. им. газ. «Красноярский рабочий», 31
E-mail: [email protected]
Рассматривается задача нахождения оптимального управления для нелинейных динамических систем. Предложенный подход основан на частично аналитическом и частично численном решении задачи оптимального управления. Исходная задача сводится к задаче поиска экстремума функций без ограничений, решением которой являются начальные координаты для сопряженных переменных, при которых удовлетворяются граничные условия. Для решения приведенной задачи сравнивались различные широко известные методы глобальной оптимизации. Эффективность метода эволюционных стратегий была повышена через введение специального условия на перезапуск алгоритма.
Ключевые слова: стохастическая оптимизация, оптимальное управление, косвенный метод, вариационная задача, динамическая система.
In this paper the idea of modified evolutionary strategies algorithm performance improvement is investigated. Some hypotheses about the ways to increase the efficiency were put forward. In previous work the different evolutionary and nature-based algorithm were examined and it was shown that these techniques are not reliable, so there is a need in special operands to be implemented. Current work consists of problem definition, previous results and suggested improvements.
The optimal control problem for dynamic systems
with one control input and integral functional is considered. Since the problem is old and it originates from the practical needs, there exist many techniques to solve the optimal control problem in different problem definitions and for different systems. But the developing of the modern technologies creates new optimal control problems that cannot be solved via well-known and classical approaches. The main problem is nonlinearity of the system model or the criterion. In general case, there is no universal analytical technique that guarantees the solution of
nonlinear differential equation to be found. But using the maximum principle, we can always determine the characteristics of the function that is suspected to be the solution of the optimal control problem.
On the other hand, the numerical approaches are useful and efficient but only for some problems that they were designed for. Any control function approximation technique that is being used to determine the solution for the initial optimal control problem is related with reduction of the problem to extremum seeking on the real vector field. And the problem reduction uses a convolution of different objective functions and penalty functions for all the constraints, and it requires more computational resources and more efficient optimization algorithms. There is no doubt that the direct method based techniques are efficient, but increasing of accuracy of the function approximation leads to increasing of extremum problem dimension.
The indirect method of solving the optimal control problem is related with solving the extremely difficult boundary-value problem, but the found solution gives us the proper control function with the known structure. In the given study the shooting method is based on the modified evolutionary optimization algorithm.
It is important to highlight that there is a sufficient benefit of using the information science techniques of solving the complex optimization problems. The modern methods and algorithms from the fields of informatics, bioinformatics and cybernetics are reliable, flexible and highly efficient techniques. And it is possible to improve them for every distinct optimization problem with unique characteristics via modifying the schemes, operators or hybridizing the algorithms.
Many works on optimal control problem solving for nonlinear dynamic systems are about some specific tasks. Many works are about the approaches to solve optimal control problems for affine nonlinear systems, like the work mentioned before, for example, [1] and [2]. In the last article the studied problem is related to optimal control of nonlinear systems via usage of the Lyapunov functions, but only one boundary in problem definition is considered. Also, in article [3] the approach of predictive optimal control for nonlinear systems is considered. There are plenty of numerical techniques application examples, [4]. Actually, since the problem is complex and there are many problems with unique features, and there are many different problem definitions for optimal control.
These techniques also require an analytical form of the system state and fit only the considered structures. And in our study, the proposed approach with implementation of some efficient global optimization technique is suggested to be applicable and reliable for solving many optimal control problems, as an effective analogue to shooting-based techniques.
In the study [5] symbolic-numeric indirect approach is considered, which is based on Newton Affine Invariant scheme for solving boundary value problem, which fits the considered systems and is being different technique of solution seeking. Following scheme can find also a local optimum.
The evolutionary strategies algorithm was used to solve the optimal control problem, but as a direct method. In the paper [6] the control function was discretized and every part of it was optimized via evolutionary strategies algorithm. That means, that there a as many optimization variables, as many discrete points are approximating the control.
The method of semi-analytical and semi-numerical optimal control problem solving is considered. The first part of the method is based on the Pontryagin’s maximum principle [7], after determination of the Hamiltonian, the system with co-state variables can be used. For the new system that is a transformation of the initial problem it becomes possible to reduce the optimal control problem to extremum seeking on a real vectors’ field.
Let the system be described with nonlinear differential equation
dx = f (x, u, t), (1)
dt
where f (•):Rn x R x R + ^ Rn is a vector function of its
arguments; x e Rn is a vector of system state; u e R is a continuous control function; n is the system dimension.
We need to find a control function u(t) that would bring the system from the initial point x(0) = x° to the
end point x(T) = x within a finite time T. Also, the control function and the system state are the functions that deliver the extremum to the given functional
I(x, u) = | F(x, u)dt ^ extr.
(2)
To find the solution for the variational problem with constrains one can use a common technique. First of all, one should find the Hamiltonian [7], that is defined by the following equation:
H(x, u, t) = -F(x,u) + p • f (x,u, t), (3)
therefore the system with co-state variables p can be determined with equations
dx N dp -dH
— = j (x, u, t), — =--------------
dt dt dx
(4)
The given system (4) is completed with system state and co-state variables’ starting points x(0) = x0 and p(0) = p0, respectively. It actually means, that the control function u(t) can be determined by choosing different values for starting point for the co-state variables, p0 . To close up the system and to determine the control function as the function of state or co-state variables, the condition of Hamilton stationary is used,
dH
du
= 0.
(5)
Since the differentiation of the symbolic expression is not a common problem, the forming of the system in the current study was not made automatically. Anyway, some mathematical softwares are able to operate with analytical problems, simplify expressions and differentiate them.
That is why it is seems promising that the general method can be realized in one programm in the future.
The structure of the control function is determined by equation (5). After using of the transversability conditions, by changing the starting point of the co-state variables, we change the control function and the solution of the optimal control problem.
Normally, it means that the proper vector of the costate variables initial point, which provides the condition x(T) = x* would give us the solution for the whole problem, since the functional (2) and the differential equation (1) are forming the system (4). Initial point is the real vector and it could be searched with some optimization technique.
Since the main problem is reduced to optimization problem on the field Rn, let the x(t), p(t)| p0 be the
solution for the system (4) in case of p(0) = p0 is being the starting point fort he co-state variables. Now it is possible to define a criterion,
K (p0) =
:- %T )|
^ min.
p0
(6)
so the fitness function is a mapping: Rn ^ [0,1]. The
greater fitness is, the better current solution is.
To prove the high complexity of the optimization problem let the system be defined by equation ^ln( xj) + cos( x0)N t • sin( x1) + u(t)
'0 ( 2' x(T) =
f (x, t, u) =
T = 1.
x0 =
(7)
and the integrand function for the functional of the optimal control problem
F(x,u) = u ^ min.
(8)
is considered. Then, it is necessary to define the extended system (4),
( ln( x1) + cos( x0) ^
Fp (x, p, t) =
p,
t • sin( x1) + -2 sin(x0) • P0 -1 • cos(x0) • P1
- P0
The proposed criterion generally is multimodal, complex function of its arguments. It is not known, of course, where any extremum is located. Moreover, if the initial system (1) or functional (2) that forms the Hamiltonian (3) is nonlinear, so there is no analytical solution for the given criterion (6) and it can be evaluated only numerically.
The given criterion (6) is being transformed into fitness function for the evolutionary algorithms
f'tmss( p0)=I+kT) '
since we closed up the system with condition (5),
dH „ p,
-----= 0 ^ u(t) = — .
du 2
Now, having the system with co-state variables and the structure of the control function it is possible to form an optimization problem for initial point of co-state variables, so the end point of the system state would be achieved at time T .
As one can see, the nonlinear differential equation consists of logarithm function, trigonometric functions and the system itself is nonstationary.
Kes (P0)
Fig. 1. The surface of the criterion (6) for optimal control problem (7)-(8)
K„ (P0)
Fig. 2. The surface of the criterion (6) for optimal control problem (9)—(10)
The mapping (6) for the given problem is shown on the fig. 1. The surface was made via evaluating numerically the nonlinear differential equation for extended system, varying the initial point of the co-state variables. As it can be shown on the current surface some extremum problems that are reduced from the optimal control problems have a lot of local maximums that are less than 1 and so do not satisfy two-point problem, and among them there could be closed sets or distinct points, that delivers extremum to criterion (6) and it equals 1.
Let us describe the next optimal control problem for the plant with inverted pendulum, which movement is determined with system of nonlinear differential equation
f (X, t, u) =
- x1 + sin( x0 ) + u (t) • cos( x0 )
r=5, X"=11) • /=f0
and functional
F(x,u) = u + x0 ^ min.
(9)
(10)
u, xo
is being considered. Then, it is necessary to define the extended system (4),
Fp (x, p, t) =
- x1 + sin( x0) +
Pi • cos (Xo)
-2 • Xo + Pi •cos(Xo) -
Pi • sin(2• Xo)
- Pi + Po
because we closed up the system with condition (5),
— = 0 ^ u(,) = Pl •cos(Xo>.
du 2
The mapping (6) for current problem is shown on the fig. 2.
Every optimal control problem reduced to extremum problem for vector function with unknown characteristics and behavior of the criterion.
As it can be seen on figures, the problem is complex. Moreover, there is no any information about the location of the extremum.
To sum up, seeking for the solution of the reduced problem, in general, is associated with the global optimization technique that works on the vector field with no constraints. Anyway, it is possible to use the optimization techniques, which works on the compact, but then the special procedure of extending the compact or switching to different one should be implemented.
Since many optimization techniques are suitable for the considered problem and deal with its features, it was suggested to compare these well-known techniques: evolutionary strategies, differential evolution and particle swarm optimization.
The main principle of evolutionary strategies (ES) is described in [8]. To provide the efficiency growth evolutionary strategies algorithm was modified. Another modification of the evolutionary strategies algorithm suggested is the CMA-ES, which is described in [9] and uses the covariance matrix adaptation. As the next
optimization technique the differential evolution (DE) algorithm is suggested, which main principle is described in [10]. The last considered method of extremum seeking is the partial swarm optimization (PSO), which is described in [11].
The random coordinate-wise real-valued genes optimization (LO) has been implemented for the algorithms performance improvement. The optimization is fulfilled in the following way. For every N2 randomly
chosen real-valued genes for N1 randomly chosen
individuals N3 steps in random direction with step
size ht are executed.
For problems that were described above: (7)-(8), (9)-(10), we set the maximum numbers of criterion evaluation to 8000, and tested different setting of the given algorithms. The number of algorithms’ iterations and the size of populations were varied too: 1600 and 5, 800 and
10, 400 and 20, 200 and 40, 100 and 80, respectively.
Since the proposed optimization techniques have different natures and their settings were varied regarding to the features of algorithms. For the evolutionary strategies techniques the selection was varied: proportional, rank, tournament; crossover operator was varied: intermediate, weighted intermediate and discrete; mutation: classical and modified, with mutation
probability equals to 1/k. For differential evolution technique the settings were chosen due to
recommendation given: Cr = 0,5 and
F e{at = 0,2 • i: i < 10, i e N} . For the particle swarm optimization settings were taken from the followig sets: rae{0,5 • i: i < 4, i e N} , q1 2 e {0,4 • i: i < 6, i e N} .The initial population was randomly generated, oPi e N(0,10), sPi e N(0,1) and vi e N(0,1). For the ES+LO technique, the settings for LO and the number of individuals and populations were chosen as the numbers, which sum is equal to maximum number evaluation. The settings for the CMA-ES algorithm were set as it is recommended in reference, for this technique the only numbers of populations and individuals were varied.
It is important to highlight the fact of stagnation of some algorithms. For the problem (9)-(10) the average of the fitness function for every population and the fitness of the best individual are shown on the fig. 3. These curves are the averaging of the presented variables after 20 restarts of algorithms with the same settings. The horizontal axis is the number of iteration for every algorithm; vertical axis is the fitness function value. As one can see, there are many iterations made by the algorithm that gives no solution improvement. Moreover, the scouting of the surface does not give any sufficient result.
So the first problem is related with the stagnation of the algorithms, and the second important thing is reaching the global extremum point, for dissipative fitness functions. Let us compare the efficiency of algorithms for two different problems (7)-(8) and (9)-(10). In the table 1 the average values of the fitness function for the found solution are presented.
In table 2 the probability estimation of
1 - fitness (op*) < 0.05 is considered, since that
algorithms can reach local optimum point for (9)-(10), it is easier to add one more characteristic - chances to reach the global optimum.
1----------------,---------------,---------------,---------------
0.9-f -
0. J
0.6 j/ -
°-5fi; /-
a4| /-
°3^ /
02{/
1/
0.1 j -
0I---------------1---------------1---------------1---------------
0 50 100 1 50 200
Fig. 3. Behaviour of average fitness function value and fitness of the best individual. Thick lines are the average fitness and thin lines are the best fitness
Table 1
Average values of the fitness function for different techniques with the most efficient settings
Algorithm
Problem ES DE PSO CMA-ES ES+LO
(7)-(8) 0,97 0,98 0,95 0,99 0,99
(9)-(10) 0,93 0,95 0,96 0,94 0,97
Table 2
The estimation of probability to return solution that is close to the global optimum
Algorithm
ES DE PSO CMA-ES ES+LO
Problem 0,3 0,5 0,6 0,35 0,65
One can put forward a hypothesis about improvement of the algorithm performance via implementing the
if /' * ’
r
$
i:
i:
[/
critique, which aim would be detecting the stagnation of an algorithm. If for chosen numer of populations there is no improvement of the best solution the population is being regenerated.
In the current study the following scheme was investigated:
- If last ntail populations best solution is not
changing, i. e. max(tail) - min(tail) > boardertail, where tail, \tail\ = ntail is the vector of best solutions and boardertail is critique parameter - algorithm is restarting;
- If algorithm is restarting, then the database is extended with new solution found: database = database|^| best _ solution ;
- If the algorithm is not restarting and
database ^ 0 , then if
||best _ set - databasei || < boarderbest, i = 1,\database\ , where boarderbest is another critique parameter;
- There is also a possibility to change the expected
value of the initial population distribution, i. e. center = center + N(0, Grestart) ■
In the current investigation, all the algorithms’ settings and population, individual numbers were varied. Due to given problems, the hybrid evolutionary strategies algorithm was the most effective in searching the extremum and reliable in case of the problems with objective function that have a surface as it is shown on figure 2. After all the runs, the best settings were estimated as following ones: 20 individuals for 200 populations, tournament selection (10 %), discrete crossover, modified mutation (mutation probability 0,75) and N1 = N2 = NI /2, where NI is population size, N3 = 0.1, with evaluations number limitation equal to 8000.
To investigate the algorithm performance two different schemes were examined: with changing of the expected value for the initial population and without it. The size of tail was varied from 5 to 20 and boarder was
0,005, 0,01, 0,05, 0,1. The examination was done on (9)-(10) problem. For every setting of the critique, the global solution was found at least once. It means that the restarting sufficienly increases efficiency oft he algorithm.
Fig. 4. Relation between the number of restarts (left) and the number of global optimum points found (right) for different boarders and tail sizes
If we compare two schemes, the sum of cases when algorithm founded global optimum point for different settings is 512 and 476 for algorithm with changing of expected value and not, respectively. The number of total runs was 796 and 819, the number of populations that was aborted because of their best solution being close to one from the set was 534 and 516, respectively. The last fact means that the checking the distance between point that was already suspected to be «final» improves the performance as well and prevent from extra evaluations. On the fig. 4 the relation between increasing of the tail size and number of restarts for different boarders is on the left diagram, and number of global optimum points found for different boarder values and increasing of the tail size is on the right diagram.As it can be seen on the figures, there is nonlinear influence of increasing the size of the tail, but the size of the tail does change the algorithm efficiency, as well as the boarder size. The further study will be focused on different schemes of critique’s action and detection and ways to adapt the new parameters. Anyway, even now, with the same number of function evaluation we increased the estimated probability to find the desired solution from 0.65 to 1.
References
1. Popescu M., Dumitrache M. 2005: On the optimal control of affine nonlinear systems. Mathematical Problems in Engineering (4): p. 465-475.
2. Chen W. H., Balance D. J., Gawthrop P. J., 2003: Optimal control of nonlinear systems: a predictive control approach. Automatica 39(4): p. 633-641.
3. Primbs J. A., Nevistic V., Doyle J. C., 1999: Nonlinear optimal control: a control Lyapunov function and receding horizon perspective. Asian Journal of Control, vol. 1, no. 1: p. 14-24.
4. Rao A. V., 2009: Survey of Numerical methods for Optimal Control, AAS/AIAA Astrodynamics Specialist Conference, AAS Paper 09-334, Pittsburg, PA.
5. Bertolazzi E., Biral F., Lio M., 2005: Symbolic-Numeric Indirect Method for Solving Optimal Control Problems for Large Multibody Systems: The Time-Optimal Racing Vehicle Example. Multibody System Dynamics, vol. 13, no. 2, p. 233-252.
6. Cruz P., Torres D., 2007, Evolution strategies in optimization problems, Proc. Estonian Acad. Sci. Phys. Math., 56, 4, p. 299-309
7. Kirk D. E., 1970: Optimal Control Theory: An Introduction. Englewood Cliffs, NJ: Prentice-Hall.
8. Schwefel H.-P., 1995: Evolution and Optimum Seeking. New York: Wiley & Sons.
9. Hansen N., 2006: The CMA evolution strategy: a comparing review. Towards a new evolutionary computation. Advances on estimation of distribution algorithms, Springer, p. 1769-1776.
10. Storn R., Price K., 1997. Differential Evolution -A Simple and E?cient Heuristic for Global Optimization over Continuous Spaces, Journal of Global Optimization,
11, p. 341-359.
11. Kennedy, J., Eberhart, R., 2001. Swarm
Intelligence. Morgan Kaufmann Publishers, Inc., San Francisco, CA.
© Pbi^HKOB H. C., 2013
УДК 519.6
INTELLIGENT INFORMATION TECHNOLOGIES IN TIME SERIES FORECASTING
E. S. Semenkin, A. A. Shabalov
Siberian State Aerospace University named after academician M. F. Reshetnev 31, Krasnoyarsky Rabochy Av., Krasnoyarsk, 660014, Russian Federation E-mail: [email protected], [email protected]
Intelligent information technologies enable to solve complex data mining problems in various domains of human activity. In this paper such popular techniques as artificial neural networks, fuzzy rule based systems and neuro-fuzzy systems are considered. A genetic programming algorithm is used for building intelligent systems ensembles in order to improve the performance and reliability of decision making. The methods proposed are applied to time series prediction task. The results obtained are compared to other state-of-the-art time series forecasting techniques.
Keywords: artificial neural networks, fuzzy rule based systems, neuro-fuzzy systems, evolutionary algorithms, ensembles of intelligent systems.
ИНТЕЛЛЕКТУАЛЬНЫЕ ИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИ В ПРОГНОЗИРОВАНИИ ВРЕМЕННЫХ РЯДОВ
Е. С. Семенкин, А. А. Шабалов
Сибирский государственный аэрокосмический университет имени академика М. Ф. Решетнева Российская Федерация, 660014, Красноярск, просп. им. газ. «Красноярский рабочий», 31 E-mail: [email protected], [email protected]
