CC BY
0
MSC 93E12
DOI: 10.14529/ mmp240308
APPLICATION OF PARTICLE SWARM OPTIMIZATION FOR PARAMETER ESTIMATION OF THE LOGISTIC MAP
A.S. Sheludko, South Ural State University, Chelyabinsk, Russian Federation, [email protected]
This work considers the parameter estimation from measurements of the logistic map. The problem is solved in the context of optimization approach, which assumes minimization of a cost function that measures the difference between the time series obtained from the model equation and measurements. Complex dynamics of the logistic map leads to the multi-extremal optimization problem. It requires using appropriate computational techniques. This work presents the application of particle swarm optimization in searching for the global minimum of the cost function.
Keywords: logistic map; parameter estimation; cost function; particle swarm optimization.
Introduction
One-dimensional nonlinear maps are a class of models that describe the dynamics of time series in many fields. The general form of these models is
where xk is the state variable, A is the model parameter, and k is the time step. For certain values of the parameter A the map (1) can generate time series x0, xi, x2,... with complex chaotic dynamics [1,2].
According to Devaney's definition [1], a one-dimensional map f : X ^ X is chaotic on X if f satisfies the following conditions:
1. f has sensitive dependence on initial conditions: there exists 8 > 0 such that, for any x £ X and any e > 0, there exists y £ X such that |x — y| < e and |fk(x) — fk(y)| > 8 for some k > 0.
2. f is topologically transitive: for any pair of open sets U, V C X there exists k > 0 such that fk(U) n V = 0.
3. Periodic points of the map f are dense in X: there exists at least one periodic point in any neighborhood of any x £ X.
One of the famous examples is the logistic map, which is given by
The logistic map is chaotic for A £ (3, 569945...; 4] and xo £ (0; 1). Complex dynamics of the chaotic maps can be characterized by bifurcation diagram, Lyapunov exponent, autocorrelation function, power spectrum [2, 3]. From the computational viewpoint, complex dynamics leads to difficulties in parameter estimation due to the multi-extremal optimization problem [4].
One of the most common approaches for parameter estimation from measurements is the least squares method (LSM) and its modifications [4-6]. The LSM is based on the
xk = f (xk-i, A),
(1)
xk = Axk-i(1 — xk-i)-
(2)
minimization of a cost function that measures the similarity between the data obtained from the model equation (1) and measurements. The goal is to find the global minimum of the cost function. In this context, many computational techniques from the area of artificial intelligence [7] were developed in the past decades, such as chaotic ant swarm [8], particle swarm optimization [9], evolutionary programming [10], differential evolution [11], gravitational search [12], artificial bee colony [13].
Note that, besides the LSM, there are other approaches, such as recursive estimation using the modifications of the Kalman filter for nonlinear systems [4], combined state and parameter estimation [14]. Some approaches use other forms of the cost function [15].
This work is focused on the application of particle swarm optimization (PSO). The algorithm was developed in the area of swarm intelligence [16] and has become one of the most widely used optimization techniques [17]. PSO searches for the global minimum of the cost function using a swarm of particles. Evolution of the swarm is described by the iterative procedure, which has only a few control parameters, simple implementation and low computational cost.
In the past decades, PSO was successfully applied for parameter estimation of different types of chaotic system [9,18,19]. It also has special modifications for chaotic systems [20-22]. In [23,24] PSO is combined with the local search algorithms. In [25] PSO is used for real-time estimation.
There are not so many studies, which present the application of PSO for parameter estimation of one-dimensional chaotic maps, and this topic is studied in the presented work. This work considers the parameter estimation from measurements of the logistic map. The main goal of this research is to compare the estimation results for different combinations of the control parameters.
1. Cost Function
Consider the parameter estimation problem for a chaotic map (1). It is assumed that the parameter A is unknown and the available data is given by
Vk = Xk + vk, k = 1, 2,..., N, (3)
where yk is the measurement, vk is an additive error, and N is the number of measurements. The problem is to estimate the unknown parameter A from measurements (3). In this problem statement, the initial condition x0 is also unknown, since it defines the value of the state variable xk at time step k:
xk = fk(x0, A) = /(/(/■■ ./Or o, A) ...))•
k
Application of the LSM leads to the minimization of the cost function defined as
F(xo, A) = (vi - f (xo, A))2 + (V2 - f (f (xo, A), A))2 + ... =
N (4)
= £ (Vk - fk(xo, A))2. 1 J
k=1
Fig. 1 shows the cost function for parameter estimation of the logistic map (2). In this example, the true value of the parameter is A* = 3, 8, the initial condition is x* = 0, 3,
and the number of measurements is N = 30. The mesh with steps Ax = 10-2, AA = 10-2 is used to plot the surface shown in Fig. 1. It seems visually that the cost function is not smooth and has only a few local minima, which are candidates for the global best. In fact, the function (4) is differentiable, and there are many local minima near the true solution. This fact is clearly seen in Fig. 2, where the mesh steps Ax = 10-5, AA = 10-6. Note that in the presented example there are no abnormal changes in the cost function due to the small number of measurements.
There are a few reasons why the cost function becomes multi-extremal:
1. Chaotic map is very sensitive to the initial condition and parameter (due to this property, the initial value approach may also lead to an incorrect estimation [7]).
2. Dynamics of the state variable and measurement errors is very similar.
3. For a small number N of measurements, the difference in the cost function value at the global and some local minima is very small.
In order to use PSO, consider the cost function (4) in general form. Let
p = [ xo A ]T
be the vector of the unknown variables. In this way, the cost function can be written as
N
F(P) = E (yk — fk(p))2 .
k=i
2. Particle Swarm Optimization
Consider the minimization problem of the cost function F(p), where p £ Rn is the vector of the unknown variables, and n is the dimension of the search space. PSO searches for the global minimum using a swarm of m particles that updates from iteration to iteration. Each particle is described by its position p^- £ Rn and individual velocity Vjj £ Rn, where i is the particle index, j is the iteration index. Each particle moves in the search space to find a better solution and saves the best position pj £ Rn, which can be interpreted as a possible local minima. The best position p £ Rn among all particles in the swarm is considered as a possible global minimum of the cost function. The idea of PSO is shown in Fig. 2and the following steps represent the evolution of the swarm. Step 1. Updating the velocity of each particle:
Vij = Wj Vij-i + Ciri(pj — pij-i) + C2f2 (p — pij-i),
where wj > 0 is the inertia weight, ci, c2 > 0 are acceleration coefficients, and ri, r2 £ [0; 1] are uniformly distributed random numbers (the values are generated by the function rand of Matlab). Usually the inertia weight decreases linearly as follows [9,19]:
__wmax wmin
Wj = Wmax X J,
where wmax,wmin are the maximum and minimum values, respectively. Step 2. Each particle moves to the new position:
pjj = pij-i + vij •
F
Fig. 1. Cost function for parameter estimation of the logistic map
F
Fig. 2. Cost function near the true solution
Step 3. If the new position is better than the best position, then the last one is updated:
F (p i,j ) < F (pi ) ^ pi = p i,j.
Step 4. If the best position of the particle was updated in the previous step, then try to update the global best position:
F (pi ) < F (p) ^ p = pi.
The listed steps represent the evolution of the swarm from iteration j — 1 to iteration j, and they are repeated until the maximum number M of iterations is reached.
Fig. 3. Calculation of the particle position
3. Numerical Simulations
Consider the parameter estimation from measurements of the logistic map (2). In the test problem, the true value of the parameter is A* = 3, 8, and the initial condition is = 0, 3. The number of measurements is N = 30. The measurement errors are pseudorandom numbers with normal distribution, zero mean and standard deviation a = 0, 05. The values of vk are generated by the function randn of Matlab. The corresponding cost function is shown above in Fig. 1.
In the simulations, four different combinations of the control parameters are studied. The control parameters are the acceleration coefficients ci, c2 and the maximum and minimum values wmaxof the inertia weight. The studied combinations are given in Table 1. The first combination is recommended by many authors [9,19]. In all cases the number of particles is m = 100, and the maximum number of iterations is M = 200. The initial swarm is generated randomly in the search set P = X0 x A, where X0 = [0; 0, 5] and A = [3, 5; 4] are the intervals of possible values of the initial condition x0 and parameter A, respectively. The estimation results are presented in Table 2 and Table 3.
Table 1
Combinations of control parameters of PSO
Control parameter Combination
1 2 3 4
Cl 2 0,5 0,2 0,05
с2 2 0,5 0,2 0,05
Wmax 0,8 1 1 1
Wmin 0,4 0,9 0,9 0,96
Table 2
Comparison of the estimates of the initial condition x0
Combination of control parameters
3 1 2 3 4
50 0,28463885 0,26943005 0,29992404 0,29999780
100 0,29063151 0,26936041 0,29928637 0,29895155
150 0,29970289 0,30074711 0,29928637 0,29930475
200 0,29977539 0,30074256 0,30075355 0,29930475
Table 3
Comparison of the estimates of the parameter A
Combination of control parameters
3 1 2 3 4
50 3,90726872 3,80379671 3,80363632 3,80451150
100 3,86885450 3,80711139 3,80307440 3,80635968
150 3,80887617 3,79946499 3,80307440 3,80298607
200 3,80792517 3,79879552 3,79940143 3,80298607
Fig. 4 shows the evolution of the cost function value through iterations. In all cases the best cost function values belong to the interval [0, 08; 0,12], and the final estimation errors have the same order of precision:
10-4 < |x0 - X0| < 10
-3
10
-3
<
A* - A
< 10
-2
Thus, all considered combinations of the control parameters provide similar results.
3 4
Fig. 4. Evolution of the cost function value through iterations
3.82
3.81
3.8
3.79
3.78
0.28 0.29 0.3 0.31 0.32 *
3.82
3.81
3.8
3.79
3.78
0.28 0.29 0.3 0.31 0.32 *
34 Fig. 5. Comparison of the trajectories of the swarm center. Each figure shows the final part of the trajectory: 150 < j < 200 for combinations 1 and 2, 100 < j < 200 for combinations 3 and 4
1
2
1
2
However, for each combination, the searching for the global minimum has different dynamics. To show this difference, consider the motion of the swarm center:
Comparison of the trajectories is shown in Fig. 5. The first trajectory is very irregular, and the other have a spiral-like shape. In addition, decreasing of the acceleration coefficients leads to smoothing of the trajectory. Such trajectory of searching increases the chance to find the global minimum. It is promising since the cost function has many local minima near the true solution. Note, that the obtained results are valid not only for the presented simulations.
Conclusion
This work considers the parameter estimation from measurements of the logistic map. The most common solution technique for this problem is the LSM. The main difficulty of the LSM is that the cost function has a large number of local minima. This multi-extremal optimization problem was solved using PSO. The algorithm has only a few control parameters, simple implementation and low computational cost. The presented study proposes values of the control parameters that are different from the recommended ones. This provides searching for the global minimum by taking into account the properties of the cost function. Further research will be related to the comparison of PSO with other techniques.
References
1. Devaney R.L. An Introduction to Chaotic Dynamical Systems. New York, Addison-Wesley,
2. Sprott J.C. Chaos and Time-Series Analysis. Oxford, Oxford University Press, 2003.
3. Anishchenko V.S., Astakhov V., Neiman A., Vadivasova T., Schimansky-Geier L. Nonlinear Dynamics of Chaotic and Stochastic Systems. Springer, 2007. DOI: 10.1007/978-3-540-38168-6
4. Voss H.U., Timmer J., Kurths J. Nonlinear Dynamical System Identification from Uncertain and Indirect Measurements. International Journal of Bifurcation and Chaos, 2004, vol. 14, no. 6, pp. 1905-1933. DOI: 10.1142/S0218127404010345
5. Bezruchko B.P., Smirnov D.A., Sysoev I.V. Identification of Chaotic Systems with Hidden Variables (Modified Bock's Algorithm). Chaos, Solitons and Fractals, 2006, vol. 29, no. 1, pp. 82-90. DOI: 10.1016/j.chaos.2005.08.204
6. Smirnov D.A., Vlaskin V.S., Ponomarenko V.I. Estimation of Parameters in One-Dimensional Maps from Noisy Chaotic Time Series. Physics Letters A, 2005, vol. 336, no. 6, pp. 448-458. DOI: 10.1016/j.physleta.2004.12.092
7. Quaranta G., Lacarbonara W., Masri S.F. A Review on Computational Intelligence for Identification of Nonlinear Dynamical Systems. Nonlinear Dynamics, 2020, vol. 99, pp. 1709-1761. DOI: 10.1007/s11071-019-05430-7
8. Li Lixiang, Yang Yixian, Peng Haipeng, Wang Xiangdong. Parameters Identification of Chaotic Systems via Chaotic Ant Swarm. Chaos, Solitons and Fractals, 2006, vol. 28, no. 5, pp. 1204-1211. DOI: 10.1016/j.chaos.2005.04.110
i= 1
1989.
9. He Qie, Wang Ling, Liu Bo. Parameter Estimation for Chaotic Systems by Particle Swarm Optimization. Chaos, Solitons and Fractals, 2007, vol. 32, no. 2, pp. 654-661. DOI: 10.1016/j.chaos.2006.03.079
10. Chang Jen-Fuh, Yang Yi-Sung, Liao Teh-Lu, Yan Jun-Juh. Parameter Identification of Chaotic Systems Using Evolutionary Programming Approach. Expert Systems with Applications, 2008, vol. 35, no. 4, pp. 2074-2079. DOI: 10.1016/j.eswa.2007.09.021
11. Peng Bo, Liu Bo, Zhang Fu-Yi, Wang Ling. Differential Evolution Algorithm-Based Parameter Estimation for Chaotic Systems. Chaos, Solitons and Fractals, 2009, vol. 39, no. 5, pp. 2110-2119. DOI: 10.1016/j.chaos.2007.06.084
12. Li Chaoshun, Zhou Jianzhong, Xiao Jian, Xiao Han. Parameters Identification of Chaotic System by Chaotic Gravitational Search Algorithm. Chaos, Solitons and Fractals, 2012, vol. 45, no. 4, pp. 539-547. DOI: 10.1016/j.chaos.2012.02.005
13. Gu Wenjuan, Yu Yongguang, Hu Wei. Parameter Estimation of Unknown Fractional-Order Memristor-Based Chaotic Systems by a Hybrid Artificial Bee Colony Algorithm Combined with Differential Evolution. Nonlinear Dynamics, 2015, vol. 84, pp. 779-795. DOI: 10.1007/ s11071-015-2527-x
14. Nakamura T., Hirata Y., Judd K., Kilminster D. Improved Parameter Estimation from Noisy Time Series for Nonlinear Dynamical Systems. International Journal of Bifurcation and Chaos, 2007, vol. 17, no. 5, pp. 1741-1752. DOI: 10.1142/S021812740701804X
15. Jafari S., Sprott J.C., Pham V.-T., Golpayegani S.M.R.H., Jafari A.H. A New Cost Function for Parameter Estimation of Chaotic Systems Using Return Maps as Fingerprints. International Journal of Bifurcation and Chaos, 2014, vol. 24, no. 10. DOI: 10.1142/S021812741450134X
16. Kennedy J., Eberhart R.C., Shi Yuhui. Swarm Intelligence. San Francisco, Morgan Kaufmann, 2001.
17. Zhang Yudong, Wang Shuihua, Ji Genlin. A Comprehensive Survey on Particle Swarm Optimization Algorithm and its Applications. Mathematical Problems in Engineering, 2015, article ID 931256. DOI: 10.1155/2015/931256
18. Gao Fei, Lee Ju-Jang, Li Zhuoqiu, Tong Hengqing, Lu Xiaohong. Parameter Estimation for Chaotic System with Initial Random Noises by Particle Swarm Optimization. Chaos, Solitons and Fractals, 2009, vol. 42, no. 2, pp. 1262-1291. DOI: 10.1016/j.chaos.2009.03.074
19. Tang Yinggan, Guan Xinping. Parameter Estimation for Time-Delay Chaotic System by Particle Swarm Optimization. Chaos, Solitons and Fractals, 2009, vol. 40, no. 3, pp. 1391-1398. DOI: 10.1016/j.chaos.2007.09.055
20. Alatas B., Akin E. Chaotically Encoded Particle Swarm Optimization Algorithm and Its Applications. Chaos, Solitons and Fractals, 2009, vol. 41, no. 2, pp. 939-950. DOI: 10.1016/j.chaos.2008.04.024
21. Jiang Yunxiang, Lau Francis C.M., Wang Shiyuan, Tse Chi K. Parameter Identification of Chaotic Systems by a Novel Dual Particle Swarm Optimization. International Journal of Bifurcation and Chaos, 2016, vol. 26, no. 2. DOI: 10.1142/S0218127416500243
22. Sun Jun, Zhao Ji, Wu Xiaojun, Fang Wei, Cai Yujie, Xu Wenbo. Parameter Estimation for Chaotic Systems with a Drift Particle Swarm Optimization Method. Physics Letters A, 2010, vol. 374, no. 28, pp. 2816-2822. DOI: 10.1016/j.physleta.2010.04.071
23. Wang Ling, Xu Ye, Li Lingpo. Parameter Identification of Chaotic Systems by Hybrid Nelder-Mead Simplex Search and Differential Evolution Algorithm. Expert Systems with Applications, 2011, vol. 38, no. 4, pp. 3238-3245. DOI: 10.1016/j.eswa.2010.08.110
24. Zhang Hongjun, Li Baozhu, Zhang Jun, Qin Yuanhui, Feng Xiaoyi, Liu Bo. Parameter Estimation of Nonlinear Chaotic System by Improved TLBO Strategy. Soft Computing, 2015, vol. 20, pp. 4965-4980. DOI: 10.1007/s00500-015-1786-2
25. Modares H., Alfi A., Fateh M.-M. Parameter Identification of Chaotic Dynamic Systems through an Improved Particle Swarm Optimization. Expert Systems with Applications, 2010, vol. 37, no. 5, pp. 3714-3720. DOI: 10.1016/j.eswa.2009.11.054
Received May 7, 2024
УДК 519.7 DOI: 10.14529/mmp240308
ПРИМЕНЕНИЕ МЕТОДА РОЯ ЧАСТИЦ ДЛЯ ОЦЕНИВАНИЯ ПАРАМЕТРА КВАДРАТИЧНОГО ОТОБРАЖЕНИЯ
А.С. Шелудько, Южно-Уральский государственный университет, г. Челябинск, Российская Федерация
В работе рассматривается задача оценивания параметра квадратичного отображения по зашумленным измерениям. Решение задачи оценивания в рамках оптимизационного подхода основано на минимизации целевой функции, которая определяет разность между реализацией модельного уравнения и измерениями. Вследствие сложной динамики квадратичного отображения целевая функция является многоэкстремальной, что приводит к необходимости применения соответствующих вычислительных алгоритмов. В данной работе исследуется применение метода роя частиц для поиска глобального минимума целевой функции.
Ключевые слова: квадратичное отображение; оценивание параметров; целевая функция; метод роя частиц.
Антон Сергеевич Шелудько, старший преподаватель, кафедра «Прикладная математика и программирование:», Южно-Уральский государственный университет (г. Челябинск, Российская Федерация), [email protected].
Поступила в редакцию 7 мая 2024 г-
