CC BY
10
Journal of Siberian Federal University. Mathematics & Physics 2016, 9(1), 3—10
УДК 517.9
Bionic Algorithm with Biogeography Based Operator for Binary Optimization
Shakhnaz A. Akhmedova* Eugene S. Semenkin^
Department of System Analysis and Operation Research Reshetnev Siberian State Aerospace University Krasnoyarskiy Rabochiy, 31, Krasnoyarsk, 660037
Russia
Received 26.11.2015, received in revised form 30.12.2015, accepted 25.01.2016 The meta-heuristic called Co-Operation of Biology Related Algorithms (COBRA) developed earlier for solving real-valued optimization problems has also been modified for solving optimization problems with binary variables (COBRA-b). The algorithm COBRA-b is based on a collective work of five nature-inspired algorithms' binary modifications such as Particle Swarm Optimization (PSO), the Wolf Pack Search Algorithm (WPS), the Firefly Algorithm (FFA), the Cuckoo Search Algorithm (CSA) and Bat Algorithm (BA). Its usefulness and workability were demonstrated on various benchmarks, and COBRA-b also outperformed its components. But solving problems sometimes required too many function evaluations, so the COBRA-b migration operator was modified by integrating biogeography principles for the speedup of the algorithm. Numerical experiments showed that the new modification exhibits high performance and outperforms COBRA-b and therefore its components.
Keywords: biology inspired algorithms, biogeography, migration operator, optimization, binary variables. DOI: 10.17516/1997-1397-2016-9-1-3-10.
Introduction
The Particle Swarm Optimization Algorithm (PSO) [1], the Wolf Pack Search Algorithm (WPS) [2], the Firefly Algorithm (FFA) [3], the Cuckoo Search Algorithm (CSA) [4] and the Bat Algorithm (BA) [5] are biology-related optimization techniques originally developed for continuous variable space. These algorithms mimic the collective behaviour of corresponding animal groups that allows the global optima of real-valued functions of real variables to be found. The mentioned heuristics were used for the developing of a new collective nature-inspired meta-heuristic called Co-Operation of Biology Related Algorithms (COBRA) [6].
However many applied problems are defined in discrete valued spaces where the domain of the variables is finite. For this purpose the binary modification of COBRA namely COBRA-b was developed [7]. Experiments showed that the COBRA-b method works successfully and reliably but much slowlier than the original COBRA for the same problems with a smaller success rate obtained [8].
Biogeography-based optimization (BBO) [9] is an evolutionary algorithm that optimizes a function by stochastically and iteratively improving candidate solutions with regard to a given
* shahnaz@inbox.ru 1 eugenesemenkin@yandex.ru © Siberian Federal University. All rights reserved
Collective Migration
measure of quality, or fitness function. BBO was motivated by biogeography, which is the study of the distribution of biological species through time and space [10]. In this paper the BBO migration operator was implemented to COBRA-b in an attempt to decrease the number of function evaluations during the solving of a given optimization problem.
The rest of the paper is organized as follows. Section 1 briefly describes the method of COBRA and its modification COBRA-b. The biogeography-based optimization algorithm and how it was used for the meta-heuristic COBRA-b (for its migration operator) are presented in Section 2. In Section 3 the developed approach was applied to different optimization problems whose dimensions were also varied. In the Conclusion the results and directions for further research are discussed.
1. Co-Operation of Biology Related Algorithms
A collective meta-heuristic called Co-Operation of Biology Related Algorithms (COBRA) [6] was developed based on five well-known and similar bionic algorithms such as Particle Swarm Optimization (PSO) [1], Wolf Pack Search (WPS) [2], the Firefly Algorithm (FFA) [3], the Cuckoo Search Algorithm (CSA) [4] and the Bat Algorithm (BA) [5]. These nature-inspired algorithms were originally developed for solving real-parameter unconstrained optimization problems and imitate a nature process or the behaviour of an animal group. Namely, the Bat Algorithm is based on the echolocation behaviour of bats; the Cuckoo Search Algorithm was inspired by the obligate brood parasitism of some cuckoo species by laying their eggs in the nests of other host birds (of other species); the Firefly Algorithm was inspired by the flashing behaviour of fireflies; and the Wolf Pack Search imitates wolves' hunting process.
A precondition for the new algorithm was the fact that one cannot say which approach is the most appropriate for the given function and the given number of variables. More specifically on the basis of investigation into the effectiveness of these five optimization methods, it was established that the best results were obtained by different methods for different problems and for different dimensions; in some cases the best algorithm differs even for the same test problem if the dimension varies because each strategy has not only advantages but also disadvantages.
The meta-heuristic approach COBRA combines the major advantages of the bionic algorithms listed above. Its basic idea consists in generating five populations (precisely one population for each algorithm) which are then executed in parallel cooperating with each other.
The algorithm proposed in [6] is a self-tuning meta-heuristic so there is no necessity to choose the population size for each algorithm. The number of individuals in the population of each component algorithm can increase or decrease depending on whether the fitness value improves on the current stage or not. If the fitness value does not improve during a given number of generations, then the size of all populations increases. And vice versa, if the fitness value constantly improves, then the size of all populations decreases. Also each population can "grow" by accepting individuals removed from other populations. The population "grows" only if its average fitness is better than the average fitness of all other populations. Thereby the "winner component algorithm" can be determined on each generation. The result of this kind of competition allows the presenting of the biggest resource (population size) to the most appropriate (in the current generation) algorithm.
The migration operator of the COBRA method involves communication between populations. Specifically, all populations communicate with each other by exchanging individuals in such a way that a part of the worst individuals of each population is replaced by the best individuals of
other populations. It brings up-to-date information on the best achievements to all component algorithms and prevents their preliminary convergence to its own local optimum, which improves the group performance of all component algorithms.
The performance of the COBRA algorithm was evaluated on the set of 28 benchmark problems from the CEC'2013 competition [11]. Experiments showed that COBRA works successfully and is reliable on this benchmark and demonstrates competitive behaviour. Results also showed that COBRA outperforms its component algorithms when the dimension grows and more complicated problems are solved [6].
As has already been mentioned, the COBRA approach was developed for solving real-valued optimization problems, but many applied problems are defined in discrete valued spaces where the domain of the variables is finite. So, COBRA-b, i.e. the modification of COBRA that can be used for solving optimization problems with binary variables was proposed in [8].
COBRA was adapted to search in binary spaces by applying a sigmoid transformation to the velocity component (PSO, BA) and coordinates (FFA, CSA, WPS) to squash them into a range [0,1] and force the component values of the positions of the particles to be 0's or 1's. The basic idea of this adaptation was taken from [12]; firstly it was used for the PSO algorithm. The sigmoid function is also given in [12].
Six test problems whose number of dimensions varied from 2 to 4 were used for the investigation into the effectiveness of COBRA-b. The mentioned benchmarks are real-parameter optimization problems therefore each real-valued variable was represented by a binary string with length 10. Therefore the number of binary variables varied from 20 to 40. Experiments showed that the binary modification of COBRA works successfully and that it is reliable but in some cases requires too many calculations. However, the COBRA-b approach can be recommended for solving optimization problems with binary representation of solutions.
2. Biogeography-based optimization
Biogeography-based optimization (BBO) is a new population-based evolutionary algorithm that was introduced in 2008 [9]. BBO algorithm is based on the theory of island biogeography [10]. Habitat, in biogeography, is the locality, site and particular type of local environment occupied by an organism, where the island is any area of suitable habitat surrounded by an expense of unsuitable habitat and is endowed with exceptionally rich reservoirs of endemic, exclusive, strange and relict species.
Biogeography-based optimization translates the natural distribution of species into a general problem solution. Each island represents one solution, where a good problem solution means that the island has lots of good biotic and abiotic factors, which attracts more species than the other islands. Each feature is called a suitability index variable SIV, which represents the independent variable of such a problem in BBO. As these features change, the island suitability index ISI changes too; thus in BBO, ISI is the dependent variable [9]. A problem with n-independent variables and fc-islands or individuals can be expressed as:
ISI., = f (SIVU SIV2,SIVn), i = 1, 2,...,k. (1)
The algorithm of BBO consists of two main stages, migration and mutation. In this paper only the migration operator which was used for the COBRA-b modification will be described.
The theory of island biogeography proposes that the number of inhabited species on an island is based on the dynamic between new immigrated species onto an island and the extinct species
from that island. Let's consider a migration model with a linear immigration rate A and emigration rate n, where they can be plotted as logistic, exponential or any proper function [13]. I and E are the maximum possible immigration and emigration rates, respectively. The maximum immigration rate I occurs when the island is empty of any species and thus it offers a maximum opportunity to the species on the other islands for immigrating and settling on it; whereas if the arrivals on that island increase, the opportunity for settling will decrease, which means that the immigration rate will also decrease. Also, as A decreases, the species density increases, and thus the predation, competition and parasitism factors will increase too; and as a result, the emigration rate ^ will increase and reach its maximum value E when A reaches its minimum value [14].
Let's assume that the recipient island i has S species with As and ns being the immigration and emigration rates at present of S species on that island. If the i-th island has lots of features, then lots of species will colonize it, which means that As becomes low and ns becomes high. Thus, the high ISI for island i represents a good solution, and vice versa for a poor solution which has a lack of feature diversity, and is reflected on the total available number of species; in this condition, As is high and ns is low.
The purpose of the migration process is to use high ISI islands as a source of modification to share their features with low ISI islands, so the poor solutions can be probabilistically enhanced and may become better than those good solutions.
The migration process of standard BBO can be described by the following scheme:
Let ISIi denote the i-th population member and contains n features; for each island ISIi (where i = 1, 2,...,k) do for each SIVs (where s = 1, 2,... ,n) do
Use Ai to probabilistically select the immigrating island ISIi if rand < Ai then for j = 1,... ,k do
Use Hj to probabilistically decide whether to emigrate to ISIi if ISIj is selected then
Randomly select an SIVi from ISIj Replace a random SIVs in ISIi with SIVl end end end end end
This migration operator was applied to the binary modification of COBRA. More specifically now populations do not communicate with each other by exchanging individuals in such a way that a part of the worst individuals of each population is replaced by the best individuals of other populations. In the new version of the algorithm COBRA-b each population's individuals can be updated by individuals of the other populations. However a certain number of individuals with the highest fitness value will not be changed during "migration" but can be used for updating other individuals.
3. Experimental results
Different real-parameter test optimization problems with the number of variables changed from 2 to 4 were used for the preliminary investigation of the binary versions of component algorithms (PSO, WPS, FFA, CSA and BA): Rosenbrock's function, Sphere function, Ackley's function, Griewank's function, Hyper-Ellipsoidal function and Rastrigin's function [15]. Then the developed meta-heuristic COBRA-b was used to solve them. Therefore each real-valued variable was represented by a binary string whose length was equal to 10. Consequently the number of variables while solving the above-mentioned test problems by the optimization method COBRA-b and the binary modifications of its components varied from 20 to 40.
For each problem and dimension the number of program runs was equal to 100 so the results in Tab. 1 were averaged by that number. The maximum number of function evaluations was equal to 10000; however, if the result obtained by the algorithm differed from the known optimum by less than 0.001, calculations were stopped.
Thus it was established that the meta-heuristic COBRA-b outperforms its components. Experimental results obtained by the algorithm COBRA-b are presented in Tab. 1 where D denotes dimensions and STD is the standard deviation.
Table 1. Results obtained by algorithm COBRA-b
Function D Average population size Average number of function evaluations Average function value STD
Sphere function 2 31 740 0.000182069 0.282119
3 68 3473 0.000188191 6.94652e-005
4 80 6730 0.00579879 0.0129499
Griewank's function 2 27 567 0.000236274 9.93911
3 30 775 0.000150127 8.85592e-005
4 32 916 0.000355086 0.0669021
Ackley's function 2 32 1439 0.00019874 2.82798
3 51 2046 0.00150713 0.222684
4 62 3030 0.00126295 0.0515319
Hyper- Ellipsoidal function 2 33 931 0.000209168 0.515766
3 32 868 0.000191162 1.67698
4 79 1710 0.000347666 0.306151
Rosenbrock's function 2 30 899 0.00032841 0.00046868
3 65 1332 0.000506847 0.00140048
4 160 2258 0.00411721 0.158903
Rastrigin's function 2 28 1734 180.0002 0.00018536
3 36 3294 169.801 0.169149
4 41 5462 159.2 0.279294
Then the same six optimization problems were solved by the new biogeography-based modification of COBRA-b. The experimental settings were also not changed. The obtained results averaged by the number of program runs are presented in Tab. 2 where D denotes dimensions and STD is the standard deviation.
As the Tables show, the modification of the migration operator of the COBRA-b approach allows better solutions with a smaller number of calculations to be found. The new version of
Table 2. Results obtained by biogeography-based modification of the algorithm COBRA-b
Function D Average population size Average number of function evaluations Average function value STD
Sphere function 2 40 116 1.10078e-005 3.05451e-005
3 41 435 0.000162824 0.001195637
4 43 555 4.12793e-005 0.000133099
Griewank's function 2 40 103 0.000260617 2.32735e-005
3 42 298 4.1847e-005 4.90123e-005
4 44 419 0.000658365 9.9126e-005
Ackley's function 2 42 1041 0.00275265 0.11245036
3 56 1861 0.00150873 0.3043927
4 66 2580 0.00283 0.465772932
Hyper- Ellipsoidal function 2 40 116 4.0362e-005 7.43944e-005
3 41 327 0.000330693 5.33858e-005
4 42 518 0.000734619 0.000147079
Rosenbrock's function 2 41 157 9.87119e-005 1.85875e-005
3 42 527 0.000644101 0.000376341
4 48 1082 0.00030445 6.7481e-005
Rastrigin's function 2 56 1554 180.002 0.029217289
3 72 2567 170.015 0.084792983
4 90 4583 160.023 0.122501061
the algorithm outperforms the original COBRA-b for all benchmarks by all criteria, namely the average number of function evaluations and the standard deviation of obtained results and also averaged over 100 program runs the best function value achieved during the work of the algorithm. Besides, the average population size for the new modification did not change significantly while solving test problems even when the number of variables was varied.
Conclusion
In this paper a new modification of an earlier proposed meta-heuristic called Co-Operation of Biology Related Algorithms is introduced for solving optimization problems with binary variables (COBRA-b) based on five nature-inspired algorithms such as Particle Swarm Optimization, Wolf Pack Search, the Firefly Algorithm, the Cuckoo Search Algorithm and the Bat Algorithm. The modification of the algorithm COBRA-b consisted in the implementation of a migration operator from a biogeography-based optimization algorithm instead of the simple exchange of individuals between populations. The main purpose was to lessen the number of function evaluations required for a solving optimization problem and therefore to add urgency to the work of the algorithm.
The new version of the COBRA-b approach was tested by using a set of six well-known benchmarks with differing numbers of variables. Experiments showed that proposed algorithm works successfully and is reliable on different benchmark problems and demonstrates competitive behaviour. Moreover, the new optimization technique demonstrated better results than original COBRA-b, so it outperformed not only the component algorithms but also the meta-heuristic COBRA-b.
Directions for future research are heterogeneous: improvement of the cooperation and competition scheme within the approach, extension of this potentially powerful optimization strategy to study real-parameter constrained and unconstrained optimization problems. As far as applications are concerned, the proposed technique can be used for the adjustment of the structure of a neural network for solving for example classification problems and there is the intention to modify the algorithm for tuning the neural network weight coefficients.
Research is performed with the financial support of the Ministry of Education and Science of the Russian Federation within the State Assignment for the Siberian State Aerospace University, project 2.1889.2014/K.
References
[1] J.Kennedy, R.Eberhart, Particle Swarm Optimization, Proceedings of International Conference on Neural networks IV, 1995, 1942-1948.
[2] Ch.Yang, X.Tu, J.Chen, Algorithm of Marriage in Honey Bees Optimization Based on the Wolf Pack Search, Proceedings of International Conference on Intelligent Pervasive Computing (IPC2007), 2007, 462-467.
[3] X.S.Yang, Firefly algorithms for multimodal optimization, Proceedings of 5th Symposium on Stochastic Algorithms, Foundations and Applications (SAGA 2009), 2009, 169-178.
[4] X.S.Yang, S.Deb, Cuckoo Search via Levy flights, Proceedings of World Congress on Nature and Biologically Inspired Computing (NaBic2009), 2009, pp. 210-214.
[5] X.S.Yang, A new metaheuristic bat-inspired algorithm, Nature Inspired Cooperative Strategies for Optimization, Studies in Computational Intelligence, 284(2010), 65-74.
[6] Sh.Akhmedova, E.Semenkin, Co-Operation of Biology Related Algorithms, Proceedings of the IEEE Congress on Evolutionary Computation (CEC'2013), 2013, 2207-2214.
[7] Sh.Akhmedova, E.Semenkin, Co-Operation of Biology Related Algorithms Meta-Heuristic in ANN-Based Classifiers Design, Proceedings of the IEEE World Congress on Computational Intelligence (WCCI'2014), 2014, 867-873.
[8] Sh.Akhmedova, E.Semenkin, New optimization metaheuristic based on co-operation of biology related algorithms, Vestnik. Bulletine of Siberian State Aerospace University, 50(2013), no. 4, 92-99.
[9] D.Simon, Biogeography-Based Optimization, IEEE Transactions on Evolutionary Computation, 12(2008), no. 6, 702-713.
[10] R.H.MacArthur, E.O.Wilson, An Equilibrium Theory of Insular Zoogeography, Journal of Evolution, 17(1963), no. 4, 373-387.
[11] J.J.Liang, B.Y.Qu, P.N.Suganthan, A.G.Hernandez-Diaz, Problem Definitions and Evaluation Criteria for the CEC 2013 Special Session on Real-Parameter Optimization, Technical Report, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China; and Technical Report, Nanyang Technological University, Singapore, 2012.
[12] J.Kennedy, R.C.Eberhart, A discrete binary version of the particle swarm algorithm, Proceedings of the World Multiconference on Systemics, Cybernetics and Informatics, 1997, 4104-4109.
[13] D.Simon, A Probabilistic Analysis of a Simplified Biogeography-Based Optimization Algorithm, Evolutionary Computation, (2011), 167-188.
[14] A.Alroomi, F.Albasri, J.Talaq, Performance comparison between the original forms of biogeography-based optimization algorithms, Proceedings of the Second International Conference on Advanced Information Technologies and Applications, 2013, 121-140.
[15] M.Molga, Cz.Smutnicki, Test functions for optimization need, 3 kwietnia, 2005.
Коллективный бионический алгоритм с биогеографическим оператором миграции для решения задач оптимизации с бинарными переменными
Шахназ А. Ахмедова Евгений С. Семенкин
Кооперативный бионический алгоритм оптимизации, Co-Operation of Biology Related Algorithms (COBRA), разработанный ранее для решения задач оптимизации с вещественными переменными, был модифицирован также для решения задач оптимизации с бинарными переменными (COBRA-b). Алгоритм COBRA-b основан на коллективной 'работе бинарных модификаций пяти бионических эвристик, а именно стайного алгоритма (Particle Swarm Optimization, PSO), алгоритма поиска стаей волков (Wolf Pack Search Algorithm, WPS), алгоритма светлячков (Firefly Algorithm, FFA), алгоритма поиска кукушек (Cuckoo Search Algorithm, CSA) и алгоритма летучих мышей (Bat Algorithm, BA). Работоспособность и целесообразность применения метода COBRA-b были продемонстрированы на различных тестовых задачах, COBRA-b превосходила по результатам свои алгоритмы-компоненты. Однако иногда для решения задач оптимизации требовалось слишком много вычислений целевой функции, поэтому был применен биогеографический оператор миграции для ускорения работы метода COBRA-b. Численные эксперименты показали, что новая модификация превосходит метод COBRA-b и, как следствие, его компоненты.
Ключевые слова: бионические алгоритмы, биогеография, оператор миграции, оптимизация, бинарные переменные.
