CC BY
45
Classifier Generating with Coevolutionary Algorithm for Strategy Adaptation // Proc. of 2011 IEEE Congress on Evolutionary Computation. New Orleans, LA, USA 2011.
7. Sergienko R. B., Semenkin, E. S. Competitive cooperation for strategy adaptation in coevolutionary genetic algorithm for constrained optimization // Proc. of 2010 IEEE Congress on Evolutionary Computation. 2010. P. 1626-1631.
8. UCI Machine Learning Repository. URL: http://kdd.ics.uci.edu/.
9. Schapire. R. The boosting approach to machine learning: An overview // MSRI Workshop on Nonlinear Estimation and Classification. Berkeley, CA. 2001.
10. Breiman, L. Arcing classifiers. In: The Annals of Statistics. 1998. Vol. 26. P. 801-849.
11. Ho T. K. The random subspace method for constructing decision forests // IEEE Trans. on Pattern Analysis and Machine Intelligence. 1998. Vol. 20. P. 832-844.
12. Zhuravlev J. I. An algebraic approach to recognition or classifications problems // Pattern Recognition and Image Analysis. 1998. Vol. 8, № 1. P. 59-100.
13. Voroncov V. V., Kanevsky, D. U. Cooperative coevolution algorithm ensemble learning // Tavricheskiy Vestnik Informatiki I Matematiki (Tavria's Gerald of Informatics and Mathematics). 2005. P. 51-66.
Р. Б. Сергиенко
ПРИМЕНЕНИЕ МЕТОДА ФОРМИРОВАНИЯ НЕЧЕТКОГО КЛАССИФИКАТОРА САМОНАСТРАИВАЮЩИМИСЯ КОЭВОЛЮЦИОННЫМИ АЛГОРИТМАМИ В ЗАДАЧЕ РАСПОЗНАВАНИЯ ГОВОРЯЩЕГО
Рассматривается задача распознавания говорящего. В качестве исходных данных взята задача «Японские гласные» из ПС!репозитория. Эта задача была решена с использованием нечеткого классификатора как метода классификации, способного извлекать причинно-следственные закономерности из исходных данных. Был применён новый метод формирования нечеткого классификатора, самонастраивающимися коэволюционными алгоритмами, а именно многошаговый метод формирования нечеткого классификатора, основанный на многократном повторении ранее разработанного метода формирования нечеткого классификатора. Были проведены численные исследования метода формирования нечеткого классификатора для различного числа говорящих и различного числа используемых нечетких правил. Метод показал приемлемую эффективность на тестовой выборке: от 0,985 для двух говорящих до 0,786 для девяти.
Ключевые слова: нечеткий классификатор, коэволюционный алгоритм, классификация, распознавание говорящего.
© Sergienko R. B., 2012
UDC 519.S
A. A. Shabalov
APPLICATION OF NEURO-FUZZY SYSTEMS IN BANK SCORING PROBLEMS*
As the generation of a neuro-fuzzy model from scratch by hand is complex and time consuming process, evolutionary computations are used for this. By virtue of genetic algorithms, the development of neuro-fuzzy systems is simplified and becomes automatic. The proposed scheme is applied to well-known Australian and German Credit Approval problems. Comparison of different algorithms is given.
Keywords: neuro-fuzzy modeling, evolutionary calculations, fuzzy systems, neural networks.
Neuro-fuzzy modeling is applied to soft computing paradigm. It combines the advantages of neural networks and fuzzy rule based systems.
While fuzzy systems implement effective approximate reasoning in uncertain environment, neural networks provide efficient learning algorithms from data. Meanwhile, neuro-fuzzy systems chiefly represent a knowledge base
with fuzzy rules and membership functions where neural network algorithms such are used for parameters learning.
Typically, the learning phase of neuro-fuzzy modeling consists of two stages. The first one is an unsupervised mode where any clustering algorithm could by applied for determination of initial size of rule base, i. e. number of rules.
**The study was supported by The Ministry of education and science of Russian Federation, project № 16.740.11.0742, 14.740.12.1341, 11.519.11.4002 and the Grant of the President of the Russian Federation MK-2835.2012.9.
Mamdani inference type
Rk
ai °i an a1 am ^m
Y
Input parameters
yv.
Y
Output parameters
Takagi-Sugeno inference type
Rk
___________Ж_______________
a1 °1 an °n V V m
i j
--------v--------
Input parameters
yv
"v--------
Output parameters
Fig. 1. Chromosome coding schemes
Comparison with different algorithms
Australian Data Set Proposed algorithm LR Bayesian RSM k-NN CART
0.8696 0.8696 0.8470 0.8660 0.8744 0.8986
C4.5 CCEL 2SGP GP+SRF Boosting Bagging
0.8986 0.7150 0.9027 0.8889 0.7600 0.8470
German Data Set Proposed algorithm LR Bayesian RSM k-NN CART
0.8700 0.7837 0.6790 0.7460 0.7565 0.7618
C4.5 CCEL 2SGP GP+SRF Boosting Bagging
0.7773 0.7151 0.8015 0.7834 0.7000 0.6840
Here a competitive learning with rival penalized mechanism is used. In comparison with other clustering techniques (such as k-mean, fuzzy k-means, conventional competitive learning) where a specific number of clusters is to be set, competitive learning with rival penalized mechanism requires a maximum number of clusters. During the learning procedure it will eliminate extra clusters that are beyond the universe of discourse. After such a stage the “rough” fuzzy rule base is established. The second stage (supervised mode) consists in tuning of parameters of membership functions. At this stage the learning capabilities of neural networks are applied. As a rule, a modified version of steepest descent method is used. The drawbacks of gradient based techniques are well known. They could be trapped in a local extremum due to a complex shape of objective function, go to stagnation or, otherwise, pulse. Under these considerations it is rational to apply evolutionary algorithms as well-established technique in global multiextremal optimization. Genetic algorithms are robust stochastic search procedure based on principles of natural evolution. It possesses flexible coding structure making it applicable to problems in different areas of human life. The next paragraph presents the proposed coding schemes of bit strings in genetics algorithms for automatic adjustment of membership function parameters.
Coding structure of neuro-fuzzy model parameters. There are three main evolutionary coding structures of fuzzy rule base while designing a system. In our case the Pittsburgh approach was implemented where each individual in population represents single rule base. The developed algorithmic scheme requires only settings parameters of a genetic algorithm to be set. As for member-
ship function, Gaussian membership functions were used. The coding schemes are given below (Fig. 1).
Application to bank credit assignment problems. There are two data sets concerning customer credit card applications. The first one (Australian data set) contains 690 instances and 14 attributes and the second one (German data set) - 1000 instances and 24 attributes. The sets are to be classified into two classes. 10 % of instances in every set were randomly picked out for test sample. In Table 1 the results obtained and comparison with other techniques are given. The results of other algorithms were found in [1] and [2].
As it can be seen from the table our proposed algorithm is comparable with well-known techniques and can be applied to many real-world applications in different areas of human life.
In this paper evolutionary tuning technique of a neuro-fuzzy system is described. The method proposed was applied to Australian and German Credit Approval problems and showed comparable results in comparison with other algorithms found in the literature. Future work of investigation is aimed at conducting additional numeric experiments and solving other real-world problems.
Bibliography
1. Huang J.-J., Tzeng G.-H., Ong Ch.-Sh. Two-stage genetic programming (2SGP) for the credit scoring model // Applied Mathematics and Computation, 174. 2006. P. 1039-1053.
2. Sergienko R., Semenkin E., Bukhtoyarov V. Michigan and Pittsburgh Methods Combining for Fuzzy Classifier Generating with Coevolutionary Algorithm for Strategy Adaptation // IEEE Congress on Evolutionary Computation. June 5-8, New Orleans, LA. 2011.
А. А. Шабалов
ПРИМЕНЕНИЕ НЕЙРО-НЕЧЕТКИХ СИСТЕМ В ЗАДАЧАХ ПОДСЧЕТА КРЕДИТНОГО БАЛЛА БАНКОВСКОГО СЕКТОРА
В виду сложности и значительных временных затрат проектирования нейро-нечетких моделей вручную с нуля в рассмотрение вводятся эволюционные алгоритмы. С помощью генетических алгоритмов разработка нейро-нечетких систем упрощается и становится автоматической. Предложенная схема применяется к известным задачам о кредитах в Австралии и Германии. Приводится сравнение с другими подходами.
Ключевые слова: нейро-нечеткое моделирование, эволюционные вычисления, нечеткие системы, нейронные сети.
© БИаЪаЬу А. А., 2012
UDC 519.234
M. Yu. Sidorov, S. G. Zablotskiy, E. S. Semenkin, W. Minker
EVOLUTIONARY DESIGN OF NEURAL NETWORKS FOR FORECASTING OF FINANCIAL TIME SERIES
The problem offorecasting in various technical, economic, and other systems is an important problem of nowadays. The methods of artificial intelligence and machine learning analyze very effectively various data including financial ones. The main problem of such techniques is the choice of model structure and the configuration of its parameters. In this paper we propose an evolutionary method for the neural network designing that does not require any expert knowledge in the area of neural networks and optimization theory from the user. This algorithm has been applied to the FOREX forecasting task of 13 different currency pairs based on the historical data for 12,5 years. The performance of the proposed algorithm has been compared to the forecasting results of other 6 algorithms. The proposed algorithm has shown the best performance on more than half of the tasks. On remaining tasks the algorithm yields slightly to the multi-layer perceptron trained by the particle swarm optimization algorithm. However, the predominance of the proposed algorithm is more significant.
Keywords: neural networks, evolutionary algorithms, particle swarm optimization, FOREXforecasting.
One of the expressive and pragmatic applications of artificial intelligence and machine learning is the prediction of financial time series in various markets. The FOREX market is the largest (about $4 trillion daily turnover) international currency market. According to the positive market theory there is a deterministic component in the stochastic price fluctuations on the FOREX market. Therefore, using a fairly accurate predictor it is possible to achieve some speculative success.
Recently, an increasing number of papers present the advantage of artificial intelligence methods and machine learning algorithms over the standard econometric methods for solving the problem of financial time series prediction. In particular, neural networks successfully cope with the challenges of the financial forecasting. Thus, the most popular econometric technology for the problem of time series forecasting is called ARIMA [1]. However, in [2] it was shown that the multi-layer perceptron trained by different algorithms outperforms the ARIMA (1, 0, 1) model for the problem of FOREX forecasting.
The main problem of artificial neural networks which prevents their widespread exploiting is the challenge of choosing their optimal parameters for a particular problem. There are many parameters to be set up by the user,
for example, the type of neural network, the learning algorithm, the number of hidden layers and neurons, activation functions, etc.
In addition, the most of modern artificial neural networks have a fixed structure with the predefined types of activation functions. What if the neural network with a more flexible structure will be able to solve a problem more accurately?
In this paper we propose a method for evolutionary forming of neural networks, which on the one hand does not require any expert knowledge in the fields of information technology and artificial intelligence from the user and on the other hand creates the neural network with the flexible architecture that could potentially increase the prediction quality.
The structure of this article is as follows. Section 2 describes the source data and its statistical characteristics. Section 3 provides the description of the proposed method. Section 4 describes others methods for solving the forecasting problem. The experimental setup is described in Section 5. Section 6 presents the forecasting results of the proposed neural network technology, as well as the comparison to the other forecasting models. The conclusions are done at the end of the paper.
