CC BY
56
УДК 519 ББК В19
Фу Цай Лю, Шу Энъ Ван, Цзинъ Мэй Доу,
кафедра автоматизации,
Главная Лаборатория компьютерного контроля промышленной инженерии провинции Хэбэй, Янъшанъский университет (Цинъхуандао, Китай), e-mail: shuenwangl968@263.net
Усовершенствованный метод нечеткой идентификации на основе трансформации данных Бокса-Кокса
Практическая проблема идентификации нечетких систем состоит в проектировании и настройке функций принадлежности. В отличие от традиционных подходов, которые используют оригинальные модели данных для построения нечеткой модели, подход, использующий технологию преобразования данных и эвристический метод, предлагается для упрощения процедуры моделирования. Для передаваемых данных, в первую очередь, начальное значение нечетких «если-то» правил с нефазивными синглтонами в последующих частях создается с помощью эвристического метода. Затем, осуществляется настройка с помощью алгоритма изучения градиентного снижения. Предлагаемый метод имеет большую точность аппроксимации и лучшую обобщаемость. Метод продемонстри-рован на проблеме DISO, используя преобразования Бокса-Кокса.
Ключевые слова: идентификация нечетких систем, обработка данных, эвристический метод, метод градиентного снижения, преобразование Бокса-Кокса.
Fu-Саг Liu, Shu-En Wang, Jin-Mei Dou
Department of Automation, Key Lab of Industrial Computer Control Engineering of Hebei Province, Yanshan University (Qinhuangdao, China), e-mail: shuenwangl968@263.net
An Improved Fuzzy Identification Method Based on Box-Cox Data
Transformation
A practical problem in the identification of fuzzy systems from data is the design and the tuning of the membership functions. Unlike the traditional approaches that utilize original data patterns to construct the fuzzy model, an approach exploiting both data transformation techniques and heuristic method is proposed to simplify the modeling procedures. For the transferred data, firstly, the initial value of fuzzy if-then rules with nonfuzzy singletons in the consequent parts is generated by the heuristic method. Then, fine-tuning is done by gradient descent learning algorithm. The proposed method has better approximation accuracy and better generalization. The method is demonstrated on a DISO problem, using the Box-Cox transform.
Keywords: fuzzy systems identification; data processing; heuristic method; gradient descent method; Box-Cox transform.
Introduction
Data transformation is that transforming the original data to the new data by some kind of computing. Data transformation can change the smoothness and comparability of the data, and selecting the appropriate data transformation can improve the smoothness of the sequence, then you can transform the incomparable data to comparable data [1]. Recently, data transformation technology is widely used. Paper [2] has studied in-depth for data transformation in the system theory. Starting from the mechanism of data transformation, the construction guidelines of data transformation is proposed. On the basis of the criteria, it constructs a new data transformation and the transformation is applied to the GM (1,1) model. The transformation has improved the prediction accuracy of GM (1,1), and it has a certain practicality. Paper [3] applies the matrix theory to the control process of electronic jacquard machine. It has simplified the development and maintenance of electronic jacquard machine control system, and makes it easy for the users to quickly change the weaving products. This method has solved the problems of dissatisfaction needle use of electronic jacquard machine and multiple data channels, and realized
© Фу Цай Лю, Шу Энь Ван, Цзинь Мэй Доу, 2012
133
the independent of control system and hardware system structure and loading made electronic jacquard machine. Paper [4] applies the data transformation technology to outlier detection. Firstly, it transformed nonlinear problem into a linear problem in high dimensional feature space, and then the non-linear data transformation is used for dimensionality reduction. Experiments show that this algorithm can be used not only to outlier detection of linear separable data sets, but also to outlier detection of linearly inseparable data sets, which improves the efficiency of outlier detection of high-dimensional data sets. Data transformation is also used for image enhancement processing and it can improve the understanding of remote sensing image and accuracy of information extraction and interpretation. In these data transformation methods, KL transformation^], KT transformation, HIS transformation, Fourier transformation, wavelet transformation, multi-change detection transformation are widely used. Zeng Zhiyuan has proposed the LBV data transformation method[6] to highlight typical objects, such as vegetation, water bodies and bare surface, which provides convenience for information identification and extraction. Simultaneously, there is a logical relationship between the colors on the LBV color composite image and feature types, and accordingly we can accurately extract surface features. Application of LBV data transformation method can study the remote sensing images of coastal zone in classification, which can improve the classification accuracy of remote sensing images[7].
In this paper, the Box-Cox transformation method is proposed for fuzzy identification. Instead of following the traditional approaches which directly optimize the membership functions to meet a given sample data , this method directly transform the data to another, and the transformed data is similar to a Gaussian distribution. In this method, we can avoid the adjustment of membership function. The application of data transformation techniques and heuristic methods simplifies the process of data modeling. For the transformed data, firstly, the initial value of fuzzy if-then rules with nonfuzzy singletons (i.e., real numbers) in the consequent parts are generated by the heuristic method, then fine tuning is done by gradient descent learning algorithm. The advantage of proposed method is not only the more accurate approximation, but also the faster convergence speed. The method is demonstrated on a DISO problem, using the Box-Cox transform.
1. Description of the problem
Generally speaking, the identification of a fuzzy model includes the structure and parameter identifications [8]. In the structure identification, not only the system variables need to be chosen but also proper partitions of both input and output spaces are required. After identifying the structure of a fuzzy model, the parameter identification follows in order to adjust the membership functions and the parameters used in the fuzzy rules such that a more satisfactory performance can be obtained.
Depending on the types in the consequent parts, the fuzzy rules can be roughly classified into three different types, i. e., types 1-3 [2]. The following briefly reviews those three different types of fuzzy model, which are commonly used.
Typel:
If xiis An&nd a^is A{2 and ... and x ¡s Ain then yi is bi (1)
Where bj s are singletons. For the typel fuzzy model, the inferred output can be calculated as follows:
C______
7^ wibi
Vc = ^-------- (2)
E »i
i= 1
Where Wj is the firing degree of the antecedent part in the ith rule and c is the number of rules. Assume that the product operation is used, then
r
^i ^ {Xj^j Aij (^¿) ^ Ai2 (^2) ^ ••• ^ ^ir (xr) (3)
Where Ajj (Xj) represents the membership degree for the data Xj on fuzzy set Ajj.
Type 1 are applied widely because of it is simple and easy to be transferred into fuzzy rules described by linguistic information [9-10].
Since the roughly designed membership function or rule bases may not satisfy our control or identification purpose, a tuning scheme must be succeeded into adjusting the fuzzy model. Before
proceeding to the tuning job, we usually rely on a cost function to evaluate the identified fuzzy models. Define the error function for a given pattern as follows:
E=\{yc~ Vdf_____________________________________________(4)
Where yd denotes the desired output. Based on the defined error function, we can use genetic algorithms or some other approaches to adjusting the related parameters that show some degrees of effect to the system performance. Despite the initial setting problem, the gradient descent method is a strong candidate for type 1 fuzzy model.
Although a fuzzy model can be directly built from the numerical data, preprocessing the original data may be beneficial to the determination of the membership functions [9]. The paper focuses on presenting a method to improve the quality of fuzzy models based on type 1 model. Here, the statistical aspects of the transform are considered, rather than the function approximation aspects. Since fuzzy models are user oriented, the specific transform used will depend on the type of the application. The identification process will be performed on the transformed data, for a standard family of membership functions that have a good accuracy-complexity tradeoff. An inverse transformation will be used to test the final quality of the resulting fuzzy model [11].
2. The heuristic method and gradient descent method for the transformed models
2.1. Heuristic method
Let us assume that m input-output pairs (xp,yp), p = 1,2, ...m are given. Our problem is to determine the value of bi of each fuzzy if-then rule in (1) from the training data (xp, yp). For this problem, we propose the following heuristic method:
m m
B?M =y2Wi (xp)Vp/ YhWi (xp)’i = l,2,...,c (5)
p= i pi
Where bfM is the average of yp weighted by Wi(xp).
The excellence of heuristic method is its simple. Ref. [12] has proved heuristic method apply to practical problem with noise.
2.2. Learning method
To fit the control or identification requirement, the roughly identified fuzzy model must be further optimized. If available computation time is enough, learning method may generate fuzzy if-then rules with better performance than heuristic method. Type 1 model’s conclusion part, bi can be modified by the following form:
bt (t+l) = bt (t) + p(-dE/dbi) =
E
= bt (t) - pb (yc -yd)wi/^Wj,i = 1,2,..., c___________________________(6)
i = 1
Where fa is the learning rate, bi (0) = b^M.
2.3. Data transformation
The Box-Cox transformation is widely used to transform hydrological data to make it approximately Gaussian.
i3^, ^0,
%,A = yw = I y > 0 (7)
\logey, A = 0,
Since A itself is unknown, it has to be estimated as A, the value that minimizes the mean square error of the fit between the transformed data and its fuzzy model. In applications, to simplify the optimization procedure, rather then finding A . The set of "preferred"values for the Box-Cox transformation is taken as -1, -1/2, 0, 1/2, 1.
3. Simulation examples
To verify the effectiveness of the proposed model and to compare ours with previous simulation result, 50 data derived from the following equation and depicted by Sugeno and Yasukawa (1993) are used:
( \2 y=( l + x^+x^1’5) , 1 < xi, X2 < 5 (8)
The first step in establishing our fuzzy models is to define the premise parameters. Here we adopt Gauss- shaped membership.
(xj) = exp (- (xj - djf /2cr?-)________________________________(9)
Where Cjj and ajj are centers and crossover slope, respectively. Usually Cjj can get from FCM. Here, in order to avoid recursive process to find the cluster center, the equalized universe method (EUM) are adopt [13].
The most direct way to initialize the parameters of membership functions is to equally partition the input space into grid types with chosen number of clusters and each grid representing a fuzzy if-then rule. The initial parameters of MFs are set in such a way that the centers of MFs are equally spaced along the range of the data dimension. The equalized universe method is described briefly in the following. Given MFs (or clusters), the cluster center and crossover slope of membership functions are determined as
M+ - V~
Cij=Mr+ (j-1)
M+-Mr / x
aij = —i------TT". 1<3<!> 1 <*<c 10)
__________s(c~!)_____________________;___________________________________
Where M~ and Mf are the minimum and maximum values of the universe. The crossover slop of
MF is chosen such that the adjacent membership functions are overlapped approximately 25 %. EUM is simple and effective if the data sets are uniformly sampled. Unfortunately, there is not a prior knowledge about the distributions of data for most cases. Another problem of such kind of partition is that the number of fuzzy rules increases exponentially as the dimensions of the input space increase.
In computer simulation, the training set was transformed with five A values of the Box-Cox transform. EUM is used to decide the transferred fuzzy sets. The conclusion parameters were calculated by heuristic method and gradient descent method. Here, learning speed is 7w = 0.1 , and steps is 300.
4. Simulation results and analyses 4-1. Training performance
Fig. 1 and Fig. 2 present the MSE for original data and transferred data of five models respectively. Each of models’ conclusion parameters is identified by heuristic method and gradient descent method. Fig. 3 shows the fuzzy sets used and their inverse transform. Fig. 4 shows the simulation result from the transformed data using heuristic method, and Fig.5 using gradient descent method. Where c = 6, A = —1. In which solid line is actual output and dashed line is model’s output.
From Fig. 1 and Fig. 2, we can conclude:
(1) In two methods, the conformity become better with the c increase.
(2) After transformation, the train performance of the model is improved obviously, and the conform ability for gradient descent method is better than the heuristic method.
(3) In Box-Cox transform, transform A = — 1 is the best. In other words, the little the effectively compress domain for the transferred data the higher of the model precision.
As we can see from Fig. 1 and Fig. 2, the value A = — 1 turned out to be significantly larger to the
affection of the model performance than the next value. Since the domains of the fuzzy sets is designed rather arbitrarily, we cannot really conclude which transformation is best. Look for the Fig. 4 and Fig. 5; the difference is very small if any. While in both case, further tuning of the fuzzy sets is possible, the potential improvement is indicated to be small.
0.7 0.6 . 0.5
ОЩ.^ЛЛ/ МЛ-*» Ul" юАнЛІ ‘іИпМИііі'
A=1
A=0
A=-1
A=-0.5
A=0.5
^ 0.4
R’
5‘ 0.3 0.2 0.1 0Л
0.35
0.3
0.25
Ktf 0.2' 5 0.15 0.1 0.05 0
шт% Кр'ї- )j M>i )м ІЯІ ! і I [l! j 11 1 J ILi
A=1
A=0
A=-1
A»-0.5
A=0.5
3.5 4 4.5
3.5 4 4.5
Fig. 1. The comparison of simulation results before and Fig. 2. The comparison of simulation results before and after the data transformation using heuristic method after the data transformation using heuristic method
a) Original axis
b) Linear transformation Л = 1
d) Log transformation axis Л = — 1
c) Linear transformation Л = 0
4-2. Generalized ability of the test data
To illustrate the generalization of the proposed method, we use 50 data sampled from Eq. (8) to be our training patterns. Fig. 6 and Fig. 7 present the comparison of test results before and after the data transformation for the five models respectively. Each of models’ conclusion parameters are also identified by heuristic method and gradient descent method. Fig.8 shows the test result from the transformed data using heuristic method, and Fig.9 using gradient descent method. Where c = 6, A = — 1. In which solid line is actual output and dashed line is model output.
Form Fig. 6 and Fig. 7, we can conclude:
(1)The test performance of the fuzzy model based on data transform and gradient descent method to the value is sensitive to the model based on hcuristic method.
(2) After transformation, the test performance of the model is improved obviously, and the conform ability for gradient descent method is better than the heuristic method.
5.5 5
4.5 4
3.5 3
2.5 2
1.5 TL
e) Transform with A = —1/2 / ^ Transform with A = 1/2
.Fig. 3. The fuzzy sets used and their inverse transform
5.5 5
4.5 4
3.5
3
2.5 2
1.5 1
Fig. 4. Simulation results from the Transformed data 5. Simulation results from the transformed data
using heuristic method (A = —1, MSE = 0.0119) using gradient descent method (A = —1, MSE — 0.0008)
(3)With the increase of value, the generalized ability become worse when we use the gradient descent method.
Note that the Box-Cox transform is defined only for y>0. The data has to be translated, or other has to be used in case this condition is not met»
0.7
0.6
0.5
Sir 0.4
-Ю
0.3
0.2
0.1
0.
0.25
&Л1ШЙ: VfâJfUïi¡11/
A=1
A=0
A=-1
A=-0.5
A=0.5
A=1
A=0
A=-1
A=-0.5
A=0.5
2 2.5 3 3.5
4.5 5 5.5 6
Fig. 6. The comparison of test results before and after the data transformation using heuristic method
Fig. 7. The comparison of test results before and after the data transformation using gradient descent method
5. Conclusions
A practical problem in the modeling of fuzzy systems from data, is the design and the tuning of the membership functions. Instead of following the traditional approaches which utilize original data patterns
5.5 5
4.5 4
3.5 3
2.5 2
1.5 1«
5.5
5
4.5 4
3.5 3
2.5 2
1.5 1
& Ії:
Fig. 8. The Test results from the Transformed data Fig. 9. The test results from the Transformed datausing using heuristic method (A = 0, c = 5, MSE = 0.0485) gradient descent method (A = 0, c = 5, MSE = 0.0025)
to construct the fuzzy model, this paper proposed a methods to improve the fuzzy system performance by using heuristic method and data transformation technique. The improved fuzzy modeling method based on Box-Cox transformation. The transformation method allows us to map the original data to other domains so without adjusting membership functions and the fuzzification is simply since the membership functions is fixed. For the transformed data, firstly, the initial value of fuzzy if-then rules with nonfuzzy singletons (i. e., real numbers) in the consequent parts are generated by the heuristic method, then fine tuning is done by gradient descent learning algorithm. The advantage of proposed method is not only the more accurate approximation, but also the faster convergence speed. Simulation results show the effectivity of the proposed method.
References
1. Liu Sifeng, Dang Yaoguo, Fang Zhigeng. Grey system theory and its applications! M[. Beijing: Science Press, 2004 (in Chinese).
2. Qian Wuyong, Dang Yaoguo. New type of data transformation and its application in GM (1,1) model [J]. Systems Engineering and Electronics,2009,31(12). P. 2879-2908 (in Chinese).
3. Shen Wei, Hu Hongbo, Yuan Yanhong, Data transform control of electronic jacquard [J], Journal of Textile Research,2010,31(6). P. 125-128 (in Chinese).
4. Xu Xuesong, Zhang Xu, Song Dongming, Zhang Hong, Liu Fengyu. Outliers detection algorithm based on nonlinear data transformation [J]. Engineering Science,2008,10(9). P. 74-78 (in Chinese).
5. BOYD R K. A comparison of the usefulness of canonical analysis, principal components analysis and band selection for extraction of features from TMs data for land cover analysis. Proceedings of the Seventeenth International Symposium on Remote Sensing of Environment [Cl, 1983. C. 1333-1342.
6. ZENG Z Y. A new method of data transformation for satellite images: I. Methodology and transformation equations for TM images [J]. Remote Sensing, 2007, 28(18). P. 4095-4124.
7. Zhang Chengwen, Ma Yi, Guo Liping. Research and Evaluation of Coastal Zone Multi-spectral Image Classification Based on LBV Transformation [J]. Geography and Geoinformation Science,2010,26(4). P. 53-56 (in Chinese).
8. Takagi T., M. Sugeno. Fuzzy Identification of System and Its Application on Modeling and Control [C[. IEEE Trans, on Systems, Man and Cybernetics, 1985, 15(1). P. 116-132.
9. Huang Y. P., Chen Y. R.. Preprocessing Data Patterns to Simplify the Identification of Fuzzy Models. International [J] Journal of Systems Science, 1999, 30(5). P. 479-489.
10. Huang Y. P., Chen Y. R., Chu H C. Identification a fuzzy model by using the bipartite membership functions [J], Fussy Sets and Systems, 2001, 118(2). P. 199-214.
11. Shmilovici A., Martin J. A.. Improving Fuzzy Systems Identification with Date Transformations [J]. International Journal of Approximate Reasoning, 1999, (22) P. 93-107.
12. Ishibuchi H., Nozaki K., Tanaka H. Empirical Study on Learning in Fuzzy Systems by Rice Taste Analysis [J]. Fuzzy Sets and Systems, 1994, 64(2). P. 129-144.
13. Chen M. SWang., S. W.. Fuzzy Clustering for Optimizing Fuzzy Membership Functions [J], Fuzzy Sets and System, 1999, 103(2). P. 239-254.
Статья поступила в редакцию 25.04.2012 г.
