CC BY
64
Unsupervised color texture segmentation based on multi-scale region-level
Markov random field models
X. Song L. Wu 1, G. Liu w 1 School of Computer and Information Engineering, Anyang Normal University, Anyang 455000, Henan, China, 2Collaborative Innovation Center of International Dissemination of Chinese Language Henan Province,
Anyang 455000, Henan, China, 3 Henan Key Laboratory of Oracle Bone Inscriptions Information Processing, Anyang 455000, Henan, China
Abstract
In the field of color texture segmentation, region-level Markov random field model (RMRF) has become a focal problem because of its efficiency in modeling the large-range spatial constraints. However, the RMRF defined on a single scale cannot describe the un-stationary essence of the image, which highly limits its robustness. Hence, by combining wavelet transformation and the RMRF model, we present a multi-scale RMRF (MsRMRF) model in wavelet domainin this paper. In the Bayesian framework, the proposed model seamlessly integrates the multi-scale information stemmed from both the original image and the region-level spatial constraints. Therefore, the new model can accurately describe the characteristics of different kinds of texture. Based on MsRMRF, an unsupervised segmentation algorithm is designed for segmenting color texture images. Both synthetic color texture images and remote sensing images are employed in the comparative experiments, and the experimental results show that the proposed method can obtain more accurate segmentation results than the competitors.
Keywords: region-level Markov random field model, color texture image, image segmentation, wavelet transformation, multi-scale.
Citation: Song X, Wu L, Liu G. Unsupervised color texture segmentation based on multi-scale region-level Markov random field models. Computer Optics 2019; 43(2): 264-269. DOI: 10.18287/2412-6179-2019-43-2-264-269.
Acknowledgments: This work was financially supported by the Key Technology Projects of Henan province of China under Grant 15210241004, Supported by Program for Changjiang Scholars and Innovative Research Team in University, the Key Technology Projects of Henan Educational Department of China under Grant 16A520036, the Key Technology Projects of Henan Educational Department of China under Grant 16B520001,the National Natural Science Foundation of China under Grant 41001251, Anyang science and technology plan project: Researches on Road Extraction Algorithm based on MRF for High Resolution Remote Sensing Image, and the Research and Cultivation Fund Project of Anyang Normal University under Grant AYNU-KP-B08.
1. Introduction
Image segmentation is one of the key techniques in the field of remote sensing and computer vision. It aims at dividing an image into a series of non-overlapping image regions with similar characteristics such as spectral, texture, etc. Although many different kinds of methods have been proposed to solve this problem [1 - 3], it still keeps challenging because of noisy perturbation, outliers and imaging artifacts, due to these methods ignorance of spatial information.
In order to solve the problem, recently, many improved algorithms have been proposed to incorporate spatial information. S. Krinidis and V. Chatzis proposed a novel robust fuzzy local c-means algorithm (FLICM), where a fuzzy factor was introduced into its objective function to guarantee noise insensitiveness and image detail preservation [4]. This method was lately improved by M.Gongfrom introducing a kernel distance measure into its objective function [5]. Besides, Zhang et al.utilized the mean templates of distance function and membership function to consider spatial information [6]. All of these methods mentioned above have shown their superiority in their realms.
In the past decades, fuzzy clustering algorithms have been widely used in image segmentation. because they have robust characteristics for ambiguity and are able to retain more original image information than hard or crisp segmentation methods.Besides, they requires experts with adequate domain knowledge to determinate the number of clusters before clustering, which greatly limits their application.
The Markov random field models (MRF) have a perfect theoretical basis and can easily describe the spatial information of the image. So, they have a wide application in the field of image segmentation [7]. In the Bayesian framework, the MRF model based on pixels can integrate spatial interaction between pixels into the tagged verdict through the form of a priori probability, which can obtain better regional segmentation results. However, in the segmentation of texture images, especially macro texture images, it is difficult to accurately describe the characteristics of different textures through the spatial relationship between pixels. So, a larger scope of spatial interactions should be used to obtain accurate segmentation results. S. P. Chatzis and T. A. Varvarigou explored the spatial coherency modeling capacities of the hidden
Markov random field (HMRF) model in the fuzzy clustering procedure and proposed the HMRF-FCM algorithm [8]. We extended this method into a multi-scale version by capturing and utilizing the multi-scale spatial constrains [9] and enhanced its accuracy of selecting local information by incorporating region-level information into the fuzzy clustering [10]. The region-level MRF (RMRF)
[11] models introduce the interregional interaction into the image segmentation, which will make full use of the larger scope of the image space information. However, the RMRF model on a single scale does not take into account the non-stationary characteristics of the image itself, which will seriously affect its stability.
In this study, we present a multi-scale RMRF modelin wavelet domain (MsRMRF), On the basis of wavelet decomposition for the compensation of the inability of image non-stationary characterization the MsRMRF makes full use of the regional spatial interaction between different scales to get the exact texture description.
The reminder of this paper is organized as follows: in Section 2, we provide a brief review of the region-level markov random field model. In Section 3, we introduce our algorithm based on multi-scale region-level Markov random field models. The experimental results of the proposed approach are given in Section 4. Finally, Conclusions are drawn in Section 5.
2. Region-level Markov random field model
Suppose S= {1, 2, ..., X} represents pixel index assemblage of pending image Y. Ys (s e S) represents the B dimensional pixel color vector on the position s. In the initial segmentation, there are M different regions corresponding to the image Y. These regions and their spatial relationships can be described by the Region Adjacency Graph (RAG). Suppose that RAG is recorded as G = { V,E}. In this set, Vrepresents the set of vertices, and E represents the set of edges. Each vertex veV represents an area in the initial segmentation, which connects region v1 and region v2, and their spatial interactions are represented by the edges e122eE. A regional set of adjacent relations forms a neighborhood system n = {nv I veV} at a regional level. nv represents a set of regions that have side connections with the region v. Suppose that the image Y is divided into K categories, and all the pixels of the same region are the same tagged symbol as the region, then the segmentation result X can be represented as a RMRF tagged field: X = {Xv | ve V}, Xv = {Xs|seV}(1 <Xv<K) represents the category markers of region v. Meanwhile, it also represents the category markers of all the pixels in region v.
In the Bayesian framework, the segmentation model
[12] based on RMRF can be expressed as:
X = argmax x {p (X )P (y|X )}
(1)
P(Y | X) is regional class conditional probability and P(X ) is regional priori probability.
Assuming that every class is subject to Gauss segmentation and the class condition is independent, then P(YX) can be expressed as:
p (YX )=n ,e,p № ).
(2)
P(Yv | Xv) the class conditional probability of the region v, can be expressed as:
p (X )=n iEvP [Y]Xs ; ^, S Xv ).
(3)
|Xv and EXv represent the mean vector and covariance matrix of class Xv respectively. Gaussian distribution P (Ys | Xs; |Xv, EXv) can be expressed as:
1
P ( X ; p Xv, S X,)
cexp
(2*r/2 |SXv
(Ys ) ' Sx1. (Ys )
(4)
A priori model P(X ) can be expressed by the Gibbs distribution:
P (X ) = iexp {-U (X )/T }, (5)
Z =X x exp{-U (X )/T } is a normalization constant, and
U (X) en ^^ v env Q (v, ^ )
is an energy function. In this paperwe only consider the interaction between two adjacent regions, so Q(v,vr) is defined as:
Q (v, v') =
= ((Xv, XvO)J)V
(6)
V
av, bv and represent the area, edge length and color mean vector of the region v respectively. bvy is the common boundary length of region v and v'. 8 (Xv,Xv), which is the delta function, can be expressed as:
[1, Xv = Xv,
0, Xv ^ Xv.
^(Xv, Xv0 =
(7)
The formula (1) is equivalent to the formula (8): X = arg min x {- log (P (X))- log (p (y |X))}.
(8)
The corresponding energy function E can be expressed as the sum of the marked field energy function EX=-log (P (X)) and the characteristic field energy function Ey=-log (P (Y|X)):
E = Ex + Ey . (9)
The minimum energy segmentation result is the optimal segmentation result. The following formula can be achieved by putting formula (2) and (5) into formula (8):
x
E = EveV {ZveV [fö - V-X, )' Exl (Ys - ^ ) + log (|EXv ESEv +Zv'E
(1 -S( Xv, Xv) )
(10)
The minimization of the energy function can be solved by the optimization algorithm, such as the Iterative Conditional Modes [13], the Expectation Maximization (EM) [14], the Gibbs [15] sampling and the Graph Cut (GC) [16].
Though image segmentation based on RMRF model can take into account the large range of spatial information through a priori probability function at the region level, the model only describes the statistical characteristics of the image on a single spatial resolution scale, which is obviously incompatible with the non-stationary characteristics of the image, and needs to be improved.
3. Multi-scale Region-level Markov Random Field Model In order to improve the RMRF segmentation model, we put forward a multi-scale RMRF model (MsRMRF) in wavelet domain. Suppose the image Y is decomposed into D scale, and the corresponding wavelet coefficient is expressed as W = {w d\1 <d<D}. wd = {wd 15 e Sd} represents the image of the wavelet coefficients vector on the scale d. Sd is the pixel position index set on the scale d. The wavelet coefficients vector on each scale can be expressed as:
= J[wd (LL), wd (LH), wd (HL), wd (HH)]', d =D, (11) [[wd (LH), wd (HL), wd (HH))', d * D.
If the scale corresponding to the original image is recorded as d=0, that is to say w0 = Y, then there are multi-resolution data with (D + 1) scales altogether. Assuming that the RAG obtained by the initial segmentation on the scale d is expressed as G d = {Vd, e d}, the corresponding neighborhood system is expressed as nd={ nd |ve V d}, and then the segmentation results on each scale can be expressed as a RMRF tagged field on the scale: X d = { Xd |veV d}. Xd represents the common marks of the pixel set {s|seV d}on the scale d. The tagged field on all scales consists of a multi-scale tagged set Xms={Xd | d=1, 2, ..., D}.
In the Bayesian framework, segmentation model based on MsRMRF can be expressed as:
Xms = argmaxXm, {p((Xms)P(Xms)} . (12)
P (W | X ms) represents multi-scale class conditional probability, and P (X ms) represents multi-scale priori probabilities.
If the class condition is independent, then the P(W|Xms) can be expressed as:
p {w\xms )=n ;=o n ( wdAxd ).
(13)
wvd represents a set of wavelet coefficient vectors corresponding to region v on the scale d. If it obeys the
Gauss distribution, the regional conditional probability density P (wd | Xd) can be calculated by the following formula:
P {wdv | Xd ; v e Vd ) =
= n SEvP (w( ; VXd, EXd ).
(14)
If the segmentation results of high resolution scale are affected only by the segmentation results of adjacent low resolution scale, then the multi-scale priori probability P (X ms) can be expressed as:
p ( Xms ) =p ( XD )n P ( Xd \Xd+1 ) .
(15)
It's hard to get a complete expression of P (Xd | X d+1), so we describe the dependencies between scales by using Xd+1 to provide initial tags for Xd. The image segmentation algorithm based on MsRMRF model can be decomposed into the combination of RMRF segmentation for each scale, and can transfer the correlation between scales through tagged results of the scales. On the minimum resolution scale, we use the pixel level ICM algorithm to obtain the segmentation results. On other scales, the segmentation results are obtained by the minimum energy function expressed by the formula (16).
Ed = EvEVd {Esev [(wsd - VXd )' EXd (ys -VXd ) + +log (|E Xd |)^ +
(16)
+ Eve
(1 -S( Xd, Xd ))
|Vv - Vv
In this paper, the Mean Shift (MS) algorithm is used to obtain the initial region of each scale. The segmentation process is shown in Figure 1. The specific segmentation algorithm is as follows:
Step 1: Input image Y, set the class number K and wavelet decomposition scale D, MS segmentation of the minimum area ar, space bandwidth hs and range bandwidth hb.
Step 2: Carry out D scale wavelet transform for image Y, obtain the initial region of the scale based on the low frequency component w (LL) on each scale, set up RGA and calculate the area av, the edge length bv, and the color mean |av of each region.
Step 3: Execute pixel level ICM segmentation on scale d = D, and obtain the segmentation results XD.
Step 4: Let d = d + 1. If d< 0, the algorithm operation stops. Then output the segmentation results on the scale d + 1 as the final result; otherwise, turn to step 5.
Step 5: Project the segmentation results on the scale d + 1 to the current scale Xd+and use the voting method to get the segmentation tags in each region of the current scale.
Step 6: Adopt the ICM algorithm optimal formula (16) to obtain the segmentation results X d. Turn to step 4.
wd =
4. Experimental analysis
In order to verify the effectiveness of the algorithm, a combination of color texture images and two remote sensing images are used in this paper. And the algorithm proposed in this paper has been compared with the methods in ICM, MRMRF [17] and literature [18]. ICM is a classic single scale MRF segmentation algorithm, MRMRF is a multi-scale segmentation algorithm in wavelet domain, and the method in literature [18] is a single scale RMRF segmentation algorithm. In the experiment, both MRMRF algorithm and the algorithm proposed in this paper use 3-scale haar wavelet transform.
The algorithm proposed in this paper is compared with that of MRMRF, which is used to verify the ability of the wavelet domain MsRMRF segmentation algorithm proposed in this paper in accurately describing the texture characteristics by using a wide range of spatial information. Meanwhile, the algorithm proposed in this paper is also comparing with the method in literature [18], which is used to verify the ability of this algorithm in using multi-scale information and in describing the nonstationarity of the image. Besides, the MS algorithm used in this algorithm uses a set of fixed parameters: ar = 10, hs= 6, hb = 6.
Wavelet transform
Fig. 1. The segmentation process based on the MsRMRF model in the wavelet domain
Mean Shift Clustering
Multi-scale image segmentation
Fig. 2. The segmentation results of color texture image: The first column is the original images, the second column is the segmentation results of ICM, the third column is the segmentation results of MRMRF, the fourth column is the segmentation results of literature [18], and the fifth column is the segmentation results of MsRMRF
Synthetic texture images are shown in the first column in Figure 2. (a1), (b1) and (c1) in Figure 2 are obtained by Prague texture segmentation data generator [19]. (d1) in Figure 2 is manually synthesized by using remote sensing image texture. The class numbers of four images are 5, 4, 3 and 4 respectively. The segmentation results of ICM algorithm, the MRMRF algorithm, the method in literature [18] and the algorithm proposed in this paper are shown in the second, third, fourth and fifth columns in Figure 2 respectively. According to the diagram, the results of the ICM algorithm are the worst. It only takes into account the spatial interaction between pixels on a single scale and does not accurately describe the features of the textures. Obviously, the fifth column is better than the third column. The corresponding segmentation results have good regional consistency, and there is no obvious boundary deformation at the same time. Because modeling the spatial information on each scale by using RMRF can describe the characteristics of each texture
more accurately. Meanwhile, it can avoid the mixture of
Table 1. Quantitative comparison of segmentation results (%)
texture boundaries. The results of the fifth column in Figure 2 are also significantly better than the results of the fourth column, which shows that the wavelet domain MsRMRF segmentation algorithm proposed in this paper can make full use of the regional interaction information on multi-scale, and has the ability to describe the non-stationarity of the image.
In order to compare the advantages and disadvantages of the segmentation results directly, the segmentation error rate is used to evaluate each segmentation result, and is recorded in Table 1. The segmentation error rate is defined as the ratio of the number of the mis-segmented pixels to the number of the pixels of the image, and the values are between 0 ~1. The closer to 0, the better the segmentation results are, and vice versa. It's easy to see from table 1 that the average segmentation error rate of this algorithm is minimal (close to 0.015), which further proves the effectiveness of the algorithm proposed in this paper.
ICM MRMRF Literature [18] MsRMRF
Figure 2(a1) 14.39 9.72 3.47 1.25
Figure 2(b1) 37.66 14.36 11.09 3.54
Figure 2(c1) 13.91 4.78 6.32 1.07
Figure 2(d1) 27.99 0.84 0.93 0.23
Mean 23.49 7.43 5.45 1.52
In order to further verify the segmentation performance of the algorithm, we use two remote sensing images, as shown in Figure 3, to carry out a segmentation experiment. (a1) in Figure 3 is a QuickBird remote sensing image. It can be clearly seen from the image that mainly three types of ground objects are included: forest, built-up area and farmland. So the corresponding classification number is 3. (b1) in Figure 3 is a remote sensing image of Spot5. There are four main types of ground objects in the image: built-up area, woodland and two different types of farmland. Therefore, the corresponding classification number is 4. As there is no real object as reference, we only make a qualitative evaluation on the segmentation results of these two remote sensing images. Firstly, the mixed segmentation
between different ground object types is serious in the segmentation results of ICM algorithm, and the regional consistency is poor. Secondly, although the MRMRF segmentation algorithm can obtain better regional segmentation results, the mixed segmentation is still serious, and the boundary of the region is deformed obviously. Besides, in the segmentation results of literature [18], the mixed segmentation has been reduced, but the classification error is still more significant. The segmentation results of the algorithm proposed in this paper are obviously superior to the reference algorithm both in regional consistency and boundary smoothness, which further proves that the use of the wavelet regionlevel MsRMRF can better model the texture information and obtain more accurate segmentation results.
Fig. 3. The segmentation results of remote sensing image: The first column is the original images, the second column is the segmentation results of ICM, the third column is the segmentation results of MRMRF, the fourth column is the segmentation results of literature [18], and the fifth is column the segmentation results ofMsRMRF
5. Conclusion
This paper proposes an image segmentation algorithm based on multi-scale region MRF model in wavelet domain. The algorithm can take into account the wavelet information and regional interaction information at different scales and can take into account the wide range of spatial information. Meanwhile, it makes up for the deficiency of single scale MRF model in describing image non-stationarity. Therefore, it is able to describe the characteristics of different textures more accurately. The segmentation experiment of synthetic texture image and remote sensing image shows that this algorithm can obtain more accurate segmentation results.
References
[1] Drira F, Lebourgeois F. Mean-Shift segmentation and PDE-based nonlinear diffusion: toward a common variational framework for foreground/background document image segmentation. International Journal on Document Analysis & Recognition 2017; 20(3): 1-22.
[2] Kim TH, Lee KM, Sang UL. Learning full pairwise affinities for spectral segmentation. IEEE Trans Pattern Anal Mach Intell 2013; 35(7): 1690-1703.
[3] Abdelsamea M, Gnecco G, Gaber M. A SOM-based Chan-Vese model for unsupervised image segmentation. Soft Computing 2017; 21(8): 1-21.
[4] Krinidis S, Chatzis V. A robust fuzzy local information C-Means clustering algorithm. IEEE Trans Image Process 2010; 19(5): 1328-1337.
[5] Gong M, Liang Y, Shi J, Ma W, Ma J. Fuzzy C-means clustering with local information and kernel metric for image segmentation. IEEE Trans Image Process 2013; 22(2): 573-584.
[6] Zhang H, Wu QMJ, Zheng Y, Nguyen TM, Wang D. Effective fuzzy clustering algorithm with Bayesian model and mean template for image segmentation. IET Image Processing 2014; 8(10): 571-581.
[7] Yu Q, Clausi DA. IRGS: image segmentation using edge penalties and region growing. IEEE Trans Pattern Anal Mach Intell 2008; 30(12): 2126-2139.
[8] Chatzis SP, Varvarigou TA. A fuzzy clustering approach toward hidden Markov random field models for enhanced spatially constrained image segmentation. IEEE Transactions on Fuzzy Systems 2008; 16(5): 1351-1361.
[9] Liu G-Y, Wang A-M. Fuzzy clustering algorithm for integrating multiscale spatial context in image segmentation by hidden markov random field models. International Journal of Pattern Recognition and Artificial Intelligence 2013; 27(3): 1355005.
[10] Liu G, Zhang Y, Wang A. Incorporating adaptive local information into fuzzy clustering for image segmentation. IEEE Transactions on Image Processing 2015; 24(11): 3990-4000.
[11] Qin AK, Clausi DA. Multivariate image segmentation using semantic region growing with adaptive edge penalty. IEEE Transactions on Image Processing 2010; 19(8): 2157-2170.
[12] Yang F, Jiang T. Pixon-based image segmentation with Markov random fields. IEEE Transactions on Image Processing 2003; 12(12): 1552-1559.
[13] Besag J. On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society 1986; 48(3): 259-302.
[14] Zhang Y, Brady M, Smith S. Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Transactions on Medical Imaging 2001; 20(1): 45-57.
[15] Xia G-S, He C, Sun H. Integration of synthetic aperture radar image segmentation method using Markov random field on region adjacency graph. IET Radar, Sonar and Navigation 2007; 1(5): 348-353.
[16] Boykov Y, Funkalea G. Graph cuts and efficient N-D image segmentation. International Journal of Computer Vision 2006; 70(2): 109-131.
[17] Liu G, Luo L, Mei T, Qin G. A multispectral textured image segmentation method based on MRMRF. Geomatics and Information Science of Wuhan University 2008; 33(9): 963-966.
[18] Lin L, Zhu L, Yang F, Jiang T. A novel pixon-representation for image segmentation based on Markov random field. Image and Vision Computing 2008; 26(11): 1507-1514.
[19] The Prague texture segmentation datagenerator and benchmark - Introductory page. Source: (http://mosaic.utia.cas.cz/).
Author's information
Xu Song (b. 1980) graduated from Beijing Institute of Technology in 2008, majoring in Systems Engineering. He is currently a lecturer in Anyang Normal University; Research interests are statistical image processing, in particular, image segmentation and fusion.
Liang Wu (b. 1982) graduated from Xidian University in 2009, majoring in Computer Science and Technology. Currently he is a lecturer in School of Software Engineering Anyang Normal University. Research interests are Image processing, Information Safety.
Guoying Liu (b. 1979) graduated from Wuhan University in 2009, majoring in Geoscience and Remote Sensing He is currently a professorin Anyang Normal University. Research interests are statistical image processing, in particular, image segmentation and classification.
Received July 24, 2018. The final version - Desember 10, 2018.
