CC BY
9УДК 1
Snehal Pawar
Department of E&TC Engineering (Maharashtra, India)
Amarsinh Deshmukh
Department of E&TC Engineering (Maharashtra, India)
Rahul Mulujkar
Department of E&TC Engineering (Maharashtra, India)
ENDOSCOPIC IMAGE ENHANCEMENT USING BLIND DENOISING
Abstract: Biomedical images such as endoscopic images, retina, MRI, X-ray plays important role in the analysis and diagnosis of the internal body structure. Endoscopic image is used during pregnancy, plastic surgery, orthopedic surgery, spinal surgery etc. to examine internal body structure. Endoscopic images are corrupted with various types of noise. The noisy image results into inaccurate diagnosis and thus the endoscopic image denoising is essential. In this paper a method known as blind denoising has been used to improve the visual quality of the images. In the proposed method we first estimate the noise level in the image obtained. Now having known the noise level we apply BM3D algorithm to denoise the endoscopic image. By the proposed method it is found that the PSNR of the test image is improved. The enhanced image will help the doctors for accurate diagnosis.
Key words: blind denoising, noise level estimation, BM3D I. INTRODUCTION
Endoscopy is used to observe an internal body organ, structure or tissue. In this process long, thin tube is inserted into the body for diagnosis. The main application of endoscopy is imaging, minor surgery, diagnosis and so on. Few times in endoscopy due to internal bleeding
and some other problems we cannot get a clear image. Thus the noisy image results into inaccurate diagnosis. So we need an engineering solution to this problem.
Every real time image which is captured from the camera consists of some sort of noise. The noise may be from different types of source such as photon noise, thermal noise and quantization noise.
Image denoising is important in many image processing applications and analysis. The study of image denoising started a few decades ago i.e. since 1970, but still we are lagging behind the mark of perfection. Image denoising is classified on different basis such as domain based approach, noise level based approach. According to the noise level based approach denoising is divided into two types non-blind denoising and blind denoising. This classification is based on whether the noise level is known or unknown.
In case of non-blind denoising, the noise level(an) is considered as known parameter, this is conventional way of denoising. On the other hand in case of blind denoising the noise level(an) is unknown. We have to estimate the noise level parameter along with the denoising process. The accomplishment of image denoising algorithm predominantly depends upon the noise level (an) estimation.
In most of the commonly used noisy image model generally the noise is AWGN (Additive White Gaussian Noise). In the noise level estimation we mainly estimate the standard deviation (an) for given single noise image. Lots of work is done on this topic, many algorithms [3-9] have been implemented. These algorithms are basically classified into three types of approaches i.e. filter based approach, patch based approach & statistical approach.
Fig.1. Different types of Image Denoising
In filter based approach [3],[5],[7] noisy image is passed through the high pass filter to get the suppressed image structure. Then the difference between the filtered image and the original image is considered as the noise. The problem with this denoising method is that the difference between the images is considered as noise, but this assumption is not always true especially in case of image with complex structure.
In patch-based approaches [4], [7], [9], the image is divided into number of patches i.e. rectangular window of size N * N, and select the smooth patch among the separated patches. The smooth patch is selected on the basis of intensity level depending on the standard deviation. Here the consideration is the smooth patch consists of large amount of noise as compare to the true image contents of the patch. So approximate the true image contents to zero and hence by assuming the smooth patch consisting of only the noise, one can estimate the noise level. But the disadvantage of this method is that if the consideration goes wrong then overestimation or underestimation of the noise level takes place.
II. NOISE LEVEL ESTIMATION
Xinhao Liu [1] proposed a method for noise level estimation based on PCA. This method comes under patch based noise level estimation, here the input noisy image is divided into number of small patches in raster scan. Then we slide the window pixel by pixel so the patches are overlapped and the data model of the patches is represented as noisy image patch which is the combination of true image patch and noise. By taking the advantage of properties of the natural image i.e. the data of natural image spans only low dimensional sub space because of redundancy of natural image. If the data patch spans the subspace whose dimension is very less than the patches dimensions then that patch is known as low rank patches. Now here is the assumption that the minimum eigenvalue of the covariance matrix is equal to zero. The variance of the Gaussian noise is equally distributed in all the direction and all eigenvalues are same, so we can estimate the noise level.
The main disadvantage of this method is that our assumption is not always true, especially in case of images with complex structure. So when the image with very fine details is given we can overestimate the noise level. To overcome this disadvantage we go for proposed method in which we choose the low rank patches. The low rank patches may consists of the patches with similar structure which includes the high frequency components like edges, corners or texture.
A. Patch selection:
There are many algorithms used for the patch selection depending on their applications.
In an image patch local variance is an important parameter and it is useful to analyze the image
88
structure as well as to select the image patch for noise level estimation. Lee and Popper [12] proposed an algorithm in which homogeneous patches are required to estimate the noise level, but here the homogeneous patches are known as the patches with small local variance. Similarly Pyatykh et al. [9], proposed an algorithm where he discarded the patches with large variances. The advantage of above two methods is that, both the algorithms are simple and fast but the major disadvantage is that it overestimates the noise level. To overcome the above disadvantage Shin et al.[7] proposed a method in which instead of selecting homogeneous patches or discarding the patches with large variance, he suggest to use the adaptive threshold of variance to select the patches. By using this method the performance is improved but not up to the mark.
To deal with the above problem, Aishy Amer et al[4] proposed an algorithm in which high-pass operator as well as threshold is used to calculate the homogeneity measures, but the high pass operator is easily affected by the noise. Hence in case of high noise level estimation this method fails. So by analyzing above results we can conclude that noise level estimation using only the variance parameter is not accurate, rather we can say suitable patch selection is the first step for accurate noise level estimation and it depends not only on the image variance but also on the image structure.
Zhu and Milanfar [13] concluded that image structure analysis can be done on the basis of gradient covariance matrix. Xinho Liu[1] proposed an algorithm for patch selection which is based on local image gradient matrix and its statistical properties to select low rank patches. The proposed algorithm for low rank patch selection is as follows:
Algorithm:
1. Let us take an input patch yi (NxN)
2. Find the N2 x 2 gradient matrix Gyi, if the gradient matrix is null matrix go to
step 1.
3. Calculate the gradient covariance matrix Cyi for the image patch yi
4. Find the eigenvalues and eigenvectors of Cyi to calculate the dominant direction and its energy
5. Calculate the texture strength (^n) by using trace operator i.e. sum of all eigenvalues of covariance matrix.
6. To analyze the statistical properties of texture strength ^n, apply the Gamma Distribution
7. Apply the threshold to select the weak texture patches which is the function of given significant level 5 and noise level an
Applying the above algorithm to different patches
CASE 1: Let us take perfectly noise free flat patch yf, as the input. Now find the gradient matrix Gyf,
(1)
Where Dh and Dv are the horizontal and vertical derivative operators respectively. As patch yf is perfectly noise free flat patch, hence the gradient matrix Gyf is,
(2)
CASE 2: Now take noisy flat patch yf with Gaussian noise,
(3)
Where 'zf is actual noise free image contains and 'n' is the Gaussian noise patch with zero mean and standard deviation an. From the CASE 1 we know that the gradient matrix of flat patch is zero so the gradient matrix of noisy patch is,
(4)
Now calculate the eigenvalue and eigenvector of Gyf and by applying the trace operator we can get the texture strength of the patch as,
(5)
To analyze the statistical properties of ^n we have to apply the gamma approximation to the above equation. By simplifying we get,
(6)
Where, in the Gamma approximation first term is shape parameter and second term is scale parameter and an2 is the standard deviation of Gaussian noise.
To estimate the unknown noise level we have to select the weak texture patches, for that we have to set the threshold value. Below that threshold value the patch is consider as weak texture patch. Xinho Liu[1] proposed a formula for the threshold value which is depending upon the gamma approximation of texture strength of patch as below
(7)
Where F-1 stands for inverse gamma distribution function and 5 is the confidence level (values of texture strength within the threshold range). an2 is the standard deviation of Gaussian noise.
B. Iterative structure:
The noise level estimation totally depends upon selection of weak texture patches. We will take the noise level as a variable while threshold for the weak texture patch selection process. As number of iteration increases the accuracy of estimated noise level increases up to certain point after that we will get a constant value.
The algorithm for iterative structure is as shown below:
1. Estimate initial noise level
2. Calculate threshold Tk+1
3. Select patch Wk+1
4. Estimate new noise level if it is not stable go to stage 2
5. Final estimated noise level.
By using the above algorithm we can get the accurate and stable noise level.
III. BLIND DENOISING
In case of blind denoising the noise level (an) is unknown, we have to estimate the noise level parameter along with the denoising process. Till now in this paper we have estimated the noise level (an) that means now we have to choose or develop the best denoising algorithm for enhancing the image details such edge preserving, structural similarity etc.
Image denoising is very vast area. Different types of image denoising methods are as shown in Fig.1. Rather than going for non-blind denoising which is very easy and commonly used method we have chosen a blind denoising algorithm which is complex but gives more accurate results. Dabov [2] proposed an algorithm block matching and 3D transform which is based on grouping by matching and collaborative filtering which is as follows:
A. Grouping & collaborative filtering
i. Grouping of similar patches
Grouping of similar patches or blocks, the name itself indicates the meaning that is collocation of similar patches. For simplicity divide these stages into two parts, first gets the similar patches or blocks and second stack them together by matching.
Fig.2. Formation of 3D group from 2D image patches
Some patches of approximately same intensity level are selected, amongst them any one patch is selected as reference patch. The similarity of the patch depends on the distance between the patch, as the distance increases similarity decreases. From the reference patch some fixed distance threshold is taken. The patches within this threshold are selected for grouping. The similar 2D image patches stacked together is known as 'group', so instead of group it referred as 3D here one extra D stands for grouping of similar 2D image patches. This group is formed by many block or patch matching so the name is give as BM3D. The forming of groups is as shown in Fig.2 and Fig.3.
ii. Collaborative filtering and reduction in transform domain
Given a bunch of n small parts of image which is also known as group of patches. By applying collaborative filtering on that patches we get an estimate for each individual patch. This estimate preserves the difference between the patches and details of it. Here we apply filtering to group of patches so the word collaborative filtering occurs.
The collaborative filtering achieve the best results when preform the shrinkage in transform domain. Now let us consider the 3D groups of similar image patches that are already constructed as discussed above. The collaborative reduction includes following steps,
1. Transform the 3D group
2. Apply reduction (by wavelet or winner filtering)
3. Inverse the linear transform
The collaborative filtering is effective in case of natural images which is characterized by both intra fragment and inter fragment correlation. The 3D transform can produce the sparse representation of the signal in a group. Sparsity is defined a number of non-zero elements in a vector or matrix. Sparsity achieves great results while preserving the structural details of an image.
Fig.3. Example of grouping block or patches from noisy image
B. Algorithm
Dabov [2] proposed an algorithm for image denoising by grouping and collaborative filtering. The image is divided into the number of small patches. Depending upon the structure, intensity and some more parameters of patch the similar patches are grouped together. This grouping is achieved by block matching and stacking of some patch in a group, is referred as 3D. So the name of this algorithm is BM3D.
Algorithm:
1. Form a group of similar patches that is 3D block
2. Denoise the 3D block by wavelet thersholding
3. Aggregate each estimate of the denoised patch to form the image
4. Repeat the same algorithm using wiener filtering in 2nd stage
IV. EXPERIMENTATION
Xinhao Liu [1] performed noise level estimation on complex image structure such as mountain image, gravel image and so on. In this paper we have taken an endoscopic image (source: worldendo.org) as shown in Fig.4 (a) and Fig.5 (a). Let us consider the noise free test image in which known White Additive Gaussian Noise is added, then we estimate the noise level (an) as per our proposed algorithm and remove the noise by BM3D. Now we check the
PSNR in both the cases of noise image and denoised image.
93
Let us take a test image as an input, now add additive white Gaussian noise of zero mean and variance an. We have added an AWGN (Additive White Gaussian Noise) in an image because it is uniformly distributed over the image. For blind denoising first we have to estimate the noise level. Then using Block matching algorithm we denoise the image. For noise level estimation we have to select a weak texture patch from input noisy image. Selection of weak texture patch is important parameter in the noise level estimation process. Weak texture patch is selected by analyzing the image structure and the strength of the patches (^n). Now in the selected patch assume that the selected image patch is flat patch. As the selected patch is weak textured patch in the image so our assumption is true. We can estimate the true noise level by iterative structure.
Apply the grouping and collaborative filtering to the noisy image as discussed in the section III. In step 1 we denoise the image by smoothing and edge preserving algorithm in which we use hard (wavelet) thersholding. In step 2 rather than using hard threshold we have used wiener filtering keeping rest of the process unchanged.
V. RESULTS AND DISCUSSION
In this section we present and discuss the experimental results obtained by the proposed method. For endoscopic image enhancement we improve the PSNR value by 60% in case of AWGN. The PSNR of the estimated image y of the true image is calculated by the following
For the test image 1 as shown in Fig.3(a),(b),(c) the PSNR value increase from 20.178 dB to 33.499dB and similarly for test image 2 as shown in Fig 4 (a),(b),(c) the PSNR value increases from 20.178dB to 35.800 dB. The proposed algorithm is tested and sampled with addition of AWGN and the PSNR is calculated. It was found that the algorithm is working as expected. And thus the noisy image is taken as input and PSNR is calculated. Now we take noise image as a input and then denoise it by using our proposed method.
The Fig 6(a) is taken as the input from gastroscopy done by Dr. Pankaj Bansode(MS, FIAGES, FICS), Bharati Vidyapeeth medical college, Pune. Processor used for the gastroscopy is OYLMPUS CV-150. Now, by applying the proposed algorithm to the input image we have enhanced the image structure as highlighted in Fig 6(b). We have successfully enhanced the image structure which will help the doctors for accurate diagnosis.
VI. CONCLUSION
The proposed work presents the noise level estimation, block matching and collaborative filtering for blind denoising of endoscopic image. This is a novel approach in case of endoscopic image enhancement. By the observation of PSNR value it is clear that the blind denoising using BM3D method can enhance the image structure which will help doctors for an accurate diagnosis. The proposed algorithm was applied on real images and found to be successfully working. The enhancement in PSNR is clearly seen.
REFERENCE
Xinhao Liu, Student Member, Masayuki Tanaka, Member "Single-Image Noise Level Estimation for Blind Denoising" IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 12, DECEMBER 2013
Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian, Senior Member, IEEE" Image denoising by sparse 3D transform-domain collaborative filtering" IEEE transactions on image processing, vol. 16, no. 8, august 2007.
K. Rank, M. Lendl, and R. Unbehauen, "Estimation of image noise variance," IEE Proc. Vis., Image, Signal Process., vol. 146, no. 2, pp. 80-84, Aug. 1999.
A. Amer, A. Mitiche, and E. Dubois, "Reliable and fast structureoriented video noise estimation," in Proc. Int. Conf. Image Process., vol. 51. 2002, pp. I-840-I-843.
B. R. Cornera, R. M. Narayanana, and S. E. Reichenbach, "Noise estimation in remote sensing imagery using data masking," Int. J. RemoteSens., vol. 24, no. 4, pp. 689-702, 2003.
S.-C. Tai and S.-M. Yang, "A fast method for image noise estimation using Laplacian operator and adaptive edge detection," in Proc. 3rd Int.Symp. Commun., Control Signal Process., Mar. 2008, pp. 1077-1081.
D.-H. Shin, R.-H. Park, S. Yang, and J.-H. Jung, "Block-based noise estimation using adaptive Gaussian filtering," IEEE Trans. ConsumerElectron., vol. 51, no. 1, pp. 218-226, Feb. 2005.
C. Liu, R. Szeliski, S. B. Kang, C. L. Zitnick, and W. T. Freeman, "Automatic estimation and removal of noise from a single image," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 2, pp. 299-314, Feb. 2008.
S. Pyatykh, J. Hesser, and L. Zheng, "Image noise level estimation by principal component analysis," IEEE Trans. Image Process., vol. 22, no. 2, pp. 687-99, Feb. 2013. Z. Daniel and W. Yair, "Scale invariance and noise in natural images," in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 2209-2216.
C. M. Bishop, Pattern Recognition and Machine Learning. New York, NY, USA: SpringerVerlag, 2006.
J. Lee and K. Hoppel, "Noise modeling and estimation of remotelysensed images," in Proc. 12th Can. Symp. IGARSS, Jul. 1989, pp. 1005-1008.
X. Zhu and P. Milanfar, "Automatic parameter selection for denoising algorithms using a no-reference measure of image content," IEEE Trans. Image Process., vol. 19, no. 12, pp. 311632, Dec. 201
