CC BY
78
IMAGE PROCESSING, PATTERN RECOGNITION
AN EFFICIENT BLOCK-BASED ALGORITHM FOR HAIR REMOVAL IN DERMOSCOPIC IMAGES
I.S. Zaqout1
1 Department of Information Technology, Faculty of Engineering and Information Technology
Al-Azhar University, Gaza, Palestine
Abstract
Hair occlusion in dermoscopy images affects the diagnostic operation of the skin lesion. Segmentation and classification of skin lesions are two major steps of the diagnostic operation required by dermatologists. We propose a new algorithm for hair removal in dermoscopy images that includes two main stages: hair detection and inpainting. In hair detection, a morphological bottom-hat operation is implemented on Y-channel image of YIQ color space followed by a bina-rization operation. In inpainting, the repaired Y-channel is partitioned into 256 non-overlapped blocks and for each block, white pixels are replaced by locating the highest peak, using a histogram function and a morphological close operation. The proposed algorithm reports a true positive rate (sensitivity) of 97.36 %, a false positive rate (fall-out) of 4.25 %, and a true negative rate (specificity) of 95.75 %. The diagnostic accuracy achieved is recorded at a high level of 95.78 %. Keywords: dermoscopy image, melanoma, hair detection, hair removal, inpainting, skin lesion. Citation: Zaqout IS. An efficient block-based algorithm for hair removal in dermoscopic images. Computer Optics 2017; 41(4): 521-527. DOI: 10.18287/2412-6179-2017-41-4-521-527.
The rest of this research is organized as follows: Section 1 describes an overview of related work. Section 2 describes the proposed technique. The implementation is presented and discussed in Section 3 followed by some remarks and future work.
1. Related work The pixel-based interpolation technique was proposed by [8] to find a quadratic curve which detects curved hairs in the binary image mask for removal and replacement. Gabor filtering and PDE-based image reconstruction was proposed by [9] for hair removal problem. In addition, for edge sharpening, they have used a warping algorithm to move pixels from the neighborhood of the blurred edge closer to the edge while the overall luminosity and texture patterns of skin lesions are preserved.
In [10], two main steps are proposed to automatically detect and remove hairs from dermoscopy images: firstly, generation of a binary image mask by isolating hairs and ruler marking. The red channel of the RGB dermoscopy image is utilized to perform noise removal followed by an adaptive canny edge detector to generate the binary image mask. Secondly, a repaired operation based on polygons inpainting is implemented on the white regions of the generated mask.
The work of [11] relied on two classes of images, grayscale and RGB images. In grayscale images, based on edge property a circular mask used to remove the non-skin pixels followed by a repair operation achieved by a normalization process of pixel values. In RGB images, based on histogram values a frequency of occurrence of each bin is measured followed by the calculation of minimum distance among neighborhood pixels.
An algorithm presented by [12] for automatically detecting and repairing hair occlusion in dermoscopy images. In the detection stage, hairs are segmented using MF-FDOG, thresholding, and morphological edge-based techniques applied for enhancement. In the repair stage, the fast marching technique is implemented to inpaint the image without loss of texture patterns of skin lesions.
Introduction
Malignant melanoma is the most serious form of skin cancer. An early detection and diagnosis of skin cancer prevents its progression to later stages. Menzies method, the 7-point checklist, the CASH algorithm, and the widely used algorithm is the ABCD/ABCDE method are computational algorithms have been developed using image processing techniques to assist dermatologists in early diagnosis of skin lesions [1- 4].
Great efforts have been done by researchers to create effective and reliable automated diagnostic methods of skin lesions, but not much research has been focused on the hair removal problem. It's obvious that human body may be entirely covered by hair which has a range of different textures, orientations, and colors; therefore affect partially/completely the appearance of skin lesions as shown in Fig. 1. Hair removal is an important step in dermoscopy images to classify the skin lesion correctly into benign, suspicious, or malignant.
Fig. 1. Digital dermoscopic images with hair pixels [5]
Various techniques were applied to remove hairs automatically from dermoscopic images are discussed in detail by [6, 7].
Dual-Channel Quaternion Tubularness Filters and MRF-based Multi-Label Optimization are proposed by [13] for hair enhancements in dermoscopy images. There method was validated and compared to other methods in terms of: hair segmentation accuracy, image inpainting quality, and image classification accuracy. The Generalized Radon Transform used to remove hairs by detecting hair pixels in a binary image mask followed by replacement through pixel interpolation. The Radon Transform was chosen to locate quadratic curves characterized by rational angle and scaling [14].
A two-stage artifact detection termed Fast Image Restoration (FIR) via Canny algorithm and Line Segment Detection (LSD) operation for effective detection of artifacts proposed by [15]. To remove artifacts from dermoscopic images, the Fast Marching Method (FMM) was applied at each stage while preserving morphological features during artefacts removal. In [16, 17], they proposed a threshold set model for digital hair removal from dermoscopic images. A gap-detection algorithm was adapted to locate hairs for every threshold and merge results in a single mask image. Morphological filters and medial descriptors are combined to locate hairs in generated mask.
The proposed work of [18] is automatically detects and removes hairs and ruler markings from dermoscopy images. In detection stage, they used a curvilinear structure and modeling, as well as feature guided exemplar-based in inpainting stage. Extensions to the fast marching method are introduced [19] with the aim to enhance the segmentation of medical image data. The proposed algorithm used to minimize the occurrence of bleeding across boundaries, including automatic starting point selection and statistical region combination.
Two removal hair approaches are applied by [20]. The first method is based on a simple morphological closing operation with a disk-shaped structural element while the top-hat transform combined with a bicubic interpolation used in the second approach.
The proposed algorithm by [21] divided into two stages, detection and removal. In detection, light and dark hairs and ruler marking are segmented through adaptive canny edge detector and refinement by morphological operators. In removal, the hairs are repaired based on multiresolution coherence transport inpainting.
In addition to the above mentioned hair removal methods, several aspects are captured in Table 1.
Table 1. Comparison of existing digital hair removal methods
Method Hair detector Inpainting method #test images
DullRazor [22] generalized morphological closing bilinear interpolation 5
E-shaver [23] Prewitt edge detector color averaging 5
Fiorese et al. [24] top-hat operator PDE-based [25] 20
Huang et al. [26] multiscale matched filters median filtering 20
Xie et al. [27] top-hat operator anisotropic diffusion [28] 40
Abbas et al. [6] derivative of Gaussian coherence transport [29] 100
Koehoorn et al. [16, 17] multiscale skeletons and morphological operators fast marching [30] s 300
Our method top-hat operator block-based histogram function & morphological close 200
2. The proposed technique
In this research, we propose a novel technique to remove hair pixels from dermoscopic images. The YIQ or NTSC color space is chosen because the hair pixels are well-demonstrated by only luminance (Y-channel) image, as for example, compared to RGB as shown in Fig. 2. In addition to RGB color space, the HSV and YCbCr color spaces present the hair pixels in more than one channel too. This issue complicates the hair removal task and may affects the performance.
Fig. 2. A digital dermoscopic image presented in RGB (a-d) and YIQ (e-h) color spaces
The Y-channel image is partitioned into 256 non-overlapped blocks. During experimental studies, several block sizes are tested such as 4*4, 8*8, 16*16, etc. We concluded that the implementation of block size 16*16 introduced better results for inpainting stage compared with other block sizes. For each block, morphological operators and histogram analysis are implemented to detect hair pixels and inpainting operation as well to replace hair pixels by non-hair skin pixels.
This section describes the proposed algorithm for an automatic hair detection and inpainting operations. To achieve the aims of this research, Fig. 3 describes the work mechanism and each step is described in the following subsections.
Fig. 3. Flowchart of the proposed method 2.1. Color space conversion As depicted in Fig. 4 the conversion operation from the input image (RGB) into YIQ color space.
Fig. 4. RGB (left) and YIQ (right) color spaces 2.2. Hair detection
To detect hair pixels, a morphological "bottom hat" operation is implemented on Y-channel image, returning the image minus the morphological closing of the image (dilation followed by erosion) to highlight dark hair on a light background as shown in Fig. 5. Because the image closing
expands the white areas in an image but does not significantly alter those areas which are already white, the only areas left after subtracting the original are those that were originally black but surrounded by white. In general, bottom hat filtering produces highlighted areas which more truly follow the shape of the hair. However, the main motivation behind utilizing a bottom hat filter is still the ability to better preserve the true shape of the hair.
Fig. 5. Hair detection, (a) Y-channel image, (b) result of bottom-hat operation
2.3. Binary image conversion As shown in Fig. 6 is the conversion operation of the image resulted from the previous step (repaired Y-channel) into binary image.
2.4. Inpainting operation Divide the repaired Y-channel and the binarized image into 256 non-overlapped blocks. During experimental studies, several block sizes are tested such as 4*4, 8*8, 16*16, etc. We concluded that the implementation of block size 16*16 introduced better results for inpainting stage compared with other block sizes.
Fig. 6. Result of the binarization operation
- For each block do
• Apply histogram function using 32 bins. The histogram function is imhist constructed from the image processing toolbox in the MATLAB software. The first parameter used is the sub-image of size 16*16 and the second parameter is the number of bins which is equal to 32 bins. Based on experimental studies, several number of bins tested and found that 32 bins is sufficiently utilized the intensity pixels ranged in [0, 1] into 32 intervals of size 0.0313 each. Furthermore, there were no improvements when number of binds was increased over 32 bins.
• Find the bin number that contains maximum occurrences (highest peak) of gray-scale pixels in each sub-image or block.
Let a = interval lower value and b = interval upper value.
Find locations of white pixels in binary subimage.
For each white pixel do
1. Generate a random number r in [0, 1].
2. Replace the pixel in the Y-channel by:
a + (b - a) ■ r (1)
where the purpose of having r is to keep a dynamic change in the repaired pixel value among all repaired pixels in each block. End.
Perform the morphological "close" operation (dilation followed by erosion) on repaired Y-channel image as depicted in Fig. 7b.
Fig. 7. Repaired Y-channel, (a) Y-channel before close operation, (b) Y"-channel after close operation 2.5. Repaired RGB image The result of the conversion operation from the repaired YIQ image to the RGB color space is depicted in Fig. 8.
Fig. 8. The repaired RGB image
3. Results and discussions The experiments are executed on processor Intel, core i3-2330M @ 2.20GHz and RAM 4GB. The system type is windows 7 ultimate of 64-bit OS and the software used for research implementation is MATLAB R2013 a.
The proposed methodology is tested on PH2 dataset [5]. It consists of 200 8-bit RGB dermoscopic images of melanocytic lesions with a resolution of 768*560 pixels. The dermoscopic images were obtained at the Dermatology Service of Hospital Pedro Hispano, Portugal under the same conditions through Tuebinger Mole Analyzer system using a magnification of 20*.
The efficiency of the proposed algorithm is the detection and removal of thin/thick and light/dark hair from dermoscopic images with the preservation of the texture
pattern, shape, and colors of skin lesion. Furthermore, any dermoscopic image does not contain hair, the algorithm preserves its features. Fig. 9 depicts the initial and last stages of our proposed algorithm.
Fig. 9. Sample of results
The statistical analysis based on the metrics of sensitivity, specificity and diagnostic accuracy was used to determine the performance of hair detection and inpainting operation. Our proposed algorithm reports a true positive rate (sensitivity) of 97.36 %, a false positive rate (fall-out) of 4.25 %, and a true negative rate (specificity) of 95.75 %. The diagnostic accuracy achieved is recorded at level high of 95.78 %.
To estimate the accuracy of the proposed algorithm and to quantify the automatic hair detection error, quantitative evaluations were performed using three statistical metrics: Sensitivity or True Detection Rate (TDR), Specificity or True Negative Rate (TNR), and Diagnostic Accuracy (DA). TDR measures the rate of pixels which were classified as hair by both the automatic algorithm and the medical expert, and FPR measures the rate of pixels which were not
classified as hair by both the automatic segmentation and the medical expert.
These metrics are calculated as follows:
TP
Sensitivity(TDR) =
Specificity(TNR) =
Fall - Out(FPR) =
FP + TN
Diagnostic Accuracy (DA) = TP + TN
-X100,
TP + FN
TN TN + FP FP
-x100,
x 100,
100,
(2)
(3)
(4)
(5)
TP + FN + FP + TN where TP, FP, FN, and TN stand for the number of true positive, false positive, false negative, and true negative respec-
tively. The quantitative results of the proposed algorithm are summarized in Table 2.
Table 2. Performance Evaluation (Confusion Matrix)
Count # Hair Pixels (Predicted)
Class = Yes Class = No
# Hair Pixels (Actual) Class = Yes TP 1924779 FN 52256
Class = No FP 3664600 TN 82521688
They were calculated as follows: • False Negative (FN): Find the differences between the repaired Y-channel (Y") and the original Y-channel, apply a binarization operation, and then count the white pixels. The results of these sequence of operations are depicted in Fig. 10.
Fig. 10. False negative calculation: (a, e) Y-channel; (b, f Y"-channel; (c, g) differences between (a, b) and (e, f illustrated by red dots; (d, g) Y-channel with false negative pixels represented by red dots
True Positive (TP): Apply the binarization operation on the hair segmented image (Y') yields to the hair segmented binary image (BW). Visually, it's better to represent the white pixels which are hair pixels in red color and black pixels for non-hair pixels in white background as shown in Fig. 11a, c. The white pixels exist in BW and not exist in the images shown in Fig. 10d, h are counted and preserved in another images as true positive pixels shown in Fig. 11b, d. True Negative (TN): Perform the complement operation on the hair segmented binary image (as shown Fig. 11a, c) yields to the images shown in Fig. 12, respectively. The TN is the count of the white pixels exist in the complement image. False Positive (FP): Count of the remained white pixels.
Unfortunately, up to the submission of the manuscript, we couldn't find a common database that can be shared with other researchers and there is no related work used PH2 dataset [5] to compare the proposed algorithm with others. However, Table 3 compares the proposed hair detection algorithm with some other methods.
Conclusion and future work
In this study, a fast and effective method is proposed for hair-occluded removal in dermoscopic images. The hair detection involved a morphological bottom-hat operation and the removal stage relied on the processes of the 256 non-overlapped blocks. Each block is processed by a histogram function followed by a morphological close operation. Our achieved results indicate high accuracy
and the proposed method can be dedicated to Dermat-logists as a pre-processing stage before the lesion segmentation and classification.
m Mm
illW
Fig. 11. True positive calculation. (a) Hair segmented binary image. (b) Truly classified hair pixels
Fig. 12. Sample of results Table 3. Comparison of the hair detection algorithms
Artifact Detection Method TDR (%) TNR (%) FPR (%) DA (%) # test images
The Pro-
posed Al- 97.36 95.75 4.25 95.78 200
gorithm
Multi-
Resolution 93.2 - 4 88.3 50
[21]
top-hat operator [27] - - - 72.5 40
DullRazor [22] 70.2 - 33.4 48.6 50
Fast Image Restoration
(FIR) + Line Seg- 98.27 93.75 96.10 299
ment De-
tection
(LSD) [15]
The following opportunities are suggested for future
work:
- Allocate a dataset to be common among researchers.
- Other artifacts such as air bubbles can be added for further studies.
References
[1] Iyatomi H. Computer-based diagnosis of pigmented skin lesions. In book: Campolo D, ed. New developments in biomedical engineering. Chap 10. "In Tech" Publisher; 2010: 183-200. ISBN: 978-953-7619-57-2.
[2] Garnavi R. Computer-aided diagnosis of melanoma. PhD thesis. Australia: "The University of Melbourne" Publisher; 2011.
[3] American academy of dermatology (AAD). Basal cell carcinoma. Source: (http://www.aad.org/skin-conditions/derma-tology- a-to -z/skin-c anc er).
[4] Zaqout I. Diagnosis of skin lesions based on dermoscopic images using image processing techniques. International Journal of Signal Processing, Image Processing and Pattern Recognition 2016; 9(9): 189-204. DOI: 10.14257/ij-sip.2016.9.9.18.
[5] PH2 Database. Source:
(http s:// www. fc.up. pt/addi/ph2%20database.html).
[6] Abbas Q, Celebi ME, Garcia IF. Hair removal methods: A comparative study for dermoscopy images. Biomedical Signal Process and Control 2011; 6(4): 395-404. DOI: 10.1016/j.bspc.2011.01.003.
[7] Somnathe PA, Gumaste PP. A review of existing hair removal methods in dermoscopic images. IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) 2015; 1: 73-76.
[8] Satheesha TY, Satyanarayana D, Giriprasad MN. A pixel interpolation technique for curved hair removal in skin images to support melanoma detection. Journal of Theoretical and Applied Information Technology 2014; 70(3): 559-565.
[9] Nasonova A, Nasonov A, Krylov A, Pechenko I, Umnov A, Makhneva N. Image warping in dermatological image hair removal. Image Analysis and Recognition (ICIAR 2014): 159-166. DOI: 10.1007/978-3-319-11755-3_18.
[10] Mahmood AF, Mahmood HA. Artifact removal from skin dermoscopy images to support automated melanoma diagnosis. Al-Rafadain Engineering Journal 2015; 23(5): 22-30.
[11] Das P, Gangopadhyay M. Removal of hair particles from skin disease images using pixel based approach. International Journal of Computer Science and Information Technologies 2015; 6(5): 4154-4158.
[12] Abbas Q, Garcia IF, Celebi ME, Ahmad W. A feature-preserving hair removal algorithm for dermoscopy images. Skin Research and Technology 2011; 19(1): e27-e36. DOI: 10.1111/j. 1600-0846.2011.00603.x.
[13] Mirzaalian H, Lee TK, Hamarneh G. Hair enhancement in dermoscopic images using dual-channel quaternion tubu-larness filters and MRF-based multi-label optimization. IEEE Transaction on Image Processing 2014; 23(12): 5486-5496. DOI: 10.1109/TIP.2014.2362054.
[14] Kretzler M. Automated curved hair detection and removal in skin images to support automated melanoma detection. Case Western Reserve University; 2013.
[15] Okuboyejo D, Olugbara O, Odunaike S. Unsupervised restoration of hair-occluded lesion in dermoscopic images. MIUA, Epham, UK, 2014; 91-96.
[16] Koehoorn J, Sobiecki A, Boda D, Diaconeasa A, Doshi S, Paisey S, Jalba A, Telea A. Automated digital hair removal by threshold decomposition and morphological analysis.
12th International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing ISMM 2015, Reykjavik, Iceland, 2015: 15-26. DOI: 10.1007/978-3-319-18720-4_2.
[17] Koehoorn J, Sobiecki A, Rauber P, Jalba A, Telea A. Efficient and effective automated digital hair removal from dermoscopy images. Math Morphol Theory Appl 2016; 1(1): 1-17. DOI: 10.1515/mathm-2016-0001.
[18] Zhou H, Chen M, Gass R, Rehg JM, Ferris L, Ho J, Drogowski L. Feature-preserving artifact removal from dermoscopy images. Proc SPIE 2008; 6914: 69141B. DOI: 10.1117/12.770824.
[19] Hearn J. Competitive medical image segmentation with the fast marching method, Case Western Reserve University; 2008.
[20] Sultana A, Ciuc M, Radulescu T, Wanyu L, Petrache D. Preliminary work on dermatoscopic lesion segmentation. Proceedings of the 20th European Signal Processing Conference (EUSIPCO) 2012; 2273-2277.
[21] Toossi MT, Pourreza HR, Zare H, Sigari MH, Layegh P, Azimi A. An effective hair removal algorithm for dermoscopy images. Skin Research and Technology 2013; 19(3): 230-235. DOI: 10.1111/srt.12015.
[22] Lee T, Ng V, Gallager R, Coldman A, McLean D. DullRa-zor®: A software approach to hair removal from images. Computers in Biology and Medicine 1997; 27(6): 533-543. DOI: 10.1016/S0010-4825(97)00020-6.
[23] Kiani K, Sharafat AR. E-Shaver: An improved DullRazor® for digitally removing dark and light-colored hairs in der-moscopic images. Computers in Biology and Medicine
2011; 41(3): 139-145. DOI: 10.1016/j.compbio-med.2011.01.003.
[24] Fiorese M, Peserico E, Silletti A. VirtualShave: automated hair removal from digital dermatoscopic images. 33rd Annual International Conference of the IEEE EMBS 2011; 5145-5148. DOI: 10.1109/IEMBS.2011.6091274.
[25] Bertalmio M, Sapiro G, Caselles V, Ballester C. Image inpainting. Proceedings of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH '00) 2000; 417-424. DOI: 10.1145/344779.344972.
[26] Huang A, Kwan S, Chang W, Liu M, Chi M, Chen G. A robust hair segmentation and removal approach for clinical images of skin lesions. 35th Annual International Conference of the IEEE EMBS 2013; 3315-3318. DOI: 10.1109/EMBC.2013.6610250.
[27] Xie FY, Qin SY, Jiang ZG, Meng RS. PDE-based unsupervised repair of hair-occluded information in dermoscopy images of melanoma. Computerized Medical Imaging and Graphics 2009; 33(4): 275-282. DOI: 10.1016/j.compmedimag.2009.01.003.
[28] Perona P, Malik J. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 1990; 12(7): 629-639. DOI: 10.1109/34.56205.
[29] Bornemann F, MSrz T. Fast image inpainting based on coherence transport. Journal of Mathematical Imaging and Vision 2007; 28(3): 259-278. DOI: 10.1007/s10851-007-0017-6.
[30] Telea A. An image inpainting technique based on the fast marching method. Journal of Graphics Tools 2004; 9(1): 25-36. DOI: 10.1080/10867651.2004.10487596.
Author's information
Ihab Salah Zaqout (b. 1965) holds a PhD degree from University of Malaya, Malaysia since 2006 and currently is an associate professor in Computer Science working of Information Technology department, faculty of Engineering and Information Technology, Al-Azhar University, Gaza, Palestine. Research interests are digital image processing and data mining. E-mail: [email protected] .
Received May 20, 2017. The final version - June 27, 2017.
