CC BY
28
HAND GESTURE RECOGNITION USING MORPHOLOGICAL OPERATION
Manisha H. Devnani, Dr. Udesang K Jaliya
Birla Vishvakarma Mahavidyalaya, Gujarat. udesang.j aliya@bvmengineering. ac.in
Abstract
A nonverbal communication involves movements of different body parts to communicate a particular message. Hand, face and lips movements or other movement of body parts are regarded as gestures. In this paper I have proposed a method based on hand gesture recognition to recognize the different gestures used by deaf people to communicate using morphological operation. This will help to communicate between deaf and dumb people and normal public. In earlier systems the use of colour markers and gloves for gesture recognition has been used but it resulted in delay in processing time and sometimes inconvenient for the user.
Keywords:image recognition; image processing; image acquiring; morphological operations; Human computer interface
I. Introduction
The People who cannot speak or hear use sign language to express the thoughts and communicate with the common people or among them. Sign language (Cowie, 2017) includes movement of different parts like the hand, lips, mouth and many others which are referred to as gestures. But the challenge of the people who are deaf and dumb is the sign they use to speak are often not understandable by common people who does not have proper knowledge of different sign they use to communicate. It becomes a problem for this disabled people to communicate with others. To resolve this problem often a translator is used who has better knowledge of the sign language but that becomes very costly and often getting a person who has knowledge of that specific sign language is not an easy job which is another problem. In this paper I have tried to resolve this issue by proposing a method which will be cost effective, easily accessible and won't be time consuming. The work flow of hand gesture recognition is described as follows. First the hand area is detected from the gesture of the hand that will be captured. Then some features are extracted to describe the hand gestures. Lastly the gesture of hand is recognized by the dataset. The camera detects the hand area but if the area of skin and hand is the same the camera cannot detect the gesture properly. The hand gesture recognition system involves use of color marker gloves (Chaudhary, Raheja, Das, & Raheja, 2014), but this often results in inconvenience for the user. I have focused on developing a system without using color markers or gloves which will be more convenient. I used mathematical operations (Alvarez, Baumela, Marquez-Neila, & Henriquez, 2012) and each step has been defined accurately so that some error occurred during the program or in the process of the recognition which will be a more convenient system for the user. This system reduces the processing time and enhances the accuracy. The development of human computer interface (HCI) (Bevan, 1995) (Karray, Alemzadeh, Saleh, & Arab, 2008) has also been done for virtual recognition of gestures. Gestures are non-vocal ways of communication like posing a
24
victory sign from fingers in front of a smartphone camera for clicking photos.
II. Methods Overview of the Hand Recognition Method:
The overview of the hand recognition is described in Figure 1. First the webcam captures the gesture of the hand using background subtraction. Then the captured image is converted into a binary image. The morphological operations are applied and then the image is flipped so that the fingers can be detected.
( \ Hand Detection
V_J
Hand Detection:
The process of hand detection has two parts. First the background image is captured and in the second process the gesture image is captured using the webcam. The two different images are then processed with the help of background subtraction (Vasavi & Sanku, 2013) and thus the processed image is formed. The subtraction of an image is important because the background of the image is identical. However, in some cases, there are other moving objects included in the result of background subtraction. The skin colour can be used to discriminate the hand region from the other moving objects. The Figure 3 of the background image subtraction is shown below:
Figure:3 Background Image Subtraction
Pre-Processing of the image:
The pre-processing of the image takes place after the image background subtraction. There are four steps containing the pre-processing of the image. The explanation of each steps are as follows:
Formation of GreyScale image:
Background Subtraction
Flipping of image
Gesture Recognition
Figure 1: Flowchart of the Hand Gesture Recognition
Figure:2 The process of Hand Gesture Recognition.
After the formation of the subtracted image this is converted into the Gray scale image. That is the image is processed through the grey scale and is converted to black and white image. This is helpful for the computer to recognize the finger and palm region correctly. There are two ways in which we can convert an image into the Gray scale form. They are:
a. Average Method
b. Weighted Method or luminosity method
The equation of converting the image into Gray scale is shown below: New grayscale image = ((0.3 * R) + (0.59 * G) + (0.11 * B)) The Figure 4 of the binary image is shown below.
Grayscale
1 »
/ -J /
r
Figure: 4 Conversion of Image into GreyScale.
Black and White image:
Now the image is converted into the binary image or also we can say that the image is converted into black and white image. The morphological operation is applied in this process and thus the processed image is formed. The equation of the conversion of the image into binary or black and white image is as follow:
BW=imbinarize(X,cmap,level)
Black&White
Figure: 5 Black and White image. Erosion, Dilation and Median Filter:
In this method of hand gesture recognition, the process of erosion is used to shrink the image pixels so that the pixels which are located in the boundary can be removed.
The process of dilation in the hand gesture recognition is used to convert images in an expanded form. The image is dilated and eroded because the result of the image produced is better for the result. The median filter is also known as nonlinear filtering. It is used to eliminate salt and pepper noise. Here the pixel value is replaced by the median value of the neighboring pixel. The Figure 6 below shows the formation of the processed image.
Eroded,Dilated & Median Filtered
Figure: 6 Erosion, Dilation and Median applied to an image.
Flipping of the image:
In the process of hand gesture recognition, the last step is the flipping of the image. The image is flipped to get the better result in the process. The Figure 7 below shows the output result of the flipping of an image.
Flip Image
Figure:7 Flipping of the Image.
III. EXPERIMENTAL RESULTS
In the experiment, to obtain the accurate result I obtained the result using different hand gestures at different angles. This will help us to obtain the more accurate result and will be helpful for the user to use it properly. I have shown the four different gestures of an image that have been used by me as a database. The result obtained is of four different structures. The Figure 8 below shows us the experiment result of hand gesture recognition.
Figure: 8 The input of the Hand Gesture.
Figure: 9 The corresponding output of the Hand Gesture.
IV. APPLICATION, ADVANTAGES AND LIMITATIONS
A. Applications
1. It can be used by deaf and dumb people for communication with the normal people.
2. It can be used in smart classes for real time translation of language where deaf and dumb students can be taught.
3. It can be used in computer games as a robust system to help its translation for the people who are mute.
4. It can also be used to classify the number. Which means that when the hand gesture creates a number the output will be produced respectively.
B. ADVANTAGES
1. The result of hand gesture recognition gives the result in an appropriate time.
2. The time taken is less than the other processes that have been made.
3. Takes low power consumption.
4. Automated results are obtained.
5. An effective way of communication for physically challenged people.
6. Takes less time as compared to others.
C. LIMITATIONS
1. The intensity of the light plays an important role in this project. If the colour of skin and background is the same there would be a problem.
2. The input image should be properly taken.
V. CONCLUSIONS
The hand gesture recognition proposed in this paper uses a background subtraction method. This method reduces the delay of time and is more helpful and understandable for the user to use this project. The hand gesture recognition forms the bridge between the physically disabled person and the normal person.
References
[1] R. Harshitha, I. A. Syed, and S. Srivasthava, "Hci using hand gesture recognition for digital sand model," in Proceedings of the 2nd IEEE International Conference on Image Information Processing (ICIIP '13), pp. 453-457, 2013.
[2] Processing (ICIIP '13), pp. 453-457, 2013. [2] Real-Time Hand Gesture Recognition Using Finger Segmentation Zhi-hua Chen,1 Jung-Tae Kim,1 Jianning Liang,1 Jing Zhang,1,2 and Yu-Bo Yuan 1 Received 24 April 2014; Accepted 29 May 2014; Published 25 June 2014
[3] Akshay Dekate; Anam Kamal; K. S. Surekha " Magic Glove - wireless hand gesture hardware controller" : 2014 International Conference on Electronics and Communication Systems (ICECS).
[4] Gurwinder Kaur* and Gourav Bathla "Hand Gesture Recognition based on Invariant Features and Artificial Neural Network" Indian Journal of Science and Technology, Vol 9(43) November 2016
[5] Remimol.A.M, Sekar.K. "A Method of DWT with Bicubic Interpolation for Image Scaling": International Journal of Computer Science Engineering (IJCSE), Vol. 3 No.02 Mar 2014.
[6] M.A.Nuno-Maganda, M.O.Arias-Estrada "Real-time FPGA-based architecture for bicubic interpolation: an application for digital image scaling" 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05)
[7] Xudong Kang; Shutao Li; Jianwen Hu " Fusing soft decision adaptive and bicubic methods for image interpolation" Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).
[8] Yu Huang; T. S. Huang; H. Niemann "Two-handed gesture tracking incorporating template warping with static segmentation " :Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition
[9] Asongu L. Tambo; Bir Bhanu "Dynamic bi-modal fusion of images for the segmentation of pollen tubes in video": 2015 IEEE International Conference on Image Processing (ICIP).
[10] Manar Maraqa, Raed Abu-Zaiter. (2008). "Recognition of Arabic Sign Language (ArSL) Using Recurrent Neural Networks," IEEE First International Conference on the Applications of Digital Information and Web Technologies, (ICADIWT), pp. 478-48. doi: 10.1109/ICADIWT.2008.4664396.
[11] Tin Hninn H. Maung. (2009)."Real-Time Hand Tracking and Gesture Recognition System Using Neural Networks," World Academy of Science, Engineering and Technology 50, pp. 466- 470
