CC BY
75
The Impact of Implementing the Gamification Method in Learning Indonesian Sign Language with Bisindo Vocabulary
Kevin Hosea Widjaya, Wirawan Istiono
Abstract—Human is a social being, where humans need to socialize with the other humans. One of the interactions that do by humans is speaking. When humans speaks, humans use a language because it's one of the important roles in communicating, but some humans are born with special needs where they can't hear which commonly called deaf. In communicating, they cannot hear voices, which makes it difficult to communicate with the other people. Therefore, deaf people are using sign language to communicate, but not many man can communicate with sign language, to can communicate with deaf people, it's very important to know the meaning of the sign language. This study aims to build a sign language mobile application which uses BISINDO (Indonesian Sign Language) to can be learned by beginners who want to learn sign language to can communicate with the deaf people with method learning sign language with basics of BISINDO vocabulary. Also, in creating the applications, this application using Six Steps to Gamification method as reference for application, to make this game fun to be learned by deaf people or by normal peoples. Application testing was conducted on 30 people and the survey was based on Hedonic Motivation System Adoption Model (HMSAM). The evaluation results show that 88.22% of respondents stated that they were helped and will use this application to learn basic sign language, and 82% of respondents stated that they liked the sign language learning application with BISINDO basic vocabulary and gamification methods.
Keywords—Bisindo, Gamification, Sign Language, Mobile Application, Education Application.
I. Introduction
Human as social beings need to communicate and interact with other persons because human need each other, in other word, humans cannot live alone, therefore humans have social needs like interacting with one another. Language has an important role in interacting with each other because language is a tool for communicating with one another. Language is divided into two things, such as, written language and spoken language [1], both of these have a function to communicate which is used for social interaction. But, there is some people are born with special needs where they have difficulties to hear or speech like a hearing impairment or speech impaired. Hearing impairment commonly called deafness. In terms of communicating, the deaf is having difficulties to communicate with each other, these difficulties are very influential in social relations
between each other, therefore the deaf need a sign language to help communicate with each other. In the development of this industry, most individuals have smart phones which are used as learning media which is commonly called Mobile Learning (M-learning). M-learning has an important role in the learning component where students can access, collaborate, and share ideas with each other [2]. There is also a similar study related to the application of the Indonesian language studied by Fadhlina Nor Fahira and Nor intan entitled "I-Sign Language Learning Application via Gamification" [3] in their research has a core that is more or less different from this research that they are using SIBI (Indonesian Sign Language System) in their application.
There are another study conducted by Riadi and Patra [4] stated in their research that SIBI cannot be used in the daily communication of the deaf because the SIBI language is too difficult to use and the placement of vocabulary is not in accordance with the aspirations and conscience of the deaf. There is also a study conducted by Rohmah Ageng [5] regarding the response of the deaf to the use of SIBI and Bisindo (Indonesian Sign Language) in communication, in the study there is a graph which states that Bisindo is more friendly to use and is more often used by deaf person to communicate on a daily basis. Therefore, in this research will used use Bisindo. Therefore, if you want to be able to understand Indonesian sign language, it is better to learn the basics of sign language first. For this application, the test uses the Hedonic Motivation System Adoption Model (HMSAM) method. Because this HMSAM focuses more on fun than productivity, this method is used for testing this application [6]. HMSAM can measure whether the user will use this application again or not. Several aspects were chosen because an application that uses the gamification method aims to motivate or interest users to learn while helping users get carried away when using the application. The framework used in this research to build and create an application is Six Steps to Gamification. Wono [7] wrote that the framework is a general gamification framework that has the most complete features. So from the existing problems, an application will be created and built using the Six Steps to Gamification framework to teach the wider community to learn Indonesian sign language (Bisindo). Then it will be assessed the level of user aspects in using the application.
II. Literature Study
III. Research Methodologies
Sign language is the language used daily by deaf people. In general, sign language is studied by deaf people in special schools. In Indonesia there are two sign languages that are used, namely SIBI (Indonesian Sign Language) and Bisindo (Indonesian Sign Language), the difference from SIBI (Indonesian Sign Language) and Bisindo (Indonesian Sign Language) is that BISINDO is easier to apply everyday because Bisindo is the mother tongue and natural sign language of indigenous Indonesian culture [4]. Meanwhile, SIBI is a sign system that is recognized by the government and is used in teaching at Special Schools for the Deaf (SLB/B). The main difference between SIBI and BISINDO lies in the language procedure. BISINDO which has been used daily by deaf peoples contains a symbolic vocabulary of signs. As long as the meaning of a word is represented, then BISINDO's simple sign vocabulary is sufficient. Meanwhile, SIBI was created to teach the Indonesian system to deaf peoples. The rules for using sign vocabulary in SIBI are more complicated and tiered, such as the use of prefixes and suffixes for each word. In SIBI, a lot of sign vocabulary has been adapted from American Sign Language, so that the context is not suitable for deaf friends. One of the most visible differences between BISINDO and SIBI is that BISINDO uses two hands to indicate the alphabet, while SIBI only uses one hand [10].
Figure 1, Sign language Alphabet hand pattern in Bisindo (source: ypedulikasihabk.org)
In Figure 1 [10] shown an Indonesian sign language by Bisindo vocabulary that explains about the movement of the hand pattern of the alphabet from a to z. In making the Indonesian sign language application using a companion dictionary that has been made by pusbisindo which is the selection of the most basic vocabulary for learning sign language [11]. BISINDO are not limited to being used only for deaf peoples, but can also be used by hearing friends or people who do not have hearing or speech impairments. Normal peoples can also learn it so that they can communicate widely with deaf peoples [10].
There are some methodologies that being used on this research, first step is literature study, at this step, the retrieval of information needed to meet research needs is carried out. Information related to gamification and Indonesian sign language or Bisindo. The second step is planning, at the system design stage, it consists of 3 activities, including making application assets such as icons, buttons, animations and others, designing game mechanics in applications such as application game rules and creating a game system for the application. And the next step is create the application, this step is the process of making Indonesian sign language learning applications using the gamification method. Activities that will be carried out in this stage are making displays, functions, buttons that have been made in the system design stage and using the Unity application. The next step is testing the application, at this stage, the application that has been made will be tested with target user are children aged seven years old and over. And after the testing step, the next step is evaluation, where in this step, will do evaluation Indonesian sign language learning applications using the gamification method where the evaluation is obtained after carrying out the testing process and conducting a survey of application usage. And the final step is writing a report, to record the results that obtained from the survey and application testing that has been carried out.
System planning is consisted by gamification planning, application model and making an assets. In the process of building an application, Werbach using 3 things in pyramid there are dynamics, mechanics and components. Dynamics is a description of the aspect that you want to gamify for example emotion, progression and others. Mechanics is the explanation how the system works to gain a feedback from user that using the application in the form of engagements like competition, rewards, feedback and many more. Components is the representation of dynamic and mechanics in the system like there are levels, leaderboard and score. In the define step, this application is named "Bisindoku" where this application aims to give access to people who have never studied Bisindo or Indonesian Sign Language. Besides being able to learn, users will be able to play at the same time. In addition, the purpose of making this application is expected to increase enthusiasm and interest in learning Bisindo for those who do not know it at all. In this application there is a mechanic in the form of a quiz in the form of a quiz where users can take a test in the form of a quiz where in the quiz the user will get a reward in the form of a score that will be entered into the database if the user's score is the best three in the database then the user's score will be displayed on the database. Scoreboard therefore it increases competitiveness and motivates users to use the application besides that there is a time limit and life while taking the quiz.
This application targets users among children over the age of seven years old, because this application has two parts, namely learning and trying tests where users are expected to learn and take tests so that points are earned and appear on the leaderboard. If the test fails then the user must repeat the
test again for the next level. In learning mode the user can access the images presented at each level respectively. With the system created, it is hoped that users can learn Bisindo and understand the steps to be able to understand Bisindo well and at each level of the test such as level 2 it can only be accessed when completing level 1 and level 3 will be opened after the user has completed level 2. This application is targeted for users over the age of 7 years old because users are expected to be able to spell and understand words. This application is limited because the choice of Bisindo vocabulary is only as far as the scope of communicating in the family.
This application also uses score and leaderboard to motivate users to compete for a higher score where these two elements affect the engagement loop and stairs progression. This application uses the score and leaderboard to motivate users to compete for a higher score where these two elements affect the engagement loop and the stairs progression. Here's the explanation. Progression Stairs where if the user completes a level then the user will have the right to continue to the next level which was previously locked. The engagement loop given in this application is that every question answered correctly will get a score where the score is feedback from the application and game over when wrong. According to the Framework explanation that is calculated in the "fun" category, there are 4, the following are the "fun" aspects. "Hard fun": gives a score when successfully guessing Bisindo's picture and opens a new level when completing another level."Easy fun": there is a quit button if the user is reluctant or confused during the quiz. "Altered states": The image provided has a special symbol that distinguishes the movement of the hand. "The people factors": use the leaderboard to experience competition between users. To realize this application, the development of this application is limited to the android platform. The tools used are Visual Studio Code, Firebase and Unity.
Figure 2. Main Flowchart Belajar Sign Language Bisindo
In Figure 2 shown the main flow of the Bisindo Sign Language Learning system, where the main flowchart above explains the flow of the system that works when the user selects the menu, either level1 or level2 and level3 menus. Where level 2 and level 3 will be able to be played when the user has completed the game at the previous level, this is intended so that the user can understand the basic sign language material first before moving on to more difficult learning materials. In the main flowchart it is also explained that the user can see the user menu profile with scores each level, besides that there is also a logout menu which is useful for changing profiles if the user wants to learn sign language material with a different profile.
Figure 3. Flow Gameplay Level 1
Figure 3 shown a flowchart module from game level 1, where the system will ask questions in the form of pictures. On that page, there is a button that is useful for filling in the question box in the form of an array and each button has its own value. If the user presses the alphabet button, the answer will be assembled into a word that was previously only one character. If the length of the answer has not been met answer criteria, the user must press the alphabetical button until the length of the answer is met. If the user has met the length of the answer to the question, the system will check whether the answer array is match with the answer key, if the result is matching, the user will get a score of 10 points. But if not match, then the life point decrease by one and score will be deducted by 10 points. However, if the answer is wrong and the life reach zero, the user fails to complete the level, but if the life is greater than zero, the system will continue to the next question. If the questions run out, the results that obtained will be entered into the database and directed to the next page.
Figure 4. Flow Gameplay Level 2
Figure 4 shown the load module level two, where at the second level, the system will display an image in the form of buttons, where each button will have a boolean value, namely true and false. The user will be given an initial score of zero, and has two chance and also a time of 10 seconds. The time will be counted down to zero, where if the time has reached zero or the question has run out, then the total score that obtained by the player will be inserted into the database and the system will give two options, namely repeating or returning to the home module. If the user chooses to repeat, then the second game load module will be repeated and the game will start from the beginning again. However, if the answer is correct, the user will get a score of 20 and the system will shown the next question.
y
score+=10, ¡+=1 Question(i)
Figure 6. Login and Register
Figure 6 shown the screen of the program when the application Bisindoku was launched. To be able to start the application, the user must register first in the register screen and then after register, user can login to enter the application. User registration is used to store data and scores for each player.
Figure 5. Flow Gameplay Level 3
Figure 5 shown the load module level three, where the players will be presented with questions in the form of pictures and players will see the gameplay is not too different with level one, but in level three, difficulty level will be increase, where at this level there are only two lives and a timer is also added to increase the level difficulty.
After designing the Bisindo-based sign language learning application, the next stage is making the application, where the gamification element will be added to the application. In Figures 6 to Figure 10, several screenshots of the results of the Bisindo-based sign language learning application development are shown, which will be used as a tool to measure the impact of sign language learning using applications with Bisindo vocabulary and gamification methods.
{R-llJt-.') [ J ftiM'J [JKMJ [OflBi'iJ ;t«i>sj
\äl k
Figure 7. Home Menu and Learning Page
Figure 7 shown the main page if the user successfully logged in. In main page, there are two button to direct the user to learn or plays, in button "belajar" or "bermain". Also in main page, there are three buttons to learn Bisindo vocabulary, such as "abjad", "panggilan" and "waktu", each button will redirect the user to different Bisindo vocabulary. The last buttons is profile button, which will redirect the user to their profile. In Learning page in button "belajar" and when user choose "panggilan" button in home menu, learning page section will be shown, where in this page user can pressed the many word Bisindo vocabulary button like "saya", "dia" and many more, when user click, there is a hand gesture image or sign language on the learning pages according to Bisindo dictionary. When user choose the "bermain" button. there are three buttons level will shown, such as "level 1", "level 2", and "level 3". In page "bermain", the levels button will shown depending playing progress user, there is three levels on each
gameplay, and the button levels will appear when the user successfully to finish the previous level, the higher level the challenge will become more difficult.
the questions that appear. If the user succeeds in answering the questions that appear, the popup menu score will be shown with the score that user get, but if the user answers the questions incorrectly until the lives given are zero, a gameover popup menu will appear.
Figure 8. Level 1, Level2 and Level3 page
Figure 8 shown the page for each level, in this pages, there are differences in each exercise depending on the type of menu selected by the user. When the user selects the alphabet or "abjad" menu, it will show menu alphabet questions, in this menu, the user will be asked to choose the right letter according to the hand image that appears, while on the "panggilan" menu option, a basic sign language question will appear for calling someone. While in the "waktu" game, the user will be asked to write a word according to the image that appears on the screen. In each type of exercise menu, a "lives" gamification element is adding, to give a chance, if the user give the wrong answer. Also in a view types menus and level there are more gamification elements, such as timers, to give more challenges to the user to answering the questions quickly and accurately.
Figure 9. Scoring and Game Over
Figure 9 shows an additional popup menu that will appear when the game is over and the user successfully answers the questions correctly or the user fails to answer
Figure 10. profile page and Leaderboard page
Figure 10 is profile page if the user choose the profile button on main page and it will be show the user their username and their unique id which was "UID". The leaderboard button will redirecting user to leaderboard page, in leaderboard or scoreboard page shown the top three user that have the highest score on database.
The application trial was carried out using a google form with questionnaire questions given based on the Hedonic-Motivation System Adoption Model (HMSAM). Application testing is done by distributing the application apk to the surrounding environment such as social media (discord and line), churches and others. The user will try the application through an application that has been installed on the user's mobile phone that has been assisted by the researcher or the researcher's mobile phone because during the survey taking process some users do not use an Android-based mobile phone. If you have finished using the application, the user will fill out the google form that has been given.
IV. Result And Decision
These are the number of data that have been gained using questionnaire and calculated using equation formulas. The application trial was carried out using a google form with questionnaire questions given based on the Hedonic-Motivation System Adoption Model (HMSAM).
Table 1. Interval Result Categories
Interval Categories
0%-20% Strongly disagree (STS)
>20%-40% Disagree (TS)
>40%-60% Neutral (N)
>60%-80% Agree (S)
>80%-100% Strongly Agree (SS)
Table 1 is a list or list that shows the value intervals and categories of google form answers. The interval has 5 sections, namely 0%-20% in the disagree category, >20%-
40% in the disagree category, >40%-60% in the neutral category, >60%-80% in the agree category,>80%- 100% with the category strongly agree.
Table 2. Google Form Questionnaire Answers_
Immersion 82%. The following is the final result of the percentage and classification of applications obtained with the HMSAM aspect.
Table 5 Final Calculations
STS
TS
N
I3
I6
SS
I5
II
Aspects
Perceived usefulness
Perceived ease of use
HMSAM Aspect
Perceived usefulness
Perceived ease of use
Curiosity
Joy
Presentase dan klasifikasi
88% (SS)
84,67% (SS)
78% (S)
80,67% (S)
I4
Curiosity
Control
87,33% (SS)
I3
Joy
Behavioral Intention to use
82,22% (SS)
I3
I4
Control
Immersion
82% (SS)
I3
Behavioral Intention of use
I2
Immersion
Tabel 2 is the result of the answers to the questionnaire survey from the google form after the user uses the application. The seven points are surveys built on the basis of HMSAM. The age of the respondents varied from 17 years to the age of 40 years and the number of respondents was 30 people, but in the process of using the application there were no children as users. The data collection was answered from December 24 to December 28, 2021.
Tabel 3. Calculation of HMSAM using Likert Scale
HMSAM Aspects Likert Scale Precentagee and classification
Perceived usefulness 0,88 88% (SS)
Perceived ease of use 0,846666667 84,67% (SS)
Curiosity 0,78 78% (S)
Joy 0,806666667 80,67% (SS)
Control 0,87333333 87,33% (SS)
Behavioural Intention to Use 0,78 78% (S)
Immersion 0,76 76% (S)
Table 3 are the results of the calculation of the seven aspects of HMSAM that have been obtained through the google form. The average percentage of the seven aspects is added first and divided by seven ((88% + 84.67% + 78% + 80.67% + 87.33% + 78% + 76%)) / 7) produces an average an average of 82% where the application received by the user is very positive.
Table 4. Calculating using HMSAM relatives
HMSAM Aspects Calculation Percentage and Classification
Behavioral Intention to Use 0,8222222 82,22% (SS)
Immersion 0,82 82% (SS)
Table 4 is the result of HMSAM relatives, when compared with Table 3 there is a difference in results of 4%-8%, where from the results of the two tables in the BIU and Immersion aspects the results are taken from Table 4 because the percentage from Table 4 is greater than Table 3 where the percentage of BIU in Table 4 is 82, 22% and
Table 5 is the final calculation of the seven aspects of HMSAM, the largest percentage of aspects obtained is Perceived usefulness with a percentage value of 88% where users feel that this application is important. The lowest aspect value of the seven aspects is curiosity with a percentage value of 78% where users feel quite curious to know more with further Indonesian sign language lessons.
V. Conclusion
Based on the research that has been done, some conclusions can be drawn as follows; The Indonesian sign language learning application using the gamification method has been successfully designed and created. In the process of making this application using Unity software, Visual Studio Code, and Windows using the C # programming language and using the Six Steps to Gamification framework..
This application has been tested for its usefulness and has been evaluated by 30 (thirty) users of the Indonesian sign language learning application using the gamification method which is built based on the Hedonic Motivation System Adoption Model (HMSAM) model. The results of a survey conducted through filling out Google forms where the percentage value of 82.22% in the behavioral aspect of intention to use from the survey results show that users have the intention to use the application in the future and users are carried away by the atmosphere in the Immersion aspect, the percentage is 82%.
Acknowledgment
Thank you to the Universitas Multimedia Nusantara, Indonesia which has become a place for researchers to develop this journal research. Hopefully, this research can make a major contribution to the advancement of technology in Indonesia.
References
[1] F. Luis and G. Moncayo, "DEVELOPING WRITING MATERIALS FOR DEAF STUDENTS," Journal of English Education, Literature and Culture, vol. 2, no. 2, pp. 415-430, 2017.
[2] A. M. Rekkedal, "Assistive hearing technologies among students with hearing impairment: Factors that promote satisfaction," Journal of Deaf Studies and Deaf Education, vol. 17, no. 4, pp. 499-517, 2012.
[3] P. H. Prastyo, S. E. Prasetyo, and S. Arti, "A Sign Language Prediction Model using Convolution Neural Network," IJID
S
0
I
I
2
0
I
2
I
I
6
8
0
0
8
9
0
0
3
I
0
8
8
0
2
9
7
(International Journal on Informatics for Development), vol. 10, no. 1, pp. 47-52, 2021.
[4] M. L. Hall, W. C. Hall, and N. K. Caselli, "Deaf children need language, not (just) speech," First Language, vol. 39, no. 4, pp. 367-395, 2019.
[5] A. Van Staden, G. Badenhorst, and E. Ridge, "The benefits of sign language for deaf learners with language challenges," Per Linguam, vol. 25, no. 1, 2011.
[6] K. N. Andrew, J. Hoshooley, and M. F. Joanisse, "Sign language ability in young deaf signers predicts comprehension of written sentences in english," PLoS ONE, vol. 9, no. 2, 2014.
[7] M. Al-Emran, H. M. Elsherif, and K. Shaalan, "Investigating attitudes towards the use of mobile learning in higher education," Computers in Human Behavior, vol. 56, pp. 93-102, 2016.
[8] A. Riadi and P. Aditia, "Buku Ilustrasi Bahasa Isyarat Indonesia (Bisindo) Bagi Anak Tunatungu Illustrated Book Of Indonesian Sign Language For Deaf Children," Metode, 2007.
[9] R. A. Mursita, "Respon Tunarungu Terhadap Penggunaan Sistem Bahasa Isyarat Indonesa (Sibi) Dan Bahasa Isyarat Indonesia (Bisindo) Dalam Komunikasi," Inklusi, vol. 2, no. 2, p. 221, 2015.
[10] A. Baker, B. van den Bogaerde, and B. Woll, "Methods and procedures in sign language acquisition studies," Sign Language & Linguistics, vol. 8, no. 1-2, pp. 7-59, 2005.
[11] F. I. Saman, N. F. M. Shariff, and N. I. S. Nasaruddin, "i-Sign: Sign language learning application via gamification," Asian Journal of University Education, vol. 15, no. 3, pp. 187-197, 2019.
[12] Hartelina, R. L. Batu, and A. Hidayanti, "What can hedonic motivation do on decisions to use online learning services?," International Journal of Data and Network Science, vol. 5, no. 2, pp. 121-126, 2021.
[13] P. B. Lowry, J. E. Gaskin, N. W. Twyman, B. Hammer, and T. L. Roberts, "Taking 'fun and games' seriously: Proposing the hedonic-motivation system adoption model (HMSAM)," Journal of the Association for Information Systems, vol. 14, no. 11, pp. 617-671, 2013.
[14] D. Lie, J. C. Young, and S. Hansun, "Dietary application for diabetic patients using gamification method," International Journal of Scientific and Technology Research, vol. 9, no. 4, pp. 24172423, 2020.
[15] A. W. Istiono and A. Waworuntu, "INTERNATIONAL JOURNAL OF ADVANCED STUDIES IN COMPUTER SCIENCE & ENGINEERING What element that influence preschool and elementary school children to enjoy playing education games ?,"
International Journal of Advanced Studies, vol. 9, no. 12, pp. 9-13, 2021.
[16] S. A. Nurpiena, E. S. Wihidayat, and A. Budianto, "Developing Indonesia Sign Language (BISINDO) Application with Android Based for Learning Sign Language," Journal of Informatics and Vocational Education, vol. 4, no. 1, 2021.
[17] A. R. Syulistyo, D. S. Hormansyah, and P. Y. Saputra, "SIBI (Sistem Isyarat Bahasa Indonesia) translation using Convolutional Neural Network (CNN)," IOP Conference Series: Materials Science and Engineering, vol. 732, no. 1, 2020.
[18] T. Handhika, R. I. M. Zen, Murni, D. P. Lestari, and I. Sari, "Gesture recognition for Indonesian Sign Language (BISINDO)," Journal of Physics: Conference Series, vol. 1028, no. 1, pp. 0-8, 2018.
[19] H. Bestari, "Mengenal Bahasa Isyarat," Peduli Kasih ABK, 2018. [Online]. Available: https://www.ypedulikasihabk.org/2018/11/09/mengenal-bahasa-isyarat/. [Accessed: 25-Dec-2021].
[20] H. kevin werbach, For the Win. Philadelphia: Wharton Digital Press, 2015.
Kevin Hosea Widjaya was born in South Tangerang, Indonesia. He received Bachelor's degree in Infomatics from Multimedia Nusantara University. His focus research in Information technology, Gamification, Information System and Artificial Intellegence. Now, he works as fullstack intern developer specialized in flask (python) and CodeIgniter in PT. Global Loyalty Indonesia.
Wirawan Istiono was born in Jakarta Indonesia, he received
his Masters of Computer Science degree in Budi Luhur University, Indonesia, in 2018, focusing in the Software Engineering field. He is currently a lecturer and researcher in Universitas Multimedia Nusantara and also serving as the head coordinator of the Game Development Laboratory. His research interests include requirements engineering in
software application development, computer engineering, and human computer interaction. His last publication entitled "Bilingual Color Learning Application as Alternative Color Learning for Preschool Student" in International Journal of Interactive Mobile Technologies (IJIM) in 2022.
