CC BY
59
УНИВЕРСИТЕТ итмо
НАУЧНО-ТЕХНИЧЕСКИИ ВЕСТНИК ИНФОРМАЦИОННЫХ ТЕХНОЛОГИИ, МЕХАНИКИ И ОПТИКИ май-июнь 2022 Том 22 № 3 http://ntv.ifmo.ru/
SCIENTIFIC AND TECHNICAL JOURNAL OF INFORMATION TECHNOLOGIES, MECHANICS AND OPTICS May-June 2022 Vol. 22 No 3 http://ntv.ifmo.ru/en/
ISSN 2226-1494 (print) ISSN 2500-0373 (online)
ИНФОРМАЦИОННЫХ ТЕХНОЛОГИЙ. МЕХАНИКИ И йПТИКИ
doi: 10.17586/2226-1494-2022-22-3-574-584
Modelling of basic Indonesian Sign Language translator based on Raspberry Pi technology Umi Fadlilah1 Raden Adrian Rafli Prasetyo2, Abd Kadir Mahamad3, Bana Handaga4,
Sharifah Saon5, Endah Sudarmilah6
i,2,4,6 Universitas Muhammadiyah Surakarta, Surakarta, 57169, Indonesia, !,3,5 Universiti Tun Hussein Onn Malaysia, Batu Pahat, Johor, 86400, Malaysia
1 uf138@ums.ac.idH, https://orcid.org/0000-0002-2261-3663
2 adrian.raflip@gmail.com, https://orcid.org/0000-0001-8747-9437
3 kadir@uthm.edu.my, https://orcid.org/0000-0002-3985-6549
4 bana.handaga@ums.ac.id, https://orcid.org/0000-0001-6797-6233
5 sharifa@uthm.edu.my, https://orcid.org/0000-0003-2706-3258
6 Endah.Sudarmilah@ums.ac.id, https://orcid.org/0000-0002-9244-4762
Abstract
Deaf people have hearing loss from mild to very severe. Such people have difficulty processing language information both with and without hearing aids. Deaf people who do not use hearing aids use sign language in their everyday conversations. At the same time, it is difficult for general people to communicate with the deaf, so in order to communicate with the deaf they must know sign language. There are two sign languages in Indonesia, namely SIBI (Indonesian Sign Language System) and BISINDO (Indonesian Sign Language). To help with communication between deaf and normal people, we developed a model using the one-handed SIBI method as an example, and then further developed it using the one-handed and two-handed BISINDO. The main function of the method is the recognition of basic letters, words, sentences and numbers using a Raspberry Pi single-board computer and a camera which are designed to detect the movements of language gestures. With the help of a special program, images are translated into text on the monitor screen. The method used is image processing and machine learning using the Python programming language and Convolutional Neural Network techniques. The device prototype issues a warning to repeat the sign language if the translation fails, and delete the translation if it doesn't match the database. The prototype of the device requires further development providing its flexibility: to provide reading of dynamic movements, facial expressions, to provide translation of words not included in the existing database. You need to add a database other than SIBI, such as BISINDO, or sign languages from other regions or countries. Keywords
CNN, deaf people, droidcam, image processing, machine learning, Python, Raspberry Pi, SIBI, sign language,
smartphone camera, webcam
Acknowledgements
This work was supported by a research grant TIER 1, H917.
The researchers thank all those who contributed to the preparation of the work, especially the UMS (Muhammadiyya University Surakarta) Electrical and Electronics Engineering Curriculum and Tun Hussein Onn Malaysia (UTHM) Electrical Engineering Curriculum, Research Management. We thank the Center, which contributed to the implementation and testing of this study.
For citation: Fadlilah U., Prasetyo R.A.R., Mahamad A.K., Handaga B., Saon S., Sudarmilah E. Modelling of basic Indonesian Sign Language translator based on Raspberry Pi technology. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2022, vol. 22, no. 3, pp. 574-584. doi: 10.17586/2226-1494-2022-22-3-574-584
© Fadlilah U., Prasetyo R.A.R., Mahamad A.K., Handaga B., Saon S., Sudarmilah E., 2022
УДК 004.89
Моделирование базового переводчика индонезийского языка жестов на одноплатном компьютере Raspberry Pi
Уми Фадлила1Н, Раден Адриан Рафли Прасетьо2, Абд Кадир Махамад3, Бана Хандага4, Шарифа Саон5, Энда Судармила6
1,2,4,6 Университет Мухаммадия Суракарта, Суракарта, 57169, Индонезия
1,3,5 Университет Тун Хусейн Онн Малайзия, Бату Пахат, Джохор, 86400, Малайзия
1 uf138@ums.ac.ids, https://orcid.org/0000-0002-2261-3663
2 adrian.raflip@gmail.com, https://orcid.org/0000-0001-8747-9437
3 kadir@uthm.edu.my, https://orcid.org/0000-0002-3985-6549
4 bana.handaga@ums.ac.id, https://orcid.org/0000-0001-6797-6233
5 sharifa@uthm.edu.my, https://orcid.org/0000-0003-2706-3258
6 Endah.Sudarmilah@ums.ac.id, https://orcid.org/0000-0002-9244-4762
Аннотация
Глухие люди имеют потерю слуха от легкой формы до очень тяжелой. Такие люди испытывают трудности при обработке языковой информации, как со слуховыми аппаратами, так и без них. Глухие люди, которые не применяют слуховые аппараты, в своих повседневных разговорах используют язык жестов. В то же время здоровым людям трудно общаться с глухими, поэтому для общения они должны знать язык жестов. В Индонезии существует два жестовых языка, а именно Indonesian Sign Language System (SIBI) и Indonesian Sign Language (BISINDO). Разработана модель для помощи в общении между глухими и здоровыми людьми. Модель рассмотрена на примере использования одноручного метода SIBI и доработана с использованием одноручного и двуручного BISINDO. Основная функция метода — распознавание основных букв, слов, предложений и цифр с помощью одноплатного компьютера Raspberry Pi и камеры, которые предназначены для обнаружения движений языковых жестов. Полученные изображения переводятся в текст на экране монитора с помощью специальной программы. Используемый метод заключается в обработке изображений и машинном обучении с использованием языка программирования Python и техники сверточной нейронной сети. Прототип устройства выдает предупреждение о необходимости повторить язык жестов, если перевод не удался, и удалить перевод, если он не соответствует базе данных. Прототип устройства требует дополнительных исследований для обеспечения гибкости при считывании динамических движений, выражений лиц, и перевода слов, не включенных в существующую базу данных. Также требуется расширение базы данных, отличной от языка жестов SIBI, например, BISINDO, или языков жестов из других регионов или стран. Ключевые слова
CNN, глухие люди, дроид-камера, обработка изображений, машинное обучение, Python, Raspberry Pi, SIBI, язык
жестов, камера смартфона, веб-камера
Благодарности
Работа поддержана исследовательским грантом TIER 1, H917.
Исследователи благодарят всех, кто внес свой вклад в подготовку работы, особенно в учебные программы по электротехнике и электронике UMS (Университет Мухаммадия Суракарта) и по электротехнике Университета Тун Хусейн Онн Малайзия (UTHM), управление исследованиями. Благодарим Центр, который способствовал реализации и тестированию этого исследования.
Ссылка для цитирования: Фадлила У, Прасетьо Р.А.Р., Махамад А.К., Хандага Б., Саон Ш., Судармила Э. Моделирование базового переводчика индонезийского языка жестов на одноплатном компьютере Raspberry Pi // Научно-технический вестник информационных технологий, механики и оптики. 2022. Т. 22, № 3. С. 574-584 (на англ. яз.). doi: 10.17586/2226-1494-2022-22-3-574-584
Introduction
Deaf people have hearing loss ranging from easy to very hard. Furthermore, deaf people have obstacles in processing language information through their hearing with or without using hearing tools. His hearing is quite successful in processing language information if the deaf person uses a hearing aid [1-7]. In fact, sign language is one of the most popular communication techniques in deaf communication [8]. At the same time, Sign Language Recognition is a breakthrough for helping deaf-mute people [9-20]. The results of the study [21] said that based on data analysis, it is known that there are still many people who do not know what sign language is and some people already know what sign language is but still do not know how to communicate with other people. Yet sign language needs to be learned to support communication between
the deaf and normal people. Based on this, there must be an intermediary media to be a solution to the problem, for example, sign language translator using Raspberry Pi.
Literature Review
Many ways have been taken to create a communication aid medium between deaf people and ordinary people, for example, by developing a prototype that translates words into sign language. The technologies such as the Python programming language, Natural Language Tool Kit, etc. had been used for developed Pakistan Sign Language (PSL) prototype [22]. The study about sign language also provides a literature review to highlight existing technology work being carried out around the world, for example, using deep learning [23-32]. Another way is to change from sign language to writing or conversation by utilizing
human gesture, especially hand gesture recognition technology and image processing [33-44]. The technology has the potential for application in sign language. But, the challenge or obstacle of this technology is achieving the accuracy of readings in various conditions, such as achieving an accurate and robust system remain, hand occlusion, transformation, database scalability, differential background illumination, position control, self-location, high computation costs, etc. [37, 45]. Meanwhile, to train the model on spatial features, usually use the inception model which is a deep Convolutional Neural Network (CNN) method to carry out basic things in bridging the communication gap in sign language recognition [23, 4648]. By using technologies such as image processing, databases, and the Python programming language, we can create a communication aid that translates sign language into written form [22, 41]. Then, using the camera as an image sensor to extract sign language data, it is processed by the Raspberry Pi microprocessor module, and the results are displayed on the monitor screen [41, 49]. In Indonesia there are various sign languages, the main ones being Indonesian Sign Language System (SIBI) and Indonesian Sign Language (BISINDO). However, not all deaf people in Indonesia use the sign language; some prefer or are forced to communicate orally. Many other deaf people are considered linguistically isolated, living in rural areas where there is no opportunity to meet other deaf people. Meanwhile, BISINDO sign language is used in urban centers by tens and even hundreds of thousands of deaf people throughout the archipelago [36, 50-53]. However, in this project, we only take a few examples of the sign language used in SIBI (using one hand), before using BISINDO in the next project that is more flexible. The author hopes that this prototype can be a testing device for communication between the deaf and the ordinary people in their communicating using sign language, in this case SIBI method is a model in hand gestures.
Method
The sign language used for this experiment is the SIBI method. It can be seen that the SIBI sign language movements captured by the camera will be entered into a gesture database with a certain number of movements to create a CNN movement model. The camera captures input in the form of a photo in 2 dimensions of sign language movement, then it is processed using a Python algorithm and CNN technique, so that it can classify the movements made by the user when using sign language. The model is tested using an identifier that can display the results in the form of a translation of the sign language movements captured by the camera. The sequence of how this tool works starts from taking pictures, creating a gesture database, creating a CNN model, reading the model, and displaying the results. The processed photo database is converted into a model file which is used by the main program as sign recognition data to produce the display of letters, numbers, or words on the monitor screen so that ordinary people who do not understand sign language can know the meaning of sign language hand gestures. For all of those workflows, some software and methods are
required, such as Anaconda and Python programming [10]. Python is the main programming language in making sign language translation software. Background subtraction technology with calibration in translating sign language is implemented using Python programming. The details of how Python is used in this work begins with compiling scripts in Python programming to make objects or images other than the objects, or objects used to disappear in order to separate the object from the background. Data retrieval and software testing require several Python packages or libraries according to their specifications which are installed through the Anaconda Python distribution software so that the program runs more smoothly. The Background subtraction method is very influential on the change in lighting conditions of images that are processed in translation. Therefore, a calibration feature is needed to reset the processed image using frames from the previous sign language motion capture as a reference. This method can be implemented with the help of the OpenCV Package which is able to process image data captured by the camera and process the information in it [37]. In addition, machine learning methods using CNN are also explored in this project to process information data that can recognize two-dimensional objects [54]. The object is captured from hand tracking, a method where the program recognizes the hand that is in an identification area, and if the hand movement is recognized by the program, the area will follow the movement of the hand [55]. This study uses sign language as the object. Sign language is the language used by people who can't hear or are deaf. In Indonesia, there are two types of sign language, namely, BISINDO and SIBI. BISINDO, or Indonesian Sign Language, is one of the sign languages that apply in Indonesia which is natural and can be easily used in the daily interactions of the deaf. SIBI is one of the sign language systems adopted from American Sign Language (ASL). The SIBI sign language has been formalized by the government and is often used in special schools and television broadcasting in Indonesia.
There have been studies that have helped the development of sign language in Indonesia, one of which is the BISINDO community website in Surakarta [51]. Actually, the next project in this research is to build a smartphone application to translate BISINDO. Fig. 1 shows the difference between BISINDO and SIBI. SIBI uses one hand in all alphabets and in a number of sign languages while BISINDO uses not only one hand. So, maybe the level of difficulty in BISINDO tends to be higher than SIBI hand gestures. This project first uses SIBI as a sign language model before developing BISINDO application using Android smartphone in the next project.
Results and Discussion
The sequence in data collection is in accordance with the research flow diagram in Fig. 2. The program starts by creating a photo database folder from sign language, then retrieve the database followed by basic database modeling. After that, we test the model (algorithm testing) of the hand movement whether it matches the expected sign language. If not yet, then retrieve again to photo database, but if it is suitable, immediately proceed to the next testing to read
Fig. 1. Two types of sign language: BISINDO1 (a), SIBI2 (b)
1 Available at: https://www.klobility.id/post/perbedaan-bisindo-dan-sibi (accessed: 14.02.2022).
2 Available at: https://ekoslbkuncupmas.wordpress.com/2015/03/30/belajar-sibi/ (accessed: 14.02.2022).
the sign language movement, translate it to 2D black & white frame and then match it with suitable model to get the translated language. If the translation results do not match, then look back at the hand movement model in the database. If the translation results are suitable, proceed with analyzing the data and then creating the right script to run the program repeatedly before completing the program flow until display the outcomes in text form.
In data collection and software testing, several Python packages or libraries are needed so that the program can run smoothly. The packages or libraries used are installed through the Python Anaconda distribution software. This data collection uses a sample of letters A, B, C, numbers 0, 1, 2, and the words NAMA (NAME), SAYA (I), KAMU (YOU). Each data sample is taken using two different types of cameras, namely a webcam camera installed on a laptop, and a smartphone camera using the DroidCam application (Fig. 3, a). This testing tool uses two cameras, namely a laptop webcam camera with 0.31 MP specifications and a resolution of 640 x 480 and a smartphone camera with 12 MP specifications and 1080 x 720 resolution and a laptop with an Intel I3 processor specification with 4 GB RAM.
The reading data on the hand signal model (having been made) is done in real time using the CNN technique which can detect and recognize objects in an image. Furthermore, it is necessary to test the final stage in collecting accuracy data, because the reading results will be used to communicate. If the reading is not accurate, it is necessary to add a database and create a new model from training data and validation data so that the accuracy increases.
In the test data from taking pictures two types of cameras used (laptop webcam and smartphone camera), like in Fig. 3, b. The reading results are in the form of two states: Good and Not Good. Good is a reading that produces a clear background subtraction image and no
Translated Display on Monitor Screen
Data Analysis
Making Script
> 1
End
Fig. 2. Research flow
LAPTOP
0 1 1 2 _ З 4
SMARTPHON E
r \
p'à à'Vt 9 À'àët
f à* à •
M àÀr&SÊ 1 l ^l'f ggQ X fà Лв т Й è
йЫ à Л
rvvm rrrei
Fig. 3. DroidCam software (a); Results of the background subtraction of laptop and smartphone cameras (b)
pixels that obscure hand shapes, this parameter is good for database modeling and model testing. False Acceptance Rate (FAR) can be calculated as
FAR =
The number of false acceptances The number of identification attempts
x 100 % .
Not Good is a reading that is the opposite of a Good state. Based on the experimental data, the number of identification attempts was 78 times and the number of incorrect receipts was 33 ("Not Good"). Then the FAR value will be the value 42.3 % and the error value remains very large.
In Fig. 4, a, the right side shows the results of the background subtraction process using a high resolution camera that looks like the shape of a human hand in a smartphone. Meanwhile, on the left there are still many pixels that the program cannot read due to the small resolution in laptop. Therefore, for the high level of accuracy of the database model training, the authors use a smartphone camera. The 2D photo database includes data taken from the camera and directly processed using the Background subtraction method through Python programming which produces black & white photos which then stored in a folder that was created beforehand. There are two types of data taken, namely training data
and validation data. Training data is the data used to create a new model, while validation data is the data used to validate data generated by machine learning training using Python algorithms that focus on code readability so that the model formed is not too biased or hyperparameter. The total number of photo data made in this study is 35,100 photos. In the training data folder, there are 600 photos; it is more than the photo data in the validation data folder containing 300 photos in total. So the total photo data in each movement includes 900 photos. The data were used for the creation of CNN training and the creation of a reading model. Meanwhile, the validation data folder is used to validate the results of the model that has been made.
In Fig. 4, a, a folder presents letters A-Z, numbers 0-9, and the words NAMA, SAYA, and KAMU. While Fig. 4, b shows the contents of the number 0 folder as an example of the contents of each photo database folder according to Table. The reading data on the hand model that has been made is done in real time and it is necessary to test the final stage of data collection for its accuracy, because the results of the reading will be used to communicate. If the reading is inaccurate, it is necessary to add databases and create new models to increase the accuracy.
Graph in Fig. 5, b is the result of the model training in the training data folder Database which contains photos of the results of the background subtraction of each hand
■ ■ ■ s In ■т
■ ■ |H |nama
■ ■ A ■ > lo ■ v
■ ■ ■ lp lw
|4 ■ ■ ■ Q ■ x
■ И KAMU |V
■ Ш-
Ш' В SAYA
Fig. 4. Database folder (a); Database photo (b)
0 20 40 0 20 40
Epoch # Epoch #
Fig. 5. Results of the model training: CNN Training accuracy chart (a), CNN Training loss (b)
movement totaling 600 pieces of each type of movement using CNN technique and Python programming. The validation result of the training model created by machine learning using CNN with image as input. The combination of CNN and Python programming is easier and faster in machine learning research. In this study, the value of each graph always changes as the epoch changes, in the sense that the epoch language means an event at a certain time.
For example, if there are 600 images to be trained into a CNN, with a batch size of 10, it will take 10 iterations to get 1 epoch (in 1 epoch, every weights on CNN will be changed 10 times as well).
The accuracy (value and the loss value) are inversely proportional to each other: the higher the accuracy value, like in Table, the smaller the loss or loss value, like in the total accuracy percentage divided to data total:
Table. .Reading Accuracy Values
Data Author's Hand Reading Data, Times Other People's Hand Reading Data, Times Hand Reading Data for Deaf Disability, Times Total Reading Error Accuracy, % Data Author's Hand Reading Data, Times Other People's Hand Reading Data Times Hand Reading Data for Deaf Disability, Times Total Reading Error Accuracy, %
0 10 5 5 5 75 K 10 5 5 5 75
1 10 5 5 7 65 L 10 5 5 4 80
2 10 5 5 4 80 M 10 5 5 9 55
3 10 5 5 5 75 N 10 5 5 10 50
4 10 5 5 8 60 O 10 5 5 7 65
5 10 5 5 4 80 P 10 5 5 6 70
6 10 5 5 8 60 Q 10 5 5 4 80
7 10 5 5 8 60 R 10 5 5 5 75
8 10 5 5 5 75 S 10 5 5 5 75
9 10 5 5 4 80 T 10 5 5 9 55
A 10 5 5 9 55 U 10 5 5 6 70
B 10 5 5 9 55 V 10 5 5 6 70
C 10 5 5 7 65 W 10 5 5 8 60
D 10 5 5 5 75 X 10 5 5 6 70
E 10 5 5 8 60 Y 10 5 5 4 80
F 10 5 5 9 55 Z 10 5 5 5 75
G 10 5 5 4 80 NAMA 10 5 5 4 80
H 10 5 5 4 80 KAMU 10 5 5 7 65
I 10 5 5 4 80 SAYA 10 5 5 8 60
J 10 5 5 6 70 TOTAL 390 195 195 241 2695
Fig. 6. Testing tools along with smartphone camera (a); read error in letter N (b); deletion of letter N (c); addition of letters A and T (d)
J
/ (7 weeH
k4«®librqpi ulonj^piggо кзуаг domlnan hitem
g . f. fr«me. copy( )
•nd_bbox - (116, 116, i:
Fig. 7. Testing with deaf people
Percentage of Average Accuracy = YReadine data - Terror
= —-5-^- x 100 %.
Yfieading data
The results of calculating the Percentage of Average Accuracy give the value 69.1 %.
When data reading is done, every program successfully recognizes and retrieves data ten times to find the accuracy of comparisons from the models that have been made from the database folder. The reading is done using one's own hands 10 times, then five times of one person and five times of one deaf person. The number of identification attempts is A + B + C is equal to 780. Thus, FRR (False Recognition Rate) can be known from
^ ^ The ration of the number offalse recognitions The number of identification attempts D
X 100 % =-x 100 %.
A+B + C
Calculation results for FRR give the value 30.9 %. FRR was less than FAR after testing with smartphone cameras on 3 people. So, the more data models, the lower the error rate, because the application must be trained by the data models first before launching to translate sign language hand gesture to text. In Fig. 6, a, the examiner is running a reading program using a smartphone camera, but it can be seen in Fig. 6, b that there is an error in reading the letter N which should be the letter A.
Fig. 6, c shows the deletion of the wrong letter, namely N. In Fig. 6, d it can be seen the addition of the letter A and T to complete the word "SIG". In the display window there is also an indicator of the number of Frames displayed Per Second (FPS), information on the function of the 'r' button for calibration, and information on the function of the 'p' button for returning to the initial display. There is also a small window display that is the result of the background
subtraction process from reading in the blue box area. There are differences in readings that affect the level of accuracy, so added a feature to delete letters, numbers or words if there are errors in reading hand movements.
In Fig. 7, the test is carried out with the deaf that translates the sign language gestures of the letters A, L, V, I, and N that are the name of the person. The problem when working on and testing this tool is when the photo database is taken because the amount of data taken is very large, so that some of the photos taken contain noise. This is especially related to hardware specifications, the laptop being used, data retrieval and running tests with system operating conditions that are not smooth. However, the testers were able to anticipate these problems by using a laptop with a higher specification and increasing the light for clean shots. A computer should be used with minimum specification of an Intel I3 processor, 4 GB RAM, and a 12 MP camera.
Conclusion
Based on the results of research and testing that have been carried out, the following conclusions can be drawn.
1. Before taking the photo database, we recommend to prepare a folder structure model according to the number of database types and use compatible camera. The camera specifications may affect readings even if there is an image calibration feature. The better the camera specifications, the clearer the resulting image for the database. The better the use of cameras and the number of models generated from CNN training, the better the computer specifications needed for the program to run smoothly. The computer used to run this program should have a minimum specification of an Intel I3 processor, 4GB RAM, and a 12 MP camera.
2. The modeling and reading of sign language gestures in this study were assisted by the Python programming
language using packages designed to run Image Processing, Machine Learning, and CNN. The more data models, the lower the error rate, because the application is trained with the data models first before launching.
3. This tool can read the SIBI method sign language from letters A-Z, numbers 0-9, and the words NAME, ME, YOU with different levels of accuracy, so there is a feature to delete the wrong letters, numbers, or words.
Future research should consider further the design and manufacture, so that the tool can read dynamic movements and facial expressions needed to translate words. It should use a programming language that is compatible with Android operation system or others. In addition, the design of the device should be made portable since it is easier to use and carry anywhere. The next research plan will use BISINDO because it is more dynamic, and it uses 1 and 2 hand gestures that will be captured by a smartphone camera.
References
1. Hernawati T. Pengembangan kemampuan berbahasa dan berbicara anak tunarungu. Jurnal Jurusan PLB FIP Universitas Pendidikan Indonesia, 2007, vol. 7, no. 1, pp. 101110.
2. Shiraishi Y., Zhang J., Wakatsuki D., Kumai K., Morishima A. Crowdsourced real-time captioning of sign language by deaf and hard-of-hearing people. International Journal of Pervasive Computing and Communications, 2017, vol. 13, no. 1, pp. 2-25. https://doi.org/10.1108/IJPCC-02-2017-0014
3. Islalm M.S., Rahman M.M., Rahman M.H., Arifuzzaman M., Sassi R., Aktaruzzaman M. Recognition bangla sign language using convolutional neural network. Proc. of the 2019 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT), 2019, pp. 8910301. https://doi. org/10.1109/3ICT.2019.8910301
4. Lim M.Y. MySlang — An Electronic Malaysian Sign Language Dictionary. 2008, pp. 145.
5. Karpov A., Kipyatkova I., Zelezny M. Automatic Technologies for processing spoken sign languages. Procedia Computer Science, 2016, vol. 81, pp. 201-207. https://doi.org/10.1016/j.procs.2016.04.050
6. Brour M., Benabbou A. ATLASLang NMT: Arabic text language into Arabic sign language neural machine translation. Journal of King Saud University — Computer and Information Sciences, 2019, vol. 33, no. 9, pp. 1121-1131. https://doi.org/10.1016/j. jksuci.2019.07.006
7. Latif G., Mohammad N., Alghazo J., AlKhalaf R., AlKhalaf R. ArASL: Arabic alphabets sign language dataset. Data in Brief, 2019, vol. 23, pp. 103777. https://doi.org/10.1016/j.dib.2019.103777
8. Suharjito S., Thiracitta N., Gunawan H., Witjaksono G. The comparison of some hidden Markov models for sign language recognition. Proc. of the 1st Indonesian Association for Pattern Recognition International Conference (INAPR), 2019, pp. 6-10. https://doi.org/10.1109/INAPR.2018.8627031
9. Muthu Mariappan H., Gomathi V. Real-time recognition of Indian sign language. Proc. of the 2nd International Conference on Computational Intelligence in Data Science (ICCIDS), 2019, pp. 8862125. https://doi.org/10.1109/ICCIDS.2019.8862125
10. Kadiyala A., Kumar A. Applications of Python to evaluate environmental data science problems. Environmental Progress and Sustainable Energy, 2017, vol. 36, no. 6, pp. 1580-1586. https://doi. org/10.1002/ep.12786
11. Xie M., Ma X. End-to-End residual neural network with data augmentation for sign language recognition. Proc. of the 4th IEEE Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), 2019, pp. 1629-1633. https://doi. org/10.1109/IAEAC47372.2019.8998073
12. Inoue K., Shiraishi T., Yoshioka M., Yanagimoto H. Depth sensor based automatic hand region extraction by using time-series curve and its application to Japanese finger-spelled sign language recognition. Procedia Computer Science, 2015, vol. 60, no. 1, pp. 371-380. https://doi.org/10.1016/j.procs.2015.08.145
13. Xue Y., Gao S., Sun H., Qin W. A Chinese sign language recognition system using leap motion. Proc. of the 7th International Conference on Virtual Reality and Visualization (ICVRV), 2017, pp. 180-185. https://doi.org/10.1109/ICVRV.2017.00044
14. Kumar E.K., Kishore P.V.V., Kiran Kumar M.T., Kumar D.A. 3D sign language recognition with joint distance and angular coded color topographical descriptor on a 2 — stream CNN. Neurocomputing, 2020, vol. 372, pp. 40-54. https://doi.org/10.1016Aj. neucom.2019.09.059
Литература
1. Hernawati T. Pengembangan kemampuan berbahasa dan berbicara anak tunarungu // Jurnal Jurusan PLB FIP Universitas Pendidikan Indonesia. 2007. V. 7. N 1. P. 101110.
2. Shiraishi Y., Zhang J., Wakatsuki D., Kumai K., Morishima A. Crowdsourced real-time captioning of sign language by deaf and hard-of-hearing people // International Journal of Pervasive Computing and Communications. 2017. V. 13. N 1. P. 2-25. https:// doi.org/10.1108/IJPCC-02-2017-0014
3. Islalm M.S., Rahman M.M., Rahman M.H., Arifuzzaman M., Sassi R., Aktaruzzaman M. Recognition bangla sign language using convolutional neural network // Proc. of the 2019 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT). 2019. P. 8910301. https://doi. org/10.1109/3ICT.2019.8910301
4. Lim M.Y. MySlang — An Electronic Malaysian Sign Language Dictionary. 2008. p. 145.
5. Karpov A., Kipyatkova I., Zelezny M. Automatic Technologies for processing spoken sign languages // Procedia Computer Science. 2016. V. 81. P. 201-207. https://doi.org/10.1016/j.procs.2016.04.050
6. Brour M., Benabbou A. ATLASLang NMT: Arabic text language into Arabic sign language neural machine translation // Journal of King Saud University — Computer and Information Sciences. 2019. V. 33. N 9. P. 1121-1131. https://doi.org/10.1016/jjksuci.2019.07.006
7. Latif G., Mohammad N., Alghazo J., AlKhalaf R., AlKhalaf R. ArASL: Arabic alphabets sign language dataset // Data in Brief. 2019. V. 23. P. 103777. https://doi.org/10.1016/j.dib.2019.103777
8. Suharjito S., Thiracitta N., Gunawan H., Witjaksono G. The comparison of some hidden Markov models for sign language recognition // Proc. of the 1st Indonesian Association for Pattern Recognition International Conference (INAPR). 2019. P. 6-10. https://doi.org/10.1109/INAPR.2018.8627031
9. Muthu Mariappan H., Gomathi V. Real-time recognition of Indian sign language // Proc. of the 2nd International Conference on Computational Intelligence in Data Science (ICCIDS). 2019. P. 8862125. https://doi.org/10.1109/ICCIDS.2019.8862125
10. Kadiyala A., Kumar A. Applications of Python to evaluate environmental data science problems // Environmental Progress and Sustainable Energy. 2017. V. 36. N 6. P. 1580-1586. https://doi. org/10.1002/ep.12786
11. Xie M., Ma X. End-to-End residual neural network with data augmentation for sign language recognition // Proc. of the 4th IEEE Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). 2019. P. 1629-1633. https://doi. org/10.1109/IAEAC47372.2019.8998073
12. Inoue K., Shiraishi T., Yoshioka M., Yanagimoto H. Depth sensor based automatic hand region extraction by using time-series curve and its application to Japanese finger-spelled sign language recognition // Procedia Computer Science. 2015. V. 60. N 1. P. 371380. https://doi.org/10.1016/j.procs.2015.08.145
13. Xue Y., Gao S., Sun H., Qin W. A Chinese sign language recognition system using leap motion // Proc. of the 7th International Conference on Virtual Reality and Visualization (ICVRV). 2017. P. 180-185. https://doi.org/10.1109/ICVRV. 2017.00044
14. Kumar E.K., Kishore P.V.V., Kiran Kumar M.T., Kumar D.A. 3D sign language recognition with joint distance and angular coded color topographical descriptor on a 2 — stream CNN // Neurocomputing. 2020. V. 372. P. 40-54. https://doi.org/10.1016/j.neucom.2019.09.059
15. Yang X., Chen X., Cao X., Wei S., Zhang X. Chinese sign language recognition based on an optimized tree-structure framework // IEEE
15. Yang X., Chen X., Cao X., Wei S., Zhang X. Chinese sign language recognition based on an optimized tree-structure framework. IEEE Journal of Biomedical and Health Informatics, 2017, vol. 21, no. 4, pp. 994-1004. https://doi.org/10.1109/JBHI.2016.2560907
16. Rincon Vega A.M., Vasquez A., Amador W., Rojas A. Deep learning for the recognition of facial expression in the Colombian sign language. Annals of Physical and Rehabilitation Medicine, 2018, vol. 61, pp. e96. https://doi.org/10.1016Zj.rehab.2018.05.204
17. Joy J., Balakrishnan K., Sreeraj M. SignQuiz: A quiz based tool for learning fingerspelled signs in indian sign language using ASLR. IEEE Access, 2019, vol. 7, pp. 28363-28371. https://doi.org/10.1109/ ACCESS.2019.2901863
18. Gupta B., Shukla P., Mittal A. K-nearest correlated neighbor classification for Indian sign language gesture recognition using feature fusion. Proc. of the 6th International Conference on Computer Communication and Informatics (ICCCI), 2016, pp. 7479951. https:// doi.org/10.1109/ICCCI.2016.7479951
19. Singh S. Sign language to text by feed forward back propagation neural network. International Journal of Advanced Research in Computer Science, 2012, vol. 3, no. 3, pp. 146-152. https://doi. org/10.26483/ijarcs.v3i3.1209
20. Abid M.R., Petriu E.M., Amjadian E. Dynamic sign language recognition for smart home interactive application using stochastic linear formal grammar. IEEE Transactions on Instrumentation and Measurement, 2015, vol. 64, no. 3, pp. 596-605. https://doi. org/10.1109/TIM.2014.2351331
21. Bayu Ramadhani Fajri B., Ramadhani Fajri B., Kusumastuti G. Perceptions of 'hearing' people on sign language learning. Advances in Social Science, Education and Humanities Research, 2019, vol. 382, pp. 364-367. https://doi.org/10.2991/icet-19.2019.91
22. Abbas A., Sarfraz S. Developing a prototype to translate text and speech to Pakistan sign language with bilingual subtitles: A framework. Journal of Educational Technology Systems, 2018, vol. 47, no. 2, pp. 248-266. https://doi. org/10.1177/0047239518794168
23. Tao W., Leu M.C., Yin Z. American Sign Language alphabet recognition using Convolutional Neural Networks with multiview augmentation and inference fusion. Engineering Applications of Artificial Intelligence, 2018, vol. 76, pp. 202-213. https://doi. org/10.1016/j.engappai.2018.09.006
24. Sudharshan D.P., Raj S. Object recognition in images using convolutional neural network. Proc. of the 2nd International Conference on Inventive Systems and Control (ICISC), 2018, pp. 718821. https://doi.org/10.1109/ICISC.2018.8398893
25. Hernández-Blanco A., Herrera-Flores B., Tomás D., Navarro-Colorado B. A Systematic review of deep learning approaches to educational data mining. Complexity, 2019, vol. 2019, pp. 1306039. https://doi.org/10.1155/2019/1306039
26. Zhang Q., Zhang M., Chen T., Sun Z., Ma Y., Yu B. Recent advances in convolutional neural network acceleration. Neurocomputing, 2019, vol. 323, pp. 37-51. https://doi.org/10.1016/j.neucom.2018.09.038
27. Bantupalli K., Xie Y. American sign language recognition using deep learning and computer vision. Proc. of the 2018 IEEE International Conference on Big Data, 2018, pp. 4896-4899. https://doi. org/10.1109/BigData.2018.8622141
28. Liao Y., Xiong P., Min W., Min W., Lu J. Dynamic sign language recognition based on video sequence with BLSTM-3D residual networks. IEEE Access, 2019, vol. 7, pp. 38044-38054. https://doi. org/10.1109/ACCESS.2019.2904749
29. Nasreddine K., Benzinou A. Shape geodesics for robust sign language recognition. IET Image Process, 2019, vol. 13, no. 5, pp. 825-832. https://doi.org/10.1049/iet-ipr.2018.5282
30. Tolentino L.K.S., Serfa Juan R.O., Thio-ac A.C., Pamahoy M.A.B., Forteza J.R.R., Garcia X.J.O. Static sign language recognition using deep learning. International Journal of Machine Learning and Computing, 2019, vol. 9, no. 6, pp. 821-827. https://doi.org/10.18178/ ijmlc.2019.9.6.879
31. Sreedevi J., Rama Bai M., Ahmed M.M. An approach to translate hand gestures into telugu text. International Journal of Scientific and Technology Research, 2020, vol. 9, no. 4, pp. 258-264.
32. Azar S.G., Seyedarabi H. Trajectory-based recognition of dynamic Persian sign language using hidden Markov model. Computer Speech and Language, 2020, vol. 61, pp. 101053. https://doi.org/10.1016/j. csl.2019.101053
33. Rajam P.S., Balakrishnan G. Recognition of Tamil sign language alphabet using image processing to aid deaf-dumb people. Procedia
Journal of Biomedical and Health Informatics. 2017. V. 21. N 4. P. 994-1004. https://doi.org/10.1109/JBHI.2016.2560907
16. Rincon Vega A.M., Vasquez A., Amador W., Rojas A. Deep learning for the recognition of facial expression in the Colombian sign language // Annals of Physical and Rehabilitation Medicine. 2018. V. 61. p. e96. https://doi.org/10.1016/j.rehab.2018.05.204
17. Joy J., Balakrishnan K., Sreeraj M. SignQuiz: A quiz based tool for learning fingerspelled signs in indian sign language using ASLR // IEEE Access. 2019. V. 7. P. 28363-28371. https://doi.org/10.1109/ ACCESS.2019.2901863
18. Gupta B., Shukla P., Mittal A. K-nearest correlated neighbor classification for Indian sign language gesture recognition using feature fusion // Proc. of the 6th International Conference on Computer Communication and Informatics (ICCCI). 2016. P. 7479951. https:// doi.org/10.1109/ICCCI.2016.7479951
19. Singh S. Sign language to text by feed forward back propagation neural network // International Journal of Advanced Research in Computer Science. 2012. V. 3. N 3. P. 146-152. https://doi. org/10.26483/ijarcs.v3i3.1209
20. Abid M.R., Petriu E.M., Amjadian E. Dynamic sign language recognition for smart home interactive application using stochastic linear formal grammar // IEEE Transactions on Instrumentation and Measurement. 2015. V. 64. N 3. P. 596-605. https://doi.org/10.1109/ TIM.2014.2351331
21. Bayu Ramadhani Fajri B., Ramadhani Fajri B., Kusumastuti G. Perceptions of 'hearing' people on sign language learning // Advances in Social Science, Education and Humanities Research. 2019. V. 382. P. 364-367. https://doi.org/10.2991/icet-19.2019.91
22. Abbas A., Sarfraz S. Developing a prototype to translate text and speech to Pakistan sign language with bilingual subtitles: A framework // Journal of Educational Technology Systems. 2018. V. 47. N 2. P. 248-266. https://doi.org/10.1177/0047239518794168
23. Tao W., Leu M.C., Yin Z. American Sign Language alphabet recognition using Convolutional Neural Networks with multiview augmentation and inference fusion // Engineering Applications of Artificial Intelligence. 2018. V. 76. P. 202-213. https://doi. org/10.1016/j.engappai.2018.09.006
24. Sudharshan D.P., Raj S. Object recognition in images using convolutional neural network // Proc. of the 2nd International Conference on Inventive Systems and Control (ICISC). 2018. P. 718821. https://doi.org/10.1109/ICISC.2018.8398893
25. Hernández-Blanco A., Herrera-Flores B., Tomás D., Navarro-Colorado B. A Systematic review of deep learning approaches to educational data mining // Complexity. 2019. V. 2019. P. 1306039. https://doi.org/10.1155/2019/1306039
26. Zhang Q., Zhang M., Chen T., Sun Z., Ma Y., Yu B. Recent advances in convolutional neural network acceleration // Neurocomputing. 2019. V. 323. P. 37-51. https://doi.org/10.1016/j. neucom.2018.09.038
27. Bantupalli K., Xie Y. American sign language recognition using deep learning and computer vision // Proc. of the 2018 IEEE International Conference on Big Data. 2018. P. 4896-4899. https://doi.org/10.1109/ BigData.2018.8622141
28. Liao Y., Xiong P., Min W., Min W., Lu J. Dynamic sign language recognition based on video sequence with BLSTM-3D residual networks // IEEE Access. 2019. V. 7. P. 38044-38054. https://doi. org/10.1109/ACCESS.2019.2904749
29. Nasreddine K., Benzinou A. Shape geodesics for robust sign language recognition // IET Image Process. 2019. V. 13. N 5. P. 825-832. https://doi.org/10.1049/iet-ipr.2018.5282
30. Tolentino L.K.S., Serfa Juan R.O., Thio-ac A.C., Pamahoy M.A.B., Forteza J.R.R., Garcia X.J.O. Static sign language recognition using deep learning // International Journal of Machine Learning and Computing. 2019. V. 9. N 6. P. 821-827. https://doi.org/10.18178/ ijmlc.2019.9.6.879
31. Sreedevi J., Rama Bai M., Ahmed M.M. An approach to translate hand gestures into telugu text // International Journal of Scientific and Technology Research. 2020. V. 9. N 4. P. 258-264.
32. Azar S.G., Seyedarabi H. Trajectory-based recognition of dynamic Persian sign language using hidden Markov model // Computer Speech and Language. 2020. V. 61. P. 101053. https://doi. org/10.1016/j.csl.2019.101053
33. Rajam P.S., Balakrishnan G. Recognition of Tamil sign language alphabet using image processing to aid deaf-dumb people // Procedia Engineering. 2012. V. 30. P. 861-868. https://doi.org/10.1016/j. proeng.2012.01.938
Engineering, 2012, vol. 30, pp. 861-868. https://doi.org/10.1016/j. proeng.2012.01.938
34. Tan T.S., Salleh S.H., Ariff A.K., Ting C.M., Siew K.S., Leong S.H. Malay sign language gesture recognition system. Proc. of the 2007 International Conference on Intelligent and Advanced Systems (ICIAS), 2007, pp. 982-985. https://doi.org/10.1109/ ICIAS.2007.4658532
35. Gulhane V.M., Joshi M.S., Ingole M.D. Neural network based hand gesture recognition. International Journal of Advanced Research in Computer Science, 2012, vol. 3, no. 3, pp. 849-853. https://doi. org/10.26483/ijarcs.v3i3.1244
36. Indra D., Purnawansyah S., Madenda S., Wibowo E.P. Indonesian sign language recognition based on shape of hand gesture. Procedia Computer Science, 2019, vol. 161, pp. 74-81. https://doi. org/10.1016/j.procs.2019.11.101
37. Cheok M.J., Omar Z., Jaward M.H. A review of hand gesture and sign language recognition techniques. International Journal of Machine Learning and Cybernetics, 2019, vol. 10, no. 1, pp. 131-153. https:// doi.org/10.1007/s13042-017-0705-5
38. Tewari D., Srivastava S.K. A visual recognition of static hand gestures in Indian sign language based on kohonen self-organizing map algorithm. International Journal of Engineering and Advanced Technology, 2012, no. 2, pp. 165-170.
39. Rajaganapathy S., Aravind B., Keerthana B., Sivagami M. Conversation of sign language to speech with human gestures. Procedia Computer Science, 2015, vol. 50, pp. 10-15. https://doi. org/10.1016/j.procs.2015.04.004
40. Chen Y., Zhang W. Research and implementation of sign language recognition method based on Kinect. Proc. of the 2nd International Conference on Computer and Communications (ICCC), 2016, pp. 1947-1951. https://doi.org/10.1109/CompComm.2016.7925041
41. Abed A.A., Rahman S.A. Python-based Raspberry Pi for hand gesture recognition. International Journal of Computer Applications,
2017, vol. 173, no. 4, pp. 18-24. https://doi.org/10.5120/ ijca2017915285
42. Suharjito, Ariesta M.C., Wiryana F., Kusuma G.P. A survey of hand gesture recognition methods in sign language recognition. Pertanika Journal of Science and Technology, 2018, vol. 26, no. 4, pp. 16591675.
43. Asadi-Aghbolaghi M., Clapés A., Bellantonio M., Escalante H., Ponce-López V., Baró X., Guyon I., Kasaei S., Escalera S. Deep learning for action and gesture recognition in image sequences: A survey. Gesture Recognition. Springer, 2017, pp. 539-578. https://doi. org/10.1007/978-3-319-57021-1_19
44. Manikandan K., Patidar A., Walia P., Roy A.B. Hand gesture detection and conversion to speech and text. arXiv, 2018, arXiv:1811.11997. https://doi.org/10.48550/arXiv. 1811.11997
45. Cañas J.M., Martín-Martín D., Arias P., Vega J., Roldán-álvarez D., García-Pérez I., Fernández-Conde J. Open-source drone programming course for distance engineering education. Electronics, 2020, vol. 9, no. 12, pp. 1-18. https://doi.org/10.3390/electronics9122163
46. Masood S., Srivastava A., Thuwal H.C., Ahmad M. Real-time sign language gesture (word) recognition from video sequences using CNN and RNN. Advances in Intelligent Systems and Computing,
2018, vol. 695, pp. 623-632. https://doi.org/10.1007/978-981-10-7566-7_63
47. Islam M.S., Mousumi S.S.S., Azad Rabby A.K.M.S., Hossain S.A., Abujar S. A potent model to recognize bangla sign language digits using convolutional neural network. Procedia Computer Science, 2018, vol. 143, pp. 611-618. https://doi.org/10.1016/j. procs.2018.10.438
48. Zhong B., Xing X., Love P., Wang X., Luo H. Convolutional neural network: Deep learning-based classification of building quality problems. Advanced Engineering Informatics, 2019, vol. 40, pp. 4657. https://doi.org/10.1016Zj.aei.2019.02.009
49. Dawane S.P., Sayyed H.G.A. Hand gesture recognition for deaf and dumb people using GSM module. International Journal of Science and Research, 2017, vol. 6, no. 5, pp. 2226-2230.
50. Handhika T., Zen R.I.M., Murni, Lestari D.P., Sari I. Gesture recognition for Indonesian Sign Language (BISINDO). Journal of Physics: Conference Series, 2018, vol. 1028, no. 1, pp. 012173. https://doi.org/10.1088/1742-6596/1028A/012173
51. Fadlilah U., Wismoyohadi D., Mahamad A.K., Handaga B. Bisindo information system as potential daily sign language learning. AIP Conference Proceedings, 2019, vol. 2114, pp. 060021. https://doi. org/10.1063/1.5112492
34. Tan T.S., Salleh S.H., Ariff A.K., Ting C.M., Siew K.S., Leong S.H. Malay sign language gesture recognition system // Proc. of the 2007 International Conference on Intelligent and Advanced Systems (ICIAS). 2007. P. 982-985. https://doi.org/10.1109/ ICIAS.2007.4658532
35. Gulhane V.M., Joshi M.S., Ingole M.D. Neural network based hand gesture recognition // International Journal of Advanced Research in Computer Science. 2012. V. 3. N 3. P. 849-853. https://doi. org/10.26483/ijarcs.v3i3.1244
36. Indra D., Purnawansyah S., Madenda S., Wibowo E.P. Indonesian sign language recognition based on shape of hand gesture // Procedia Computer Science. 2019. V. 161. P. 74-81. https://doi.org/10.1016/j. procs.2019.11.101
37. Cheok M.J., Omar Z., Jaward M.H. A review of hand gesture and sign language recognition techniques // International Journal of Machine Learning and Cybernetics. 2019. V. 10. N 1. P. 131-153. https://doi. org/10.1007/s13042-017-0705-5
38. Tewari D., Srivastava S.K. A visual recognition of static hand gestures in Indian sign language based on kohonen self-organizing map algorithm // International Journal of Engineering and Advanced Technology. 2012. N 2. P. 165-170.
39. Rajaganapathy S., Aravind B., Keerthana B., Sivagami M. Conversation of sign language to speech with human gestures // Procedia Computer Science. 2015. V. 50. P. 10-15. https://doi. org/10.1016/j.procs.2015.04.004
40. Chen Y., Zhang W. Research and implementation of sign language recognition method based on kinect // Proc. of the 2nd International Conference on Computer and Communications (ICCC). 2016. P. 1947-1951. https://doi.org/10.1109/CompComm.2016.7925041
41. Abed A.A., Rahman S.A. Python-based Raspberry Pi for hand gesture recognition // International Journal of Computer Applications. 2017. V. 173. N 4. P. 18-24. https://doi.org/10.5120/ijca2017915285
42. Suharjito, Ariesta M.C., Wiryana F., Kusuma G.P. A survey of hand gesture recognition methods in sign language recognition // Pertanika Journal of Science and Technology. 2018. V. 26. N 4. P. 1659-1675.
43. Asadi-Aghbolaghi M., Clapés A., Bellantonio M., Escalante H., Ponce-López V., Baró X., Guyon I., Kasaei S., Escalera S. Deep learning for action and gesture recognition in image sequences: A survey // Gesture Recognition. Springer, 2017. P. 539-578. https:// doi.org/10.1007/978-3-319-57021-1_19
44. Manikandan K., Patidar A., Walia P., Roy A.B. Hand gesture detection and conversion to speech and text // arXiv. 2018. arXiv:1811.11997. https://doi.org/10.48550/arXiv.1811.11997
45. Cañas J.M., Martín-Martín D., Arias P., Vega J., Roldán-álvarez D., García-Pérez I., Fernández-Conde J. Open-source drone programming course for distance engineering education // Electronics. 2020. V. 9. N 12. P. 1-18. https://doi.org/10.3390/electronics9122163
46. Masood S., Srivastava A., Thuwal H.C., Ahmad M. Real-time sign language gesture (word) recognition from video sequences using CNN and RNN // Advances in Intelligent Systems and Computing. 2018. V. 695. P. 623-632. https://doi.org/10.1007/978-981-10-7566-7_63
47. Islam M.S., Mousumi S.S.S., Azad Rabby A.K.M.S., Hossain S.A., Abujar S. A potent model to recognize bangla sign language digits using convolutional neural network // Procedia Computer Science. 2018. V. 143. P. 611-618. https://doi.org/10.1016/j.procs.2018.10.438
48. Zhong B., Xing X., Love P., Wang X., Luo H. Convolutional neural network: Deep learning-based classification of building quality problems // Advanced Engineering Informatics. 2019. V. 40. P. 46-57. https://doi.org/10.1016/j.aei.2019.02.009
49. Dawane S.P., Sayyed H.G.A. Hand gesture recognition for deaf and dumb people using GSM module // International Journal of Science and Research. 2017. V. 6. N 5. P. 2226-2230.
50. Handhika T., Zen R.I.M., Murni, Lestari D.P., Sari I. Gesture recognition for Indonesian Sign Language (BISINDO) // Journal of Physics: Conference Series. 2018. V. 1028. N 1. P. 012173. https:// doi.org/10.1088/1742-6596/1028/1/012173
51. Fadlilah U., Wismoyohadi D., Mahamad A.K., Handaga B. Bisindo information system as potential daily sign language learning // AIP Conference Proceedings. 2019. V. 2114. P. 060021. https://doi. org/10.1063/1.5112492
52. Palfreyman N. Social meanings of linguistic variation in BISINDO (Indonesian sign language) // Asia-Pacific Language Variation. 2020. V. 6. N 1. P. 89-118. https://doi.org/10.1075/aplv.00008.pal
53. Fadlilah U., Mahamad A.K., Handaga B. The Development of android for Indonesian sign language using tensorflow lite and CNN: An
52. Palfreyman N. Social meanings of linguistic variation in BISINDO (Indonesian sign language). Asia-Pacific Language Variation, 2020, vol. 6, no. 1, pp. 89-118. https://doi.org/10.1075/aplv.00008.pal
53. Fadlilah U., Mahamad A.K., Handaga B. The Development of android for Indonesian sign language using tensorflow lite and CNN: An initial study. Journal of Physics: Conference Series, 2021, vol. 1858, no. 1, pp. 012085. https://doi.org/10.1088/1742-6596/1858/1Z012085
54. Dewa C.K., Fadhilah A.L., Afiahayati A. Convolutional neural networks for handwritten Javanese character recognition. IJCCS (Indonesian Journal of Computing and Cybernetics Systems), 2018, vol. 12, no 1, pp. 83. https://doi.org/10.22146/ijccs.31144
55. Kurien M., Kim M.K., Kopsida M., Brilakis I. Real-time simulation of construction workers using combined human body and hand tracking for robotic construction worker system. Automation in Construction, 2018, vol. 86, pp. 125-137. https://doi.org/10.1016/j. autcon.2017.11.005
Authors
Umi Fadlilah — M. Eng., Lecturer, Universitas Muhammadiyah Surakarta, Surakarta, 57169, Indonesia; Student, Universiti Tun Hussein Onn Malaysia, Batu Pahat, Johor, 86400, Malaysia, S3 57202821436, https://orcid.org/0000-0002-2261-3663, uf138@ums.ac.id
Raden A.R. Prasetyo — Bach. of Elec. Eng., Alumni, Universitas Muhammadiyah Surakarta, Surakarta, 57169, Indonesia, https://orcid. org/0000-0001-8747-9437, adrian.raflip@gmail.com Abd K. Mahamad — D. Sc., Associate Professor, Universiti Tun Hussein Onn Malaysia, Batu Pahat, Johor, 86400, Malaysia, S3 26423181700, https://orcid.org/0000-0002-3985-6549, kadir@uthm.edu.my
Bana Handaga — PhD, Lecturer, Universitas Muhammadiyah Surakarta, Surakarta, 57169, Indonesia, S3 43061222800, https://orcid.org/0000-0001-6797-6233, bana.handaga@ums.ac.id
Sharifah Saon — M. of Elec. Eng., Senior Lecturer, Universiti Tun Hussein Onn Malaysia, Batu Pahat, Johor, 86400, Malaysia, S3 35280559600, https://orcid.org/0000-0003-2706-3258, sharifa@uthm. edu.my
Endah Sudarmilah — D. Sc., Researcher, Senior Lecturer, Universitas Muhammadiyah Surakarta, Surakarta, 57169, Indonesia, S3 56122440200, https://orcid.org/0000-0002-9244-4762, Endah.Sudarmilah@ums.ac.id
Received 10.03.2022
Approved after reviewing 08.04.2022
Accepted 19.05.2022
initial study // Journal of Physics: Conference Series. 2021. V. 1858. N 1. P. 012085. https://doi.org/10.1088/1742-6596/1858/1/012085
54. Dewa C.K., Fadhilah A.L., Afiahayati A. Convolutional neural networks for handwritten Javanese character recognition // IJCCS (Indonesian Journal of Computing and Cybernetics Systems). 2018. V. 12. N 1. p. 83. https://doi.org/10.22146/ijccs.31144
55. Kurien M., Kim M.K., Kopsida M., Brilakis I. Real-time simulation of construction workers using combined human body and hand tracking for robotic construction worker system // Automation in Construction. 2018. V. 86. P. 125-137. https://doi.org/10.1016/j. autcon.2017.11.005
Авторы
Фадлила Уми — магистр, преподаватель, Университет Мухаммадия Суракарта, Суракарта, 57169, Индонезия; студент, Университет Тун Хусейн Онн Малайзия, Бату Пахат, Джохор, 86400, Малайзия, S3 57202821436, https://orcid.org/0000-0002-2261-3663, uf138@ums. ac.id
Прасетьо Раден Адриан Рафли — бакалавр, выпускник, Университет Мухаммадия Суракарта, Суракарта, 57169, Индонезия, https://orcid.org/0000-0001-8747-9437, adrian.raflip@gmail.com Махамад Абд Кадир — доктор наук, доцент, Университет Тун Хусейн Онн Малайзия, Бату Пахат, Джохор, 86400, Малайзия, S3 26423181700, https://orcid.org/0000-0002-3985-6549, kadir@uthm. edu.my
Хандага Бана — PhD, преподаватель, Университет Мухаммадия
Суракарта, Суракарта, 57169, Индонезия, S3 43061222800, https://orcid.
org/0000-0001-6797-6233, bana.handaga@ums.ac.id
Саон Шарифа — магистр, старший преподаватель, Университет
Тун Хусейн Онн Малайзия, Бату Пахат, Джохор, 86400, Малайзия,
S3 35280559600, https://orcid.org/0000-0003-2706-3258, sharifa@uthm.
edu.my
Судармила Энда — доктор наук, исследователь, старший преподаватель, Университет Мухаммадия Суракарта, Суракарта, 57169, Индонезия, S3 56122440200, https://orcid.org/0000-0002-9244-4762, Endah.Sudarmilah@ums.ac.id
Статья поступила в редакцию 10.03.2022 Одобрена после рецензирования 08.04.2022 Принята к печати 19.05.2022
Работа доступна по лицензии Creative Commons «Attribution-NonCommercial»
