CC BY
63
DOI: 10.12731/2218-7405-2013-7-28
LINGUISTIC ASPECTS OF THE DEVELOPMENT OF COMPUTER SIGN LANGUAGE INTERPRETERS RUSSIAN LANGUAGE
Ganelina N.D., Grif M.G., Korolkova O.O.
The article is devoted to the actual direction of modern linguistics - describing the features of Russian sign language identified by the analysis of theoretical and empirical lexicographical sources, as well as in the process of communication with native «speakers» - people using Russian sign language as native language. Another purpose of the article is detecting and confirming the practical significance of such research - solving problems of social adaptation for hard of hearing people. The paper presents the results of linguistic research of Russian sign language that the developers of the computer Russian sign language translator rely on: peculiarities of lexis, word derivation, morphology and syntax of sign language for deaf citizens of the Russian Federation are described. Approaches to developing a system for computer sign language translation used by the employees of the Institute of Social Rehabilitation, Novosibirsk State Technical University, are justified in the paper. The results of the research can be interested to specialists studying and describing lexical and grammatical systems of Russian sign language, developers of the system for a two-way machine translation from Russian spoken language to Russian sign language and vice versa, sign language interpreters and teachers of sign language.
Keywords: Russian sign language; Russian language; Computer Sign Language Translation.
ЛИНГВИСТИЧЕСКИЕ АСПЕКТЫ РАЗРАБОТКИ КОМПЬЮТЕРНОГО СУРДОПЕРЕВОДЧИКА РУССКОГО ЯЗЫКА
Ганелина Н.Д., Гриф М.Г., Королькова О.О.
Статья посвящена актуальному направлению современного языкознания - описанию особенностей русского жестового языка, выявленных при анализе теоретических, лексикографических и эмпирических источников, а также в процессе общения с носителями русского жестового языка и раскрытию практической значимости подобных исследований для решения проблемы социальной адаптации инвалидов по слуху. В работе представлены результаты лингвистического исследования русского жестового языка, на которые опираются разработчики компьютерного сурдопереводчика русского языка: дано описание особенностей лексики, словообразования, морфологии и синтаксиса жестового языка глухих и слабослышащих граждан Российской Федерации. В работе обоснованы подходы к созданию системы компьютерного сурдоперевода русского языка, используемые сотрудниками института социальной реабилитации Новосибирского государственного технического университета. Результаты исследования представляют интерес для специалистов, занимающихся изучением и описанием лексической и грамматической системы русского жестового языка, разработчиков системы двустороннего машинного перевода с русского звучащего языка на русский жестовый язык и с русского жестового языка на русский звучащий язык, специалистов-сурдопереводчиков и сурдопедагогов.
Ключевые слова: русский язык; русский жестовый язык; компьютерный сурдопереводчик.
Today world community is actively resolving problems of breaking the barriers in communication between hearing and hear-impaired people and their social adaptation. It is known that deaf and people with hearing disabilities are about 10% of
world population. These people use for communication sign language which can be unknown for hearing people they speak to. Discussing sign languages we should mention that during a long time sign languages were not considered to be full and independent languages in spite of the fact that such a language is one of the types of natural languages distinguishing by the kind of presenting «based only on gesture and mimic» [7, p.44].
Different machine sign language translation systems are developed to resolve several problems, including deficiency of professional sign language interpreters, problem of communication and social integration for hard of hearing people. In Novosibirsk State Technical University, for example, a team of specialists is designing a two- way computer sign language translator [11, 13, 14].
To address this important social problem it is necessary to make a research of linguistic aspects of Russian sign language (RSL). By comparative analysis of RSL and Russian spoken language lexical and grammatical features of RSL used by deaf and hear-impaired citizens of Russia should be detected. The idea of necessary study of RSL was formulated by A.A. Kibrik in 1990s [4]. In 2000s under his direction investigation of some aspects of RSL were started. But grammatical system of RSL is not described yet. The most complete and holistic paper devoted to RSL is a textbook «Gestural language. Dactylology» written by well-known Russian defectologist G.L. Zaytseva [1].
Importance and urgency of our research is proved by announcing Russian sign language to be official language of deaf and hard of hearing citizens of the Russian Federation in December, 2012. In Federal Law № 296-FL «On alteration of 14 and 19 articles of Federal Law «On social defense of disabled people in Russian Federation» passed at December 30, 2012 it is stated: «in article 14: a) the second part to be as follows: Russian sign language is avowed to be a language for communication in case of hear and/or speech difficulties, including spheres where verbal official language of the Russian Federation is used... g) to annex by the fifth part with the following content: Training, raising of qualification and professional retraining for
teachers and interpreters of Russian sign language, as well as Russian sign language development is provided» [10].
Our linguistic research of Russian sign language is based on G.L. Zaytseva thesis: «RSL is a full human language with own lexical and grammatical systems» [1, p. 32].
The analysis of theoretical, empirical and lexicographical sources, as well as communication with people for whom Russian sign language is native, let us conclude that RSL has the following features of vocabulary, derivation, morphology and syntax.
Russian sign language vocabulary is characterized by a tendency to syncretism and separation. On one hand, in RSL the same gesture can be used to present an action, an executor and a tool for action. The actual denotatum is defined by the context. On the other hand, in Russian sign language there is not always a certain sign to transfer any information and people have to resolve this problem in the process of communication using available gestures.
Gestures are iconic, it is a feature of vocabulary of sign languages including RSL. It means that signs perform denotatum, show it or look like and resemble it.
The third feature of RSL vocabulary is a possibility to form figurative meaning.
One more peculiarity of RSL lexical system is distinguishable vocabulary, i.e. there are expressions that can not be translated to spoken language by one word.
There are system relations in RSL vocabulary. Thus, we defined and described gestures that are similar to antonym, synonym and polysemantic words.
An important feature of RSL vocabulary is a possibility to be enriched by loan words and adoptions from world sign and spoken languages.
RSL vocabulary is also characterized by changing its composition, gestures appear and disappear or perform in different way.
We defined that Russian sign language vocabulary has thematic groups which can be divided into subgroups and micro groups.
And the last feature of RSL vocabulary is small scope as compared to Russian spoken language. This fact perhaps let scientists assume that sign language was not full and complete.
The analysis of gestures in terms of word formation allowed us to define the following peculiarities of RSL derivation system [6].
RSL word formation is more focused on expressing morphological features of words than on creating new gestures.
Principal units of RSL derivation system are word formative chains, paradigms and families of words. Motivating and motivated gestures are defined in and motivating gestures in Russian spoken language do not always title motivating gestures.
There are no tools in RSL derivation system that fully correspond to word formants of Russian language. But the system has specific means for new gestures formation: combination of two separate gestures, appending to nominative gesture special additional gestures (for example, a gesture meaning PERSON), repeating additional gesture, changing gesture amplitude or intensity and its localization, transforming one-handed gesture to two-handed, using facial expression and/or body turning in the process of gesture performing. Applying systematically tools mentioned above let us define in Russian sign language special word formative models which sometimes have analogues in Russian spoken language, for example prefixal, suffix and stem composing methods.
There are special cases of gesture formation in RSL: gestures that in terms of word formation in Russian language are cognate words, but not included in derivation chains, are similar in performing; gestures which analogues in Russian language are cognate words are identical in performing; gestures which names in Russian language are cognate words are diverse in performing.
In RSL research tradition initiated by G.L. Zaytseva [1] gestures were not divided into parts of speech. The well-known Russian defectologist explains this by the fact that there are gestures which can denote an action, agent or tool depending on the
context. For example, the same gesture in RSL can denote TO SKI, SKIER and SKIING.
As a result of analyzing lexicographical, theoretical and empirical sources we concluded that classes of gestures analogous to parts of speech in Russian language (noun, adjective, numeral, pronoun, verb, predicate noun, adverb, participle, modal words, preposition, conjunction, particle, interjection) can be defined in RSL. These classes also can be described by categories of analogues in spoken language [5].
The feature of RSL morphology is that gestures analogous to parts of speech do not have all grammatical categories peculiar to their spoken equivalents. For example, noun gestures do not have case category, adjective gestures do not have gender and case categories.
The key part of speech in Russian sign language is verb. Gestures that are analogous to verbs can present aspect meaning, mode, tense (the paradigm of tense forms includes preterit and perfect), person, number and reflexive meaning. We should note that all these forms are created analytically. There is no gender category for verb gestures. As in Russian spoken language verb gestures in RSL realize a predicate function.
There are groups of numeral gestures in RSL (TWO, THREE, FOUR, FIVE), which combine paradigm subgroups. Gestures in subgroups denoting some values or dimension (time, weight, cost and others) differ from each other by the type of movement. RSL peculiarity is gesture incorporation, that is special way of presenting quantitative relation when a gesture denoting a number less than 5 is incorporated to nominative gesture denoting noun. As a result of incorporation new complicated gestures identical to word combination noun + numeral are formed.
All personal pronouns in RSL perform designative or indicating function. In Russian spoken language deictic function is attributed only to pronouns of the first and second person.
In RSL as opposed to spoken language there is a tendency for interrogative pronouns and negative participles to take postpositions in statements.
Prepositions in RSL are not tools for hypotaxis of one meaningful word to another in a phrase or sentence, but a way to present spatial relations.
Describing features of RSL morphology we can not but mention that basic means for presenting grammatical meaning are gesture configuration and movement.
Syntax in RSL like in Russian spoken language performs a leading function. RSL syntax system has some features.
Basic syntactic element in RSL as opposed to Russian spoken language is a statement.
An important feature of RSL syntax is basic order of gestures in a statement, including interrogative sentence. This order depends on reversibility of the sentence, aspect mark of a verb, as well as on animateness and prevalence of object as well as on the part of speech the gesture belongs to.
The syntax of Russian sign language can be described in terms of subject (subject in a sentence) - object (object in a sentence) - predicate (verb). Differentiation of subject and object is based on the features of gesture order in a statement, changing direction of gesture performing and localization of referents.
As sentences in Russian spoken language statements in RSL are classified to simple and composite, and not all of them have analogues in spoken language.
Sign languages are soundless, so to express a question you should use not intonation but other tools. These tools are non-manual component, interrogative gestures and particles.
One more important feature of RSL syntax is predicate doubling, repeating parts of a statement or whole phrases. The necessity of adjustment, explanation and appending of the stated is a reason of these phenomena.
The last feature of RSL syntax is segmentation of discourse to elementary discourse units (EDU) and super discourse units (SDU) that are analogous to word combinations in a simple sentence and to a simple sentence in composite sentences [10].
So everything stated above let us conclude that Russian sign language is a full natural language with specific features at every level of language system which
should be taken into account in the process of developing machine sign language translation systems.
There are two approaches to designing automated sign language translators: rule-based and data-based translation [12, 15]. In the first type systems rules of translation are created manually based on knowledge about both languages. In the systems of the second type rules are derived automatically based on computer analysis of languages data. The analysis doesn't rely on prior knowledge about structure of languages considered. An example of the second type system is statistical automated translation. As soon as there is no sufficiently representative corpus of parallel texts which the system can use to learn, elicit rules for correlation between texts in Russian spoken language and RSL from it, this approach doesn't seem to be realizable at the current moment.
Systems of the first type can differ in depth analysis of the translated text.
Direct translation is based on found lexical matching without syntactic (and of course, semantic) analysis. «Transfer» translation is based on more sophisticated analysis at syntax and semantic level. Syntactic or semantic representation of the text translated transforms to syntactic or semantic representation of another language based on the rules of the translation system. This transformed representation is the basis for generating output text. Such a translation strategy goes to ideas stated in the end of 1950s by V. Yngve.
Even deeper analysis takes place in translation based on designing "interlin-gua" - artificial interim language. This strategy of translation originates in W. Weaver's ideas. It consists in using a translation circuit which transforms a text in one language to a text in another language on the basis of reduction both texts to a common for them semantic representation in interlingua. This notation theoretically doesn't depend on the chosen languages L1 an L2.
There are two types of strategy for designing interlingua:
1. Designing a universal language description that generalizes syntactic and semantic means of different natural languages.
2. Designing an interlingua that doesn't simulate natural languages, but a world which these languages are used to speak about. Formalization of knowledge about the world makes it possible to include in translation process procedures of logical inference based on common sense reasons.
As an example let's consider software package for computer translation from Russian text to RSL (programming language is C++, OS Win 32) [11]. Russian text can be arbitrary. It can include numbers and acronyms. The version of colloquial Russian sign language is based on notations for sign writing SignWriting [2, 3, 9]. Software developed includes modules for morphologic, syntactic and semantic analysis of Russian texts as well as the module for translating to colloquial Russian sign language. Any Russian text (sentence) entered to the program is provided for morphologic, syntactic and semantic analysis. Then a standard scheme of sentence in colloquial Russian sign language is formed:
SUBJECT - ATTRIBUTE - ADJUNCT - PREDICATE - OBJECT.
This scheme is consecutively applied to simple statements.
If a word in a sentence has a corresponding gesture then the word is changed by the notation system in SignWriting for this gesture. If there is no corresponding gesture - the word is translated by dactyl alphabet in the notation system. To remove polysemy in gesture performing a base of ontology is used. Possibility of parallel performing of statements is analyzed.
In the SignWriting notation system 500 Russian signs are described. All gestures were normalized and approved by Russian experts in colloquial RSL. The strategy for translating Russian text to Russian sign language is focused on the closest in the meaning and ergonomic translation. It includes loan Russian sign language and dactyl alphabet.
References
1. Zaytseva G.L. Zhestovaya rech'. Daktilologiya [Gestural language. Dactylology]: A textbook for university students. Moscow, 2000. 192 p.
2. Karpov A.A. Komp'yuternyy analiz i sintez russkogo zhestovogo yazyka [Computer analysis and synthesis of Russian sign language]. Voprosy yazykoznaniya [Problems of Linguistics], no. 6 (2011): 41-53.
3. Karpov A.A., Kagirov I.A. Formalizatsiya leksikona sistemy komp'-yuternogo sinteza yazyka zhestov [Formalization of lexicon for the system of computer sign language synthesis]. St. Petersburg: Proceedings of SPIIRAS. 2011, Issue 1(16): 123-140.
4. Kibrik A.A. O vazhnosti lingvisticheskogo izucheniya russkogo zhestovogo yazyka [The importance of the linguistic study of Russian Sign Language]. http://signlang.ru/science/read/kibrik1/ (accessed September 3, 2010).
5. Berezina O.A., Bizyukov N.V., Korolkova O.O. et al. Morfologiya russkogo zhestovogo yazyka [The morphology of Russian Sign Language]. Lingvistika i mezhkul'turnaya kommunikatsiya [Linguistics and Intercultural Communication]. Vol. 3. Krasnoyarsk: Research and Innovation Center, 2011. pp. 60-87.
6. Korolkova O.O. Osobennosti slovoobrazovaniya russkogo zhestovogo yazyka [Features of derivation in Russian Sign Language]. Sovremennaya lingvistika i mezhkul'turnaya kommunikatsiya [Modern linguistics and intercultural communication]: Monograph. Krasnoyarsk: Research and Innovation Center, 2012. pp. 98-152.
7. Prozorova E.V. Rossiyskiy zhestovyy yazyk kak predmet lingvisticheskogo issledovaniya [Russian sign language as a subject of linguistic investigation]. Voprosy yazykoznaniya [Problems of Linguistics], no. 1 (2007): 44-61.
8. Prozorova E.V. Markery lokal'noy struktury diskursa v russkom zhestovom yazyke [Markers of the local structure of discourse in Russian sign language]: Abstract for PhD thesis in Philolgy. Moscow, 2009. 24 p.
9. Ronzhin A.L., Karpov A.A., Li I.V. Rechevoy i mnogomodalnyy in-terfeysy [Speech and multimodal interfaces]. Moscow, 2006. 173 p.
10. The Federal Law № 296-FZ of 30 December 2012 «On Amendments to Articles 14 and 19 of the Federal Law «On social protection of disabled people in the
Russian Federation». http://docs.cntd.ru/document/902389665 (accessed December 31, 2012)
11. Grif M.G., Korolkova O.O. International Journal of Advanced Studies 2, no. 3 (2012). http://ijournal-as.com/issues/2012/3/korolkova.pdf (accessed November 30, 2012).
12. Huenerfauth M. A Survey and Critique of American Sign Language Natural Language Generation and Machine Translation Systems. Technical Report MS-CIS-03-32, Computer and Information Science, University of Pennsylvania. 2003. 36 p.
13. Mikhail G. Grif, Olga O. Korolkova, Yevgeny B. Tsoy. Development of Computer Sign Language Translation Technology for Deaf People regarding Russian Sign Language Peculiarities. MERI Journal of Management & IT (Special Issue). Vol. (5). Num. (2). April 2012: 40-46.
14. Mikhail G. Grif, Yelena A. Demyanenko, Olga O. Korolkova, Yevgeny B. Tsoy. Computer Sign Language Translation System for Hearing Impaired Users. Proceedings of IFOST 2012. The 7th International Forum of Strategic Technology IFOST 2012. September, 17-21, 2012. Tomsk Polytechnic University. Vol. 1. pp. 617-620.
15. Morrissey S. Data-Driven Machine Translation for Sign Languages. PhD Thesis, Dublin City University, Dublin, Ireland, 2008. 192 p.
Список литературы
1. Зайцева Г.Л. Жестовая речь. Дактилология: Учебник для студентов высших учебных заведений. М., 2000. 192 c.
2. Карпов А.А. Компьютерный анализ и синтез русского жестового языка // Вопросы языкознания, 2011. № 6. С. 41-53.
3. Карпов А.А., Кагиров И.А. Формализация лексикона системы компьютерного синтеза языка жестов. СПб: Труды СПИИРАН. 2011. Вып. 1(16). С. 123—140.
4. Кибрик А.А. О важности лингвистического изучения русского жесто-вого языка. URL: http://signlang.ru/science/read/kibrik1/ (дата обращения: 03.09.2010)
5. Королькова О.О. Морфология русского жестового языка // Лингвистика и межкультурная коммуникация: монография. Кн. 3. / О.А. Березина, Н.В. Бизюков, О.О. Королькова [и др.]. Красноярск, 2011. С. 60-87.
6. Королькова О.О. Особенности словообразования русского жестового языка // Современная лингвистика и межкультурная коммуникация: монография / О.А. Березина, Е.Н. Грушецкая, Т.В. Ицкович [и др.]. Красноярск, 2012. С. 98-152.
7. Прозорова Е.В. Российский жестовый язык как предмет лингвистического исследования // Вопросы языкознания. 2007. № 1. С. 44-61.
8. Прозорова Е.В. Маркеры локальной структуры дискурса в русском жестовом языке: Автореф... канд. филол. наук. М., 2009. 24 c.
9. Ронжин А.Л., Карпов А.А., Ли И.В. Речевой и многомодальный интерфейсы. М., 2006. 173 с.
10. Федеральный закон N 296-ФЗ от 30 декабря 2012 г. «О внесении изменений в статьи 14 и 19 Федерального закона «О социальной защите инвалидов в Российской Федерации». URL: http://docs.cntd.ru/document/902389665 (дата обращения: 31.12.2012).
11. Grif M.G., Korolkova O.O. Development of systems for computer sign language // International Journal of Advanced Studies, vol. 2, no. 3 (2012). URL: http://ijournal-as.com/issues/2012/3/korolkova.pdf (accessed November 30, 2012).
12. Huenerfauth M. A Survey and Critique of American Sign Language Natural Language Generation and Machine Translation Systems. Technical Report MS-CIS-03-32, Computer and Information Science, University of Pennsylvania. 2003. 36 p.
13. Mikhail G. Grif, Olga O. Korolkova, Yevgeny B. Tsoy. Development of Computer Sign Language Translation Technology for Deaf People regarding Russian
Sign Language Peculiarities // MERI Journal of Management & IT (Special Issue). Vol. (5). Number (2). April 2012. P. 40-46.
14. Mikhail G. Grif, Yelena A. Demyanenko, Olga O. Korolkova, Yevgeny B. Tsoy. Computer Sign Language Translation System for Hearing Impaired Users // Proceedings of IFOST 2012. The 7th Internation Forum of Strategic Technology IFOST 2012. September, 17-21, 2012. Tomsk Polytechnic University.Volume 1. P. 617-620.
15. Morrissey S. Data-Driven Machine Translation for Sign Languages. PhD Thesis, Dublin City University, Dublin, Ireland, 2008. 192 p.
DATA ABOUT THE AUTHORS
Ganelina Natalia Davidovna, Associate Professor of Department of Automatic Control Systems, PhD in Engineering
Novosibirsk State Technical University
20, K.Marksaprospectus, Novosibirsk, Novosibirsk Region, 630073, Russia e-mail: natalie_ganelina@ngs. ru
Grif Mikhail Gennadyevich, Head of Department of Automatic Control Systems, PhD in Engineering, professor
Novosibirsk State Technical University
20, K.Marksa prospectus, Novosibirsk, Novosibirsk Region, 630073, Russia e-mail: grifmg@mail. ru
Korolkova Olga Оlegovna, Associate Professor of Psychology and Pedagogy Institute of the natural and socio-economic sciences, PhD in Philology
Novosibisk State Pedagogical University
28, Viluyskaya street, Novosibirsk, Novosibirsk Region, 630126, Russia e-mail: ookorol@mail. ru
ДАННЫЕ ОБ АВТОРАХ
Ганелина Наталья Давидовна, доцент кафедры автоматизированных систем управления, кандидат технических наук
Новосибирский государственный технический университет
Пр. К.Маркса, 20, г. Новосибирск, Новосибирская область, 630073, Россия
e-mail: natalie_ganelina@ngs. ru
Гриф Михаил Геннадьевич, заведующий кафедрой автоматизированных систем управления, доктор технических наук, профессор Новосибирский государственный технический университет Пр. К.Маркса, 20, г. Новосибирск, Новосибирская область, 630073, Россия e-mail:grifmg@mail. ru
Королькова Ольга Олеговна, доцент кафедры психологии и педагогики Института естественных и социально-экономических наук, кандидат филологических наук
Новосибирский государственный педагогический университет
Ул. Вилюйская, 28, г. Новосибирск, Новосибирская область, 630126, Россия
e-mail: ookorol@mail. ru
