CC BY
36
UDC 81'139 DOI: 10.18413/2313-8912-2023-9-1-0-6
Alan Cienki1 Anna V. Leonteva2 Olga V. Agafonova3 Andrey A. Petrov4
Numbers in simultaneous interpreting: a multimodal analysis
1 Vrije Universiteit Amsterdam 1105 De Boelelaan, Amsterdam, 1081 HV, Netherlands E-mail: a.cienki@yu.nl
2 Moscow State Linguistic University
38 Ostozhenka St., Moscow, 119034, Russia
Institute of Linguistics, RAS 1 B. Kislovsky Ln., Moscow, 119019, Russia E-mail: lentevania27@gmail.com
3 Moscow State Linguistic University
38 Ostozhenka St., Moscow, 119034, Russia E-mail: olga.agafonova92@gmail.com
4 Moscow State Linguistic University
38 Ostozhenka St., Moscow, 119034, Russia E-mail: petrov@linguanet.ru
Received 10 January 2023; accepted 01 March 2023; published 30 March 2023
Acknowledgements. The research is financially supported by the Russian Science Foundation, Project 19-18-00357 "Verbal and co-verbal behavior under cognitive load: analyses of speech, gesture, and eye gaze" at Moscow State Linguistic University.
Abstract. Simultaneous interpreting (SI) is considered to be a highly demanding cognitive task, especially in regard to elements which represent a stumbling block during SI (e.g. numbers, idiomatic expressions, etc.). The current study examines the use of numbers in SI in combination with co-speech gestures. The issue of interpreting numbers has been addressed by different scholars, however it has rarely been studied from the multimodal perspective so far. Given the known role of gesture during processes of lexical retrieval, we hypothesize that the use of co-speech manual gestures plays a significant role in the process of interpreting numbers, i.e. facilitates mental retrieval of the target number. In addition to the hypothesis to be tested, we consider the exploratory question as to whether any gestures with numbers correlate more with an externalized or internalized function, that is: either representation of the numerical concept externally (metaphorical depiction of the quantified entity as an object or space that could be touched or pointed to) or more with self-adapters to help the interpreter manage the cognitive load/stress internally. A corpus of 10 recordings of English-to-Russian SI sessions was analyzed. The interpreting of numbers was analyzed as being correct, incorrect or absent. The
analysis of the material showed that the participants mostly interpreted numbers correctly or omitted them in their speech. The results of the analysis of co-speech gestures demonstrated nearly equal use of gesture and no gesture when interpreting numbers, however, there was a significant correlation between interpreting certain number categories and gesture use. Then the following types of gesture functions were analyzed: adapters, pragmatic, representational and deictic. The results indicate that self-adapters were the most frequent gesture type used. This finding can be interpreted based on the literature showing self-adapters ease tension and anxiety and/or help one gain control of stressful situations.
Keywords: Simultaneous interpreting; Cognitive load; Multimodality; Interpreting numbers; Gestures
How to cite: Cienki, A., Leonteva, A. V., Agafonova, O. V. and Petrov, A. A. (2023). Numbers in simultaneous interpreting: a multimodal analysis, Research Result. Theoretical and Applied Linguistics, 9 (1), 82-98. DOI: 10.18413/2313-8912-20239-1-0-6
УДК 81-139 DOI: 10.18413/2313-8912-2023-9-1-0-6
Ченки A.1
Леонтьева A. B.2 Числительные в синхронном переводе:
Агафонова О. В.3 полимодальный анализ
Петров А. А.4
1 Свободный университет Амстердама De Boelelaan, 1105, Амстердам, 1081 HV, Нидерланды E-mail: a.cienki@vu.nl
2 Московский государственный лингвистический университет
ул. Остоженка, 38, Москва, 119034, Россия Институт языкознания РАН Б. Кисловский пер., 1, Москва, 119019, Россия E-mail: lentevanja27@,gmail. com
3 Московский государственный лингвистический университет
ул. Остоженка, 38, Москва, 119034, Россия E-mail: olga. agafonova92@gmail. com
4 Московский государственный лингвистический университет
ул. Остоженка, 38, Москва, 119034, Россия E-mail: petrov@linguanet ru
Статья поступила 10 января 2023 г.; принята 01 марта 2023 г.; опубликована 30 марта 2023 г.
Информация об источниках финансирования или грантах: Исследование выполнено при финансовой поддержке Российского научного фонда, проект № 19-18-00357 «Вербально-телесное поведение говорящего в условиях
повышенной когнитивной нагрузки: анализ речи, жестов и движения глаз» в Московском государственном лингвистическом университете.
Аннотация. Синхронный перевод (СП) считается сложной когнитивной задачей, особенно в отношении элементов, которые представляют собой затруднения во время СП (например, числа, идиоматические выражения и т.д.). В данном исследовании рассматривается использование цифр в СП в сочетании с жестами в речи. Проблема перевода чисел рассматривалась разными учеными, однако до сих пор она редко изучалась в полимодальных исследованиях. Учитывая известную роль жестов в процессах лексического поиска, мы выдвигаем гипотезу, что использование жестов рук в речи играет значительную роль в процессе перевода числительных, т.е. облегчает мысленный поиск нужного числа. В дополнение к гипотезе, нами был поставлен исследовательский вопрос о том, соотносятся ли жесты с числами в большей степени с экстернализированной или интернализованной функцией, то есть: либо представление числовой концепции происходит извне (метафорическое изображение количественной сущности как объекта или пространства, к которому можно прикоснуться или на который можно указать), или сопровождается самоадаптерами, чтобы помочь переводчику внутренне справиться с когнитивной нагрузкой / стрессом. Был проанализирован корпус из 10 записей СП с английского языка на русский. Перевод чисел был проанализирован как правильный, неправильный или опущенный. Анализ материала показал, что участники в основном либо правильно переводили цифры, либо опускали их в своей речи. Результаты анализа жестов, сопровождающих речь, продемонстрировали почти равное использование жестов или их отсутствие при переводе чисел, однако наблюдалась значительная корреляция между переводом определенных категорий чисел и использованием жестов. Затем были проанализированы следующие типы функций жеста: адаптеры, прагматические, репрезентирующие и дейктические. Результаты показывают, что адаптеры были наиболее часто используемым типом жестов. Этот вывод может быть интерпретирован на основе литературы, показывающей, что использование самоадаптеров снимает напряжение и тревогу или / и помогает человеку обрести контроль над стрессовыми ситуациями.
Ключевые слова: Синхронный перевод; Когнитивная нагрузка; Полимодальность; Перевод числительных; Жесты
Информация для цитирования: Ченки А., Леонтьева А. В., Агафонова О. В., Петров А. А. Числительные в синхронном переводе: полимодальный анализ // Научный результат. Вопросы теоретической и прикладной лингвистики. 2023. Т. 9. № 1. С. 82-98. DOI: 10.18413/2313-8912-2023-9-1-0-6
1. Introduction
Simultaneous interpreting (SI) is regarded as a cognitively demanding task which requires a lot of effort from interpreters. According to the "Tightrope Hypothesis" (Gile, 1995) different processes and requirements emerging during the working process involve nearly the entire
cognitive processing capacity of the speaker. As Gile mentions, when total processing capacity requirements are high, interpreters might find it difficult to perform the task. Such situations can occur due to the quality of the source speech e. g. it can be too fast, heavily accented, etc., or it can contain words which are regarded to be difficult to interpret
such as unfamiliar names, idiomatic expressions, numbers etc. The overall saturation which might appear due to such triggers can lead to different errors and influence the quality of the target speech (Gile, 2008).
1.1 Simultaneous interpreting and cognitive load
In the literature, the process of SI is commonly viewed from the perspective of the Effort Model (Gile, 2008), a cognitive framework which presents SI as cognitive online operations. Such operations can be grouped into "Listening (and Analysis) Effort", e.g. operations which help interpreters comprehend the source speech; "Production Effort", aimed at producing target speech, using self-monitoring and self-correction; and "Memory Effort" operations responsible for storage and retrieval of information in short-term memory; "Coordination Effort" is responsible for attention distribution and shifts between the aforementioned types.
Cognitive load is a multidimensional phenomenon that presumes the existence of a strain on mental processes (i.e., attention, thinking, working memory, etc.) while performing a cognitive task. SI is one such task that is highly demanding of cognitive resources (Gosy, 2007; Dayter, 2020). There is a peculiarity of SI, however: while perceiving the input signal (the source text), the interpreters need to produce the output signal (the interpreting). This process includes information decoding (from one language to the other) and involves different types of mental processes working at the same time: attention, thinking, perception, reasoning and memory. Cognitive load may result in speech disfluencies in SI (Stachowiak-Szymczak, 2019). According to the Cognitive Load Model by K. Seeber (Seeber, 2011) and D. Gile's Effort Model (Gile, 2009) interpreting incorporates a language comprehension task, a language production task, memory storage and a coordination task. Achieving balance between these cognitive challenges requires self-management in order to reduce cognitive
load. Hand gestures can serve such a function, along with interactive and communicative functions (Poyatos, 1987/2002).
Research performed by K. Seeber (Seeber, 2012) shows that simultaneous interpreters are searching for information that is complementary to the speech, which can be available in the form of visual stimuli (e.g., hand / body movements of the speaker, additional information given on the slides of a presentation). This is relevant especially while translating numerals, taking into account that they are associated with an increase of cognitive load.
1.2 Cognitive load and gesture use
Gestures have been demonstrated to play various roles in relation to how people manage a cognitive load. One way is in how they help speakers manage the concepts they are working with. While McNeill's (1985, 1992) work particularly focused on the role gestures play as people formulate their ideas when speaking, Kita et al. (2017) argue that gestures have a broader self-oriented function, namely in helping people with processes of conceptualization, particularly if they involve spatial concepts. For example, participants in one experiment (Chu and Kita, 2008) viewing unusual three-dimensional objects had to picture what they would look like if rotated in certain directions; the participants made significant use of gestures while imagining rotating the objects, even though they did not speak during the task.
Another well-known function of gestures in assisting people manage a heavy cognitive load is in helping them maintain focus and even calming themselves. Here what are called self-adapters (Ekman and Friesen, 1969) play a fundamental role. Such body-focused movements (Freedman, 1972), such as rubbing one's hands, stroking one's hair, etc., play a role not only in calming oneself down, but also in gathering one's ideas and maintaining a mental focus (Neff et al., 2011). In this regard, gesture use has an internal-regulating function, when one is experiencing various kinds of cognitive stress in general (Lin et al., 2020), and specifically
when engaged in the process of SI, as noted in Cienki and Iriskhanova (2020).
1.3 Numbers in interpreting
Lexical units used for naming numbers and numerals are a special category which lies between language and mathematics. Such words are an interesting case of shaping and expressing these abstract categories from our mind into speech. Our cognitive ability to subitize, i.e., to understand the number of things from the first glance, is regarded to be inborn (Mandler and Shebo, 1982). We also have several cognitive capacities that help us to deal with numbers. We have numerosity, which is reflected in our ability to estimate roughly the number of objects in a group; grouping, which allows us to group objects mentally, visually, etc.; ordering and pairing as we put objects into a specific order and group them; memory capacity to keep track of objects that are being counted; exhaustion-detection capacity to detect the absence of objects to be counted; cardinal-number assignment to assign a cardinal number of the group of objects to estimate its size; independent-order capacity to understand that the assigned cardinal number is independent of the order of the counted objects; combinatorial-grouping capacity to put small groups into larger units; and a symbolizing capacity to associate words with numbers (Lakoff and Nunez, 2000).
In general, as we know, numbers are used to count, as this is the first thing we learn about numbers as children through the counting sequence in one's native language. Numbers express cardinality, which is applied to some set of objects. However, it is argued that cardinality is not the only property of numbers. Numbers possess a certain flexibility, as there are no objects, imagined or real, that they cannot be applied to. They can be applied to objects in three ways: cardinal (used with sets of objects, e.g. five boys), ordinal (used with objects, that are a part of some progression, e.g. the fifth boy) and nominal (used with objects, that are a part of a set, e.g. boy #5). The role of number words is still being debated in linguistics due to their
semantics and relations to the context. There are different semantic types of numerals. Numerals are regarded as determiners, modifiers, number-denoting words, and degree quantifiers, and are also used to denote exhaustivity and scope (Bylinina and Nouwen, 2020).
There are two types of representation which are involved in the processing of numbers: syntactic and lexical. Syntactic representation specifies the number of digits involved in writing numbers of that category (e.g. tens, thousands, etc.). Lexical representation adds to the syntactic by specifying the basic quantity (from 1 to 9).
There are various categories which give rise to objective difficulties for interpreters in their work. These categories are known as problem triggers and result in high error rates, compared to the interpretation of other words (Gile, 2009). Numbers are regarded as one of the main categories that pose a difficulty to interpreters of any level (Mazza, 2001; Mead, 2015; Korpal, 2017).
Interpreting numbers can be performed in different ways: when an interpreter is alone (stays isolated in the booth), they can write down numbers; they can use complementary information given on slides; an interpreter can render prepared speech (e. g. a lecture or any other type of prepared talk); when an interpreter in not alone in the booth, their colleague can write down numbers and then they can be read off. A study conducted by C. Collard and B. Defranq with interpreters in the European Parliament showed that error rates for numbers amounted to approximately 18% (Desmet et al., 2018).
There are different reasons why numbers might be difficult to interpret. Numbers are regarded to be very informative as they have neither redundant material (Seeber, 2015), nor a conceptual representation (Timarova, 2012; Seeber, 2015); when we imagine a number, we usually imagine some objects that have this number, rather than the abstract notion itself. Thus, they are less predictable for interpreters and it creates a difficulty (Pinochi, 2009;
Mead, 2015). Due to this, interpreters cannot use such strategies as reformulation or paraphrasing, as components of numbers can have only one exact meaning (Pinochi, 2009). It causes a switch between types of interpreting: from 'intelligent' to 'literal' interpreting (Braun and Clarici, 1996), which might cause more errors.
Numbers might also be difficult to interpret due to the source text and differences in language structures (Pinochi, 2009). If a number is uttered, the chance for it to be held for interpretation later in the utterance is minimal. Moreover, numbers carry precise pieces of information: when this piece of information is omitted or misinterpreted, the communication process may suffer (Korpal and Stachowiak-Szymczak, 2020). Simultaneous interpreting of numbers is considered to be difficult in all languages (Pinochi, 2009).
In the current study we explore: (1) the interpreters' verbal behavior while they are rendering numbers from the source text into their target language (whether there are more cases of correct interpreting of numbers than incorrect); (2) which subcategory of interpreting numbers is more frequent: 'correct', 'approximation', 'absent' or 'wrong'; (3) if interpreting numbers is mostly accompanied by gestures, i.e., whether there is a correlation between such categories as 'correct', 'approximation', 'wrong' and the presence of a gesture.
2. Material and methods
The analysis is based on material taken from 10 videos, approximately 10 minutes each, with each produced by a different interpreter. The total video corpus comprises 100 minutes. The participants performed the
L1). The interpreters were master's students; beginners, who had been interpreting for less than 3 years; and experienced professionals, who had been interpreting for more than 3 years. They each heard the same portion of a popular science lecture (a TED Talk) in English, only hearing the audio through headphones and not seeing the video (so as not to be influenced by the original speakers' gestures). Due to the initial purposes of our studies, the interpreters were not allowed to take any objects (e.g. pen and paper) to write down any remarks, including numbers, during the process. Though this is a common practice in the interpreting field, our study focused on how interpreters would handle a high cognitive load using only 'natural media' (Gibbon, 2005), including the potential role of gesture with speech. Allowing use of any implements or technology would change the nature of the interpreters' processes of thinking for speaking (Slobin, 1987), which would have fundamentally altered the nature of the research project.
The video material was recorded using three cameras. A GoPro camera was installed in front of the participants to record the frontal view of the speaker, which allowed us to see their gestures precisely in order to annotate them later. For even better precision, for example in case the interpreter put his/her hands on their knees under the table, a Sony camera was installed on a tripod behind them to see such gestures. The third camera was in the eye-tracking glasses, Tobii Pro II, which the participants had on during the whole process of interpreting. These glasses showed the first-person perspective and allowed the coders to double check hand movements of the speakers (see Figure 1).
SI from English into Russian (from L2 into
Figure 1. The three perspectives that were recorded: frontal, over-the-shoulder, and from the eye-tracking glasses
Рисунок 1. Расположение камер, на которые осуществлялась запись: на столе напротив переводчика, в верхнем углу переводческой кабины и камера на очках-айтрекере
After the videos were recorded, they were annotated in ELAN, a special program used to analyze verbal and nonverbal behavior (https://archive.mpi.nl/tla/elan). For the purposes of our study, we analyzed numerals in terms of their assignment to the object (cardinal, ordinal or nominal) and their use in the source text compared to how they were interpreted by the participants. Then the interpretations of the numbers were put into two major categories according to their accuracy: correct and incorrect. Under 'correct interpreting' we understand the direct translation of the source number e. g. five hundred species - пять сотен/пятьсот животных or five tons of weight -пятитоннный. The correct category means that the idea is being interpreted without any corruptions and is expressed in similar categories in the target language. The 'incorrect' category has three subcategories: approximation, wrong and absent. The first subcategory is assigned to the number if it was interpreted with some corruption in its meaning, thus the idea is partly lost e. g. one hundred years - несколько лет; millions and millions years - сто миллионов лет;
1598 (year) - в шестнадцатом веке. The second category, wrong, was applied in case the main idea of the source number was lost and the word used in the target language had a completely different meaning, e. g. 1844 (year) - в двадцатом веке; sixty - шесть; thousands and thousands - сотни и сотни. The last category, absent, was marked if the numeral from the source text was not interpreted by a participant at all, thus was absent in their speech.
The analysis of the nonverbal behavior of the participants included the study of their manual co-speech gestures. There were several highlighted types of gestures, taken into account: adapters, representational, pragmatic and deictic.
Adapters are such gestures that have no referential semantic function. They consist of simple movements like scratching oneself, rubbing one's own fingers, touching one's hair, etc. However, even without any semantic contribution to the speech, such gestures have an important role as they can show the increase in cognitive load, as speakers often use such gestures to deal with a stressful situation by regaining cognitive control over
it (Freedman, 1972). For instance, the speaker in Figure 2 is rubbing her hands during the interpreting. Such small gestures are quite
typical for this type of activity, as they are not very visible, but might help to cope with the stress.
Figure 2. Adapter gesture Рисунок 2. Жест самоадаптер
«десять тысяч птиц» "ten thousand (birds)"
Representational gestures are hand movements that have semantic meaning conveyed by their form and/or movement. Such gestures are based on the notion of iconicity as they resemble the object/notion they refer to. The way they resemble the referent is usually called a "mode of representation" (Müller, 1998; 2014). In our study we distinguish five modes: holding, molding, embodying, acting and tracing. In the example shown in Figure 3, the speaker uses a holding gesture when representing a number. The holding mode of representing things nonverbally is one of the most frequently used in speech as it can have a metaphorical meaning of a container (see McNeill's, 1992, discussion of metaphoric gestures and Lapaire, 2016), which helps to put one's idea into a form. This can be of aid to interpreters, as the representation of the
subject might lead to a better search for the equivalent lexical unit in the target language.
Pragmatic gestures are the type of co-speech hand movements that have discourse-related properties, like emphasizing verbal units, helping in searching for words (Streeck, 2009; Ladewig, 2014; Hirrel, 2018; Nikolaeva, 2017; Dressel, 2020) or structuring some parts of discourse (Fricke, 2013); they can also be used to express one's attitude or evaluation of the subject being talked about. As it is represented in Figure 4, the speaker is using her left hand, raised up, palm open, as she is translating the number one hundred. This gesture can have several discourse functions, as it might help search for the words to translate the construction, but also be as-if presenting part of the discourse, which is being interpreted; the form does not relate iconically to the referent mentioned simultaneously in the speech.
Figure 3. Example of a representational gesture Рисунок 3. Репрезентирующий жест
«восемь сотен» "eight hundred"
Figure 4. Example of a pragmatic gesture Рисунок 4. Прагматический жест
«сто (видов)» 'one hundred (species)"
Deictic gestures are different from the types described above in term of their ability to create vectors in space in order to show the location of a concrete or abstract notion. These can be such hand movements as pointing or touching gestures, as they are used to show the direction through the vector, created by hand with the speaker as the starting point or "origo" (Fricke, 2002). As it can be observed in Figure 5, the speaker is
touching the table in front of her when enumerating objects. These movements accompany the speech and as they help to locate the notions in space in front of the interpreter; they might bear the function of facilitating the interpreting process by making the objects as if tangible, thus creating an order in correspondence with growth of the amount of objects.
Figure 5. Examples of deictic gestures Рисунок 5. Дейктический жест
или триста...» or three hundred"
For the purposes of our study, we conducted a semantic analysis of the speech in order to compare the interpreting of the numbers from the source language (English) into the target language of the participants (Russian). Then the analysis of the nonverbal behavior was performed, as gestures were analyzed for their form and semantic properties. The final step was the statistical analysis of the correlation between the categories of gestures and numbers, except for the 'absent' category, as it was impossible to predict the exact possible occurrence of the untranslated number in speech, which made it impossible to line it up with any possible corresponding gesture. We examined the cooccurrence of gestures and the aforementioned types of number interpreting using the Pearson x2 criterion as it shows the linear dependence of the two categories.
3. Results
The total amount of the numbers appearing during the interpreting was 438 units. Of these, 277 units were interpreted correctly and 160 incorrectly: 98 units were absent, 43 were wrong and 19 were interpreted approximately. Mostly cardinal numbers were omitted, including simple numbers (like 2 and 5) and also more complex ones (50,000, millions and millions). The tendency to omit cardinal numbers might be explained by the absence of entire phrases
which included the numbers, as sometimes the interpreters omitted some parts of the source text. However, it cannot be determined whether such gaps were provoked by the presence of numbers in them or were difficult to interpret in general.
The category 'approximation', though not frequent in our results, displayed a particular tendency throughout the interpreting of different participants, as they interpreted the same two phrases using approximation, e.g., from half a species to four species; thousands and thousands (of pages). However, these phrases were also sometimes interpreted incorrectly (our category of 'wrong'). The results of the analysis of this category also showed the predominance of cardinal numbers, but it also showed quite a number of cases when the number of a year (as a time period) was misinterpreted, e.g. in 1884 - в 1848 (this same mistake was made by three of the interpreters)/ в 1544 году. However, such results might be influenced by the predominance of cardinal numbers in the source text in general, in comparison to ordinal or nominal numbers.
The analysis of gesture use revealed that, for the most part, numbers in interpreting were accompanied by gestures (169 cases), however the amount of cases when numbers were used without gestures is not significantly
different (117 cases). The most frequent gesture type was adapters (107 cases). Other types were used less frequently: pragmatic gestures (42 cases), representational (13 cases) and deictic (7 cases).
The analysis of the correspondence of gesture types to the interpreting categories showed that the 'correct' category was mostly accompanied by no gestures (117 cases) and by adapters (86 cases). It was also the only category accompanied by representational (13 cases) and deictic gestures (7 cases). Similar results were shown by the analysis of the other two categories. The 'approximation' category was not accompanied by gestures in 10 cases, and accompanied by adapters (7 cases) and pragmatic gestures (3 cases), whereas the 'wrong' category was mostly accompanied by gestures (in 27 out of the 43 instances): of these, 13 cases involved the use of pragmatic gestures and 14 cases - the use of adapters.
The last step was to establish any correlations existing between gestures and categories of interpreting. A statistical analysis was performed to establish any linear correlations (Pearson x2) between the presence/absence of gesture and interpreting categories 'correct', 'wrong' and 'approximation'. The results showed the correlation between the presence of a gesture and interpreting categories 'correct' (Pearson x2 = 0.7) and 'wrong' (Pearson x2 = 0.86) categories.
During the analysis it was revealed that similar representational gestures can be used with the same lexical units in different interpreting. In the examples below, we can observe that different participants used an embodying gesture (Figure 6) when talking about two positions. The idea behind the gesture is the division of the two notions, showing them metaphorically and through space highlighting the opposition between the positions and their difference.
Figure 6. Embodying gestures
Рисунок 6. Репрезентирующий жест (олицетворение)
«двумя позициями» «две позиции»
"two positions" "two positions"
Another case of embodying was found in the material which can be called a classical representation of a numeral. The interpreter in Figure 7 used her index finger extended on
two occasions when talking about one (thing). The depiction can relate to both category of one, highlighting it, as well as to its referent, embodying it.
Figure 7. Embodying gestures
Рисунок 7. Репрезентирующий жест (олицетворение)
«первая (тема)» "the first topic"
In another case (Figure 8), the participants used a molding gesture when interpreting the phrase 3D model. This gesture here reflects not the numeral, but rather the object it refers to, a model. The idea of a Figure 8. Molding gestures
Рисунок 8. Репрезентирующий жест (лепка / придание формы)
«один (вид)» "one species"
model as something that has physical borders which can be touched and even altered might be common background knowledge shared by people and reflected in the gesture use.
«3д модель» "3D model"
«3д модель» "3D model"
4. Discussion
The results of the study support the ideas of previous research on numbers in interpreting, which regard them as words difficult to interpret, as nearly half of the
numbers (160) were interpreted incorrectly. When the participants seemed to have difficulties in the interpreting, they chose to omit numbers (97 cases) or to use approximation (20 cases). This can indicate
that omitting is a more preferable tactic for dealing with the rise in cognitive load during the interpreting.
The results also showed a statistical correlation between one category of interpreting and gesture type. A high correlation between the 'wrong' category and the presence of gesture might indicate the importance of hand movements during the interpreting process, especially when a speaker faces a moment of difficulty. The correlation with the 'correct' category showed lower results, but they are still sufficient to say that there might be a dependence on gestures, in that gestures might support the idea of speech facilitation during the interpreting. The results, obtained with the 'approximation' category might show that either there is not enough data or that this category does not really depend on the presence or absence of a gesture in general and there can be additional factors which influence it.
As it was mentioned above, representational and deictic gestures co-occurred only with the 'correct' interpreting of numbers. These results might support the idea of the growth point, introduced by D. McNeill (1992). That is, McNeill argues that idea units unfold in relation to each other moment by moment as they are expressed in speech and gesture. In terms of the present findings, when the interpreters have the number and whatever is being quantified correctly in mind and are able to interpret that correctly, this provides a coherent 'image' that has the potential to also appear in gesture. While this was apparently not the situation in the majority of the cases (117 correct interpretations of numbers were not accompanied by any gestures), it is known from previous research (including McNeill, 1992) that gestures are used more often when presenting new information than when speaking about something deemed by the speaker to be already known or less noteworthy. Therefore, the representational gestures may have just come into play when
the interpreters were presenting the quantities or objects that were seen as key new information, e.g., один вид, две позиции (one species, two positions). In any case, the point remains that they would need to be able to mentally simulate the referent in some coherent way in order to have a gesturable image.
This contrasts with the cases when the interpretation of numbers was approximate or missing. When the interpreters have some difficulty with the number and/or what is being referred to, it does not provide a coherent growth point of an idea, and so is not capable of being rendered in a representational gesture or conceptualized as something that can be pointed to with a deictic gesture. In these cases, we see the pragmatic gestures coming into use. Rather than depicting or indicating a referent's location, pragmatic gestures often concern discursive functions, such as emphasizing a point (with a beat movement), highlighting some parts of discourse with a palm open hand gesture or structuring the speech by using the space around the speakers, and thereby simply presenting an idea to an addressee, regardless of the specifics of the idea being presented (Kendon, 2017). In this respect, the production of pragmatic gestures during approximate or incorrect interpreting of numbers could be a device whereby the interpreter facilitates their own speech production through 'phatic' behavior-that is, moving themselves to show they are expressing something, regardless of what actual words will come out. This could be a way in which they facilitate their own speech production (i.e., acting like they are about to provide a rendering of the idea from the source text audio).
Adapters, however, were found to be used both with correct, approximate, and incorrect interpretation of numbers. These movements appear to be all-purpose stress-relievers, helping the interpreters to manage the cognitive load on an ongoing basis as they do their work.
5. Conclusion
The research presented here shows advantages of taking a multimodal perspective in researching the process of number interpreting. This approach provides insights into the different processes involved that lead to the different outcomes (correct, approximate, or incorrect interpretation of the numbers) as well as insights into the kinds of conceptualization taking place, and the thinking for speaking involved during interpreting in these moments.
In addition, one could note that in the bulk of gesture research from at least the past 30 years, self-adapters have largely been excluded as an object of study. Many researchers have focused instead on the ostensibly communicative functions of representational, deictic and pragmatic gestures. However, research on high-stress contexts of communication, such as SI, shows frequent use of self-adapters. This suggests avenues for future research, ones which harken back to early work on body-oriented movements, such as that of Ekman and Friesen (1969) and Freedman (1972), cited earlier. While that research arose in part due to interest in what it could offer psychoanalysis, future research might focus on professional contexts of communication under stress, such as SI, in which the use of gesture, and of self-adapters in particular, may provide insights into processes of speech (and idea) production pertinent to those communicative contexts.
In general, the results show that hand gestures might help to deal with the cognitive load in SI and facilitate the process of word search, thus aiding the flow of speech and helping avoid interruptions and omissions. We finish the current study with a recommendation not to restrict oneself when performing interpreting but to use pragmatic and representational gestures when having difficulty in remembering a word and to use adapters to cope with stress and reduce it during this kind of work.
References
Braun, S. and Clarici, A. (1996). Inaccuracy for numerals in simultaneous interpretation: Neu-rolinguistic and neuropsychological perspectives, The Interpreters' Newsletter, 7, 85-102. (In English)
Bylinina, L. and Nouwen, R. (2020). Numeral semantics, Language and Linguistics Compass, 14, e12390. https://doi.org/10.1111/lnc3.12390 (In English)
Chu, M. and Kita, S. (2008). Spontaneous gestures during mental rotation tasks: Insights into the microdevelopment of the motor strategy, Journal of Experimental Psychology: General, 137, 706-723. https://doi.org/10.1037/a0013157 (In English)
Cienki, A. and Iriskhanova, O. K. (2020). Patterns of multimodal behavior under cognitive load: An analysis of simultaneous interpretation from L2 to L1, Voprosy Kognitivnoy Lingvistiki [Issues in Cognitive Linguistics], 1, 5-11. https://doi.org/10.20916/1812-3228-2020-1-5-11 (In English)
Dayter, D. (2020). Variation in non-fluencies in a corpus of simultaneous interpreting vs. non-interpreted English, Perspectives, 1-18. https://doi.org/10.1080/0907676X.2020.1718170 (In English)
Desmet, B., Vandierendonck, M. and De-francq, B. (2018). Simultaneous interpretation of numbers and the impact of technological support, in Fantinuoli, C. (ed.), Interpreting and technology, Language Science Press, Berlin, Germany, 13-27. https://doi.org/10.5281/zenodo.1493281 (In English)
Dressel, D. (2020). Multimodal word searches in collaborative storytelling: On the local mobilization and negotiation of coparticipation, Journal of Pragmatics, 170, 37-54. https://doi.org/10.1016/j.pragma.2020.08.010 (In English)
Ekman, P. and Friesen, W. V. (1969). The repertoire of non-verbal behavior: Categories, origins, usage, and coding, Semiotica, 1, 49-98. https://doi.org/10.1515/semi.1969.L1.49 (In English)
Freedman, N. (1972). The analysis of movement behavior during the clinical interview, in Siegman, A. W. and Pope, B. (eds.), Studies in dyadic communication, 7, 153-175. (In English)
Fricke, E. (2002). Origo, pointing, and speech: the impact of co-speech gestures on linguistic deixis theory, Gesture, 2 (2), 207-226. https://doi.org/10.1075/gest.2.2.05fri (In English)
Fricke, E. (2013). Towards a unified grammar of gesture and speech: A multimodal approach, in Müller, C., Cienki, A., Fricke, E., Ladewig, S., McNeill, D. and Tessendorf, S. (eds.), Body - Language - Communication, Volume 1, De Gruyter Mouton, Berlin, Germany 733-754.
https://doi.org/10.1515/9783110261318.733 (In
English)
Gibbon, D. (2005). Prerequisites for a multimodal semantics of gesture and prosody, Proceedings of the International Workshop on Computational Semantics, 6, 2-15. (In English)
Gile, D. (1995). Fidelity assessment in consecutive interpretation: an experiment, Target. International Journal of Translation Studies, 7 (1), 151-164.
https://doi.org/10.1075/target7.L12gil (In English)
Gile, D. (2008). Local cognitive load in simultaneous interpreting and its implications for empirical research, FORUM. Revue internationale d'interprétation et de traduction/International Journal of Interpretation and Translation, 6 (2), 59-77. https://doi.org/10.1075/forum.6.2.04gil (In English)
Gile, D. (2009). Basic concepts and models for interpreter and translator training, John Benjamins, Amsterdam/Philadelphia, Netherlands/USA. https://doi.org/10.1075/btl.8 (In English)
Gosy, M. (2007). Disfluencies and Self-monitoring, Govor = Speech, XXIV, 91-110. (In
English)
Hirrel, L. (2018). Cyclic gestures and multimodal symbolic assemblies: An argument for symbolic complexity in gesture, Doctoral dissertation, The University of New Mexico, Albuquerque, New Mexico, USA. (In English)
Kendon, A. (2017). Pragmatic functions of gestures: Some observations on the history of their study and their nature, Gesture, 16 (2), 157175. https://doi.org/10.1075/gest.16.2.01ken (In English)
Kita, S., Alibali, M. W. and Chu, M. (2017). How do gestures influence thinking and
speaking? The gesture-for-conceptualization hypothesis, Psychological Review, 14 (3), 245-266. http://dx.doi.org/10.1037/rev0000059 (In English) Korpal, P. (2017). Linguistic and psychological indicators of stress in simultaneous interpreting, Wydawnictwo Naukowe UAM, Poznan, Poland. (In English)
Korpal, P. and Stachowiak-Szymczak, K. (2020). Combined problem triggers in simultaneous interpreting: Exploring the effect of delivery rate on processing and rendering numbers, Perspectives, 28 (1), 126-143. https://doi.org/10.1080/0907676X.2019.1628285 (In English)
Ladewig, S. H. (2014). The cyclic gesture, Body-language-communication: An international handbook on multimodality in human interaction, 38, 1605-1618. (In English)
Lakoff, G. and Nunez, R. (2000). Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being, New York, NY, USA. (In English)
Lapaire, J.-R. (2016). From ontological metaphor to semiotic make-believe: Giving shape and substance to fictive objects of conception with the "globe gesture", Signo, 41 (70), 29-44. https://doi.org/10.17058/signo.v41i70.6413 (In English)
Lin, W., Orton, I., Liu, M. and Mahmoud, M. (2020). Automatic detection of self-adaptors for psychological distress, 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), 371378. https://doi.org/10.1109/FG47880.2020.00032 (In English)
Mandler, G. and Shebo, B. J. (1982). Subi-tizing: an analysis of its component processes, Journal of experimental psychology: General, 111 (1), 1-22. https://doi.org/10.1037/0096-3445.111.1.1 (In English)
Mazza, C. (2001). Numbers in simultaneous interpretation, The Interpreters' Newsletter, 11, 87-104. (In English)
McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92 (3), 350371. https://doi.org/10.1037/0033-295X.92.3.350 (In English)
McNeill, D. (1992). Hand and mind: What gestures reveal about thought, University of Chicago Press, Chicago, USA. (In English)
Mead, P. (2015). Numbers, in Pöchhacker, F. (ed.), Routledge encyclopedia of interpreting studies, London/New York, UK/USA, 286-288.
https://doi.org/10.4324/9781315678467 (In English)
Müller, C. (1998). Iconicity and gesture, in Santi, S. (ed.), Oralite et gestualite. Communication multimodale, interaction, L'Harmattan, Paris, France, 321-328. (In English)
Müller, C. (2014). Gestural modes of representation as techniques of depiction, in Müller, C., Cienki, A., Fricke, E., Ladewig, S. H., McNeill, D., and Bressem, J. (eds.), Body - Language - Communication, De Gruyter Mouton, Berlin, Germany, 1687-1702. (In English)
Neff, M., Toothman, N., Bowmani, R., Fox Tree, J. E. and Walker, M. A. (2011). Don't scratch! Self-adaptors reflect emotional stability, in Vilhjalmsson, H. H., Kopp, S., Marsella, S. and Thorisson, K. R. (eds.), Intelligent Virtual Agents, IVA 2011, Lecture Notes in Computer Science, 6895, 398-411. Reykjavik, Iceland, 398-411. https://doi.org/10.1007/978-3-642-23974-8 43 (In English)
Nikolaeva, J. V. (2017). Pragmatic gestures in Russian retellings of "The Pear Stories", The Russian Journal of Cognitive Science, 4 (2-3), 612. (In English)
Pinochi, D. (2009). Simultaneous interpretation of numbers: Comparing German and English to Italian. An experimental study, The Interpreters' Newsletter, 14, 33-57. (In English)
Poyatos, F. (1987/2002). Nonverbal communication in simultaneous and consecutive interpretation: A theoretical model and new perspectives, in Pöchhacker, F. and Shlesinger, M. (eds.), The Interpreting Studies Reader, Routledge, London, UK, 235-246. (In English)
Seeber, K. (2011). Cognitive load in simultaneous interpreting: Existing theories - New models, Interpreting, 13, 176-204. https://doi.org/10.1075/intp.13.2.02see (In English)
Seeber, K. G. (2012). Multimodal input in simultaneous interpreting: An eye-tracking experiment, in Zybatow, L. N., Petrova, A. and Usta-
szewski, M. (eds.), Proceeding of the 1st International Conference TRANSLATA, Translation and Interpreting Research: Yesterday-Today-
Tomorrow, Peter Lang, Frankfurt a.M., Germany, 341-347. https://doi.org/10.3726/978-3-653-01625-3 (In English)
Seeber, K. G. (2015). Simultaneous interpreting, in Mikkelson, H. and Jourdenais, R. (eds.), The Routledge Handbook of Interpreting, Routledge Handbooks Online.
DOI: 10.4324/9781315745381.ch5 (In English)
Slobin, D. I. (1987). Thinking for speaking, Proceedings of the Thirteenth Annual Meeting of the Berkeley Linguistics Society, 13, 435-445. https://doi.org/10.3765/bls.v13i0.1826 (In
English)
Stachowiak-Szymczak, K. (2019). Eye Movements and Gestures in Simultaneous and Consecutive Interpreting, Springer, New York, USA. https://doi.org/10.1007/978-3-030-19443-7 (In English)
Streeck, J. (2009). Forward-gesturing, Discourse Processes, 46 (2-3), 161-179. https://doi.org/10.1080/01638530902728793 (In English)
Timarova, S. (2012). Working memory in simultaneous interpreting, PhD Thesis, University of Leuven, Leuven, Belgium.
https://doi.org/10.1075/intp.16.2.01tim (In English)
Все авторы прочитали и одобрили окончательный вариант рукописи.
All authors have read and approved the final manuscript.
Конфликты интересов: у авторов нет конфликтов интересов для декларации.
Conflicts of interests: the authors have no conflicts of interest to declare.
Alan Cienki, Ph.D., Professor of Language Use & Cognition and English Linguistics, Vrije Universiteit Amsterdam, Amsterdam, Netherlands. Алан Ченки, Ph.D., профессор Свободного университета Амстердама, Амстердам, Нидерланды.
Anna V. Leonteva, Researcher at the Center for Socio-Cognitive Discourse Studies (SCoDis), Moscow State Linguistic University, Junior Researcher at the Laboratory of Multi-Channel Communication, Institute of Linguistics of Russian Academy of Sciences, Moscow, Russia.
Анна Васильевна Леонтьева, научный сотрудник Центра Социокогнитивных исследований дискурса (СКоДис) Московского государственного лингвистического
университета, младший научный сотрудник Лаборатории мультиканальной коммуникации Института языкознания РАН, Москва, Россия.
Olga V. Agafonova, Junior Researcher at the Center for Socio-Cognitive Discourse Studies (SCoDis), Moscow State Linguistic University, Moscow, Russia.
Ольга Владимировна Агафонова, младший научный сотрудник Центра Социокогнитивных
исследований дискурса (СКоДис) Московского государственного лингвистического
университета, Москва, Россия.
Andrey A. Petrov, Researcher at the Center for Socio-cognitive Discourse Studies (SCoDis), Moscow State Linguistic University, Moscow, Russia.
Андрей Андреевич Петров, научный сотрудник Центра социокогнитивных исследований дискурса (СКоДис) Московского государственного лингвистического
университета, Москва, Россия.
