CC BY
15
УДК 811.11 А. Ченки
Ph.D., профессор лингвистики английского языка Свободного университета Амстердама, Нидерланды; директор когнитивной лаборатории полимодальной коммуникации (ПолиМод) при МГЛУ; e-mail: a.cienki@vu.nl
ПОВТОРЫ В АСПЕКТЕ ВАРИАТИВНОСТИ ПОЛИМОДАЛЬНОГО ОБЩЕНИЯ
В данном исследовании рассматривается понятие динамического диапазона релевантных действий (ДРД), которое позволяет принимать во внимание варьирование семиотического статуса жестов, используемых коммуникантами. На примере вербальных и / или невербальных повторов демонстрируются процессы расширения и / или сокращения ДРД говорящего. Действия, которые повторяются в ДРД и выполняют определенную функцию, должны c большей вероятностью приобретать символический (знаковый) статус. Исследование показывает, что знаки могут иметь относительно устойчивый центр и вариативные границы.
Ключевые слова: когнитивная грамматика; устная речь; жест; диапазон релевантных действий; повторы.
Cienki A.
Ph.D., Professor of Language Use and Cognition, VU University Amsterdam, Professor and Director of Multimodal Communication and Cognition Lab (PoliMod), MSLU; e-mail: a.cienki@vu.nl
REPETITIONS IN VIEW OF TALK AS VARIABLY MULTIMODAL1
The notion of a dynamic scope of relevant behaviors (SRB) is introduced as a way of taking into account the varying kinds of sign-status that speakers' gestures can have. This is demonstrated on the basis of a set of examples of verbal and/ or gestural repetitions that illustrate processes of expansion and/or contraction of the speaker's SRB. Behaviors that repeatedly occur in the SRB paired with certain functions should be more likely to gain symbolic status. But the present study suggests that signs may have relatively stable centers and variability in their boundaries.
Key words: Cognitive Grammar; spoken language; gesture; scope of relevant behaviors; repetitions.
1 This research has been supported by Russian Science Foundation grant #14-48-00067.
1. Background: Cognitive Grammar
The discipline of linguistics has long been biased by an implicit view of language as it is written [18]. It is an ironic situation, given De Saussure's [6] starting point of theorizing about language based on face to face talk. Witness his famous diagram of two heads facing each other and of speech as signals going from the brain through the mouth of one to the ear of the other [6, p. 11]. However, within the field of gesture studies, the many ways in which speech and gesture interrelate in the production of utterances has been the object of research for decades (at least since [11]). Nevertheless, it has only been in recent years that more linguists have begun to think seriously about the implications of this research for their theories of grammar [3; 4; 6; 10; 27].
In cognitive linguistics, the framework of Cognitive Grammar (CG) is one that potentially provides a means of moving beyond the historic limitations on what can fall under the purview of a theory of grammar. This is a consequence of the usage-based [13] nature of the theory. A fundamental tenet in CG is that lexicon and grammar form a continuum consisting in assemblies of symbolic structures [14, p. 15]. Langacker exemplifies the continuum by showing how each of the following English words and compounds can be considered a symbolic structure, with each progressive one involving more internal complexity: sharp, sharpen, sharpener, pencil sharpener [15, p. 16-18].Those symbolic structures are them selves constituted by linked semantic and phonological structures -the two poles linked together through conventionalized association of meaning / function and form of expression.
To be consistent with the theoretical framework, analyses in CG should work from the ground up to see what form-meaning (phonological-semantic) associations are abstracted by language users, that is: which ones become schematized and entrenched from communicative usage events. Schematization here concerns «the process of extracting the commonality inherent in multiple experiences to arrive at a conception representing a higher level of abstraction»[14, p. 17] and entrenchment has to do with establishing the fixedness of that symbolic relation through the reinforcement of commonalities, and the ignoring of fine-grained differences, across usage events [14, p. 458]. It is important to bear in mind that in the theory of CG, schematization and entrenchment are matters of degree rather than absolutes (of a unit being schematized or entrenched
versus not). The phonological pole of a linguistic sign could conceivably include signals that are not just the audible sounds of spoken language; Langacker [14, p. 457] adds that «any other signals, such as gestures and body language» could also be involved in the phonological pole to the degree that they also become schematized and entrenched in sufficiently consistent association with concepts (the semantic pole of a sign). The theory thus allows for linguistic signs to be multimodal (audio-visual in the case of speech and gesture) to varying degrees, based on the extent of schematization and entrenchment.
However, this elegant theoretical characterization, that appears to quite validly capture what language users do in practice, is a difficult one for linguists to do justice to in their analyses. In traditional studies within cognitive linguistics, language is treated as if it were a discrete category, separate from gesture, and the assumptions behind this are not problematized - this despite the fact that in CG, linguistic categories on various levels (phonemes, constructions, word meanings, etc.) are considered to involve continua and prototype categories rather than categories with strict boundaries.
2. The dynamic scope of relevant behaviors
Taking the notion of communicative usage events seriously as a way to approach the study of what linguistic signs are and how they function in actual practice, I propose we need to adjust how we theorize about the nature of the linguistic sign itself. Going back to the roots of CG theory, we find the importance that attention plays in accounting for the nature of grammar. Attention is what guides processes that have central explanatory value in CG, such as selection of which facets of scenes will be focused on for linguistic characterization, focal adjustments made within selection, the perspective taken, determing figure and ground relations, etc. [16, ch. 3]. If we turn to research on attention as it applies to communication, we find notions such as the following. Oakley [23] describes the Distributed Adjustable Capacity of attention that we possess, which widens or narrows like a zoom lens depending on different factors. Some are bottom-up factors, dependent on the perceptual density of the environment; e.g., denser perceptual input (such as a more complex visual image) may result in a rise in the energy level for processing visual information and a more intense focus on part of the visual field. Other factors are top-down, such as
priorities of the individual, or social and cultural demands, which may bring the perceiver to focus more or less attention on certain sensory input (see, for example, [8; 17]). Müller [22] discusses the devices that speakers have at their disposal to foreground their use of particular verbal expressions. For example, the use of metaphors can be highlighted by repeating the same imagery in other words (e.g., «I get what you're saying» and «I grasp what you mean»); by co-expressing the same metaphoric idea in words and in gesture (closing one's hand into a fist when using one of the expressions above); increasing the size or duration of a metaphorically used gesture, which increases its visibility; directing one's gaze towards one's gesture, which not only involves the speaker's heightened awareness of his / her communicative behavior, but also increases the saliency of the gesture for those paying attention to the speaker / gesturer [24, ch. 5].
The points above, combined with the view in cognitive linguistics that behaviors provide cues for constructing meaning [9; 16, p. 154-163] leads me to propose the notion of a dynamic scope of relevant behaviors (SRB) as a way to take into account the varying kinds of sign-status that gestures can have. Elsewhere [5] I consider how the SRB can also help handle the variable relation of other behaviors to talk, such as the use of non-lexical utterances (such as uh and mm in English) and the use of hummed intonation contours without speech. We see the SRB playing out in various ways in language if we take off the blinders that have been imposed upon us by traditional views of language and grammar, taking a step back to look at usage events of spoken language, rather than considering language as just symbols written on a page or a computer screen.
The proposal is that in a communicative context, there is a dynamic scope of relevant behaviors, the scope of which may differ for the producer of the behaviors and for anyone paying attention to him or her (the attender[s]) at any given moment. The scope has a focus and a periphery. In face-to-face communicative usage events between hearing people, spoken language is the default focus of the scope of relevant behaviors. A producer can flexibly make use of a smaller or larger scope of expressive behaviors, and an atttender's focus can also be narrower or broader and can change in size over the course of a conversation. The SRB is thus dynamic in terms of zooming in (taking just one behavior into account) or zooming out (including more than one behavior at a time) and in terms of its shifting focus, determing which behavior(s) is / are in focus. We
can claim thatbehaviors that repeatedly occur in usage events paired with certain meanings/functions become more entrenched (linguistic) signs.
The present study begins by considering the different degrees to which speakers' manual gestures are conventionally communicative. Kendon [12] and McNeill [19] discuss a continuum of gesture types, from more to less conventionally communicative signs. On one end are «emblem» gestures (e.g., think of a «thumbs up» gesture to show a positive evaluation of something). These are the most word-like in the stability of their sign status. However, on the other end is «spontaneous gesticulation»: the manual movements that may relate to the contents of the speech in idiosyncratic ways, such as depicting selected aspects of the forms of entities one is talking about. Such depiction can take place in more detailed or more schematic ways, and can vary greatly in form across speakers and across usage events. Is there any way in which such gestures can sensibly be accounted for in a theory such as CG?
3. Empirical demonstration
The phenomenon of self-repetition (of the verbal or gestural part of an utterance) provides a useful context for exploring how the properties of the construct proposed above operate in practice. Repetitions have been an important topic in Conversation Analysis (witness classic works on the topic such as [25]), but have been more peripheral in linguistic theorizing. A few exceptions that prove the rule come, perhaps not surprisingly, from linguistic research based on actual usage of spoken language through analysis of video data, including [10] and [1].
Self-repetitions within topic units from a set of five interviews in English from an American television talk show («The Ellen DeGeneres Show») were analyzed. Verbal repetitions were counted when they consisted of more than one word repeated, and gestural repetitions consisted of repeated use of more than two form features of the set of four that have become customary in gesture analysis since McNeill [19], namely: hand shape, palm orientation, location, and movement type. The number of times that the repetition took place was also counted (two, three, or four productions).
The analysis revealed a number of patterns according to which the SRB may be expanded and / or contracted in the process of talk on the time scale of seconds or minutes. The types were interpreted as follows:
• Repetition involving contraction of the SRB: An idea goes from being presented with multiple articulators (spoken words produced orally and gestures produced manually) and / or more elaborate use of one or more form features to being presented with one mode of expression. For example: the speaker saying «I [do] love it. I do love it», while producing a gesture with «do» the first time (both hands turning from palm-down to palm-up and then back down again). The repetition thus involves reduction of behaviors on the part of the producer.
• Expansion and then contraction of the SRB: An idea first presented verbally is reintroduced using multiple articulators (speech and gesture), with the speaker settling on one form of expression. In one example, the speaker says, «We haven't decided yet. [We...] haven't decided. We haven't decided yet», and while uttering [We.] in the second utterance makes a circular gesture in the horizontal plane with her left hand loosely open, palm down, such that the circle moves out toward the interviewer and back toward the speaker. This pattern can be interpreted as a momentary elaboration into a multimodal sign ensemble is followed by stabilization on the use of a monomodal sign.
• Maintenance of an expanded SRB: An ideafirst expressed with multiple articulators is repeatedand part of it is symbolized in gesture for a small stretch of discourse. An example occurs when one speaker tells a rathere laborate story about something that happened when he was being introduced to present an award at a nationally televised awards show. He says, «Ifyou do-funny enough, I don't know if you [still have that whole clip of you-of] you two introducing me, but, [... uh, if you'll notice, there was] someone, running, behind you». The interviewer says, «No,» and the speaker continues, «[An' an'--] Well, obviously, you didn't see it, wh-heh, ... uhm [.., but-- but uh, one of the stage hands] comes running up and says, < they want me to switch the batteries in you rmicrophone!>» During each of the utterances in square brackets above he makes a two-handed gesture, hands abouts houlder-width, palms facing a way from his body, witht humband index finger extended at 90 degreesto make two Ls
facing each other forward and backwards, like this: |__|. The form of the
gesturecanbeinterpreted as partialrepresentation of what he says the first time, referringto the «the clip»from the TV show: the hands iconically outline the bottom corners of a object that appears to be rectangular - the prototypicals hape of a small segment of film, particularlyold-fashioned movie film consisting of a row of square frames. But the word
«clip» is notrepeated, although the gesture is produced three more times (indicatedby the square brackets around the accompanying speech), reintroducing the image of «clip» with accompanying speech aboutwhat was shown in the clip. It is anexample of what McNeill [19] calls a gestural catchment: «A catchment is recognized from recurrences of gesture form features over a stretch of discourse. Catchments are recognized from two or more gestures (not necessarily consecutive) with partially or fully recurring features of shape, movement, space, orientation, dynamics, etc.» [20, p. 11]. Information that was originally temporally integrated in what Enfield [7] calls a «composite utterance» becomes decompressed, as itwere, as relevant behaviors (speech and gesture) continue to serve some what different expressive functions over perhaps a few minutes of talk.
4. Discussion and conclusions
The SRB that we make use of varies, and this variation occurs not only in different ways but also along different time scales. Consequently, symbolization processes can also play out along different time scales, e. g., within topic units of less than a minute (microsymbolization via gestural catchments), across topics within a usage event (in the use of ad hoc words or gestures spontaneously ascribed a symbolic function within a conversation) and across genre events (in the gradual codification of functions / meanings with certain lexico-grammatical and / or gestural forms within communities of practice). The phenomena discussed here are in line with the research in cognitive linguistics on how linguistics categories of various types function with a center-periphery structure, as prototypes [26] whereby any category has a center that has a kind of solidity in its consistency across instantiations but a flexible boundary. As Bybee [2, p. 2] notes, «All types of units proposedbylinguists show gradience, in the sense that there is a lot of variation within the domain of the unit (different types of words, morphemes, syllables) and difficulty in setting the boundaries of the unit.» As suggested in [3], on a broaderscale, language and gesture are them selves prototype categories, and they overlap tovarying degrees and in dynamic ways in communicative usage events. Future research will consider connections to dynamic(al) systems theory and theory about complex adaptive systems, in that spoken language and gesture may them selves been seen as variably overlapping complex adaptive systems (compare [28] on the proposed relation of sign languages to gesture).
In conclusion, this research helps us reflect theoretically about how linguistic signs function in practice. Behaviors that repeatedly occur in the SRB paired with certain functions should be more likely to become more entrenched as symbolic structures or signs. But the present study suggests that signs need not be considered static entities with rigid boundaries, but rather may have relatively stable centers and variability in their boundaries. This also allows us to explain the overlap between the semiotic systems of spoken language and gesture (and other concurrent behaviors) as communicative signs in face-to-face interaction - an overlap which varies in degree over the time course of any usage event of talk.
Acknowledgements
Thanks to Suwei Wu for collection and transcription of the video data.
REFERENCES
1. Bressem J. Repetitions in gesture // Body-Language-Communication: An International Handbook on Multimodality in Human Interaction, Vol. 2 / C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & J. Bressem (eds.). - Berlin : De Gruyter Mouton, 2014. - P. 1641-1649.
2. Bybee J. L. Language, Usage and Cognition. - Cambridge : Cambridge University Press, 2010. - 262 p.
3. Cienki A. Cognitive Linguistics: Spoken language and gesture as expressions of conceptualization // Body-Language-Communication: An International Handbook on Multimodality in Human Interaction // C. Müller, A. Cienki, S. H. Ladewig, D. McNeill & S. Teßendorf (eds.). - Vol. 1. - Berlin : De Gruyter Mouton. 2013a. - P. 182-201.
4. Cienki A. Gesture, space, grammar, and cognition // Space in Language and Linguistics: Geographical, Interactional, and Cognitive Perspectives / P. Auer, M. Hilpert, A. Stukenbrock& B. Szmrecsanyi (eds.). - Berlin : Walter de Gruyter, 2013b. - P. 667-686.
5. Cienki A. Usage events of spoken language and the symbolic units we (may) abstract from them // Cognitive Processes in Language / J. Badio, K. Kosecki (eds.) - Bern : Peter Lang, 2012. - P. 149-158.
6. de Saussure F. Course in General Linguistics. - N. Y.: Duckworth, 2013 [1916]. - 236 p.
7. Enfield N. J. The Anatomy of Meaning: Speech, Gesture, and Composite Utterances. - Cambridge : Cambridge University Press, 2009. - 264 p.
8. Eriksen C. W., James J. D. St. Visual attention within and around the field of focal attention // Perception and Psychophysics. - 40. - 1986. - P. 225-240.
9. Fillmore Ch. J. Frames and the semantics of understanding // Quaderni di Semantica. - 6(2). - 1985. - P. 222-254.
10. FrickeE. Grammatik multimodal: Wie Wörter und Gesten zusammenwirken. -Berlin : Walter de Gruyter, 2012. - 296 p.
11. Kendon A. Gesticulation and speech: Two aspects of the process of utterance // The Relation between Verbal and Nonverbal Communication / M. R. Key (ed.). -The Hague : Mouton, 1980. - P. 207-227.
12. Kendon A. How gestures can become like words // Cross-cultural Perspectives in Nonverbal Communication / F. Poyatos (ed.). - Lewiston-N Y : C. J. Hogrefe, 1988. - P. 131-141.
13. Langacker R. W. A usage-based model // Topics in Cognitive Linguistics / B. Rudzka-Ostyn (ed.). - Amsterdam : John Benjamins, 1988. - P. 127-161.
14. LangackerR. W. Cognitive Grammar: A Basic Introduction. - Oxford : Oxford University Press, 2008. - 580 p.
15. Langacker R. W. Concept, Image, and Symbol: The Cognitive Basis of Grammar. - Berlin : Mouton de Gruyter, 1994. - 408 p.
16. Langacker R. W. Foundations of Cognitive Grammar, Volume I: Theoretical Prerequisites. - Stanford : Stanford University Press, 1987. - 539 p.
17. Lavie N., Hirst A., Fockert J., Vidin E. Load theory of selective attention and cognitive control // Journal of Experimental Psychology. - 133(3). - 2004. -P. 339-354.
18. Linell P. The Written Language Bias in Linguistics: Its Nature, Origins, and Transformations. - L. : Routledge, 2005. - 256 p.
19. McNeillD. Hand and Mind: What Gestures Reveal about Thought. - Chicago : University of Chicago Press, 1992. - 423 p.
20. McNeill D., Quek F., McCullough K.-E., Duncan S., Furuyama N., Bryll R., Ma X.-F., Ansari R. Catchments, prosody and discourse // Gesture. - 1(1). -2001. - P. 9-33.
21. Müller C. Gestures as a medium of expression: The linguistic potential of gestures // Body - Language - Communication: An International Handbook on Multimodality in Human Interaction / C. Müller, A. Cienki, E. Fricke, S. Ladewig, D. McNeill & S. Tessendorf (Eds.). - Vol. 1. - Berlin : De Gruyter Mouton, 2013. - P. 202-217.
22. Müller C. What gestures reveal about the nature of metaphor // Metaphor and Gesture / A. Cienki & C. Müller (eds). - Amsterdam : John Benjamins, 2008. - P. 219-245.
23. Oakley T. From Attention to Meaning. - Bern : Peter Lang., 2009. - 268 p.
24. Streeck J. Gesturecraft: The Manu-facture of Meaning. - Amsterdam : John Benjamins, 2009. - 235 p.
25. TannenD. Talking Voices: Repetition, Dialogue, and Imagery in Conversational Discourse. - Cambridge : Cambridge University Press, 1989. - 240 p.
26. Taylor J. R. Linguistic Categorization: Prototypes in Linguistics Theory. -Oxford : Clarendon Press, 1995. - 331 p.
27. Turner M., Steen F. Multimodal Construction Grammar. Social Science Research Network, 2012. - URL: http://ssrn.com/abstract=2168035).
28. Wilcox S., André Nogueira X. A framework for unifying spoken language, signed language, and gesture // Todas as Letras-Revista de Lingua e Literatura, 15 (1), 2013. - P. 88-110.
