CC BY
35
GRAMMATICAL ASPECT AND GESTURE IN FRENCH: A KINESIOLOGICAL APPROACH
Dominique Boutet1, Aliyah Morgenstern2, Alan Cienki3
'Université de Rouen 140, rue de Belleville 75020 Paris, France
^Université Sorbonne Nouvelle
12, rue St Fiacre, 75002 Paris, France
3Universiteit Amsterdam & Moscow State Linguistic University
VU, De Boelelaan 1105 1081 HV Amsterdam, Netherlands
In this paper, we defend the idea that research on Gesture with Speech can provide ways of studying speakers' conceptualization of grammatical notions as they are speaking. Expressing an idea involves a dynamic interplay between our construal, shaped by the sensori-motoric and interactive experiences linked to that idea, the plurisemiotic means at our disposal for expressing it, and the linguistic category available for its expression in our language. By analyzing the expression of aspect in Speech with Gesture (GeSp) in semi-guided oral interactions, we would like to make a new contribution to the field of aspect by exploring how speakers' construal of aspectual differences grammaticalized in their language, may be enacted and visible in gesture.
More specifically we want to see the degree to which event structure differences expressed in different grammatical aspects (perfective and imperfective) correlate with kinesiological features of the gestures. To this end, we will focus on the speed and flow of the movements as well as on the segments involved (fingers, hand, forearm, arm, shoulder). A kinesiological approach to gestures enables us to analyze the movements of human bodies according to a biomechanical point of view that includes physiological features. This study is the first contribution focused on the links between speech and gesture in French in the domain of grammatical aspect.
Grammatical aspect was defined by Comrie (1976) [1989] as involving the internal unfurling of the process, «[...] tense is a deictic category, i.e. locates situations in time, usually with reference to the present moment [...]. Aspect is not concerned with relating time of the situation to any other time-point, but rather with the internal temporal constituency of the one situation; one could state the difference as one between situation-internal time (aspect) and situation-external time (tense) » (Comrie, 1976 [1989]: 5).
Can kinesic features express and make those semantic differences tangible? When a speaker produces a gesture with specific physiological properties, for example a certain speed, is there an underlying source that motivates that speed? Could a speaker's con-strual of the event, seen in her choice of grammatical aspect, lead to her performing a gesture with specific properties? This study is an attempt to tackle these questions and present new methods to analyze gestures.
1. INTRODUCTION
1.1. Aspect in speech: focus on French
The great Alexandrian grammarian, Dionysios Thrax (170—90 B.C., see Lallot & Thrace, 1998) had already indicated in what remains of his written work that Greek verbal forms express both time and aspect. Indeed, if grammatical tenses merely expressed chronological time, there would only be one tense per time meaning. This paradox in the Greek verbal system is also illustrated in past tenses in French. The existence of several forms can be explained by aspectual differences added to reference to chronological time.
Aspect became extensively described thanks to the analysis of Slavic languages. The study of Slavic verb aspect influenced Indo-European linguistics and was then transmitted to the French linguistic tradition mostly in the 19th century according to Wilmet (2003). For de Boer (1947), as for a number of subsequent linguists working on French, the only aspectual difference that he finds preserved in French is the opposition between the three past tenses, passé composé, imparfait, and passé simple. Despite the fact that Damourette & Pichon (1929) explain that aspect is not present in a systematic way in the French system as a whole, according to Gosselin (2005), French is a language in which lexical and grammatical aspect can be clearly distinguished and for which indeed grammatical aspect is expressed in addition to reference to time in the past tenses.
What Gosselin calls a clear distinction was not very precisely described at first by linguists working on French. Verbal aspect is mentioned throughout Ferdinand Bru-not's book (1922) that is fundamental for many specialists of French, but the distinction between grammatical and lexical aspect is rather blurred. Le bon usage by Maurice Grevisse (1953) however, offers quite a good general definition of aspect: 'L'aspect du verbe est le caractère de l'action considérée dans son développement, l'angle particulier sous lequel l'accomplissement (le "processus") de cette action est envisagé' (the aspect of a verb is the character of the action considered in its progress; the particular angle from which the accomplishment (the "process") of the action is viewed). A more theoretical analysis of aspectual differences for French past tenses was introduced extensively and in detail by Guillaume (1929). In his terminology, simple verbs are in the TENSIVE aspect, compound verbs in the EXTENSIVE aspect. Syntax is thus to be taken into account when describing grammatical aspect. Guillaume skillfuly characterizes passé simple (now used in written French only), passé composé and imparfait. According to him, the three types of aspects involved are:
— The "global" aspect, which is described as being an aoristic, perfective or inceptive aspect. It is expressed by the "passé simple" and gives an account of the event in its entirety: il entra dans la maison (he entered the house).
— The aspect "accompli", which expresses a resulting state: il a terminé son travail (he has finished his work).
— The imperfective (aspect "inaccompli") which views the event from an internal perspective: the temporal boundaries of the event are not taken into account and only part of the process is presented: il mangeait (he was eating).
The concept of open and closed boundaries, which is considered by Desclés & Gu-enchéva (Desclés & Guenchéva, 1996: 27) as fundamental for the study of aspect is taken up, reintroduced and reinforced after Guillaume by Culioli around 1970 in his seminars (see Culioli, 1999).
In contemporary spontaneous oral French, the passé simple is not used anymore. It is described as having been replaced by the passé composé and in a lot of cases of vivid descriptions, by the narrative présent as we will see in our data. It is difficult to formally distinguish what Guillaume called "aspect global" and "aspect accompli" as they are very often both expressed by the passé composé. In this paper based on French data of oral interactions, we will thus mostly focus on the distinction between passé composé, which we will associate to the perfective aspect, and the imparfait, associated to the imperfective aspect.
1.2. The gestural expression of aspect
The notion of boundary, which comes from linguistic analyses of events in several languages, is also present in a variety of studies on aspect and gesture. Those studies indicate that verbal forms with perfective aspect are co-produced with gestures characterized by sudden halts, and those with imperfective aspect by continuous movement.
Duncan (2003), in her paper on perfective and imperfective aspect in English and in Mandarin, expected that the gestures associated with the imperfective would express the temporal unfurling of the event and that the gestures associated with the perfective would be less imagistic, as perfective involves an external viewpoint on the event. Her study indicates no gestural difference between the two languages. The gestures associated to the different categories of verbal forms have the same features: for the imperfective verbal forms, their length is greater and the shape of the gestures is more complex. Perfective only enacts path (Talmy 1985), whereas gestures with imperfective forms indicate mostly manner or even sometimes figure and ground in Talmy's sense. If significant results are given as far as duration is concerned, the complexity of the gestures associated to the imperfective is not quantified in the study.
The author attributes features that are grounded in the very essence of the gestures produced with perfective and imperfective verbal forms. The aspect-gesture correspondence is transparent in her study: the gestures with imperfective are rendered by a continuous movement and project an image of the action, which explains the rich categories involved. The question concerning the justification of the structure of the event expressed in gestures is an important issue in the paper. The goal is to capture the relation between speech and gesture by studying the expression of aspect. Despite formal differences between Mandarin and English in the expression of aspect, no formal differences between the gestures in the two languages are found. However, common gestural distinctive features are described for perfective versus imperfective aspect as if a common element linked each grammatical category and its gestural enactment. The aspectual dichotomy is expressed by different spoken forms in each language but is based on the same type of features in gestures, but as we will see, also in sign languages. In a study on gesture and American Sign Language, Malaia and Wilbur (2012) give arguments in favor of si-
milarities in gesture and sign as far as the dichotomy between telicity (when the event expressed is presented as having an endpoint) and atelicity (the event or state is presented as having no endpoint) is concerned. They had already presented the kinematic dimension on which this aspectual dichotomy is based in a study published in 2008. In their research, five properties differentiate telic and atelic forms. 1) The length of telic signs is shorter than atelic signs. 2) The peak speed is more important for telic than for atelic signs. 3) The deceleration is more important for telic signs. 4) The peak speed is reached faster for atelic than for telic signs. 5) The slope between the peak speed and the local minimum speed is significantly stronger for telic signs. This means that deceleration is either stronger or reached more quickly: the stopping is more sudden for telic signs, which indicates a boundary.
If telicity and perfectivity are of course different notions involving lexical versus grammatical aspect depending on the languages under study, there are however common features that might help distinguish both types of aspectual categories according to the course of action of the GeSp or signs. At this level, the difference is not based on the shape of the gestures or the signs. It takes path in motion into account and views movement from within the motion itself. What is usually called path and manner in the literature is no longer relevant. The study questions a self-explaining view of the gestural expression of event structure. Wilbur (2003) contends that "Since crossing a distance (path) involves elapsing time, it is not surprising that path movement also provides the meaning 'time between events' in the habitual (sort) and iterative (long, elongated semi-circular) aspects" (Wilbur, 2003:10).
This purely actional view of gestuality is grounded in physics and geometry (Wilbur 2003), — we will call them kinematic features — as if speed, acceleration and deceleration provided a sufficiently tangible basis. Is it really possible to analyze gestures without taking our body and its physiological properties into account? This paper is an attempt to show that the bio-mechanical properties of gestures have an impact on their form and function. Our approach that takes form AND function into account is a kinesiological approach.
A recent study by Strickland et al. (2015) on telic and atelic signs1 and pseudo-signs extends Wilbur's observations to three sign languages and tests hearing non-signers' perception of telicity. We will not give the details of the complex protocol consisting of a series of experiments used in the study. However, the results show on the one hand that those three sign languages have similar encoding of telicity and atelicity and that on the other hand, non signers significantly recognize signs' telic and atelic properties even when pseudo-signs are used in the experiments. Thus, apparently, signers and non-signers encode telicity in signs and gestures in what the authors call a "universal" manner.
1 We use Wilbur's (2003) definition of telicity in this paper: "The notion of telicity used here is associated with the presence of a final end-State in the event structure, that is, with Transitions" ... "An atelic Process that occurs in the context of a final state or a bounding event is thus interpreted as part of a telic predicate" (p. 356).
1.3. Gestural rendering of (un)boundedness
1.3.1. Boundary schemas
The concept of boundaries (and of their absence) is constantly focused on throughout the studies on verbal aspect as well as on the production of signed or gestural aspect. We have thus taken up that concept in the present study.
The notion of boundaries has been applied to gesture schemas as early as 1998 (Müller). Coding "Boundary schemas" for aspectual patterns is a way of capturing how they express event structures. Müller made a bottom-up analysis of 647 gestures referring to events from 10 conversations between pairs of German speakers and 10 between pairs of Spanish speakers. Motion schemas found there were compared with the existing Aktionsart and aspect categories in the scholarly literature and the overlap between them resulted in a set of "boundary schemas". These have to do with whether events are being portrayed gesturally as involving boundedness on the action, and if so, how (initially, finally, repeatedly through iteration, totally via punctuality), or not.
In our common project, we have used the schemas presented in table 1.
Table 1
Schemas for the boundaries
Bounded Unbounded
Onset |.......... Unbounded
Offset ...........|
Double bounded |..........|
Multiple bounded -Ч-Ч---- Iterative л л л л
Punctual ......|.....
The dashes represent the unfurling of the event, the vertical bars represent boundaries. The circomflex accents indicate the repetition of the same sequence of events (based on Muller 1998).
1.3.2. Kinesiological features of (un)boundedness Along with Muller's (1998) gesture study using the notion of boundary for Aktionsaaart and Culioli's (1999) notion of "aoristic" in French with the boundary ("fermé" i.e. "closed"), we consider that perfective aspect corresponds to a bounded event and imperfective aspect to an unbounded event. By taking a kinesiological approach to gestures, we can uncover two key components of the movement of gestures used to express boundedness and unboundedness: the path of the gesture, including taking its shape into consideration, and the quality of its movement. Those two components are somewhat independent from each other and can therefore be associated.
Path in motion is rendered by the movement of at least one part of the upper limb in a bounded or unbounded manner. For example, a circle that is traced thanks to the gesture can be either viewed as bounded or unbounded. Boundedness can be expressed by discontinuity such as a sudden halt or a back and forth movement. In those two cases, the tracing of the straight line or the curve involves a sudden change in the unfurling of the movement, which corresponds to a variation in speed, either acceleration or deceleration. Unboundedness is rendered by continuous or homogeneous speed.
Via the tracing of a path and the consequent description of a shape, any gesture can be viewed as either bounded or unbounded. Possible combinations between a shape (such as a circle) and a path in motion indicate that (un)bounding effects can either be added to each other or can diverge such as when a non-bounded shape is produced with an acceleration of the movement which creates a boundary. Path is produced thanks to movement — acceleration/deceleration versus homogeneous speed are essential characteristics of the gestural rendering of (un)boundedness.
In addition to path and shape, quality of the movement enables us to view the movement from within its execution. It gives us an internal viewpoint contrary to path in motion and shape, which are considered with an external viewpoint. Movement quality is the most "intimate" part of a gesture. By using this component, we do not consider gesture as a result, but as a process: we analyze the unfolding of the movement from segment to segment (the arm, the forearm, hand, fingers) from a kinesiological point of view, i.e. an explanatory approach to the movement, (for a good introduction to the physiology of the upper limb, see Kapandji 1997). Gesture is thus captured from inside its very motion in relation to the physical properties involved in its production.
The first characteristic of the quality of the movement we will consider here is inertia: each gesture depends on the specific inertia of each segment involved. The hand represents 25% of the mass of the forearm, and the forearm constitutes 65% of the inertial mass of the arm (Dumas et al. 2007, for a similar notion see Laban & Lange 1975 and the concept of "effort" in dance). The movement of the gestures therefore flows more naturally when it unfolds from shoulder to fingers. This asymmetry between segments leads to three consequences:
1) The movement of a gesture is more naturally transferred from a proximal2 segment — shoulder or arm — to a distal segment — hand or even fingers. The number of segments in motion can be high (maximum 5).
2) When the movement propagates the other way — from the segments with the lowest inertia, fingers-hand, towards the segment with higher inertia — the energy to deploy must be so strong when segments with lower inertia are involved that the gesture often remains in the fingers, the hand or the forearm. The number of segments engaged in the movement is thus lower (1 to 3 most of the time).
3) Speed accumulates on the most distal segments — hand and fingers. Gestures that originate on the shoulder or the arm and propagate to the forearm then the hand often become faster once the hand is involved but without any specific jerk. Before it moves on its own, the hand is in motion (because of the movement of the arm and forearm). When the hand starts moving on its own, the movement of the arm itself is already decreasing. We will call this process the propagation flow of the proximal-distal movement (Boutet 2001, 2010). In the case of a movement that originates on the hand and is propagated up to the forearm, the hand acquires a certain speed with accelerations. As speed depends on the length of the segment, it is slower when measured on the joints of the fingers than on the wrist for a movement of the forearm (the wrist being the joint
2 Proximal segments of the upper limb are the segments that are closest to the torso, distal segments are the farthest to the torso.
between the hand and the forearm). The length of the forearm is about three times the length of the hand. When the movement is transferred from the hand to the forearm, the jolt in speed is potentially important for the hand, leading to acceleration, which is often visible in the blurry quality of the video recording (on average quality recordings). We call the direction of the propagation of the movement distal-proximal flow.
The number of segments in motion is potentially more important for the proximal-distal flow than it is for the distal-proximal flow.
Therefore, the bounding process of the quality of the movement component consisting in the sudden variation in velocity (acceleration, deceleration) is much more likely to be performed with distal-proximal flow and the absence of bounding is more likely to be performed with proximal-distal flow.
For this third component, bounding is marked by speed and acceleration/deceleration. As in the path in motion component, the quality of the movement can help us characterize all gestures in terms of the bounded/non-bounded expression of perfec-tive/imperfective aspect. The parameter used for both components is the same: speed. Our study will help us determine whether gestural bounding and non-bounding depend on the path in motion or on the quality of movement.
If gestures do indeed embody mentally simulated actions (Hostetter & Alibali 2008), a kinesiological approach to gesture analysis, closely connected to the embodied properties of muscular exertion or effort (Laban & Lawrence 1974/1947) and to biomechanical properties, can provide clues to understand speakers' dynamic construals of events (McNeill 1992).
1.4. A Kinesiological approach to gestuality
Kinesiologic analysis is based on the biomechanics of the segments involved in a gesture and their degree of freedom (Viviani & Flash 1995, Berthoz 1997, Kapandji 1997, Boutet 2007). A degree of freedom is defined as the relative movement of each segment independently of the adjacent segment. More concretely, each degree of freedom corresponds to the rotation of a segment around an axis relatively to another segment. Flexion/extension is an example of degree of freedom. There are 3 degrees of freedom on the arm, 2 on the forearm and 2 on the hand (for more details see Kapandji 1997, Boutet 2010). A movement initiated on one of the degrees of freedom can be transferred to another one either on the same segment, or another segment. Movement transfer is thus dependent on very specific biomechanical considerations that enable us to anticipate in which direction and according to which amplitude a movement might be performed after its initial impulse. The propagation flow of the movement can also be a structuring component of gestures. The distinction between proximal-distal flow and distal-proximal flow is not easy to measure with the naked eye. Using slow motion is often the only way to assess the order in which the segments are set in motion and thus their temporal intervals (Allen 1983). In order to achieve that assessment, we must distinguish between the motion of a segment and its own "active" movement. For a proximal-distal gesture, which begins with a movement of the arm, the forearm and the hand are involved in the motion without initiating the movement on their own. If we can distinguish those two types of motions (the motion that a segment is subjected to and the motion that the
segment actually initiates), we can determine the propagation flow. This forces us to examine the actual course of the gesture in time. This kinesiological approach enables us to analyze gesture as a process and to shed light on the biomechanical factors that explain the various parameters that express (im)perfectivity or (a)telicity (variations in speed, contacts, change of shape, duration of the gesture) and that seem quite unconnected to each other. Our approach provides analytic tools for our investigation of the most hidden features of the grammar of gestures.
Our hypothesis in the Polimod project based on the theoretical literature on aspect and previous studies on sign language and gesture was that there might be a correlation between imperfective and unbounded gestures and perfective aspect and bounded gestures. In this paper we test this hypothesis on our French data.
2. DATA AND METHOD
Our method of data elicitation was built on what Becker et al. (2011) devised in their study. It is a compromise between two methods: video recordings of natural interaction and retelling specific narratives. After having participants discuss a warm-up question about their favorite place in the world, the study involves giving them prompts to tell stories from their own personal experience about events of different types: ones that took a long time to play out (e.g., dealing with bureaucracy) and ones that involved a sudden event (e.g., an accident that they witnessed). This offers a medium between spontaneous and controlled discourse, and between narrative and conversation, the combination of the warm-up question and the conversation prompts targeted towards narratives results in a mix of uses of present and past verb tenses. This proves important since our study focuses on past tense verbs characterizing events of various types.
The data was then transcribed by a native French transcriber, checked by author 2 and aligned with the video in ELAN. The coding system was devised with the Polimod team. We decided to code verb forms, chronological time, tense, the lemma, whether there was a co-produced gesture and what we called the boundary schema of the gesture.
We coded the gestures that had any temporal overlap with the utterance of a verb. Gesture-verb overlap, even if only during the beginning preparation phase or final retraction phase of a gesture, helped us capture a large proportion of gestures that "go with" verbs (GeSp), without also including gestures that are affiliated with concepts expressed in other parts of the clause.
Building on Müller (1998), we defined bounded gestures as involving a pulse of effort at the onset, offset, or in the gesture stroke, while unbounded gestures involve smooth, controlled motion (see Table 1).
An ELAN template with controlled vocabulary was created and implemented by Author13. The first two authors coded 15% of the French data at the verbal level (verb
3 In the Polimod project, the coding system was discussed and established as a team by (alphabetical order) Dominique Boutet, Alan Cienki, Olga Irishkanova, Aliyah Morgenstern, Cornelia Mueller. Coding of tense was specific to each language (German, French, Russian) but followed the same format.
form, lemma, tense, time, presence or absence of co-verbal gesture) together in order to reach an agreement. The coding categories at this level are objective as they are descriptive. Author 2 then coded about 50% of the data with a second native speaker of French and the rest of the data by herself.
The Polimod group worked on the coding of the boundary schemas in the three languages, did double blind coding, and collectively discussed differences until they reached an agreement. The boundary schemas were coded with the sound off and without any other tier visible in ELAN. 30% of the French data was coded for gestural boundary schemas by the two first authors of this paper together in order to create coding habits. Author 1 then coded the rest of the data and discussed the coding with author 24.
Using commands in ELAN, we then counted the frequency of occurrences of the different verb forms as well as their frequency of overlap with gestures and the rate of each type of boundary schema according to tense of the co-occurring spoken verb form.
The third step of the coding which was only done for data from four of the French conversational pairs included separate coding of propagation flow of the movement, the segments involved in the movement of each gesture, the length of strokes and their speed. The coding of speed was too difficult to conduct with the naked eye and is thus not accounted for in the results presented in section 3.
3. QUANTITATIVE RESULTS FOR FRENCH
3.1. Use of tenses in French
Our results for the coding of time meaning confirm the fact that we were able to design a protocol that elicited a majority of past tenses (table 2). 52% of the verb forms used in the data referred to past time.
Table 2
Number and percentage of form according to time meaning
Past Present Future atemporal Total
1 439 1 096 77 143 2 755
52% 40% 3% 5% 100
We focused on the forms used in the imparfait, passé composé, plus-que-parfait and narrative present.
Table 3
Number and percentage of past tense forms
imparfait Passé composé Plus-que-parfait Présent Other total
457 443 70 130 339 1 439
32% 31% 5% 9% 24% 100
4 This method was chosen in order to ensure more homogeneity to the data: the whole data was thus analyzed by author 2 for speech and author 1 for gesture.
3.2. Tenses with and without gestures
Within the forms referring to past tense, 392 (27%) were produced with gestures.
Table 4
Number of forms with and without gestures and % GeSp per tense
with gestures without gestures Total % with gesture
imparfait 157 300 457 34%
passй composй 150 293 443 34%
pmsent 63 67 130 48%
plus-que-parfait 37 33 70 53%
The speakers used more imparfait (Impf) and passé composé (pc) than other tenses to refer to past events in the data but interestingly enough, narrative present and plus-que-parfait (pqp) were more often produced with gestures. For the purpose of this study we will nevertheless focus on the two main tenses, for which we know there is a clear aspectual contrast in French. They were used in equal proportion and were co-produced with gestures in equal proportion (34% of the time) which makes them quite comparable for our study.
3.3. Boundary schemas
The results of our coding of the boundary schemas are presented in graph 1.
ut ts e a ■c
a in
pi p
la lr
S
o
o p
e b
25G
2GG
150
100
5G
□ unbounded ■bounded
Impf
pc
pqp
pres
Graph 1. Distribution of boundary schemas for the co-verbal gestures produced with past tense forms
Plus-que-parfait and présent (pres) don't have marked aspectual differences, and there is no significant difference in the boundary schemas in the data.
However the difference for Imparfait and plus-que-parfait is quite significant. For the imparfait, 66.88 % GeSp are unbounded and 33.12% are bounded. For the passé composé, 28.43% GeSp are unbounded and 71.57% are bounded.
0
Here are examples to illustrate this difference.
Example 1
Figure 1. "Tout le monde passait à côté" (everyone was walking past them). The photos of the video are taken every two images. This gesture is co-occurrent to "was walking past". The movement is quite slower at the end of the gesture (images 4 and 5). Movement is much more homogeneous from image 1 to 4
Ludivine et Caroline 8 time code 2.29 *CAR: elle est tombée. she fell.
et en fait elle a dû dévaler les [/] les marches.
and in fact she must have tumbled down the stairs.
et personne n' est allé la voir.
and no one went to check on her.
*LUD: puzzled expression.
*CAR: donc euh tout le monde passait à côté.
so, um, everyone was walking passed her. *CAR: et i(l)s en avaient rien à faire. and they couldn't care less.
The unbounded gesture is co-produced with the verbal form in the imparfait.
Example 2
Figure 2. J'y suis allée trois fois. (I went there three times). These movements are co-occurrent to "went there three times".The 5 first photos are taken every four images. The following ones every 2 images. There is a strong acceleration between the 4th and 5th photo corresponding to the past participle «allée»
Example 2: Marion and Aurore. Passé composé with bounded gesture. They are talking about their problems with university registrations.
*MAR: (en)fin bon j' y suis allée trois fois avant mais bon.
oh well, anyway, I went there three times before that. quand j' ai réussi à y aller (laughs). When I was able to go there.
*AUR: (laughs)
*MAR: et j' y retourne deux semaines après. and I go back three weeks later.
The bounded gesture is coproduced with the passé composé. The strong acceleration between images 4 and 5 marks a clear frontier. The other key images (7 to 11) are taken when MAR says «trois fois» (three times). The gestures are also abrupt especially in photo 7, which shows the first occurrence, but also in 8 and 9 which show a flexion of the little finger and of the ring finger. There is also a lowering of the hand on the second occurrence, and a flexion of the ring finger in photos 10 and 11.
The results for the two main French tenses used to refer to past events match our hypothesis. Passé composé, which corresponds to perfective/delimited aspect highly correlates with bounded gestures and imparfait, which corresponds to imperfective/undeli-mited highly correlates with unbounded gestures. This seems to indicate that the linguistic and cognitive differences between imparfait and passé composé could be co-expressed in a majority of the gestures produced with the forms or the path in motion.
However the use of boundary schemas does not help us describe the specific properties of the GeSp involved and to capture what exactly in gesture production might enact the conceptual properties of the two opposing aspects in French. We therefore turned to the coding of specific kinesic features of the gestures.
4. KINESIOLOGIC FEATURES OFTHE BOUNDARYSCHEMAS
When we coded the boundary schemas, we had a set of visual criteria to help us code the boundedness or unboundedness of the gestures we were coding.
Beyond those visual criteria, we wanted to get a grasp of the actual physiological features of the gestures produced. We therefore made a move towards a fine-grained kinesiological system of analysis, which provides a way of analyzing the details of what it means physiologically to talk about bounded/unbounded motion in gestures of the hands and arms.
When we determined the boundaries, our discrimination was mostly based both on velocity and acceleration; we will focus on those features adding the flow of the movement and the number of segments involved in the gesture
4.1. Results of the coding of kinesiological features
Three types of results will be presented here:
— the propagation flow of the movement of the gesture in relation to imperfec-tive and perfective aspect in the verbal tier;
— the number and type of segments involved in the movement associated to the two aspects
— length of the flows associated to perfective and imperfective.
4.1.1. Propagation flow and perfectivity
The propagation flow expected for the gestures associated to the imparfait (imper-fective) is proximal-distal (from the arm to the fingers), whereas for the gestures associated to the passé composé the flow is expected to be distal-proximal (from the fingers to the arm).
ет с a a
ro
ь- <л g g
15 S
И и
■О ф ф »
> .с
ч- а
2 * с
ф
о ^
ф CL
90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
Proximal-distal Flow □ Distal-proximal Flow
Impf
pc
Graph 2. Distribution of the two propagation flows according to the tenses used imparfait (Impf, imperfective and passé composé (pc, perfective)
Our analysis was conducted on 4 out of the 10 sessions chosen at random. The results indicate a marked tendency, even stronger than the one we found with the boundaries. The proximal-distal propagation flow is predominantly associated to the imparfait (for 81.3% of our occurrences). The distal-proximal flow is associated predominantly to the passé composé (74%). This dichotomy seems to be even more relevant to gestures associated with aspect than the boundary schemas for our sample. We will try to give an explanation for this strong correlation.
4.1.2. Number and type of segments and perfectivity
Another level of differentiation inherent to the propagation flow and to inertia concerns the number and type of segments involved in the movement. Let us briefly summarize the nature of that link. In the case of a proximal-distal propagation flow, inertia drives the transfer of the movement to the segments with weaker inertia. Segments have decreasing inertia as they go from shoulder to arm, to forearm, to hand and finally to the last phalanx of the fingers. Thus, when the arm initiates a gesture, the transfer of the movement to the fingers will not encounter inertial resistance. The shoulder to arm direction involves a naturally declining slope. Whatever segment initiates the movement at the beginning of the gesture, transfer will be made onto a more distal segment.
The number of segments involved in the gesture will therefore be higher in this case. For the opposite flow — distal-proximal — it will be the contrary. Within that flow, for each new segment involved by the movement transfer, inertia will increase. Thus for the
gestures with a distal-proximal flow, the number of segments involved by a movement should be reduced. This difference is not marked in our data. We will come back to that in our discussion.
4.1.3. Length of verbs and gestures The other difference associated to flow is linked to the type of segment that initiates motion. For the proximal-distal flow, initial motion should be started on a segment with greater inertia, close to the chest. For the distal-proximal flow, the motion should be initiated on the fingers or the hand.
E Ë E
^ 4-1
<U =
.Q <U
<o £
3 ®
S »
m .c
<D U
w re
4- (V
0 E
Ê O
<u u
<u
CL
60%
50%
40%
30%
20%
10%
0%
■ Proximal-distal flow □ Distal-proximal flow
Fingers Hand Forearm Arm Shoulder Head Graph 3. Distribution of the first segment in motion according to the flow
Results meet our expectation. Over 90% of the gestures with distal-proximal flow start on the fingers or the hand, whereas over 80% of the gestures with proximal-distal flow are initiated by the shoulder, the arm or the forearm.
Our data does not show significant difference in the length of the gestures associated to the verbs in the imparfait and those associated to the passé composé. The results on the length of gestures in Duncan's study (2003) as well as Malaia & Wilbur (2012) for the three sign languages are not confirmed in this study. However a similar tendency is observed: the gestures associated to the perfectives are shorter than the gestures associated to the imperfective (see last two columns of table 5) although verbs in the imparfait, which is an inflectional form, are on average shorter to pronounce than verbs in the passé composé which combine an auxiliary and a past participle (see first column of table 5). The speech flow can influence the timing and length of the gestures that are associated. In the 4 sessions under study, the average length of the verbal constituent of the verbs in the passé composé is 364 ms (N = 51, SD = 154) whereas the length of imparfait is 309 ms (N = 52, SD = 110). The perfective is longer than the imperfective for these verbs. Do those durations have an impact on the gestures? The analysis of the gestures associated to each tense indicates that imperfective is associated to longer gestures than perfective (columns 3 and 4 in table 5).
Table 5
Length of verbs in the imparfait and passé composé and gestures produced with those verbs
Duration (ms) Duration Flow proximal-distal (mean) Duration Flow distal-proximal (mean)
imparfait 309 594 SD: 439 502 SD: 293
passé composé 364 548 SD: 386 493 SD: 284
4.2. Discussion of the results of the analyses of kinesiological features
The analysis of the specificities of flow gives coherence to apparently unrelated phenomena. Perfectivity can be semantically related to the propagation flow of the movement.
The distal-proximal flow involves the initiation of the movement with segments whose amplitude is reduced (see Boutet 2015:121—122). The articulatory stop is reached more quickly on the fingers and the hands than on the forearm and the arm. The movement is blocked more rapidly even if the transfer of the gesture continues onto the forearm. This natural boundary echoes the perfectivity of the event — or the accomplishment of the event.
On the contrary, the proximal-distal flow involves a movement that is predominantly executed on the arm or the forearm. The amplitude of those segments, being much greater, rarely reaches a stop before the adjacent distal segment is in turn in motion. As inertia decreases, the movement propagates progressively and seems to be more homogeneous and without borderlines, even when it involves the hand. The imperfective aspect of the event, without marked boundaries corresponds to the proximal-distal flow. Moreover, the internal point of view carried by the imperfective (Comrie 1981:3) is echoed in the involvement of the segments closer to the chest — that are more internal —, whereas the gestures of the distal-proximal flow set more distal segments in motion, which can correspond to an external point of view associated to the perfective.
The length of the gestures associated to the imparfait and the passé composé is interesting to consider. The distal-proximal gestures are executed more quickly (means in milliseconds: 493 with passé composé and 502 with imparfait) and the proximal-distal gestures are lengthier (means: 548 and 594). The difference in standard deviation for each flow is also important (see values SD in columns 3 and 4, table 5). These trends are not statistically significant and indicate that a more thorough investigation of the duration of each gesture according to the number of segments involved in the movement could be relevant. Our results illustrate the importance of taking co-verbal gestures into account in the analyses of aspect in French for past events — raising questions about what the results might be like for other languages with grammaticalized aspectual distinctions.
5. CONCLUSION
In this study, we have shown that in our data of interactions in French, gestures that mark a boundary are predominantly associated with the passé composé (perfective) whereas the gestures without boundaries are more often associated with the imparfait (imperfective).
The notion of boundary associated to aspect in the literature was conceptual. As far as gestures are concerned, that notion is anchored in kinesiological features and boundedness is thus enacted in our French speakers' gestures. Boundaries are marked by change or preservation of speed. It is a purely kinematic feature.
In our more detailed kinesiological study of 4 sessions of those French interactions, the specific features of flow correspond to verbal perfectivity even more closely than the boundary schemas we had defined based on kinematic criteria. Indeed kinematic features represent the study of the movements independently from their cause, whereas a kinesiological approach to gestures helps us determine the links between form and function. The question raised is whether physical and kinematic features are the ultimate criteria we should take into account for the gestural expression of perfective and imperfective aspect.
Flow, which is a kinesiological notion that takes both biomechanic properties and their cause into account, corresponds better to the enactment of (im)perfectivity in French than gestural boundaries based on kinematic criteria. The question of the status of kinematics and kinesiology is thus at play in this study. Are the physical and kinematic properties the ultimate criteria to understand the gestural expression of aspect? If they are, at a kinesiological level, flow would simply be a means. Another option is to consider flow as a marker whereas the kinematic elements (speed, duration, acceleration) would only be its natural consequences. The issue is to understand whether the gestural expression of aspect is linked to a logic that is external to gestuality itself (kinematic features independent of the speakers' conceptualizations and intentions), or whether they derive from the internal functioning of gestures.
Further investigations are needed in order to pursue the difference between the kinematic option in which the gestures' boundedness would be an enactment of the aspects marked in speech, and the kinesiological option that accounts for the biomechanic properties of the movement enacting (im) perfectivity.
We have shown in this paper that both kinematic features and kinesiological features are expressed in the gestures associated to speech about past events narrated by French speakers. Either way, co-verbal gestures associated with the passé composé and the imparfait could thus be enactments of perfectivity and imperfectivity.
© Dominique Boutet, Aliyah Morgenstern, Alan Cienki, 2016 REFERENCES
Allen, James F. (1983). Maintaining knowledge about temporal intervals. Communications of the
ACM 26, no 11, 832—43.
Becker, Raymond, Alan Cienki, Austin Bennett, Christina Cudina, Camille Debras, Zuzanna Fleischer,
Michael Haaheim, Torsten Müller, Kashmiri Stec, Alessandra Zarcone. (2011). Aktionsarten,
Speech and Gesture », 2011. http://pub.uni-bielefeld.de/publication/2300874.
Berthoz, Alain (1997). Le sens du mouvement. Bibliothèque (Paris. 2007). Paris: O. Jacob.
Boutet, Dominique (2001). Approche morpho-dynamique du sens dans la gestuelle conversationnelle. Université de Paris VIII.
Boutet, Dominique (2015). Conditions formelles d'une analyse de la négation gestuelle». Vestnik of Moscow State Linguitic University Discourse as social practice: priorities and prospects, no 6 (717), 116—29.
Boutet, Dominique (2007). Gesturing as Substratum and Support: a Case of Continuity. In Interacting Bodies. Lyon; France. http://hal.archives-ouvertes.fr/index.php?halsid=rrlb1bfqut9aihl7fVqvjorig0& view_this_doc=halshs-00186622&version=1.
Boutet, Dominique (2010). Structuration physiologique de la gestuelle: modèle et tests». Édité par Jean-Marc Colletta, Agnès Millet, et Catherine Pellenq. Lidil, Multimodalité de la communication chez l'enfant, no 42 , 77—96.
Bressem, Jana, et Cornelia Müller (2014). The family of AWAY-gestures. In Body-Language-Communication: An International Handbook on Multimodality in Human Interaction. Handbooks of Linguistics and Communication Science, 38.2. Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Jana Bressem.
Brunot, F. La pensée.
Calbris, Geneviève (2011). Elements of Meaning. Amsterdam, Philadelphia: John Benjamins Publishing Company.
The Semiotics of French Gestures. (1990). Advances in semiotics. Bloomington: Indiana University Press.
Comrie, B. (1976). Aspect An introduction to the study of verbal aspect and related problems.
Comrie, Bernard (1981). Aspect and Voice: Some Reflections on Perfect and Passive in Tense and Aspect. In Philip Tedeschi and Annie Zaenen.
Culioli A. (1999). Valeurs aspectuelles et opérations énonciatives: l'aoristique, in Pour une linguistique de l'énonciation, tome II, Paris: Ophrys, 127—143.
Damourette, Jacques, Edouard Pichon (1935). Des Mots à la pensée: essai de grammaire de la langue française, 1911—1940. 8 vol. Paris, France: édition d'Artrey.
de Boer, Cornelis. (1947). Syntaxe du français moderne. Universitaire Pers Leiden.
Desclés, Jean-Pierre, Zlatka Guentchéva (1996). Convergences et divergences dans quelques modèles du temps et de l'aspect. INSemantyka a konfrontacja jezykowa, I, Varsovie, 23—42.
Dumas, R., Chèze L., Verriest J.-P. (2007). Adjustments to McConville et Al. and Young et Al. Body Segment Inertial Parameter. Journal of Biomechanics 40, no 3, 543—53. doi: 10.1016/j.jbiomech.2006.02.013.
Duncan, Susan D. (2002). Gesture, verb aspect, and the nature of iconic imagery in natural discourse». Gesture 2, no 2, 183—206.
Gosselin, L. (2005). Temporalité et modalité. (postface de B. Victorri), Bruxelles, Duculot-de Boeck, col. Recherches. 254 p.
Gosselin, Laurent, Cédric Person (2005). Temporalité. In Sémantique et traitement automatique des langues, édité par Patrice Enjalbert, 173—214. Traité IC2, série Cognition et traitement de l'information. Hermès Sciences, Lavoisier.
Grevisse, Maurice, André Goosse (1991). Le bon usage: grammaire française. Duculot. Paris, France, Belgique.
Guillaume, Gustave (1929). Temps et verbe: théorie des aspects, des modes, et des temps: suivi de L'architectonique du temps dans les langues classiques. H. Champion.
Hostetter, A.B., M.W. Alibali (2008). Visible Embodiment: Gestures as Simulated Action. Psycho-
nomic Bulletin & Review 15, no 3, 495—514. doi: 10.3758/PBR.15.3.495. Kapandji, I.A. (1997). Physiologie articulaire. 1, Membre supérieur. Paris: Maloine. Kendon, Adam (2004). Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. Laban, Rudolf, Frederick Charles Lawrence (1974). Effort: economy of human movement. Macdonald and Evans.
Lallot, J. de Thrace, D. (1998). La grammaire de Denys de Thrace. Paris: CNRS Editions.
Malaia, Evie, Ronnie B. Wilbur (2012). Kinematic signatures of telic and atelic events in ASL
predicates. Language and speech 55, no 3, 407—421. McNeill, David (1992). Hand and mind: what gestures reveal about thought. Chicago; London:
University of Chicago press. Müller, Cornelia (1998). Redebegleitende Gesten: Kulturgeschichte, Theorie, Sprachvergleich.
Strickland, Brent, Carlo Geraci, Emmanuel Chemla, Philippe Schlenker, Meltem Kelepir, Roland Pfau. «Event Representations Constrain the Structure of Language: Sign Language as a Window into Universally Accessible Linguistic Biases». Proceedings of the National Academy of Sciences, 27 avril 2015, 201423080. doi: 10.1073/pnas.1423080112. Talmy, Leonard (1985). Lexicalization patterns: Semantic structure in lexical forms. Language typology and syntactic description 3, 57—149. Viviani, Paolo, Tamar Flash (1995). Minimum-jerk, two-thirds power law, and isochrony: converging approaches to movement planning. Journal of Experimental Psychology: Human Perception and Performance 21, no 1, 32.
von Laban, Rudolf, Roderyk Lange (1975). Laban's principles of dance and movement notation. Princeton Book Co Pub.
Wilbur, Ronnie (2003). Representations of telicity in ASL. In Proceedings from the Annual Meeting of the Chicago Linguistic Society, 39:354—368. Chicago Linguistic Society. http://www.ingentaconnect.com/content/cls/pcls/2003/00000039/00000001/art00023. Wilmet, Marc (2003). Grammaire critique du français. Grammaire critique du français, Bruxelles: Duculot.
Article history:
Received: 18 July 2016 Revised: 16 August 2016 Accepted: 18 August 2016
Finance and Acknowledgement
This paper is part of a larger research project led by Alan Cienki at Moscow State Linguistic University and has been supported by grant number RSF 14-48-00067 from the Russian Scientific Foundation. The main members who have participated in the elaboration of the ideas expressed in this paper are Olga Iriskhanova, Cornelia Müller, Nicole Richter and the authors of this paper. We thank our anonymous reviewers for their insightful comments.
For citation:
Dominique Boutet, Aliyah Morgenstern, Alan Cienki (2016). Grammatical Aspect and Gesture in French: A kinesiological approach. Russian Journal of Linguistics, 20 (3), 132—151.
Bio Note
Dominique Boutet is associate Professor at the University of Rouen. He is a linguist in the domain of gestures and Sign Language. His research focuses on the interface between gestuality and Sign Languages especially on semantic invariant features with a kinesiological approach to gestures. Contact information: e-mail: dominique_jean.boutet@orange.fr
Aliyah Morgenstern is currently Professor at the Sorbonne Nouvelle University where she teaches English linguistics and language acquisition. She supervises a number of research projects on children's communication and linguistic development, on sign language and on gestures in spontaneous interaction financed by several agencies. She has published books and papers on language acquisition and multimodal communication using socio-pragmatic, constructionist, cognitive and functionalist perspectives. Contact information: e-mail Aliyah.Morgenstern@univ-paris3.fr
Alan Cienki, Professor of Language Use and Cognition at the Vrije Universiteit Amsterdam, Netherlands and Professor at Moscow State Linguistic University, Russian Federation, where he founded and directs the Multimodal Communication and Cognition Lab (PoliMod). His research interests include cognitive linguistics, gesture studies, and comparative/contrastive linguistics with a focus on spoken language. Contact information: a.cienki@vu.nl
ЖЕСТЫ И ГРАММАТИЧЕСКАЯ КАТЕГОРИЯ АСПЕКТА ВО ФРАНЦУЗСКОМ ЯЗЫКЕ: КИНЕСИОЛОГИЧЕСКИЙ ПОДХОД
Доминик Бутэ1, Алия Моргенстерн2, Алан Ченки3
Университет Руана 5020 Франция, Париж, ул. Belleville, 7
2Университет Париж III Новая Сорбонна 75002 Франция, Париж, ул. St Fiacre, 12
3Свободный университет Амстердама 1105 1081 HVАмстердам, Нидерланды, VU, De Boelelaan
Цель данной статьи — доказать, что исследование жестов в речи может пролить свет на понимание особенностей концептуализации грамматических категорий в процессе говорения. Выражение мысли говорящим предполагает динамическое взаимодействие нескольких факторов: высказывание формируется под воздействием связанных с передаваемой мыслью сенсомоторных ощущений, множественных семиотических средств, которыми мы располагаем для выражения мысли, и имеющейся в языке той или иной лингвистической категории. Анализ категории аспекта в речи, сопровождаемой жестами, в устных интеракциях в условиях эксперимента, нацелен на изучение того, как конструирование говорящими аспектуальных различий, грамматикализованных в языке, может проявляться в жестикуляции. Ставится вопрос о том, до какой степени различия в структуре события, выраженные грамматическим аспектом (совершенным и несовершенным видом), коррелируют с кинесиологическими особенностями жестикуляции. Для этой цели анализу подвергаются скорость и последовательность движений, а также участвующие в жестикуляции части тела (пальцы, кисть руки, предплечья, руки, плечи). Кинесиологический подход к жестикуляции позволяет анализировать движения человеческого тела в соответствии с биомеханической точкой зрения, которая принимает во внимание физиологические особенности.
Грамматический вид, по Б. Комри (1976) [1989], предполагает внутреннее развертывание процесса. В его интерпретации, «время — это дейктическая категория, т.е. оно локализует ситуацию во времени, соотнося ее, главным образом, с настоящим моментом. Вид не соотносит время ситуации с какой-либо другой временной точкой, он скорее характеризует внутреннюю темпоральную структуру ситуации; различие между временем и видом можно определить как разницу между внутренним временем ситуации (категорией аспекта) и внешним временем ситуации (категорией времени)» (Comrie, 1976 [1989]:5).
Может ли кинесика быть связанной с выражением семантических различий такого рода? Когда говорящий делает жест, характеризующийся специфическими физиологическими свойствами, например определенной скоростью, есть ли внутренний источник, который влияет на эту скорость? Может ли трактовка события говорящим, которая просматривается в выборе видовой формы, вести к выбору жеста, который характеризуется специфическими свойствами? В данном исследовании предпринимается попытка ответить на поставленные таким образом вопросы и апробировать новые методы анализа жестовой стороны речи.
История статьи:
Дата поступления в редакцию: 18 июля 2016 Дата принятия к печати: 18 августа 2016
Для цитирования:
Доминик Бутэ, Алия Моргенстерн, Алан Ченки. Жесты и грамматическая категория аспекта во французском языке: кинесиологический подход // Вестник Российского университета дружбы народов. Серия: Лингвистика. 2016. Т. 20. № 3. С. 132—151.
Сведения об авторах:
Доминик Бутэ, доктор, Университет Руана. Сфера научных интересов: невербальная коммуникация, язык глухих, кинезиологический подоход к изучению языка жестов. Контактная информация: e-mail: dominique_jean.boutet@orange.fr
Алия Моргенстерн, профессор, Университет Париж III Новая Сорбонна. Сфера научных интересов: биллингвизм и коммуникативная лингвистика, когнитивная лингвистика, социолингвистика, прагматика, невербальная коммуникация (язык жестов). Руководитель международных исследовательских проектов, автор ряда научных трудов в русле вышеуказанных направлений. Контактная информация: e-mail Aliyah. Morgenstern@univ-paris3.fr
Алан Ченки, профессор, Свободный Университет Амстердама. Директор и основатель Когнитивной лаборатории полимодальной коммуникации на базе Московского государственного лингвистического университета. Сфера научных интересов: когнитивная лингвистика, невербальная коммуникация, сравнительное языкознание. Контактная информация: e-mail a.cienki@vu.nl
