Научная статья на тему 'Language, gesture, bodily stance: grammar as a multimodal system'

Language, gesture, bodily stance: grammar as a multimodal system Текст научной статьи по специальности «Языкознание и литературоведение»

CC BY
306
88
i Надоели баннеры? Вы всегда можете отключить рекламу.
Ключевые слова
BLEND / COGNITIVE LINGUISTICS / COMMUNICATION / DATA

Аннотация научной статьи по языкознанию и литературоведению, автор научной работы — Mark Turner, Francis Steen

Television news represents a uniquely available dataset of common communicative practices, with the additional attraction that it has been deliberately crafted by professional teams to grab attention, convey complex meanings in a rapid and compressed format, and persuade by implication. The way in which television succeeds in tapping into ancient interpretive structures in our cognitive system makes its detailed examination particularly revealing.

i Надоели баннеры? Вы всегда можете отключить рекламу.
iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.
i Надоели баннеры? Вы всегда можете отключить рекламу.

Текст научной работы на тему «Language, gesture, bodily stance: grammar as a multimodal system»

9 Бел^ А. Петербург. СПб., 2004. C. 41. 11 Белый, А. Петербург... C. 173.

10 Тихонов A. H. Словарь русских личных имен / A. H. Ти- 12 Белый, А. Петербург\.. C. 236.

хонов, Л. З. Бояринова, А. Г. Рыжкова. М., 1995. С. 180.

Вестник Челябинского государственного университета. 2014. № 7 (336). Филология. Искусствоведение. Вып. 89. С. 40-47.

Mark Turner

LANGUAGE, GESTURE, BODILY STANCE: GRAMMAR AS A MULTIMODAL SYSTEM*

* The co-author of this article is Francis Steen. The article partly duplicates the article: Francis Steen & Mark Turner. 2013. "Multimodal Construction Grammar." In Borkent, Michael, Barbara Dancygier, and Jennifer Hinnell, editors, Language and the Creative Mind. Stanford, CA: CSLI Publications. Pages 255-274. With the given publication the authors and the editorial board propose to expand the reading audience due to the state-of-art cognitive-linguistic approach to subject matter and its high scientific importance.

Television news represents a uniquely available dataset of common communicative practices, with the additional attraction that it has been deliberately crafted by professional teams to grab attention, convey complex meanings in a rapid and compressed format, and persuade by implication. The way in which television succeeds in tapping into ancient interpretive structures in our cognitive system makes its detailed examination particularly revealing.

Keywords: blend, cognitive linguistics, communication, data.

Nearly all scenes involving human beings include communication in multiple modalities. Facial expressions, eyes, hand gestures, bodily stance, and voice are combined with the perceived affordances of the surroundings into an integrated and partially shared experience, including touch and audiovisual signs that are often not verbal or, anyway, not strictly verbal. There are 7 billion human beings on earth now, and they have been communicating in this fashion for at least 50,000 years. The result is a vast amount of potential data, comparable in complexity to the information available to astrophysicists, zoologists, and geneticists. Yet the datasets linguists rely on for deciphering the hidden orders of human communication remain almost entirely textual. The relatively few and small corpora we have of multimodal communication often come from specialized circumstances, such as interviews conducted by experimenters in whit-eroom lab settings. How can we broaden the foundations of our study of human communication?

Methodologically, we need to check our hypotheses against data. In the face of small, biased, and narrow archives of data, language scientists have often relied on personal introspection to choose between hypotheses. Yet, it is well-recognized by now in cognitive linguistics that although personal intuition can be a source of hypotheses, it has weaknesses as a test of hypotheses. For example, as D^browska [3] shows, judgments made by linguists diverge from those made by the gen-

eral population. She concludes that "syntacticians should not rely on their own intuitions when testing their theories."

For all these reasons - the sparseness and the biases of the data, the relative lack of systematicity, the relative lack of multimodality, the unreliability of introspective data - there has been a push in linguistics to develop corpora and methods for investigating cognitive linguistic questions through big datasets, some of them multimodal.

The Distributed Little Red Hen Lab, or "Red Hen," is a research program dedicated to this goal. Red Hen is a global enterprise designed to create a massive systematic corpus of ecologically valid multimodal data, along with new tools and practices to analyze this data. Red Hen records audiovisual news broadcasts systematically, and supplements the resulting dataset with other audiovisual records. This is made possible by section 108 of the U.S. Copyright Act, which authorizes libraries and archives to record and loan audiovisual news programs.

The Data

Red Hen's core dataset consists of the NewsScape Library of International Television News, housed and maintained securely at the library of the University of California, Los Angeles. Other datasets are in development. NewsScape consists of roughly 200,000 hours of broadcast network news, in a variety of languages. These data reveal the quick cultural creativity and extraordinary

cultural variation of network news. They include roughly a billion words of timestamped closed-captioned texts, and roughly a billion words of transcripts. Red Hen ingests another hundred hours or so of global network news daily. We have begun to capture several Scandinavian channels and two channels from Spain, and we are currently working on expansion to other languages.

Red Hen has developed code for putting transcripts, where they exist, into time-stamped registration with the closed-caption text and aligning both with the audiovisual stream. Red Hen is also extracting on-screen text with optical character recognition and exploring ways to deploy speech-to-text transcription for broadcasts that lack closed-captions. Funded by a four-year National Science Foundation grant, Red Hen employs graduate students in Statistics, Computer Science, Information Studies, Communication Studies, and Political Science at UCLA and the University of Illinois Urbana-Champaign (UIUC) to develop techniques for automatic story segmentation, topic hierarchies, named entity recognition, speaker identification, and geospatial tagging. Computer vision students work on person detection, face recognition, and the characterization of visual features in facial expressions, clothing, and pose. Landscapes, buildings, vehicles, objects, and symbols are also being tagged, to permit richer description of the af-fordances of the environment and their systematic use in visual communication. Scholars of audio analysis examine the collection for strong emotions, vocal characteristics, and patterns of music use. Information scholars are developing new and interactive forms of data visualization. Broadcast news presents a series of hard problems to multiple disciplines, calling for transdisciplinary collaboration.

Red Hen data are highly multimodal, including speech, on-screen text, gesture (broadly defined), bodily stance, music, sound effects, graphics, and a range of other audiovisual expressions. Red Hen data are legendarily ecologically valid: human beings find TV news immersive throughout their lifespan, in all developed cultures and in all of the most-commonly studied languages. The data are broadcast daily to billions of people worldwide - for an impression, see the supporting website, which presents pictures of people in various environments attending to the news.

While the experience of watching the news is now ubiquitous, as a mode of human communication it remains a cultural novelty. The technologies and practices of multimodal news communication

have undergone rapid developments over the past century - a blink of an eye compared to the history of language. They represent significant cognitive innovations: at no previous point in human history have people had the ability to assemble representations of events that to a high degree reproduce the human experience of vision and hearing. The technologically mediated communicative practices of television news present an opportunity and a challenge to our understanding of the human communicative system, inviting us to extend our study of linguistic phenomena into new multimodal forms of expression.

What do we think is happening when we watch the news? How do we make sense of the scene? Human beings have a rich experiential understanding of scenes of face-to-face "communicative joint attention." Joint attention is a human-scale scene in which some people are attending to something and know they are all attending to something and know also that they are engaged with each other in attending to it [18 ; 17]. In "communicative joint attention," people are not only jointly attending but also communicating with each other about the focus of their attention, even if the communication is very sparse, consisting perhaps of only pointing. We use the term «classic joint attention» to refer to perhaps the most fundamental scene of communicative joint attention, in which two or a few people in face-to-face presence are not only attending to something that is directly perceptible but are moreover communicating about it in a sustained way (Thomas & Turner 2011, Part 3.). The multimodal patterns of managing classic joint attention have been closely studied in the tradition of Erving Goffman [9], notably in e.g. [14].

Watching the news is not a scene of classic joint attention, but it tacitly builds upon that understanding. The many mental spaces needed to make sense of a scene of watching network news include classic joint attention, the broadcast viewer, everyone in the viewer's environment involved in jointly watching the broadcast, everyone outside the viewer's environment involved in watching the broadcast, the staff involved in crafting the communication, the crews that handle the technology, the technology itself, the items that are the focus of the news and to which our attention is directed, and so on and on and on. This diffuse mental network would be intractable to the viewer except that we can blend its many connected mental spaces into an anchoring scene of blended joint attention, much of whose structure is provided by the all-important input space of classic joint attention. In

the blend, we are in a scene of human-scale classic joint attention. Some of the management and communication techniques available in classic joint attention project down easily to the blend, some do not, and the blend develops emergent techniques of its own, which it is our purpose to study.

We are not deluded by this blend; contrary to the claims of Reeves & Nass (1996), it is not simply the case that we equate the media with reality. The blend integrates conceptual structure from multiple sources into a seamless and mentally manageable whole. We know, for example, that, in the full mental network, the anchor who addresses us as "you" or who says "I will show you that in a minute" and who is looking at "us" does not actually know us or see us. In the vast mental network surrounding and anchored by the human-scale scene of blended joint attention, we know that there are hundreds of agents, and people we do not see, and technological manipulation, and so on. Our attention, however, is typically focused on the meanings made available through high levels of compression achieved inside the blend. Inside the blend, we are in a congenial scene structured by the concept of classic joint attention. Accordingly, many of the linguistic constructions and words and gestures developed for classic joint attention can be projected to the blend and used directly of the blend. We do not need to invent entirely new language or gestures in order to run and understand this scene of news broadcasts. To be sure, as we will see, new or extended communicative constructions can also emerge in this blend, but they are based on constructions we already know for our normal scenes of classic joint attention.

One of the basic research missions of Red Hen is to analyze such multimodal constructions.

What is a Construction?

Construction grammarians use the term "construction" in a range of ways, but in all of them, a construction is a mental packet, consisting of a form-meaning pairing that speakers of a language know. Goldberg writes:

A construction is defined to be a pairing of form with meaning/use such that some aspect of the form or some aspect of the meaning/use is not strictly predictable from the component parts or from other constructions already established to exist in the language. On this view, phrasal patterns, including the constructions of traditional grammarians, such as relative clauses, questions, locative inversion, etc. are given theoretical status. Morphemes are also constructions, according to the definition, since their form is not predictable

from their meaning or use. Given this, it follows that the lexicon is not neatly delimited from the rest of grammar, although phrasal constructions differ from lexical items in their internal complexity. Both phrasal patterns and lexical items are stored in an extended 'construction' [10].

A construction might include various related elements for which traditional linguistics has various names - phonology, internal syntax, external syntax, semantics, pragmatics, and so on. Constructions can be more or less "lexically filled." For example, the "Way" construction [12] and the "What's X doing Y?" construction [13] require that the lexical item "way" and the lexical items "what" and "doing" fill the right spots in instances of the constructions. Examples of the "Way" construction include "she found her way to the market" and "he lied his way to the top." Examples of the "What's X doing Y?" construction include "What are you doing talking to me?" and "What's this coffee cup doing on my coffee table?"

The basic vocabulary we need to talk about research into multimodal constructions comes from the notion that to know a language is to know a relational network of constructions and how they can be blended to create forms that when expressed count for (many) other speakers of the language as belonging to the language. They count as expressions in the language because those other speakers are able to find in their own relational network of constructions some established constructions that blend to produce just those forms. By "construc-ticon," we will mean this knowledge of constructions and how to blend them. Broadly, knowing a language is knowing such a relational network of constructions and how its constructions can be blended to produce forms that can be expressed. Accordingly, the two-year-old and the eighty-year-old from different geographical areas can both be thought to "know" a single language even though what they know might be considerably different. We modify our knowledge of constructions and the way they blend as we develop.

Crucially, the constructicon is multimodal. We can sum up our project as the investigation through the powers of Red Hen of multimodal constructions and how they blend. In exploring the ways in which the idea of a construction in linguistics extends and generalizes to technologically mediated forms of multimodal communication, as they have been extended to interpersonal forms of multimodal communication, we remain open to the possibility of radical discontinuities and unprecedented innovations. Our point of departure

is that network news relies heavily on a carefully orchestrated system of generating blended joint attention, creatively building on a series of constructions we recognize from the analysis of linguistic communication.

Audiovisual Correlates of Linguistic Constructions

Cognitive linguists routinely study basic mental operations and phenomena that are not exclusive to language but that are deployed in language and leave their mark on its structure: mental space phenomena, conceptual integration, categorization, image-schematic structuring and transformation, fictive motion, force dynamics, viewpoint phenomena, scanning... Since the news deploys other modalities than speech and text, it is an obvious project to look for the ways in which these basic mental operations and phenomena are deployed in those other modalities.

Many words in many languages prompt us to build conceptual integration networks containing relations of counterfactuality, such as "safe", "accident", "dent", "mistake", "Good thing...", "Too bad...", and so on. The news presents counter-factuality so routinely that it is surprising that the stereotype of the news is that it presents "what's happening." The news has standard routines for audiovisual prompting of the counterfactual. One of the most straightforward of these standard routines is to show a visual prompt for the actual counterfactual scene. For example, a young woman who stars as a student in a based-on-a-true-story film about a high-school teacher who was fabulously successful in teaching mathematics to inner-city poor youth reports, "I thought if only I had come to this school, if only I had him for a teacher, I would have learned math", and the TV news immediately shows a picture of a young woman dressed as a student standing with the legendary teacher, so we can map the speaker onto the student. (See the supporting website.) A woman at the beauty parlor says "If only I had some shoes", and the TV news immediately shows a vast array of shoes. (See the supporting website.) It is easy for us to imagine the counter-factual, but not to perceive it. Television is using its standard technique in these cases of moving an operation that is normal to human imagination into actual human perception. The bizarre perceptual phenomenon does not seem bizarre to us at all because it is a familiar imaginative phenomenon.

Such data, in which the news visually presents a representation of the counterfactual scene or counterfactual elements needed for constructing the blend, are vast and often subtle and nuanced,

falling into a variety of types. Consider the word "detour", which prompts us to construct one space with one path A on which something travels and another space with another path B on which the same thing travels. When we blend these so as to take both paths but travel on A, then the counter-factual relationship between travel on A and travel on B is compressed to absence of travel on B. Language for expressing this blend includes "direct", "quick», and so on. If we blend the same inputs so as to take both paths but travel on B, then the counterfactual relationship between travel on A and travel on B is compressed to absence of travel on A. Language for expressing this blend includes "detour", "delay", and so on. Red Hen offers a specific case in which police traveling to an island took path B by inflatable boat, and during the period of the travel, scores of young people were murdered on an island not far from Oslo, Norway. In this presentation, the "detour" causes a "delay", used as an invitation to the audience to construct a network with a counterfactual connection between the two journeys. Without explicitly accusing the police, the news is designed to lead its viewers to contemplate the alternative scenario and to compare the two outcomes. The delta of this operation is additional deaths: taking the "shortest route" would accordingly have "saved lives." All during the linguistic presentation, the news is showing a satellite image of the geographical area, with the diagrammatic trace of the possible and the actual travel by the police. The dynamic travel that in fact did not happen is shown first, followed by the route actually taken (see the supporting web site). The words "Dette er den korteste veien til Ut0ya" 'this is the shortest route to Ut0ya' and "politiet valgte a kj0re denne omveien" 'the police chose to take this detour' are accordingly paired with visual presentations for the blend that prompt for the activation of the different input spaces, the counterfactual relationship between them, and the compression to features like absence, detour, delay, and so on.

This depiction of the counterfactual exemplifies how the news uses the multimodality of the presentation to prompt for the understanding that we linguists usually analyze by inspecting the linguistic phenomena only. Red Hen would seem to be a natural dataset for such investigations. Presumably, for example, one could investigate gestural prompts for the construction of counterfactuality and blending to produce emergent structure.

Let us consider a specific grammatical construction for which there was originally an im-

plicit hypothesis about its natural alignment with multimodal presentation. Nikiforidou [15 ; 16] analyzes the role of blending in a construction she calls "Past tense + proximal deictic», with emphasis on the cases where the proximal deictic is "now." This well-known construction is also broadly analyzed in [5. Ch. 7 ; 5]. The preferred patterns are "was/were + now", as in "It was now possible..." and, for a non-copula verb, "now + past tense", as in "He now saw that." Nikifori-dou provides "a detailed blueprint of the blending mappings cued by the [past + proximal deictic] pattern" [15. P. 177]. Essentially, the pattern calls for a blend of viewpoints, in which our overall understanding is stage-managed from the point of view of a narrator but some self or consciousness located in a previous time is contextually available and prominent, and the events experienced in that previous time are to be construed "from the point of view of that consciousness, as that character's thoughts, speech or perceptions" [16. P. 266]. The blended viewpoint takes on elements of different perspectives and compresses a time relation. The mental space of the narrator's condition is still the mental space from which the narrated space is accessed and built up, but the experiential perspective comes from inside the narrated events. There is considerable emergent structure in the blend. In the blend, it is possible to have not only knowledge that is available only at a distance but also to have the experience, perception, and realization available only up close.

In a study of the British National Corpus, Niki-foridou shows that this is a highly productive construction, even outside of literary genres.

Presciently, Nikiforidou writes that the grammatical pattern has the "effect of zooming in on the events" [15. P. 180]. Let us consider "zooming in." A human being can shift focus and attention and can locomote or lean forward or back in order to change the angle that objects subtend in the visual field, but, absent assisting technology, a human being cannot zoom in on or zoom out from an actual percept - that is, a human being cannot, without changing the location of the head, change the angle that objects subtend in the visual field. Yet, it is easy to "zoom" in visual imagination, as anyone can demonstrate. In your mind's eye, picture some landmark near where you live - a familiar building, a bridge, a body of water, for example - or picture a person. Now, in visual imagination, zoom in. Now zoom out. Zooming seems to be a very common and easy manipulation in imagination. Perhaps imagination works this way because

we do have the actual experience of an image in the visual field getting "larger" as we move toward it or "smaller" as we move away from it, and we can of course do time-scaling in imagination because of blending, so zooming in imagination is available as a product of blending over experience, but not projecting to the blend the movement of the viewer. Does the news adopt this mental functionality of the zooming imagination by using a photographic visual-field analogue of mental zooming? That is, does it move an operation from imagination into perception? Yes. For screens that result from camerawork, we have available a mental blend in which our eye is fused with the camera lens or with the through-the-lens viewfinder. Now, when the camera zooms - which it can do because it has a lens array that our eyes do not have - our vision, in the blend for the news, zooms. So the next question is: When we are in a news scene of narration and the narrator uses the past + now construction, does the camera zoom in on the self or consciousness that had the experiences?

Yes. Or rather, that is what we often find in Red Hen. There is a hitch in providing this past + now visual zoom, because the narrator speaks at one time about a consciousness at a previous time. That is a mismatch. The consciousness and its experiences are not available in the narrator's immediate environment, or indeed in any of the mental spaces we have for considering the production and broadcast of the narration. How do we deal with the mismatch? The news production team must provide some suitable prompt for that consciousness in the past, but it isn't with them. There are several ways to resolve the mismatch. The three most common appear to be (1) have the person who is coreferential with the consciousness we are narrating re-enact the events, with the appropriate setting and staging and so on, and film that scene; (2) find archival still photos of that person at the time and present them, perhaps, e.g., with a Kens Burns effect, as the narrator uses the past + now construction; (3) find historical film footage containing the person and run that footage as the narrator uses the past + now construction. One can, of course, do all three. For all three of these expedients, one can zoom in on the images. Of course, narrators on the news usually just narrate, without using this extra production, but such zooming production is easy to find in Red Hen. Ni-kiforidou did not explicitly predict these data, but implicitly she did, and the data support her intuition of what is going on mentally.

Here are two examples (see the supporting website for the clips). In the first, the narrator is

telling the story of Kim, who as an adult had to deal with her mother's continuing to invest in what Kim came to understand was a fraudulent business scam. While the narrator uses phrases like "Kim now saw..." and "Kim now wondered...", we see scenes of what must be a more recent Kim balancing a checkbook, shaking her head, doing sums, comparing documents, and so on. Kim is re-enacting her consciousness and behavior from the past. The production team uses camera zoom and jump cut to get closer to the image of Kim's re-enactment of her scenes of realization as the narrator uses the past + now construction.

Nikiforidou writes of the linguistic construction.

In blending terms, ... resolution of (apparent) conflict is often achieved through the mechanism of compression, whereby elements that are conceptually separate in the input spaces are construed as one in the blended space. The construction at hand, I suggest, cues a particular kind of compression, namely compression of a time relation. The dynamic, continuously updated character of such blending networks renders them particularly suitable for representing meaning in a narrative, where formal clues may often give conflicting instructions even within the same sentence (as is the case with FIS [Free Indirect Speech]) [15. P. 179].

We see in the news the audiovisual analogue of this blending prompt: in the news, we have input spaces in which the narrator and the production team have a viewpoint on the past and on the orchestration and arrangement of all the mental spaces involved in the network of the narrative, and we have a particular input space that has Kim and her experiences, and we are prompted to create a blended space that has projections from both the orchestrating viewpoints and Kim's viewpoint, creating there a great compression not only of viewpoint but also of time.

One of the most interesting and common uses of past + now occurs when the narrator and the narrated consciousness are coreferential. We find many such examples in Red Hen, such as a documentary on the Pentagon Papers, in which Daniel Ellsberg, who leaked the Pentagon Papers, is narrating in advanced age his exploits in the period 1967-1971. (See the supporting website.) As he narrates, we see historical footage of several presidents discussing the Vietnam conflict publicly, along with historical footage of the war. But we also see, in montage, close-ups and zooms of Ellsberg at the time. Perhaps these scenes are acted, using an actor to play the young Ellsberg - the audience is not told. In montage, as the his-

torical footage runs, we are treated to close-ups of fingers - which we are to take as belonging to Ellsberg - running over the typed words of P. of the Pentagon Papers. We see "Ellsberg" opening volumes, closing them, shifting them, inspecting the documents, all in great close-up, and using various zooming techniques. Ellsberg the narrator is using the past + now construction: "I now saw that [President Lyndon] Johnson was continuing a pattern of presidential lying." We construct a viewpoint blend in which the viewpoint of the orchestration of the entire narrative is projected from the mental space with Ellsberg the narrator but the viewpoint on the experience of realizing what was happening is projected from the space of Ellsberg as he reads the Pentagon Papers for the first time. There is extraordinary emergent structure in this blend, including Ellsberg's ability to speak for his young self in a way that probably would not have been available to him at the time, and of course and enduring, manufactured, compressed character for "Ellsberg" the man: young Ellsberg and old Ellsberg are of course extremely different things, but the analogies between them, including analogies of viewpoint, can be compressed to a charac-terological unity in the blend.

Just in passing, we acknowledge that it is possible to deploy not only reinforcing or supplementary linguistic and audiovisual constructions but also conflicting constructions. This is a standard technique of humor and various forms of entertainment. It is also a technique of news broadcasts whose brand is based in an explicit and reliable political or ideological stamp. These shows routinely ridicule the opposition by providing footage of the opposition with accompanying audiovisual constructions designed to make them look stupid.

Novel Broadcast Constructions

Since new constructions arise by blending existing constructions and conceptual arrays, there is really never any construction that is truly "novel." Human cognition and expression is highly conservative, although what we focus on is often the emergent structure. Accordingly, when we discuss "novel broadcast constructions», we do not mean to indicate that they are not based on antecedent constructions. On the contrary, everything new human beings contrive is deeply based in established input spaces. Otherwise, it would be not novel but rather unintelligible.

The power of broadcast news to use so many modalities simultaneously, not as a combination but rather as a system that different disciplines have tried to approach as a linear sum, produces

a wealth of partly-novel broadcast constructions. Consider, for example, the extraordinary work of news broadcasts to talk about the future, as in weather forecasts. What is required in this case is amazing compressions over not only time, space, causation, and agency, but also possibility. Some of the audiovisual prompts for these compressions are spectacular but look utterly natural. For example, in weather reports about impending meteorological reports such as hurricanes, one can find the usual linguistic expressions, such as "likely", "perhaps", "threat", and so on, both in the speech of the weather forecaster and in the on-screen text. Graphics often accompany these verbal reports, and the graphics can be extremely sophisticated while striking viewers as entirely ordinary. It is common, for example, to display, using compression of time and space, the future of a hurricane. One sees representations on a map that we are to take as prompting for a conception of the path of the hurricane, and also the hurricane's size, but there is more: As the path moves into the future, there is an expanding width to the cone of incidence for the hurricane, not because the radius of the hurricane itself is expanding, but because our ignorance of the probable location of the hurricane is increasing. The conical graph on the map compresses not only time and space but also epistemic stance. We seem to have here a visual correlate of an evidential marker. (See website for an example).

Conclusion

The multimodal dimensions of human communication present a rich field of discovery and insight that extends existing linguistic theories into other modalities, providing new opportunities for systematically testing and validating intuitions. The different modalities of a communicative act are not simply redundant, but provide new aspects of meaning that a multimodal analysis can uncover. Nor is the construction of meaning across modalities mechanically additive; rather, meanings emerge as crossmodal blends that rapidly synthesize selected features of the information into new wholes. To understand human communication, we must develop a new facility for understanding the grammar of multimodal meaning construction.

The Red Hen project aims to integrate a multimodal data collection of global television news with data enhancement, multimodal data mining, and cognitive analysis of multimodal communicative practices. Television news represents a uniquely available dataset of common communicative practices, with the additional attraction that it has been deliberately crafted by professional

teams to grab attention, convey complex meanings in a rapid and compressed format, and persuade by implication. The way in which television succeeds in tapping into ancient interpretive structures in our cognitive system makes its detailed examination particularly revealing.

References

1. Cohen, Gerald. 1987. Syntactic Blends in English Parole. Frankfurt, Bern, New York: Peter Lang.

2. Coulson, Seana. 2013. "Cognitive neuroscience of creative language." This volume.

3. D^browska, Ewa. 2010. "Naive v. expert intuitions: an empirical study of acceptability judgments», The Linguistic Review 27 (2010), DOI 10.1515/tlir.2010.001

4. Dancygier, Barbara. 2012. The Language of Stories: A Cognitive Approach. Cambridge: Cambridge University Press.

5. Dancygier, Barbara & Eve Sweetser, editors. 2012. Viewpoint in Language: A Multimodal Perspective. Cambridge UK: Cambridge University Press.

6. Fauconnier, Gilles & Mark Turner. 2002. The Way We Think: Conceptual Blending and the Mind's Hidden Complexities. New York: Basic Books.

7. Fillmore, C. J. 2006. Articulating Lexicon and Constructicon. Presentation at the Fourth International Conference on Construction Grammar, Tokyo,Japan.

8. Fillmore, Charles J., Russell R. Lee-Goldman, Russell Rhodes. 2011. "The FrameNet Construction." In Boas, H.C. and Sag, I.A. (eds.) Sign-based Construction Grammar. Stanford: CSLI Publications.

9. Goffman, Erving. 1967. Interaction Ritual: Essays in Face-to-Face Behavior. Chicago: Aldine

10. Goldberg, Adele. 1996. "Construction Grammar." In Brown, E.K & J. E. Miller, editors, Concise Encyclopedia of Syntactic Theories. N.Y.: Pergamon.

11. Hofstadter, Douglas & David J. Moser. 1989. "To Err is Human; To Study Error-making is Cognitive Science." Michigan Quarterly Review, Volume 28, number 2, P. 185-215.

12. Israel, Michael. 1996. "The Way Constructions Grow." In Goldberg, Adele, editor, Conceptual Structure, Discourse and Language. Stanford: CSLI. P. 217-230.

13. Kay, Paul & Charles Fillmore. 1999. "Grammatical Constructions and Linguistic Gen-

eralizations: The What's X Doing Y? Construction." Language, Vol. 75, №. 1., P. 1-33.

14. Kendon, Adam. 1990. Conducting Interaction: Patterns of behavior in focused interactions. Cambridge: Cambridge University Press.

15. Nikiforidou, Kiki. 2012. "The constructional underpinnings of viewpoint blends: The Past + now in language and literature." In: B. Dancygier & E. Sweetser (eds.), Viewpoint and Perspective in Language and Gesture. Cambridge: Cambridge University Press. P. 177-197

16. Nikiforidou, Kiki. 2010. "Viewpoint and construction grammar: The case of past + now." Language and Literature 19(2) 265-284.

17. Tobin, Vera. 2008. "Literary Joint Attention: Social Cognition and the Puzzles of Modernism." (unpublished dissertation).

18. Tomasello, M. 1995. "Joint Attention as Social Cognition." In Moore, C. and Dunham, P. (Eds.) Joint Attention: Its Origins and Role in Development. Mahwah, NJ: Lawrence Erlbaum Associates.

19. Turner, Mark. 1991. Reading Minds: The Study of English in the Age of Cognitive Science. Princeton, New Jersey: Princeton University Press.

20. Turner, Mark. 1998. "Figure." In Cristina Cacciari, Ray Gibbs, Jr., Albert Katz, and Mark Turner, Figurative Language and Thought. N.-Y.: Oxford University Press, 1998.

21. Turner, Mark. 1987. Death is the Mother of Beauty: Mind, Metaphor, Criticism. Chicago: University of Chicago Press.

Вестник Челябинского государственного университета. 2014. № 7 (336). Филология. Искусствоведение. Вып. 89. С. 47-50.

Г. Г. Москальчук

АКТУАЛЬНЫЕ ВОПРОСЫ ФОРМООБРАЗОВАНИЯ ТЕКСТА

Рассматриваются пути синтеза различных текстов в макроструктуры на основе пространственных параметров. Намечены некоторые направления межуровневого описания языка, исходя из текста как единицы высшего уровня сложности. Представлена архитектоника системы «Текст ^ среда» на базе категорий симметрия/ асимметрия, типология текстов, построенная от их аттракторов. Рассматриваются нерешенные и актуальные вопросы теории формообразования (морфогенеза) текстовых структур.

Ключевые слова: текст, морфогенез текстовых структур, форма текста, аттрактор, среда, структура, симметрия, асимметрия, синтез, фрактальность.

В уровневой модели описания языка сложилась непростая терминологическая ситуация: единицы и единства каждого уровня описываются слабо пересекающейся терминологией, а переходы между выделяемыми уровнями изучаются лишь эпизодически. Тем не менее, коммуникативная, деривационная и просодическая природа языка нацеливает лингвистов на некоторое единообразие в описании, как единиц, так и единств языка не только в статике, но и в динамике, а также должным образом изучать процессы синтеза, осуществляющиеся в единицах на стыке различных уровней организации. Поскольку текст и дискурсы разного рода являются реальными полями информационно-семиотического и коммуникативно-информационного взаимодействия языков, лич-

ностей, сообществ, культур, то именно этот уровень определяет характер многих процессов, протекающих в структурах иных порядков организации.

Только текст может полагаться важнейшей единицей человеческого общения, концентрирования и хранения значимой информации. Поэтому синтез результатов разнообразных направлений изучения текста, обширных массивов сведений, накопленных на этом пути, качественно-количественных данных о текстах на различных языках, может быть осуществлен на основе инвариантных представлений о структурной организации текста как базовой единицы языка.

Текст обладает собственной геометрией квазипространства, распределенного в физиче-

i Надоели баннеры? Вы всегда можете отключить рекламу.