CC BY
32
_LINGUISTICS AND LANGUAGES (UDC 81)_
DOI: https://doi.org/10.18454/RULB.2020.23.3.6
МУЛЬТИМОДАЛЬНЫЙ МЕДИАТЕКСТ: МЕТОДИКИ АНАЛИЗА
Научная статья
Пром Н.А.*
ORCID: 0000-0003-2711-4916, Волгоградский государственный технический университет, Волгоград, Россия
* Корреспондирующий автор (natalyprom77[at]mail.ru)
Аннотация
Рассматривается мультимодальная коммуникация, которая учитывает информацию, передаваемую через несколько каналов восприятия, и реализуется через ряд параллельных каналов. Исследование ориентировано на медиадискурс, ресурсы которого - материалы и инструменты- используются для производства интегрированного текста. Актуальность работы состоит в том, что дальнейшее развитие технологий создает такие условия, которые расширят возможности мультимодальности и, следовательно, повысят релевантность лингвистических исследований в этой области. Цель работы мы видим в том, чтобы изучить лингвистическую литературу на предмет аналитических моделей мультимодального медиатекста. Исследование установило, что в лингвистике предлагаются разные интегративные модели представления мультимедийного текста и дискурса. Наиболее точным, на наш взгляд, является многоуровневый анализ режимов, где структурные, семантические и стилистические свойства извлекаются и представляются для каждого режима отдельно, а также во взаимодействии со слоями других режимов. Перспективу нашего исследования мы видим в разработке интегрированной модели анализа мультимодального медиатекста, гармонично учитывающей различные подходы.
Ключевые слова: мультимодальность, медиатекст, мультимодальный анализ, коммуникация.
MULTIMODAL MEDIATEXT: ANALYSIS PROCEDURES
Research article
Prom N.A.*
ORCID: 0000-0003-2711-4916, Volgograd State Technical University, Volgograd, Russia
* Corresponding author (natalyprom77[at]mail.ru)
Abstract
The article deals with multimodal communication that considers information provided through several parallel sensory canals. The investigation takes into account the specifics of media discourse, since its resources—materials and tools—are used to produce an integrated text. The relevance of the work is that further development of technologies will create conditions that will expand the capabilities of multimodality and, consequently, foreground linguistic research in this area. The purpose of the work was to overview linguistic literature with respect to analytical models of a multimodal media text. The study has found that in linguistics, different integrative models for analyzing a multimedia text were developed. The most accurate multimodal description, in our opinion, is ensured by a multi-level analysis that extracts and presents structural, semantic, and stylistic properties of each text level separately and in interaction. The prospect of our work is seen in developing an integrated model for analyzinga multimodal media text, which harmoniously takes into account various approaches.
Keywords: multimodality, media text, multimodal analysis, communication.
Introduction
At the end of the last century, linguists realized that language is neither the only, nor even the dominant semiotic system [9, P. 113]. In natural communication, a multimodal system is formed around the key verbal component accompanied with non-verbal components—para- and extra-verbal ones. Paralinguistic components of non-verbal communicationinclude loudness, tempo, rhythm, and pitch of tone and determine the voice quality, its range, and tonality. Extralinguistic components are atypical individual pronunciation features, i.e. speech pauses, laughter, coughing, sighing, crying, stuttering, etc. All of them are full of visual and acoustic keys that fall out of focus in a classical linguistic research.
Communicativists drew their attention to the phenomenon of multimodality a short time ago, but it has already acquired a wide range of interpretations. The literature overview revealed a common understanding that various semiotic resources or modes are used in human communication to express meaning, including values, ideology, and social roles [5, P. 452-453]. According to Schmitz, multimodal communication involves several feelings and is implemented through a number of parallel channels. In order to perform such actions and produce multimodal texts, various components that are integrated into a syntactic, semantic, and functional whole are required [8, P. 24-42]. At the same time, media possess resources—materials and tools—that are used to produce an integrated text.
The basis of the cognitive-linguistic understanding of multimodality involves the term "modality"that is adopted in psychology, neurophysiology, and computer scienceand means a type of external stimulus perceived by one of the human senses, primarily vision and hearing. Studies of the multimodal discourse focus on how people simultaneously engage several communication channels, contributing to the general sense, and how information is shared between them [3, P. 135]. Therefore, if modern analysis is aimed at interpreting the meaning of a text or a communicative phenomenon, it can no longer rely mainly on the verbal component, since the latter explains only the use of speech or its individual aspects. To ignore other
modesimplies to miss a lot of potential meanings of media texts [5, P. 457], [6, P. 14]. Being limited in capabilities, channels not only additively participate in the transmission of information, but are interdependent and interact with each other.Information is distributed between them in a complex way [3, P. 148].
It is the spontaneous discourse when a recipient gets information in its multimodality that many linguists consider to be true communication. Its analysis adheres to the postulate of the equivalence of all modality levels and the procedure for the sequential study of each individual level [2, P. 27]. Modern research of multimodalityconducted in strict accordance with the principles of cognitive psychology and linguistics studies in detail each freeze frame of the analyzed video text.
Resultsand Discussion
In linguistics, different models for analyzing a multimedia text and discourse were developed. In this interdisciplinary field, research methods are clearly integrative. Discourse analysis includes such components as
1) source data (text name, source, author, and creation date);
2) mediatopic;
3) linguistic characteristics, i.e. vocabulary, phraseology, morphology, syntax, and stylistics (language game and rhetorical figures);
4) media technologies, first of all, visual means (location of material, font, graphics, and images), sound means, and video support;
5) communicative strategy that a specific media text implements and its communicative tactics, namely, the purpose of the media text, communicative ways to achieve it, addressant, and addressee of the media text; and
6) selection of facts, relationship between them, precedence, and hypertext [4, P. 26].
We can see that the discourse analysis scheme of a media text takes into account multimodality, however, is not attached with priority importance.
The media linguistic analysis pays more attention to multimodality, reveals the patterns of interaction between the verbal and media aspects and studies media signs and combinations of elements of all media text levels, i.e."word - sound - image," "word - graphic design - image," etc. Functions of the components of the media text—illustrations, complements, amplification, contrast, and clash of meanings—are also taken into account [1, P. 58]. The main function of photography is the evidentiality. For example,
(Photo)Afer last night's meeting Downing Street released a picture of a stern-faced prime minister not looking at the prince. The Saudis released footage of Mrs May greeting him, apparently cordially, with a handshake. In reference to Khashoggi's murder, Downing Street issued a statement saying that Mrs May had told the prince that Saudi Arabia needed "to build confidence that such a deplorable incident could not happen again " [10].
The text refers to the inconsistency between the information provided by the British official media (Downing Street)and news in the Saudi Arabian media. After the assassination of a journalist at the Saudi embassy, the official position of the British prime minister was disapproving the leadership of this country, and the government-controlled media considered it necessary to publish a picture of a stern-faced prime minister not looking at the prince at the G20 summit in Buenos Aires. But The Times presented a (Photo) that contradictedthe cold attitude to the prince. The photo, where the prime minister was greeting him[Mohammed bin Salman - N.P.], apparently cordially, had beenpublished earlier by the Saudi media,reporting that theprince was only slightly "upbraided." Thus, the author of the article used a photo as a multimodal tool to evidence a conflict between real attitude and political necessity, with the latter being always more expedient.
Multimodal research studies need to understand, how semiotic modes work together in a multimodal text, where the verbal text coexists with visual, musical, and other elements that are not the sum of individual semiotic codes, but aorganic unity and itsindividual parts contribute to a new text quality [7, P. 381]. When analyzing the text, four main points should have respect to, i.e.
1) language is always combined with non-linguistic resources to create meaning;
2) in a multimodal text, each mode can perform various communicative functions, depending on its capabilities;
3) people create meaning due to choosing and combiningavailable resources;
4) multimodal meaning is formed with interests of people as social actors [6].
In other words, accurate is a studythat considers a multilevel description of modes, where structural, semantic, and stylistic properties are extracted and presented for each mode separately and in interaction.
The study of multimodality addresses the problem of how modes in a mixed media text influence its interpretation. Siefkes identified three layers, or levels, of a text, namely,
1) a layer of form as the material organization that includes properties of a structure or arrangement without taking into account possible meanings, as well as the syntactic properties of the language;
2) a layer of semantics, implying discourse relations as an aspect of the rhetorical structurethat contributes to the cohesion of the text; and
3) a layer of style, which is a tool necessary to explain non-verbal interaction in various cultural and media contexts [9, P. 120].
We can see, that this model focuses on multimodal layers, their semantic potential, and socio-cultural context and should be revealedby linguistic, and, to a greater extent, cognitive-psychological and sociological procedures.
Conclusion
Thus, multimodality in media, especially network media, attracts attention of many researchers. The discourse analysis scheme of a media text takes into account multimodality that, however, is not attached with priority importance. The medialinguistic analysis pays more attention to multimodality, reveals the patterns of interaction between the verbal and media aspects and studies media signs and combinations of elements of all levels in a media text. A more accurate description, in our
opinion, is ensured by a multi-level analysis that extracts and presents structural, semantic, and stylistic properties of each text level separately and in interaction.
Researchers focus on such phenomena as log visualization—combined verbal and graphic elements that appear on the screen and participate in creating a television environment, context, and inscription; suggestive and counter-suggestive behavior of multimodal interaction participants as communicative behavior that facilitates the efficient manipulation of recipient's decision-making, on the one hand, and prevents such an impact, on the other; musical quote in the structure of a media text—reproduction of a musical fragment in a finished multimodal media text; the prosody of media speech in convergent media, etc. Further development of technologies undoubtedly will create conditions that will expand the capabilities of multimodality and, consequently, foreground linguistic research in this area. The prospect of our work is seen in developing an integrated model for analyzing a multimodal media text, which harmoniously takes into account various approaches.
Конфликт интересов Conflict of Interest
Не указан. None declared.
Список литературы / References
1. Добросклонская Т. Г. Медиалингвистика: системный подход к изучению языка СМИ: современная английская медиаречь / Т. Г. Добросклонская. - М.: ФЛИНТА, 2014. - 264 с.
2. Егорченкова Н. Б. Стратегический потенциал мультимодальной интеракции в медиа-политическом дискурсе / Н.Б. Егорченкова // Вестник Томского государственного университета. Филология. - 2014. - № 5 (31). - C. 24-37.
3. Кибрик А. А. Мультимодальная лингвистика / А. А. Кибрик // Когнитивные исследования IV. - М.: ИП РАН, 2010. - С. 134-152.
4. Кудрявцева Л. А. Дискурс-анализ медиатекста / Л. А. Кудрявцева, К. А. Иванова // Русский язык, литература, культура в школе и вузе. - 2014. - № 4(58). - С. 22-26.
5. Adami E. Multimodality / E. Adami // Oxford Handbook of Language and Society / O. Garcia(ed.). - Oxford: Oxford University Press, 2017. - Pp. 452-471.
6. Jewitt C. The Routledge Handbook of Multimodal Analysis/ C.Jewitt. - London: Routledge. 2009. - Pp. 14-27.
7. Krauz M. Zatrzymacuwag^widza. J^zykoweipozaj^zykowesrodkiperswazji w telewizyjnychprogramachinformacyjnych / M. Krauz // Medialinguistics. - 2018. - No. 5 (3). - Pp. 380-393. https://doi. org/10.21638/spbu22.2018.311
8. Schmitz U. Sehflächenforschung / U. Schmitz // Bildlinguistik. Theorien - Methoden - Fallbeispiele / H. Diekmannshenke (ed.). - Berlin, 2011. - Pp. 23-42.
9. Siefkes М. How Semiotic Modes Work Together in Multimodal Texts: Defining and Representing Intermodal Relations / М. Siefkes // 10plus1: Living Linguistics. - 2015. - Is. 1. Media Linguistics. - Pp. 113-131.
10. The New York Times [Электронный ресурс]. - URL: https://www.thetimes.co.uk/article/theresa-may-welcomes-mohammed-bin-salman-at-the-g20-conference-l0220vqwp (дата обращения: 02.21.2020)
Список литературы на английском / References in English
1. Dobrosklonskaya T.G. Medialingvistika: sistemnyj podhod k izucheniyu yazyka SMI: sovremennaya anglijskaya mediarech' [Medialinguistics: a systematic approach to the study of the language of the media: modern English media speech] / T. G. Dobrosklonskaya.- Moscow: FLINTA, 2014. - 264 p. [in Russian]
2. Egorchenkova N.B. Strategicheskij potencial mul'timodal'noj interakcii v media-politicheskom diskurse [Strategic potential of multimodal interaction in media-political discourse] / N. B. Egorchenkova // Tomsk State University Journal of Philology. - 2014. - No. 5 (31). - Pp. 24-37. [in Russian]
3. Kibrik A.A. Mul'timodal'naya lingvistika [Multimodal Linguistics] / A.A. Kibrik // Cognitive Research IV. - Moscow: IP RAS, 2010. - Pp. 134-152. [in Russian]
4. Kudryavtseva L.A. Diskurs-analiz mediateksta [Discourse analysis of a media text] / L.A. Kudryavtseva, K. A. Ivanova // Russian language, literature, culture at school and university. - 2014. - No. 4 (58). - Pp. 22-26. [in Russian]
5. Adami E. Multimodality / E. Adami // Oxford Handbook of Language and Society / O. Garcia(ed.). - Oxford: Oxford University Press, 2017. - Pp. 452-471.
6. Jewitt C. The Routledge Handbook of Multimodal Analysis/ C.Jewitt. - London: Routledge. 2009. - Pp. 14-27.
7. Krauz M. Zatrzymacuwag^widza. J^zykoweipozaj^zykowesrodkiperswazji w telewizyjnychprogramachinformacyjnych [Linguistic and non-linguistic persuasion agents in TV news programs] / M. Krauz // Medialinguistics. - 2018. - No. 5 (3). -Pp. 380-393. https://doi. org/10.21638/spbu22.2018.311
8. Schmitz U. Sehflächenforschung / U. Schmitz // Bildlinguistik. Theorien - Methoden - Fallbeispiele / H. Diekmannshenke (ed.). - Berlin, 2011. - Pp. 23-42.
9. Siefkes М. How Semiotic Modes Work Together in Multimodal Texts: Defining and Representing Intermodal Relations / М. Siefkes // 10plus1: Living Linguistics. - 2015. - Is. 1. Media Linguistics. - Pp. 113-131.
10. The New York Times [Electronic resource]. - URL: https://www.thetimes.co.uk/article/theresa-may-welcomes-mohammed-bin-salman-at-the-g20-conference-l0220vqwp (accessed date: 02.21.2019)
