CC BY
13DOI: https://doi.org/10.46991/AFA2023.19.L083
DECODING MULTIMODAL TEXTS OF MEDIA DISCOURSE
Narine Harutyunyan*
Yerevan State University
In modern print and online media, the verbal text is increasingly being replaced by a multicode material, whose verbal component is illustrated, supplemented, and multiplied by the visual component. At the same time, the value and impact of the visual component increases immeasurably: vivid images are imprinted in the mind, making a powerful effect on readers and viewers. Since most media texts being multicode, special studies of such texts have become especially relevant today. Hence, based on the descriptive, analytical, non-verbal-culturological methods of research, this article discusses the role and relationship of visual and verbal texts, the advantages of the visual code as a carrier of implicit information on the covers of periodicals. The decoding of the multimodal text from The Economist magazine demonstrates clearly that, as a rule, the non-verbal component attracts more attention and inspires more confidence in the recipient. Because the non-verbal component is quite difficult for the addressee to track consciously, the additional manipulative potential of the metaphorical image is used in the text.
Keywords: multimodal text, media discourse, multicode text, clip culture, decoding, visual component, verbal component.
Introduction
At the turn of the 20-21st centuries, the Polish researcher M. Chervinsky (1981) noted that the flourishing of visual information, the development of new technical means of communication, the emergence of computer hypertexts that use pictures to categorize stored information allow specialists to talk more often about the "crisis of the verbal text" and its displacement in the 21st century by figurative visual messages, heralding the beginning of the post-literary era.
Today we are already living in a visual era when our consciousness is ready to work with the information expressed in the illustration without being
* narineharutyunyan@ysu.am Received: 05.03.2023
Revised: 24.03.2023
__Accepted: 15.04.2023
iicc^ CD © 1 This work is licensed under a Creative Commons
Attribution-NonCommercial 4.0 International License. © The Authors) 2023
embarrassed by the lack of a full-fledged verbal form. The dominance of visual content in the mass media is associated with the acceleration of the pace of life, the increase in the diversity and quantity of incoming information, the clipization of the thinking of the digital society. A feature of modern clip culture1 is the mosaic and fragmentation of the image, its brightness, relevance and short duration. Clip culture products include covers of The Economist, The New Yorker, The Spectator, The Week UK publications as a kind of modern multimodal texts that require the reader to decode meaning using a variety of cognitive mechanisms.
Nowadays, the management of a modern person behavior in society is carried out mainly through multimodal mass media texts and visualization of mental models. The above confirms the relevance and timeliness of the study. Despite the increased scientific interest of linguists in the analysis of the media, it should be noted that magazine covers have not been sufficiently studied in terms of their multimodality, which also determines the relevance of this article.
Exploring multimodal covers of The Economist.
Some magazines tend to preface the content of the issue with visual-verbal text on the cover. Their multi-modal nature is expressed in combining visual and verbal elements into complex, compelling messages which have a great impact on the mind of the reader. One of the clearest examples of such publications is the British weekly political and economic magazine The Economist.
On the one hand, its cover is highly inventive and achieves high expressiveness by presenting a complex form of announcement whose visualverbal rhetoric, in the form of a rebus, enhances the enjoyment of the text decoding process On the other hand, the verbal-figurative component of the cover realizes the function of a "visiting card", which provides an opportunity for preliminary acquaintance with the content of a particular issue of the journal.
Texts that connect verbal and visual codes are called multimodal (Voloskovich, 2011; Iriskhanova, 2012). It should be noted that the issue of the interaction of verbal and visual components in multimodal texts has not been fully resolved. Each of the researchers considers this issue from the standpoint of their theoretical orientation. However, one thing is clear that the presence of a non-verbal component is an essential attribute of a modern media text.
Media text, as a multi-level sign that combines different semiotic codes (verbal and non-verbal) that have a significant impact on the content of the media text, on the form of its reproduction and the level of its perception, is a vivid example of multimodality.
Visualization becomes an element of text formation and expansion of the semantic space of this text. It enables the author of the article to succinctly encrypt, veil information, and give the reader food for thought, entering into a preliminary dialogue with the author, the possibility of generating semantic associations, freedom of creative interpretation, etc. "The text of a newspaper/magazine message and the accompanying illustrative material form a single semantic space and perform a single pragmatic task, which allows us to speak about the implementation in one English-language media text of various forms of pragmatic influence on the recipient of information" (Eremina, 2007, p. 4).
It should be noted that the text of media discourse can be quite informative even in the presence of only a visual component because the human nature allows to quickly recognize images, understand the connections and patterns of a drawing. Based on this, George A. Miller coined the term "informavore" to describe modern people who collect and consume information (in Kulpinova, 2020).
Visual information is of greater interest to a person, as it operates with visually perceived images that contribute to the understanding of "deep" topics, and gives emotionality to the message. Due to the variety of intertextual connections in the media discourse, the formation of expressive content is carried out. In addition, visual information most often looks unusual, and according to the study of Tony Buzan (2014), it is the unusual information that is best remembered.
In modern media discourse, along with the traditional verbal metaphor, visual metaphor is used as a more complex type of coding meaning. Illustrated magazine covers provide an opportunity to visualize the impossible; anticipate the perception of certain topics and events (in particular, those related to politics and social problems); predict the near future and influence the process of perception of certain events, facts and ideas through visual metaphor.
The visual message is created in such a way that it implies the variability of interpretation and the complexity of the decoding process, and invites the reader to immerse himself in the work of a "detective" and unravel the "tangle" of metaphors and other devices of metaphorization.
Figure 1
The texts of The Economist magazine are self-sufficient in the presence of only a figurative code, as a rule, built on metaphorization. In the above figure (Figure 1) of this conspiracy cover of The Economist (January 24, 2019), the neologism Slowbalisation (slow + globalisation) is used. It describes a phenomenon that followed the global financial crisis, characterized by a prolonged slowdown in trade reform and waning political support for open trade amid growing geopolitical tensions. In the decade and a half following the global financial crisis, the era of globalisation has given way to an era of sluggish Slowbalisation. To enhance the reader's perception of this new social and economic phenomenon, the verbal metaphor is correlated to the visual one, which, in this case, is the snail. As we know, the snail (Figure 1) is distinguished by a very slow speed of movement, and this image of a snail with a globe instead of a shell, is a metaphorical transfer from an animal (mollusk) to a slow process of development of the world economy, and in particular, world trade, which seems to freeze. As a clarifying component, the subtitle The future of global commerce is used. I would like to note that the magazine with this cover was published in January 2019 (before the coronavirus pandemic), but the forecast turned out to be quite accurate.
The Economist
Amarlia't next mining boom Canada'* ataault on trmt »pe«th Who win« from a wafaprKa ipirnl? A Stanhlp It twrn
Putin's botched job
War or not, ha hat miscalculated ft
Figure 2
The next cover from The Economist (February 17, 2022) depicted in Fig. 2 is an example of the use of an image in a symbolic function. An important function of the symbolic and semantic highlighting of the verbal component is the choice of the color. Quite often, the coloring of certain parts of the cover in the appropriate colors serves as a hidden allusion to certain political forces and government leaders, and is associated with universal color symbols. It is the red color that plays a special role in this case - it causes an instant association with the blood shed by soldiers and civilians. In the picture we see Putin, who is standing in the corner in thought, and under his feet the map of Ukraine is spread. Here, in the literal sense, the phraseological unit drive yourself into a corner (put yourself in a difficult situation) is played out. The use of red color is involved in the implementation of the author's artistic intention, in creating an image and influencing the reader's aesthetic feelings. The created pictorial metaphor symbolizes the beginning of the "bloody" war and reinforces the verbal assessment of the military operation as a failure (see the title Putin's botched job). At the same time, Putin stands in the center of the composition, on the map of Ukraine as the main "designer" and leader of the war scenario. The red color creates an associative link with the component war at the epicenter of which is the President.
The title of the article Putin's botched job, as well as the subtitle War or not, he was miscalculated, makes it clear that the Russian President is the main "loser" who has put everything on the line. A conscious reminiscence (reference) to Shakespeare's Hamlet and his famous phrase to be or not to be, creates a certain context for the perception of this article.
Correct decoding of the meaning generated by the combination of image and text requires the reader not only to understand the literal meaning of the text, but also to comprehend the extralinguistic component.
However, the most interesting combinations of the textual and illustrative parts, which create a kind of semantic tension, are obtained by the striving to use the function of expanding the semantic space of the media text, when "the visual component of the media text contains information that is absent in the verbal part of the text, thereby contributing to a deeper understanding of the author's main idea" (Eremina, 2007, p. 22).
Figure 3
In the edition of The Economist for December 24, 2022 (Figure 3), under the title Holiday double issue, a Christmas tree is presented. Through the image of the tree several functions are realized: attractive (providing visual perception of the media text), expressive (for expressing the feelings of the sender and influencing the emotions of the addressee), informative (the purpose of which is to convey certain information). In this example, the visual
component acts as the main carrier of the aesthetic and content-conceptual information. The Christmas tree is the key element of the composition as a symbol of the New Year. Instead of Christmas decorations, various symbols are depicted, the function of which is to visually express the key concepts and ideas of the coming 2023. Some of the visual symbols suggest many single interpretations. For example, at the top of the tree we see the sign of the atom, which can be interpreted both as the super importance of nuclear energy in a period of high energy prices, and as an aggravation of the nuclear crisis, in particular around Iran, and as a warning about a possible nuclear catastrophe. The depiction of a human head also evokes a number of interpretations. For example, it can symbolize artificial intelligence, human chipization, the development of bioengineering and the replacement of parts of the human brain. In the picture, we can also see a stomach, instead of the brain, which may suggest a world food shortage, which will force people to think with their stomachs and do various radical things. The image of the skull is an allusion to the death of mankind from hunger. A cricket-like symbol crossed with a baseball bat can be interpreted as mutual interests of Great Britain (cricket) and the United States of America (baseball).
The visual metaphor of a man and a woman leaving can mean the departure of the traditional institution of the family and family values into the past. An alternative interpretation of these figures arises upon careful consideration of the figure. Both have their hands in their pockets, and their heads seem to have been pierced. If we correlate the visual component and the verbal text (Lessons of inflations past) located directly next to it, then there is an association with inflation and financial difficulties.
Other symbolic images are conventional and limited in their interpretation. For example, symbols of a new deadly virus, radioactive danger, Chinese poetry, a soccer ball, etc. In the floral ornament at the bottom of the picture, we see the symbolism of both the East (stars and crescent moon) and Europe (Christmas candles, Christmas holly).
According to researchers, visually perceived information causes more confidence in the addressee (Better to see something once than to hear about it a thousand time). So, as claimed by L. Wojtasik (1981, pp. 198-199): "What we see is faster, easier accepted as truth and causes less fear. This refers to "pictures" in a fairly broad sense of the word. Visual agitation can convey some information more convincingly than oral agitation." In the example under consideration, the verbal and visual parts of the text are characterized by
relative autonomy since they have semantic independence and can be decoded independently of each other. The verbal part implies the variability of interpretation and a certain complexity of the decoding process. Based on the world events of recent months, we offer our vision of verbal information encoded in a multimodal text:
A Chinese city in Europe - Chinese cities want to use tracking methods, giving the world an example of building a digital totalitarian state.
Lessons of inflations past - The resumption of inflation in the United States raises political concerns and encourages turning to history to learn lessons and find ways out of the impasse.
The new tech world view - Developments in areas such as artificial intelligence and machine learning, robotics, nanotechnology, 3D printing, genetics and biotechnology form a new type of person and a new mentality.
Copyright loses its grip - Copyright affects everything from what entertainment we watch to which of our devices we can repair. Almost everyone on the Internet has seen some version of the "this media is no longer available due to a copyright complaint" notice. This is why we must fight for copyright law.
The Amish in Indonesia - promotes unity with nature, eco-activism, a new type of "segregation" of people (e.g. the 15-minute city concept).
The Haj's future - Keeping pilgrims safe will become more difficult and expensive as the world gets hotter.
The author of the illustration of the December 24, 2022 edition fills his work with important implicative meaning and prediction with the help of a biblical allusion. This allusion refers to a passage from the Book of Genesis:
Then Pharaoh said to Joseph, "In my dream I was standing on the bank of the Nile. Seven cows, shimmering with health, came up out of the river and grazed on the marsh grass. On their heels seven more cows, all skin and bones, came up. I've never seen uglier cows anywhere in Egypt. Then the seven skinny, ugly cows ate up the first seven healthy cows. But you couldn't tell by looking - after eating them up they were just as skinny and ugly as before. Then I woke up. "In my second dream I saw seven ears of grain, full-bodied and lush, growing out of a single stalk, and right behind them, seven other ears, shriveled, thin, and dried out by the east
wind. And the thin ears swallowed up the full ears. I've told all this to the magicians but they can't figure it out." Joseph said to Pharaoh, "Pharaoh's two dreams both mean the same thing. God is telling Pharaoh what he is going to do. The seven healthy cows are seven years and the seven healthy ears of grain are seven years - they're the same dream. The seven sick and ugly cows that followed them up are seven years and the seven scrawny ears of grain dried out by the east wind are the same - seven years of famine. "The meaning is what I said earlier: God is letting Pharaoh in on what he is going to do. Seven years of plenty are on their way throughout Egypt. But on their heels will come seven years of famine, leaving no trace of the Egyptian plenty. As the country is emptied by famine, there won't be even a scrap left of the previous plenty - the famine will be total. The fact that Pharaoh dreamed the same dream twice emphasizes God's determination to do this and do it soon. (GEN.41:17-31)
The drawing is built on the contrast of the left and right, red and green halves. Almost all the symbols of the image are a reference to the dream of the pharaoh. On the left side of the picture is a well-fed cow (a symbol of abundance), and on the right is a thin, bony dog (a symbol of scarcity). On the left - an apple, a goblet and an olive branch (symbols of abundance), on the right - a dried branch and a bird looking for food (symbols of scarcity). Consequently, the main implicit, metaphorical meaning of the image is a warning about world hunger. Seven candles on the Christmas tree (three on the left, three on the right and the top of the tree) correlate the biblical allusion seven sick and ugly cows - seven years of famine, with the UN report: "The world is not on track to achieve Zero Hunger by 2030. If recent trends continue, the number of people affected by hunger would surpass 840 million by 2030. According to the World Food Programme, 135 million suffer from acute hunger largely due to man-made conflicts, climate change and economic downturns" (United Nations Sustainable Development Goals, Goal 2: Zero Hunger, 2021).
Conclusion
On the example of the illustrated covers of The Economist magazine, we see that the image can be not only auxiliary, but also the leading link in relation to the verbal text. The visual component is an important element of the media text, a dominant, independent material, as well as an integral part of the corporate identity of a quality press. The use of multimodal texts allows us to attract attention and keep the interest of a larger audience. Unusual words, pictures, symbols, allusive images and quotations are remembered better than their typical counterparts. Visualization that conveys the content of the message in a non-textual form is an independent journalistic work. Summing up, it can be noted as an obvious fact that the use of expressive visual means in a media text not only gives it a visual sensitive visibility, but is also an inexhaustible source of its information saturation.
Notes
1. Clip culture refers to the culture of technologically advanced regions of the world, in which the media largely determine the existence of cultural subjects and, most importantly, the associated ways of representing information content in the media. Romanov N. A. Klipovaja kul'tura v sovremennom mediaprostranstve. Chelovek. Kul'tura. Obrazovanie. [Clip culture in modern media space. Human. Culture. Education]. 3 (25), 2017. Retrieved March 11, 2023.
References
B'yuzen, T. (2014). Nauchite sebya dumat (Gerasimchik Yu.I. trans.) [Teach
yourself to think]. Minsk: Poppuri. (in Russian) Chervinskij, M., & Zberskij T. (1981). Sistema knigi. Semiotika knigi. [Book
system. Semiotics of the book]. Moscow: Kniga. (in Russian) Eremina, E.A. (2007). Mnozhestvennost' form pragmaticheskogo vozdeystviya angloyazychnogo mediadiskursa (Thesis Abstract) [Plurality of forms of pragmatic influence of the English-language media discourse]. Retrieved March 9, 2023. (in Russian). Genesis 41:17-31. In YouVersion Retrieved March 11, 2023. Irisxanova, O. K. (2012). Semiotika plakata v sociokognitivnom osveshhenii: lejboristy', kapitalisty' i uglerodny'e sledy'. [Poster Semiotics in Sociocognitive Illumination: Labourists, Capitalists and Carbon
Footprints]. Aktualxny"e problemy" angliiskogo yazy"koznaniya: sbornik nauchny x statei. Moskva: MAKS Press. (in Russian)
KuFpinova, E.V. Vizualizaciya informacii v obrazovatelnom prostranstve. [Visualization of information in the educational space]. Retrieved March 9, 2023. (in Russian)
University of the West of Scotland, UNITED NATIONS SUSTAINABLE DEVELOPMENT GOALS, GOAL 2: ZERO HUNGER. Retrieved March 11, 2023
Voloskovich, A. M. (2011). Ikonotekst kak raznovidnosf polimodal "noi gibridizacii. [Iconotext as a kind of polymodal hybridization]. Vestnik Moskovskogo Gosudarstvennogo Lingvisticheskogo Universiteta, 5, (611), 25-35. (in Russian)
Wojtasik, L. (1981). Psixologiya politicheskoj propagandy. [Psychology of political propaganda]. Moscow: Progress. (in Russian)
Sources of Data
Holiday double issue. (2022, December 24). In The Economist. Retrieved March 6, 2023
Putin's botched job. (2022, February 17). In The Economist. Retrieved March 6, 2023
Slowbalisation: The future of global commerce. (2019, January 24). In The Economist. Retrieved March 07, 2023.
T^u^nhrufr puaummusFu Sb£USfc№ U^U^n^U^nrnhUC
Lmp^Ut ¿mpnip]ni.UjmU
tfmtfmUm^m^g mqmq^p U mngmUg ipmrn^mtf^ngUtpnttf pmUm^np mtpump qUmjn^ ^n^mp^U^nLtf t pmqtfm^nqmj^U mtpumtpn^: ^pm pmUm^np pmqmqp^p ^mrn^tp^n^tf, jprng^nLtf U mnm^t^ mqqtg^ t qmnUnttf mtunqm^mU pmqmqp^^ mn^mjntpjmtfp: U^mdmtfmUm^ st-unqm^mU pmqmqp^^ mpdtpp ¿m^mqmUg tft&mUnttf t: Hmn ^mm^tp-Utpp tU q^mm^gntpjmU hqnp mqqtgntpjntU pnqUtjn^
pUptpgnq^ ^mtf q^mnq^ ^pm: ZtmUmpmp, pmqtfm^nqmj^U mtpumtp^ nLunL^Umu^pnLpjntUp mpq^m^mU t U mUhpmdt2s: S^jm^ hnq^m&nttf gUUmp^^n^J tU mtunqm^mU U pmUm^np mtpumtp^ qtpU ni. ^n^hm-
pmptpntpjntUp, ^qntmL Inrfr mnm^tinLpjnLUUtpp, tpp qpmUp np^tu mUnLrrmlfr mtqt^mm^nLpjmU IpnrUtp ^mmltp^nLtf tU ^mpptpm-ImUUtpfr Imqltpfr ^pm: PpfrmmUm^mU The Economist mlumqpfr lmq-itpfr ^pm Utp^mjmg^nq pmq^m^nqmjfrU mtpumtpfr ^tpinL^ntpjntUp Immmp^t^ t U^mpmqpm^mU, ^tpinL&m^mU, n^ ^nupmjfrU-^mlnLpm-pmUm^mU ItpnrUtpfr IfrpmnJmJp:
Pmhmj]i pmnhp' pxuqiSxul}ni}xujfrU intpuw, iiti}]iw qjiuljiupu, litrfrw intpuw, hfiiqwjfrU ii2wlniJP, wiqmlinrimi[npnLii, mtunqmljmU pmrpu-ipji^, pwUwrfnp pwrwrpfe:
