CC BY
15
бш
МЕЖКУЛЬТУРНАЯ КОММУНИКАЦИЯ И СОПОСТАВИТЕЛЬНОЕ ИЗУЧЕНИЕ ЯЗЫКОВ
DOI: https:// doi.org/ 10.15688/jvolsu2.2022.6.9
UDC 81'42 LBC 81.055.1
Submitted: 13.04.2022 Accepted: 20.06.2022
THE ART OF STORYTELLING IN THE DIGITAL AGE: A MULTIMODAL PERSPECTIVE
Zinaida M. Chemodurova
Herzen State Pedagogical University of Russia, Saint Petersburg, Russia
Abstract. Using Dan Brown's Wild Symphony, Craig Silvey's The Amber Amulet, and Jon McGregor's In Winter the Sky as its case studies, this article attempts to explore a number of mechanisms and strategies contributing to the multimodal fiction construction and posing challenges for modern readers of all ages. The article proposes to examine multimodal resonance as a cognitive mechanism enhancing the emotional impact of multimodal stories on recipients and triggering the receptive mechanism of narrative empathy. Various semiotic resources, both free resources, such as photos, maps, and illustrations, and bound ones, including font types and typographical experiments, are analyzed from the standpoint of their role in fictional world creation. The article makes an original contribution to the Multimodal Stylistics Studies by hypothesizing the relevance of identifying two distinctive mechanisms of foregrounding: visual foregrounding as a range of formal ways to construct a multimodal text that focuses the reader's attention with the help of free semiotic resources, and graphic foregrounding, which has bound semiotic resources at its heart. The findings of the research prove that it is the combination of elements of stylistic, cognitive and linguo-semiotic analyses that might provide a new and effective methodology for interpreting a multimodal literary text. The overall outcome testifies to the importance of further research into varied mechanisms of foregrounding as a complex phenomenon essential for our better understanding of the embodied character of reading.
Key words: multimodal fiction, multimodal resonance, semiotic resources, foregrounding, narrative empathy, ludic modality.
Citation. Chemodurova Z.M. The Art of Storytelling in the Digital Age: A Multimodal Perspective. Vestnik Volgogradskogo gosudarstvennogo universiteta. Seriya 2. Yazykoznanie [Science Journal of Volgograd State University. Linguistics], 2022, vol. 21, no. 6, pp. 110-120. DOI: https://doi.org/10.15688/jvolsu2.2022.6.9
^ УДК 81'42 § ББК 81.055.1
Дата поступления статьи: 13.04.2022 Дата принятия статьи: 20.06.2022
(N
ИСКУССТВО ПОВЕСТВОВАНИЯ В ЦИФРОВУЮ ЭПОХУ: МУЛЬТИМОДАЛЬНЫЙ АСПЕКТ
Зинаида Марковна Чемодурова
Российский государственный педагогический университет им. А.И. Герцена, г. Санкт-Петербург, Россия
©
Аннотация. Статья посвящена изучению механизмов и стратегий, значимых для процессов создания мультимодальных художественных текстов и представляющих собой несомненные трудности для читателей разных возрастов. В качестве когнитивного механизма, благодаря которому усиливается эмоциональное воздействие таких текстов на читателей и запускается рецептивный механизм нарративной эмпатии, определен мультимодальный резонанс. Источниками типичных случаев создания мультимодального повествования послужили «Звериная симфония» Дэна Брауна, «Янтарный амулет» Крейга Сильви и «Небо зимой» Джона Макгрегора. Охарактеризованы разнообразные семиотические ресурсы, как свободные (фотографии, карты, иллюстрации), так и связанные (виды шрифтов и типографические эксперименты), в аспекте их функций в создании фикционального текста. Доказана авторская гипотеза об актуальности выделения двух самостоятельных механизмов выдвижения: визуального выдвижения как совокупности способов формальной организации мультимодального текста, внимание читателей в котором привлекается с помощью свободных семиотических ресурсов, и графического выдвижения, в основе которого лежат связанные семиотические ресурсы. Полученные выводы подтверждают эффективность методологии, предполагающей использование элементов стилистического, когнитивного и лингвосемиотического анализа при интерпретации мультимодального художественного текста. Результаты исследования свидетельствуют о важности дальнейшего изучения разнообразных механизмов выдвижения как сложного феномена, играющего существенную роль в понимании лингвистами воплощенного характера процесса чтения.
Ключевые слова: мультимодальная литература, мультимодальный резонанс, семиотический ресурс, выдвижение, нарративная эмпатия, игровая модальность.
Цитирование. Чемодурова З. М. Искусство повествования в цифровую эпоху: мультимодальный аспект // Вестник Волгоградского государственного университета. Серия 2, Языкознание. - 2022. - Т. 21, №2 6. -С. 110-120. - (На англ. яз.). - DOI: https://doi.Org/10.15688/jvolsu2.2022.6.9
Introduction
Since the end of the 20th century the scholars in various fields of linguistics have claimed practically unanimously that the "multimodal turn" [Jewitt (ed.), 2009] in stylistics, cognitive linguistics, social semiotics has entailed an increased relevance of elaborating a new methodology of reading contemporary fiction. The emergence of the so-called multimodal fiction, defined by A. Gibbons as "a body of literary texts that feature a multitude of semiotic modes in the communication and progression oftheir narratives" [Gibbons, 2012, p. 420], poses new challenges for both students of philology interested in text interpretation and stylistic analysis and readers of modern fiction, while they are trying to come to terms with the writers' use of unusual page designs, experimental typography and innovative strategies of constructing multimodal narratives.
M.-L. Ryan, developing the concept of "transmedial narratology", argues in favour of the interdisciplinary approach to narrative stating that "the diversity of games that narrative can play with the resources of its medium is one of the many reasons, that make the intersection of narratology and media studies, an area still largely unexplored, into a productive field of investigation" [Ryan, 2004, p. 13].
Multimodal Studies, focusing, among various problems, on the analysis of multimodal storytelling, might be viewed as a very productive field of investigation, indeed, for several reasons. First, a remarkable advance in digital technologies led to a sharp spike in the publication of diverse subgenres of multimodal literary texts, encompassing books for very young readers, teenage novels and multimodal fiction. The statistical survey of the hybrid texts (i.e. novels combining images and texts) published from 1922 to 2012, carried out by J. Luke as part of his PhD research, testifies to the fact that there has been "a considerable increase in hybrid texts over time, particularly since the year 2000" [Luke, 2013, p. 20], with more than half of such works published in the new millennium, with the onset of the Digital Age when the publication of print-based multimodal books became more affordable financially.
Second, our intuitive exposure, as readers, to multimodal texts, which are in no way a recent invention, if we think about the seminal novel by Lawrence Sterne "The Life and Opinions of Tristram Shandy, Gentleman" with its remarkable "marble page" and unique typography and consider the concrete prose created by postmodernists in the 60-80s of the 20th century, cannot at the moment be supported by a systematic
theoretical analysis of the intricate mechanisms of transmodal meaning-making (see: [Hallet, 2018]). One of the many challenges related to the problem of semiotic modes integration within the multimodal narrative reflects a need to look upon an act of reading a multimodal text as a multiliterate act, when readers of all ages are expected to "engage in acts of signification across semiotic modes and in the construction of transmodal meaning of a single text as opposed to 'simple' acts of alphabetical decoding and signifying" [Hallet, 2018, p. 27].
This multimodal literacy can be developed as an essential component of High School and University curricula with students being made aware of a range of strategies modern authors employ to effectively help their readers immerse in the stories told in the Digital Era.
This article aims at probing deeper into the analysis of a number of cognitive mechanisms and pragmatic strategies used by contemporary writers to construct their multimodal stories where various semiotic resources, both bound and free (see: [Hallet, 2018]), are creatively integrated to express a holistic meaning of the literary text.
Material and methods
The article uses Dan Brown's book for kids "Wild Symphony" (2020) as its case study to explore the innovative mechanisms that our youngest readers might encounter in the multimodal texts nowadays. The main focus in the second section of this paper, therefore, is made on the further study of the multimodal resonance treated here as a cognitive mechanism ensuring an increased impact of semiotic modes on addressees. The third section of the paper explores the combination of various types of foregrounding enhancing the expressivity of multimodal literature. The novella by Craig Silvey "The Amber Amulet" (2012) has been chosen for this section as a sample of multimodal youth fiction in which a gifted Australian author practices a wide range of multimodal devices that might help to engage young adult readers in his fictional storyworld. Finally, in Section 4 of the article, the short story by Jon McGregor "In Winter the Sky" (2012) will be examined, its details will clearly demonstrate how the experimental narrative
strategies making an ample use of diverse modalities testify to the embodied character of reading. The elements of stylistic, cognitive and linguo-semiotic analyses will be incorporated in the discussion of these three multimodal texts as one of the potential contributions of this research is related to the challenge of paving the way for the development of a new methodology which will help to produce a coherent multimodal text analysis.
Results and discussions
Multimodal resonance as a cognitive mechanism in "Wild Symphony"
Dan Brown published the picture book for children titled "Wild Symphony" in 2020 giving all his fans quite an unexpected chance to appreciate his debut multimodal story for little ones. Apart from his poems for kids describing all sorts of animals, and simple morals of a didactic nature meant both for children and their caregivers alike, this wonderfully illustrated book includes a free app and music, composed by the author himself (see Fig. 1). In the afterword Dan Brown explains to his readers that "music is a kind of storytelling, and the orchestral movements in Wild Symphony - combined with their accompanying poems and illustrations - all work together (like a code, of sorts) to tell a story and reveal a funny or interesting side of an animal's personality. If you listen carefully, you might be able to find each animal hiding in the music" (Brown, 2020):
"Busy Beetles
Busy beetles, busy beetles,
Digging with their little feetles.
Working slowly as a turtle,
Building homes in garden dirtle.
But the beetles, bit by bittle,
Keep on going, never quittle.
When at last their work is done,
Then it's time to have some fun!" (Brown, 2020).
" There's time for work,
And there's time for play.
There's time for both every day!" (Brown, 2020).
Children's picture books that have already been the object of interdisciplinary investigation
Fig. 1. A multimodal page of "Wild Symphony'
for several decades typically represent a clear-cut case of bimodality with verbal and visual semiotic resources, that "range from the wordless or near-wordless picture book to 'illustrated stories'"[Fryer, 2014, p. 313]. The picture book as a complex genre accounts for a growing body of research encompassing diverse approaches of linguists, educationists, ethnologists focusing either on text analysis, or the studies of "children's, adults', and children and adults' reading practices, how readers engage with, respond to, and make meaning of such texts, and the 'talk around text' typical of adult-and-child joint-reading" [Fryer, 2014, p. 313]. Multimodal stylistics is one of the potential areas of research examining the image/ text interaction and scrutinizing implications of the groundbreaking research in social semiotics by G. Kress and T. van Leeuwen [2006] for the analysis of such texts (see: [Painter, Martin, Unsworth, 2013]). One of the controversial points in multimodal stylistics concerns the definition of a truly multimodal text as opposed to a visually conventional one [Norgaard, 2014]. W. Hallet, for example, insists on the creative use of free, or unbound semiotic resources, i.e. photographs, maps, illustrations, handwritten letters, as inherent components of a fictional world "that are produced, circulated or displayed in the fictional world" [Hallet, 2018, p. 26].
Within this perspective not all illustrated books are likely to be qualified as multimodal texts, for a frequent use of illustrated materials has
always been associated with visually conventional literature both for adult and young readers. The book written by Dan Brown, nevertheless, creates a fully immersive experience of storytelling combining verbal, visual and auditory semiotic modes, the experience the author himself describes as "an immersive feast for eyes, ears, and mind" (Brown, 2020). Young children and their parents are able to listen to the original music compositions accompanying adventures of each animal of the book whilst augmented reality helps readers to simultaneously read the story and enjoy music pieces created by the author.
This innovative multimodal approach, used by Dan Brown, instantiates quite convincingly the opinions of many modern educationists pointing out the need "to recognize the diversity of children's experience of finely shaped imaginative worlds, and the ways in which new technologies and possibilities for meaning-making are creating new textual forms" [Beavis, 2013, p. 245]. Such new forms of storytelling, related to the emergence of "participatory culture" [Beavis, 2013, p. 245], raise challenges for cognitive linguistics, psycholinguistics, neurolinguistics. Thus, nowadays we witness an increased interest in the affordances of diverse sensory modalities which are involved creatively in the process of storyworld construction and perception.
The imaginative capacity of readers (visual, aural, tactile, etc.), young and adult alike, has
recently become the focus of interdisciplinary investigation aiming at exploring cognitive mechanisms underpinning the multimodal meaning-making. According to A. Gibbons, "not only do humans experience the world in multisensory terms, the media-saturated environment and highly visual culture that has developed into the twenty-first century complements the impression of multimodality as contemporary reality" [Gibbons, 2009, p. 100]. Studying the latest book by D. Brown, who implicitly acknowledges a paradigmatic shift from a monomodal page of a story to a multimodal construct, might yield an insight into the interplay of sensory modalities.
The hypothesis proposed in this article underlines the significance of multimodal resonance as a cognitive mechanism responsible for an increased concordance among verbal, visual and aural modalities. In the light of the latest findings in neuroscientific works which have proved that the combination of senses "result in enhanced neurological response" [Gibbons, 2009, p. 101], it transpires that the mechanism of multimodal resonance (see: [Iriskhanova, 2018]) appears to activate for recipients of "Wild Symphony" a game of make-believe involving the literary text (poems), art (illustrations) and music (classical orchestra pieces) as imaginative props (cf.: [Walton, 1990]). Our imagination is supposed "to be a multimodal dynamic system exhibiting features that are referred to as self-organization or emergence, implying simulation, emotional reward and in general the activation of embodied multiple neural circuits involved in superior associative cognitive functions" [Abramo, Gambino, Pulvirenti, 2017, p. 55]. Thus, such visual and aural props converge, activating multiple neural circuits, and enhance the emotional impact on children, emotive function being one of the metafunctions in such books for kids (see: [Painter, Martin, Unsworth, 2013]). The effect of multimodal resonance is characterized by a complex network of associative links, which, on the one hand, exist between verbal and visual semiotic resources, and, according to the seminal study by Van Leeuwen, semantically "extend" each other [Van Leeuwen, 2004, p. 230]. On the other hand, multimodal resonance is enhanced through readers' further immersion in the "musical narration" [Kramer, 1990] and its own intricate
"extracompositional relations" [Wolf, 2009] with the verbal mode.
Another important function of this multimodal text is ludic, helping to engage young readers in exciting puzzle-solving activities generated by Dan Brown, famous for offering his readers various intellectual conundrums in his novels. The poems in his picture book are full of verbal games, alliterations, nonce-words, and puns, captivating in their humour (see example above) and paired with thought-provoking short morals. There are jumbled letters hidden on each page of the book inviting kids to identify a musical instrument associated with each character in the narrative. For example, the page with Busy Beetles contains a key to marimba, Eager Elephant is trombone, Happy Hippos is bass, etc. There is even a secret message encrypted in the book, which says "music is a universal language", and, indeed, this interplay of semiotic codes seems to be entertaining, moving and educational at the same time setting an example of multimodal storytelling for young children many writers might find inspiring to emulate.
Foregrounding mechanisms in "The Amber Amulet"
in the light of multimodal stylistics
The question how textual strategies can trigger specific readerly responses, i.e. what kind of "interpretational programme", devised by an author, can be elicited by addressees of a literary text, is evidently a pivotal problem in many fields of philological studies.
One of such interdisciplinary theories combining efforts of linguists, neuroscientists, psychologists and literary critics is Foregrounding theory. According to W. Van Peer and Frank Hakemulder, the notion of foregrounding, if approached from this wider interdisciplinary perspective, "has come to mean several things at once. First of all, it is used to indicate the (psycholinguistic) process by which - during the reading act - something may be given special prominence. Second, it may refer to specific devices (as produced by the author) located in the text itself. It is also employed to indicate the specific poetic effect on the reader. Furthermore, it may be used as an analytic category in order to evaluate literary texts, or to situate them
historically, or to explain their importance and cultural significance" [Van Peer, Hakemulder, 2006, p. 547]. From the neurohermeneutic standpoint, foregrounding "surface" features of a literary text are believed to be "generated by the mindbrain of the author as possibly triggering specific reader responses in terms of immersion, empathy, emotional arousal, aesthetic appreciations, pleasure" [Abramo, Gambino, Pulvirenti, 2017, p. 52] and are treated as "'traces' of the author's brain processes, i.e. as specific highly formalized elements of the 'poetic language' (Lotman), expressing the thought processes of the creative mindbrain of the author" [Abramo, Gambino, Pulvirenti, 2017, p. 52].
Foregrounding, therefore, is often interpreted both as a cognitive mechanism providing readers with important clues as to an interpretational programme of a literary text, and as a wide range of linguistic devices triggering deautomatization of the reading process and the ensuing intensification of reader cognitive activities.
This article hypothesizes the relevance of identifying two separate mechanisms of foregrounding - graphic and visual - generated by modern authors of multimodal texts to give special salience to the two wide groups of semiotic resources, described by W. Hallet as bound and free resources [Hallet, 2018]. In his classification "bound" semiotic modes, such as colours, fonts, creative typography, are always "directly tied to distinct semiotic units of representation or communication" [Hallet, 2018, p. 26], whereas "free, or unbound" semiotic modes - maps, photos, pictures - are used independently of verbal resources and contribute significantly to the multimodal meaning-making. Thus, the graphic foregrounding, which has always been associated with literary text construction and drawing readers' attention to the elements, most prominent in this text from the point of view of its creator, uses a creative potential of the bound semiotic resources. Visual foregrounding [Chemodurova, 2021] can, on the other hand, be defined as a formal feature of modern multimodal texts focusing readers' attention on various unbound semiotic resources contributing to the transmodal meaning-making process and performing a range of functions in the narrative.
This section of the paper, therefore, proposes a further analysis of the dynamic combination of these two types of foregrounding and their
functions in the meaning-making process of the multimodal novella called "The Amber Amulet", used as a case study in this article.
"The Amber Amulet" is a story about a twelve-year old boy who seems to be a quiet enough teen during the daytime and turns into a heroic persona of "the Masked Avenger" fighting against injustices and trying to right wrongs in his neighbourhood during the nighttime. The story is a third person narration with this Superhero of a boy as its reflector so the opening paragraph of the novella might easily be taken for a revised pulp fiction story for teens:
"The Masked Avenger can make things happen. Though at twelve he is considered young for a Justice Fighter, he has already proved himself highly effective in the pursuit of peace. His developing powers are so potent, so vast, that not even he can fully comprehend their extent. He has a number of astonishing abilities. For example, on certain occasions he can summon a wall of cloud and make the sky roar" (Silvey, 2012, p. 1).
This kind of the beginning helps Craig Silvey to focus our attention on the imaginary world of Liam McKenzie, who patrols his suburban neighbourhood and sincerely believes in mysterious powers of energy which is "imbued with the properties of Good and Evil" (Silvey, 2012, p. 2). The combination of visual and graphic foregrounding, used extensively to construct the narrative, helps to characterize this boy as a naive and open-hearted dreamer, sharing his most secret thoughts and ideas with his devoted sidekick, Richie the Powerbeagle, and keeping a sort of scrap-book, both hilarious and touching in its simplicity and old-fashioned charm.
The scrap-book as a feature of many contemporary books with adolescent protagonists (cf. a well-known example can be found in J.S. Foer's "Extremely Loud and Incredibly Close") is a powerful multimodal mechanism contributing to the creation of a multi-layered narrative and enhancing its expressivity. One of the most important concepts of the story is the energy "hidden from common citizens" but available to The Masked Avenger is foregrounded graphically (the boy's captions for his pictures and illustrations, his handwritten notes and formulas) and visually (through a number of pictures, photos, maps) with the converged effect of immersing readers in a superhero's world of astonishing
abilities and their sources, setting a humorous and playful tone of the story and triggering narrative empathy for a lonely boy longing for warmth and safety (see Fig. 2). The technique of depicting cognitive abilities and emotional states of characters with the help of visual props in the fictional game of imagination and graphic devices effectively converging on the page is supposed to contribute to "empathetic experiences, opening readers' minds to others, changing attitudes, and even predisposing readers to altruism" [Keen, 2006, p. 213].
Empathy, though a loaded term, might be broadly defined as "an affective response stemming from the understanding of another's emotional state or condition similar to what the other person is feeling or would be expected to feel in the given situation" [Decety, Meyer, 2008, p. 1054]. According to the findings in developmental psychology, narratology and cognitive psychology, empathy might be triggered by "the deliberate acts of imagination" and is believed to be particularly "sensitive to perspective-taking" [Decety, Meyer, 2008, p. 1070]. Fiction for young adults such as "The Amber Amulet" relies quite actively on the combined use of different types of semiotic
foregrounding (visual and graphic) which has been reported to significantly increase the general emotional response and empathic distress" [Koopman, 2016, p. 90].
The story about Liam, which starts as a light-hearted adventure tale, is quickly transformed into the coming-of-age narrative, developing a philosophical subtext and showing life in all its complexity and unpredictability. The author's narrative strategy skillfully steers his readers away from a purely entertaining impact of his multimodal textual elements when Craig Silvey introduces one more element of graphic foregrounding, viewed as a bound semiotic resource (see Fig. 3). The theme of handwritten notes that Liam initiates by leaving a short letter to his beautiful neighbor, who seems to be troubled and unhappy in her family life, gives a game of make-believe we have been engaged in so far a more profound aspect inviting both young adults and more mature readers to change perspective and think about the issues of identity, courage and happiness. The culmination of this truly moving story foregrounds the materiality of the little book [Gibbons, 2012] we are reading intensifying the game of our
Fig. 2. The multimodal scrap-book in "The Amber Amulet" (pp. 2-3)
<
/ A CA ' ' аЛ*. л St s. Sc,
< -г^с S * Jf*
с/ i- ci ^ ?
^KM ^il f.<" ^¡/¿t^y '-<*
и^ л- дЛ
l/Ub ^/Tïi -i" , ** (jj - V ^ -tXï
'¿«a ' '/^v ;
.-«¡¡F -y " t^
.«.-С • _ A -'".«
Алдю yC^dtiC ij /f^Ji/Vi .¿¿Г iViiï
^fe л т^л/ ^dcA йа^ -efaeé ¿M
/ ^/W йДКЛ1,- .-¿К*'i Л.
«i^e. OJ^ sew -ЙО ^»C /
if
^TuU^y if*'*' -
/ ôwAbùî л ^¿Ал •
Fig. 3. Joan's farewell letter as an example of the creative use of the bound semiotic resources (pp. 84-85)
imagination even more and thus causing even a stronger emotional response.
Thus, the convergence of graphic foregrounding (bound semiotic resources) and visual salience (free semiotic resources) might be viewed as a powerful attention phenomenon that impacts readers on various levels of the text and contributes to the transmodal meaning-making.
Narrative experimentation from the multimodal perspective
M.-L. Ryan, analyzing the phenomenon of narrative and its transformations in the digital era, stresses the cultural significance of this concept pointing out that "it is because knowledge was encoded as tales that it was effectively transmitted and remembered in oral societies; it is because of its narrative power that the novel emerged as the dominant literary genre of the nineteenth and twentieth centuries; and it is because it gave new narrative dimensions to the novel and to the theater that cinema became "the art of the twentieth century" [Ryan, 2002, p. 582].
Playing with typography, creatively approaching the page layout, using various font types and experimenting with their colours can be considered a typical feature of much postmodernist fiction of the 60s - 80s of the 20th century prompting Brian McHale to describe such literature as "concrete prose" [McHale, 1987]. Many 21st century writers have developed
this postmodernist tendency of foregrounding the intricate relations between fiction and reality, resorting to what some critics tend to define rather disparagingly as "narrative gimmicks" and drawing their reader's attention to the ludic modality of such fictional texts [Chemodurova, 2019].
The short story by Jon McGregor "In Winter the Sky", included in his collection of short stories "This isn't the Sort of Thing That Happens to Someone Like You" [2013], is characterized by such a distinctively unconventional use of various semiotic resources that it immediately signals to those readers who are not so well-versed in multimodal literary practices yet that reading this story poses a challenge and might even cause a cognitive dissonance [Chemodurova, 2019]. The design of the page and the graphic foregrounding of the explicitly polyphonic narration leave a lasting impression of an original and immensely disturbing story told by the husband haunted by the memory of one life-changing event in his past and the wife desperately trying to grope her way to happiness in their marriage. The heteroglossia is created with the help of the page layout, with the left page telling us a story of their teenage romance, a tragic incident changing the course of both their lives and their marriage, and the right page offering us lyrical experiments very loosely connected to the left-page narrative. The third person narration of the left part of the story is regularly interrupted by the sad ruminative lines composed by the wife (see Fig. 4).
Fig. 4. The page layout foregrounding a polyphonic nature of McGregor's short story (pp. 20-21)
waist, fie backed away towards the embankment and the man's legs unfolded from beneath him, his head rolling down into his armpit, his half-bottle of whisky falling from his pocket and breaking on the road. He didn't stop. He kept dragging him away, away from the road, down the bank, into the field.
She'd said, when he finally told her all this, that she wanted to know it all. How it was done. How it had felt. So now she knew.
I le laid the man down beside the hole in the earth and rolled him into it. The man fell face down, and he felt bad about that, about the man's face being in the mud. He went back to the place on the road and picked up the pieces of glass. He threw them down on to the man's back, and then he took the shovel and began to pile the earth back into the hole.
He threw soil over the man until he was gone, until the soil pressed down on him so that he was no k»nger a man or a body or a victim or anything. Just an absence, hidden under the ground. It was only then that he looked up at the sky, dark and silent over him, the moon hidden by a cloud. He drove past her house in March again, and then back to his father's house. He put the car-keys awav in the kitchen drawer, and the shovel in the barn, and he stood in the shower until the hot-water tank had emptied and he was left standing beneath a trickle of water as cold as stone.
The words we'vv been given Our waterway* arc called drains not rivers or streams or brooks or bums.
(and the closest to grandeur, this)
to name these place*
Thirty Foot Drain Sixteen Foot Drain Hundred Foot Drain Lower Field Farm Middle Field Farm Sixteen Foot Farm
itound Ы the place, the landscape
People arc not drawn here b\ People don't much come hen
retains its beautv and
the beauty ot thi> place о not in the name* but the »hape* the flatness / hugeness I completeness of the landscape Only what t» beneath the surface ot the earth is hidden
and everything else и» made visible beneath the sky
So now she knevs
The two parts of the story seem to require a certain readjustment on the part of its readers involved in a hypertextual fictional game and expected to choose a reading strategy: they can either carry on in a traditional linear manner facing a deliberate narrative disruption when moving from a left page of the story to its right page; or they can select a different tack reading only the left-side part of the story and then turning to its poetic counterpart. The ludic modality of the text is also sustained through the typographical signs of the performative nature of art: crossed out lines of the poems in prose, left on the right-side pages by the author, foreground a painful process of creativity strangely resonating with the darkest moments of our life and causing cognitive dissonance in the minds of unsophisticated readers trying to piece together parts of the narrative puzzle. The typographical difference in type size, crossed out words and phrases, disjointed textual elements signal a change of discourse accentuating multisensory perception of the story (cf.: [Gibbons, 2009]).
The choice of the reading strategy we face when looking at the pages of this story and taking in its visual peculiarities makes us acutely aware of the materiality of the book and the physicality of the reading process which inevitably leads us to the fuller understanding of the embodied character of reading.
Conclusion
The emergence of the extended body of the multimodal fiction in the 21st century has raised quite a number of challenging problems for various areas of the philological research. Multimodal stylistics is interested, among many aspects of multimodal text construction and interpretation, in identifying cognitive mechanisms responsible for the multimodal meaning-making and describing diverse strategies providing an increased pragmatic impact on the readers. The cognitive mechanism of multimodal resonance, hypothesized in this article, is believed to play a significant role in the meaning-making process, contributing to ensuring immersion, triggering empathy and increasing text expressivity.
Visual and graphic foregrounding, viewed in this paper as distinct though by no means independent ways of focusing reader's attention on numerous semiotic modes in this fast developing genre of literature, can account for the enhanced emotivity of such texts and their explicit ludic modality.
The further study of creative ways that many contemporary authors use to share exciting stories with their readers, whether it is a hypertext narration or an original combination of free and bound semiotic resources, will yield new insights into the complex and captivating art of storytelling.
REFERENCES
Abramo F.C., Gambino R., Pulvirenti, G., 2017. Cognitive Literary Anthropology and Neurohermeneutics. A Theoretical Proposal. Enthymema, no. XVIII, pp. 44-62.
Beavis C., 2013. Literary English and the Challenge of Multimodality. Changing English: Studies in Culture and Education, vol. 20, no. 3, pp. 241252. DOI: 10.1080/1358684X.2013.816527
Chemodurova Z., 2019. The Mechanisms and Types of Foregrounding in Postmodernist Fiction. Haase Ch., Orlova N., eds. English Language Teaching: Through the Lens of Experience. Newcastle, Cambridge Scholars Publishing, pp. 277-295.
Chemodurova Z.M., 2021. Visual Foregrounding in Contemporary Fiction. Voprosy kognitivnoy lingvistiki, no. 2, pp. 5-15. DOI: 10.20916/18123228-2021-2-5-15
Decety J., Meyer M., 2008. From Emotion Resonance to Empathic Understanding: A Social Developmental Neuroscience Account. Development and Psychopathology, no. 20, pp. 1053-1080.
Fryer D.L., 2014. Reading Visual Narratives: Image Analysis of Children's Picture Books. Linguistics and the Human Sciences, vol. 10.3, pp. 312-321. DOI: 10.1558/lhs.v10.3.19618
Gibbons A., 2009. "I contain Multitudes": Narrative Multimodality and the Book That Bleeds. Page R., ed. New Perspectives on Narrative and Multimodality. London, Routledge, pp. 99-114.
Gibbons A., 2012. Multimodal Literature and Experimentation. Bray J., Gibbons A., McHale B., eds. Routledge Companion to Experimental Literature. London, New York, Routledge, pp. 420-434.
Hallet W., 2018. Reading Multimodal Fiction: A Methodological Approach. Anglistik: International Journal of English Studies, vol. 29, iss. 1, pp. 25-40.
Iriskhanova O.K., 2018. Fedya, dich! Ili o rezonanse v rechi i zhestakh [Fedya, dich'! On Resonance in Speech and Gestures]. Kognitivnye issledovaniya yazyka [The Cognitive Studies of the Language], iss. 33, pp. 598-604.
Jewitt C., ed., 2009. Handbook ofMultimodal Analysis, London, Routledge. 340 p.
Keen S., 2006. A Theory of Narrative Empathy. Narrative, vol. 14, no. 3, pp. 207-236.
Koopman E.M., 2016. Effects of "Literariness" on Emotions and on Empathy and Reflection After
Reading. Psychology of Aesthetics, Creativity, and the Arts, vol. 10, iss. 1, pp. 82-98. DOI: https:// doi.org/10.1037/aca0000041
Kramer L., 1991. "Musical Narratology: A Theoretical Outline". Indiana Theory Review, vol. 12, pp. 141-162.
Kress G., Van Leeuwen T., 2006. Reading Images: The Grammar of Visual Design. London, Routledge. 310 p.
Luke J., 2013. Writing the Visible Page: A Multimodal Approach to Graphic Devices in Literary Fiction. PhD by Creative Works. QUT ePrints. URL: https://eprints.qut.edu.au/63020/
McHale B., 1987. Postmodernist Fiction. London, New York, Routledge. 278 p.
Norgaard N., 2014. Multimodaly and Stylistics. Burke M., ed. The Routledge Handbook of Stylistics. London, New York, Routledge, pp. 471-485.
Painter C., Martin J.R., Unsworth L., 2013. Reading Visual Narratives: Image Analysis of Children's Picture Books. Sheffield, Equinox. 181 p.
Ryan M.-L., 2002. Beyond Myth and Metaphor: Narrative in Digital Media. Poetics Today, vol. 23, iss. 4, pp. 581-609.
Ryan M.-L., 2004. Introduction. Ryan M.-L., ed. Narrative Across Media: The Languages of Storytelling. Lincoln, U of Nebraska, pp. 1-40.
Van Leeuwen T., 2004. Introducing Social Semiotics. London, New York, Routledge. 320 p.
Van Peer W., Hakemulder F., 2006. Foregrounding. Brown K., ed. Encyclopedia of Language & Linguistics. Vol. 4. Oxford, Elsevier, pp. 546-550.
Walton R., 1990. Mimesis as Make-Believe. London, Harvard Un. Press. 450 p.
Wolf W., 2009. Relations Between Literature and Music in the Context of a General Typology of Intermediality. Comparative Literature: Sharing Knowledges for Preserving Cultural Diversity. Vol. 1. URL: https://www.eolss.net/ sample-chapters/c04/E6-87-02-02.pdf
SOURCES
Brown D. Wild Symphony. 2020. URL: www. wildsymphony. com
McGregor J. This isn't the Sort of Thing That Happens to Someone Like You. London, New Delhi, New York, Sydney, Bloomsbury, 2013. 262 p.
Silvey C. The Amber Amulet. Sydney, Allen and Unwin, 2012. 104 p.
