CC BY-NC-ND
119
Strategies of Reading Digital Texts for Performing Educational Reading Tasks: Study Based on the Think-Aloud Protocols
M.Yu. Lebedeva
This article was Maria Lebedeva, Candidate of Philological Sciences, leading research fellow, submitted to the Edi- Pushkin State Russian Language Institute. Email: m.u.lebedeva@gmail.com. Ad-tonal Board dress: 6 Akademika Volgina St., Moscow, 117485, Russia in November 2021
Abstract For success in education and life in our informationally saturated digital society, one must be able to select and interpret digital texts of different genres, choose optimal ways of interacting with these texts, and extract and assess information from them. Contemporary education specialists believe that skills of working with digital texts are an integral part of reading literacy; their publications model successful results of interacting with digital texts. Nevertheless, the means of attaining these results remains a very important and topical question for the education system. What strategies allow one to interact with digital texts effectively? How should one teach these strategies to contemporary schoolchildren? The present article aims to identify and classify metacognitive strategies used by competent Russian-speaking lower secondary students for performing learning assignments based on digital texts. It is based on the analysis of think-aloud protocols and data from the online monitoring of readers' activities on the screen. The study describes and analyzes seven groups of digital reading strategies. The results contribute to basic knowledge about the processes at the root of effective digital reading and hence of the development of approaches to teaching and assessing reading literacy in the digital age.
Keywords reading literacy, digital reading, metacognitive reading strategies, teaching digital reading, online monitoring, think-aloud method.
For citing Lebedeva M.Yu. (2022) Strategii raboty s tsifrovym tekstom dlya resheniya ucheb-nykh chitatel'skikh zadach [Strategies of Reading Digital Texts for Performing Educational Reading Tasks: Study Based on the Think-Aloud Protocols]. Vo-prosy obrazovaniya / Educational Studies Moscow, no 1, pp. 244-270. https://doi. org/10.17323/1814-9545-2022-1-244-270
Digital textbooks, interactive workbooks, and digital content ranging from educational to popular science — these tools have taken a strong position in the modern educational process. The sources from which today's schoolchildren draw information important for their education and development are mostly digital [Lebedeva et al., 2020. P. 262].
At the same time, there is a strong view held in the science of reading that interaction with digital text is fundamentally different from working with printed text or its digitized version. Digital reading is defined as working with texts characterized by distinctive properties that can only be found in the digital environment [Singer, Alexander, 2017]. This type of reading relies on specific cognitive and metacognitive processes. For the participants in today's education system, it is crucial to understand how digital educational reading, that is, extracting information from digital text and interpreting it to solve educational tasks, works. In particular, the "learning from the strongest" approach appears to be productive: understanding the strategies used by proficient digital readers can serve as a basis for teaching successful digital reading.
This article reports the results of a qualitative study of metacognitive strategies employed by proficient Russian-speaking digital readers at the end of lower secondary (or "basic", according to the terminology of the Russian education system) school when reading a digital text. The purpose of the study is to identify, describe, and classify these strategies.
The purpose of the study determines the structure of the present article. The article consists of an introduction, a review of the theoretical background, a description of the research methods and materials, a presentation and discussion of the results, and a conclusion.
1. Theoretical background of the research
1.1. Reading strategies for teaching and assessing reading literacy
The tools for measuring reading skills and competencies are based predominantly on the outcome-based approach to the assessment of reading activities. For instance, the sections on semantic reading in the international PIRLS [Mullis, Martin, 2019] and PISA [OECD, 2019; Zuckerman, 2010] surveys and the Russian monitoring and control tools for measuring reading literacy1 [Ryabinina, Chaban, 2019] assess how participants solve the tasks of finding, selecting, interpreting, and evaluating information from text. The outcome-based approach is based on the assumption that in case of adequate ("correct") text comprehension the reader's projection of the text will be close to the author's projection [Zalevskaya et al., 1998. P. 35].
At the same time, in reading literacy instruction the outcome-based approach has a number of limitations. Focusing only on reading outcomes does not provide an opportunity to identify
Description of the testing and assessment materials for the 2020 Russian language test, 4th grade // Federal Service for the Supervision of Education and Science. https://fioco.m/Media/Default/Documents/%D0%92%D0%9F%D 0%A02020/%D0%92%D0%9F%D0%A0_%D0%A0%D0%A3-4_%D0%9E%D0%B-F%D0%B8%D1%81%D0%B0%D0%BD%D0%B8%D0%B5_2020.pdf
the underlying mechanisms that can help to improve reading skills [McNamara, Kendeou, 2011. P. 35]. When applying a strictly outcome-based approach, one does not measure semantic reading skills per se, but rather the reader's ability to answer text-based questions and complete text-based tasks.
The process-based approach, an alternative to that focused on the outcome, is used more often in basic research on reading and less often — in instruction and assessment. This approach focuses on the cognitive and metacognitive processes that take place during reading and text comprehension, from word decoding to making meaning from text. Of particular interest are the processes that readers consciously activate during and after reading, such as self-explanation and asking questions about the text.
There is a substantial body of evidence on how particular reading processes and strategies lead to effective or, conversely, poor comprehension [Magliano, Millis, 2003; Millis, Magliano, Todaro, 2006; Magliano, Millis, Ozuru, McNamara, 2007]. The information about the process of reading is usually complemented by an assessment of its results [McNamara, Kendeou, 2011]. This approach allows for a deeper understanding of how to teach semantic reading and helps students overcome difficulties in comprehending the text.
This research combines the process- and outcome-based approaches to the study of reading: we focus on the metacognitive strategies of digital reading, and we measure the effectiveness of these strategies by reading outcomes, i.e. accurate and complete responses to text-based questions.
1.2. Reading In the literature, strategies are defined as techniques that help strategies as readers to overcome difficulties and obstacles to successful text a special kind comprehension [McNamara et al., 2007] and as "voluntary actions of process that of the reader aimed at the most effective and full comprehension occurs during of the text" [Oganov, Kornev, 2017. P. 118].
reading in didactic studies, reading strategies are defined as complex units — combinations of individual strategies representing behavioral or cognitive actions, for example, pausing while reading (Self Monitoring Approach to Reading and Thinking, or SMART), "Know — Want to know — Learned" [Smetannikova, 2018. P. 57-58], SQ3R and SQ4R [Prantsova, Romanicheva, 2015. P. 33].
In order to distinguish individual reading strategies from comprehensive instructional strategies, we will refer to the latter as technologies of strategy-based reading instruction. A technology is a particular sequence in which multiple strategies are applied for more effective comprehension and memorization of a text.
Reading strategies are defined in this article as deliberate, goal-directed actions that readers take to effectively accomplish
their reading goals. The goals can be self-determined by the reader or externally imposed, ranging from getting a "full comprehension of the text" [Oganov, Kornev, 2017. P. 118] to cursory reading or scanning the text for specific information.
Another category is metacognitive reading strategies. Using these strategies, readers consciously plan, control, evaluate, and correct their interaction with the text. The employment of such strategies involves "cognition about cognition" in the process of reading, i.e., being aware of the cognitive processing of the text and correcting the cognitive reading strategies being applied [Baker, Brown, 1984]. Three groups of metacognitive strategies are distinguished: Global Reading Strategies for reading planning, regulation, and assessment; Support Reading Strategies, such as taking notes or using reference materials; and Problem-Solving Strategies for solving problems while reading [Mokhtari, Reichard, 2002].
1.3. Digital This study focuses on strategies that readers use when reading a reading digital text, i.e., a text that is characterized by multimodality, non-strategies linearity, and interactivity [Lebedeva, Veselovskaya, Kupreshchen-ko, 2020].
Studies of reading distinguish between texts that are the result of digitization of print sources and texts that have specific properties that can only be found in the digital environment [Singer, Alexander 2017. P. 1035]. There is no doubt that to achieve learning objectives, schoolchildren also interact with digitized linear ("ordinary") texts. However, we are particularly interested in digital texts that require dealing with specific tasks: concentrating and overcoming distractors, choosing a reading trajectory, etc.
The differences in digital and print reading processes are often described in terms of strategies: strategies for text navigation, control of scrolling (flipping pages on the screen), and distribution of attention between the components of digital text, including components of different semiotic nature.
In particular, researchers have been studying strategies for navigating through digital text components that are specific to digital reading [Salmeron et al., 2005. P. 174], coping with the unexpected structure and interactivity of online texts [Afflerbach, Cho, 2010], and resisting the inclination to read in a cursory and fragmented manner typical of the digital environment [Zhang, 2012. P. 138].
It is important for our study that scholars conceptualize reading digital multimedia hypertexts as self-directed text construction [Coiro, Dobler, 2007]). The very nature of digital text encourages nonlinear, selective interaction with any text (or multiple texts) on the Internet, which contrasts with the conventional sequential reading of printed text. Every reader constructs his or her unique read-
ing path in the digital environment — or follows the path offered by that environment. According to N. N. Smetannikova, "readers of the virtual text do not follow the author, do not perceive the text according to the author's logical structure, but establish connections in the text independently, thus creating their own structure <...> Today a machine becomes an active 'reader', it offers its text to a human, thereby changing the relationship between reader and text" [Smetannikova, 2019. P. 8].
This conceptualization of digital reading as a construction process dictates the urgent need for a high level of reading awareness, which is essential to effectively and optimally construct the path and mode of interaction with digital text. We, therefore, assume that a proficient reader of digital text differs from a traditional reader in his or her ability to apply metacognitive reading strategies.
1.4. The strategy-based approach to digital reading instruction and reading skills assessment
The application of research on reading strategies in the classroom is based on the assumption that the strategies of proficient readers differ from those of readers who have difficulty comprehending what they read. This assumption is well supported empirically.
A whole range of studies have found significant differences between proficient and less proficient readers in the application of specific strategies (building coherence [Magliano, Millis, 2003; Millis, Magliano, Todaro, 2006], preliminary analysis of text structure [Block, 1986]), and in quantitative indicators [McNamara, McDaniel, 2004; Anderson, 1991; Yayli, 2010].
These data inform educational practices. Reading strategies are developed through both teacher-assisted instruction2 and interaction with automated learning systems [McNamara et al., 2006]. This approach focuses not only on the outcome of reading (correct or incorrect comprehension) but also on the process of reading and making sense of the text. The effectiveness of this approach has been supported by research findings showing that didactic interventions significantly contribute to the development of reading competence.
Of particular interest is the use of the process-based approach to measuring reading skills. For example, evidence of significant differences in coherence-building strategies between proficient and less proficient readers has provided the basis for the Reading Strategy and Assessment Tool (RSAT) which enables automated assessment of reading competencies [Magliano et al., 2011]. In digital assessment systems, it also appears promising to consider reading
2 See [Mulcahy-Ernt, Caverly 2009; Smetannikova, 2018; Prantsova, Romanicheva, 2015] for a description of approaches and techniques of strategy-based reading instruction.
behavior, i. e. user actions in the digital environment, which, as we assume, may reflect their reading strategies [Lebedeva et al., 2021].
In order to apply a strategy-based approach to digital reading instruction, we need to find out which strategies for interacting with digital texts, summarizing, and didactically making sense of the accumulated observations are used by competent readers. In particular, the international Online Reading Comprehension Assessments project (ORCA) has tested a model for teaching digital reading strategies based on teachers' demonstration (voicing) of strategies and techniques they use when working with online texts [Coiro, 2011].
This paper describes the results of a study on digital reading strategies employed by competent readers attending lower secondary school, conducted on the Russian-language materials and a sample of Russian-speaking readers.
2. Research methods and materials
2.1. Research participants
The study was conducted in two stages. At the first, screening, stage 12 students in grades 7-9 of schools in and near Moscow (six boys and six girls) took part in the study. This stage resulted in the selection of the two main participants — a boy and a girl in the ninth grade — who had been identified by the experts (language and literature teachers and reading researchers) as the most proficient readers. The second stage of the study involved individual sessions with the selected students.
Choosing senior students as participants was motivated by psychophysiological research on reading, according to which the ability to self-monitor comprehension develops in children after age 13, and the strategy-based approach to reading begins to develop closer to upper secondary school [Oganov, Kornev, 2017].
2.2. Research The study uses a combination of think-aloud verbal protocols and methods structured online observation.
Think-aloud verbal protocols involve participants thinking about and explicitly commenting on what they are reading [Bereiter, Bird, 1985]. This method gives insight into the cognitive and metacognitive strategies employed by the reader as he or she is making sense of the text [Leslie, Caldwell, 2009. P. 416]. The effectiveness of the think-aloud method for studying reading processes has been confirmed in a significant body of research on traditional reading ([Magliano, Trabasso, and Graesser, 1999; Magliano and Millis, 2003; Millis, Magliano, and Todaro, 2006; Leslie and Caldwell, 2009; Bohn-Gettler and Kendeou, 2014; Wang, 2016; Bohn-Gettler, 2018], etc.) and in a smaller number of studies of digital reading [Coiro and Dobler, 2007; Salmeron et al., 2017; Latini, Braten, 2021].
The structured online observation method implies recording the observed facts and events in the online environment [Polukhina, 2014]. For the purpose of this study, we were interested in observing the actions performed by the respondent while reading, such as scrolling up or down, slowing down or speeding up scrolling, engaging with interactive text elements, clicking hyperlinks, following text with the mouse pointer, highlighting text.
2.3. Research The study simulated the conditions of educational reading, i.e., inmaterials teraction with text to perform educational tasks. The participants were given the following reading prompt:
At school, your class has been asked to prepare for a lesson on "The Differences between Human and Animal". You need to find materials on the topic and prepare answers to the questions. You found this online material: <link>.
The link led to a post in the community of the Schrodinger's Cat popular science magazine3 containing an abstract of the text offered for reading. The text itself was published in the popular science section "Vsenauka" in the digital edition of the Novaya Gazeta newspaper4. Such an indirect path to the target text is intended to track participants' use of strategies for a preliminary evaluation of the text content and for making decisions on clicking hyperlinks in the digital text.
The reading material is a popular science text that possesses all the properties of digital texts. The structure of the text is unconventional, which leads to additional reading difficulties: the text opens with an interactive test to check the pre-reading knowledge on the topic of the text and to create reading expectations; the main part is written in the genres of expository text and interview; the conclusion contains the description of the books on the topic of the text.
The reading prompt consisted of three text-based questions. In the first question, the participants were asked to find specific, directly stated information in the text: "Find an example in the text of how animals use memes". According to the second question, the readers were expected to find information located in different parts of the text and interpret it: "How do different scientists answer the question of what makes humans different from animals?" The third question required thoughtful, analytical reading and summarization of information from throughout the text: "From what you
3 The post can be accessed at: https://vk.com/kot_sch?w=wall-78004698_29921.
4 The text can be accessed at: https://novayagazeta.ru/articles/2021/08/05/vo-pros-po-sushchestvu.
understood in the text, what makes humans significantly different from animals?"
2.4. Data collection The data were collected online using the Zoom platform.
The introductory phase included the establishment of contact between the respondent and the instructor, the explanation of the research protocol, and the testing of the technical and organizational conditions of the study.
During the demonstration phase, the participants were given an example of how to comment on the reading process using the think-aloud method. While reading a sample digital text, the instructor was verbalizing his or her thoughts and explaining his or her actions to the study participants.
The main phase involved the participants performing the assigned reading tasks and commenting aloud on their thoughts and actions. At the beginning of the main phase, the respondents activated the screen sharing mode on their device, so that the instructor could observe visible actions that they were taking while reading. In most cases, the participants chose at which points to pause and verbalize their thoughts independently, but the instructor sometimes stopped the participants and asked them to comment on specific actions. The participants were not assisted in the reading process; the instructor's questions did not prompt a correct interpretation of the text, but only motivated the participants to think aloud, for instance, "What did you just think about?" and "Why did you flip through this fragment?" The success of the participant's completion of the reading tasks was assessed by the instructor based on the oral responses of the former.
As a result, two verbal protocols of 63 and 39 minutes were collected, transcribed, and marked.
2.5. Data analysis The purpose of the study was to identify the specific digital reading methods strategies characteristic of competent readers and to classify them.
The data analysis method most relevant for reaching this goal is content analysis [Weber, 1990. P. 9-10; Krippendorp, 2004. P. 48-53].
According to the verbal protocol analysis procedures [Bohn-Get-tler, Olson, 2019], the records obtained from the research sessions were transcribed into a text format and segmented into single utterances, each of which was further encoded. Utterances containing multiple thoughts and ideas were split into smaller fragments. The coding was performed using the inductive approach [Bohn-Get-tler, Olson, 2019. P. 5], which ensured that the observation was not distorted by studies done in other languages, populations, and settings.
Using the inductive analysis of verbal protocols, a coding scheme was developed. Table 1 presents a fragment of the scheme, showing the codes related to the preliminary evaluation of the text by the participants.
Table 1. A Fragment of the Marked-up Verbal Protocol
Code j Interpretation Reader commentary
GENERAL_ PREVIEW j General preliminary evaluation of the j text: the reader explicitly communicates j his or her intention to evaluate the text before reading Here, I'll probably first look at what's on this page...
SCROLL_FROM_ i Scrolling through the text from beginning BEGINNING_TO_END j to end: the reader explicitly comments on this action ...scroll through it completely...
STRUCTURE_PREVIEW j Preliminary evaluation of text structure: j the reader explicitly communicates his or j her intention to evaluate the text struc-1 ture before reading ...look at the sections...
LENGTH_PREVIEW | Preliminary evaluation of text length: the j reader explicitly communicates his or her intention to evaluate the text length before reading ...see how long it is...
3. Results By analyzing the verbal protocols, we have identified the strategies and discussion used by the study participants, classified them, and examined in detail some of the strategies specific to digital reading. A detailed description of the identified strategies can be found on the website of the study5.
During the study, the participants demonstrated a high level of reading literacy and reading awareness, thus confirming that they were proficient readers: both participants gave correct and complete answers to the questions, and their commentary during the reading process displayed a high density, which is indicative of a competent reader [Anderson, 1991; Yayli, 2010].
In the general corpus of comments made in the course of reading, we identified comments that verbalize the following categories of strategies:
• text comprehension strategies;
• pre-reading strategies;
• strategies related to setting and pursuing a reading goal;
• strategies for choosing and changing the type, method, and trajectory of reading;
5 https://digitalpushkin.tilda.ws/digitalreading#strategies
• strategies for monitoring the quality of reading and resolving difficulties that arise;
• strategies for using information from the non-verbal components of the text;
• support strategies.
Text comprehension strategies are fundamental, universal strategies for semantic reading. They include the following:
• paraphrasing;
• making inferences based on what has been read;
• activating background knowledge;
• summarizing what has been read.
The set of strategies identified in our verbal protocol analysis is similar to those described in other studies and does not depend on the reading format [Bohn-Gettler, Kendeou, 2014].
Other categories of strategies used in digital text reading have their specific features.
3.2. Pre-reading Pre-reading strategies include the following: strategies
• preliminary evaluation of the text value and its relevance to the reading task;
• preliminary evaluation of the text volume, structure, and content;
• predicting the content of the text from the title, subtitle, and first lines.
Purposeful employment of these strategies indicates a well-developed level of skimming skills that is characteristic of a competent reader's repertoire of techniques, regardless of whether they are reading from paper or a screen. Yet, in the digital environment skimming has specific features. The most common action in skimming is scrolling, i.e., a special way of flipping that is not used in print reading. However, the difference is not only in mechanics.
A number of researchers have concluded that skimming is the predominant type of digital reading [Liu, 2005; Hillesund, 2010]. It can be assumed that to some extent the spreading of skimming is an evolutionary necessity: in the information explosion era, people face a prodigious amount of information and critically need the techniques for quickly determining the value and relevance of the incoming information. In the case of textual information, skimming is such a technique. A competent digital reader must be able
3.1. Text comprehension strategies
to quickly and consciously decide whether or not to read a text or text fragment, and such decisions are made based on a preliminary evaluation of various text parameters.
At the same time, as studies by other authors and our observations show, skimming is not always applied consciously and purposefully. For instance, according to the shallowing hypothesis [Annisette, Lafreniere, 2017], which is confirmed in recent experiments [Delgado, Salmerón, 2021], the daily mass experience of reading on digital media leads to a superficial perception of textual information, regardless of the reader's intentions [Alexander, the DRLRL, 2012; Delgado et al., 2018]. Consequently, a competent digital reader is characterized not so much by the high mastery level of their skimming skills as by the conscious employment of scanning techniques when beginning to work with the text in order to decide on the next steps to take.
Digital texts offer readers supports and cues that are not always present in a print text. In our study, for example, the participants paid attention to the slider on the right side of the text, and the structure of the digital text, when employing the strategies of preliminary text evaluation. When constructing a reading path, the participants relied on the following:
• their knowledge of the specific structure of texts in certain genres and formats:
If this were a Wikipedia article where I needed to find specific information, I would pay special attention to the large print headings because they usually separate blocks of information;
• the visual arrangement of the text. In our study, for example, due to a typical visual arrangement the readers identified an advertisement fragment at the end of the text and did not take it into account when working with the text. Such selective attention mechanisms are known as banner blindness [Pagendarm, Schaumburg, 2001];
• other features of hypertexts and interactive texts. For instance, the readers separately evaluated the value of text fragments referenced by in-text and external links.
Strategies related to setting and pursuing a reading goal include the following:
• focusing on the reading goal;
• tracking deviations from the reading goal and getting back to it;
• evaluating the accuracy and completeness of how the reading goal is accomplished.
3.3. Strategies related to setting and pursuing a reading goal
Strategies related to the adjustment of reading behavior to the reading goal are especially important in digital reading. Researchers have suggested that when reading a print source, readers adapt to the reading goal better than in digital reading [Latini et al., 2019]. Purposeless and mindless wandering in the online environment has been documented as a recurring behavior pattern [Burbules, Callis-ter, 1996. P. 41]. The so-called distracted reading is becoming a hallmark of today's multitasking world [Thain, 2018]. The prevalence of such behavior patterns is partly due to the high amount of distrac-tors in the digital environment. Full of various stimuli claiming the reader's attention, the digital environment impedes concentrated, goal-oriented reading. The reader needs to make a special effort to stay focused on the reading goal.
In our case, there was a special distractor — an entertaining test at the beginning of the article, asking readers to check their ideas against the opinions of scholars. The two readers evaluated the value of this test differently as they performed the assignment, but, importantly, they did so in relation to the reading goal: one participant refused to take the test because they thought it was not conducive to reaching the goal; another participant answered several test questions to see if the answers contained information that was valuable for accomplishing the reading goal.
While avoiding the most distracting trap of the proposed text, participants nevertheless lost concentration:
I think I got distracted from the question. I guess I was distracted by this text because it stood out, I got curious to see what was in it. There, that's it [back to goal-oriented reading].
Thus, a competent digital reader is characterized by the ability to define a goal, verbalize it explicitly, and check regularly against this goal while reading:
I probably don't need to take this test. What is it for? Let's move on to the text. I have the assignment to work on.
3.4. Strategies for choosing and changing the type, method, and trajectory of reading
Strategies for choosing and changing the type, method, and trajectory of reading include the following:
• choosing the type, method, and trajectory of reading depending on the reading goal, type, content, and the visual arrangement of the text;
• controlling the reading speed, depending on the content of individual text fragments.
In the digital environment, the reader has to construct the optimal reading path independently, i.e. to make decisions about which parts of the text and in what order to read, whether to follow hyperlinks, get interactively involved with the text, and go beyond the text by using additional resources (for example, clarifying the meaning of unknown words in electronic dictionaries or checking the facts from the text in other sources), etc.
The study supported the findings of the non-linear nature of digital reading: observing reading behavior revealed the participants' frequently going back through the text, and their comments showed how decisions about these actions were made. That being said, non-linearity is not always a sign of aimless wandering through the text; non-linear interaction with the text is often a consequence of the conscious application of reading strategies:
It looks like a conclusion, so it makes sense to read some of the preceding text. I'll take a look at the beginning of this paragraph here.
One of the key questions in this study was the reader's ability to switch between different types of reading to solve different reading tasks. Several of the tasks required search-reading techniques (reading for specific information), one of which was to find information located in a specific place in the text (an example of an animal meme), another was to collect and summarize the meaning of several text fragments (opinions of different scientists), while the third question required reading the whole text for detail (summarize the differences between animals and humans).
The question that required searching for multiple fragments proved to be more difficult for the participants than the question that required finding information located in one place. During search reading, the participants employed general strategies that could be applied regardless of the text format (e.g.,
I am looking for some names. Usually they are capital letters <...> in the middle of a sentence. And then I will read the text to understand whose name it is and what it relates to),
as well as strategies specific to digital reading (e.g., a search reading strategy where the reader uses such technical capabilities as the Ctrl+F key combination to search a page). Both study participants said that they often used this method when they were looking for an answer to a specific question (Table 2).
Such an appeal to new technological tools to implement standard reading operations requires a separate commentary. The strategy of finding information in a text through automatic search in-
Table 2. Search Reading Strategy with Automatic Text Search
Question: "How do different scientists answer the question of what makes humans different from animals?"
Action 1 Reader commentary example
Identifying the key (pivot) word i The key word here is "scientists" <...>. But we want to find specifically \ the part with "different scientists". There won't necessarily be the word j "different", so I'd put "scientists" in there
Search by keyword in the text \ Better without the ending. i [Typing "учен" — the Russian equivalent for "scientists" without j the ending — in the search field] 1 Seventeen results.
Reading the search re-suits with the keyword and evaluating their relevance to the question \ "Scientists managed to teach them the sign language" is hardly about different opinions. \ "Scientists from the University of California" — this is more interesting, i but here we see the heading "It's not only humans who can empathize", i so it's not really what I need, it's still not about different opinions. j "Many respected scientists have also been convinced of this" — there may be something here about differing opinions.
stead of using human eye only is an example of how humans are delegating some of their cognitive processes to emerging digital technologies. Word search entrusted to the computer ranks with such tasks as memorizing information, performing arithmetic operations, or translating text, which refers us to E. Clark and D. Chalmers' concept of extended cognition [Clark, Chalmers, 1998]. The assumption that the agent of the modern educational process can be seen as a "personality extended into digital media" [Semenov, 2020] requires thought and the revision of educational practices and methods of measuring educational outcomes.
While reading the text, the participants obtained two different results by using automatic search. For one, the strategy was successful and produced a quick result, while for the other it led to an incomplete solution to the reading task: the actions described in the table prevented the second participant to identify several scientists named hyponymically (anthropologist, biologist). This issue, however, was later solved with the use of the following group of strategies.
3.5. Strategies for monitoring the quality of reading and resolving difficulties that arise
The employment of the strategies for monitoring the quality of reading and resolving difficulties that arise is the most important indicator of a competent reader regardless of the reading medium. These strategies include the following:
• tracking text comprehension;
• going back through the text to resolve comprehension difficulties;
• reducing the reading speed when difficulties in comprehension arise;
• switching to whisper reading when difficulties in comprehension arise;
• consulting the context to resolve comprehension difficulties;
• consulting external sources to resolve comprehension difficulties.
The desire to solve comprehension difficulties often results in a non-linear reading trajectory. For example, the participants first paid attention to the text in larger font, then realized that they did not understand part of the text and, using linguistic cues (in this case, the anaphoric pronoun), returned to the preceding fragment:
"One female came up with this for some reason"... Okay, "this", means there is some information before that. After all, I need to read what came before.
Furthermore, in the course of the study, we observed how the desire to solve comprehension difficulties competed with the desire to pursue the reading task: when confronted with an unclear fragment, the participants tended to skip it, considering it unimportant for answering the question:
"...theological dogma. What complicated words! Okay, that's not important right now".
The reading time was unlimited; participants were asked to read as they would in a natural setting, so there were presumably other reasons for not resolving comprehension difficulties.
To increase understanding, the participants drew on the text, for example, inferring the meaning of words from the context:
They have their own memes... stick a blade of grass in their ear. Apparently, they thought it was funny. Oh, no, I see: "non-biological objects, memes — ideas and technology". Okay, so a meme is some kind of joke that someone came up with and others picked up on.
The readers could also use external sources to increase understanding. For instance, the other participant used the search on the Internet to clarify the meaning of the word "meme".
The text offered to the study participants did not contain many non-verbal components, such as graphs, diagrams, and tables, so we were able to observe only a limited repertoire of strategies from this group, including the following:
• using graphic text design and illustrations to choose a reading path;
3.6. Strategies for using information from the nonverbal components of the text
• relying on illustrations and the arrangement of the text to choose a reading path.
The first strategy was used a lot by both participants. This strategy relies on the reader's prior knowledge that a highlighted fragment is very important for text comprehension:
I saw the text in large print there. I'll read it because it is catchy. It must be something that contains the main ideas, most important ones in the article.
The illustrations also helped the readers to navigate the body of digital text and understand its structure:
Here begins some new part <...> and the first thing that catches the eye is the photo.
3.7. Support Support reading strategies involve actions other than reading (such strategies as writing, retelling, highlighting, etc.) that help readers interact more effectively with the text and understand and memorize what they read.
Our study identified the following support reading strategies:
• taking notes;
• highlighting a text fragment with the cursor.
Other studies have described a wide range of support strategies, in particular 9 types in [Mokhtari, Reichard, 2002; Anderson, 2003]. The limited repertoire employed by the study participants is presumably due to the limitation of the chosen data collection method: despite the instruction to behave as naturally as possible, as if they were performing a real-life learning assignment, the participants were aware of the artificial nature of what was happening. They even mentioned it in their speech:
If I were now writing some text for myself, I would put it somewhere in Word, for example.
We expect to see a wider range of support strategies when the observations are carried out in more natural conditions.
4. Conclusion In the study of the reading process of two competent readers at the age of 14 performing an educational task, we have identified and classified metacognitive strategies for digital reading. Seven
groups of strategies were described and analyzed. Some of them are universal and do not depend on the text format, others are specific to digital reading.
We chose a research design that allowed us to observe how competent ninth-graders were performing reading tasks. Participants did not always use optimal strategies, but they did track and address difficulties that occurred during the reading process. Thus, we modeled the behavior of a proficient reader, and this model can provide the basis for both teacher-assisted reading instruction and automated learning systems.
Below we briefly formulate the main conclusions of the study. This study confirmed the view of digital reading as text construction. The study participants avoided linear, sequential reading from first to last line, preferring nonlinear self-constructed reading trajectories. During the digital reading process, the participants made a lot of decisions about which text fragments to read, how attentively, and for what purpose. Through online observation, we found that the visual arrangement of digital text and the opportunity to scroll quickly supported nonlinear reading.
It was also found that the effectiveness of digital reading depends on the reader's self-control: on their ability to focus on the reading goal and overcome the distractors that hinder the reading process. An important group of digital reading strategies is strategies for the preliminary evaluation of text — of its relevance to the reading task, volume, structure, and general content. An observable indicator of the use of these strategies is quick scrolling through the text from beginning to end before reading.
In the context of digital reading, strategies that delegate some of the reading tasks to technology deserve special attention. Both study participants employed a search reading strategy with automatic page search, the successful application of which requires a range of skills from the reader. In particular, it is important to critically evaluate the results obtained with technology: while delegating, the reader should control how the search is performed.
The participants demonstrated how in the reading process they draw on their experience of reading digital texts and their idea of the typical arrangement of such texts and in-text visual cues. Since the arrangement of digital texts differs significantly from that of print texts, print reading instruction does not always result in a high level of digital reading proficiency [Ortlieb, Sargent, Moreland, 2014]. Consequently, digital reading practice guided by a teacher or other competent reader is important for developing a high level of digital reading proficiency.
The think-aloud method — the research method of this study — seems a promising instruction method. Using the explicit commenting technique, the teacher can model digital reading strategies explic-
itly for students. As evidenced in practice, by applying the think-aloud method in reading class, students are better at solving reading tasks independently, if they have the opportunity to observe an example from a competent adult [Dobler, 2015]. In addition, thinking aloud reinforces the practice of slow conscious reading and can thus prevent distracted reading characteristic of many contemporary readers.
The goal of digital reading instruction for today's students is to provide students with a repertoire of digital reading strategies and develop in them the ability to select from this repertoire those techniques that are useful for solving specific reading tasks.
The results of the present study can also be used to develop instruments for measuring reading literacy in upper secondary school. For example, the observable reader's actions described in the study can serve as the basis for automated reading assessment systems.
The research was conducted with the financial support of the Russian Foundation for Basic Research within the framework of the research project No. 19-2914148 "Text in the digital educational environment: a study of reading strategies and parameters influencing the quality of digital reading among Russian schoolchildren".
The author would like to thank the students and teachers of Moscow schools Nos. 1514 and 2097, Letovo School, the New School, the Gymnasium Named after Podolsk Cadets, and Balashikha Lyceum who participated in the study.
References Afflerbach P.A., Cho B.Y. (2010) Determining and Describing Reading Strategies: Internet and Traditional Forms of Reading. Metacognition, Strategy Use, and Instruction (ed. W. Schneider), NY: Guilford, pp. 201-255. Alexander P.A., Disciplined Reading and Learning Research Laboratory (2012) Reading into the Future: Competence for the 21st Century. Educational Psychologist, vol. 47, no 4, pp. 259-280. doi:10.1080/00461 520.2012.722511 Anderson N. (2003) Scrolling, Clicking, and Reading English: Online Reading Strategies in a Second/Foreign Language. The Reading Matrix, vol. 3, no 3. Anderson N. (1991) Individual Differences in Strategy Use in Second Language Reading and Testing. The Modern Language Journal, vol. 75, no 4, pp. 460472. doi: 10.1111/j.1540-4781.1991.tb05384.x Annisette L.E., Lafreniere K.D. (2017) Social Media, Texting, and Personality: A Test of the Shallowing Hypothesis. Personality and Individual Differences, vol. 115, February, pp. 154-158. doi: 10.1016/j.paid.2016.02.043 Baker L., Brown A.L. (1984) Metacognitive Skills and Reading. Handbook of Reading Research (eds P.D. Pearson, R. Barr, M.L. Kamil, P. Mosenthal), New York: Longman, pp. 353-394. Bereiter C., Bird M. (1985) Use of Thinking Aloud in Identification and Teaching of Reading Comprehension Strategies. Cognition and Instruction, vol. 2, no 2, pp. 131-156. doi:10.1207/S1532690XCI0202_2 Block E. (1986) The Comprehension Strategies of Second Language Readers. TESOL
Quarterly, vol. 20, no 3, pp. 463-493. Bohn-Gettler C.M. (2018) The Complexities of Comprehension: Conducting Think-Aloud Studies with Multiple Interacting Variables. SAGE Research Methods Cases. doi:10.4135/9781526439550
Bohn-Gettler C.M., Kendeou P. (2014) The Interplay of Reader Goals, Working Memory, and Text Structure during Reading. Contemporary Educational Psychology, vol. 39, no 3, pp. 206-219. doi: 10.1016/ j.cedpsych.2014.05.003 Bohn-Gettler C.M., Olson O. (2019) Analyzing Verbal Protocols: Thinking Aloud during Reading in Cognitive and Educational Psychology. London: Sage. doi:10.4135/9781526494306 Burbules N.C., Callister T.A. (1996) Knowledge at the Crossroads: Some Alternative Futures of Hypertext Learning Environments. Educational Theory, vol. 46, no 1, pp. 23-50. doi:10.1111/j.1741-5446.1996.00023.x Clark A., Chalmers D. (1998) The Extended Mind. Analysis, vol. 58, iss. 1, pp. 7-19.
doi:10.1093/analys/58.1.7 Coiro J. (2011) Talking About Reading as Thinking: Modeling the Hidden Complexities of Online Reading Comprehension. Theory Into Practice, vol. 50, no 2, pp. 107-115. doi:10.1080/00405841.2011.558435 Coiro J., Dobler E. (2007) Exploring the Online Reading Comprehension Strategies Used by Sixth-Grade Skilled Readers to Search for and Locate Information on the Internet. Reading Research Quarterly, vol. 42, no 2, pp. 214-257. doi:10.1598/RRQ.42.2.2 Delgado P., Salmerón L. (2021) The Inattentive On-Screen Reading: Reading Medium Affects Attention and Reading Comprehension under Time Pressure. Learning and Instruction, vol. 71, Article no 101396. doi:10.1016/j.learnin-struc.2020.101396
Delgado P., Vargas C., Ackerman R., Salmerón L. (2018) Don't Throw away Your Printed Books: A Meta-Analysis on the Effects of Reading Media on Reading Comprehension. Educational Research Review, vol. 25, November, pp. 23-38. doi:10.1016/j.edurev.2018.09.003 Dobler E. (2015) E-Textbooks: A Personalized Learning Experience or a Digital Distraction? Journal of Adolescent & Adult Literacy, vol. 58, no 6, pp. 482-491. Hillesund T. (2010) Digital Reading Spaces: How Expert Readers Handle Books, the Web and Electronic Paper. First Monday, vol. 15, no 4. doi:10.5210/ fm.v15i4.2762
Krippendorp K. (2004) Content Analysis: An Introduction to its Methodology. Thousand Oaks, CA: Sage. Latini N., Braten I. (2021) Strategic Text Processing Across Mediums: A Verbal Protocol Study. Reading Research Quarterly, May, pp. 1-22. doi:10.1002/rrq.418 Latini N., Braten I., Anmarkrud 0., Salmerón L. (2019) Investigating Effects of Reading Medium and Reading Purpose on Behavioral Engagement and Textual Integration in a Multiple Text Context. Contemporary Educational Psychology, vol. 59, July, Article no 101797. doi:10.1016/j.cedpsych.2019.101797 Lebedeva M.Yu., Kupreshchenko O.F., Berlin Khenis A.A., Veselovskaya T.S. (2021) Skrolly, kliki i tsifrovaya didaktika: opyt sbora dannykh o pol'zovatel'skom po-vedenii dlya issledovaniya chteniya shkol'nikami tsifrovykh uchebnykh teks-tov [Scrolls, Ckicks and Digital Didactics: Experience in Collecting User Behavior Data for a Study of School Students' Digital Reading]. Proceedings of the V International Scientific Conference "Informatization of Education and Methods of e-Learning: Digital Technologies in Education" (Krasnoyarsk, 2020, 6-9 October), Krasnoyarsk: Siberian Federal University, pp. 187-191. Lebedeva M.Yu., Vergeles K.P., Kupreshchenko O.F., Zhiltsova L.Yu., Veselovskaya T.S. (2020) Faktory vybora tsifrovogo formata dlya chteniya i samootsenka chitatel'skogo povedeniya v tsifrovoy srede (po dannym oprosa rossiyskikh shkol'nikov) [The Russian Schoolchildren's Digital Reading: Factors Affecting Medium Preferences and Self-Evaluation of Digital Reading Practice]. Science for Education Today, vol. 10, no 6, pp. 252-272. doi:10.15293/2658-6762.2006.14 Leslie L., Caldwell J. (2009) Formal and Informal Measures of Reading Comprehension. Handbook of Research on Reading Comprehension (eds S.E. Israel, G.G. Duffy), New York: Routledge, pp. 403-427.
Liu Z. (2005) Reading Behavior in the Digital Environment: Changes in Reading Behavior Over the Past Ten Years. Journal of Documentation, vol. 61, no 6, pp. 700-712. doi:10.1108/00220410510632040 Magliano J.P., Millis K.K. (2003) Assessing Reading Skill with a Think-Aloud Procedure and Latent Semantic Analysis. Cognition and Instruction, vol. 21, no 3, pp. 251-284. doi:10.1207/S1532690XCI2103_02 Magliano J.P., Millis K., Ozuru Y., McNamara D.S. (2007) A Multidimensional Framework to Evaluate Reading Assessment Tools. Reading Comprehension Strategies: Theories, Interventions, and Technologies (ed. D.S. Mcnamara), New York, London: Taylor and Francis Group, pp. 107-134. Magliano J.P., Millis K., R-SAT Development Team, Levinstein I., Boonthum C. (2011) Assessing Comprehension during Reading with the Reading Strategy Assessment Tool (RSAT). Metacognition and Learning, vol. 6, no 2, pp. 131-154. doi:10.1007/s11409-010-9064-2 Magliano J.P., Trabasso T., Graesser A.C. (1999) Strategic Processing during Comprehension. Journal of Educational Psychology, vol. 91, no 4, pp. 615-629. doi:10.1037/0022-0663.91.4.615 McNamara D., Kendeou P. (2011) Translating Advances in Reading Comprehension Research to Educational Practice. International Electronic Journal of Elementary Education, vol. 4, no 1, pp. 33-46. McNamara D.S., McDaniel M.A. (2004) Suppressing Irrelevant Information: Knowledge Activation or Inhibition? Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 30, no 2, pp. 465-482. doi:10.1037/0278-7393.30.2.465
McNamara D.S., O'Reilly T.P., Best R.M., Ozuru Y. (2006) Improving Adolescent Students' Reading Comprehension with Istart. Journal of Educational Computing Research, vol. 34, no 2, pp. 147-171. doi:10.2190/1RU5-HDTJ-A5C8-JVWE McNamara D.S., Ozuru Y., Best R., O'Reilly T. (2007) The 4-Pronged Comprehension Strategy Framework. Reading Comprehension Strategies: Theories, Interventions, and Technologies (ed. D.S. McNamara), New York: Lawrence Erlbaum Associates, pp. 465-496. Millis K.K., Magliano J.P., Todaro S. (2006) Measuring Discourse-Level Processes with Verbal Protocols and Latent Semantic Analysis. Scientific Studies of Reading, vol. 10, no 3, pp. 225-240. doi:10.1207/s1532799xssr1003_2 Mokhtari K., Reichard C. (2002) Assessing Students' Metacognitive Awareness of Reading Strategies. Journal of Educational Psychology, vol. 94, no 2, pp. 249259. doi:10.1037/0022-0663.94.2.249 Mulcahy-Ernt P., Caverly D. (2009) Strategic Study-Reading. Handbook of College Reading and Study Strategy Research (eds R.F. Flippo, D.S. Caverly), New York: Routledge, pp. 177-198. Mullis I.V., Martin M.O. (eds) (2019) PIRLS 2021 Assessment Frameworks. Available at: http://pirls2021.org/wp-content/uploads/sites/2/2019/04/P21 Frameworks. pdf (accessed 20 lanuary 2022). OECD (2019) PISA 2018. Assessment and Analytical Framework. Paris: OECD.
doi:10.1787/b25efab8-en Oganov S.R., Kornev A.N. (2017) Okulomotornye kharakteristiki kak pokazatel' sformirovannosti navyka analiza pis'mennogo teksta u detey 9-11 i 12-14 let [Oculomotor Characteristics as Indicators of Written Text Analysis Skills Formation in Children Aged 9-11 and 12-14]. Special Education, no 3, pp. 112-121. Ortlieb E., Sargent S., Moreland M. (2014) Evaluating the Efficacy of Using a Digital Reading Environment to Improve Reading Comprehension within a Reading Clinic. Reading Psychology, vol. 35, no 5, pp. 397-421. doi:10.1080/02702 711.2012.683236
Pagendarm M., Schaumburg H. (2001) Why Are Users Banner-Blind? The Impact of Navigation Style on the Perception of Web Banners. Journal of Digital Information. Vol. 2. No 1.
Polukhina E.V. (2014) Onlain-nablyudenie kak metod sbora dannykh [Online Observation as a Method for Collecting Data]. Interaction. Interview. Interpretation, no 7, pp. 95-106.
Prantsova G.V., Romanicheva E.S. (2015) Sovremennye strategii chteniya: teoriya i praktika. Smyslovoe chtenie i rabota s tekstom [Modern Reading Strategies: Theory and Practice. Semantic Reading and Working with Text]. Moscow: Forum.
Ryabinina L.A., Chaban T.Yu. (2019) Monitoring chitatel'skoy gramotnosti: regionally opyt [Monitoring of Reading Literacy: Regional Experience]. Otechestven-naya i Zarubezhnaya Pedagogika, vol. 1, no 4 (61), pp. 236-247.
Salmeron L., Canas J.J., Kintsch W., Fajardo I. (2005) Reading Strategies and Hypertext Comprehension. Discourse Processes, vol. 40, no 3, pp. 171-191. doi:10.1207/s15326950dp4003_1
Salmerón L., Naumann J., Garcia V., Fajardo I. (2017) Scanning and Deep Processing of Information in Hypertext: An Eye Tracking and Cued Retrospective Think-Aloud Study. Journal of Computer Assisted Learning, vol. 33, no 3, pp. 222-233. doi:10.1111/jcal.12152
Semenov A.L. (2020) Rezul'tativnoe obrazovanie rasshirennoy lichnosti v prozrach-nom mire na tsifrovoy platforme [Productive Education of Extended Human in the Transparent World on Digital Platform]. The Herzen University Conference on Psychology in Education, no 3, pp. 590-596. doi: 10.33910/herzenpsy-conf-2020-3-27
Singer L.M., Alexander P.A. (2017) Reading on Paper and Digitally: What the Past Decades of Empirical Research Reveal. Review of Educational Research, vol. 87, no 6, pp. 1007-1041. doi: 10.3102/0034654317722961
Smetannikova N.N. (2019) Obuchenie professional'no-spetsializirovannomu cht-eniyu v svete teorii innovatsiy [Teaching Professionally-Specialized Reading in the Light of Innovation Theory]. World of Science. Pedagogy and Psychology, no 1. Available at: https://mir-nauki.com/PDF/53PDMN119.pdf (accessed 20 January 2022).
Smetannikova N.N. (2018) Strategial'ny podkhod k obucheniyu profession-al'no-spetsializirovannomu chteniyu [A Strategic Approach to Teaching Professionally Specialized Reading]. Higher Education Today, no 9, pp. 54-59. doi: 10.25586/RNU.HET.18.09.P.54
Thain M. (2018) Distracted Reading: Acts of Attention in the Age of the Internet. Digital Humanities Quarterly, vol. 12, no 2. Available at: http://digitalhuman-ities.org/dhq/vol/12/2/000393/000393.html (accessed 20 January 2022).
Wang Y.H. (2016) Reading Strategy Use and Comprehension Performance of More Successful and Less Successful Readers: A Think-Aloud Study. Educational Sciences: Theory and Practice, vol. 16, no 5, pp. 1789-1813. doi:10.12738/ estp.2016.5.0116
Weber R.P. (1990) Basic Content Analysis. Thousand Oaks, CA: Sage.
Yayli D. (2010) A Think-Aloud Study: Cognitive and Metacognitive Reading Strategies of ELT Department Students. Eurasian Journal of Educational Research, iss. 38, pp. 234-251.
Zalevskaya A.A., Kaminskaya E.E., Medvedeva I.L., Rafikova N.V. (1998) Slovo na puti k ponimaniyu teksta [A Word on the Way to Understanding the Text] Psik-holingvisticheskie aspekty vzaimodeystviya slova i teksta [Psycholinguistic Aspects of Word-Text Interaction]. Tver: Tver State University, pp. 4-46
Zhang M. (2012) Supporting Middle School Students' Online Reading of Scientific Resources: Moving beyond Cursory, Fragmented, and Opportunistic Reading. Journal of Computer Assisted Learning, vol. 29, no 2, pp. 138-152. doi:10.1111/ j.1365-2729.2012.00478.x
Zuckerman G.A. (2010) Otsenka chitatel'skoy gramotnosti. Materialy k obsuzhdeni-yu [Assessment of Reader Literacy. Materials for Discussion]. Moscow: Center for Education Quality Assessment. Available at: http://2020strategy.ru/ data/2011/07/15/1214720557/4.pdf (accessed 20 January 2022).
