CC BY
21
PSYCHOPHYSIOLOGICAL MARKERS OF EPISTEMIC EVALUATION: EVIDENCE FROM EYE TRACKING IN READING FAMILIAR AND UNFAMILIAR WORDS
V.A. Demareva1*, A.V. Golubinskaya1, T.V. Mayasova2, V.V. Vyakhireva1, M.V. Zhukova1, R.V. Golubin1
1 Lobachevsky State University (Nizhny Novgorod, Russia);
2 Minin Nizhny Novgorod State Pedagogical University (Nizhny Novgorod, Russia).
* Corresponding author: valeriia.demareva@fsn.unn.ru
Abstract. The article focuses on identifying specific eye movement parameters during reading unfamiliar words to study the formation of epistemic evaluation. The sample consisted of 40 students. 240 eye-tracking records were registered while the participants were reading 6 texts in Russian. The study revealed that the speed of saccades decreased, and the duration of fixations increased while reading an unknown word. Eye movements at first encounter of a new term may be an indicator of epistemic evaluation formation.
Keywords: epistemic evaluation, cognitive-oriented models, fixations, saccades, reading, eye-tracking.
List of Abbreviations
SAS - Serial Attention Shift
GAG - Guidance by Attentional Gradients
SWIFT - Saccade-generation With Inhibition by Foveal Targets
EMMA - Eye Movements and Movement of Attention
Introduction
Epistemic evaluation (from the Greek epis-teme - knowledge) describes a state of a person when they process information. In logics, this term refers to a subject's attitude to a judgemental statement, expressed by the terms «to know», «to believe», «to be convinced», «to doubt», etc. In psychology, research on epis-temic evaluation involves analysis of the cognitive mechanisms by which a person assigns the status of «true», «trustworthy», «likely», «doubtful», «false», and experiences specific cognitive events (agreement, concentration, thoughtfulness, certainty, and interest) to information new to him (Rahman et al., 2020).
A key element of such mechanisms is the peculiar procedure that humans perform to assess the validity of claims (epistemic evaluation). Epistemic evaluations can be deliberate, when they occur as a part of purposive reasoning, or accidental, as well as direct, when humans ana-
lyze a claim, and indirect, when humans evaluate the validity of conclusions reached by third parties. There can also be other typologies. The most elaborate one is the differentiation of epistemic evaluations by subject, that is, what exactly the person's reasoning is aimed at (e.g., reasons for typical and atypical events, facts, etc.) (Kirefel et al., 2021; Henne et al., 2021).
However, in this paper, it is appropriate to focus on direct epistemic evaluations. Unlike indirect evaluation, which is influenced by social factors (e.g., the authority of a third party), direct epistemic evaluation is most susceptible to psycho-physiological measurements, because dealing with new information involves parts of the sensory system responsible for perception. The most frequent perceptual source of new information for humans is reading. It is easy to see that purely lexical parameters of a text are not sufficient to explain individual epistemic processes: often different people come to different conclusions after reading the same text (Chertakova et al., 2021). Consequently, this difference is due to some internal factors of epistemic evaluations.
There are several concepts of epistemic evaluation formation that consider different internal factors (Perrine, 2020). Among them there are cognitive and non-cognitive concepts. From the point of view of the somatic markers hypothesis
formulated by A. Damasio, epistemic states depend on emotional events arising in the process of reasoning (Damasio, 1996). This is an example of a non-cognitive concept. Proponents of cognitive concepts, which argue that epistemic evaluations depend on cognitive factors, have not yet reached a compromise on the nature of these very factors. Fundamentalist and coher-entist approaches suggest that epistemic evaluations are determined by the conformity to previously existing beliefs (Kole & Eshimome, 2021). The probabilistic approach assumes that the most significant internal factor in the process of evaluation is the determination of possibilities (Loewer, 1998). In this approach, any epistemic evaluations are represented as degrees satisfying probability axioms that are formed in the process of assigning a percentage of probability information.
A major difficulty with these and other approaches is the inaccessibility of the epistemic process and any of its markers to the observer. Some tentative alternative is the relational approach, which assumes that a significant factor for epistemic estimation is the search for connections (Kozlova et al., 2021). We did not find literature devoted to a detailed description of such an approach, however, its basic theoretical positions can be traced in various sources. For example, Wittgenstein's «wheel,» a metaphor for the structure of human knowledge: «a wheel that doesn't spin though anything else moves with it» is not part of a mechanism; knowledge forms a kind of system, like a mechanism with gears, the function of which must manifest itself in the word's performance (Wittgenstein, 2009). A similar idea is expressed in E. Yudkowsky as follows: what does not receive connections with other knowledge is stored in memory in isolation (Yudkowsky, 2015). This leads to the question of the importance of connections for epistemic evaluation. If it is indeed a factor in the processing of new information, for example, when reading, then such reading must be accompanied by specific fixations of the eye on those words and text fragments that are the subject of epistemic evaluation.
In psychophysiology, there are several related statements. The process of reading is sus-
tained by foveal and parafoveal vision. Foveal vision ensures the perception of letters fixated by the gaze (carried out by receptors in the central fossa of the retina and characterized by maximum visual acuity). Parafoveal vision provides the perception of non-fixated letters (Va-silyeva, 2019). With a proper degree of background knowledge of information in texts to be read, some part of the verbal information is processed parafoveally (Vasilev & Angele, 2017). Parafoveal preview of a word («parafoveal preview») reduces its subsequent fixation time -this effect is called «parafoveal preview bene-fit» (Niefind & Dimigen, 2016; Vasilev et al., 2020) and is taken account of by most current models of eye movement control during reading (hereafter «reading models»).
Reading models describe the cognitive and/or oculomotor processes that control the reading process. They are precise algorithms that contain mathematical formulas and rely on a large amount of empirical data. There are different versions of the models (Reingold et al., 2016). In the current contribution, cognitive-oriented models will be considered that argue that eye movements are controlled by processes of lexical processing, and take visual-motor factors into account. In turn, these models are divided into two main types: Sequential Attention Shift (SAS) (Hans & Engbert, 2012) with implementations of E-Z Reader (Reichle & Sheridan, 2015), EMMA (Salvucci, 2000), etc.; Guidance by Attentional Gradients (GAG) with implementations of SWIFT (Engbert et al., 2005), Glenmore (Reilly & Radach, 2002), etc.
In SAS models, lexical word processing is closely related to a sequential shift in attention from word to word; words are processed in the order in which they appear in the text; only one word can be the focus of attention (Risse & Seelig, 2019). SAS models assume a close relationship between lexical processing, attention shifts, and eye movements. These models rely on the assumption that a latent shift of attention to the next word occurs simultaneously with saccade programming to that word. At the same moment, lexical processing of that (parafoveal) word begins. If the processing of the parafoveal word is completed before the saccade is per-
formed (which is quite likely for simple and predictable words), the saccade is repro-grammed to another word following the skipped word. This mechanism explains selective skipping of short, high-frequency words.
In 1998, Reichle and colleagues suggested that: (1) lexical processing occurs in two steps (word familiarity assessment and subsequent word search in the lexicon), and (2) saccade programming also occurs in two steps (Reichle et al., 1998). This article begins the description of the best-known reading model, the E-Z Reader (Madi et al., 2020). In this model, saccade programming to the next word and lexical processing of the current word occurs simultaneously from the moment word familiarity check is completed. Saccade reprogramming is possible only in the first phase. A latent shift of attention to the next word occurs after the completion of lexical processing, regardless of the saccadic programming phase. Partial independence of the lexical and visual-motor programs goes beyond Morrison's model and explains average fixation durations and the probability of missing words in connection with word frequency, as well as the negative effect of the difficulty of processing a foveal word on the subsequent processing of a parafoveal word (in this case a parafoveal word takes longer to process than the same word that follows an easy (e.g., high-frequency) foveal word). In the next version, the model is expanded to include the parameters of visualmotor control and thereby - to consider achievements of visual-motor models.
In GAG models it is possible to process several words in parallel, the attention is not focused on one word, but distributed throughout the text perception area according to the gradient principle, so that the closer a word is to the foveal area, the higher its processing speed, i.e., words in the parafoveal area are processed slower. GAG models include the best-known model SWIFT. SWIFT is a dynamic systematic approach to eye movement control in reading, based on the concept of spatially distributed or parallel word processing. One of the main motivations for developing SWIFT was to propose
one common mechanism to describe all types of saccades observed in reading experiments, i.e., right-hand forward saccades, word skips, repeated fixations, and regressions. In addition, as a key principle borrowed from neurophysio-logical work, SWIFT separates the «when» and «where» saccade preparation pathways (Findlay & Walker, 1999). Saccades according to this model are generated autonomously (Godfroid, 2021).
Empirical data for the construction and verification of reading models are collected using eye-tracking which registers eye movements. Modern devices allow to accurately measure the duration of fixations and saccades during reading, to fix the direction of gaze, and to trace the trajectory of saccades. Experiments with eye-tracking revealed basic patterns of eye movements during reading. Reading is characterized by a sawtooth pattern of sequentially scanning text from left to right and from top to bottom or in the opposite direction, depending on the writing system. Eye movements are an alternation of stationary fixations and saccades, whose ratio of durations depends on multiple factors. One saccade is approximately equal to 8 symbols (Clifton et al., 2016). There can be different numbers of fixations on a word. During a saccade, as well as immediately before and after, it is almost impossible to perceive any visual information. Information enters the processing system predominantly during fixations, and during saccades there is low-level processing of visual information (Fabius et al., 2020).
Thus, there is a suggestion that by analyzing eye movements when a person is introduced to new information (new terms), it can be established whether visual seeking is an essential element in the process of the formation of epistemic evaluation of information.
The aim of this article was to identify eye movement patterns that are indicative of epis-temic evaluation associated with the reading of new terms.
To achieve the goal, an empirical study was conducted to record eye movements while reading texts in which adult respondents marked terms previously unfamiliar to them.
Materials and Methods
Forty Russian-speaking subjects aged 18 to 22 (12 males and 28 females) participated in the study.
The study consisted of two stages. At the first stage, eye movements while reading texts were recorded. Eye movements were recorded using the SMI HiSpeed eye-tracking system. The data on eye-motor activity parameters were extracted using SMI BeGaze 3.4 program (Event Statistics module), and were processed using StatSoft Statistica v. 13.0 Eng.
Six texts in Russian (containing 32, 36, 38, 43, 31, 33 words respectively) were used as stimulus material.
Excerpts of texts were taken from the following sources: Yarovitsky V. «100 great psychologists», Solso R. «Cognitive psychology», Berdyaev N. «Science of religion and Christian apologetics», «Igneous rocks - plutonites and vein rocks formed because of magma breakthrough into the Earth's crust», Gudilin E.A. Shlyakhtin O.A. «Actuator», Arzamasova O. A. «Carbohydrate-protein complexes of liver and blood serum in prenatally alcoholized rats».
At the second stage, after completing the recording of eye movements, the subjects were given the same texts on paper, in which it was necessary to highlight unfamiliar words.
Further statistical processing of the received data was carried out. Methods of descriptive statistics and analysis of the reliability of differences using Student paired t-test were used.
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of Faculty of Social Sciences of Lobachevsky State University.
Results
1. Identification of unfamiliar words in texts
240 eye tracking recordings were registered while reading six texts in Russian.
The subjects identified the following words as unfamiliar - Table 1.
Then, 105 eye movement recordings were selected where the subject was reading an unfamiliar word: unfamiliar words were high-lighted, which were preceded or followed by 3 words familiar to the subject, and which were read for the first time, since some unfamiliar words were repeated several times in the text. The fixation durations, duration, amplitude, and average speed of saccades when reading the unfamiliar words, as well as the first, second, and third word preceding or following the unknown word, were analyzed.
Statistically significant differences in the duration of fixations on words in Russian texts and differences in the duration, amplitude, and average speed of saccades were calculated.
2. Comparison of eye movement patterns when reading familiar and unfamiliar words
2.1. Fixation features analysis
The following patterns were found when an unfamiliar word occurs within a text (Fig. 1). A longer fixation is made for the word which immediately precedes the unknown word than for the second word to the left (t = 2.4;p < 0.05). On the third subsequent word after the unfamiliar word, a shorter fixation is made compared to the following words: the word before the unfamiliar word (t = 2.2; p < 0.05), the unfamiliar word (t = = 2.5; p < 0.05), the first after the unfamiliar word (t = 2.1; p < 0.05), the second after the unfamiliar word (t = 2.5; p < 0.05).
2.2. Saccade feature analysis
No statistical difference in the duration and amplitude of saccades during reading was found.
The following regularities were found in the presence of an unfamiliar word in Russian texts (Fig. 2). The average saccade speed before the unknown word was higher than during its reading (t = 2.5;p < 0.05) and after reading (t = 2.4; p < 0.05).
Table 1
List of unfamiliar words, marked by the participants
No. A word marked as unfamiliar (in Russian) A word marked as unfamiliar (English translation)
1 «кремнозема» «silica»
2 «диорит» «diorite»
3 «габбро» «gabbro»
4 «пренатальной» «prenatal»
5 «интенциональности» «intentionality»
6 «актуаторов» «actuators»
7 «гексоз» «hexosis»
8 «мукопротеинов» «mucoproteins»
9 «нейрокогнитологией» «neurocognitology»
10 «гликопротеинов» «glycoproteins»
300
250
200 - -
c
o _
• -
cn
^ 150 ~o c o
<o 100
iff
50 0
-3 -2 -1 unfamiliar +1 +2 +3
word
Fig. 1. Mean duration of fixations on familiar and unfamiliar words
100
90 -
80 70 50 50 40 30 20 10 0
before unfamiliar word after
Fig. 2. Average saccade speed before, during, and after reading an unfamiliar word
Discussion
Understanding represents only one of the dimensions that are involved in the processes of epistemic evaluation. At the same time, it is one of the primary preconditions for a person to to make any kind of evaluation at all. It follows, that peculiarities of statement processing at the stage of primary evaluation of information can affect the result of cognitive and, consequently, epistemic situation. The above results show that the area of the text, within which the search for connections between familiar and unfamiliar terms is manifested, turns out to be relatively small, but its size is regular, and it can be determined in accordance with the modern models of reading.
The study found that the more complex the words are, the longer the fixation time is, which is consistent with the results of other studies (Loberg et al., 2019). Such characteristics as frequency, predictability, etc., which are cognitive/lexical properties of a word that affect fixation duration, are estimated differently in the E-Z Reader and SWIFT models. In the E-Z Reader these characteristics control eye movements during reading (Reichle & Sheridan, 2015), in the SWIFT model saccades are generated autonomously (Rabe et al., 2021). Thus, we can conclude that, in our case, the results obtained fit best into the SWIFT model.
According to the E-Z Reader and SWIFT models of reading, eye movements during reading are controlled by the processes of lexical word processing. The main difference between these two models is that lexical processing in E-Z Reader is sequential (Reichle & Sheridan, 2015), while in SWIFT it is parallel (simultaneous processing of multiple words) (Rabe et al., 2021; Godfroid, 2021). Concerning the saccades programming, it was found that the statistical difference was only in the average speed: the speed decreases during the reading of an unknown word.
Figure 2 shows that the speed on the unfamiliar word decreases significantly. Perhaps this is one of the subjects' strategies of seeing an unfamiliar word in advance when reading in their native language and immediately determining more time to review it. In addition, dec-
reased saccade rate on an unknown word is indicative of new information processing, making connections between the words, and thus finding subjects in a state of epistemic evaluation. It is likely that when programming a saccade, there was simultaneous processing of several words, and this may have affected the speed of the subsequent saccade, which confirms the SWIFT model (Rabe et al., 2021; Godfroid, 2021).
There is quite a large empirical base supporting the performance of each of the two models described. However, there are discussions about which of the models is the most universal, and the search and development of the most perfect reading model is underway. It can be concluded that in this study, during the study of the gaze movement when reading unfamiliar words, the most appropriate model, which best explains the results obtained, is the SWIFT model.
Conclusion
The aim of the current research study was to determine the characteristics of eye movements when reading unfamiliar words in the context of studying the process of epistemic evaluation formation. The analysis of the obtained data revealed that the speed of saccades decreased, and the duration of fixations increased during the reading of an unfamiliar word.
Overall, the data obtained regarding the dynamics of saccades and fixations characteristics agree with the SWIFT model and testify to the simultaneous processing of several words, and this means that our assumption, that the analysis of eye movements when a person is familiar with new information (new terms) reveals that the search for associations is an essential element for epistemic evaluation formation, is confirmed.
Acknowledgements
The study was supported by the grant of the President of the Russian Federation for governmental support of young Russian scientists -candidates of sciences N МК-6208.2021.2. The authors thank Anastasia Vervai for her participation in data collection.
References
CHERTAKOVA EM, LAPSHOVA A.V, VAGANOVA O.I, BULAEVA M.N. & BYSTROVA N.V. (2021): Formation of students' ability to cognitive activity in the learning process. Propositos y representaciones 9, 217-230. http://dx.doi.org/10.20511/pyr2021.v9nSPE1.919.
CLIFTON C., FERREIRA F., HENDERSON J. M., INHOFF A.W., LIVERSEDGE S.P., REICHLE ED. & SCHOTTER E.R. (2016): Eye movements in reading and information processing: Keith Rayner's 40 year legacy. Journal of Memory and Language 86, 1-19. https://doi.org/10.1016/j.jml.2015.07.004.
DAMASIO A.R. (1996): The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences 351(1346), 1413-1420. https://doi.org/10.1098/RSTB.1996.0125.
ENGBERT R., NUTHMANN A., RICHTER E., & KLIEGL R. (2005): SWIFT: A dynamical model of saccade generation during reading. Psychological Review 112, 777-813. https://doi.org/10.1037/0033-295X.112.4.777.
FABIUS J. H., FRACASSO A., ACUNZO D. J., VAN DER STIGCHEL S. & MELCHER D. (2020): Low-level visual information is maintained across saccades, allowing for a postsaccadic handoff between visual areas. Journal of Neuroscience 40(49), 9476-9486. https://doi.org/10.1523/JNEUR0SCI.1169-20.2020.
FINDLAY J.M. & WALKER R. (1999): A model of saccade generation based on parallel processing and competitive inhibition. Behavioral and Brain Sciences 22(4), 661-674. https://doi.org/10.1017/ s0140525x99002150.
GODFROID A. (2021): Eye tracking in second language acquisition and bilingualism: A research synthesis and methodological guide. Routledge: New York, 2020. 414 pp. https://doi.org/10.1111/ijal.12330.
HENNE P., KULESZA A., PEREZ K. & HOUCEK A. (2021): Counterfactual thinking and recency effects in causal judgment. Cognition 212, 104708. https://doi.org/10.1016/j.cognition.2021.104708.
KIRFEL L. & LAGNADO D. (2021): Causal judgments about atypical actions are influenced by agents' epistemic states. Cognition 212, 104721. https://doi.org/10.1016/j.cognition.2021.104721.
KOLE J.A. & ESHIMIOME I.P. (2021): Foundationalism and the Project of Epistemic Justification. Sapien-tia: Journal of Philosophy 14, 1-11.
KOZLOVA E.P., MOROZOVA G.A., KUZNETSOV V P., ROMANOVSKAYA E.V. & ANDRYASHINA N.S. (2020): Interaction of Human and Smart Technologies in Modern Conditions. In: Institute of Scientific Communications Conference, pp. 609-618, Springer, Cham.
LOBERG O., HAUTALA J., HÄMÄLÄINEN JA. & LEPPÄNEN P.H.T. (2019): Influence of reading skill and word length on fixation-related brain activity in school-aged children during natural reading. Vision Res 165, 109-122. https://doi.org/10.1016Zj.visres.2019.07.008.
LOEWER B. (1998): Probability theory and epistemology. In: Routledge Encyclopedia of Philosophy (Ed. E. Craig), pp. 6880-6885, Routledge, London and New York.
MADI N.A., PETERSON C.S, SHARIF B. & MALETIC J. (2020): Can the E-Z Reader Model Predict Eye Movements Over Code? Towards a Model of Eye Movements Over Source Code. Association for Computing Machinery 57, 1-4. https://doi.org/10.1145/3379156.3391983.
NIEFIND F. & DIMIGEN O. (2016): Dissociating parafoveal preview benefit and parafovea-on-fovea effects during reading: A combined eye tracking and EEG study. Psychophysiol 53, 1784-1798. https: //doi.org/10.1111/psyp.12765.
PERRINE T. (2020): Strong internalism, doxastic involuntarism, and the costs of compatibilism. Synthese 197(7), 3171-3191. https://doi.org/10.1007/s11229-018-1879-4.
RABE MM., CHANDRA J., KRÜGEL A., SEELIG S.A., VASISHTH S. & ENGBERT R. (2021): A Bayes-ian approach to dynamical modeling of eye-movement control in reading of normal, mirrored, and scrambled texts. Psychological Review 128(5), 803-823. https://doi.org/10.1037/rev0000268.
RAHMAN A., ANAM A.I. & YEASIN M. (2020): Robust modeling of epistemic mental states. Multimedia Tools and Applications 79(47), 35785-35809. https://doi.org/10.1007/s11042-020-09145-5.
REICHLE E D. & SHERIDAN H. (2015): E-Z Reader: An overview of the model and two recent applications. In: The Oxford handbook of reading (Eds. A. Pollatsek & R. Treiman), pp. 277-290, Oxford University Press.
REICHLE E.D., POLLATSEK A., FISHER D.L. & RAYNER K. (1998): Toward a model of eye movement control in reading. Psychological review 105(1), 125. https://doi.org/10.1037/0033-295x.105.1.125.
REILLY R.G. & RADACH R. (2002): Glenmore: an interactive activation model of eye movement control in reading. In: Proceedings of the 9th International Conference on Neural Information Processing 3, pp. 1194-1200. https://doi.org/10.1109/ICONIP.2002.1202810.
REINGOLD E.M., SHERIDAN H., MEADMORE K.L., DRIEGHE D. & LIVERSEDGE S.P. (2016): Attention and eye-movement control in reading: the selective reading paradigm. Journal of Experimental Psychology: Human Perception and Performance 42(12), 2003. https://doi.org/10.1037/xhp0000291.
RISSE S. & SEELIG S. (2019): Stable preview difficulty effects in reading with an improved variant of the boundary paradigm. Quarterly Journal of Experimental Psychology 72(7), 1632-1645. https: //doi.org/10.1177/1747021818819990.
SALVUCCI D.D. (2000): A model of eye movements and visual attention. In: Proceedings of the Third International Conference on Cognitive Modeling (Eds. N. Taatgen, & J. Aasman), pp. 252-259, Veenen-dal, The Netherlands: Universal Press.
SEELIG S.A., RABE MM., MALEM-SHINITSKI N., RISSE S., REICH S. & ENGBERT R. (2020): Bayes-ian parameter estimation for the SWIFT model of eye-movement control during reading. Journal of Mathematical Psychology 95, 102313 https://doi.org/10.1016/jjmp.2019.102313.
TRUKENBROD H.A. & ENGBERT R. (2012): Eye movements in a sequential scanning task: Evidence for distributed processing. Journal of Vision 12(1), 5. https://doi.org/10.1167/12.L5.
VASILEV M.R. & ANGELE B. (2017): Parafoveal preview effects from word N + 1 and word N + 2 during reading: A critical review and Bayesian meta-analysis. Psychon Bull Rev 24, 666-689. https: //doi.org/10.3758/s13423-016-1147-x.
VASILEV M.R., YATES M., PRUEITT E. & SLATTERY T.J. (2021): Parafoveal degradation during reading reduces preview costs only when it is not perceptually distinct. Quarterly Journal of Experimental Psychology 74(2), 254-276. https://doi.org/10.1177/1747021820959661.
VASILEVA M.D. (2019): Paraphoveal processing: the effects of word morpheme structure. Journal of Psy-cholinguistics 1(39), 206-231. https://doi.org/10.30982/2077-5911-2019-39-1-206-231.
WITTGENSTEIN L. (2009): Philosophical investigations. John Wiley & Sons, 592 pp.
YUDKOWSKY E. (2015): Rationality: from AI to zombies. Berkeley, MIRI, 1813 pp.
