CC BY
127
Psychology in Russia: State of the Art Volume 12, Issue 1, 2019
Russian I Lomonosov
Psychological |MlJi . A . Moscow State Society University
Eye Movements and Word Recognition during Visual semantic search: Differences between expert and Novice Language Learners
Irina V. Blinnikovaa*, Maria D. Rabesona, Anna I. Izmalkovaa
a Lomonosov Moscow State University, Moscow, Russia * Corresponding author. E-mail: [email protected]
Background. The empirical studies in visual word recognition done over the past years have been focused on the influence of contextual, lexical, and semantic properties. Researchers also have taken into consideration the role of individual differences in the word recognition process, e.g., vocabulary knowledge.
objective. This study focuses on the cognitive strategies used by expert and novice language learners in a visual semantic search task. Our hypothesis is that the level of ESL (English as a Second Language) mastery would influence the word recognition and oculomotor patterns applied by the participants.
Design. The participants-native Russian speakers-were divided into three groups according to their level of English language mastery. The experimental task involved a search for horizontally- or vertically-oriented English words in letter matrices (15*15); the frequency and length of the words varied. Performance measures (number and orientation of the found words) were registered, along with the participants' eye movements.
Results. Word search efficiency depended on the frequency, length, and orientation of the words and the participant's language mastery; however, these factors did not interact. The data show that oculomotor events are denser in experts' results. Learners with different levels of language mastery use different information-processing patterns, which are reflected in the proportions of fixation and saccade durations. Two complementary trends were found: word search efficiency is effected, first, by a longer gaze scan path, and second, by the focal mode of visual information-processing, manifested in a combination of longer fixations and shorter saccades.
conclusion. The registration of eye-movement patterns in visual semantic search tasks reveals the characteristics of effective and non-effective cognitive strategies used by ESL students at different levels of language competence.
Keywords:
visual semantic
search, word
recognition,
expert and
novice language
learners,
SL competence,
eye-movement,
cognitive
strategies,
visual
information processing.
ISSN 2074-6857 (Print) / ISSN 2307-2202 (Online) © Lomonosov Moscow State University, 2019 © Russian Psychological Society, 2019 http://psychologyinrussia.com
Introduction
Visual word recognition is a dynamic field in which there have been a lot of broad theoretical developments and rich empirical literature (Yap & Balota, 2015). The empirical research in visual word recognition has focused on the influence of contextual, lexical, and semantic properties (Brysbaert & New, 2009; Pexman, 2012). A more recent approach takes into consideration the role of individual differences in the word recognition process, e.g. vocabulary knowledge (Yap, Tse, & Balota, 2009) and reading disabilities (David & Metsala, 2015). However, the cognitive microstructure for word recognition in skilled and novice language learners has not yet been subjected to empirical study, as it has been for reading tasks, although numerous attempts have been made to reveal the nature of the cognitive processes used in reading through the analysis of eye movements (Dare & Shillcock, 2013; Rayner, 2009).
The transformation in information processing that occurs while learning, has attracted the attention of researchers since the beginning of the 20th century (Haider & Frensch, 1999). The issue is often investigated through comparing the performance of experts and novices in cognitive tasks (Chase & Simon, 1973; Reingold & Sheridan, 2011). Experts have extensive and deep knowledge, as well as resources for effective problem-solving in their professional areas. They contrast with people who are beginning to master the area of knowledge or skill, or those who have not achieved high levels of mastery (Feldon, 2007). K.A. Ericsson emphasized that "one of the most exciting challenges in cognitive science is to understand the mechanisms mediating the superior performance of experts in various domains, such as chess, physics, medicine, sport, dance, and music" (Ericsson, 1991: vii). Indeed, this understanding opens up new opportunities not only for the modeling of cognitive processes, but also for developing new systems of assessment and training for professionals.
Eye-tracking methodology1 has provided significant insight into the perceptual mechanisms underlying the difference in performance between experts and novices. There is a lot of evidence identifying the specific perceptual patterns used by experts as they solve visual tasks in their fields. The level of expertise in visual tasks has been investigated in eye-movement studies of chess players (Grigorovich & Zyzlova, 2016; Reingold & Sheridan, 2011); medical staff (Wood, Batt, Appel-boam, Harris, & Wilson, 2014); teachers (Asaba, 2018; Wolff, Bogert, Jarodzka, & Boshuizen, 2014); chemists (Blinnikova & Ishmuratova, 2017) and others. All the above-mentioned studies showed significant distinctions between the eye movement patterns of experts and novices in their fields.
The main characteristic of experts' eye movements in visual tasks is greater "fluency" of viewing and attention to informative details, which is probably due to their deeper knowledge of the subject (Gegenfurtner, Lehtinen, & Saljo, 2011). For instance, it took only one fixation for grandmasters to find the position of a given
1 Eye-movement measures such as fixation duration, fixation count, AOI first hit and dwell times, saccadic amplitude, and regression count are most often taken into account (see Holmqvist et al., 2011). Attempts have been made to elaborate integrative oculomotor indicators: the saccade-fixation ratio (Kotval & Goldberg, 1998), focal and ambient modes of exploration (Velichkovsky, Joos, Helmert, & Pannasch, 2005), and change in the saccadic angles (Blinnikova & Izmalkova,
2017;.
piece on the chess board (Chase & Simon, 1973; Reingold & Sheridan, 2011). In the task of proof validation, skilled mathematicians tended to shift their attention between consecutive lines of purported proofs and distribute their attention in favor of implicit warrants, whereas undergraduate students spent more time focusing on the algebraic computation ("surface features" of the arguments) (Inglis & Alcock, 2012). Moreover, experts tended to find "shortcuts" to relevant information in complex visual stimuli (Jarodzka, Scheiter, Gerjets, & Van Gog, 2010). This idea is called the information-reduction hypothesis, which postulates that "with practice, people learn to distinguish task-relevant from task-redundant information and to ignore task-irrelevant information" (Haider & Frensch, 1999: 129). Similar changes can take place with the increase in language proficiency.
The fields of language acquisition and linguistic competence necessary to provide verbal information processing stand somewhat apart. When N. Chomsky (1965) introduced the concept of linguistic competence, he implied competence in one's native language. It is acquired early in childhood, which means that many of the fundamental cognitive attributes that ensure a high level of performance in verbal tasks remain hidden, despite attempts to reveal them (Carlson, Seipel, & Mc-Master, 2014). When analyzing language development in childhood, it is difficult to distinguish between the linguistic system proper and the systems of perception, memory, and thinking (Verhoeven & Leeuwe, 2009). Some significant characteristics of linguistic competence can be revealed in the process of mastering a foreign language (Blinnikova & Izmalkova, 2016; Mishra, Singh, Pandey, & Huettig, 2012). But this approach gives rise to several difficulties, the main one being the interaction between the developing systems of a second language (SL), and the already existing systems of the native language. However, a careful analysis of the change in verbal information-processing as the SL is being mastered, makes it possible to identify the main trends in the development of linguistic competence.
In the research on foreign language learning, the most frequent index for SL experience is language proficiency. One of the necessary skills formed during language learning is the ability to recognize lexical units, which involves matching a perceived stimulus to the representation in a person's mental lexicon1. The ways a mental lexicon is addressed have been broadly discussed recently (Libben & Titone, 2009). Attempts were made to establish whether this process relies on literal analysis, or rather that the words are perceived as holistic patterns (Grainger & Dufau, 2012). The existing models (Yap & Balota, 2015) suggest that there are at least two levels of analysis that determine the depth of information processing: incoming information processing (sub-lexical) and semantic analysis (lexical). Some researchers identify different strategies of information processing in the process of word recognition2. For example, D.A. Balota and D.H. Spieler (1999) describe two main strategies: a fast-acting familiarity-based process and a slower more attention-demanding process. The research into visual recognition of words often in-
1 The mental lexicon is known as a mental dictionary which contains information about a word's pronunciation, meaning, syntactic attributes, and so on, (Jackendoff, 2002). In linguistics and psycholinguistics Mental Lexicon is used to refer to individual speaker's lexical, or word, representations.
2 According to Wolf and Katzi-Cohen (2001), word recognition is a summation of accuracy and speed of meaning access through decoding of printed words.
volves the following tasks: reading, naming, categorizing, perceptual identification, lexical decision task, etc. (Yap & Balota, 2015). To gain a better understanding of the processes behind word recognition, researchers try to figure out how different characteristics (such as word frequency, length, familiarity, etc.) affect performance on the recognition task (New, Ferrand, Pallier, & Brysbaert, 2006; Norvig, 2013).
Eye-movement indicators have been used to identify the cognitive architecture of information processing and word recognition while reading, in people with different linguistic competence (Kunze, Yoshimura, Kawaichi, & Kise, 2013; Leinenger & Rayner, 2017). In K. Rayner's classic work on eye movements in reading and information processing, significant distinctions were found in the eye movements of individuals with different reading expertise: more experienced readers demonstrated longer saccadic amplitude, shorter fixations, and fewer regressions, while less skilled readers exhibited shorter saccades, longer fixations, and more regressions (Ashby, Rayner & Clifton, 2005; Rayner, 1998;). These data indicate that the development of reading skills leads to faster eye movements and faster information processing. As skills and word familiarity increase, more rapid linguistic analysis strategies are used (Clifton et al., 2016; Schotter & Rayner, 2015). Also, people learn to better organize their oculomotor activity, which helps them get more information within shorter intervals (Kunze et al., 2013). With an increased proficiency level, the duration of eye fixation in recognizing words is significantly reduced, although the number of eye fixations of low-proficiency SL readers is the same as those of high-proficiency SL readers (Bernhardt, 1984; Dare & Shillcock, 2013).
Since SL reading and word recognition form a task of a higher complexity than in one's own language, fixations get longer, saccades get shorter, and more regressions occur while performing (Rayner, 1998). Research in SL reading considers a number of additional aspects, including elements such as orthographic background, print input properties, SL experience, reading skill (Rayner, 2009), and an interplay of these elements (Koda, 2007). These factors have also been considered in SL word recognition research aimed at investigating the relationship between word recognition and reading outcomes (Han, 2015; Koda, 2007; Yamashita, 2013).
In our opinion, the task of reading, especially native language reading, creates special conditions which imply that the subjects recognize words placed in a semantic context rather than individual words. This allows the readers to refer only to a limited part of his/her mental lexicon and naturally facilitates the task of recognition, sometimes to the extent that all operations become implicit. Therefore, we were interested in identifying a more difficult task that would reveal the cognitive processing through eye-movement analysis of word recognition strategies in people at varying levels of linguistic competence. We assumed that visual semantic search tasks, which involve detecting a target object on a screen filled with distrac-tors, could be used in the research.
One of the instruments for investigating semantic search is to use a stimulus matrix that consists of random letters. Participants start the search, unaware of which word to look for, and with the goal of finding any correct word. This task combines reading and lexical decision tasks. Just as in reading, the subject has to scan the space filled with letters. However, unlike in reading, eye movements are unguided and less organized. Just as in a lexical decision task, the subject must constantly analyze whether the letter string is a word, but, in contrast to the lexical
decision task, it is the subject who highlights which chain of letters to consider. We believe this task will allow us to reveal the complex processes lying behind word search and recognition, and to identify the mechanisms which differentiate people at various levels of competence.
Method Goal
Our study focuses on the eye-movement patterns of ESL learners at different levels of mastery. As previous studies of oculomotor activity patterns in the context of cognitive strategies in various tasks (Blinnikova & Izmalkova, 2016; Blinnikova, Izmalkova, & Semenova, 2016) have shown, oculomotor correlates can indicate the characteristics of information processing (Rayner, 2009). Thus, the registration of participants' eye-movement patterns while they are performing semantic search and word recognition tasks is expected to reveal the features of the cognitive strategies used by ESL students at different levels of linguistic competence. The core goal of the research is to highlight the specific oculomotor activity patterns which can be associated with an increase in linguistic competence and, therefore, can reveal the development of cognitive processing strategies leading to higher levels of semantic search performance.
Sample
The sample included 32 undergraduate students aged 18-22, and 15 postgraduates aged 23-33. The participants were selected to form three homogeneous groups based on the parameter of English language mastery tested with the Word Associates Test (Read, 2013). In group one, the average language competence of the students corresponded to A2 (elementary-novices); in group two, it was B1-B2 (intermediate/vantage-users); and in group three, it was C1 (proficiency-experts), with A2, B1-B2, and C1 being levels set by CEFR (The Common European Framework of Reference for Languages). All the participants gave their written informed consent prior to the experimental procedure.
Stimuli
A Word Search task assumes that the person tries to find meaningful lexical units in a random set of letters. The subject consistently analyzes a string of letters, which he/she checks for lexical relevance. The solution has a complex cognitive architecture, including word recognition and a set of lexical decisions. For stimulus material, we generated letter matrices from randomized letters selected in accordance with the frequency of their appearance in English language. To do so, we created a randomizing procedure which used English letter frequency data (Norvig, 2013). This procedure was carried out to form letter matrices (15 by 15 letters each).
Then, the sets of target words were placed in the matrices (see Figure 1). The target words were separated by at least one row of letters and never crossed each other. Each matrix contained 10 English words, horizontally and vertically oriented. Thus, the parameter of spatial orientation of the target word was controlled in every trial.
L B A w T P U L N s M Y m~l 1 N
° M U 1 F E S F D R o E D
R A H N 1 L E Y R T F 1 u A T
1 H s T D A lK 1 N D W s E N
Q Y E N V T E K M T C N E P E
A 0 P N E H 0 E N T s 1 0 F H
M U R S M H| L 0 E| F E G V K
1 N 0 H N 0 E H N D s 0 T 0 1
S G c T M D A R J A N H B D S
G E u 0 G W T 1 R A E s R D
Y A G s X G E V W W Z A A 1 H
A P 1 R S 1 1 E U O N F O U K
N lc H 1 L R T 1 T E E c s
R T T L 1 H R N D 0 s M 1 E s
T s H E 1' D E A |A c 0 E 0 E u
Figure 1. Sample stimulus matrix with marked target words
Two series of the experimental procedure tested the factor of word length. The first series included words of four, five, and six letters. In the second series the length of the target words exceeded six letters. The ANEW scale (Bradley & Lang, 1999) was used to perform the target word selection according to word frequency data. The three groups were labeled as frequent, average, and rare words. Thus, the stimulus material consisted of 18 letter matrices, varying in the target word length and frequency. The incidence of distractors was under control. Three experts checked and proofread the matrices prior to the experimental procedure.
Procedure
Our experimental design included two consecutive series of stimuli exposure for each participant. The experiment was conducted individually. First, a subject filled in a questionnaire on his/her biographical and educational background, language mastery, and current physical condition. Then the subjects were tested for English language verbal competence. A printed version of Word Associates Test (WAT) (Read, 2013) was used to evaluate it. Later, the WAT results were used to group the participants according to their verbal competence.
The experimental procedure called for all the participants to be provided with printed instructions, and they had time to prepare for the experimental session. They were instructed to look for meaningful English words, vertically or horizontally oriented, in the matrices they would see on the screen. The words ought to have a minimum of four letters. The identified word was to be marked by mouse clicks of the first and the final letter. This method was meant to decrease the num-
ber of incidental strikes on the target words, as can happen in the case of fixations on Areas of Interest (AOI1). Each subject took part in two consecutive experimental series, divided by a short break for releasing the muscle tension which accumulated during holding the body in a fixed position during the experiment. Each series included nine matrices presented on a screen for 40 seconds, with a calibration fixation screen between them. The apparatus was calibrated in the beginning and after the first five matrices of each series.
Registered parameters
The number and spatial orientation of the found words were registered. We also registered the following oculomotor activity parameters: fixation count and duration; saccadic count, duration, amplitude, and velocity; and scan path length. Moreover, the target words were marked as Areas of Interest on stimuli matrices, which allowed us to register the eye-movement parameters within the target areas. These included AOI dwell time, fixation time and average duration, first fixation duration, glance duration, revisits, entry time, and the first click time.
Technical equipment and software
We used the following technical equipment: "SMI Gaze & Eye-tracking System" hardware and software; "Experiment Centre" software in experimental design; and the SMI RED 250 Hz eye-tracker. The distance between the monitor and eye was set at 0.6-0.8 m; precision at 0.4°; spatial resolution (RMS) at 0.03°; maximum delay at 6 ms; automatic calibration at <10 sec. The mouse clicks were registered. IBM SPSS Statistics 19 was used for data analysis.
Results and Discussion
For each subject 18 trials were registered. In the processing of samples, we treated each record as a separate case. We considered this possible because from the beginning we aimed at identifying the strategies leading to success that could vary from trial to trial.
The influence of the main and additional factors on the word search performance
The experimental task proved to be quite difficult for the subjects. In each individual trial, the participants found from zero to six words. However, the task was quite sensitive to the level of foreign language mastery. The means comparison indicates that task performance efficiency increased through all three groups of participants (see Table 1), with the novices having the lowest scores, and the experts showing the highest result. The differences proved significant. The same trend was observed throughout the results for horizontally- and vertically-oriented words (see Table 1).
1 "AOIs define regions in the stimulus that the researcher is interested in gathering data about" (Holmqvist et al., 2011: 187).
Table 1
Word search efficiency
Recognized Words Count
General Efficiency Horizontal Orientation Vertical Orientation
Novices 0.99 0.83 0.15
Users 1.69 1.52 0.16
Experts 2.22 1.91 0.31
F(2;796) 71.31 63.65 8.27
Significance p<0.001 p<0.001 p<0.001
Differences depending on spatial orientation, word frequency, and length effects were detected. It was much easier for all the participants to find the words that were horizontally positioned. On average, the subjects found 1.39 horizontally located words per matrix, vertically located words were found about seven times less often (0.20 — the differences are highly significant). This result may indicate that the subjects used strategies similar to those they use when reading. Identifying the characteristic scanning patterns is a task for further analysis.
The data also indicate that it was much easier for subjects to find long words rather than short ones (see Figure 2). With short and long words presented in different series, participants found an average of 1.83 long words and 1.34 short words per matrix. The differences are significant (F(1;797)=30.24; p<0.001). These results were unexpected since in most word recognition tasks (such as reading, pronouncing out loud, perceptual identification, lexical decision, etc.), the difficulty of the task increases with the increase in word length — i.e. latent reaction time grows (Yap & Balota, 2015). The words containing more letters require longer latent sub-processing time (Oganian et al., 2015). In our case it turned out that longer words appeared more "noticeable" for the participants. Apparently, the search for short
■ long
0,50 0,00 -1
1 1 i Group 1 (novices) Group 2 (users) Group 3 (experts)
Figure 2. The number of found words depending on their length in groups of different language competence
words is under greater influence of distractors, when subjects have to process a lot of related information. The longer words set-up allows the reader to avoid a large number of deceptive lexical units.
The effect of frequency is shown in Figure 3. The words of higher frequency were more likely to be detected in the current study. On average the subjects found 1.97 high-frequency words, 1.66 midrange words, and 1.23 low-frequency words per matrix. The differences calculated for all the trials are highly significant F(2;796)=27.50; p<0.001). The fact that high-frequency words are easier to recognize, which is expressed in higher reaction speed and recognition rating, has been repeatedly confirmed. Nevertheless, the mechanisms explaining the effects of frequency are not completely clear (Kinoshita, 2015). In our case, the simplest explanation is that low-frequency words are more difficult to distinguish from non-words, and this makes it difficult to identify them (Yap & Balota, 2015).
3,00
2,50
2,00
1,50
1,00
0,50
0,00
I low frequency middle frequency I high frequency
Group 1 (novices) Group 2 (users) Group 3 (experts)
Figure 3. The number of found words depending on their frequency in groups with different language competence
No interactions were found between the factors of spatial orientation, length, and frequency of words, and the factor of foreign language competence. In the groups of participants at different levels of language mastery, the influence of those three additional factors was unidirectional.
Oculomotor activity patterns in participants with different levels of linguistic competence
When analyzing oculomotor activity in the participants, we found that the development of language mastery is associated with higher gaze coverage of the matrix areas. Thus, oculomotor events occur much more frequently in the experts' results than in the novices' results (see Table 2). Experts show shorter fixation durations and longer saccade durations as compared to users and novices. The same trend is reflected in saccade velocity and scan path length increase. For in-
stance, the mean scan path length rises from 16821.07 pixels for novices, through 18165.47 pixels for users, to 20776.40 pixels for experts; that result is significant at F(2;796)=46.82; p<0.001.
Table 2
Stimulus matrix coverage by eye-movements: Intergroup comparison
oculomotor Events count (Fixations/ saccades) Fixation Duration Total(ms) saccade Duration Total(ms) saccade Velocity Average ^deg per s) scan path Length(px)
Novices 135.31/137.38 34552.54 3460.76 91.75 16821.07
Users 144.22/146.55 31715.48 6167.42 94.43 18165.47
Experts 146.37/149.23 29977.12 6410.42 109.14 20776.40
F(2;796) 17.84/19.03 35.62 40.77 19.72 46.82
Significance p<0.001 p<0.001 p<0.001 p<0.001 p<0.001
Data analysis also shows that learners with different levels of language mastery use different types of information processing, which are reflected in the proportions of their fixation and saccade durations (see Table 3). The word search and recognition process is characterized by longer fixations and shorter saccades in the group of novices in the English language. The group of experts, on the other hand, demonstrates shorter fixations and longer saccades. The same pattern persists in average saccade amplitudes with the group of experts having twice as high values as the group of novices.
Table 3
Information processing measures: Intergroup comparison
oculomotor Events Frequency (Fixations/ saccades) (count/s) Fixation Duration Average (ms) saccade Duration Average (ms) saccade Amplitude average ^deg)
Novices 3.42/3.45 259.65 24.74 3.19
Users 3.62/3.69 221.22 43.69 5.63
Experts 3.65/3.72 203.90 47.68 6.92
F(2;796) 11.52/15.58 105.98 30.70 32.42
Significance p<0.001 p<0.001 p<0.001 p<0.001
In general, these data confirm the general trend established in the previous analyses of eye movements while reading. K. Rayner and his colleagues have shown that the increased competence in reading, and/or reduced complexity of the text, lead to an increase in reading speed and saccadic amplitudes, and a decrease in average fixation durations (Rayner, 2009; Schotter & Rayner, 2015). Similar trends are also observed in chess playing, visual medical information analyzing, etc. These
changes reflect skill development in a particular area of expertise (Feldon, 2007). Information processing and access to mental lexicons are becoming more and more automated as the information-reduction mechanisms start to operate (Haider & Frensch, 1999).
Areas of Interest analysis shows that there are significant differences in oculomotor activity patterns in the region of successfully identified words, compared to the AOIs with words which were not found by the respondent (see Table 4). For instance, the AOI dwell time associated with correct target word identification exceeds by almost three times the same parameter for the unidentified words. Another factor contributing to the successful search, is the number of AOI revisits which are higher for the found words. Additional analysis is required to identify the cognitive architecture of this process.
Table 4
Oculomotor events in the Areas of Interest (AOI) for the found and not-found words
AoI Dwell Time (ms) Average Fixation Duration in AoI (ms) First Fixation Duration in AoI (ms) Revisits count (number)
Not found word 852.21 160.90 158.86 2.31
Found word 2436.64 226.07 198.11 3.41
F(2;7987) 159.,72 283.16 77.86 177.43
Significance p<0.001 p<0.001 p<0.001 p<0.001
Significance p<0.001 p<0.001 p<0.001 p<0.001
The duration of the first fixation, as well as the duration of the average fixation in the AOI, is higher for the found words, which can indicate that correct lexical decisions are taken only in case of deeper semantic processing, associated in its turn with longer timespans. It is significant that both indicators are not too high as compared to the average in the case of the found words; rather, on the contrary, the fixations in the areas of the missed words are too short. These data require further analysis. Most likely, we are faced with the phenomenon of word skipping. K. Rayner assumes that approximately one third of words are skipped when reading (Rayner, 2009; Schotter, Rayner, 2015). It is noted that these words are either too short, or highly predictable and out of context. This habit of skipping words seemed to mislead the participants in our experiment. They relied on their reading skills in a task that required other strategies of verbal material processing.
Analyzing the intergroup differences in eye-movements during successful target word recognition (see Table 5), we find that, along with the gradual decrease in AOI dwell time from novices (3089.83ms), through users (2317.29 ms), to experts (2161.19 ms) (F(2;1166)=22.95; p<0,001), the duration of the first and the average fixation breaks the trend. This finding can indicate some qualitative change in semantic processing with the growth of language mastery, whereas the unsuccessful AOI hits only reflect the general trend toward shorter fixations with the increase in linguistic competence.
140 I. V. Blinnikova, M. D. Rabeson, A. I. Izmalkova Table 5
Intergroup comparison of the oculomotor events in AOIs (found and not-found words)
Aoi Dwell Average Fixation First Fixation
Time (ms) Duration in AoI (ms) Duration in AoI (ms)
Found Not found Found Not found Found Not found
Novices 3089.83 963.62 265.10 175.42 233.87 174.35
Users 2317.29 818.54 209.56 157.60 180.81 154.93
Experts 2161.19 742.89 217.37 145.36 191.94 142.79
F(2;1166)= F(2;6817)= F(2;1166)= F(2;6817)= F(2;1166)= F(2;6817)=
22.95 24.06 21.27 34.56 15.20 28.74
Significance p<0.001 p<0.001 p<0.001 p<0.001 p<0.001 p<0.001
The time between the first AOI hit and the first mouse click in the same area reflects the normal trend for shorter reaction times in experts. Novices tend to spend more time for their lexical decision (see Figure 4). Differences are significant at F(2;1166)=5.82; p=<0.003.
Trials with high and low success levels
The eye-movement patterns we observed are associated with an increase in language competence and indicate the development of a more automated processing of verbal information. However, it remains unclear which scanning strategies and information extraction types provide a more efficient recognition of words in a noisy context. In searching for a reliable answer, we attempted to compare eye movements in more and less successful trials.
12,00 -
10,00
8,00 -
6,00 -
4,00 -
2,00
0,00 -------
Group 1 (novices) Group 2 (users) Group 3 (experts)
Figure 4. Average decision time-the time from the first entry into the AOI in the letter matrix to the click on the computer mouse highlighting a recognized word -in groups with different foreign language competence
We divided each of the three groups of subjects into two subgroups, successful and unsuccessful trials, with the following results:
• In the group of novices, unsuccessful trials (n=131) amounted to 45.5%; they didn't find a single word. All other trials (n=157) were classified as successful (54.5%).
• In the group of users, unsuccessful trials were the ones in which 0 or 1 word was found (n=124), which amounted to 46.1%. Trials in which the subjects found from 2 to 6 words (n=157) were classified as successful (53.9%).
• In the group with good knowledge of English, not only the trials in which the subjects found no words (they were only 18, or 7.4%), but also those in which only one or two words were found, were classified as unsuccessful. The total volume of such trials (n=125) was 51.6%. Successful trials in this case were those in which the subjects found from 3 to 6 words (n=117); in total, they accounted for 48.4% of all trials. Similar trends were observed in certain abilities tests, where each level is determined by adding the number of problems solved, or points obtained.
Next, we compared the indicators of eye movements in the successful and unsuccessful samples. For the two groups with low and medium SL competence, we did not find any significant differences. However, some trends require reflection and further analysis. For instance, in the group with the lowest SL competence, we found that better results are associated with a longer scanning path (17238.32 vs 16228.52px; F(1;285)=2.99; p<0.1) and lower saccadic speed (88.94 vs 95.78deg/s; F(1;285)=2.83; p<0.1). In other words, the more elements of the matrix are scanned and the more slowly it is done, the more successful the word search and recognition are.
Table 6
Oculomotor parameters in high and low success trials in the group of experts
scan path Fixation saccade saccade Fixation/
Length Duration Duration Amplitude saccade
(px) Average(ms) Average(ms) Average^deg) Ratio
Trials with low success level 20066.04 196.69 55.33 8.08 7.41
Trials with high success level 21875.59 215.06 35.86 5.11 9.08
F(1; 240) 7.1 10.22 10.88 9.21 4.43
Significance p<0.01 p<0.01 p<0.01 p<0.01 p<0.05
Thus, successful word detection is linked with wider coverage of the search space. This strategy may be called "movement in breadth." Slower speed of movement along the matrix can be associated with higher awareness and control of the search process. In the group with medium language competence, the more successful trials were characterized by longer average fixation durations (225.03 vs
216.03ms; F(1.267)=2.81; p<0.1). Since average fixation duration is associated with the depth of processing, this may indicate that more successful word recognition is ensured by approaching semantic levels of information processing (Velichkovsky et al., 2005; Pannasch, Schulz, & Velichkovsky, 2011). This strategy can be described as "movement in depth."
In the group of experts, we found a number of significant differences in eye-movement indicators between successful and unsuccessful samples. The main results are presented in Table 6. Successful trials are characterized by a longer scan path and, at the same time, longer fixations and shorter saccades. Thus, the success in word search and recognition is determined, first, by larger coverage of the search space, and second, by deeper information processing. In other words, the integration of the two strategies identified earlier appears ("movement in breadth" and "movement in depth").
conclusion
In this study, subjects were instructed to look for foreign language words in alphabetic matrices which were presented for 40 second intervals. The task turned out to be quite difficult: the subjects found an average of 1.53 words out of 10 (ranging from 0 to 6 in separate trials). The effects of word frequency, length, and orientation were discovered. The important thing was that the task successfully differentiated groups on the basis of their levels of foreign language competence.
Foreign language proficiency manifested itself both in search efficiency and in patterns of oculomotor activity. The increase in language competence resulted in an increase in the number of words found, regardless of the length and frequency of words. As for the eye-movement parameters, we observed several characteristic changes associated with the better knowledge of foreign language: 1) the overall length of the gaze path; 2) a rise in the number of oculomotor events and the average saccadic velocity and saccadic amplitude; and 3) a decrease in the average fixation duration. The data are consistent with those obtained in eye-movements studies on reading patterns in people with different levels of reading skill development.
The comparison of the more and less successful trials in groups with different levels of linguistic competence proved to be of interest. It turned out that success in task performance is associated with strategies of oculomotor activity. In the low competence group (novices), success was mostly determined by slower scanning speed, which presumably may indicate the use of more conscious strategies. In the group of users, there was a trend associating word search success with longer fixations, that is, with deeper semantic processing. The clearest differences between eye movements in successful and unsuccessful trials were observed in the group of the highest competence (experts). There were two significant and complementary trends. The word search success was achieved, first, by a longer gaze scan path, and second, by a focal mode of visual information processing, manifested in a combination of longer fixations and shorter saccades. Thus, we can draw the important conclusion that, even though enhancing linguistic com-
petence leads to the development of faster and possibly automatic ways of verbal information processing, those subjects who chose to combine fast scanning with deep semantic processing proved to be more successful in the experimental task.
Limitations
Our research reveals some interesting trends in oculomotor activity with the growth of language competence, but all the participants were Russian learners of English. Significant distinctions in oculomotor activity patterns have been found in individuals who have mastered different writing systems. For example, Chinese and Hebrew native speakers' average saccade length was much shorter than that of English native speakers in reading English passages, due to the fact that linguistic information in these languages is more "densely packed" than in English (Pollatsek, Bolozky, Well, & Rayner, 1981). Thus, cross-cultural comparisons can be of great interest. Another aspect to consider is including a sample of English native speakers into the study, to evaluate how expert learners approach their eye movement patterns and word recognition models.
Moreover, the individual lexicon of each participant was not evaluated prior to the study. We assumed that the common word-frequency scale suited all the participants equally, although word familiarity can differ dramatically, depending upon a person's learning background, with beginning adult learners acquiring profession-specific vocabulary first. There is a possibility that commonly used words might have been unfamiliar to this group of respondents.
Acknowledgements
The research was supported by the Russian Foundation for Basic Research (Project No 18-013-01240a).
References
Asaba, M. (2018) A Model of Expertise: A Case Study of a Second Language Teacher Educator. TESL-EJ, 21, 4. Retrieved from http://www.teslej.org/wordpress/issues/volume21/ej84/ ej84a5/
Ashby, J., Rayner, K., & Clifton, C. (2005). Eye movements of highly skilled and average readers: Differential effects of frequency and predictability. Quarterly Journal of Experimental Psychology, 58A, 1065-1086. https:// doi.org/10.1080/02724980443000476 Balota, D.A. & Spieler, D.H. (1999). Word frequency, repetition, and lexicality effects in word recognition tasks: Beyond measures of central tendency. Journal of Experimental Psychology: General, 128(1), 32-55. https://doi.org/10.1037/0096-3445.128.L32 Blinnikova, I.V. & Izmalkova, A.I. (2016) Eye Movement Evidence of Cognitive Strategies in SL Vocabulary Learning. Smart Innovation, Systems and Technologies, 57, 311-322. https://doi. org/10.1007/978-3-319-39627-9_27 Blinnikova, I.V. & Izmalkova, A.I. (2017) Modeling search in web environment: the analysis of eye movement measures and patterns. Smart Innovation, Systems and Technologies, 58, 125-136. https://doi.org/10.1007/978-3-319-59424-8_28 Blinnikova, I.V., Izmalkova, A.I., & Semenova M. (2016) The factors of effectiveness and the organization of the search of elements within a graphic interface. International Journal In-
formation Content and Processing, 3 (2), 160-174. Retrieved from http://www.foibg.com/ ijicp/vol03/ijicp03-02.pdf Blinnikova, I.V. & Ishmuratova, Y. (2017) Cognitive strategies for solving graphically presented chemical tasks: A comparison of experts' and novices' performance. Journal of Eye Movement Research, 10, 6. https://doi.org/ 10.16910/jemr.10.6 Bernhardt, E.B. (1984). Toward an Information Processing Perspective in Foreign Language
Readin. The Modern Language Journal, 68(4), 322-331. http://dx.doi.org/10.2307/328180 Bradley, M.M. & Lang, P.J. (1999). Affective norms for English words (ANEW): Instruction
manual and affective ratings. Technical report C-1. Gainesville: University of Florida. Brysbaert, M. & New, B. (2009) Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior research methods, 41, 4, 977-990. https://doi. org/10.3758/BRM.41.4.977 Carlson, S.E., Seipel, B., & McMaster, K. (2014) Development of a new reading comprehension assessment: Identifying comprehension differences among readers. Learning and Individual Differences, 32, 40-53.https://doi.org/ 10.1016/j.lindif.2014.03.003 Chase, W.G. & Simon, H.A. (1973). The mind's eye in chess. In Chass, W.G .(Ed.) Visual information processing (pp.215-281) NY: Academic Press, Chomsky, N. (1965). Aspects of the Theory of Syntax. Cambridge, MA: MIT Press. Clifton, Jr. Ch., Ferreira, F., Henderson, J.M., Inhoff, A.W., Liversedge, S.P., Reichle, E.D., & Schotter, E.R. (2016) Eye movements in reading and information processing: Keith Rayner's 40-year legacy. Journal of Memory and Language. 86, 1-19. https://doi.org/ 10.1016/j. jml.2015.07.004
Dare, N. & Shillcock, R. (2013). Serial and parallel processing in reading: Investigating the effects of parafoveal orthographic information on non-isolated word recognition. The Quarterly Journal of Experimental Psychology, 66(3), 487-504. https://doi.org/10.1080/17470218 .2012.703212
David, M.D. & Metsala, J.L. (2015) Individual Differences in Word Recognition: Reading Acquisition and Reading Disabilities. In: P. Afflerbach (Ed.) Handbook of Individual Differences in Reading (pp. 111-124). N.Y.-London: Routledge. Ericsson, K.A. (1991) Preface. In K.A. Ericsson & J. Smith (Eds.) Toward a general theory of expertise: Prospects and limits (pp. VII-VIII). Cambridge: Cambridge University Press. Feldon, D.F. (2007) the Implications of Research on Expertise for Curriculum and Pedagogy.
Educational Psychology Review, 19, 2, 91-110. https://doi.org/ 10.1007/s10648-006-9009-0 Gegenfurtner, A., Lehtinen, E., & Saljo, R. (2011) Expertise differences in the comprehension of visualizations: A meta-analysis of eye-tracking research in professional domains. Educational Psychology Review, 23, 4, 523-552. https://doi.org/10.1007/s10648-011-9174-7 Grainger, J. & Dufau, S. (2012). The front-end of visual word recognition. In J.S. Adel-man (ed.) Visual Word Recognition (Vol. 1, 159-185). NY: Psychology Press. https://doi. org/10.4324/9780203107010 Grigorovich, S. & Zyzlova, S. (2016) Visual attention distribution during analysis of chess positions: The effect of skill level. International Journal of Psychology, 51, 1111. Haider, H. & Frensch, P.A. (1999). Eye movement during skill acquisition: More evidence for the information-reduction hypothesis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25(1), 172-190. https://doi.org/ 10.1037/0278-7393.25.1.172 Han, F. (2015). Word recognition research in foreign language reading: A systematic review. University of Sydney Papers in TESOL, 10, 57-91. https://doi.org/10.1080/02724980443000476
Holmqvist, K., Nystrom, N., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (2011) Eye tracking: A comprehensive guide to methods and measures. Oxford, UK: Oxford University Press.
Inglis, M. & Alcock, L. (2012) Expert and novice approaches to reading mathematical proofs. Journal for Research in Mathematics Education, 43, 4, 358-390. https://doi.org/ 10.5951/jre-sematheduc.43.4.0358
Jackendoff, R. (2002) Foundations of language: Brain, meaning, grammar, evolution. NY: Oxford
University Press. https://doi.org/10.1093/acprof:oso/9780198270126.001.0001 Jarodzka, H., Scheiter, K., Gerjets, P., & Van Gog, T. (2010). In the eyes of the beholder: How experts and novices interpret dynamic stimuli. Learning and Instruction, 20, 2, 146-154. https://doi.org/10.1016/j.learninstruc.2009.02.019 Kinoshita, S. (2015). Visual word recognition in the Bayesian Reader framework. In A. Pollatsek & R. Treiman (Eds.), Oxford Handbook of Reading (pp. 63-75). NY: Oxford University Press. Koda, K. (2007). Reading and language learning: Cross-linguistic constraints on second language reading development. Language Learning, 57, 1-44. https://doi.org/10.1111/ 0023-8333.101997010-i1 Kotval, X.P. & Goldberg, J.H. (1998) Eye Movements and Interface Components Grouping: An Evaluation Method. Proceedings of the Human Factors and Ergonomics Society, 42(5), 486490. https://doi.org/10.1177/154193129804200509 Kunze, K., Yoshimura, K., Kawaichi, H., & Kise, K. (2013). Towards inferring language expertise using eye tracking. In CHI EA 2013 - Extended Abstracts on Human Factors in Computing Systems: Changing Perspectives (Vol. 2013-April, 217-222). Association for Computing Machinery. https://doi.org/10.1145/2468356.2468396 Leinenger, M. & Rayner, K. (2017). What we know about skilled, beginning, and older readers from monitoring their eye movements. In J.A. León & I. Escudero (Eds.) Reading Comprehension in Educational Settings. (pp. 1-27). Amsterdam: John Benjamins. Libben, M.R. & Titone, D.A. (2009) Bilingual lexical access in context: evidence from eye movements during reading. Journal of Experimental Psychology: Learning, Memory, and Cognition. 35(2), 381. https://doi.org/ 10.1037/a0014875 Mishra, R.K., Singh, N., Pandey, A., & Huettig, F. (2012) Spoken language-mediated anticipatory eye-movements are modulated by reading ability: Evidence from Indian low and high literates. Journal of Eye Movement Research, 5, 1, 1-10. https://doi.org/10.16910/jemr.5.1.3 New, B., Ferrand, L., Pallier, C., & Brysbaert, M. (2006) Re-examining the word length effect in visual word recognition: New evidence from the English Lexicon. Psychonomic Bulletin and Review, 13, 45-52. PMID: 16724767 Norvig, P. (2013). English letter frequency counts: Mayzner revisited. Retrieved from http://
norvig. com/mayzner. html. Oganian, Y, Froehlich, E, Schlickeiser, U, Hofmann, M.J., Heekeren, H.R., & Jacobs, A.M. (2015) Slower perception followed by faster lexical decision in longer words: A diffusion model analysis. Frontiers in psychology, 6, 1958. https://doi.org/10.3389/fpsyg.2015.01958 Pannasch, S., Schulz, J., & Velichkovsky, B.M. (2011). On the control of visual fixation durations in free viewing of complex images. Attention, Perception & Psychophysics, 73, 4, 1120-1132. https://doi.org/ 10.3758/s13414-011-0090-1 Pexman, P.M. (2012). Meaning-based influences on visual word recognition. In J.S. Adelman
(Ed.). Visual word recognition (Vol.2, 24-43). NY: Psychology Press. Pollatsek, A., Bolozky, S., Well, A.D., & Rayner, K. (1981). Asymmetries in the perceptual span
for Israeli readers. Brain and Language, 14, 174-180. PMID:7272722 Rayner, K. (1998) Eye movements in reading and information processing: 20 years of research. Psychological bulletin, 124, 3, 372. https://doi.org/10.1037/0033-2909.124.3.372
Rayner, K. (2009) Eye movements and attention in reading, scene perception, and visual search. The quarterly journal of experimental psychology, 62, 8, 1457-1506. https://doi. org/10.1080/17470210902816461 Read, J. (2013). Validating a test to measure depth of vocabulary knowledge. In A.J. Kunnan
(Ed.) Validation in language assessment (pp.55-74). NY: Routledge. Reingold, E.M. & Sheridan, H. (2011). Eye movements and visual expertise in chess and medicine. In S.P. Liversedge, I.D. Gilchrist, & S. Everling (Eds.), Oxford library of psychology. The Oxford handbook of eye movements (pp. 523-550). NY: Oxford University Press. Schotter, E.R. & Rayner, K. (2015). The work of the eyes during reading. In A. Pollatsek & R.
Treiman (Eds.), Oxford Handbook of Reading (pp. 44-62). Oxford: Oxford University Press. Velichkovsky, B.M., Joos, M., Helmert, J.R., & Pannasch, S. (2005). Two visual systems and their eye movements: Evidence from static and dynamic scene perception. In B.G.Bara, L. Barsalou, & M. Bucciarelli (Eds.) Proceedings of the XXVII conference of the cognitive science society (pp. 2283-2288). New Jersey: Lawrence Erlbaum Associates. Verhoeven, L. & van Leeuwe, J. (2009). Modeling the growth ofword-decoding skills: Evidence from Dutch. Scientific Studies ofReading, 13(3), 205-223. https://doi.org/10.1080/10888430902851356 Wolf, M. & Katzir-Cohen, T. (2001). Reading fluency and its intervention. Scientific studies of
reading, 5(3), 211-239. https://doi.org/10.1207/S1532799XSSR0503_2 Wolff, C.E., Bogert, N. van den., Jarodzka, H., & Boshuizen, H.P. (2014). Keeping an eye on learning: Differences between expert and novice teachers' representations of classroom management events. Journal of Teacher Education, 66, 68-85. https://doi.org/10.1177/0022487114549810 Wood, G., Batt, J., Appelboam, A., Harris, A., & Wilson, M.R. (2014). Exploring the impact of expertise, clinical history, and visual search on electrocardiogram interpretation. Med. Decis. Mak, 34, 1, 75-83. https://doi.org/10.1177/0272989X13492016 Yamashita, J. (2013). Word recognition subcomponents and passage level reading in a foreign
language. Reading in a Foreign Language, 25, 52-71. Yap, M. J., Tse, C.S., & Balota, D.A. (2009) Individual differences in the joint effects of semantic priming and word frequency revealed by RT distributional analyses: The role of lexical integrity. Journal of Memory and Language, 61, 3, 303-325. DOI: 10.1016/j.jml.2009.07.001 Yap, M.J. & Balota, D.A. (2015). Visual word recognition. In A. Pollatsek & R. Treiman (Eds.) The Oxford handbook of reading (pp. 26-43). Oxford: Oxford University Press.
Original manuscript received December 27, 2018 Revised manuscript accepted January 10, 2019 First published online March 15, 2019
To cite this article: Blinnikova, I.V., Rabeson, M.D., Izmalkova, A.I. (2019). Eye movements and word recognition during visual semantic search: differences between expert and novice language learners. Psychology in Russia: State of the Art, 12(1), 129-146. DOI: 10.11621/pir.2019.0110
