CC BY
29International Annual Edition of Applied Psychology : Theory, Research, and Practice Volume 3, Issue 1, 2016.
EYE-TRACKING: REGULARITIES OF EDUCATIONAL INFORMATION SEARCHING
Sergey B. Venig,
Saratov State University, Saratov, Russia sergey.venig@gmail.com
Valentina A. Solovyova,
Saratov State University,
Saratov, Russia
v.a. solovyova@gmail.com
Abstract
This work describes the results of research concerning patterns of students' visual attention while searching answers using an educational source for a limited period of time. The eye-tracking technology was used to perform the experiment. The following aspects were found out: which part of an informational source attracts attention the most; the blind spots of the source; the differences in searching the information between humanity students and natural science students; the impact of using graphical material on execution of a task to solve. These findings may be used for creating the educational information source.
Key words: oculomotor characteristics, fixations, number of blinks, searching educational information, regularities of searching, text complexity.
Introduction
Everybody knows the phrase "being determines consciousness". We can create a new postulate based on the prase "the way information is represented determines users' mode of its consumption". The competent information supplying becomes one of the important steps for reaching the effectiveness in information influence on
users. This thesis is notably topical in the digital era when the main channel for receiving information is visual. A lot of specific literature concerning Web-design and using coloristic rules confirms this idea as well [1-3].
Thus, the live issue accelerates development of tools and methods, eye-tracking being one of these methods. It is based on studying users' behavior by tracking and recording their oculomotor characteristics. The application of eye-tracker is wide -usability-testing, the analysis of discourse processes, computer games' enhancement etc. However, regardless of the use, it is intended to identify humans' cognitive reactions regularities [4-6]. The research results are used as the basis for making decisions concerning quality improvements in every mentioned area.
The quality issues are relevant to Internet-education sphere more than ever - a lot of educational information resources are being created, so the demand in the regulation procedures is increasing. Changes in the students' way of thinking and availability of new ways to represent information leads to the situation when rules and trends, characteristic of the book period, can only be used partially. Therefore, we should try to understand regularities of students' interaction with the educational information when it is on screen [7]. The eye-tracking can be used for studying processes of scanning, reading and processing texts.
This work describes the results of the research concerning quantitative assessment of students' visual attention and processing information while searching answers using an educational source for a limited period of time. Stimuli were structured differently. It is assumed that the set conditions simulate situations of searching suitable educational resources on the Web. The main questions to identify:
(1) influence of the way of structuring the material on the order of content studying,
(2) a part of the source which attracts students mostly,
(3) blind spots of the source,
(4) attention deflection by entertainment content,
(5) graphical educational materials attractiveness,
(6) difference in searching information between humanity students and natural science students.
Methods
The eye-tracking technology was used to conduct the experiment. Eye movement recording was conducted by SMI iView X system.
The algorithm of the work includes the following components:
1) Defining three basic questions for students to search the answers to using stimuli. The following questions were prepared:
(1) "When the economics is regarded as effective?"1
(2) "What is being?"2
(3) "What characterizes the fact that thermodynamic equilibrium is attained?"3 Thus, different fields of studies were covered within the experiment.
2) Development of the stimuli structures. Five educational Web-pages were prepared for each question. Fifteen pages were compiled as a result. The following conditions were met:
(1) different structure of representing the information:
- text structured in one, two or three columns;
- plain text / text with supporting data - tables, charts, graphics;
- graphics with entertaining or educational content;
- using and not using indentions;
- using and not using text highlighters: color, bold, italic, linked handwriting. Bearing in mind that the main characteristics of the texts and educational
sources - font size, substate color, etc. play the important role in searching
1 "Когда экономику можно считать эффективной?"
2 "Что есть бытие?"
3 "Что характеризует достижение термодинамического равновесия?"
information [8], the differences in these characteristics have been kept to a
• • 4
minimum .
(2) themes of the texts and Web-pages design were chosen for of little participants' notice. This condition concerns the peculiarity of visual search -familiarity with an object influences main oculomotor characteristics [9];
(3) some Web-pages contained the answers to the questions, some did not. The answers location also differed - at the top of the Web-page, in the middle, at the bottom.
(4) the answers to the questions could be explicit or implicit.
3) Specification of the cognitive task for students: try to find the answer to the question you have just read on the following 5 Web-pages. Searching time is limited to 30 seconds.
Regularities of searching the information depend on the task conditions [10]. For instance, time for scanning the page was limited to simulate situation of searching information on the Web because the vast majority of users leave sites in 10-20 seconds, so this time is enough to make a decision about the source usefulness5.
4) Sampling.
The selection - students of Saratov State University:
- humanity students - 60 students.
- natural science students - 60 students.
Overall selection - 120 students.
5) Forming the final experiment scenario.
(1) A student gets acquainted with: the environment, the rules of working with the eye-tracker, the cognitive task.
(2) A moderator calibrates the eye-tracker.
4 Except the texts complexity - the length of words and sentences.
5 Nielsen J. How Long Do Users Stay on Web Pages? September 12, 2011. URL: https://www.nngroup.com/articles/how-long-do-users-stay-on-web-pages/.
(3) The student reads a slide with the question, and then 5 Web-pages are displayed in a specified sequence, 30 seconds for each one. If the student finds the answer earlier or understands that there is no answer on the Web-page, he / she asks the moderator to switch to the next page. After scanning each Web-page, the student says if he / she has found the answer. This process repeats with the 3 basic questions defined at the first step.
(4) After working with the Web-pages, the student assesses the questions difficulty and leaves comments concerning his / her own user experience of working with the texts.
Results and discussion
This research offers the analysis of the following characteristics: (1) Blink frequency. (2) Search response time. (3) Fixations (frequency and duration average). (4) The analysis of AOI charts (areas of interest). (5) The analysis of the results of the participants' decision-making. (6) Studying students' comments.
(1) Blink frequency
The analysis of blink frequency shows the average index of nearly 15 counts per minute for both humanities and natural science students.
It is believed that while blinking some information is lost. So, the index of blink frequency decreases and reaches the value of approximately 5 counts per minute when reading or scanning the texts, i.e. the stimuli required the higher attention and better information processing [11]. The increasing index may indicate mind wandering [12] and distracting from the task.
Only 25% of students blinked less than 5 counts per minute. 45% of them are humanities students, 55% are natural science students. Feasibly, they grasped the information quicker and digressed by distractors less [13].
It is worth noticing that the probability to make a correct decision by these students was higher in case the answer on the Web-page was explicit6.
The increased index may indicate other reasons too:
- inconvenience of the texts for reading (monitor brightness, font size and others);
- the students' fatigue or alertness.
(2) Search response time
Natural science students needed more time to find the answer in 13 cases out of 15 (see table 1). The resulting difference can be considered significant because Mann-Whitney U-test index is equal to 54. The possible reason is educational peculiarities - humanities students work with the large amount of information more often so they navigate the texts easier and scan it quicker. Meanwhile, natural science students are accustomed to a more fundamental text studies approach.
Table 1.
Average search response time (the results are rounded upward), and the text complexity indexes: hum. - humanities students; nat.sc. - natural science students
t hum., t nat.sc., Flesch/Fog t hum., t nat.sc., Flesch/Fog
sec sec Index sec sec Index
1 page 26,2 29,9 23/8 9 page 21,7 24,6 46/9
2 page 26,1 27,2 15/9 10 page 21,8 21,5 39/8
3 page 24,2 26,3 29/5 11 page 24,1 25,9 11/9
4 page 24,2 26 0/11 12 page 21,6 25,4 21/10
5 page 22 22,5 0/11 13 page 23,8 24,6 13/8
6 page 16 15,3 35/8 14 page 21 24,9 25/10
7 page 20,9 24,1 41/8 15 page 23,4 26,2 0/12
8 page 19 23,4 3/10
6 These students are named "selec." in table 3.
Search response time was confronted with two indexes of the text complexity -Flesch Index and Fog Index (see table 1). The correlation between these parameters is very weak (for the humanities students: -0.095 Flesch Index, Fog Index -0.01; for natural science students: -0.16 Flesch Index, the Fog Index 0.03). This result can be interpreted as follows:
- these indexes are not appropriate for Russian texts because originally they were used for assessment of English texts;
- the complexity of the texts does not have a significant impact on the search response time because the mechanism of text scanning differs from that of reading. This thesis seems to be more convincing, because the natural science students worked with the texts longer and correlation between these parameters is more obvious for them.
(3) Fixations
Due to the fact that natural science students spent more time studying the pages, the number of their fixations was also higher. So the frequency and duration of fixations were analyzed (see table 2).
Fixations frequency decreases with increasing the task complexity [14]. According to this statement, it can be concluded that the cognitive task was more difficult for natural science students (Mann-Whitney U-test equals 30). This confirms the results presented in "Search response time" once more.
Duration of fixation is used to determine the texts readability [15]. The results confirmed that students did not read the texts (the mechanism of text scanning differs from that of reading):
- significant differences between natural science students and humanities students were not found (Mann-Whitney U-test equals 97),
- correlation between fixation duration and complexity of the texts is very weak: 0,017 - with Flesch Index; 0,002 - with Fog Index. The analysis of the correlation between these parameters was carried out for the general sample.
Table 2.
Average frequency and duration average of fixations (the results are rounded upward): hum. - humanities students; nat.sc. - natural science students
Frequency Frequency Duration Duration
hum., nat.sc., Average Average
count/sec. count/sec. hum., sec. nat.sc., sec.
1 page 3,2 3,6 1 page 153,7 149,7
2 page 3 3,2 2 page 158,3 158,2
3 page 3 3,3 3 page 147,9 151
4 page 2,4 3,3 4 page 145,8 154,3
5 page 3,1 3,1 5 page 155,7 144,6
6 page 3,2 3,2 6 page 148,3 148,5
7 page 3,2 3,2 7 page 156,3 154,3
8 page 3,1 3,4 8 page 153,4 162,1
9 page 3,2 3,3 9 page 151 150,6
10 page 3,2 3,3 10 page 155 147,6
11 page 3,2 3,3 11 page 156,2 154
12 page 3 3,3 12 page 154,6 153,8
13 page 3,2 3,4 13 page 154,2 154,2
14 page 3,2 3,4 14 page 153,9 150,5
15 page 3,2 3,4 15 page 155,3 156,7
(4) AOI charts
Analysis the AOI charts (areas of interest) shows:
1) the most attention was focused on:
A. the places of keywords in two first questions: "эффективность" ("efficiency") and "бытие" ("being"). The last question caused more difficulties that can be explained by the length of the keywords "термодинамическое равновесие" ("thermodynamic equilibrium") and the subject complexity.
7 i.e. physics.
B. the first three paragraphs of the texts (mostly, the second and the third ones, see figure 1);
2) less attention was attracted by:
A. supporting data - tables, charts, graphics (but not the one with the text in them). Picture-meme with a well-known media person, which is familiar to students due to viral spread in social networks, was one of the exceptions.
Taking into account that students ignored the educational and entertaining graphics, it can be concluded that modern students consider educational materials as texts primarily. Distracting graphics do not disrupt students while searching information. So, only graphics with some useful text in it would be within students' sight.
B. Data at the bottom of the Web-page (logically appears from the F-effect)
The reasons for this situation may be the following:
- a student found the answer earlier or decided that page did not contain the answer;
- students did not have enough time to read the text at the bottom of the Webpage. This statement looks unconvincing because 30 seconds was always almost enough for participants.
However, this result shows that the ways of text scanning and reading are similar - from the top of the page to its bottom, from left to right (what is natural for Russia) but with every new line the participants read less text - this is the classic F-effect.
3) the way of structuring the material influenced the distribution of students' visual attention (there is no big difference in distribution of attention between natural science students and humanities students):
- in case the texts were not divided into structural units, the first three paragraphs attracted the most attention (see figure 1);
Figure 1. Heat maps showing F-effect (fist case - text structuring in one column; second case - in three columns)
- in case the texts were divided into structural units students looked through first phrases of each unit to decide if there is useful information (see figure 2);
Figure 2. Heat maps showing F-effect (fist case - the list of definitions; second case - numbered parts of the text)
(5) The analysis of the results of the participants' decision-making
A special focus should be paid to the students' decisions about the answers presence on the given Web-pages (see table 3). Students defined the questions complexity. According to collected data, the philosophy question was the easiest, economics question was more difficult, physics question was the most difficult.
In case of the first two questions most students found the explicit answers on the page regardless of their location. Only a half of the students found the answer to the third, the most difficult, question.
Table 3.
Decisions about the answer presence made by students: hum. - humanities students; nat.sc. - natural science students; selec. - students blinked less than 5 counts per minute; "+" - there is the explicit answer on the page; "+/-" - there is the implicit answer
on the page; "-" - there is no answer on the page
First question Second question Third question
Middle complexity question Easiest question The most difficult question
% finding Answer's % finding Answer's % finding Answer's
answer students presence and location answer students presence and location answer students presence and location
hum. 32% hum. 97% + middle hum. 52% + top
nat.sc. 18% - nat.sc. 97% nat.sc. 50%
selec. 19% selec. 100% selec. 48%
hum. 35% +/-middle hum. 20% +/-bottom hum. 52% + bottom
nat.sc. 27% nat.sc. 15% nat.sc. 53%
selec. 36% selec. 13% selec. 55%
hum. 78% + bottom hum. 15% hum. 52% + middle
nat.sc. selec. 68% 87% nat.sc. selec. 11% 10% - nat.sc. selec. 58% 61%
hum. 73% +/-top hum. 13% hum. 23%
nat.sc. 58% nat.sc. 5% - nat.sc. 10% -
selec. 68% selec. 10% selec. 6%
hum. 70% + top hum. 42% +/-top hum. 55% +/-middle
nat.sc. 83% nat.sc. 38% nat.sc. 30%
selec. 84% selec. 55% selec. 29%
Implicit answers (for example, given without keywords given in the question) were found rarely and their location played a role. The correctness of decisionmaking decreased if the answers were located at the bottom of the Web-page.
The reasons for students' finding the answer when there was no answer: students' inadvertence or carelessness. Only 4% of students (humanities) always found the answers on Web-pages even if they were not given.
Natural science students indicated the answers presence less often, which may denote their fundamental text processing approach. The following conclusions can be made:
- the possibility to find the answers decreases with the increasing questions complexity;
- the possibility to find the answers increases in case they are explicit and contain keywords;
- the lower the implicit answer was placed on the Web-page, the mor difficult it was to find it. This is congruent with the AOI charts results.
(6) Studying students' comments
Students gave their comments about the main problems they faced while working with the texts. Generally, the following statements may be used in further research as students' regards:
(A) The presence of definitions and titles.
According to the students coments, the presence of definitions and titles makes work with the texts easier because they contain the main meaning and key concepts. Moreover, some students noted that using answers keywords in the texts would simplify the solution of the task. AOI results show the same.
(B) Emphasizing definitions, keywords and titles with italic, bold, or underlining.
(C) Dividing the texts into paragraphs (the more, the better). The students noted that this way is easier for texts absorbing.
(D) Using supporting material such as diagrams, charts, tables, formulas, quotations and others.
In students' opinion, it simplifies the work with the text. However, several students stressed that supporting graphics distracted them from solving the task, especially in cases when pictures did not have links to the text and were arranged chaotically. However, the vast majority of students suggested using visual material as simplifying because they had to look through the less amount of text information.
(E) Structuring text into columns.
In students' opinion, using only the straight text, in one column is inappropriate because it is impossible to get to the next idea. Though several students noted they had difficulties in searching information while working with two- or three-columns texts. The only conclusion we can make is that the rule of optimal length of lines
o
should be complied .
(F) Answers position at the beginning of the text. This statement coincides with the results of AOI charts and the analysis of the results of the participants' decisionmaking.
Conclusion
The ways of the educational text representation on the Web should comply with modern students' demands. The place of keywords and main didactic material should be concentrated at the place of students' attention to simplify understanding of the sources usefulness. The following conclusions could be made from this research:
(1) The students pay most attention to the text, especially to finding keywords from the querying, while searching the information. Graphical materials are the blind spot of students' attention except two situations: graphics containing text or familiar to students' experience images.
(2) F-effect is remarked in case of searching educational information. However, using different ways of information structuring may modify the way students look at the text. To simplify the task solution, a text should be split into semantic structural units9. The main idea of each unit should be mentioned in its first line.
(3) Significant differences in the places where natural science and humanities students search the information were not found. Consequently, the same parts of any source should be filled with useful information - top left corner of the text.
8 50-75 symbols for 10 inch diagonal monitors, 30-45 - for phones.
9 If it is appropriate.
Natural science students used to spend more time searching information. They treated the assignment profoundly. Such students found the implicit answers more rarely than humanities students did, but made the wrong decisions about the presence of the answers in case of their absence more rarely too (see table 3).
(4) The complexity of the text does not correlate with the main oculomotor characteristics (blinks, fixations) while searching the information. Therefore, it does not play an important role in solving such tasks.
These results can be used as the foundation for creating the educational source on the Web. The futher question is what appropriate amount of data should be used on the page to attract the students' attention.
Acknowledgements
We wish to express our sincere gratitude to Prof. L.N. Aksenovskaya, Dean of Psychology Faculty, Saratov State University for the given opportunity to work with the eye-tracker.
We sincerely thank the staff of the cognitive psychology laboratory, Saratov State University, namely Prof. T.V. Belykh, postgraduate student E. Zinchenko and postgraduate student Mr. E. Knyazev for their support and giving advice about the work process.
References
[1] McKay E.N. UIis Communication. Morgan Kaufmann, 2013. 378 p.
[2] Chiu T.K.F., Churchill D. Design of learning objects for concept learning: effects of multimedia learning principles and an instructional approach. Interactive Learning Environments, 2016, Volume 24, P. 1355-1370. DOI: 10.1080/10494820.2015.1006237
[3] Tuzun H., £rnar M. Guidelines for Transferring Residential Courses into Web. International Review of Research in Open and Distributed Learning, 2016, Vol 17, No 4, P. 145-165.
[4] Blignaut A.S. Infinite Possibilities for Using Eyetracking for Mobile Serious Games in Order to Improve User Learning Experiences. 14th World Conference on Mobile and
Contextual Learning, mLearn 2015, CCIS 560, P. 70-83. DOI: 10.1007/978-3-319-25684-
9_6
[5] Dolgunsoz E. Using eye-tracking to measure lexical inferences and its effects on reading rate during EFL reading. Journal of Language and Linguistic Studies, 2016, 12(1), P. 63-78.
[6] Wojdynski B.W., Bang H. Distraction effects of contextual advertising on online news processing: an eye-tracking study. Behaviour & Information Technology, 2016, Volume 35, P. 654-664. DOI: 10.1080/0144929X.2016.1177115
[7] Anson C.M., Schwegler R.A. Tracking the Mind's Eye: A New Technology for Researching Twenty-First-Century Writing and Reading Processes. College Composition and Communication, 2012, 64(1), P. 151-171.
[8] Raney G.E., Campbell S.J., Bovee J.C. Using Eye Movements to Evaluate the Cognitive Processes Involved in Text Comprehension. Journal of Visualized Experiments, 2014, (83). DOI: 10.3791/50780
[9] Hidalgo-Sotelo B., Oliva A. Person, place, and past influence eye movements during visual search. Proc. 32nd Annual Conf. of the Cognitive Science Society, 2010, P. 820-825. Austin, TX: Cognitive Science Society.
[10] Bekkering H., Neggers S.F. Visual search is modulated by action intentions. Psychological Science, 2002, 13(4), P. 370-374.
[11] Orchard L.N., Stern J.A. Blinks as an index of cognitive activity during reading. Integrative Physiological & Behavioral Science, 1991, 26, P. 108-116.
[12] Smilek D., Carriere J.S., Cheyne J.A. Out of mind, out of sight eye blinking as indicator and embodiment of mind wandering. Psychological Science, 21(6), 2010, P. 786-789. DOI: 10.1177/0956797610368063
[13] Martens S., Munneke J., Smid, H., Johnson A. Quick minds don't blink: Electrophysiological correlates of individual differences in attentional selection. Journal of Cognitive Neuroscience, 2006, 18(9), P. 1423-1438.
[14] Barabanschikov V.A., Zhegallo A.V. Eye-tracking: methods of registration eye movements in psychological research and practice. Cogito-Centre, 2014. 128 p.
[15] Soh O. Examining the reading behaviours and performances of sixthgraders for reading instruction: evidence from eye movements. Journal of e-Learning and Knowledge Society, 2016, Vol. 12, n. 4, P. 63-79.
