CC BY-NC-ND
3
Психология. Журнал Высшей школы экономики. 2022. Т. 19. № 4. С. 725-735. Psychology. Journal of the Higher School of Economics. 2022. Vol. 19. N 4. P. 725-735. DOI: 10.17323/1813-8918-2022-4-725-735
INCIDENTAL FINDINGS IN RELATION TO SUBSEQUENT SEARCH MISSES IN VISUAL SEARCH
O.S. RUBTSOVAa, E.S. GORBUNOVAa
a HSE University, 20 Myasnitskaya Str, Moscow, 101000, Russian Federation
Внезапные находки и пропуски при продолжении поиска в задаче на
зрительный поиск
О.С. Рубцова", Е.С. Горбунова"
'Национальный исследовательский университет «Высшая школа экономики», 101000, Россия, Москва, ул. Мясницкая, д. 20
Abstract
Incidental findings defined as valuable findings that are not searched purposely by the experts were originally discovered by radiologists. Despite the importance and great practical value of this phenomenon for visual search, it was almost not studied by cognitive psychologists and vision science experts. The current study aimed to examine experimentally incidental findings in visual search. The main objective was to clarify independence of incidental findings from subsequent search misses, another well-known visual search phenomenon. In order to do that, the standard experimental paradigm for detecting subsequent search misses was used. At the same time the stimuli material and tasks were created to closely fit the definition of incidental findings. The participants were asked to find the images of plastic bags and paper wastes (targets) among the images of leaves and snags (distrac-
Резюме
Внезапные находки, определяемые как ценные находки, которые не являются изначальной целью поиска экспертов, были первоначально обнаружены радиологами. Несмотря на важ ность и большую практическую ценность этого явления для визуального поиска, когнитивные психологи и специалисты по науке о зрении практически не изучали его. Представленное исследование направлено на экспериментальное изучение внезапных находок в зрительном поиске. Основная цель заключалась в том, чтобы прояснить вопрос независимости внезапных находок от пропусков при продолжении поиска, другого известного феномена зрительного поиска. Для этого использовалась стандартная экспериментальная парадигма для изучения пропусков при продолжении поиска. В то же время стимульный материал и экспериментальная задача соответствовали определению внезапных находок. Участники исследования должны были искать изображения пластиковых пакетов и бумажных отходов (целевые стимулы) среди изображений листьев и коряг
The research was supported by RSCF, project № 20-78-10055. Исследование выполнено при поддержке РНФ, проект № 20-78-10055.
tors) on the computer screen in a simulated "garbage collection" task. Their accuracy and reaction times were analyzed. Specifically, the trials with a single target were compared with dual-target trials. The findings revealed that subsequent search misses, but not incidental findings, were observed. The results suggest that incidental findings may be closely related to subsequent search misses. As well as that, the difficulty of the task, particularly induced by target-distractor similarity, may be one of the major factors leading to the emergence of subsequent search misses instead of incidental findings.
Keywords: visual attention, visual search, incidental findings, subsequent search misses.
Olga S. Rubtsova — Junior Research Fellow, Laboratory for the Cognitive Psychology of Digital Interface Users, HSE University.
Research Area: cognitive psychology, visual attention, visual search. E-mail: olga.rubtsova98@gmail.com
Elena S. Gorbunova — Laboratory Head, Laboratory for the Cognitive Psychology of Digital Interface Users, HSE University, PhD, Associate Professor.
Research Area: Cognitive science, cognitive psychology, visual attention, visual perception, working memory, visual search, usability
E-mail: gorbunovaes@gmail.com
(дистракторы) на экране компьютера, задание имитировало ситуацию сбора мусора на субботнике. Были проанализированы точность и время реакции. В частности, пробы с одним целевым стимулом сравнивались с пробами с двумя целевыми стимулами. В результате эксперимента были обнаружены пропуски при продолжении поиска, но не внезапные находки. Сделано предположение о том, что внезапные находки могут быть тесно связаны с пропусками при продолжении поиска. Кроме того, сложность задачи, в частности, вызванная сходством целевых стимулов и дистракторов, может быть одним из основных факторов, приводящих к проявлению пропусков при продолжении поиска вместо внезапных находок.
Ключевые слова: визуальное внимание, зрительный поиск, внезапные находки, пропуски при продолжении поиска.
Рубцова Ольга Сергеевна — младший научный сотрудник, научно-учебная лаборатория когнитивной психологии пользователя цифровых интерфейсов, Национальный исследовательский университет «Высшая школа экономики». Сфера научных интересов: когнитивная психология, визуальное внимание, зрительный поиск. Контакты: olga.rubtsova98@gmail.com
Горбунова Елена Сергеевна — заведующая лабораторией, Научно-учебная лаборатория когнитивной психологии пользователя цифровых интерфейсов, Национальный исследовательский университет «Высшая школа экономики», кандидат психологических наук, доцент. Сфера научных интересов: когнитивная наука, когнитивная психология, визуальное внимание, визуальное восприятие, рабочая память, зрительный поиск, юзабилити. Контакты: gorbunovaes@gmail.com
Incidental findings are widely known and discussed in radiology. They were defined as clinically valuable findings that are not related to the initial purpose of the search (Beigelman-Aubry et al., 2007). There was almost no research from the point of vision science and cognitive psychology, though. One of very few studies was conducted by Jeremy Wolfe and colleagues (Wolfe et al., 2017). The authors describe the incidental findings as a separate phenomenon, which differs from other, more widely investigated effects. As well as that, the study provides a novel
experimental method for studying incidental findings. A hybrid search task was used, which means that subjects partially relied on working memory resources while searching for targets. Specific and categorical search were used in this study. Categorical search seems to demand more attentional resources in comparison to specific search (Maxfield & Zelinsky, 2012). The authors suggested that when categorical and specific search were performed simultaneously, incidental findings would be detected, because categorically defined targets fit the definition of the effect in this case.
One distinguishing feature of incidental findings is the low prevalence of the items. Of course, medics are aware of such artefacts as lesions while inspecting a medical image of a patient with possible pneumonia. However, it is quite a rare occasion to find a lesion in such a common, probably every-day task. The work of Hout and colleagues shows the strong low-prevalence effect for rapid serial visual presentation task in comparison to standard visual search task (Hout et al., 2015). The results stated that in both cases the high-prevalence targets were found significantly faster and more accurately then low-prevalence ones.
There is an "opponent" for incidental findings in visual search, however, which is called "subsequent search misses" (SSM). SSM is the effect of accuracy decrease for the second target after successful identification of the first target (Adamo et al., 2013). SSM errors refer to targets that are to be found after already identifying at least one. They are typically very similar to initially found targets. Incidental findings are, however, typically rare low-prevalent targets that have very little resemblance to originally discovered target. What is more, they are not considered to be primary targets of search. The question is: are incidental findings really that different from SSM? In case of incidental findings, the target still can be considered quite relative to the primary purpose of search. In real medical search, no matter what the initial suspicion is, the critical goal is to perform a thorough search and to identify all possible abnormalities. Regarding target resemblance, there were a few works examining the target differences in SSM: either in perceptual characteristics of stimuli (Gorbunova, 2017) or in a more complex way (e.g., Biggs et al., 2015). Thus, the possibility is that incidental findings and SSM differ quantitatively, rather than qualitatively. One of the arguments is that the factors, resulting in missing second targets, may be the same for the both effects.
These mechanisms may include resource depletion. When a searcher finds the first target, their attentional resources become consumed by it, so that there are fewer resources for detecting the next target (Cain & Mitroff, 2013). Another possibility is proposed by the perceptual set hypothesis: the perceptual features of the first identified target prime the following search, so that the searcher is more likely to attend to objects alike. It was revealed, for instance, in a study by Gorbunova (Gorbunova, 2017) that the factor of target similarity reduced the effect of SSM.
The aim of the proposed study is the experimental verification of independence of incidental findings from SSM. A standard experimental paradigm for SSM was chosen. The main question was: would incidental findings or, rather, SSM be detected in the experiment? The criterion for incidental findings was the decreased accuracy for finding either the only non-typical target or the second non-typical
target after detecting a typical target. The criterion for SSM was the decreased accuracy for finding any second target after detecting the first one.
Method
Participants
There were originally 15 participants, all students of National Research University Higher School of Economics, with normal or corrected to normal vision, without any neurological or psychological problems. Data from two participants was excluded from further analysis, due to misunderstanding the instructions. Therefore, there were ten females and three males, their age ranged from 18 to 22 years old (M = 19.08, SD = 1.12).
Stimuli
In order to create a task relative to common real-life situation, it was decided to simulate a "garbage collection" task. Human wastes were chosen as stimuli, since they can be distributed to several categories. Two such categories were chosen for experimental purposes as targets: plastic bags and paper wastes. Leaves and snags were chosen as distractors. Snags were wooden pieces without any branches, so that they would not differ much perceptually from other stimuli. There were five objects chosen for each category, and each of these objects was represented in three colors: green, yellow and brown. Hence, there were 60 images in total. They were all 4.26° X 4.3° in size. They were all presented on a dark-brown background, the color resembling soil. There were also two additional buttons made for partici-pantss answers, they contained the words "NO" and "OK", correspondingly.
In order to represent incidental findings in experimental settings, the salience of two types of targets was specified. It was decided to use plastic bags as typical targets, while paper wastes served as non-typical targets less significant to the initial search. There were five experimental conditions created to satisfy the occurrence of incidental findings in a real-life situation. They were conditions with two typical targets (plastic bags), one typical target, no targets, one non-typical target (paper wastes) and two targets differing in typicality (both a plastic bag and paper). The last condition was considered critical for identifying incidental findings experimentally, since it contained a non-typical item on the same set as a common one. The frequency of different trial types was varied, and it is illustrated in Table 1.
Table 1
The Distribution of Different Target Types in Experiment 1
0 targets 30%
1 typical target (plastic bag) 37%
1 non-typical target (paper wastes) 5%
2 typical targets 18%
2 targets: 1 typical and 1 non-typical 10%
The stimuli were distributed randomly across the screen within a 5/5 invisible grid. They could shift up to 118 pixels horizontally and up to 25 pixels vertically randomly from the centers of the cells in each trial. Moreover, their orientation was also varied from trial to trial. Overall, there could be 12, 16 or 20 stimuli in each individual trial, the number of targets varied from 0 to 2. Examples of experimental trials for different conditions are presented in Figures 1, 2, and 3.
Procedure
The participants were asked to search for any targets that were considered human wastes; however, it was stated that there were many plastic bags in particular. That way, the initial task implied that plastic bags were typical targets, in contrast to
Figure 1
An Example of an Experimental Trial (the Condition with Both Typical and Non-Typical Targets)
Figure 2
An Example of an Experimental Trial (the Condition with One Typical Target)
Figure 3
An Example of an Experimental Trial (the Condition with One Non-Typical Target)
paper wastes. As well as that, the participants were informed that they could find 0, 1 or 2 targets in each individual trial. They were asked to search for targets as quickly as possible, as soon the trials began. They used a computer mouse to click on targets and additional buttons at the bottom of the screen in order to report their answers. If there were two targets present, the participants needed to click on each of them sequentially. If there was only one target present, they needed to click on it and then click on the button "OK". Finally, if there were no targets, they had to click two times on the button "NO". After the end of each trial, the participants could have some rest if needed and begin the new trial by pressing the spacebar.
Before the main part of the experiment, the participants completed a short training block. If they were confident in understanding all the instructions, they proceeded to the main part. The first 60 trials did not contain non-typical targets, since it was relevant for keeping the simulation of incidental findings valid. The next trials were randomly distributed among all five experimental conditions. Overall, there were 495 trials in the main block of the experiment. An illustration of the experimental procedure is presented in Figure 4.
A standard computer and a monitor with a screen resolution of 1024X768 and a refresh rate of 85 Hz were used. The experiment was constructed in PsychoPy v.1.90.2. This version of PsychoPy was used to conduct the experiment. The partici pants used a standard computer mouse and a standard keyboard in order to report their answers.
Results
Accuracy and reaction time for both mouse clicks were analyzed. The condition with no targets was excluded from the analysis, since it was used for controlling participants^ attention to the instruction and did contain any relevant data.
Figure 4
The Design of the Experiment
second click
The error analysis was made for different conditions. For conditions with one target those parameters were computed for those trials, in which the click on the target was followed by a click on the "OK" button. For two typical targets present in one trial, the accuracy and reaction time of identifying the second target, no matter in what order the targets were clicked, were measured. For one typical and one non-typical target in the same trial, the accuracy and reaction time of the non-typical target were computed, only if it was found after the typical one. Accuracy and reaction time then were compared for the relevant experimental conditions. Repeated measures ANOVA and pairwise comparisons with Bonferroni-Holm adjustment were chosen as analysis methods. The Greenhouse-Geisser corrections were applied, when Mauchly's sphericity tests were significant.
Accuracy
ANOVA revealed the significant impact of condition factor: F(2, 25) = 6.609; p = .005; ^p2 = 0.355. Pairwise comparisons with Bonferroni-Holm adjustments revealed significant differences between following conditions: conditions with one typical target and one non-typical target (p = .045), conditions with two typical targets and one non-typical target (p = .031) and conditions with one non-typical target and both typical and non-typical targets in the same trials (p = .049). The results are presented in Figure 5.
Reaction time (first click)
ANOVA revealed the significant impact of condition factor: F(3, 36) = 63.495; p < .001; ^p2 = 0.841. Pairwise comparisons with Bonferroni adjustments revealed significant differences between all four conditions: with one typical target and two typical targets (p < .001), with one typical target and one non-typical target (p = .001), with one typical and both typical and non-typical targets (p < .001), with two typical and one non-typical target (p = .005), with two typical and both typical and non-typical targets (p = .033) and with one non-typical and both typical and non-typical targets (p < .001). The results are visualized in Figure 6.
Reaction time (second click)
ANOVA revealed the significant impact of condition factor: F(2, 22) = 15.778; p < .001; ^p2 = 0.568. Pairwise comparisons with Bonferroni-Holm adjustments
Figure 5
The Results of Accuracy (error bars represent 95% confidence intervals)
BO
llll
One typk»! tlrfel Two lyp<(«l targeti On* nan-typical target Both typic«! i«d n Ort-typical
target*
Figure 6
The Results of Reaction Time (First Click) (error bars represent 95% confidence intervals)
Figure 7
The Results of Reaction Time (Second Click) (error bars represent 95% confidence intervals)
revealed significant differences between following conditions: conditions with one typical target and two typical targets (p < .001), with one typical target and both typical and non-typical targets in the same trials (p = .008), with two typical targets and one non-typical target (p = .003) and conditions with one non-typical target and both typical and non-typical targets in the same trials (p = .008). The results are visualized in Figure 7.
Discussion
The results of the experiment illustrate the phenomenon of subsequent search misses. In the critical condition with both typical and non-typical targets in the same trials, the participants made significantly more mistakes in comparison to the condition with one non-typical target. At the same time, the decreased accuracy for finding the only non-typical target in comparison to typical target was not detected. Hence, such results suggest that overall SSM were present, while incidental findings were not observed. In case of incidental findings we would expect to see significantly better performance in the condition with one typical target compared to the condition with one non-typical target. We would also expect to see no significant differences between conditions with one non-typical target and with both typical and non-typical targets in the same trials. None of those trends were apparent. The present effect of SSM is similar to those detected in previous research (e.g. Fleck et al., 2010). It is interesting, since the stimuli material fit closely the definition of incidental findings. We suggest that the target-distractor similarity factor might have been responsible for such results. The targets shared colors with dis-tracters, making them hard to identify for the participants. Thus, incidental findings and SSM may, in fact, be separated from each other by altering the target-dis-tractor relationship: the more similarities they share, the higher probability for SSM is. This factor plays a significant role in visual search mechanisms, which was supported by data (e.g., Duncan & Humphreys, 1989). Interestingly, SSM were not detected for typical targets: there were no significant differences between conditions
with one typical and two typical targets. This finding can be explained by high prevalence of these targets: they appeared in much more trials and, moreover, were the targets of initial search. Overall, the findings regarding typicality of targets resemble those reported by Hout and colleagues, where high-prevalence targets were found more accurately than low-prevalence ones (Hout et al., 2015). A very interesting finding of this experiment was the increased accuracy for the condition with one non-typical targets in comparison to the condition with one typical target. Not only the participants were more accurate detecting less typical targets, but they also found them significantly faster. This contradicts the assumption of the prevalence-effect hypothesis, since the reversed results would be expected in this case. A possible explanation of such findings is the so-called "novel pop-out effect". It suggests that in some cases novel objects are better identified when presented among familiar items. In a study by Strayer and Johnston it was suggested that such an effect depends on the early attentional processing of objects (Strayer & Johnston, 2000). In our experiment the first sixty trials did not contain any non-typical stimuli. This was intentional, since it would make the bags more familiar, therefore, typical for the subjects. At the same time, when paper appeared for the first time, it could be processed as novel outstanding items on set. We propose that a better familiarity of bags, determined by the bigger proportion of trials with them, could be responsible for such results, resembling the mentioned novel pop-out effect, although it is still quite a controversial finding that needs further clarification.
The results of reaction time represent, on the one hand, a typical increase of time needed to find the first target for trials with one target in comparison to trials with two targets, regardless of the target typicality. It takes statistically less time to come across a target, if there are two of them on a display (e.g., Kwak et al., 1991). Similarly, it took significantly longer to report the absence of the second target in trials with only one target than to find the second object in trials with two targets. It would take statistically more time to look through all the distractors than to stumble across one target. Regarding the results for the reaction time of a second click, the critical condition with both typical and non-typical targets was associated with a significantly faster reaction time than the baseline condition with one non-typical target. In other words, it took less time to find the second non-typical target than to report the absence of the second target. The typical finding in case of SSM would be the increased results for reaction time in conditions with two targets (Gorbunova, 2017).
Conclusions
In summary, the presented study examined the effect of incidental findings in visual search. The primary goal was to clarify the independence of incidental findings from the SSM effect. The results of the experiment show that, despite the stimuli and experimental task closely fitting the definition of incidental findings, SSM were still observed. It was proposed that the difference between SSM and incidental findings may lay in the specifics of target-distractor similarity effects.
Further research may use the obtained results in order to reveal the specifics of underlying mechanisms of these effects, as well as serve for the professional fields, such as medicine and security screening.
References
Adamo, S. H., Cain, M. S., & Mitroff, S. R. (2013). Self-induced attentional blink: A cause of errors in multiple-target search. Psychological Science, 24(12), 2569-2574. https://doi.org/10.1177/0956797613497970 Beigelman-Aubry, C., Hill, C., & Grenier, P. A. (2007). Management of an incidentally discovered pulmonary nodule. European Radiology, 17(2), 449-466. https://doi.org/10.1007/s00330-006-0399-7 Biggs, A. T., Adamo, S. H., Dowd, E. W., & Mitroff, S. R. (2015). Examining perceptual and conceptual set biases in multiple-target visual search. Attention, Perception, and Psychophysics, 77(3), 844855. https://doi.org/10.3758/s13414-014-0822-0 Cain, M. S., & Mitroff, S. R. (2013). Memory for found targets interferes with subsequent performance in multiple-target visual search. Journal of Experimental Psychology: Human Perception and Performance, 39(5), 1398-1408. https://doi.org/10.1037/a0030726 Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review,
96(3), 433-458. https://doi.org/10.1037/0033-295x.96.3.433 Fleck, M. S., Samei, E., & Mitroff, S. R. (2010). Generalized "Satisfaction of Search": adverse influences on dual-target search accuracy. Journal of Experimental Psychology Applied, 16(1), 60-71. https://doi.org/10.1037/a0018629 Gorbunova, E. S. (2017). Perceptual similarity in visual search for multiple targets. Acta Psychologica,
173, 46-54. https://doi.org/10.1016Zj.actpsy.2016.11.010 Hout, M. C., Walenchok, S. C., Goldinger, S. D., & Wolfe, J. M. (2015). Failures of perception in the low-prevalence effect: Evidence from active and passive visual search. Journal of Experimental Psychology: Human Perception and Performance, 41(4), 977-994. https://doi.org/10.1037/xhp0000053 Kwak, H.-W., Dagenbach, D., & Egeth, H. (1991). Further evidence for a time-independent shift of the focus of attention. Perception & Psychophysics, 49(5), 473-480. https://doi.org/10.3758/bf03212181 Maxfield, J. T., & Zelinsky, G. J. (2012). Searching through the hierarchy: How level of target categorization affects visual search. Visual Cognition, 20(10), 1153-1163. https://doi.org/10.1080/ 13506285.2012.735718
Strayer, D. L., & Johnston, W. A. (2000). Novel popout is an attention-based phenomenon: An ERP
analysis. Perception & Psychophysics, 62(3), 459-470. https://doi.org/10.3758/bf03212098 Wolfe, J. M., Soce, A. A., & Schill, H. M. (2017). How did I miss that? Developing mixed hybrid visual search as a "model system" for incidental finding errors in radiology. Cognitive Research: Principles and Implications, 2(1), Article 35. https://doi.org/10.1186/s41235-017-0072-5
