CC BY
58
THE NATURE OF CHESS EXPERTISE: KNOWLEDGE OR SEARCH?
Ekaterina E. Vasyukova
Lomonosov Moscow State University Moscow, Russia
In this article, we briefly survey studies of the nature of expertise, and we present the results of research directed at evidence of the complicated nature of expertise, which is most effectively seen in experts' use of a transfer mechanism. The phenomenon of the transfer of verbalized operational senses (VOS) is analyzed and is investigated on the basis of the sense theory of thinking, as proposed by Tikhomirov (1969, 1984).
It is shown that VOS transfer manifests itself in diverse forms. It seems to be dependent on the factors of chess position and the age and skill level of the player. Diverse forms of transferring are associated with a change in separate indices of VOS volume, structure, depth, and degree of consciousness in a connected position. VOS transfer is found more in skilled than in unskilled chess players; skilled players demonstrate selectivity of search in a connected position. VOS transfer is associated not simply with the repetition and copying of some forecasts, which give the direction of search, but also with using and transforming the results of previous verbal searches.
Keywords: chess expertise, knowledge, search, memory, thinking, verbalized and unverbalized operational senses, transfer
Comparison of the peculiarities of the knowledge and cognitive processes of experts and novices has been one of the important directions of cognitive studies since the 1980s (Velichkovsky, 2006). According to Simon, an expert is a person who has successfully worked in a certain area during not less than 10 years (see Velichkovsky, 2006, p. 244). The goal of these studies is the creation of expert systems, which can operate along with knowledge in certain fields to develop recommendations and solve problems.
Chess has historically been the primary domain for psychological studies of human expertise. Among the advantages of chess as a research
domain is an international rating system that enables precise comparisons of players' skills.
In a number of works, the nature of expertise is reduced to the functions and content of memory. The alternative point of view is that the ability of experts to find the best solutions is connected not so much with already-existing templates that are used for the rapid identification of configurations but with the processes of active search and change and with the successful interpretation of the material. A survey of the research shows that expertise cannot be associated with only one level of cognitive processes - inferior (perception, pattern recognition, memory) or high (thinking, formation of concepts, logic, decision making, problem solving, creativity, human intellect), in the terminology of Solso (1996); verbal or nonverbal, in the terminology of Tikhomirov (1984). We propose that the complicated nature of expertise is seen most effectively in experts' use of a transfer mechanism.
The basis of our research is Tikhomirov's sense theory of thinking (1969, 1984). In it, the process of problem solving is understood as the formation, development, and complicated interaction of operational sense structures of different types and levels (verbalized and unverbal-ized operational senses, VOS and UOS). Operational sense designates individual mental reflection of the object, which stems from the different research actions undertaken by the subject and which changes as a task is being tackled.
Sense structures determine the selectivity and direction of research activity. Sense transfer is a major resource for enhancing selectivity in the problem-solving process.
UOS transfer is associated with the use of the results of unverbal-ized research activity in subsequent situations. Practical action in these situations proves to be preoriented. Situations perform as a single sense system.
UOS transfer is the main factor in the reduction of a search.
The coincidence of an anticipated change in a situation with a real one is a condition of the transfer of the research activity's products, received previously, to subsequent situations. The selectivity and purpose-fulness of the search are created in this way.
UOS and VOS are transferred from one situation to another in different degrees. But the general principle of greater VOS capacity for being transferred is a hypothesis.
To investigate the phenomenon of VOS transfer from one situation to another was the goal of our research. The specific task of the research was to reveal the VOS characteristics of chess players of different qualifications and ages while they were in the process of selecting the best moves in connected and unconnected chess positions.
Brief Review of the Research on the Nature of Expertise
In chess, expertise is based on the following characteristics:
1. functions and contents of memory (De Groot, 1946/1965; Ericsson & Kintsch, 1995; Gobet & Simon, 1996, 2000);
2. knowledge-based perceptual advantage: the broad useful field of view of experts, inside of which they simultaneously conduct an analysis of the situation and work with the relationships between the figures and the whole configurations (Reingold, Char-ness, Pomplun, & Stampe, 2001; Reingold, Charness, Schultetus, & Stampe, 2001);
3. reduced nonverbal search activity as a result of the effective action of mechanisms of anticipation and transfer by a chess master in comparison with a chess player of the third category (Tikhomirov, 1969, 1984);
4. processes of active search, change, and successful interpretation of material (Holding, 1979).
In the late 1960s, with the help of the method of registering the eye movements of players (the use of which led to the creation of the sense theory of thinking), Tikhomirov showed that masters, unlike chess players of the third category, have more reduced nonverbal search activity as a result of the effectiveness of forecasting mechanisms and the transfer of the results of nonverbal searches from one situation to another; the formation of selection criteria is more frequent in masters (activity is constructed by the type of production of the search needs in it); in masters, more emphasis is placed on the formation of forecasts than on the process of searching for the means of achieving them. A chess player of the third category spends more time making decisions (that are not even the best), more intensively examines his or her own figures compared with the figures of the rival, and has a smaller average time of fixation.
In our study of the main characteristics in the development of VOS in the process of selecting the best move in difficult chess positions, we found that the qualifications of chess players had an effect on such indices as the volume of VOS, the number of terminal nodes (variants and subvariants), the number of episodes (variants), and the null moves (vague moves) (Vasyukova, 2006). On these indices, highly skilled chess players surpass low-skilled players.
Chess research has been useful in characterizing the trade-offs seen among memory, perception, and problem-solving performance (Gobet & Charness, 2006).
The analysis of differences between experts and novices in various areas, such as reading, juggling, and medical diagnostics, has allowed researchers to single out the key characteristics of experts: not only the greater volume of knowledge, but sensitivity to relevant characteristics of the situation, flexibility, skill in examining several alternative interpretations, and moderate abstract conceptual representations (see Velichkovsky, 2006). Having moderately abstract conceptual representations makes it relatively easier to manipulate knowledge: conceptual schemes with an average amount of abstraction facilitate drawing a conclusion by analogy and transferring solutions from one subject area to another.
Review of the research on the nature of expertise shows that the development of expertise goes on at several levels of organization at once, from perceptual automatisms to metacognition strategies; it cannot be associated with only one level of cognitive activity, lower or higher, related, respectively, to perception and memory or thinking and problem solving. However, the connection between memory (knowledge) and thinking (search) as a function of skill still needs to be uncovered (Gobet & Charness, 2006).
Research Project
Subjects
The subjects were 44 tournament-rated chess players of different ages (from 16 to 80) and skill levels (from second category to international grandmaster). Participants were divided into four equal groups. Each group included chess players with Elo points > 2100 (+/- 40 years old) and < 2100 (+/- 40 years old).
Materials
Four sets of middle-game chess positions were provided (each group received its own set of two positions). Sets A and B included connected positions (A1 and A2; B1 and B2), where the second position was a continuation of the first position after one move for each side. Sets C and D included unconnected positions (B1 and A2; A1 and B2) (see chess diagrams in Figures 1-4). An audio-cassette recorder was also used.
Figure 1. Position A1. White to move Figure 2. Position A2. White to move
Figure 3. Position B1. White to move
Figure 4. Position B2. White to move
Procedure
The researchers elicited concurrent "think-aloud" verbal reports from individual participants while they attempted to select the best move for each position. There was a 5-minute time limit per position.
All verbal reports were transcribed and converted into problem-behavior graphs (Newell & Simon, 1972). In the graph of the process of decision making for each position, all variants and subvariants were indicated.
On the basis of the principal VOS characteristics, such as volume of the conscious orientation zone, structure, depth, and degree of "understanding" (or "consciousness" degree) (Vasyukova, 2001), the following indices of principal VOS characteristics were determined.
The indicators of volume of the conscious orientation zone are the number of
• unique base moves (moves/candidates);
• unique other moves (nonbase moves);
• episodes (variants);
• terminal nodes (variants and subvariants);
• null moves (vague moves): they appear when the chess player examines the plan for only one side without indicating answers of the rival, which prove indeterminate.
The indicators of structure are the number of
• repeated base moves;
• repeated other moves.
The indicators of depth are
• maximum depth;
• mean depth;
• "branchiness" (the relation of the number of terminal nodes to the number of episodes).
The indicators of the degree of "understanding" ("consciousness" degree) are the number of
• evaluations;
• evaluations of the vagueness of positions after their transformation.
The time to solution (sec) was recorded.
Treatment of statistical data was based on ANOVA, the criteria of the Kruskal-Woless test and a median test, the criteria of the Wilcoxon and Mann-Whitney tests.
The effects of four controlled factors and their interactions with the 12 indices of VOS and the decision time were investigated. The following four factors were controlled:
• set of positions (A, B, C, D);
• position (A1, A2, B1, B2);
• age (< 40 years old, > 40 years old);
• qualifications (< 2100 Elo points, > 2100 Elo points).
The effects of three controlled factors and their interactions with the 12 indices of VOS and the decision time were investigated also. The following three factors were controlled:
• the objective possibility of transfer (sets of connected positions, sets of unconnected positions);
• age (< 40 years old, > 40 years old);
• qualifications (< 2100 Elo points, > 2100 Elo points).
Four of the most skilled chess players were selected from each group of participants (their Elo points varied from 2180 to 2658, their age range was from 16 to 66 years old).
For each subject (n=16) in each subgroup for each position (A1 and A2 of set A; B1 and B2 of set B; A2 of set C; B2 of set D) the following VOS indices were analyzed: (1) number of researched (called) elements; (2) number of researches (calls); (3) mean number of researches of one element; (4) maximum number of researches of one element; (5) number of significant elements (those elements that were researched more often than one element on average); (6) number of sense substances (those elements that were researched more often than one significant element on average).
The same procedure was followed for the least skilled chess players from each group of participants (n=16).
Principal Results and Discussion
In set A, positions A1 and A2 did not differ significantly on VOS indices and solution time according to the Wilcoxon criteria.
In set B, positions B1 and B2 differed significantly on two VOS indices - on number of repeated other moves (p<0.05) and null moves (p<0.01) - according to the Wilcoxon criteria. There were more repeated other moves and fewer null moves in position B2 than in position B1 (Table 1).
Table 1
Characteristics of the Distribution of VOS Indices in Positions B1 and B2
Set B VOS indices N Median I-III (quantity) Min Max
Position B1 Number of other repeated moves 11 4 3-7 1 12
Number of null moves 11 8 4-11 1 18
Position B2 Number of other repeated moves 11 9 4.5-13 1 33
Number of null moves 11 2 1-3.5 0 10
Results revealed an effect of position on the number of repeated other moves (p<0.05), null moves (p<0.05), and branchiness (p<0.05). In position B2 the number of repeated other moves and the amount of branchiness were maximal (8.5 and 1.67); the number of null moves was minimal (3.32).
Thus, the type of position influences VOS transfer from one situation to another and is connected with it. VOS transfer may be observed in a change of separate VOS indices of structure, depth, and volume. A significant decrease in the number of null moves (plans for one side), a rise in the number of repeated other moves, and branchiness in a connected position can serve as the criteria for VOS transfer into it.
A decrease in the number of plans indicates a transition to a later stage of analysis and VOS development. Repetition of moves is connected with the phenomenon of re-examination, which is an indicator of VOS development. Consequently, VOS transfer is closely associated with VOS development. A rise in branchiness, connected with the number of subvariants, indicates the growth of changes in some parts of the variants. In cognitive psychology, variation is considered one of the general metaprocedures for understanding, and it plays an important role in the process of problem solving and creativity in art (Velichk-ovsky, 2006).
Results revealed the joint influence of set, position, and age on branchiness (F=4.25, p<0.05). For chess players younger than 40 years old branchiness was higher in position A2 than in position A1 (set A) and was higher in position B2 than in position B1 (set B).
For chess players in this age group branchiness was higher in connected positions than in analogous unconnected positions: branchiness was higher in position A2 (set A) than in position A2 (set C) and in position B2 (set B) than in position B2 (set D).
We can conclude that for chess players younger than 40 years the transfer of VOS may be observed in a rise in the branchiness of searches - that is, in the depth of VOS.
This tendency wasn't observed in chess players older than 40 years. For these players branchiness was lower in position A2 than in position A1 (set A); it was the same in positions B1 and B2 (set B). Branchiness in a connected position may be the same as in an analogous unconnected position (position A2) or lower than in an analogous unconnected position (position B2).
The effect of the interaction of two factors (set and qualifications) on decision time (p<0.01) was observed. Chess players with Elo points > 2100 on average chose the best move sooner in set A (364 sec) and set B (425 sec) than in set C (472 sec) and set D (513 sec).
The joint effect of two factors (Elo and the objective possibility of transfer) on decision time (F=10.555, p=.002), number of unique other moves (F=6.241, p=.015), mean depth (F=4.126, p=.046), and number of evaluations (F=4.810, p=.031) was observed (Figures 5-8). In sets of
□ Unconnected Positions D Connected Positions
Sets off:
Elo_rank
Figure 5. Average solution time in sets of connected and unconnected positions by chess players with Elo <2100 (Elo rank 1) and >2100 (Elo rank 2).
Sets off:
O Unconnected Positions O Connected Positions
Figure 6. Average number of unique other moves in sets of connected and unconnected positions by chess players with Elo <2100 (Elo rank 1) and >2100 (Elo rank 2).
connected positions compared with sets of unconnected positions, the values of these indices decreased for chess players with Elo > 2100 and rose for chess players with Elo < 2100.
Thus, for chess players with Elo > 2100, the transfer can be observed in the reduction of the number of unique other moves, evaluations, mean depth, and decision time.
Sets off:
□ Unconnected Positions
□ Connected Positions
Figure 7. Average mean depth in sets of connected and unconnected positions bу chess players with Elo <2100 (Elo rank 1) and >2100 (Elo rank 2).
Sets off:
G Unconnected Positions □ Connected Positions
Figure 8. Average number of evaluations in sets of connected and unconnected positions bу chess players with Elo <2100 (Elo rank 1) and >2100
(Elo rank 2).
A special analysis was made of VOS volume and structure in connected and unconnected positions by the most (and least) skilled chess players.
Among the most skilled chess players in our sample the following features of VOS volume and structure in connected and unconnected positions were found (Table 2).
Table 2
Features of VOS Volume and Structure in Connected and Unconnected Positions in Subgroups of the Most Skilled Chess Players
Set Position N Number of called elements Mean number of calls of one element Maximum number of calls of one element % of significant elements in the general number of elements % of sense substances in the general number of elements
A A1 4 19.00 2.76 8 39 14
A2 4 14.00 1.89 4 32 21
C A2 4 21.25 3.57 9 40 15
B B1 4 26.50 3.39 13 33 12
B2 4 16.50 5.00 14 39 18
D B2 4 17.75 3.91 12 31 15
In set A, in position A2 the number of called elements was 0.74 times the number of called elements in position A1. The following VOS indices decreased too: the mean number of calls of one element in position A2 was 0.68 times the mean number in position A1; the maximum number of calls of one element in position A2 was 0.5 times the maximum number in position A1; the percentage of significant elements in the general number of called elements in position A2 was 0.82 times the percentage in position A2. But the percentage of sense substances became 1.5 times higher (21% against 14%). The solution time decreased from 222 sec in position A1 to 161 sec in position A2.
In unconnected position A2 (set C) in comparison with connected position A2 (set A) the number of called elements and the mean and maximum number of calls of one element were larger. There was a higher percentage of significant elements, but a lower percentage of sense substances in the general number of called elements. In unconnected position A2 subjects spent 1.58 times more time than in connected position A2.
In set B, in position B2 the number of called elements was 0.63 times the number in position B1 (16.5 against 26.5). But in position B2 the mean number of calls of one element was 1.47 times more, and the maximum number of searches of one element was a little more than in position B1 (14 against 13).
In position B2 significant elements and sense substances constituted a greater percentage of all called elements than they did in position B1. Subjects selected the best move in position B2 faster than in position B1 (203 sec versus 261 sec).
In unconnected position B2 (set D) in comparison with connected position B2 (set B) there were 1.08 times more called elements. But the mean and maximum number of calls of one element and the percentage of significant elements and of sense substances were lower. The time of solution rose in unconnected position B2 in comparison with connected position B2 (223 sec versus 203 sec).
Analysis of the mean values of the indices of VOS volume and structure in connected and unconnected positions by the least skilled chess players in our sample showed: the number of called elements in position A2 in comparison with position A1 and in position B2 in comparison with position B1 increased a little but was almost the same as in the corresponding isolated positions A2 and B2. The solution time in position
A2 in comparison with the time in A1 decreased slightly (from 217 sec to 204 sec) and slightly exceeded the solution time in isolated position A2, which was 202 sec. The solution times in positions B1 and B2 were virtually the same, respectively 300 sec and 299 sec; the solution time in related position B2 slightly exceeded the search time in isolated position B2, which was 294 sec.
We can conclude that VOS transfer is a characteristic of all highly skilled chess players and may be connected with a decreased number of called elements and a rise (or decrease depending on the type of position) in the mean and maximum number of calls of one element, and with a rise in the percentage of sense substances and in the most often called elements in the general number of called elements. An important consequence of VOS transfer is a reduction of the time taken to select moves.
Thus, we have found that the transfer of VOS from one situation to another situation that is associated with it depends on the type of chess position and the qualifications and ages of the subjects. These factors determine the fact of transfer and its form. Different forms of transfer are related with changes in certain indices of volume, structure, and depth, as well as with the VOS degree of consciousness in a connected position.
In subgroups of the most skilled chess players (four persons in set A and four persons in set B) we analyzed modifications of the system of elements in position A2 (in comparison with position A1) and in position B2 (in comparison with position B1).
In position A1 one subject didn't anticipate the real changes in the situation (moves f4-Nc6). Three subjects anticipated them, but considered the changes to be scarcely probable. For this subgroup in position A2 on average 44% of the elements were preserved, 56% of the elements were rejected, and 26% of new elements were added.
In position B1 two subjects didn't anticipate the real changes in the situation (moves f4-h6). One subject anticipated them (on 4 ply) but considered his predictions as scarcely probable. One subject of this subgroup not only anticipated the real changes of the situation (on 9 ply) but also considered the anticipated changes to be highly probable. The best move selected by him, f4 with following h6, coincided with the real changes in the situation, which led to position B2. For this subject 71% of the elements were preserved, 29% of the elements were rejected, and 14% of new elements were added. For this subgroup, in position B2 on
average 47% of the elements were preserved, 53% of the elements were rejected, and 14% of new elements were added.
Thus, we can say that an important consequence of VOS transfer is the ability to operate in a specific situation with a ready-made set of elements (the formation of such a system can be traced in previous activity). This system is transformed only by excluding a number of elements (about 30% to 50%) and including some new elements (about 15% to 25%). The level of this transformation depends on the anticipation of real changes in the situation and the subjective assessment of the probability of these anticipated changes. It is minimal not only when the real changes are anticipated but when the predictions are considered to be highly probable.
Analysis of the graphs of solutions of the most skilled chess players who received sets A and B and anticipated the real changes in the situation has shown the following.
Subject 20, who anticipated the real changes in the situation but did not choose these forecasts as the best move in position B1, used in position B2 two move-candidates that were anticipated earlier, e5 and N:f6; offered another move-candidate, Nb5; developed a branch connected with e5 on a depth of five semi-moves. Variants became deeper, branchi-ness increased, the subjective probability of previously rejected forecasts increased, and their evaluation changed.
Subject 31, who anticipated the real changes in the situation and chose these forecasts as the best move in position B1(in other words, his anticipation of real changes coincided with the selected move), in position B2 considered three move-candidates: two of them, Dе3 and e5, were previously anticipated; one, Nb5, was new. The move fg, anticipated previously, was not considered as the move-candidate. Emphasis was placed on Dе3 (the selected move). Variants were transformed and deepened in relation to the variants anticipated previously in position B1. Forecasts of real changes in the situation made by the subject in position B1 provided the direction for the search in connected position B2. In a follow-up situation (that is, in position B2) the search was carried out mainly in the direction of subjectively probable forecasts that were previously made (that is, the forecasts most often verbalized). In position B1 the most probable subjective forecast was a chain of moves f4-h6, Dе3-Ng6. And, in position B2, resulting from B1 by means of one
move made by each side (f4 for White and h6 for Black), the continuations after moves De3-Ng6 were considered most often (in one third of the variants). A variant proposed in position B1, at the beginning of these moves, was not repeated in full in the follow-up situation; after the repeated part of this variant, other moves were already being considered. A stronger follow-up, changing the assessment of the entire chain, was chosen (variant 2 in position B2 was assessed in favor of Black). By then the transformation had already taken place because additional short chains had been projected in the previous situation. Variants from 11 to 13 were linked to the realization of the new plan, beginning with move Nb5. On the 14th variant there was a return to the initial plan for White: De3 with the subsequent e5. After that, on the 15th variant, the answer followed: De3. In other words, the previously formed VOS defined the search direction and its selection, but the transfer was not connected only with the repetition of previously made forecasts. There was a transformation and development of the senses because of the inclusion of new interactions of elements.
For subject 79 there was also a transformation of the previously anticipated chains of moves in a connected position. In position A2, resulting from A1 after moves f4, Nc6, the subject repeated the chain of moves Bd2, Re8, which arose after moves f4, Nc6, but modified it, considering other continuations, Bc3 and Dd3, not 0-0 (as in A1). As a result the probability increased of previously discarded forecasts connected with moves f4, Nc6, Bd2; the evaluation of the variant beginning with move Bd2 changed; and its depth was increased.
Thus, in the next situation (a connected position), which was a continuation of the preceding one after one move for White and one move for Black, the chess players used previously produced forecasts of real changes in the situation. In previous situations these forecasts were associated with a selected move (and sense of action) or were rejected. In a connected position the first type of forecast was transformed and became deeper. The second type of forecast was used with a rise in subjective probability. Forecasts were transformed and overestimated and became deeper; their branchiness could rise.
If previously rejected forecasts were short (only one move for White and one move for Black), in a connected position they started to develop: variants were proposed on the basis of several (4-5) move-candidates;
estimations were made for move-candidates and variants; plans were proposed (for one or both sides).
So, VOS transfer may be connected with the use and transformation of both previously accepted forecasts, associated with a sense of action, and previously rejected forecasts. Their depth (for both types of anticipations) and subjective probability (for previously rejected anticipations) rise.
Thus, VOS transfer is associated not simply with the repetition and the copying of some forecasts but with wide use of the results of verbal searches in subsequent situations. As a result, on the one hand, previous and subsequent situations unite for the subject into certain sense groups; on the other hand, new interactions of pieces are included.
Conclusion
VOS transfer is seen more often in skilled than in unskilled chess players. VOS transfer is characterized by a reduction in the number of called elements and an increase in the percentage of sense substances (the most often researched elements) in the total number of called elements and by a decrease of solution time in a connected position. It also reflects an increase in research selectivity there.
VOS transfer provides direction for the search and is associated with the use and transformation of results from previous verbal searches. VOS transfer is active.
So, on the basis of the conducted research of VOS transfer from one situation to another, which provides evidence of the greater portability of VOS by experts than by nonexperts and the active nature of the VOS transfer, which is manifested in the transformation, deepening, and increasing amount of branchiness of previously considered verbal forecasts, we can reach conclusions about the complex nature of expertise, which is linked with both knowledge and search. Our research showed that the problem of VOS transfer is closely associated with the problem of the correlation between knowledge and thinking. Our conception of the phenomenon of VOS transfer conforms with the ideas of a number of psychologists (Bartlett, 1998; Dunker, 1965; Najsser, 1981) about the role of past experience in cognition. According to their ideas, the past (past experience) is transformed; it isn't simply repeated.
References
Bartlett, F. (1998). Chelovek zapominaet [Man memorizes]. In J.B. Gippenrejter & V.J. Romanov (Eds.), Psihologija pamjati [Psychology of memory] (pp. 292-303). Moscow: CHeRo.
De Groot, A. (1965). Thought and choice in chess. Hague: Mouton (Original work published 1946).
Dunker, K. (1965). Psihologija productivnogo (tvorcheskogo) myshlenija [Psychology of productive (creative) thinking]. In A.M. Matyushkin (Ed.), Psihologija myshlenija [Psychology of thinking] (pp. 86-234). Moscow: Progress.
Ericsson, K.A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211-245.
Gobet, F., & Charness, N. (2006). Expertise in chess. In K.A. Ericsson, N. Char-ness, P. Feltovich, & R. Hoffman (Eds.), The Cambridge Handbook of Expertise and Expert Performance (pp. 523-538). Cambridge: Cambridge University Press.
Gobet, F., & Simon, H. A. (1996). Templates in chess memory: A mechanism for recalling several boards. Cognitive Psychology, 31, 1-40.
Gobet, F., & Simon, H.A. (2000). Five seconds or sixty? Presentation time in expert memory. Cognitive Science, 24, 651-682.
Holding, D. (1979). Echoic storage. In N.S. Sutherland (Ed.). Tutorial essays in psychology, vol. 2. Hillsdale, NJ: Erlbaum.
Najsser, U. (1981). Poznanie i real'nost' [Cognition and reality]. Moscow: Progress.
Newell, A., & Simon, H.A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
Reingold, E.M., Charness, N., Pomplun, M. & Stampe, D.M. (2001). Visual span in expert chess players: Evidence from eye movements. Psychological Science, 12(1), 40-55.
Reingold, E.M., Charness, N., Schultetus, R.S., & Stampe, D.M. (2001). Perceptual automaticity in expert chess players: Parallel encoding of chess relations. Psychonomic Bulletin and Review, 8, 504-510.
Solso, R.L. (1996). Kognitivnaja psihologija [Cognitive Psychology]. Moscow: Trivola.
Tikhomirov, O.K. (1969). Struktura myslitel'noj dejatel'nosti cheloveka [Human thinking activity structure]. Moscow: Moscow State University.
Tikhomirov, O.K. (1984). Psihologija myshlenija: Uchebnoeposobie [Psychology of thinking: Educational text-book]. Moscow: Moscow State University.
Vasyukova, E.E. (2001). Verbalizovannye operacional'nye smysly i ih razvitie v processe prinjatija reshenija (na materiale vybora luchshego hoda v shahmatnoj pozicii) [Verbilized operational senses and its development in the process of decision making
(on search the best move in chess position)]. Psihologicheskij zhurnal [Psychological journal], 22(4), 30-41.
Vasyukova, E.E. (2006).Verbalizovannye operacional'nye smysly v strukture pri-njatija reshenija v slozhnyh situacijah [Verbilized operational senses in the structure of decision making in difficult situations]. In B.S. Bratus & E.E. Sokolova (Eds.), Uchenye zapiski kafedry obshhej psihologii MGU [Learned papers of MSU general psychology chair] (Vol. 2, pp. 368-391). Moscow: Smysl.
Velichkovsky, B.M. (2006). Kognitivnaja nauka: Osnovypsihologiipoznanija [Cognitive science: Foundations of epistemic psychology] (vol. 2, pp. 244-249). Moscow: Smysl; Izdatel'skij centr "Akademija."
