CC BY
31
УДК 159.9.072, 159.955.6, 159.955.2, 159.955.5, 159.99 Вестник СПбГУ. Сер. 16. 2016. Вып. 4
O. V. Shcherbakova, D. N. Makarova
ARE THEY DIFFERENT OR THE SAME? EXPERTS' PSYCHOMETRIC IQ DOES NOT CORRELATE WITH CONCEPTUAL ABILITIES*
In this paper, we describe the results of a study (N = 16, aged 19 to 39), which aimed to test the hypothesis about the correlation between the level of psychometric IQ (measured with J. Raven's SPMT) and the level of conceptual abilities. The main finding of this study was that psychometric IQ shows a lack of correlation with conceptual abilities in experts who face cognitive challenges on a daily basis and manifest real-life intellectual productivity. This leads to the necessity of revising the existing approaches to understanding intellectual productivity and its measurement. Refs 21. Fig. 1. Tables 2.
Keywords: intellectual productivity, psychometric IQ, conceptual abilities, J. Raven's "Standard Progressive Matrices" test, "Integral Conceptual Structures" test.
О. В. Щербакова, Д. Н. Макарова
ТАКИЕ ЖЕ ИЛИ ДРУГИЕ? ПСИХОМЕТРИЧЕСКИЙ ИНТЕЛЛЕКТ ЭКСПЕРТОВ НЕ СВЯЗАН С ИХ КОНЦЕПТУАЛЬНЫМИ СПОСОБНОСТЯМИ
Представлены результаты исследования (n = 16, от 19 до 39 лет), направленного на проверку гипотезы о связи уровня психометрического интеллекта, измеренного с помощью «Стандартных прогрессивных матриц» Дж. Равена, с уровнем концептуальных способностей. Результаты регрессионного анализа показали, что у людей, регулярно сталкивающихся с необходимостью решать когнитивные задачи и характеризующихся реальной умственной продуктивностью в предметно-специфической деятельности, психометрический интеллект оказывается практически не связанным с концептуальными способностями. Делается вывод о необходимости формулирования нового подхода к пониманию природы умственной продуктивности и ее измерению. Библиогр 21 назв. Ил. 1. Табл. 2.
Ключевые слова: интеллектуальная продуктивность, психометрический интеллект, концептуальные способности, «Стандартные прогрессивные матрицы» Дж. Равена, «Интегральные концептуальные структуры».
In almost 150 years of intelligence research in psychology, there have been many promising (though sometimes a bit too inductive) findings with concern to various aspects of human intelligence. Despite this progress, the question of cognitive mechanisms underlying intellectual productivity remains one of the biggest unanswered theoretical questions in intelligence research.
Shcherbakova O. V. — PhD, Associate Professor, Saint Petersburg State University, 7-9, Universitetskaya nab., St. Petersburg, 199034, Russian Federation; o.shcherbakova@spbu.ru
Makarova D. N. — M.A., Postgraduate, Saint Petersburg State University, 7-9, Universitetskaya nab., St. Petersburg, 199034, Russian Federation; d.makarova23@gmail.com
Щербакова Ольга Владимировна — кандидат психологических наук, доцент, Санкт-Петербургский государственный университет, Российская Федерация, 199034, Санкт-Петербург, Университетская наб., 7-9; o.shcherbakova@spbu.ru
Макарова Дарья Николаевна — магистр психологии, аспирант, Санкт-Петербургский государственный университет, Российская Федерация, 199034, Санкт-Петербург, Университетская наб., 7-9; d.makarova23@gmail.com
* The research was supported by the grant of the President of Russia № МК-7507.2016.6 "Psychological and psychophysiological bases of experts' intellectual productivity".
© Санкт-Петербургский государственный университет, 2016
In the past, psychometric intelligence used to be one of the first theoretical constructs viewed as a possible predictor of one's real-life intellectual achievements. Although the concept of Intelligence Quotient (IQ) lacked theoretical clarity, it was undoubtedly supposed to determine real-life intellectual productivity. Only in the 1930s, L. Terman who published results of his longitudinal Genetic Study of Genius showed that people with above-average and even superior IQ may not necessarily reach significant eminence in any area [1]. This was the start of further questioning and investigation in the predictive potential of IQ.
Since then, criticisms of IQ tests became some sort of a commonplace [2-7]. According to the latest intelligence research, it is suggested that results stemming from psychometric intelligence tests, cannot be considered a reliable source of information with regards to the real intellectual potential of a person [8]. However, it would be premature to discredit the concept of psychometric IQ; these IQ scores are still being taken as an indicator of intellectual productivity in majority of the studies in this area [9, 10]. This notion is more likely caused by long-standing tradition and the lack of alternative approaches rather than conceptual irreproachability of the 'IQ' term.
Despite the lack of alternative theoretical frameworks and perspectives for explaining the matter of intellectual productivity including new tools for measurement, one promising methodology emerges from the theoretical approach of M. A. Kholodnaya, which views intelligence as a conglomerate of an individual's mental experience [8, 11, 12]. One of the cornerstones of this approach is the idea of conceptual structures, which are mental formations that provide an ability to reveal hidden regularities in any kind of situations even if necessary information is lacking. These conceptual structures determine the productivity of conceptual thinking and therefore integrate all the components of one's intelligence collectively [11]. Conceptual structures are also a base for conceptual abilities development which, in their turn, are necessary for generating new mental content underlying any form of productive thinking.
Methods
The main goal of this study was to test the following hypothesis: conceptual abilities as a potential psychological basis for intellectual productivity correlate with psychometric intelligence. In order to achieve this goal, the study required a specific sample consisting of people whose everyday life required a manifestation of intellectual productivity (which was easy to measure with the use of objective tools). In order to fulfill this criteria, staff from the technical support department of one of the Saint Petersburg internet companies was asked take part in our study. The participants (N = 16, 11 male, 19-39 years old) had a variety of different educational backgrounds and varying levels of experience of working in this particular area of expertise (ranging from 2.5 months to 2.5 years).
The staff from this particular line of work were suitable as the duties in a technical support department required large amounts of problem-solving in various situations. This problem-solving was possible through applying specific knowledge and expertise in this particular area which was considered superior performance in real world domains, or expertise [13]. An expert in this area usually needs to quickly figure out a client's technical problem in tight time constraints during a phone call, then build up a full and precise mental representation of this problem and decide if it is within his/her competence,
and finally to provide appropriate assistance and a solution. Aforementioned intellectual actions have to be performed with accordance to the company rules, regulating staff — client communication on the phone (strict time limitations, guidelines aimed rather at employees' efforts to follow the rules than at helping clients in solving their technical problems). All these skills are based on a specific organization of a mental experience of an expert [8], which makes the expert performance qualitatively different from any other type of human performance [13].
Participants' psychometric intelligence was measured with J. Raven's "Standard progressive matrices" test (SPMT). The experiment also used M. A. Kholodnaya's [11, 12] tests "Integral Conceptual Structures", "Concept Synthesis" and "Generalization of Three Words" to measure the level of conceptual abilities.
J. Raven's "Standard progressive matrices" test remains one of the most reliable tools for measuring "g" — "general cognitive ability", or intelligence [14] (although it is still often mistaken even by psychologists for a test of non-verbal abilities [15] due to the nonverbal form of its tasks). Productive features of intelligence measured by this test are a better predictor of one's intellectual productivity in real life than reproductive features of intelligence measured by other verbal tests [16]. Obtaining high scores in this test may be interpreted as an indication of an ability to implicitly learn by generalizing (conceptualizing) one's experience in the absence of any given instructions [17] and, also, as one of the most relevant measures of fluid intelligence in general [18]. Therefore we may hypothesize that mental processes captured with SPMT are to some extent common with the nature of conceptual abilities.
We used the following instruction (modified version):
"Hello! The goal of this test is to reveal some aspects of your cognitive processes. In each task you will be presented with an incomplete image. Below that image, you will see several options — other images that can be used to complete the main image. Your task is to choose only one of them, which seems to be the most appropriate. You have 20 minutes to complete the tasks. When 20 minutes are over, you will have to stop working on the test."
M. A. Kholodnaya's test "Integral Conceptual Structures" reveals and makes explicit different cognitive components of concepts and also estimates the degree of their maturity. The full version of this test consists of 8 tasks. We chose 3 of them according to the main goal of our study. Their brief description is given below.
The task "Verbal to Imagery Conversion" is aimed at explication of concept's visual component. The participant was presented with a concept (in verbal form) and the task was to convert its verbal form to 3 imagery representations, corresponding to a) the first impression, b) additional associations, and c) the most essential characteristics [11]. There were no time limit in this task. The answers were rated 0, 1, 2 and 3 scores — depending on how concrete/generalized the images were. The sum of scores for all images was regarded as an indicator of productivity of verbal to imagery conversion.
The task "Imagery Interlocutor" requires to set apart relevant semantic features from irrelevant ones. We used this task to assess volitional regulation of the process of semantic features actualization. The participant was presented with two concepts ("soil" and "disease"). The task was to explain the meaning of these concepts to an imagery interlocutor naming any words corresponding to their core characteristics. There was no time limit for this task. Each answer was rated 0, 1 and 2 scores depending on how concrete/generalized
it was. The sum of scores was viewed as an indicator of participant's capacity to select essential semantic features of the given concept from out of its all semantic features.
The task "Problem Formulation" is aimed at explication of a concept's mnemonic component. The participant was presented with 2 concepts: one emotionally neutral ("soil") and one emotionally negative ("disease"). The task was to imagine him/her being a researcher studying these objects and to formulate the possible research problems concerning them. There was no time limit for this task. "Research problems" were rated 0, 1 or 2 scores. A higher score was given if generalization was based on essential semantic features; a lower scores corresponded to the answers based on just themed associations. The sum of scores was regarded as an indicator of complexity of concept's mnemonic potential.
M. A. Kholodnaya's test "Concept Synthesis" measures the capacity to devise a set of semantic contexts based on three words from remote semantic fields [11]. The participant was presented with 3 concrete words belonging to 3 different semantic fields. The task was to find out as much sense bearing semantic links between these words as possible and to write them down in 1 or 2 sentences using all three words. There were 4 word triads in total. The time limit for each three words was 3 minutes. An example of a triad: cockleshell — paper clip — thermometer. The answers were rated 0, 1, 2 and 3 scores depending on how concrete/generalized the semantic link between all three words was. The sum of the scores for all 4 triads represented the capacity to create new semantic contexts basing on concrete words from remote semantic fields.
M. A. Kholodnaya's test "Generalization of Three Words" measures the capacity for categorical generalization of concepts on the ground of their essential characteristics [11]. This task challenges the participants' capacity for a "vertical" mental shift from the particulars to generals and also requires the creation of a new, highly-generalized category. The participant was presented with three words from different semantic fields. The task was to elicit the common feature of these three words and phrase it in 1 or 2 words. There were 10 word triads altogether. Each triad was presented for 30 second. Example of a triad: "musical scale — bead necklace — staircase".
The answers were rated 0, 1 or 2 depending on how concrete/generalized the suggested category was. The sum of the scores for all 10 triads represented the categorical generalization ability.
The tests were presented to the participants in the following order:
1) J. Raven's SPMT;
2) the tasks from "Integral Conceptual Structures" test: "Verbal to Imagery Conversion", "Imagery Interlocutor" and "Problem Formulation" for the concept "soil"; the solving process of the tasks "Imagery Interlocutor" and "Problem Formulation" was audio-recorded and then transcribed verbatim;
3) the same tasks as stage 2, but for the concept "disease"; also, the solving process of the tasks "Imagery Interlocutor" and "Problem Formulation" was audio-recorded and then transcribed verbatim;
4) "Generalization of Three Words" test;
5) "Concept Synthesis" test.
For statistical analysis we used the following variables: SPMT = the sum of scores for Raven's SPMT; VIC = the sum of scores for "Verbal to Imagery Conversion" task; II = the index (the sum of scores/the quantity of answers) for "Imagery Interlocutor" task; PF = the
index (the sum of scores/the quantity of answers) for "Problem Formulation" task; CS = the index (the sum of scores/the quantity of answers) for "Concept Synthesis" test; GW = the sum of scores for "Generalization of Three Words" test. All these variables were ranked variables. We used multiple regression analysis (MRA) to test our hypothesis according to which aforementioned variables are characterized by joint variation. MRA was used because it can be considered an analog of MANOVA when dealing with ranked variables [19]. Statistical analysis was conducted with the use of SPSS Version 20 (IBM Corp., New York, USA).
Results and discussion
The results of descriptive statistics are presented below (Table 1). The value of SPMT variable ranges from 39 to 58 points out of 60, with 50 % of the participants achieving the score of 52 and higher (mean = 51,56; median = 52; mode = 57; std. deviation = 5,82). VIC and GW variables are sums of scores; they vary from 1 to 8 (mean = 3,88; median = 3; mode = 3; std. deviation = 2,06) and from 0 to 15 (mean = 8,81; median = 9; mode!=6; mode2 = 12; std. deviation=4,75) points, respectively. The values of II, PF and CS variables are indices (sums of scores divided by the numbers of given answers); they range from 0,05 to 1,29 for II (mean = 0,62; median = 0,68; mode!=0,71; mode2=1; std. deviation = 0,38), from 0,25 to 1,8 for PF (mean = 0,79; median = 0,87; mode = 1,2; std. deviation = 0,43) and from 0,33 to 1,4 for CS (mean = 0,87; median = 0,96; mode = 1; std. deviation = 0,27). In the cases of SPMT, CS and GW variables distributions of data are skewed to the left (i.e. median is less than mode), in case of VIC, II and PF variables — skewed to the right (i.e. median is greater than mode). So we can see that on average experts' scores for SPMT, CS and GW tasks are higher, scores for VIC task are lower and scores for II and PF tasks are slightly lower compared to normally distributed data.
In the regression model, scores for SPMT were a dependent variable and scores for GW, PF, II, CS, VIC were independent variables. We used backward method of multiple
Table 1. Descriptive statistics
SPMT VIC II PF CS GW
Mean 51,56 3,88 0,62 0,79 0,87 8,81
Median 52 3 0,68 0,87 0,96 9
Mode! 57 3 0,71 1,2 1 6
Mode2 - - 1 - - 12
Std. deviation 5,82 2,06 0,38 0,43 0,27 4,75
Variance 33,86 4,25 0,14 0,18 0,07 22,56
Skewness -0,69 0,66 0,04 0,7 -0,37 -0,34
Std. error of skewness 0,56 0,56 0,56 0,56 0,56 0,56
Minimum 39 1 0,05 0,25 0,33 0
Maximum 58 8 1,29 1,8 1,4 15
Percentiles 25 46,5 3 0,25 0,42 0,75 6
50 52 3 0,68 0,87 0,96 9
75 57 5 0,97 1,13 1 12,75
regression analysis. At first all independent variables were input into the model, then the parameter least contributing to the model (parameter with the least absolute meaning of t statistic) was excluded from it at each stage (Table 2). As we can also see from Table 1, only one predictor's beta coefficient was significant at all stages of analysis, and this was the coefficient of CS test. Therefore GW, VIC, II, PF were excluded from the regression model one after another and CS remained: R2=0,444; p = 0,005.
Table 2. Results of regression analysis
Model Unstandardized Coefficients Standardized Coefficients t R2
Predictors B Std. Error Beta
1 Constant 35,96 4,98 7,22 ,596
VIC 0,50 0,58 ,176 ,89
II -4,56 3,65 - ,294 -1,25
PF 3,85 2,92 ,282 1,32
CS 15,00 4,51 ,684* 3,32
GW 0,04 0,28 ,032 ,14
2 Constant 36,13 4,61 7,83 ,595
VIC 0,51 0,55 ,181 ,94
II -4,35 3,17 - ,280 -1,37
PF 3,90 2,77 ,285 1,41
CS 14,95 4,29 ,681* 3,48
3 Constant 37,73 4,26 8,85 ,563
II -4,41 3,15 - ,285 -1,40
PF 4,13 2,74 ,302 1,50
CS 15,22 4,26 ,694* 3,57
4 Constant 36,77 4,36 8,43 ,491
PF 2,99 2,72 ,219 1,10
CS 14,22 4,35 ,648* 3,27
5 Constant 38,80 3,98 9,75 ,444
CS 14,62 4,37 ,666* 3,34
* p < ,01.
The relation between SPMT and CS is depicted on the plot below (Figure 1). On the plot we can see that there is a strong positive correlation between the two variables, i.e. as CS test value increases, so does the value of SPMT variable.
This regression model (dependent variable: SPMT, independent variable: CS) explains 44 % (R2 = 0,444 • 100) of the dependent variable variation.
The results show that only the test which revealed the essential basis of intellectual productivity out of the five we used for measuring the level of conceptual abilities, correlated with IQ level measured using J. Raven's SPMT. We have not found any evidence that suggests the correlation between SPMT scores and the scores obtained in the other four conceptual abilities tests. In other words, psychometric IQ seems to be in a high degree independent of conceptual abilities in people who face the necessity to solve cognitive problems on a daily basis and manifest real-life intellectual productivity within their expertise domain.
concept_syntesis
Figure 1. Distribution of the scores for SPMT within different scores for the "Concept
Synthesis" task
The only exclusion is the 'Conceptual Synthesis' test, which requires the ability to reveal hidden, unobvious categorical links between different domains of one's cognitive experience and produce new mental contents even if there is a lack of information provided. Cognitive operations of this kind are to some extent similar to the cognitive operations underlying solving of SPMT tasks. This finding might be one of possible explanations for the existing correlation between SPMT and CS variables. Whilst other facets of conceptual abilities may also be the basis of intellectual productivity, that may exist beyond cognitive functions that could be sensitive to measurement by SPMT. Among these facets of conceptual abilities are well-developed categorical structures (measured with 'Generalization of Three Words' test), the integrity of concepts' visual components (measured by 'Verbal to Imagery Conversion' test), the ability to modulate the actualization of concepts' semantic features (measured by 'Imagery Interlocutor' test) and also the capacity and complexity of concept's mnemonic component (measured by 'Problem Formulation' test). It was shown in a number of studies [11, 20, 21] that all the above-mentioned indicators underlie high intellectual productivity. In light of the results we obtained in the current study, these properties of general cognitive ability appear to be relatively autonomous from psychometric IQ measured with J. Raven's SPMT. We consider this finding to be the evidence of a substantial difference between mental structures that are available for testing with the use of SPMT, and mental structures that can be measured with cognitive tasks designed by M. A. Kholodnaya.
If this abovementioned line of thought based on our results is true, then another serious theoretical question emerges that requires investigation: which mental reality pre-
determines the scores of psychometric IQ? This leads us to the necessary reconsideration of the existing approaches for the measurement of intellectual productivity. What is even more important is to establish a new framework that provides accurate explanation of being intellectually efficient. The achievement of these goals requires more studies to illuminate the cognitive mechanisms of intellectual productivity. One of the first steps towards moving in the right direction would be to test if there is a correlation between psychometric IQ scores, conceptual abilities' level and objective indicators of intellectual efficiency in various cognitive tasks. This forms one of the directions in our future research.
Limitations and conclusion
It is acknowledged that the sample of the present study is rather small. The sample size was limited because the participants were recruited outside of a University environment of psychology students taking part in the study for the purpose of course credits (which is, however, still common for majority of research in psychology and reflects low ecological validity.) The sample were real experts who were busy and the small sample size might be partially compensated by their qualitative characteristics. Also, it might be that the results obtained within other domains of expertise (e.g. medicine, sports, etc.) differ from the ones shown in the present study.
The main finding of this study was that psychometric IQ shows lack of correlation with conceptual abilities in experts who face cognitive challenges on a daily basis and manifest real-life intellectual productivity. This reinforces the necessity of revising the existing approaches to understanding intellectual productivity and its measurement.
References
1. Burks B. S., Jensen D. W., Terman L. M. The promise of youth. L. M. Terman. Genetic studies of genius (Vol. 3). Stanford, CA, Stanford University Press, 1930.
2. Miles T. R. On Defining Intelligence. British Journal of Educational Psychology, 1957, vol. 27, pp. 153—
167.
3. McNemar Q. Lost: Our Intelligence? Why? American Psychologist, 1964, vol. 19, no. 12, pp. 871-882.
4. Frederiksen N. Toward a broader conception of human intelligence. American Psychologist, 1986, vol. 41, no. 4, pp. 445 — 452
5. Howe M. J. Intelligence as an explanation. British Journal of Psychology, 1988, vol. 79, no. 3, pp. 349 —
360.
6. Miles T. R. Comment on Howe's Paper. British Journal of Psychology, 1988, vol. 79, no. 4, pp. 535-538.
7. Anastasi A. U., Urbina A. S. Psychological testing. Ann Arbor, MI, Prentice Hall, 1997.
8. Holodnaja M. A. Psikhologiia intellekta: paradoksy issledovaniia. 2-e izd., pererab. i dop. [Psychology of Intelligence: Research Paradoxes. 2nd ed., revised and enlarged]. St. Petersburg, Piter Publ., 2002. (In Russian)
9. Grabner R. H. The role of intelligence for performance in the prototypical expertise domain of chess. Intelligence, 2014, vol. 45, pp. 26-33.
10. Ruthsatz J. Putting practice into perspective: Child prodigies as evidence of innate talent. Intelligence, 2014, vol. 45, pp. 60-65.
11. Kholodnaia M. A. Psikhologiia poniatiinogo myshleniia: ot kontseptual'nykh struktur k poniatiinym sposobnostiam [Psychology of Conceptual Thinking: From Conceptual Structures to Conceptual Abilities]. Moscow, Institute of Psychology RAS Print, 2012. (In Russian)
12. Kholodnaya M. A., Volkova E. V. Conceptual Structures, Conceptual Abilities and Productivity of Cognitive Functioning: The Ontological Approach. Procedia-Social and Behavioral Sciences, 2016, vol. 217, pp. 914-922.
13. Ericsson K. A. Why expert performance is special and cannot be extrapolated from studies of performance in the general population: A response to criticisms. Intelligence, 2014, vol. 45, pp. 81-103.
14. Carpenter P. A., Just M. A., Shell P. What one intelligence test measures: a theoretical account of the processing in the Raven Progressive Matrices Test. Psychological review, 1990, vol. 97, no. 3, pp. 404-431.
15. McCallum R. S., Bracken B. A., Wasserman J. D. Essentials of nonverbal assessment. New York, Wiley, 2001.
16. Raven J. Raven's progressive matrices: Their origin and contribution to society. Psikhodiagnostika v sotsialisticheskikh stranakh. Psikhodiagnosticheskii biulleten'. I-IX [Psychodiagnostics in Socialistic Countries. Psychodiagnostics Bulletin. I-IX]. Bratislava, 1985, pp. 27-31.
17. Raven J. The Raven Progressive Matrices: A review of national norming studies and ethnic and socioeconomic variation within the United States. Journal of Educational Measurement, 1989, vol. 26, no. 1, pp. 1-16.
18. Cattell R. B. Abilities: Their growth, structure, and action. Boston, Houghton Mifflin, 1971.
19. Nasledov A. D. Matematicheskie metody psikhologicheskogo issledovaniia. Analiz i interpretatsiia dannykh. Uchebnoe posobie [Mathematical Techniques in Psychology Research. Data Analysis and Interpretation. A Study Guide]. St. Petersburg, Retch Publ., 2004. (In Russian)
20. Kholodnaia M. A., Shcherbakova O. V., Gorbunov I. A., Golovanova I. V., Papovian M. I. Informat-sionno-energeticheskie kharakteristiki razlichnykh tipov kognitivnoi deiatel'nosti [Information-energetic characteristics of various patterns of cognitive performance]. Psikhologicheskiizhurnal [Psychology Journal], 2013, vol. 34, no. 6, pp. 96-107. (In Russian)
21. Shcherbakova O. V., Makarova D. N. Conceptual abilities as a basis of expertise. 15th Annual Conference of the International Society for Intelligence Research (ISIR). Graz, Austria, December 12-14, 2014, pp. 62.
For citation: Shcherbakova O. V., Makarova D. N. Are they different or the same? Experts' psychometric IQ does not correlate with conceptual abilities. Vestnik SPbSU. Series 16. Psychology. Education, 2016, issue 4, pp. 88-96. DOI: 10.21638/11701/spbu16.2016.407
Received: 08.08.2016 Accepted: 11.10.2016
