CC BY-NC-ND
214
Determinants of Research Productivity: An Individual-level Lens
Konstantin Fursov
Associate Professor, and Head, Division for Analysis of R&D Performance Institute for Statistical Studies and Economics of Knowledge (ISSEK). E-mail: ksfursov@hse.ru
Yana Roschina
Associate Professor, Department of Sociology; and Senior Research Fellow, Laboratory for Studies in Economic Sociology. E-mail: yroshchina@hse.ru
Oksana Balmush
Master's Student, Program 'Applied Methods of Social Analysis of Markets' by Department of Sociology.
E-mail: obalmush@mail.ru
National Research University Higher School of Economics (NRU HSE) Address: 20 Myasnitskaya str., 101000 Moscow, Russian Federation
Abstract
The continuous growth of investment in R&D in Russia and the world increases the demand for optimal allocation of public funds to support the most productive scientific performers. These are, however, hard to conceptualize and measure. First, we need to consider the nature of research activity itself and, second, we need to evaluate a number of factors that influence such activities at the national, institutional and individual levels. One of the key issues is motivation of academic personnel, who are considered the main producers of new knowledge. Therefore, it is necessary to analyse the employment characteristics of researchers, and develop adequate mechanisms to facilitate their scientific productivity.
This paper aims to examine determinants of publication activity among doctorate holders employed in the academic sector in Russia. Data for the analysis was derived from a survey on the labour market for highly qualified R&D personnel conducted in 2010 by the HSE, within the framework of the OECD / UNESCO Institute for Statistics / Eurostat international project on Careers of Doctorate Holders (CDH). With the use of regression analysis,
we assess the effects of scientific capital, international cooperation, employment, and socio-demographic characteristics of researchers on their productivity, which is measured through their total publication output as well as through the number of papers in peer-reviewed academic journals.
The differences between factors were assessed for two generations of researchers - those under 40 years old, and above. It was shown that the quality of scientific capital, measured through diversity of research experience, has a stronger impact on research productivity, rather than the age or other socio-demographic characteristics of doctorate holders. It was also demonstrated that direct economic stimuli and actual research productivity of researchers are weakly correlated. Consequently, we identified that a potentially winning strategy for universities and research institutions that want to improve their performance indicators would be to provide younger scholars with wider opportunities for professional growth, including intense global cooperation in the professional community.
Keywords: performance-based payment; bibliometric indicators; Russian scholars; the productivity of science; human capital; scientific publications
DOI: 10.17323/1995-459X.2016.2.44.56
Citation: Fursov K., Roschina Y., Balmush O. (2016) Determinants of Research Productivity: An Individual-level Lens. Foresight and STI Governance, vol. 10, no 2, pp. 44-56. DOI: 10.17323/1995-459X.2016.2.44.56
44 I «DOPCAkiT I T. 10 № 2 I 2016
s one of the key drivers of growth and competitiveness in the economy, scientific knowledge is
directly reflected in national science and technology development strategies and in the actual
government spending figures of OECD member states [OECD, 2014]. Beginning in the 1960s, the governments of leading economies recognized the contribution of scientific research to economic development and began to provide higher levels of funding, while at the same time improving the statistical accounting system to monitor investment in knowledge production [Godin, 2013]. Following the development of methods to record resources accumulated in the research and development (R&D) sector, the question arose of effective use of allocated funds and approaches to measure the results of scientific and technological activity. This problem is particularly labour intensive on account of the creative and intellectual nature of scientific activities, as well as the need to consider multiple social, economic, political and other determinants affecting the development of a research culture at the national, institutional (organizational), and individual levels (cf. for example: [Godin, Gingras, 2000]). One of the most important conditions influencing the effectiveness of national science and technology systems is the support for and motivation of researchers, who are the key and direct producers of new knowledge. Studying the factors influencing the individual productivity of researchers is of crucial importance if we want to analyse the working conditions of academics [Gokhberg et al., 2011] and develop effective mechanisms to incentivize research activity [Gershman, Kuznetsova, 2013]. Criteria to assess these mechanisms and positive influences on related processes are of critical interest. This paper will also focus on the differences in publication activity between 'younger' and 'older' generations of academics, and their main determinants.
Literature review
Indicators of scientific productivity
In order to assess scientific productivity, we first need to define its place in the socio-economic system. Attempts to offer a more or less strict definition of the term 'science,' explain its distinguishing features, and arrive at a holistic view of scientific knowledge have been undertaken at various times by members of extremely diverse disciplines. In the 20th century, knowledge production expanded beyond the boundaries of universities and started to disperse among national laboratories, research centres, and business enterprises. In social disciplines, science has come to be seen as a public good [Godin, 2013] and institution [Merton, 1973] that produced 'certified' knowledge on the surrounding world, promoted progress, aided in the flourishing of society, and solved essential social problems. As for the state, it funded the work of academics, guaranteed their autonomy, and supported comfortable working conditions. The status quo was disrupted when formal criteria were introduced to assess researchers' output, and attempts to correlate output with actual spending. However, the tasks of conceptualizing and measuring R&D outputs have become one of the key challenges in statistics over roughly the last 80 years. After the Second World War, science and technology underwent rapid growth and intensive integration into various spheres of social life. Consequently, governments in many countries desired to regulate the support for and development of R&D. One of the instruments used to control this was statistics, which set formal boundaries for the sector and defined methods of measuring financial and other resources absorbed by R&D [Godin, 2013; Roud, Fursov, 2011]. By the mid-1960s, the first conventional definition had been developed, covering the creation of new knowledge and the development of ways to make good use of this [Godin, 2009]. However, the overall direction in statistical development was aimed primarily at elaborating approaches to measure the scientific inputs rather than outputs. The assessment of scientific results is part of a different tradition, linked primarily to quantitative methods that are used to analyse publications, rarely patent and other forms of research activity. The assessment reflects not so much their productivity, but rather their presence and visibility in the international scientific community [Godin, 2006; Garfield, 2009; Hicks et al., 2015; Kirchik, 2011].
Still, the number of publications in peer-reviewed academic journals, citations, and derivative indicators are recognized by most countries as key criteria of scientific productivity, including when deciding on funding for R&D [Hicks, 2012]. Some national jurisdictions [Ancaiani et al., 2015] and organizations use individual productivity indicators. Despite the imperfection of this methodology and its multiple criticisms [Bordons et al., 2002; Weingart, 2005; Stephan, 2012; Hicks et al., 2015], publication activity together with several other approaches (primarily, professional expert assessments) remain the primary and most representative description of national and individual scientific contributions [Ball, 2005; Moed, 2009]. Russia is no exception: articles, monographs, conference papers, and other works are also viewed as evidence of significant scientific achievements. The 1996 Federal Law No. 127-FZ ('On Science and State Science and Technology Policy') defines performance results in this field as any product containing new knowledge or solutions which are recorded on a storage medium.1 According to existing statistical
1 Federal Law No. 127-FZ 'On Science and State Science and Technology Policy' (version dated 22/12/2014 with amendments dated 20/04/2015), Article 2 (Accessible at http://www.consultant.ru/document/cons_doc_LAW_172547/, accessed 20.05.2015).
data, by the end of the period 2000-2010 Russian researchers published roughly 350,000 articles every year, but no more than 10% of these were included in the world's largest (Web of Science and Scopus) academic citation indices [HSE, 2011]. Despite growth in absolute figures, Russia's share of the global publication activity map on Web of Science in the period 2000 to 2014 shrank by almost twofold - from 3.2% to 2.1% [Fursov, 2015]. Over these years, Russia lost its position amongst the top ten international leaders in terms of number of published articles, falling six positions to the 15th place.
Russia's modest share of publications in international journals maybe attributed to the current 'national model' of scientific communication. English now serves as a lingua franca for global science as well as international communication in general. The scientific community is instilling a transnational model of knowledge exchange, where international academic publications are replacing traditional journals. This process was set in motion by the American academic Eugene Garfield2 who set about developing a mechanism to prevent national isolation of the scientific community. Moreover, while the autarchic idea of 'national science' reigned globally up to the end of the 19th century, many European countries subsequently experienced a linguistic globalization in this sphere. Thus, in France from 1973 to 1988, the number of authors publishing in English rose by 45%. By the late 1990s, English became the universal language of scientific communication in developed European countries and Japan, and Anglo-American scientific publications assumed the position of global leaders [Kirchik, 2011]. The proportion of articles co-authored by Russian academics with international partners rose from 3% to nearly 30% from 1980 to 2014 [NRU HSE, 2011, p. 42; 2016, p. 286]. The highest numbers of joint articles are those co-authored by Russian academics in collaboration with researchers from Germany, USA, France, UK, Italy and Japan, which are the absolute global leaders for this figure. However, for various reasons, Russia has not yet managed to fully transition to the international communication model.
The growth in funding for research has been accompanied by an increased demand for research productivity, which needs to be analysed without taking into consideration long-term structural trends in the sector such as ageing personnel and deterioration of infrastructure and facilities [Suslov, 2010; Kuznetsova et al., 2015]. The seriousness of these challenges calls into question the very possibility of guaranteeing growth in labour productivity in the sciences. In order to secure this, we need to examine the factors influencing productivity and the distinguishing features of 'productive' researchers' activities, Based on this, we can identify measures to encourage academics to publish research materials more actively in Russian as well as international journals.
Criteria to evaluate researcher productivity
Labour productivity and pay is directly dependent on the quality of human capital, i.e. the knowledge and skills of the worker. The term 'human capital', proposed by Theodore Schultz [Schultz, 1961], was later classified by Gary Becker into two types: general and specific. 'General human capital' refers to knowledge and skills employed to solve a wide range of tasks in different spheres, while 'specific' describes experience and competencies used in a relatively narrow sphere, which are ineffectual in others [Becker, 1964]. The latter category can include scientific capital as the individuals' accumulation of 'active properties' associated with the dissemination of academic power and recognition (as per the definition and mathematical operationalization proposed in [Katchanov, Shmatko, 2014; Shmatko, Katchanov, 2016, p. 181-182]). With a certain degree of conditionality, we can consider the notion of 'cumulative advantages' developed by Robert Merton to be encompassed by the notion of scientific capital; this refers to the social characteristics of an academic (in particular, professional status and recognition from colleagues), which facilitate the search for resources to conduct research [Merton, 1968b; 1988].
Socio-demographic and psychological factors can have a marked impact on academics' productivity -age, sex (work is a classic example of this [Cole, Zuckerman, 1984]), family, and material status [Fox, 2005]. According to Russian researchers, male teachers at higher education institutions are more inclined to research activity and publishing their results than women [Roshchina, Yudkevich, 2009], while older teachers are more active in this respect than younger. This age asymmetry confirms the existence of the so-called 'Matthew Effect' in science, reflected in the total number of citations - for example, in the Hirsch index, which is higher the older the author becomes.3 This effect mirrors an imbalance characteristic of the scientific community: academics would rather support and promote the achievements of colleagues that have already become well-known thanks to their previous credentials and undervalue or refuse to recognize not so eminent young researchers.
Psychological factors can provisionally be broken down into personal, i.e. associated with the individual qualities of an academic, and organizational or environmental. From studying the first of these types, we can compile a relatively in-depth list of qualities we may find in researchers destined for success in their work: enterprise, emotional stability, openness to communication (applied to the community of
2 The founder of the first academic citation index, subsequently becoming one of the largest databases now owned by the company Thomson Reuters.
3 The peculiarities of taking into account citation figures as the main factor influencing academic recognition, including temporal dynamics, have been described repeatedly in literature (cf. for example: [De Bellis, 2009, pp. 181-242]).
physicists, cf. for example: [Hermanowicz, 2006]), etc. The factors stimulating immersion in work and the productivity of researchers include not only the availability of resources and infrastructure [Hesli, Lee, 2011], but also the equal distribution of resources and infrastructure, including opportunities for independent work [Silman, 2014] and various forms of cooperation [Lee, Bozeman, 2005; Carayol, Matt, 2006]. These are the conditions on which job satisfaction and ultimately the performance of academics depend.
Methodology
This study defines and analyses the factors influencing publication activity among different generations of Russian academics. It relies on data from the Research Personnel Labour Market Monitoring survey conducted in 2010 by the HSE Institute for Statistical Studies and the Economics of Knowledge (ISSEK) as part of the OECD, Eurostat, and the UNESCO Institute for Statistics international project on 'Careers of Doctorate Holders' (CDH).4 This comparative study uses a harmonized methodology and covers 25 participating countries [Shmatko, 2011; Gokhberg et al., 2016]. The research population is made up of candidates and doctors of science (or Ph.D. holders) - employees at research institutions, higher education institutions, and business enterprises. The total sample numbers 3,451 people, i.e. roughly 1% of all doctorate holders in Russia. The data sample was compiled using a multi-stage stratified sample of respondents selected according to a specific quota based on the following criteria: academic level, sex, age, scientific field, territorial affiliation (Federal district). The data were collected during individual interviews at the place of work of each respondent.
For the purposes of our research, we imposed further restrictions: the final sample included only those who were employed at research organizations5, even if this was one of several jobs. This reduced the final sample to 3,034 doctorate holders. However, only 2,633 respondents for whom data included dependent and independent variables in the regression models were selected for further analysis. Over half (58%) of those surveyed were men and the average age was 50 years, with a minimum of 23 and a maximum of 70 years old. In order to analyse generation-specific effects, the formulated sample was split into two groups according to age. The 'younger' cohort of academics included respondents up to the age of 40, including those who completed their higher education after 1991. Given that there has been a gradual reduction in the number of educational programmes, especially in the social sciences and humanities in line with global trends, the human capital of higher education institution graduates has significantly changed in its qualitative characteristics since 1991. New forms of mobility such as student exchanges and teacher training abroad have now become widespread. Foreign language study and work with foreign sources have taken pride of place in many programmes. The final sub-sample of younger researchers totalled 943 people, or 36% of the total number. Respondents aged 41 years or more, which accounted for 64% of the sample, were placed in a second cohort of 'older' academics.
The primary research hypothesis posits that the factors affecting the quality of the scientific capital have a stronger impact on publication activity than the socio-demographic characteristics of academics. The available empirical data allow us to use four indicators of researchers' publication activity over the three years preceding the survey:
1) total number of publications in Russia and abroad;
2) number of publications in foreign outlets;
3) number of publications in Russian outlets;
4) number of articles in peer-reviewed Russian and foreign academic journals.
Since the available data do not allow us to measure the number of publications by respondents and their types as indexed in the Russian Science Citation Index (RSCI), Scopus, or Web of Science databases, we will look at different groups of scientific works. The most representative sample when it comes to evaluating a researcher's level would appear to be the number of publications in scientific peer-reviewed journals -either in Russia or abroad (indicator 4). These works guarantee a certain level of specialization, high text quality, widespread coverage, and citations. However, publications in Russian and foreign journals have different statuses (indicators 2 and 3), as the latter require a command over a foreign language and compliance with international research standards. Using all four indicators allows us to examine different aspects of an academic's publication activity. These indicators will serve as dependent variables in the regression analysis using the Tobit model.6 The model determinants are individual indicators of scientific capital such as work experience, and institutional and socio-demographic characteristics. Some of these variables were measured over the same time frame as the number of publications, which could lead to some endogeneity in the data. Each of the models was also applied to the samples of 'younger' and
4 For more on the project, see: https://www.hse.ru/monitoring/mnk/.
5 The organizations covered by the study included RAS research organizations, research organizations in the higher education sector, research organizations run by state bodies, and industry research institutes.
6 This model was chosen due to the presence of zero values in the dependent variables.
'older' academics. The possession of a doctorate degree, the number of foreign languages used in work, work experience or studies abroad, involvement in various forms of international collaboration, and professional development were all selected as criteria to evaluate individual scientific capital.
Results
Primary characteristics of researchers employed at research organizations
Respondents were asked to talk about all their jobs because multiple employment is common for S&T personnel in Russia. This study has established that roughly 71% of respondents are primarily employed at a research organization, 21% are employed by two research organizations, 4% enjoy a position at a research organization only as their secondary employment, and the remaining 4% hold jobs at three research organizations. We consider the most important characteristic of scientific capital to be the possession of a degree. All of those surveyed had defended a candidate of science dissertation, one quarter of respondents were doctors of science, and roughly 1% held PhDs. More than one third specialize in the natural sciences with the remaining almost equally distributed between the social sciences and engineering. A little over one quarter of respondents received their most recent qualification at a research institute and the remainder in a higher education institution. Almost a third of researchers are not involved in any way in international collaboration; however, over half have participated in conferences, seminars, and other research events taking place in Russia involving colleagues from abroad. A quarter of respondents had taken part in international programmes or conferences. Over two thirds (71%) of those surveyed actively use one foreign - predominantly English - language in their work, while 17% use two. Efforts by academics to improve their knowledge continually, in some cases through further education, can help improve the quality of scientific capital. More than one third of respondents have attended courses, training sessions, or seminars either on their own or an adjoining specialization in the last three years; 14% of respondents have taken computer courses; and only 9% have studied foreign languages. While 78% of academics consider their primary employment to tie in with their specialization, 9% noted a significant shift in their research profile during their lifetime. In addition, the surveyed research personnel changed their place of employment 0.3 times on average over the last ten years and 8% had experienced unemployment. The results of the survey are shown in Table 1.
One fifth of respondents had no publications in the last three years (the average number of publications for all groups was 6.0) and 29% had publications in peer-reviewed journals (average - 4.4). A quarter of those surveyed had foreign publications (average - 1.0), while three quarters had Russian publications (on average, 4.2 publications per person). Thus, Russian academics publish on average roughly two works per year with only one foreign publication every three years. This confirms the prevalence of Russian academics publishing in Russian within the country, which reduces the chances of being noticed by foreign colleagues. It is also worth noting that one in five Russian academics has had no publications in the last three years.
Overall determinants of publication activity
The results of the regression analysis of the factors influencing the publication activity of Russian academics are presented in Table 2 (maximum effects are shown).
We will now look at the impact of variables reflecting scientific capital. We hypothesize that holders of a doctor of sciences degree show the highest levels of productivity both by total publication activity and the number of publications in peer-reviewed academic journals. The effect of this variable was the highest among all the determinants analysed. Surprisingly, holding a PhD does not have a significant impact on the number of foreign publications, but, as expected, it does have an influence on overall productivity. As a general rule, a degree in natural sciences points to higher performance compared with social sciences or humanities. Those academics actively using a foreign language in their work publish more articles than their monolingual colleagues. In other words, the more languages the researcher speaks, the higher the number of publications of all types he or she enjoys. This dependence can easily be explained: the requirement that the research has novelty value is much harder to achieve without access to recent findings and contemporary theories available in the original language. Thus, the ability to read and analyse foreign texts endows an academic with more advantages, in particular the ability to render their own text in a foreign language.
Of all the different forms of international collaboration, participation in foreign conferences reaps the most positive benefits. The impact of such participation on the number of foreign publications even exceeds that of having a doctorate. Other formats of research activity abroad are also significant: work on dissertations, participation in residential schools, forums and international stipendiary programmes, secondments, teaching foreign students, welcoming delegations from abroad, holding international conferences, and so on.7 Giving lectures, joint projects, involvement in research work, and studying
7 The variants mentioned by respondents in an open question.
Table 1. Average values of variables included in the regression analyses, for the three respondent samples
Indicator Total 'Younger' 'Older'
Total number of publications in last three years 6.0 4.7 6.5
Number of foreign publications in last three years 1.0 0.6 1.1
Number of Russian publications in last three years 4.2 3.3 4.6
Number of articles in peer-reviewed journals 4.4 3.3 4.8
Academic position only in primary employment 0.71 0.73 0.70
Academic position in primary employment and other employment (yes=1) 0.21 0.18 0.22
Academic position in all three places of employment (yes=1) 0.04 0.04 0.04
Academic position only in secondary employment (yes=1) 0.04 0.04 0.04
PhD degree 0.01 0.01 0.01
Doctor of Sciences degree 0.23 0.04 0.30
Degree in engineering (yes=1) 0.20 0.23 0.16
Degree in natural sciences (yes=1) 0.38 0.30 0.41
Degree in medical sciences (yes=1) 0.06 0.09 0.05
Degree in agricultural sciences (yes=1) 0.05 0.05 0.05
Degree in social sciences (yes=1) 0.19 0.26 0.16
Degree in humanities (yes=1) 0.07 0.10 0.06
Attended courses in own or adjoining specialization in last three years (yes=1) 0.39 0.46 0.36
Use one foreign language 0.71 0.74 0.70
Use two or more foreign languages 0.17 0.15 0.17
Foreign collaboration: delivered lectures (yes=1) 0.06 0.03 0.07
Foreign collaboration: studied (yes=1) 0.07 0.10 0.06
Foreign collaboration: worked abroad (yes=1) 0.07 0.05 0.07
Foreign collaboration: participated in joint projects (yes=1) 0.25 0.23 0.25
Foreign collaboration: participated in international conferences (yes=1) 0.28 0.24 0.29
Foreign collaboration: other (yes=1) 0.02 0.02 0.02
Directed research projects (yes=1) 0.50 0.35 0.55
Senior position at an educational organization (yes=1) 0.19 0.12 0.21
Senior position at a research organization (yes=1) 0.17 0.08 0.20
Number of times changed place of employment 0.27 0.44 0.22
Defended dissertation at research institute (yes=1) 0.28 0.21 0.30
Work in a higher education institution as primary employment (yes=1) 0.66 0.69 0.64
Permanent employment contract at primary employment (yes=1) 0.60 0.55 0.61
Complete match between primary employment and specialization (yes=1) 0.78 0.72 0.80
Changed research area (yes=1) 0.09 0.09 0.09
Have been unemployed at some time (yes=1) 0.08 0.09 0.07
Studied or worked abroad (yes=1) 0.16 0.14 0.17
Degree held by at least one parent (yes=1) 0.15 0.17 0.14
Age /10 5.04 3.30 5.68
Male (yes=1) 0.58 0.49 0.62
Married (registered or unregistered) (yes=1) 0.76 0.68 0.78
Number of observations 2633 1690 943
Source: authors' calculations.
abroad also help to increase publications, including foreign ones. In other words, any regular international activity by the researcher will raise both their overall productivity and the chances of successfully presenting research abroad.
Paradoxically, developing foreign language skills (by which we mean attending professional development courses) in one's own or adjoining specialization can have a negative impact on the number of publications in peer-reviewed scientific journals. This could be linked to the fact that attending language courses and training during a certain career period detracts researchers from direct work time as well as lowers the initial level of human capital of students on these courses. The skills acquired through such courses have a positive impact on the future level of publication activity, but in the short-term they do not help researchers to improve this indicator significantly.
Table 2. Regression analysis of factors influencing publication activity for all respondents, df/dx
Indicator All publications Foreign Russian Peer-reviewed
Academic position in primary employment and other employment (yes=1) 0.415 0.488** 0.613**
Academic position in all three places of employment (yes=1) -0.464 -0.758 0.195 -0.285
Academic position only in secondary employment (yes=1) -1.362** -1.298* -0.109 -0.421
PhD degree 2.821* 1.988 3.573** 2.816**
Doctor of Sciences degree 4.226*** 2.186*** 3.696*** 3.892***
Baseline - degree in engineering 0.202
Degree in natural sciences (yes=1) 0.672** 1.177*** 0.155 0.678**
Degree in medical sciences (yes=1) 0.439 0.374 0.884 0.523
Degree in agricultural sciences (yes=1) -0.026 -2.494*** 0.479 0.230
Degree in social sciences (yes=1) 0.641 -2.852*** -0.272 -0.636*
Degree in humanities (yes=1) 0.606 -2.386*** -0.215 -1.082**
Attended courses in own or adjoining specialization in last three years (yes=1) -0.434 -0.121 0.796*** -0.530**
Use one foreign language 1.268*** 2.443*** 1.047*** 1.170***
Use two or more foreign languages 1.859*** 2.915*** -0.326 1.333***
Foreign collaboration: delivered lectures (yes=1) 1.598*** 1.020* 0.427 0.119
Foreign collaboration: studied (yes=1) 0.509 0.989* -0.399 0.612
Foreign collaboration: worked abroad (yes=1) -0.057 1.317** 0.086 0.648
Foreign collaboration: participated in joint projects (yes=1) 0.658* 1.581*** 0.733** 0.318
Foreign collaboration: participated in international conferences (yes=1) 1.783*** 2.741*** 0.558 1.122***
Foreign collaboration: other (yes=1) 3.161*** 2.338** 1.179 1.021
Directed research projects (yes=1) 1.637*** 0.618* 0.292 1.328***
Senior position at an educational organization (yes=1) -0.003 -0.417 0.293 -0.132
Senior position at a research organization (yes=1) 0.393 -0.552 -0.462*** 0.339
Number of times changed place of work -0.625*** -0.646** 0.526** -0.608***
Defended dissertation at research institute (yes=1) 0.626* 0.194 -0.350 0.703**
Work in a higher education institution as primary employment (yes=1) -0.565 -1.663*** 0.333* -0.760**
Permanent employment contract at primary employment (yes=1) 0.590** 0.496 0.919*** 0.377
Complete match between primary employment and specialization (yes=1) 1.224*** 0.768* 0.172 1.160***
Changed research area (yes=1) 0.235 0.207 -0.092 0.179
Have been unemployed at some time (yes=1) 0.233 0.259 -0.232 -0.140
Studied or worked abroad (yes=1) 0.309 1.395*** -0.071 0.388
Degree held by at least one parent (yes=1) 0.326 0.737* 0.555 0.078
Age /10 1.424 0.897 -0.065 0.690
Age /10 (square function of) -0.160* -0.059 0.701*** -0.094
Male (yes=1) 0.984*** 0.344 -0.316 0.825***
Married (registered or unregistered) (yes=1) -0.232 0.395 -1.159 -0.043
Number of observations 2633 2633 2633 2633
Number of uncensored observations 2141 626 629 1927
Pseudo-coefficient of determination (R Square) 0.04 0.14 0.04 0.05
Model significance (Prob> chi2) 0.000 0.000 0.0000 0.000
Coefficient significance: * — 10%, ** — 5%, *** — 1%. Source: authors' calculations.
Those only employed by research organizations enjoy more publications in peer-reviewed academic journals than colleagues who work in such places for at least one of their jobs (even if this is their primary job). Primary employment in a non-academic sector, even with a secondary job in academia, has a negative impact on all research productivity indicators.
In the paper [Gershman, Kuznetsova, 2013], the authors describe the benefits of an 'effective contract,' which encourages researchers to increase their research performance. The pay principles bound up in this concept are actively used at foreign universities. It is also significant that having one's primary employment at a higher education institution can have a negative impact on publication activity in foreign journals. This seems to indicate that the relevant incentives are insufficient and demonstrate a weak dependence between teachers' incomes and their research effectiveness.
Out of all the socio-demographic characteristics, there is only a significant correlation for respondents' gender: males demonstrate a higher overall level of productivity and publish more frequently. Contrary
to expectations, age does not appear to have a significant impact on this indicator in any of the models. However, the model for all samples (Table 2) showed that over a researcher's lifetime the total number of publications initially shows an upward trend and then declines. For older academics, publication in Russian outlets is the predominant trend. The models for this age cohort (Table 3) show that age has a negative impact on the number of peer-reviewed publications (10% significance), which can be explained partly by the tradition of publishing monographs over articles.
A one-way analysis of variance confirms that older academics publish more intensively on average than their younger colleagues (Table 4). However, since the dependence of publication in foreign outlets and peer-reviewed scientific journals on age is not significant in the regression analysis (Table 2), then,
Table 3. Regression model of factors influencing publication activity among researchers
in the older age cohort, df/dx
Indicator All publications Foreign Russian Peer-reviewed
Academic position in primary employment and other employment (yes=1) Peer-reviewed 0.723 0.437 0.782**
Academic position in all three places of employment (yes=1) -0.161 -0.285 0.387 -0.071
Academic position only in secondary employment (yes=1) -1.699* -1.143 -0.365 -0.636
PhD degree 3.459 0.417 4.755*** 2.954
Doctor of Sciences degree 4.280*** 2.144*** 3.721*** 3.942***
Baseline - degree in engineering
Degree in natural sciences (yes=1) 0.623 1.165** 0.043 0.613
Degree in medical sciences (yes=1) 0.240 -0.043 0.114 0.327
Degree in agricultural sciences (yes=1) -0.336 -2.924*** 0.674 -0.311
Degree in social sciences (yes=1) 0.296 -2.809*** 0.131 -1.136**
Degree in humanities (yes=1) -0.134 -2.894*** -0.417 -1.585**
Attended courses in own or adjoining specialization in last three years (yes=1) -0.451 0.190 -0.134 -0.540
Use one foreign language 1.641*** 2.389*** 1.026** 1.172**
Use two or more foreign languages 2.150*** 2.556*** 1.663*** 1.696***
Foreign collaboration: delivered lectures (yes=1) 1.262* 1.055 -0.021 0.208
Foreign collaboration: studied (yes=1) 1.248* 1.523** 0.836 1.290**
Foreign collaboration: worked abroad (yes=1) 0.129 1.459** -0.499 0.603
Foreign collaboration: participated in joint projects (yes=1) 0.617 1.694*** 0.084 0.387
Foreign collaboration: participated in international conferences (yes=1) 1.879*** 2.951*** 0.772** 1.141***
Foreign collaboration: other (yes=1) 3.012** 1.991 0.644 1.165
Directed research projects (yes=1) 1.655*** 0.791* 1.319*** 1.460***
Senior position at an educational organization (yes=1) 0.269 -0.284 0.533 0.224
Senior position at a research organization (yes=1) 0.110 -0.557 0.066 0.247
Number of times changed place of employment -0.899*** -1.178*** -0.507** -0.742***
Defended dissertation at research institute (yes=1) 0.348 -0.084 0.391 0.518
Work in a higher education institution as primary employment (yes=1) -0.851* -1.941*** -0.636 -1.003**
Permanent employment contract at primary employment (yes=1) 0.654* 0.552 0.525 0.632*
Complete match between primary employment and specialization (yes=1) 1.022** 0.803 0.843** 1.121***
Changed research area (yes=1) 0.087 0.601 -0.339 -0.469
Have been unemployed at some time (yes=1) -0.004 -0.441 -0.106 -0.211
Studied or worked abroad (yes=1) 0.267 1.526*** -0.275 0.366
Degree held by at least one parent (yes=1) 0.269 1.210** -0.221 0.097
Age /10 -0.484 0.276 -0.218 -0.432*
Male (yes=1) 0.573 0.005 0.474* 0.509
Married (registered or unregistered) (yes=1) 0.361 0.816 -0.072 0.339
Number of observations 1690 1690 1690 1690
Number of uncensored observations 1387 442 409 0.05
Pseudo-coefficient of determination (R Square) 0.04 0.13 0.04 0.78
Model significance (Prob> chi2) 0.000 0.000 0.000 0.000
Source: authors' calculations.
Table 4. Regression model of factors influencing publication activity among younger researchers, df/dx
Indicator All publications Foreign Russian Peer-reviewed
Academic position in primary employment and other employment (yes=1) 0.358 -0.414 0.643* 0.232
Academic position in all three places of employment (yes=1) -1.107 -2.294* -0.213 -0.573
Academic position only in secondary employment (yes=1) 0.559 -0.740 0.751 0.442
PhD degree 2.644 3.357** 2.131* 2.716**
Doctor of Sciences degree 4.116*** 2.446*** 3.774*** 3.715***
Baseline - degree in engineering
Degree in natural sciences (yes=1) 0.647 1.117* 0.643* 0.786*
Degree in medical sciences (yes=1) 0.796 0.988 0.462 0.957
Degree in agricultural sciences (yes=1) 0.319 -1.169 1.078* 1.073
Degree in social sciences (yes=1) 1.332** -2.190*** 1.126*** 0.334
Degree in humanities (yes=1) 1.998** -0.800 0.429 0.035
Attended courses in own or adjoining specialization in last three years (yes=1) -0.362 -0.914** -0.263 -0.424
Use one foreign language -0.164 2.864** 0.097 0.794
Use two or more foreign languages 1.206 4.355*** -0.486 0.361
Foreign collaboration: delivered lectures (yes=1) 2.989** 0.692 -1.679** -0.756
Foreign collaboration: studied (yes=1) -1.369 -0.271 -0.164 -0.524
Foreign collaboration: worked abroad (yes=1) 0.516 2.070** 0.072 1.297*
Foreign collaboration: participated in joint projects (yes=1) 0.963* 1.060* 0.214 0.229
Foreign collaboration: participated in international conferences (yes=1) 1.480*** 1.914*** 0.586* 0.992***
Foreign collaboration: other (yes=1) 3.505** 2.352* 0.412 0.472
Directed research projects (yes=1) 1.460*** 0.041 0.793*** 0.982***
Senior position at an educational organization (yes=1) -1.251* -1.387* -0.566 -1.342
Senior position at a research organization (yes=1) 2.500*** -0.448 1.205** 0.678
Number of times changed place of employment -0.190 -0.008 -0.374** -0.423**
Defended dissertation at research institute (yes=1) 2.262*** 1.168* 1.217*** 1.579***
Work in a higher education institution as primary employment (yes=1) 0.675 -0.731 0.424 0.141
Permanent employment contract at primary employment (yes=1) 0.319 0.288 -0.075 -0.108
Complete match between primary employment and specialization (yes=1) 1.886*** 0.696 1.119*** 1.207***
Changed research area (yes=1) 0.717 -0.651 1.388*** 1.445***
Have been unemployed at some time (yes=1) 0.673 1.219** 0.070 0.307
Studied or worked abroad (yes=1) 0.498 1.072 0.126 0.676
Degree held by at least one parent (yes=1) 0.479 -0.562 0.146 -0.069
Age /10 -0.706 0.043 -0.328 -0.647
Male (yes=1) 1.565*** 0.783* 1.080*** 1.204***
Married (registered or unregistered) (yes=1) -1.221*** -0.122 -0.568** -0.451
Number of observations 943 943 943 943
Number of uncensored observations 762 184 220 695
Pseudo-coefficient of determination (R Square) 0.03 0.15 0.03 0.04
Model significance (Prob> chi2) 0.000 0.000 0.0000 0.000
Source: authors' calculations.
all things being equal (e.g. having a doctoral degree, international collaboration), the impact of age is negligible. In other words, the actual level of academics' publication activity is determined by the quality of their scientific capital. This hypothesis is also supported by the inclination among older researchers to publish in Russian, which (excluding the those in peer-reviewed journals) does not always guarantee high quality.
Factors influencing younger and older academics' publication activity
One of the hypotheses of this study is the disproportionate distribution of 'accumulated advantages' between academics (the 'Matthew Effect') , and their role on income received by colleagues from different generations (for a definition of these age categories, see section 'Methodology'). The data confirm the assumption that older academics demonstrate better performance level than their younger colleagues (Table 1) but do not allow for an unequivocal conclusion as to the higher relative productivity of the first
group compared with the second. We will now look in more detail at how scientific capital, employment conditions, and socio-demographic characteristics can influence the publication activity of different generations of researchers.
The formal level of scientific capital among younger generation academics is still relatively low (Table 1). Although they often have a PhD that shows a positive effect on research productivity, the proportion of PhD holders among young researchers is not enough to influence greater publication activity. There is almost no difference in command of foreign languages between the two generations. The differences in their involvement in international collaborations are also negligible, although older researchers tend to give lectures, participate in joint projects, and go to conferences abroad slightly more often. Among younger academics, the proportion of those who have attended professional development courses in their own or related specialization is higher. Predictably, the older generations tend to hold senior positions in science and education and, as a general rule, work in their specialization on a permanent basis. Younger colleagues are more active in changing their place of work, often experience difficulty in finding work, and are typically affiliated with higher education institutions.
The regression analysis helps to identify the factors that help younger academics to overcome the 'Matthew Effect', secure additional advantages, and raise their productivity (Tables 3 and 4). Thus, a Doctor of Sciences degree (the highest doctorate award) is the most significant factor influencing publication activity for both generations of academics. A PhD has a significant impact on the number of journal publications (both Russian and foreign) by young researchers and on the total number of Russian publications by their older colleagues. This relationship can be explained by the fact that during their study on foreign doctoral programmes, younger academics acquire a significant store of up-to-date knowledge and a mastery of international article authoring standards.
Experience of research abroad helps to increase the publication activity of both generations. Various forms of academic mobility (business trips, residential schools, grants and scholarships from foreign higher education institutions) and international collaboration (student exchange, welcoming foreign delegations, conferences) can have a marked impact on growth in the number of articles published. However, participating in international conferences unlike giving lectures has a weak impact on the productivity of younger academics. Joint international projects have a moderate effect on the number of publications in foreign scientific journals of both age cohorts. Study or placements abroad only have a significant impact on overall performance and international publication activity for the older generation of academics. Management of research projects can increase both of these indicators for younger and older academics in equal measure; furthermore, older academics can increase their presence in foreign publications. Frequent changes in work place can have a serious adverse effect on publication indicators for older researchers, while for younger colleagues it can have significant negative as well as moderate consequences.
There is likely to be some connection between a researcher's working conditions and publication activity: low publication figures under an 'Effective Contract' mechanism can deter a research organization from hiring such an employee on a permanent contract. At a new place of work, an employee settles down to work on new projects, often discarding previously initiated projects; this is corroborated by the growth in publication activity - both overall and in peer-reviewed journals - among the older generation of researchers who work under permanent employment contracts.
The match between the researcher's primary employment and specialization also has a positive impact on overall publication activity and the number of articles in peer-reviewed journals for both generations. However, for younger researchers the opposite dependence is more accurate: a fundamental shift in research area can lead to growth in peer reviewed journal publications. The same dependence is true of unemployment among this age cohort, which can increase the number of foreign publications. This can likely be explained by the fact that periods of formal unemployment often bring about an intensification of the academic process or foreign travel. Predictably, foreign educational programmes improve the corresponding indicators of the older generations, and are higher among those holding engineering or medical degrees in comparison to other specializations. There are no such differences in foreign publications of younger academics; only those with a degree in social sciences are less published abroad compared with members of engineering disciplines, while those with engineering degrees have less of a presence in Russian publications.
Relationship between publication activity and job satisfaction
Another hypothesis posited by this study was the relationship between academics' publication activity and their motivation and job satisfaction. Due to significant institutional differences between the academic and non-academic spheres, in this study we only selected those respondents who were employed primarily at research or educational organizations and who responded to questions about satisfaction with working conditions.
Researchers were most satisfied with the length of leave (average of 3.5 on a four-point scale), job security (3.2), social importance (3.2), and the geographic location (3.0) of their work, as well as the opportunity for creative expression (3.1) and the responsibility entrusted to them (3.1). Academics were least satisfied
Table 5. Factor analysis of academics' satisfaction with w< arking conditions
Satisfaction at primary employment Factor loadings
1 2 3
Opportunity for creative expression 0.756 0.079 0.186
Opportunities for professional development 0.716 0.254 0.119
Level of responsibility 0.652 0.069 0.293
Degree of freedom/level of independence 0.646 0.028 0.274
Social importance of work 0.642 0.110 0.155
Prestige of work 0.625 0.272 0.060
Availability of the required S&T information 0.601 0.287 0.152
Opportunities for international collaboration 0.581 0.225 -0.013
Primary wage 0.229 0.811 0.005
Bonuses, allowances 0.224 0.820 0.47
Benefits 0.111 0.698 0.275
Availability of modern devices and equipment 0.509 0.446 0.053
Length of leave 0.208 -0.065 0.786
Proximity to home 0.054 0.144 0.596
Job security 0.366 0.228 0.587
| Source: authors' calculations.
with payment-related issues: wages (2.2), bonuses (2.2), and benefits (2.5). The results largely confirm the theory regarding internal motivations of research personnel taking precedence over external ones [Volodarskaya, Kiseleva, 2012]. To reduce the dimensions and determine the internal structure of the attribute space, we applied a factor analysis model (principal component analysis, Varimax rotation). The results identified three factors, explaining 53% of the overall dispersion (Table 5).
The first factor can be described as satisfaction with opportunities for creative work and professional development. Research allows academics to manage their schedule, which is not administratively regulated, as the outcome depends on the individual rhythm of an employee's creative activity. Professional development and social importance are additional elements of the subjective satisfaction of research personnel, driven not only by curiosity, but also a need to have the fruits of their labour approved by the professional community.
The second factor is linked to satisfaction with material incentives (pay and benefits). While internal motivation dominates, a worthy wage and social protection are both important conditions underpinning academics' professional activities. Low primary income forces academics to search for additional sources of earnings, resort to fundraising, and participate in commercial projects. Arguably, the main incentive for academics with set-ups such as this is the system of internal competition or additional payments at the primary place of employment.
The third factor can be characterized as satisfaction with working conditions, expressed through indicators such as proximity to home, job security, and length of leave. These intangible incentives, together with the creative nature of research and direct material rewards, are most frequently used by universities and research institutes.
The existence of relationships between these factors and key productivity indicators was verified using a correlation analysis. A significant dependence but sparse correlation, was established only for the first and third factors. In particular, satisfaction with opportunities for creative expression is positively correlated with researchers' publication activity. Thus, in order to achieve the very best productivity indicators, academics need a certain degree of academic freedom. The negative correlation between number of foreign publications and job satisfaction could indicate academics' willingness to forego comfort for the sake of achieving a significant outcome. However, the reasons for this dependence require further study. Importantly, in our view, the lack of significant correlation between satisfaction with income and the level of publication activity can likely be explained by the fact that material incentives are essential for research personnel, but inadequate when it comes to working conditions.
Conclusion
According to our findings, academics who have their primary employment at research organizations publish on average six scientific works every three years (Table 1). Researchers with no employment elsewhere are less productive compared to those who combine research assignments with employment in other sectors of the economy (but do research work as their primary activity). The majority of texts are published in Russian: only one in six articles published over a three-year period were issued abroad.
Table 6. Comparison of publication activity and pay in the older and younger age cohorts of academics
Group Average number of publications Average income at primary place of employment (thousands of roubles)
All publications Foreign Russian Peer-reviewed Research organizations Other organizations
Older (41-70 years) 6.53 0.98 4.34 4.78 269.2 448.5
Younger (23-40 years) 4.82 0.56 3.20 3.38 242.7 363.1
Source: authors' calculations.
Generally, older academics are more productive and publish one to two articles more than their younger colleagues. Younger academics, in turn, are more active when it comes to foreign publications.
Academics' satisfaction with their primary employment at research organizations, namely opportunities for creative expression and development, has a positive correlation with productivity. However, no significant relationship between the amount of material remuneration and publication activity was identified (Table 6), which points to the secondary nature of financial incentives in relation to other factors. Academics' publication activity can be buoyed not only by a high level of pay, but also stimulating working conditions that include opportunities for creative expression and a certain degree of academic freedom.
The regression analysis allowed us to identify the key determinants of publication activity. The greatest impact on academic productivity came from various characteristics of scientific capital, (or 'accumulated advantages'). Primarily, these include a highest doctorate level, experience in international collaboration, and management of research projects. On the other hand, academic career characteristics such as a senior position in an educational institution can play a negative role. Ultimately, a positive factor influencing publication activity among Russian researchers is working within one's specialization and subscribing to the notion of a research career as a long-term life choice.
Older generations of academics exhibit higher levels of publication activity. However, the field of specialization in which the researcher obtained their degree can affect both age cohorts in different ways. Among the older generations, those specialized in the medical sciences and engineering have an advantage, while among younger academics, these differences are negligible; a more perceptible effect comes from having a PhD and engagement in international collaboration, primarily conference participation. Finally, success in the publications market comes more from the quality of scientific capital (for example, mastery of foreign languages, a doctoral degree, management and involvement in international projects and various programmes) than from age and other socio-demographic characteristics. Against this backdrop, the more successful strategies are expected at those universities and research centres that are able to provide their younger employees with the necessary conditions for professional development by offering material incentives as well as the possibility of integration into a global professional community.
This article was prepared from data obtained from the project 'Scientific, Methodical and Analytical Support for the Monitoring and Extra-Departmental Evaluation of the Performance of Research Organizations Carrying Out Scientific, Research, Experimental, Design And Technological Work For Civilian Use', at NRU HSE. The project was supported by the Ministry of Education and Science of the Russian Federation in 2014-2015 (Agreement No. 14.602.21.0004, identification number RFMEFI60214X0004).
References
Ancaiani A., Anfossi A.F., Barbara A., Benedetto S., Blasi B., Carletti V., Sileoni S. (2015) Evaluating scientific research in Italy: The 2004-10 research evaluation exercise. Research Evaluation, vol. 24, no 3, pp. 242-255.
Ball P. (2005) Index aims for fair ranking of scientists. Nature, no 436, pp. 900-900.
Becker G. (1964) Human capital: A theoretical and empirical analysis, with special reference to education, Chicago: The University of Chicago.
Bordons M., Fernández M.T., Gómez I. (2002) Advantages and limitations in the use of impact factor measures for the assessment of research performance in a peripheral country. Scientometrics, vol. 53, no 2, pp. 195-206.
Carayol N., Matt M. (2006) Individual and collective determinants of academic scientists' productivity. Information Economics and Policy, vol. 18, no 1, pp. 55-72.
Cole J.R., Zuckerman H. (1984) The Productivity Puzzle: Persistence and change in patterns of publication among men and women scientists. Advances in Motivation and Achievement (eds. M.W. Steimkamp, M. Maehr), vol. 2, Greenwich, CT: JAI Press.
De Bellis N. (2009) Bibliometrics and citation analysis: From the Science citation index to cybermetrics, Lanham, Toronto, Plymouth: Scarecrow Press.
Fox M.F. (2005) Gender, family characteristics, and publication productivity among scientists. Social Studies of Science, vol. 35, no 1, pp. 131-150.
Fursov K. (2015) Rossiya v global'noi nauke: Rezul'taty bibliometricheskogo analiza [Russia in Global Science: Results of Bibliometric Analysis]. Naukovedcheskie issledovaniya (Ezhegodnik: Sb. nauchn. tr.) [Research in Sociology of Science. Scientific Yearbook], Moscow: INION RAN, pp. 61-80 (in Russian).
Garfield E. (2009) From the science of science to Scientometrics: Visualizing the history of science with HistCite software. Journal of Informetrics, vol. 3, no 3, pp. 173-179.
Gershman M., Kuznetsova T. (2013) Efficient Contracting in the R&D Sector: Key Parameters. Foresight-Russia, vol. 7, no 3, pp. 26-36 (in Russian).
Godin B. (2006) On the origins of bibliometrics. Scientometrics, vol. 68, no 1, pp. 109-133.
Godin B. (2009) Chto takoe nauka? Razvitie statisticheskogo opredeleniya: 1920-2000 [What is Science? Defining Science by Numbers, 1920-2000]. Foresight-Russia, vol. 3, no 2, pp. 48-60 (in Russian).
Godin B. (2013) Measurement and Statistics on Science and Technology: 1920 to the Present, London: Routledge.
Godin B., Gingras Y. (2000) What is scientific and technological culture and how is it measured? A multidimensional model. Public Understanding of Science, no 9, pp. 43-58.
Gokhberg L., Shmatko N., Auriol L. (eds.) (2016) The Science and Technology Labor Force: The Value of Doctorate Holders and Development of Professional Careers, London, New York, Dordrecht, Heidelberg: Springer International Publishing.
Gokhberg L., Zaichenko S., Kitova G., Kuznetsova T. (2011) Nauchnaya politika: global'niy kontekst i rossiiskaya praktika [Science Policy: Global Context and Russian Practice], Moscow: HSE.
Hermanowicz J.C. (2006) What Does It Take to Be Successful? Science, Technology, and Human Values, vol. 31, no 2, pp. 135-152.
Hesli V.L., Lee J.M. (2011) Faculty research productivity: Why do some of our colleagues publish more than others? PS: Political Science and Politics, vol. 44, no 2, pp. 393-408.
Hicks D. (2012) Performance-based university research funding system. Research Policy, no 41, pp. 251-261.
Hicks D., Wouters P., Waltman L., De R., Rafols I. (2015) Bibliometrics: The Leiden Manifesto for research metrics. Nature, vol. 520, no 7548, pp. 429-431.
HSE (2011) Rossiiskiy innovatsionniy indeks [Russian Innovation Index] (ed. L. Gokhberg), Moscow: HSE (in Russian).
HSE (2016) Indikatory nauki 2016: Statisticheskii sbornik [Science and Technology Indicators 2016: Data Book], Moscow: HSE (in Russian).
Katchanov Y.L., Shmatko N.A. (2014) Complexity-Based Modeling of Scientific Capital: An Outline of Mathematical Theory. International Journal of Mathematics and Mathematical Sciences, vol. 2014, article ID 785058. Available at: http://www.hindawi.com/journals/ijmms/2014/785058/, accessed 03.03.2016.
Kirchik O. (2011) «Nezametnaya» nauka. Patterny internatsionalizatsii rossiiskikh nauchnykh publikatsii ['Invisible' Science: Patterns of Internationalisation for the Russian Scientific Publications]. Foresight-Russia, vol. 5, no 3, pp. 34-42 (in Russian).
Kuznetsova I., Kuznetsova T., Martynova S., Suslov A. (2015) Otsenka material'no-tekhnicheskoi bazy nauki: voprosy klassifikatsii nauchnogo oborudovaniya [Evaluation of the material and technical base of science: Scientific equipment classification issues]. Rossiiskie nanotekhnologii [Russian Nanotechnologies], vol. 10, no 11-12, pp. 13-21 (in Russian).
Lee S., Bozeman B. (2005) The Impact of Research Collaboration on Scientific Productivity. Social Studies of Science, vol. 35, no 5, pp. 673-702.
Merton R. K. (1968) The Matthew Effect in Science. Science, vol. 159, no 3810, pp. 56-63.
Merton R.K. (1973) The Sociology of Science: Theoretical and Empirical Investigations, Chicago: University of Chicago Press.
Merton R.K. (1988) The Matthew Effect in Science, II: Cumulative Advantage and the Symbolism of Intellectual Property. Isis, vol. 79, no 4, pp. 606-623.
Moed H.F. (2009) New developments in the use of citation analysis in research evaluation. Archivum Immunologiae et Therapiae Experimentalis, no 57, pp. 13-18.
OECD (2014) OECD Science, Technology and Industry Outlook 2014, Paris: OECD.
Roshchina Ya., Yudkevich M. (2009) Faktory issledovatel'skoi deyatel'nosti prepodavatelei vuzov: politika administratsii, kontraktnaya nepolnota ili vliyanie sredy? [Factors effecting university teachers' research: Management policies, contract incompleteness or environment influences?]. Voprosy obrazovaniya [Educational Studies], no 3, pp. 203-228 (in Russian).
Roud V., Fursov K. (2011) Rol' statistiki v diskussii o nauchno-tekhnologicheskom i innovatsionnom razvitii [The Role of Statistics in the Debate on Science, Technology, and Innovation]. Voprosy ekonomiki, no 1, pp. 120-133 (in Russian).
Shmatko N. (2011) Scientific Capital as a Driver of Researchers' Social Mobility. Foresight-Russia, vol. 5, no 3, pp. 18-32 (in Russian).
Shmatko N., Katchanov Y. (2016) Professional Carreers and Mobility of Russian Doctorate Holders. The Science and Technology Labor Force: The Value of Doctorate Holders and Development of Professional Careers (eds. L. Gokhberg, N. Shmatko, L. Auriol), London, New York, Dordrecht, Heidelberg: Springer International Publishing.
Silman F. (2014) Work-related basic need satisfaction as a predictor of work engagement among academic staff in Turkey. South African Journal of Education, vol. 34, no 3, pp. 1-5.
Stephan P. (2012) Research efficiency: Perverse incentives. Nature, vol. 484, no 7392, pp. 29-31.
Suslov A. (2010) Planirovanie nauchnoi kar'ery: vzglyad skvoz' prizmu sotsiologicheskogo obsledovaniya [Scientific career planning: Glance through the prism of sociological survey]. Voprosy statistiki, no 8, pp. 35-41 (in Russian).
Volodarskaya E., Kiseleva V. (2012) Oplata truda uchenykh i motivatsiya nauchnoi deyatel'nosti [Remuneration of scientists and motivation of scientific activities]. Motivatsiya I oplata truda [Motivation and Remuneration], no 2, pp. 154-167 (in Russian).
Weingart P. (2005) Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, no 62, pp. 117-131.
