CC BY-NC-ND
130
Publication Metrics as a Tool for Measuring Research Productivity and Their Relation to Motivation
A. Kalgin, O. Kalgina, A. Lebedeva
Received in Alexander Kalgin
September 2018 PhD, Associate Professor, School of Public Administration, Faculty of Social Sciences, National Research University Higher School of Economics. Address: 20 Myasnitskaya St, 101000 Moscow, Russian Federation. Email: [email protected] Olga Kalgina
Junior Researcher, Laboratory for Political Studies, Faculty of Social Sciences, National Research University Higher School of Economics. Address: 3 Krivokolenny Lane, 101000 Moscow, Russian Federation. Email: [email protected] Anna Lebedeva
Candidate of Sciences in Psychology, Senior Researcher, International Laboratory of Positive Psychology of Personality and Motivation, National Research University Higher School of Economics. Address: Bld. 2, 4 Slavyanskaya Sq, 109074 Moscow, Russian Federation. Email: [email protected]
Abstract. The article presents an analytical review of literature on publication metrics as a tool of performance management in academia. Issues of quantitative research assessment are investigated in the light of modern views of motivation, in particular through the lens of self-determination theory (SDT). The article provides an insight into empirical studies on the effects of intrinsic and extrinsic motivation on publication productivity, research quality and subjective wellbeing. Accumulated international experience in performance management is used as a basis for developing recommendations on how to improve academic governance.
Keywords: intrinsic motivation, governance, science, performance management, publication performance measurement.
DOI: 10.17323/1814-9545-2019-1-44-86
The article was prepared within the framework of the Basic Research Program at the National Research University Higher School of Economics (HSE) and supported within the framework of a subsidy by the Russian Academic Excellence Project '5-100'. Translated from Russian by I. Zhuchkova.
Russian universities are committed to the ambitious goal of making it to prestigious international rankings. Achievement of this goal requires concentration of efforts on the priority growth areas, such as publication activities. Under the logic of performance management, goals should be transformed into a system of incentives to ensure maximum employee productivity. Publication performance measurement systems have been designed to provide quantitative assessment of research. This article suggests analyzing the potential effects of such systems from the perspective of self-determination theory. We provide a review of studies on how publication performance measurement affects motivation and productivity in academia.
Publication metrics are one of performance management tools, which are actively used in universities in Russia and beyond. The present-day doctrine of performance management has its roots in the classical works of Peter Drucker [Drucker 1954; 1964], which assert that such tools as merit pay and regular performance reviews serve to reconcile individual employee motivations with strategic goals of organizations. In theory, performance management tools are supposed to enhance employee motivation. However, evidence of their effects is rather equivocal, particularly in knowledge production, where performance is hard to measure explicitly [Kallio, Kallio 2014].
The extensive use of performance management tools has been inspired by the philosophy of new public management, which seeks to transfer managerial solutions developed in the private sector into the public domain [Dooren, Bouckaert, Halligan 2010]. Since universities with government-funded research activities are a special case of public sector organizations, they also experience the adverse effects of such transfer. Performance of public sector organizations is much more difficult to measure than that of private businesses. With some reservations, the main goal of businesses is to maximize profits, which can thus be used as a uniform measure of their performance (although it poses a number of problems, too). The public sector has no such uniform measure of performance. There have been numerous attempts to compensate for this deficiency by creating performance assessment systems based on various substitutes for what profits are in the private sector. In higher education, the role of such substitute is played by publication metrics.
This paper intends to examine the effects of applying performance management tools in universities on researchers' motivation. The article is structured as follows. In section one, we discuss the specific aspects of contemporary universities' organizational culture and Russia's academic market. Section two describes the theoretical foundations of the performance management doctrine, which manifests itself in the assessment of publication activities, and the accumulated experience in the field. As a next step, intrinsic motivation theory is used as a framework for discussing researchers' professional motives and investigating how they can be affected by performance measurement systems. Finally, an overview of empirical studies on the effects of using quantitative bibliometric indicators in universities around the world is followed by the conclusion and recommendations.
The analytical review of literature presented in the article makes no pretense to being exhaustive or consistent, yet we tried to include a large variety of publications on performance management systems, researchers' motivation and self-determination theory as well as recent empirical studies on the effects of introducing bibliographic indicators in western universities. The article makes use of both Russian and foreign publications.
1. University Traditionally, two types of university cultures coexisted, collegiate and Culture and bureaucratic [Melo, Sarrico, Radnor 2010]. Collegiate cultures were Managerialism observed in academic communities where every member had an equal vote, while bureaucracy was typical of administrative staff playing subordinate roles. This traditional paradigm is undergoing a transformation in today's universities. Researchers are becoming managers who commit to performance and organize the work of their teams so as to make good on their commitments [Sousa, de Nijs, Hendriks 2010].
Integration of publication metrics in universities is one of the manifestations of managerialism, which swept over the academic world with the advent of new public management. Obsessive attention to the number of publications is not typical of conventional academic cultures. Under the new conditions, university has basically turned from the ivory tower to the academic assembly line [Barry, Chandler, Clark 2001]. There is a tendency towards proletarianization of academic labor [Wilson 1991] and commodification of research [Willmott 1995]. The phrase "academic corporation" is acquiring a new dimension today, corporation being understood not as a merger of equals but as a large manufacturing business [Kurakin, Filippov 2006].
As centralized performance assessment systems get introduced, they change the structure of researchers' accountabilities. Instead of being accountable to their colleagues, academics find themselves accountable to managers, i. e. university administrators. The traditional collegiality standards give way to managerial accountability [Melo, Sarrico, Radnor 2010]. Researchers preserve a certain degree of autonomy in choosing the method of meeting their commitments but have to satisfy the performance requirements imposed by university administrators, which may be perceived as loss of the much-prized academic freedom [Ibid.].
1.1. Academic Market The Russian academic market is characterized by relatively low competition among universities. A small number of Russian universities possess oligopsony power (a state of the market being dominated by a very limited number of large buyers), thus capturing all of the demand for researchers. Konstantin Sonin calls the absence of an academic market the "weak spot" of Russian science [Sonin 2016]. In a situation like that, universities that seek to increase their publication counts are forced to rely on artificially created stimuli. A competitive labor market makes academics intrinsically motivated to produce more publications (in fact, such motivation is only partially intrinsic, being "foisted" by the external competitive environment). In Russia, universities find themselves in a situation where they have to use administrative leverage to stimulate publications. The ever more widespread publication metrics are one of such administrative levers, probably the most important one. This trend is intensified by Russian universities engaging actively in the ranking race and government funding programs, where assessment criteria derive from the logic of performance management.
2. Performance Management
2.1. Principal-Agent Theory
As a tool of new public management, performance management has its theoretical underpinnings in principal-agent theory, which centers on the conflict between the interests of agents (employees) and principals (managers). The fundamental premise is that their interests diverge and must be harmonized. Agents want to get maximum reward for minimum effort, whereas principals, on the contrary, want to make agents put in maximum effort for minimum reward. Principal-agent theory suggests that stricter monitoring improves efficiency of agents, simulating them to exert more work-related effort. Agents have been traditionally treated under this theory as shirkers who try to minimize their effort unless they are closely monitored [Tullock 2005]. It is suggested that agent efficiency can be improved by introducing a system of punishment and reward, which rewards achievement of results perceived as useful by the principal and punishes failure to achieve them. Principal-agent theory is focused on external pressure and extrinsic incentives. In respect to universities, integration of a performance management system is expected to enhance the efficiency of monitoring the performance of agents (researchers) by principals (university administrators).
Applying the principles of performance management to universities has resulted in using publication metrics to monitor research performance. There are two types of performance management systems used to increase the number of publications; one suggests financial incentives for publications, and the other sets minimum requirements on the number of publications as well as penalties for failure to meet them. Principal-agent theory proceeds from the premises of rational choice, meaning that rewards and punishments modify the ways agents calculate their utility, so that commitment to principals' interests becomes a subjectively more profitable choice.
2.2. Positive Effects of Performance Management
Performance evaluation and formalized systems of punishment and reward may have positive effects in different areas of management. Robert D. Behn [Behn 2003] discriminates among eight managerial purposes that performance measurement helps achieve (Table 1).
The results of performance measurement can serve the basis for managerial implications. The fact that one academic unit has more publications than another one may be the grounds for judging the former as high-performing and the latter as low-performing. The judgments may be then used in making decisions on budget allocation, evaluation of spending efficiency, rewards for high performers, and investments in training. Efficiency of such investments, in its turn, can be evaluated based on qualitative ROI. Otherwise speaking, performance measurement is a radical method of reducing administrative costs, which simplifies the view of the organization and provides a rational basis for many managerial decisions, thus reducing uncertainty that all managers face inevitably. It also facilitates the task of demonstrating the university's research output to external audiences, which
Table 1. Eight Managerial Purposes of Performance Measurement [Behn 2003] (adapted)
Managerial Purpose What the performance measure can help find out
Evaluate How well the agency is performing
Control Whether subordinates are doing the right thing
Budget On what programs the agency should spend
Motivate How to motivate staff to improve performance
Promote How to convince politicians and citizens that the agency is doing a good job
Celebrate What accomplishments are worthy of celebrating success
Learn Why something is working and something is not
Improve What exactly and who should do differently to improve performance
include officials, students and employers. When the government stiffens accountability requirements, quantitative measures become a convenient mechanism for reducing all the diversity of university research down to communicable scientometric indicators.
The mechanism of performance assessment is congruent with the rationalistic perception of organization, which depict the working of an organization as a successive progression from planning to execution to feedback. It may be that the key to long-term success of performance management is in their ability to increase perceived rationality of decisions being made. Short-term benefits of performance management systems are manifest: performance metrics allow clarifying the organization's priorities and concentrating employee effort on those priorities. However, there is a flip side to such short-term benefits.
2.3. Unintended Consequences of Performance Management
Any performance assessment system has a number of unwanted effects that manifest themselves as undesired changes in agent behavior under the influence of the assessment system. One of such negative effects is called tunnel vision, i. e. concentration on aspects that are subject to formalized assessment to the exclusion of other important areas [Smith 1995]. In case of faculty members, selective attention to publication activities is accompanied by selective inattention to other important aspects, teaching in the first place [Taylor 2003].
Peter Smith [Smith 1995] identifies eight unintended consequences of performance management (Table 2), which are based on the assumption that the controlled agents may predict principals' steps and change their behavior accordingly. Attempts to adjust the assessment
Table 2. Unintended Consequences of Performance Management (adapted from [Smith 1995]).
Unintended Consequence Description
Tunnel vision Concentration on areas included in the performance evaluation scheme to the exclusion of other important unmeasured areas
Myopia Concentration on short-term issues to the exclusion of long-term considerations which may only show up in performance measures in many years' time
Sub-optimization Pursuit of narrow local objectives at the expense of the objectives of the organization as a whole
Measure fixation Pursuit of success as measured rather than as intended
Misrepresentation Deliberate manipulation of data so that reported behavior differs from actual behavior
Misinterpretation Intentionally misleading inferences about performance on the part of the principal
Gaming Altering behavior in order to boost performance
Ossification Organizational paralysis due to an excessively rigid system of measurement
mechanisms result in new ways of evading control being invented and lead to an increase of control-associated costs.
Inadequate performance assessment systems may pervert the stimuli considerably, encouraging employees to maximize indicators at the expense of real output [Hood 2006; Smith 1995]. The impact of performance assessment systems on agent behavior depends on the structure of motivation lying at the heart of the management tools. The following section looks at the specific aspects of work motivation in the academic world.
3. The Psychology Over the past half-century, the focus in psychology of motivation of Researcher has shifted from conventional drivers and specific stimuli toward the Motivation broader phenomena of choice, autonomy, and experience of freedom, or lack of freedom, in doing an activity.
The philosophy of performance management described above, which is widely applied today, is based on the assumption that an elaborate system of monitoring would promote employee productivity. Unmonitored agents are supposed to be inefficient, so (research) performance management basically comes down to quantitative control over explicit labor outcomes (publications). However, the contemporary psychology of motivation argues that such management sys-
tems will only be effective temporarily, up to a certain point, beyond which they stop increasing employee productivity.
3.1. Self-Determina- Self-determination is autonomy in initiation of behaviors and mak-tion Theory ing choices and decisions. Edward Deci and Richard Ryan have been working on Self-Determination Theory (SDT) for over three decades [Deci, Ryan 1985]. Self-determination means feeling independent from both external and internal powers [Leontiev 2000].
Taking Abraham Maslow's ideas further, the authors of SDT identify three basic needs that are innate and universal to all human beings. Autonomy is the need to have a choice, to feel oneself the author and master of one's own destiny. Competence is the ambition to achieve results and be efficient at work. Finally, psychological relat-edness is the need to have a sense of belonging and connectedness. Satisfaction of the basic needs is a factor of psychological health and wellbeing, while frustration of such needs, conversely, leads to lower efficiency and affects physical health and self-development in negative ways [Gordeeva 2010].
Special emphasis is laid on autonomy, otherwise referred to as the need for self-determination. Satisfaction of this need is thwarted when an individual realizes that their activity is controlled from the outside. In a performance management system, intrusive external control will have adverse effects on motivation.
3.2. Intrinsic and Depending on where the reward for activity comes from, motivation Extrinsic Motivation can be extrinsic or intrinsic. Extrinsic motivation occurs when an individual is motivated to gain an external reward, while intrinsic motivation implies that an activity has an inherent value [Kallio, Kallio 2014]. An intrinsically motivated person gains pleasure and satisfaction from actually engaging in activity. For example, games and creative activities are sources of satisfaction in themselves, so people engage in them for the sake of experience. According to Deci [Deci 1971], a person is intrinsically motivated if they perform an activity for no apparent reward except the activity itself. Robert Henri describes being intrinsically motivated as feeling part of something bigger than one's own existence [Henri 1923]. As for extrinsic motivation, it refers to the promise of reward for achieving a goal or the threat of being punished for failure to achieve it. In this regard, activity is perceived as a means of gaining that reward.
The impact of the two types of motivation on behavior can be described briefly using quotes by the proponents of SDT: "Intrinsic motivation energizes and sustains activities through the spontaneous satisfactions inherent in effective volitional action" [Deci, Koestner, Ryan 1999:658]. Conversely, "when people are rewarded for performing a task, they do the job to gain the reward, but nothing above what is expected. In other words, an extrinsically motivated person tends to minimize their effort and maximize the reward" [Deci, Ryan 1985:77].
So, what will be the consequences of using reward and punishment in measuring performance in academia? Performance metrics link research activities to extrinsic motives (success criteria, research evaluation, social comparison), ignoring the intrinsic ones (academic freedoms, personal research interests, new scientific discoveries). Performance assessment of this type is based on promoting extrinsic motivation in employees while suppressing natural, intrinsic motivation.
In the past, academic labor was driven and energized by contentment with research activity as such. However, since publication metrics were introduced, faculty has got hooked on research performance indicators. We leave for the moment the discussion of dubious publication performance criteria themselves1. What is important is that publication productivity has become overwhemlingly significant, not only having an impact on pay but often even being the decisive factor of holding down a job. Stated another way, academics have been put in a situation where their research activities are extrinsically motivated and strictly controlled.
Evidence from Deci's experiments and numerous other studies shows that extrinsic incentives and pressures tend to undermine intrinsic motivation [Deci 1971; 1975; Amabile 1997; Calder, Staw 1975; Deci, Koestner, Ryan 2001; Eisenberger, Rhoades, Cameron 1999; Hennessey, Amabile 1998; Ryan, Deci 1996; 2000a; 2000b; Weiner 1980].
Investigation of problems related to the undermining of intrinsic motivation by rewards and punishments has given rise to a number of different areas of research, one of them being Motivation Crowding Theory (MCT) [Frey, Jegen 2001]. MCT claims that tangible extrinsic rewards, such as financial incentives, may undermine intrinsic motivation, while informal rewards and verbal praise have a positive effect on it. This theory has been bolstered by an extensive review of studies [Deci, Koestner, Ryan 1999] and a recent meta-analysis of research findings [Cerasoli, Nicklin, Ford 2014]. Meanwhile, there is an alternative approach that discredits the very concept of intrinsic motivation [Cameron, Pierce 1994; Cameron, Banko, Pierce 2001].
Working on SDT in the Russian context, Tamara Gordeeva considers a particular, controlled type of extrinsic motivation that occurs as a result of external attempts to control labor activities [Gordeeva 2014; Osin, Ivanova, Gordeeva 2013]. Motivation of this type frustrates the needs for autonomy and respect since the purpose of activity is extrinsic to an individual, who does not perceive it as intended (e. g. universities need publication performance management to improve their
1 For a critical discussion of the indicators and methodology of assessing research performance of individual researchers and institutions, see: [Abra-mo 2017].
rankings and other indicators, whereas the purpose of research can hardly be boiled down to publication metrics for academics). By contrast, intrinsic motivation is related to higher levels of wellbeing, and intrinsically motivated people tend to keep doing the task once the rewards have been removed and seek to advance their skills [Gordeeva, Sychev, Osin 2013].
The influence of extrinsic and intrinsic motivation on behavior has also been studied by economists [Festre, Garrouste 2015]. The issue has become especially salient with the rise of behavioral economics, which drew economists' attention to a variety of phenomena that used to be the province of psychologists. Economists of the 1970s were already concerned about how financial incentives affected altruistic behavior. In particular, Richard Titmuss maintained that paying for blood donations could undermine donors' intrinsic prosocial motivation to donate blood [Titmuss 1970]. The subject sparked the interest of major economists [Arrow 1972; Solow 1971], which has been sustained up to the present day, so there have been a lot of economic studies focusing on the "crowding-out" of intrinsic motivation by extrinsic rewards [Benabou, Tirole 2003; 2006; Falk, Gachter, Kovacs 1999; Frey, Oberholzer-Gee 1997; Gneezy, Meier, Rey-Biel 2011; James Jr 2005; Kunz, Pfaff 2002; Romaniuc 2017]. Eventually, the notion of "hidden costs of reward" [Kunz, Pfaff 2002] was introduced, meaning that incentives designed to get agents interested in performing their task (in terms of the principal-agent problem) may involve additional "costs" by undermining their intrinsic motivation. However, providing a review of this substantial body of literature is beyond the scope of this paper.
3.3. Rewards and Deci and Ryan proposed a theoretical explanation of the observed inIncentives in the fluence of external stimuli on intrinsic motivation [Deci, Ryan 1985].
Context of SDT Rewards can be experienced as controllers of behavior or as affirmations of competence. In the former case, rewards are predicted to thwart satisfaction of the need for autonomy and thus undermine intrinsic motivation. In the latter case, however, where rewards are indicators of good performance, they are predicted to enhance intrinsic motivation. Therefore, the effects of extrinsic rewards depend on whether they are experienced primarily as controllers of behavior or primarily as affirmations of competence. This theoretical framework allows distinguishing among different types of rewards as a function of their effects on intrinsic motivation.
Ryan and his co-authors [Ryan, Mims, Koestner 1983] developed a typology of reward contingencies, which has become a standard and has been used in a number of empirical studies [Deci, Koestner, Ryan 1999]:
1) Task-noncontingent rewards, given without respect to the completion or quality of task activity, e. g. for participating in a survey;
2) Task-contingent rewards, given for engaging in or completing a task; these can be divided into
3) (a) engagement-contingent rewards; and
4) (b) completion-contingent rewards;
5) Performance-contingent rewards, given for a specified level of performance (relative to a set criterion).
Task-noncontingent rewards (1) do not affect the basic needs, so they should have no effect on intrinsic motivation. Engagement-contingent rewards (2a) are experienced as controllers of behavior but not as affirmations of competence: recipients engage in a task but do not receive affirmations of their effective performance. Such rewards thwart satisfaction of the needs for autonomy and competence and thus have a negative impact on intrinsic motivation. Completion-contingent rewards (2b) are experienced as even more controlling, as people have to complete the task to get the reward. Such rewards may also be secondarily perceived as indicators of competence in case the task requires skills and the individual wants to achieve good performance. To the extent that rewards do represent competence affirmation, such implicit positive feedback could offset some of the stronger control. Ryan and his co-authors [Ryan, Mims, Koestner 1983] claim that the effect of this type of rewards is comparable to that of the previous one. Finally, with performance-contingent rewards (3), there is even stronger control—people have to meet some standard in order to maximize their rewards. The need for autonomy is frustrated here more than with any other type of rewards. However, performance-contingent rewards can also convey substantial positive competence information, thus enhancing intrinsic motivation. As a result, the effect of such rewards will depend on whether they are experienced primarily as controllers of behavior or primarily as indicators of excellent performance.
Theoretically predicted effects of different types of rewards have been confirmed empirically, as shown in a meta-analysis of 128 studies [Deci, Koestner, Ryan 1999] looking at how different types of rewards affect intrinsic motivation.
A standard experimental measure of intrinsic motivation is the so-called free-choice measure [Wiechman, Gurland 2009]. After the main part of the experiment is over, participants are left to engage in any of several interesting tasks, including more of the target task, for a short period of time when they believe they are no longer being observed. A rewarded experimental group is compared to a no-reward control group, and undermining is in evidence if, on average, the rewarded group spends significantly less time than the control group engaging in the target activity during the free-choice period.
Meta-analysis reveals [Deci, Koestner, Ryan 1999] that engagement-contingent, completion-contingent and performance-contingent rewards significantly undermine free-choice intrinsic motivation
(d = -0.40,-0.36, and -0.28, respectively^, while positive verbal feedback has been found to increase intrinsic motivation (d = 0.33).
As put by Deci, "the rewards issue is merely a special case of a more general issue, namely the control versus self-determination of human behavior in social contexts" [Ibid.:658]. Apart from rewards, other factors such as deadlines [Amabile, Dejong, Lepper 1976] or externally mediated goal setting [Mossholder 1980] can also affect intrinsic motivation, as they represent external controllers of behavior. Satisfaction of the needs for autonomy and growth should be ensured to retain intrinsic motivation [Lawler, Hall 1970].
Analyzing how publication performance measurement systems affect researchers' self-perception of autonomy or subordination is a productive method of predicting researcher motivation. For example, different rewards and penalties may have different effects on the self-perception of autonomy or subordination depending on their mechanism, size and form. Apparently, intrinsic motivation of researchers can be increased by providing an organizational context conducive to satisfaction of their need for autonomy, and the reward scheme should be designed in a way to satisfy the need for competence. That is to say, rewards should be administered so that academics have an opportunity to earn peer and administrator approval as an affirmation of their competence, their freedoms remaining intact.
3.4. Effects of In addition to minimum required indicators, some publication perfor-Financial Incentives mance measurement systems provide financial incentives for achiev-on Intrinsic Motivation ing a certain "level of success". Empirical research has shown that the and Productivity effects of monetary rewards on intrinsic motivation are controversial and contingent on the subjective perceptions of such rewards.
An international comparison has not revealed any straightforward connection between financial incentives and the efficiency of university systems [Auranen, Nieminen 2010]. However, another study has shown that monetary incentives may undermine, and under different conditions strengthen, intrinsic motivation [Frey, Jegen 2001]. Any performance management system in the public sector that sets target criteria of success exhibits the so-called "threshold effect" [Smith 1995], when the effects of financial incentives on intrinsic motivation are positive for low performers (whose baseline performance was below the threshold) and negative for high performers (with baseline performance levels above the threshold). Agents who easily demonstrate above-average performance tend to minimize their effort to only achieve the target indicators when monetary rewards are introduced.
Beside that, individual perception of financial incentives plays a role. If they are perceived as controlling, they tend to decrease re-
2 d is the standardized mean difference, corrected for sample size. This is one of the statistical effect sizes proposed for meta-analyses [Hedges, Olkin 1985].
searchers' work effort; but if they are perceived as supportive, they actually increase the number of publications [Andersen, Pallesen 2008]. Even small monetary rewards used as indicators of approval and support can boost motivation if they work as a clear manifestation of recognition by the academic community. Of course, the size of incentives matters, but of no less importance is how exactly they are given, since monetary rewards have a high symbolic value [Jimenez-Contreras, de Moya Anegon, Lopez-Cozar 2003].
It is not only work productivity but also subjective wellbeing in the workplace that is driven by intrinsic motivation. Evidence of this has been demonstrated, in particular, by Russian researchers [Osin, Ivanova, Gordeeva 2013; Osin et al. 2015; 2017]. Findings show that satisfaction of the basic needs for autonomy, competence and related-ness affects subjective wellbeing of respondents. The researchers have proved the "positive role of intrinsic motivation that is related to positive employee perceptions of the organization and subjective well-being in the workplace" [Osin, Ivanova, Gordeeva 2013:23]. Meanwhile, the negative effects of financial incentives manifest themselves "in cases where extrinsic motivation prevails so that employees feel obligated to work and do it just for money" [Ibid.:24].
Merit pay may drive out informal types of recognition and thus negatively affect the motivation of people who do not regard money as the most important part of their job [Perry 1989]. A number of experiments have reported a negative impact of financial incentives on creativity. When using a monetary reward system, one should consider the risk of undermining intrinsic motivation, as orientation toward external compensation may be detrimental to the quality of work by impairing creativity [Amabile 1979; 1983].
Another threat presented by high-powered cash incentives is the risk of failure to maintain honesty and integrity in science. A large-scale bibliometric study demonstrates that countries where individual publication performance is rewarded with cash are more likely to produce retracted papers, which is interpreted as a predictor of scientific misconduct in pursuit of monetary rewards (unfortunately, Russia was not included in the study) [Fanelli, Costas, Lariviere 2015].
Therefore, analysis of the methods to increase research productivity in the context of SDT reveals a certain degree of risk of overrating the potential of payment-based measures. The use of strong monetary incentives to achieve short-term goals may degrade the quality of performance in the long run [Frey, Jegen 2001].
3.5. The Relationship Findings from theoretical studies and meta-analyses of empirical data between Personality justify the conclusion that extrinsic incentives can only increase the and Performance quantity of performance [Cerasoli, Nicklin, Ford 2014], while its quality is a function of intrinsic motivation [Cerasoli, Nicklin, Ford 2014; Deci, Koestner, Ryan 1999].
Research productivity is not characterized by motivation alone. Other, activity-related, characteristics include commitment, perseverance and self-discipline. Cognitive maturity is seen as a prerequisite for scientific success: a successful researcher must have, or at least seek to develop, a psychologically healthy personality. Abraham Maslow [1970] states that neurotics, who are dependent on other people's approval, absorbed by their own needs, fearful of the novel and unfamiliar, and avoiding personal growth, cannot be genuinely productive at work. Success in research is more likely to come to psychologically healthy people who can acknowledge and accept their weaknesses, compensate for their performance deficiencies, take risks, work committedly, and cooperate with colleagues. Employers can contribute to the quality of performance by taking care to improve social living conditions of their employees, enhance subjective well-being in the workplace, and raise pay to allow for relative freedom of creativity at work. Maslow also stresses the need to emancipate science and the learning process as such [Ibid.].
Current trends in motivation theory and recent empirical findings are casting doubt on efficiency of the existing academic governance policies, the methods of employee performance evaluation in particular. A fundamental revision is required in the light of new research findings and the challenges of today's world, considering that the type of motivation that prevails also affects the quality of performance. When researchers are guided by extrinsic incentives alone in planning their research activities, it will inevitably degrade the quality and creativity of research, the sense of purposefulness, and job satisfaction [Amabile 1979; 1982; Hennessey, Amabile 1998; Koestner et al. 1984].
4. Publication Performance as an Indicator of Research Quality
4.1. Publication Performance Measurement as a System of Pressure and Control
Performance management systems are primarily about exerting control and pressure over employees to improve their productivity. However, scientific performance is largely contingent on the thirst for new knowledge, creative self-expression, the need to serve something bigger than oneself—which is, again, intrinsic motivation. In many cases, close monitoring over creative activity may be detrimental to its quality.
A study devoted to the increased culture of evaluation in Australian universities is a powerful example. The active use of prescriptive management has resulted in a considerable increase in journal publication productivity (Australia's share of publications in the Science Citation Index (SCI) has increased by 25 percent in the past decade), yet there has been a significant decline in citation impact relative to other countries (Australia has dropped from sixth to 10th position in a ranking of 11 OECD countries). In other words, Australian researchers have increased their publication counts but have lost in the quality of research [Butler 2003]. These findings are consistent with the observations from psychological studies which demonstrate that per-
formance quality is affected by intrinsic motivation and performance quantity by extrinsic incentives [Cerasoli, Nicklin, Ford 2014]. However, quantity and quality are not always mutually exclusive as far as scientific papers are concerned. For instance, Abramo, D'Angelo, Di Costa [2010] use the example of Italian universities to show that papers of highly productive researchers are more likely to be of better quality.
According to a study conducted across several Finnish universities [Kallio, Kallio 2014], most researchers prefer assessing their work in qualitative rather than quantitative terms, whereas formalized performance management systems are mostly based on quantitative measures. Only 15 percent of the respondents were found to be satisfied with how performance management systems at their universities worked, and many perceived them as "meaningless" [Ibid.:579]. Over 70 percent of the participants agreed that the need to meet quantitative criteria in reporting impaired the quality of work [Ibid.:582]. Over 40 percent pointed out that quantitative performance indicators had negative effects on their motivation and infringed on their academic freedom [Ibid.:583]—that is, frustrated their basic need for autonomy. In this regard, grant systems appear to be more motivating as they imply free choice of teams, foundations and research problems, freedom in planning and scheduling work efforts, etc. Autonomous choice entails assumption of responsibility for further efforts and their efficiency. Performance under grant obligations is measured upon execution of a content-based plan. By and large, attempts to control researcher productivity result in the subject of research being left out of the focus and basically come down to consideration of formal measures.
On the other hand, even when positive extrinsic incentives (such as sizeable pay rise based on publication performance) are used to motivate scientists, such additional pressure may ultimately decrease productivity. According to the Yerkes-Dodson law, formulated early in the 20th century, the relationship between pressure and performance is parabolic: performance increases with growing pressure but only up to a point—and decreases when the level of pressure becomes too high (Fig. 1). So, medium (optimal) level motivation intensity is most favorable for the quality of work being performed [Yerk-es, Dodson 1908].
In some universities, failure to meet publication performance requirements may cost researchers their jobs. Jeannette Taylor and Ranald Taylor [Taylor, Taylor 2003] warn against exercising excessive pressure on academics in a bid to make them produce more publications, as effects may be reverse.
Being pressurized by controlling systems, researchers invent "survival strategies", preferring short-term studies, inflating the number of publications by slicing their findings "as thin as salami" [Lawrence 2003; Weingart 2005], and retreating into narrow subject areas [Ter Bogt, Scapens 2012]. As a respondent in the Finnish study said, "It is
Figure 1. The Relationship Between Pressure and Effort (Performance)
Source: [Taylor, Taylor, 2003].
Effort
B medium (optimal) level motivation
>- Pressure
0
P
P
P
only the number of publications that matters. That is why many have retired into their narrow fields of study. People have become more cynical" [Kallio, Kallio 2014:579].
Findings also show that excessive pressure results in some researchers starting to ignore all other aspects of their work (such as teaching and academic self-governance) and concentrate solely on publications [Ibid.:584]. The respondents in the same study believe that the reason for this is the degradation of working conditions at universities and, consequently, the growing risk of "free-rider problem", when academics disengage themselves from the university life. Besides, conditions for "organizational myopia" are engendering: researchers prefer focusing on the issues that allow them to quickly produce publications at the expense of the problems that are much more important. All of these negative trends manifest the universal challenges of performance management systems.
4.2. Publication Performance Measures as Indicators of Recognition
It is clear from what has been said above that publication performance measurement cannot be considered a perfect method of research quality assessment. Nevertheless, there is a positive aspect to it. It has been shown that when evaluation is perceived as recognition, it can actually have positive effects on motivation.
Dutch universities that introduced publication command systems saw an increase in the number of publications [Jacobsen, Andersen 2014]—but only as long as researchers perceived the new system as a means of support and approval, not as a controlling tool. An important contribution of this study is that it proves the importance of subjective perception of assessment systems over their objective stringency. The authors also point out that such subjective perceptions are largely shaped by middle managers who mediate the relations between facul-
ty and university administrators. It is through middle management that undesired effects of formal performance assessment can be reduced.
The fact that publication metrics may play an important symbolic role as indicators of academic recognition is also proved by findings from Australian universities [Taylor, Taylor 2003]. The significance of such recognition for research performance is easy to understand, peer praise being a powerful factor in the structure of researcher motivation [Baldwin, Krotseng 1985; Schuster 1985]. In addition, knowing how colleagues perform may create conditions for the spirit of competition, which is an informal source of additional motivation [Razina 2014]. The critical role of competition for the scientific ethos has been discussed by some classic sociologists of science [Hagstrom 1965; Merton 1973]. However, an overly competitive environment may be unfavorable for creativity [Amabile 1982]. Some relatively recent studies also demonstrate the potential harmfulness of excessive competition in science: "When competition is pervasive, such effects may jeopardize the progress, efficiency and integrity of science"3 [Anderson et al. 2007:437].
4.3. Lessons Learned from International Experience in Performance Management
Extensive international experience has been built up in using performance management systems in various domains including the public sector. Scientific managers in Russia could benefit from this rich experience.
The policies of Tony Blair's government represent a bright page in the history of using performance management to improve performance of public organizations. Michael Barber, a key figure of that government, ranks the methods of improving performance in the public sector by flexibility and complexity [Barber 2007]. Prescriptive management is ranked the lowest, being the optimal choice in case the goal is to "put things in order" and raise quality from bad to satisfactory. Further improvements, however, require different methods to be used, as creativity and enthusiasm cannot be forced. Barber cites Joel Klein, chancellor of the New York City School System: "You cannot mandate greatness; it has to be unleashed" [Ibid.:337].
The patterns described by Barber also apply to performance management in universities. Coercive methods and punishments can help increase publication productivity from low to satisfactory. Threat of penalties can make scientists produce the required minimum of papers. However, there is no way this policy could work to improve the quality of publications or inspire interest and enthusiasm in researchers. Quite the opposite, a system of punishments perceived as discouraging academic freedom may undermine intrinsic motivation for research and degrade the quality of publications (or result in sneaky publication practices emerging as "survival strategies").
3 We would like to thank the anonymous reviewer of Educational Studies for their reference to the article and translation of the quote.
The problem with prescriptive management is that it regards scientists as shirkers that should be persistently monitored and punished for lazy behavior. Control and penalties are fraught with considerable costs, both direct—related to performance assessment—and symbolic, when university and faculty cannot trust each other anymore.
Opportunities of prescriptive employee behavior management are limited. Agents endowed with wit and freedom of action will always find a way to circumvent the imposed rules, as the goals of organizations and employees rarely tend to be in complete congruence. Adam Smith already raised this problem in The Theory of Moral Sentiments [Smith 1997:230], discussing the possibility of rational government:
"The man of system is apt to be very wise in his own conceit <...> He seems to imagine that he can arrange the members of a great society as easily as a hand arranges the pieces on a chess-board! He forgets that the chessmen's only source of motion is what the hand impresses on them, whereas in the great chess-board of human society every single piece has its own private source of motion, quite different from anything that the legislature might choose to impress on it."
A critical disadvantage of command and control is that outcomes achieved are not sustainable. As soon as pressure is relieved, the system returns to its previous state. Cooperation4 in a scientific institution, as in any other type of organization, may be either prescribed from above or develop from below. Forced cooperation is unsustainable due to the fear of punishment, so it subsides as soon as the threat of punishment is removed. By contrast, cooperation initiated from below transpires to be sustainable and lasts irrespective of external forces, which has been observed by Elinor Ostrom, a Laureate of the Nobel Memorial Prizein Economic Sciences [Dietz, Ostrom, Stern 2003].
Command management works to the extent of securing a certain performance minimum, but it is useless for further development.
Scientists' attitudes towards research and publishing differ across universities and countries. Using the findings from a survey of faculty in several universities, Andrey Lovakov [2015:109] points out that "faculty members in Russia do not perceive publishing—hence, research activities as such—as an attractive part of their job and rather produce publications because they have to." Under such circumstances, control and command may increase publication performance of shirkers,
4 Cooperation being understood as joint efforts to achieve organizational goals shared by all employees. Work efforts of employees in an organization constitute cooperation, i. e. co-directional activity. Ostrom uses the term "cooperation" in its broad sense. In the context of this study, we are talking about scientific cooperation as a team of researchers created to work on a scientific problem.
and yet to a limited degree. Margarita Kurbatova and Elena Kagan [2016] conducted a survey of faculty in over 40 Russian universities to demonstrate that increased control over faculty performance and integration of performance management mechanisms lead to higher incidence of shirking patterns and other negative forms of opportunistic behavior.
Meanwhile, it should not be denied that prescriptive management can be fairly efficient in specific contexts. The international bibliomet-ric study mentioned above found no high rates of scientific misconduct (indicated by the number of retractions) in countries with strong pressures to publish [Fanelli, Costas, Lariviere 2015]. Unfortunately, again, Russia did not participate in the study. The authors conclude that "policies to reduce pressures to publish might be, as currently conceived, ineffective, whereas establishing policies and structures to handle allegations of scientific misconduct, promoting transparency and mutual criticism between colleagues, and bolstering training and mentoring of young researchers might best protect the integrity of future science"5 [Ibid.:14]. That is to say, pressure to publish may be a good policy in "immature" contexts.
In addition to the threat of punishment, monetary rewards can be used to motivate for research. However, stronger financial incentives are more likely to lead to unwanted behavioral change.
The advantage of financial incentives over prescriptive management in stimulating publication performance is that they do not impinge on the need for autonomy, while satisfying the need for competence. However, monetary rewards also have a number of drawbacks, such as high costs, the threshold effect [Smith 1995], unstable results, and substitution of extrinsic motivation for intrinsic.
The threshold effect consists in that agents do not always change their behaviors as desired when they have to achieve a certain target to gain a monetary reward. Low performers, indeed, try to improve their productivity and achieve the goal, but high performers are prompted to minimize their effort and settle for the common target indicators. This is typical of any system trying to motivate agents with extrinsic incentives.
As for the costs, using a system of high-powered incentives in the long run is very similar to developing a dependence on drugs. The costs of maintaining the level attained are getting higher and higher, while sources of self-sufficient development may never emerge. Productivity "boosted" that way may bounce back as soon as financial incentives are removed.
In a study of Russian universities, Yana Roshchina and Maria Yud-kevich [2009] demonstrated the limited potential of incentive contracting in stimulating academic research. They also underline the
5 We would like to thank the anonymous reviewer of Educational Studies for their reference to the article and translation of the quote.
importance of developing intra-university academic communities, encouraging horizontal researcher mobility and creating efficient scientific teams. Overdependence on quantitative bibliometric indicators could be offset by "parallel institutions of individual and collective reputations" [Yudkevich 2004:119].
In a review article on the position of Russian universities in global competition, Isak Froumin and Jamil Salmi [2007:37-38] accentuated the importance of "investing in the creation of an institutional academic environment that would be marked by academic freedom and intellectual resonance".
Notwithstanding all those calls to action from a decade ago, recent findings show that current university practices tend to "replace the incentives hinging on informal academic standards and reputation-based control mechanisms with the incentives dictated by a quasi-market trumped up by the government and by the respective mechanisms of external assessment and control" [Kurbatova, Kagan 2016:116]. Therefore, performance management mechanisms are continuing being actively propagated despite the repeated warnings about how dangerous their overuse can be.
5. Conclusion Substituting extrinsic motivation for intrinsic may deprive research activities of their primary motive. Papers published under pressure will probably meet the quantity performance requirements but their quality may turn out to be low.
Prevalence of extrinsic incentives over intrinsic motivation may result in researchers choosing subject areas that offer the probability of quick publishing at the expense of problems that they are really interested in. In that case, the quality of publications and the sense of pur-posefulness in research may decrease significantly.
In the context of Russia, it is also the ratio of fixed and variable pay that matters. In a number of Russian universities, the base salaries of faculty are much lower than the variable components. Incentives thus acquire a strong controlling effect, making researchers feel obliged to publish in order to avoid penalties.
However, financial incentives do not only motivate scientists by the size of cash reward—they also play an important symbolic role as affirmations of approval and performance recognition (thus satisfying the need for competence). Employee surveys in the public and nonprofit sectors demonstrate that if employees are largely driven by intrinsic motivation and interest in what they are doing, even comparatively low monetary incentives may have positive effects on their motivation, being perceived as tangible indicators of their professionalism and competence being recognized.
To prevent motivation form being undermined by financial incentives, the latter should be structured so as to avoid the controlling effect and maximize the effect of affirming competence. Perhaps, aca-
demics should be given more freedom in determining the structure of their tangible incentives, being allowed to choose from a broad range of possible rewards, with the focus on reputation effects of such rewards.
If the only thing that university administrators want is to increase the number of publications, they may trade off researchers' wellbeing and let them evaluate their scientific contributions by yearly publication counts, journal quartile scores, etc. If, however, a university's ambition is to join the international community and contribute to global scientific development, in addition to caring about the rankings it should also invest money and effort in researcher wellbeing, which implies creating conditions to foster intrinsic motivation.
Formalized assessment systems, quantitative target values and payment by results replace intrinsic motivation for research with extrinsic incentives and pressure. Findings [Frey 2002] indicate that working under the pressure of extrinsic stimuli may inhibit creativity and degrade performance quality.
The scientific value of publications is extremely difficult to assess for an external observer. One example that went down the annals of history of science is Ludwik Fleck and the camp laboratory that he led as a Nazi prisoner during World War II. Under the guise of important research, the scientists were involved in the sabotage activities that were never detected by the bureaucratic management [Grzybowski, Ciesielska 2014].
A similar situation may be observed at universities today when scientific administrators want to coerce faculty into productive research activities. Instead of improving the quality of publications, administrative pressure may ultimately result in piles of low-quality papers devoted to problems of low importance.
6. Recommenda- A number of manifestos, including two international initiatives [DORA tions 2012; Hicks et al. 2015], have been designed around the world to provide recommendations on reducing the use of bibliometric indicators in measuring research productivity6. In Russian science, the problem was raised in the open message Bibliometrics for the Good of Russian Science of the Publication Ethics Board of the Association of Science Editors and Publishers7. These documents contain a number of specific recommendations on how to optimize the use of bibliometric indicators.
Below, we are trying to complement those recommendations as applicable to performance management policies pursued by universities. An array of specific recommendations has been developed based
6 We would like to thank the anonymous reviewer of Educational Studies for their reference to the documents.
7 https://rasep.ru/sovet-po-etike/bibliometriya-vo-blago-rossijskoj-nauki
on the fundamental provisions of self-development theory, formulated by Gordeeva [2016] as follows:
"From the viewpoint of self-determination theory, it is recommended to support employees' need for autonomy by providing them with opportunity to show initiative, freedom of choice and action within a clearly defined area of responsibility, and non-controlling feedback that would allow them to have their say and feel themselves the source and subject—not object—of their own activity." [Gordeeva 2016:48]
The recommendations developed in this study can be grouped into three major categories: (i) minimizing the unintended consequences of performance management; (ii) development of the academic environment; and (iii) organizational policies.
6.1. Minimizing the Unintended Consequences of Performance Management
Measures to reduce the specific unintended consequences of performance management systems are proposed as part of the fine-tuning process that does not aim at disturbing the status quo in the distribution of power between the bureaucratic apparatus and accountable researchers. A set of typical measures was proposed by Peter Smith in the same article where he identified the eight major negative side effects [Smith 1995] (Table 3).
According to Smith, the first two strategies are, in fact, universal. Engaging employees in the development of measures and using such measures flexibly are useful strategies for minimizing any unwanted effect of performance management. In case of universities, scientists must be engaged in designing research assessment systems. This practice may promote the development of collegiate self-governance. With due regard to the high symbolic value of rewards and the effectiveness of healthy competition, the following recommendation can be made:
1. Delegate the development of complementary assessment systems to researchers.
The system is referred to as complementary since it is intended not to discredit the basic assessment model designed by the managers but rather to complement it. Such complementary assessment system could be decentralized and based on less "aggressive" rewards with high symbolic value instead of "high-powered" incentives and rigid criteria. Financial incentives of relatively small size, designed using a collegiate procedure, could offset and compensate for the indiscriminately imposed rigid indicators. A system like that could be more pliable and use the fourth strategy of reducing the unintended consequences, which is regular revisions in order to adjust to the changing needs and goals of employees. In addition, it could also include elements of the seventh strategy—that of introducing alterna-
Table 3. Strategies for Reducing the Unintended Consequences of Performance Management [Smith 1995].
Reduction Strategies Unintended Consequences
TV SO M MF MR MI G O
1. Engage employees in the development of measures + + + + + + + +
2. Maintain flexibility of use + + + + + + + +
3. Measure each and every goal + + + + +
4. Review the assessment system on a regular basis + + + + + +
5. Encourage long-term career trajectories + -
6. Use a small number of measures - + - -
7. Introduce independent alternative assessment criteria +
Note: "+"—the strategy helps reduce the effect; "-"—the strategy worsens the effect;
TV—tunnel vision; SO—sub-optimization; M—myopia; MF—measure fixation; MR—misrepresentation; MI—misinterpretation; G—gaming; O—ossification.
tive criteria—to provide independent qualitative assessment of research output and contributions in the organizational performance as a complement to quantitative bibliometric indicators. Complementary assessment system is consistent with the foundations of SDT, satisfying the basic psychological needs for autonomy, respect, recognition and competence (and relatedness, too—through the mechanism of collegiate discussion) [Gordeeva 2016:48]. On top of that, the feeling of being engaged in designing the assessment system and able to influence decision-making processes makes faculty members experience a deeper sense of freedom of choice, which not only drives their intrinsic motivation but also affects their wellbeing in a positive way.
The third strategy—measure each and every goal—seeks to prevent the priorities set by assessment criteria from being too narrow. Under the logic of performance management, the assessment system should embrace everything that its developers think matters for the organization. Meanwhile, faculty members often complain about the assessment system being publication-biased at the expense of teaching, self-governance and other activities. This strategy suggests complementing the assessment system with other aspects that are important for the university. However, mere extension of the list of obligatory indicators (like those measuring participation in teaching activities) probably makes no sense. Under the logic of SDT, such prescriptive measures only stiffen control, undermining intrinsic motivation. To sum up, the following recommendation can be made:
2. Include additional activities in the performance assessment system and allow researchers to choose configurations or sets of activities that are optimal for them.
In this case, the system of financial incentives could be diversified, and every researcher could choose a configuration to suit their specific needs. A system of this type is practiced in Great Britain, where academics are allowed to balance legitimately between publishing and teaching. A somewhat similar system can be observed at the Higher School of Economics, where faculty members can participate in the Best Teacher competition that implies monetary as well as symbolic rewards.
At least three free-choice mechanisms of varying flexibility and marketability can be envisaged in the development of an assessment system: (i) "tracks", which involve predetermined arrays of activities from which employees are free to choose; (ii) a merit point system, in which employees are allowed to create their personalized "menu" of activities with different point values, the lowest passing score being the predetermined component (not the "menu" composition); (iii) monetization, where the size of reward is expressed in cash instead of points, and employees make choices not from the required minimum perspective but on the basis of their individual perceptions of the cost-benefit ratio. For instance, faculty members who prefer teaching over publishing may focus on teaching activities in case they are satisfied with the income such activities can generate in the monetized assessment system.
The very opportunity of choosing a personalized configuration is expected to have positive effects on the sense of autonomy and intrinsic motivation in researchers. Measures of this kind ensure free choice and at the same time determine the area of responsibility for the choices made. Such expansion of leverage opportunities might decrease frustration and anxiety, leading to higher levels of subjective wellbeing. In Russia, these three free-choice mechanisms have been applied in various combinations, for instance, at the Higher School of Economics.
To overcome myopia—concentration on short-term issues to the exclusion of long-term considerations which may only show up in performance measures in many years' time— temporal dimension may be introduced into the logic of free choice of "configurations" or "tracks". Myopia as a scientific strategy stems from the need to comply with short-term requirements, often at the expense of long-term results. Not infrequently, the existing assessment systems (whether based on minimum required criteria or financial incentives) have short-term horizons, compliance to minimum requirements being verified every year or two and performance incentive plans being also short-term. Scientists could experience more autonomy if they were allowed to determine individual time horizons of assessment, so that incentives are not paid out, say, within a year but are rather administered in smaller
increments during a five-year period and rewards for different sub-periods can overlap. In that case, researchers will be able to choose between the high-risk short-term and low-risk long-term incentive payout strategies. It is easy to imagine a situation where a scientist willing to work on a new topic or conceiving a book will opt for lower but stable rewards to secure longer-term compliance to the publication performance requirements. The third technical recommendation can thus be formulated as follows:
3. Introduce a "constructor" of incentive pay and rewards to allow scientists to customize the incentive schemes and schedules to suit their individual research trajectories.
Support for autonomy not only improves intrinsic motivation in this case but also expands employees' future time perspective (ability to consider a specific range of opportunities in long-term goal planning), which is a powerful predictor of life satisfaction [Zimbardo, Boyd 2010]. Putting the choice of "configurations" and "tracks" into the temporal perspective is also a way to implement the fifth strategy of dealing with the unintended consequences of performance management—encourage long-term career trajectories—because it allows for more flexible long-term planning.
The fourth strategy of using a small number of measures should be born in mind by anyone developing a performance assessment system. Uncontrolled proliferation of measures should be prevented to avoid priority dilution and excessive pressure.
Academic environments differ across universities, university departments and even research teams. If we want to increase intrinsic motivation for research, effort should be invested in fostering particular aspects of the academic environment: collegiate academic self-governance to satisfy the faculty's need for autonomy, peer recognition mechanisms (not only (and not so much) institutional but also (and rather) within research teams) to satisfy the need for competence, and conditions for active communication and socialization of young researchers in particular—which are vital for developing a sense of relatedness.
A question can be asked, rephrasing John F. Kennedy's words: "What the university and scientists can do for each other?" The governance model where academic shirkers are coerced by bureaucratic managers breeds antagonism that inhibits productive scientific development. Administrators of contemporary Russian universities should probably shift their focus from requirements, criteria, standards and assessment to engagement, delegation, customization, dialogue and support. This, however, requires changing the way we think, since the model where lazy agents have to be closely monitored to avoid shirking has been deeply ingrained in the governance practices of Russian
6.2. Development of the Academic Environment
universities. In the parlance of organizational theory, understanding of motivation should move from Theory X to Theory Y [MacGregor 1960].
Mechanisms to achieve such transformation may include the following:
4. Introduce feedback tools (employee surveys and interviews) into governance practices. Such surveys should not only assess employee satisfaction with different "tangible" aspects but also evaluate subjective wellbeing and measure the levels of motivation of different types.
Movement from punitive to supportive management could be started by measuring researcher wellbeing and making sure that those measures are considered by managers. A number of such measures have been developed by Russian researchers [Gordeeva, Sychev, Osin 2013; Osin, Leontiev 2008; Osin, Ivanova, Gordeeva 2013; Osin et al. 2015; 2017]. Under the logic of performance management, bureaucratic governance systems pursue the "what gets measured gets done" principle [Behn 2003; Wilson, Croxson, Atkinson 2006]. Only those aspects of organizational life are visible for the performance management system that are explicitly measured and evaluated. If wellbeing indicators begin to be used in intra-organiza-tional monitoring, they can be expected to be taken into account when making monitoring-based decisions, providing an opportunity to invest and assess investments in wellbeing and re-channel the potential of performance management systems to increase intrinsic motivation of researchers.
6.3. Organizational Balance of fixed and variable pay is central to the organizational poli-Policies cies of universities. Russian universities are characterized by low fixed and high variable salary components. Under such circumstances, incentives for publication activities tend to have a strong controlling effect. Researchers have to publish yearly, as they often cannot afford being left without incentive payouts. In the perspective of self-determination theory, creation of such conditions for employees may be regarded as undermining their autonomy. For example, scientists find themselves unable to take a break and devote time to delving into a new topic, writing a book or conducting a new study. As a result, they retreat into narrow subject areas and tend to produce more and more same-type publications. Scientific inquiry becomes too risky and expensive. Increasing the share of fixed pay could probably have positive effects on intrinsic motivation for research by reducing the need to work "just for money" and releasing creative effort. This recommendation, however, may interfere with the fundamental perceptions of the structure of researcher motivation and the balance of power in organizational policies, which vary across universities. Changing those perceptions is a challenging and conflict-laden process that necessitates active involvement of the stakeholders. Empirical substantiation of the motivating role of variable pay in academia requires further in-depth
research and cannot be reduced to discussing the relationship between variable pay and the number of publications (see, for instance, [Pfeffer, Langton 1993]). This study can only refer to the findings in the paradigm of SDT that have shown that publication performance is decreased when financial incentives are experienced as controlling and increased when they are perceived as supportive [Andersen, Pallesen 2008; Frey, Oberholzer-Gee 1997].
References Abramo D. (2017) Bibliometricheskaya otsenka rezultativnosti nauchno-issledo-vatel'skoy raboty: k chemu my prishli? [Bibliometric Evaluation of Research Performance: Where Do We Stand?]. Voprosy obrazovaniya / Educational Studies Moscow, no 1, pp. 112-127. DOI: 10.17323/1814-9545-2017-1112-127
Abramo G., D'Angelo C.A., Di Costa F. (2010) Testing the Trade-off between Productivity and Quality in Research Activities. Journal of the American Society for Information Science and Technology, vol. 61, no 1, pp. 132-140. doi:10.1002/asi.21254 Amabile T. M. (1982) Children's Artistic Creativity—Detrimental Effects of Competition in a Field Setting. Personality and Social Psychology Bulletin, vol. 8, no 3, pp. 573-578. doi:10.1177/0146167282083027 Amabile T. M. (1979) Effects of External Evaluation on Artistic Creativity. Journal of Personality and Social Psychology, vol. 37, no 2, pp. 221-233. doi:10.1037//0022-3514.37.2.221 Amabile T. M. (1997) Motivating Creativity in Organizations: On Doing What You Love and Loving What You Do. California Management Review, vol. 40, no 1, pp. 39-58.
Amabile T. M. (1983) The Social Psychology of Creativity: A Componential Conceptualization. Journal of Personality and Social Psychology, vol. 45, no 2, pp. 357-376. doi:10.1037//0022-3514.45.2.357 Amabile T. M., Dejong W., Lepper M. R. (1976) Effects of Externally Imposed Deadlines on Subsequent Intrinsic Motivation. Journal of Personality and Social Psychology, vol. 34, no 1, pp. 92-98. doi:10.1037//0022-3514.34.1.92 Andersen L. B., Pallesen T. (2008) "Not Just for the Money"? How Financial Incentives Affect the Number of Publications at Danish Research Institutions. International Public Management Journal, vol. 11, no 1, pp. 28-47. doi:10.1080/10967490801887889 Anderson M. S., Ronning E. A., De Vries R., Martinson B. C. (2007) The Perverse Effects of Competition on Scientists' Work and Relationships. Science and Engineering Ethics, vol. 13, no 4, pp. 437-461. Arrow K. J. (1972) Gifts and Exchanges. Philosophy & Public Affairs, vol. 1, no 4, pp. 343-362.
Auranen O., Nieminen M. (2010) University Research Funding and Publication Performance—An International Comparison. Research Policy, vol. 39, no 6, pp. 822-834. doi:10.1016/j.respol.2010.03.003 Baldwin R. G., Krotseng M. V. (1985) Incentives in the Academy: Issues and Options. New Directions for Higher Education, vol. 1985, no 51, pp. 5-20. doi:10.1002/he.36919855103 Barber M. (2007) Instruction to Deliver: Tony Blair, Public Services and the Challenge of Achieving Targets. London: Politico's. Barry J., Chandler J., Clark H. (2001) Between the Ivory Tower and the Academic Assembly Line. Journal of Management Studies, vol. 38, no 1, pp. 87-101. Behn R. D. (2003) Why Measure Performance? Different Purposes Require Different Measures. Public Administration Review, vol. 63, no 5, pp. 586-606.
Benabou R., Tirole J. (2003) Intrinsic and Extrinsic Motivation. Review of Economic Studies, vol. 70, no 3, pp. 489-520. doi:10.1111/1467-937x.00253 Benabou R., Tirole J. (2006) Incentives and Prosocial Behavior. American Economic Review, vol. 96, no 5, pp. 1652-1678. doi:10.1257/aer.96.5.1652 Butler L. (2003) Explaining Australia's Increased Share of ISI Publications—the Effects of a Funding Formula Based on Publication Counts. Research Policy, vol. 32, no 1, pp. 143-155. doi:10.1016/s0048-7333(02)00007-0 Calder B. J., Staw B. M. (1975) Self-Perception of Intrinsic and Extrinsic Motivation. Journal of Personality and Social Psychology, vol. 31, no 4, pp. 599-605. Cameron J., Banko K. M., Pierce W. D. (2001) Pervasive Negative Effects of Rewards on Intrinsic Motivation: The Myth Continues. The Behavior Analyst, vol. 24, no 1, pp. 1-44. Cameron J., Pierce W. D. (1994) Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis. Review of Educational Research, vol. 64, no 3, pp. 363-423. doi:10.3102/00346543064003363 Cerasoli C. P., Nicklin J. M., Ford M. T. (2014) Intrinsic Motivation and Extrinsic Incentives Jointly Predict Performance: A 40-Year Meta-Analysis. Psychological Bulletin, vol. 140, no 4, pp. 980-1008. doi:10.1037/a0035661 Deci E. L. (1971) Effects of Externally Mediated Rewards on Intrinsic Motivation. Journal of Personality and Social Psychology, vol. 18, no 1, pp. 105-115. doi:10.1037/h0030644 Deci E. L. (1975) Intrinsic Motivation. New York: Plenum Press. Deci E. L., Koestner R., Ryan R. M. (1999) A Meta-Analytic Review of Experiments Examining the Effects of Extrinsic Rewards on Intrinsic Motivation. Psychological Bulletin, vol. 125, no 6, pp. 627-668. doi:10.1037/0033-2909.125.6.627
Deci E. L., Koestner R., Ryan R. M. (2001) Extrinsic Rewards and Intrinsic Motivation in Education: Reconsidered Once Again. Review of Educational Research, vol. 71, no 1, pp. 1-27. doi:10.3102/00346543071001001 Deci E. L., Ryan R. M. (1985) Intrinsic Motivation and Self-Determination in Human Behavior. New York: Plenum Press. Dietz T., Ostrom E., Stern P. C. (2003) The Struggle to Govern the Commons.
Science, vol. 302, no 5652, pp. 1907-1912. doi:10.1126/science.1091015 Dooren W. v., Bouckaert G., Halligan J. (2010) Performance Management in the
Public Sector. London: Routledge. DORA (2012) San Francisco Declaration on Research Assessment. Available at:
https://sfdora.org/read/ (accessed 15 January 2019). Drucker P. F. (1954) The Practice of Management: A Study of the Most Important
Function in America Society. New York: Harper & Row. Drucker P. F. (1964) Managing for Results: Economic Tasks and Risk-Taking Decisions. Oxford: Heinemann. Eisenberger R., Rhoades L., Cameron J. (1999) Does Pay for Performance Increase or Decrease Perceived Self-Determination and Intrinsic Motivation? Journal of Personality and Social Psychology, vol. 77, no 5, pp. 1026-1040. doi:10.1037/0022-3514.77.5.1026 Falk A., Gachter S., Kovacs J. (1999) Intrinsic Motivation and Extrinsic Incentives in a Repeated Game with Incomplete Contracts. Journal of Economic Psychology, vol. 20, no 3, pp. 251-284. doi:10.1016/s0167-4870(99)00009-4 Fanelli D., Costas R., Lariviere V. (2015) Misconduct Policies, Academic Culture and Career Stage, Not Gender or Pressures to Publish, Affect Scientific Integrity. Plos One, vol. 10, no 6, pp. e0127556. Festre A., Garrouste P. (2015) Theory and Evidence in Psychology and Economics about Motivation Crowding Out: A Possible Convergence? Journal of Economic Surveys, vol. 29, no 2, pp. 339-356. doi:10.1111/joes.12059
Frey B. S. (2002) Creativity, Government and the Arts. De Economist. Vol. 150. No 4. P. 363-376.
Frey B. S., Jegen R. (2001) Motivation Crowding Theory. Journal of Economic Surveys, vol. 15, no 5, pp. 589-611. doi:10.1111/1467-6419.00150
Frey B. S., Oberholzer-Gee F. (1997) The Cost of Price Incentives: An Empirical Analysis of Motivation Crowding-Out. American Economic Review, vol. 87, no 4, pp. 746-755.
Froumin I., Salmi J. (2007) Rossiyskie vuzy v konkurentsii universitetov mirovo-go klassa [Russian Colleges in the Competition of Leading World Universities]. Voprosy obrazovaniya / Educational Studies Moscow, no 3, pp. 5-45.
Gneezy U., Meier S., Rey-Biel P. (2011) When and Why Incentives (Don't) Work to Modify Behavior. Journal of Economic Perspectives, vol. 25, no 4, pp. 191209. doi:10.1257/jep.25.4.191
Grzybowski A., Ciesielska M. (2014) Lesser Known Aspects of Ludwik Fleck's (1896-1961) Heroic Life during World War II. Journal of Medical Biography, vol. 24, no 3, pp. 402-408.
Gordeeva T. (2014) Bazovye tipy motivatsii deyatel'nosti: potrebnostnaya model' [The Fundamental Types of Work Motivation]. MSU Vestnik, Series 14. Psychology, no 3, pp. 63-78.
Gordeyeva T. (2016) Motivatsiya: novye podkhody, diagnostika, prakticheskie rekomendatsii [Motivation: New Theoretical Approaches, Diagnostics and Practical Recommendations]. Siberian Journal of Psychology, no 62, pp. 3853.
Gordeyeva T. (2010) Teoriya samodeterminatsii: nastoyashchee i budushchee. Chast 1. Problemy razvitiya teorii [Self-Determination Theory: Present and Future. Part 1: The Problems of Theory Development]. Psychological Studies, no 4. Available at: http://psystudy.ru (accessed 10 January 2019).
Gordeyeva O., Sychyov O., Osin E. (2013) Vnutrennyaya i vneshnyaya ucheb-naya motivatsiya studentov: ikh istochniki i vliyanie na psikhologicheskoe blagopoluchie [Inner and Outer Motivation in Students: Its Sources and Influence on Psychological Well-Being]. Voprosy Psychologii, no 1, pp. 35-45.
Hagstrom W. O. (1965) The Scientific Community. New York: Basic Books.
Hedges L. V., Olkin I. (1985) Statistical Methods for Meta-Analysis. San Diego, CA: Academic Press.
Hennessey B. A., Amabile T. M. (1998) Reward, Intrinsic Motivation, and Creativity. American Psychologist, vol. 53, no 6, pp. 674-675. doi:10.1037//0003-066x.53.6.674
Henri R. (1923) The Art Spirit. Compiled by Margery Ryerson. Philadelphia: JB Lippincott.
Hicks D., Wouters P., Waltman L., Rijcke S. D., Rafols I. (2015) Bibliometrics: The Leiden Manifesto for Research Metrics. Nature, vol. 520, iss.7548. P. 429. http://dx.doi.org/10.1038/520429a
Hood C. (2006) Gaming in Targetworld: The Targets Approach to Managing British Public Services. Public Administration Review, vol. 66, no 4, pp. 515-521.
Jacobsen C. B., Andersen L. B. (2014) Performance Management for Academic Researchers: How Publication Command Systems Affect Individual Behavior. Review of Public Personnel Administration, vol. 34, no 2, pp. 84-107. doi:10.1177/0734371x13510850
James H. S., Jr. (2005) Why Did You Do That? An Economic Examination of the Effect of Extrinsic Compensation on Intrinsic Motivation and Performance. Journal of Economic Psychology, vol. 26, no 4, pp. 549-566.
Jimenez-Contreras E., de Moya Anegön F., Löpez-Cözar E.D. (2003) The Evolution of Research Activity in Spain: The Impact of the National Commission for the Evaluation of Research Activity (CNEAI). Research Policy, vol. 32, no 1, pp. 123-142.
Kalgin A. (2012) Upravlenie po rezultatam na regionalnom urovne: kontrol ili re-zultativnost? [Performance Management at the Regional Level: Control and Efficiency]. Public Administration Issues, no 3, pp. 35-60. Kallio K. M., Kallio T. J. (2014) Management-by-Results and Performance Measurement in Universities—Implications for Work Motivation. Studies in Higher Education, vol. 39, no 4, pp. 574-589. doi:10.1080/03075079.2012.709497 Koestner R., Ryan R. M., Bernieri F., Holt K. (1984) Setting Limits on Children's Behavior: The Differential Effects of Controlling vs. Informational Styles on Intrinsic Motivation and Creativity. Journal of Personality, vol. 52, no 3, pp. 233-248. doi:10.1111/j.1467-6494.1984.tb00879.x Kunz A. H., Pfaff D. (2002) Agency Theory, Performance Evaluation, and the Hypothetical Construct of Intrinsic Motivation. Accounting Organizations and Society, vol. 27, no 3, pp. 275-295. doi:10.1016/s0361-3682(01)00031-9 Kurakin D., Filippov A. (2006) Vozmozhnost' korporatsii: k sotsiologicheskomu opisaniyu universiteta [The Possibility of Corporation: Towards a Sociological Description of University]. Neprikosnovenny zapas, nos. 4-5. Available at: http://magazmes.russ.ni/nz/2006/23.html (accessed 10 January 2019). Kurbatova M., Kagan E. (2016) Opportunizm prepodavateley vuzov kak sposob prisposobleniya k usileniyu vneshnego kontrolya deyatelnosti [Opportunism of University Lecturers as a Way to Adaptate the Externfl Control Activities Strenghtening]. Journal of Institutional Studies, vol. 8, no 3, pp. 116-136. Lawler E. E., Hall D. T.(1970) Relationship of Job Characteristics to Job Involvement, Satisfaction, and Intrinsic Motivation. Journal of Applied Psychology, vol. 54, no 4, pp. 305-312. Lawrence P. A. (2003) The Politics of Publication. Nature, vol. 422, iss. 6929, pp. 259-261.
Leontiev D. (2000) Psikhologiya svobody [The Psychology of Freedom]. Psikho-
logicheskiyzhurnal, vol. 21, no 1, pp. 15-26. Lovakov A. (2015) Priverzhennost vuzu i priverzhennost professii u prepodavateley rossiyskikh vuzov [Commitment of Russian University Teachers To University and Profession]. Voprosy obrazovaniya / Educational Studies Moscow, no 2, pp. 109-128. DOI: 10.17323/1814-9545-2015-2-109-128 McGregor D. (1960) The Human Side of Enterprise. New York: McGraw-Hill. Maslow A. H. (1970) Motivation and Personality. New York: Harper & Row. Melo A. I., Sarrico C. S., Radnor Z. (2010) The Influence of Performance Management Systems on Key Actors in Universities. Public Management Review, vol. 12, no 2, pp. 233-254. doi:10.1080/14719031003616479 Merton R. K. (1973) The Sociology of Science: Theoretical and Empirical Investigations. Chicago: University of Chicago. Mossholder K. W. (1980) Effects of Externally Mediated Goal Setting on Intrinsic Motivation: A Laboratory Experiment. Journal of Applied Psychology, vol. 65, no 2, pp. 202-210. doi:10.1037/0021-9010.65.2.202 Osin E., Gorbunova A., Gordeeva T., Ivanova T., Kosheleva N., Ovchinnikova E. (2017) Professionalnaya motivatsiya sotrudnikov rossiyskikh predpriyat-iy: diagnostika i svyazi s blagopoluchiem i uspeshnostyu deyatelnosti [Professional Motivation of Russian Employees: Assessment and Associations with Well-Being and Performance]. Organizational Psychology, vol. 7, no 4, pp. 21-49.
Osin E., Ivanova T., Gordeeva T. (2013) Avtonomnaya i kontroliruemaya professionalnaya motivatsiya kak prediktory subyektivnogo blagopoluchiya u sotrudnikov rossiyskikh organizatsiy [Autonomous and Controlled Professional Motivation Predict Subjective Well-Being in Russian Employees]. Organizational Psychology, vol. 3, no 1, pp. 8-29. Osin Y., Leontiev D. (2008) Aprobatsiya russkoyazychnykh versiy dvukh shkal ekspress-otsenki subyektivnogo blagopoluchiya [Testing the Russian-Lan-
guage Versions of Two Measures of Subjective Wellbeing]. Paper presented at The 3rd All-Russia Sociological Congress, Moscow: Institute of Sociology, Russian Academy of Sciences. Available at: http://www.academia. edu/23153063/Aprobatsiya russkoyazyichnyih versiy dvuh shkal eks-press-otsenki sub'ektivnogo blagopoluchiya (accessed 10 January 2019). Osin E., Suchkov D., Gordeeva T., Ivanova T. (2015) Udovletvorenie bazovykh psikhologicheskikh potrebnostey kak istochnik trudovoy motivatsii i sub-yektivnogo blagopoluchiya u rossiyskikh sotrudnikov [Basic Psychological Need Satisfaction as a Source of Work Motivation and Subjective Well-being in Russian Employees]. Psychology. Journal of the Higher School of Economics, vol. 12, no 4, pp. 103-121. Perry J. L. (1989) Making Policy by Trial and Error: Merit Pay in the Federal Service. Policy Studies Journal, vol. 17, no 2, pp. 389-405. Pfeffer J., Langton N. (1993) The Effect of Wage Dispersion on Satisfaction, Productivity, and Working Collaboratively: Evidence from College and University-Faculty. Administrative Science Quarterly, vol. 38, no 3, pp. 382-407. doi:10.2307/2393373 Razina T. (2014) Psikhologiya motivatsii nauchnoy deyatelnosti: metodologiya, teoriya, empiricheskie issledovaniya: monografiya [The Psychology of Motivation for Research: Methodology, Theory, Empirical Studies], Syktyvkar: Syktyvkar State University. Romaniuc R. (2017) Intrinsic Motivation in Economics: A History. Journal of Behavioral and Experimental Economics, vol. 67, pp. 56-64. doi:10.1016/j.so-cec.2016.10.002
Roshchina Y., Yudkevich M. (2009) Faktory issledovatelskoy deyatelnosti prep-odavateley vuzov: politika administratsii, kontraktnaya nepolnota ili vliyanie sredy? [Factors Effecting University Teachers' Research: Management Policies, Contract Incompleteness or Environment Infl uences?]. Voprosy obrazovaniya / Educational Studies Moscow, no 3, pp. 203-228. Ryan R. M., Deci E. L. (2000a) Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemporary Educational Psychology, vol. 25, no 1, pp. 54-67. doi:10.1006/ceps.1999.1020 Ryan R. M., Deci E. L. (2000b) Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being. American Psychologist, vol. 55, no 1, pp. 68-78. doi:10.1037//0003-066x.55.1.68 Ryan R. M., Deci E. L. (1996) When Paradigms Clash: Comments on Cameron and Pierce's Claim that Rewards Do Not Undermine Intrinsic Motivation. Review of Educational Research, vol. 66, no 1, pp. 33-38. doi:10.3102/00346543066001033 Ryan R. M., Mims V., Koestner R. (1983) Relation of Reward Contingency and Interpersonal Context to Intrinsic Motivation: A Review and Test Using Cognitive Evaluation Theory. Journal of Personality and Social Psychology, vol. 45, no 4, pp. 736-750. doi:10.1037/0022-3514.45.4.736 Schuster J. H. (1985) Faculty Vitality: Observations from the Field. New Directions for Higher Education, vol. 1985, no 51, pp. 21-32. doi:10.1002/ he.36919855104
Smith A. (1997) Teoriya nravstvennikh chuvstv [The Theory of Moral Sentiments].
Moscow: Respublika. Smith P. (1995) On the Unintended Consequences of Publishing Performance Data in the Public Sector. International Journal of Public Administration, vol. 18, no 2, pp. 277-310. Solow R. M. (1971) Blood and Thunder. Yale Law Journal, vol. 80, no 2, pp. 170183.
Sonin K. (2016) It Is Difficult for Doctoral Education to Survive in the Absence of an Academic Market. Higher Education in Russia and Beyond, no 3(9), pp. 6-7.
Sousa C. A.A., de Nijs W. F., Hendriks P. H.J. (2010) Secrets of the Beehive: Performance Management in University Research Organizations. Human Relations, vol. 63, no 9, pp. 1439-1460. doi:10.1177/0018726709357083
Taylor J. (2003) A Higher Education Perspective on Linking Performance Indicators to Extrinsic Rewards: An Empirical Analysis. Journal of Public Affairs Education, no 1, pp. 13-31.
Taylor J., Taylor R. (2003) Performance Indicators in Academia: An X-Efficiency Approach? Australian Journal of Public Administration, vol. 62, no 2, pp. 7182. doi:10.1111/1467-8497.00326
Ter Bogt H. J., Scapens R. W. (2012) Performance Management in Universities: Effects of the Transition to More Quantitative Measurement Systems. European Accounting Review, vol. 21, no 3, pp. 451-497. doi:10.1080/09638180 .2012.668323
Titmuss R. (1970) The Gift Relationship: From Human Blood to Social. London: Policy.
Tullock G. (2005) Bureaucracy. Indianapolis, Lancaster: Liberty Fund; Gazelle Drake Academic.
Weiner M. J. (1980) The Effect of Incentive and Control over Outcomes upon Intrinsic Motivation and Performance. The Journal of Social Psychology, vol. 112, no 2, pp. 247-254.
Weingart P. (2005) Impact of Bibliometrics upon the Science System: Inadvertent Consequences? Scientometrics, vol. 62, no 1, pp. 117-131.
Wiechman B. M., Gurland S. T. (2009) What Happens during the Free-Choice Period? Evidence of a Polarizing Effect of Extrinsic Rewards on Intrinsic Motivation. Journal of Research in Personality, vol. 43, no 4, pp. 716-719.
Willmott H. (1995) Managing the Academics: Commodification and Control in the Development of University-Education in the UK. Human Relations, vol. 48, no 9, pp. 993-1027. doi:10.1177/001872679504800902
Wilson D., Croxson B., Atkinson A. (2006) "What Gets Measured Gets Done": Headteachers' Responses to the English Secondary School Performance Management System. Policy Studies, vol. 27, no 2, pp. 153-171. doi:10.1080/01442870600637995
Wilson T. (1991) The Proletarianisation of Academic Labour. Industrial Relations Journal, vol. 22, no 4. P. 250-262. doi:10.1111/j.1468-2338.1991.tb00642.x
Yerkes R. M., Dodson J. D. (1908) The Relation of Strength of Stimulus to Rapidity of Habit Formation. Journal of Comparative Neurology, vol. 18, no 5, pp. 459-482.
Yudkevich M. (2004) Publikuy ili proigraesh [Publish or Perish]. Voprosy obra-zovaniya / Educational Studies Moscow, no 4, pp. 107-124.
Zimbardo F., Boyd J. (2010) Paradoks vremeni. Novaya psikhologiya vremeni, kotoraya uluchshit vashu zhizn [The Time Paradox. The New Psychology of Time that Will Change Your Life]. Saint Petersburg: Rech.
