CC BY-NC-ND
115
Why Embark on a PhD Today? A Typology of Motives for Doctoral Study in Russia
E. Terentev, N. Rybakov, B. Bednyi
Received in Evgeniy Terentev
September 2019 Candidate of Sciences in Sociology, Senior Researcher, Center of Sociology of Higher Education, Institute of Education, National Research University Higher School of Economics. Address: 20 Myasnitskaya St, 101000 Moscow, Russian Federation. Email: eterentev@ hse.ru
Nikolay Rybakov
Graduate Student, Department of University Management and Innovation in Education, National Research Lo-bachevsky State University of Nizhny Novgorod. Address: 23 Gagarina Ave, 603950 Nizhny Novgorod, Russian Federation. Email: rybakov-nv@phd.unn.ru Boris Bednyi
Doctor of Sciences in Physics, Professor, Director of the Institute of Doctoral Studies, National Research Lo-bachevsky State University of Nizhny Novgorod. Address: 23 Gagarina Ave, 603950 Nizhny Novgorod, Russian Federation. Email: bib@unn.ru Abstract. Data obtained in interviews with doctoral students and their aca-
demic supervisors as well as in doctoral student surveys conducted across six Russian universities is used to explore the motives for embarking on and pursuing a PhD and evaluate their incidence. Drawing on Deci and Ryan's self-determination theory, three basic types of motivation are identified — intrinsic motivation, extrinsic motivation, and amotivation — and described in the context of doctoral education. Even though academic labor has been losing its prestige in Russia, intrinsic motivation associated with interest in research, science and teaching remains the most popular motive for embarking on doctoral study. At the same time, a significant percentage of doctoral students are driven by external non-academic motives, such as specific social benefits or desire to use PhD as an asset in a non-academic career. Keywords: doctoral study, motivations of doctoral students, self-determination theory, effectiveness of doctoral programs, retention of youth in science.
DOI: 10.17323/1814-9545-2020-1-40-69
Doctoral education is undergoing a monumental transformation today, both in Russia and beyond. The prevailing trends include meeting the demands of an employment market wider than academia [Kehm 2006; Nerad 2010; Nerad, Evans 2014] and structuring the doctoral programs so that they develop generic skills as well [Gilbert et al. 2004; Park 2005; Kehm 2006; Halse 2007; Nerad 2010]. In Russia, those trends found their way into legislation in 2012, with the adoption of the Federal Law On Education in the Russian Federation, which not only changed the status of doctoral study (making it the third lev-
el of higher education) but also led to an essential reconsideration of the content of doctoral programs [Bednyi 2013; Mosicheva, Karavaeva, Petrov 2013; Shestak, Shestak 2015].
The reform stirred a heated debate in academia, its feasibility and implications being brought up for discussion. Critics have blamed it for the sharp decrease in effectiveness of doctoral programs over the past five years, emphasizing that the transition to the structured model of doctorate was unnecessary and "damaging" to the Russian science. Supporters, on the contrary, observe a "healing" effect of the reform, underlining its consistence with the global trends in doctoral education (for discussion of standpoints, see [Shestak, Shestak 2015; Sen-ashenko 2016; Bednyi 2017; Terentev, Bekova, Maloshonok 2018]). Obviously, the discourse lacks data, the majority of arguments being expert opinions with no empirical standing. For this reason, research on the development of doctoral education in Russia based on statistical and sociological evidence becomes ever more relevant today.
Motivation for embarking on and pursuing a PhD plays a key role in doctoral student success and thus requires the utmost attention [Lovitts 2001; Bair, Haworth 2004; Ivankova, Stick 2007; Spaulding, Rockinson-Szapkiw 2012; Litalien, Guay 2015]. A number of experts consider low motivation to be a major problem of doctoral education in Russia [Shafranov-Kutsev, Yefimova, Bulasheva 2017; Reznik, Cheme-zov 2018]. However, most of their arguments are based on pre-reform experience of a limited number of universities, which is inadequate for analyzing the current situation.
This study seeks to explore the actual motives of present-day doctoral students. Findings obtained in a mixed-methods study of six universities are used to identify the main types of doctoral motivation in Russia and evaluate their incidence. A typology of motives for embarking on and pursuing a PhD is built within the framework of self-determination theory proposed by American psychologists Edward Deci and Richard Ryan [Deci, Ryan 2012], which became widespread in student motivation research but has been rarely applied to doctoral education.
1. Research on the Motives of Doctoral Students in Russia and Beyond
There is empirical evidence that motivation for embarking on and pursuing a PhD is a strong predictor of successful degree completion and thesis defense [Lovitts 2001; Bair, Haworth 2004; Ivankova, Stick 2007; Spaulding, Rockinson-Szapkiw 2012; Litalien, Guay 2015; Sha-franov-Kutsev, Yefimova, Bulasheva 2017; Reznik, Chemezov 2018]. Intrinsic motivation, based on interest for learning and research, was found to be associated with better learning outcomes [Ivankova, Stick 2007; Litalien, Guay 2015]. David Litalien with colleagues observed positive correlations between doctoral students' outcomes and autonomy, the latter being understood in the context of SDT as intrinsic regulation, contrasted with external control [Litalien, Guay, Morin 2015].
Data obtained in interviews with doctoral graduates shows that candidates motivated both personally (achievement, personal goals, enjoying a challenge, and desiring the title) and professionally (career advancement, monetary incentives) are more likely to persist [Spaulding, Rockinson-Szapkiw 2012]. A survey of doctoral students at a South Korean research-focused university revealed that students with high aspirations for working in academia may perceive their learning process in a more positive way and cope better with stress associated with intensive coursework and individual research topic development, realizing that their outcomes will determine their career advancement [Shin et al. 2018].
International studies of motives for enrolling in doctoral programs are largely cross-sectional and focused on measuring the incidence of different types of motivation without looking at correlations with the learning outcomes and various aspects of perceived doctoral experience [Brailsford 2010; Tarvid 2014; Wiegerova 2016; London et al. 2014]. In addition, researchers draw on different taxonomies and use different theoretical frameworks, which makes systematical generalizations even more challenging.
There is little research on the motives for embarking on and pursuing a PhD in Russian universities, and the available findings are inconsistent. A pioneering study was conducted by Sergey Balabanov and his colleagues [Balabanov et al. 2003], who used the results of a survey of doctoral students in Volga Federal District to demonstrate that doctoral candidates are motivated most of all by the desire to defend a thesis and obtain a PhD degree, as well as by the opportunity to set their whole mind on thesis work. Professional research as a motive was mentioned by only one third of the respondents. The answers provided by doctoral students contradicted those obtained from faculty members, who nominated postponement of military service, self-fulfillment, desire to become a highly qualified professional, competitive advantage in the labor market, and prestige to be the most powerful motives. Type of motivation was one of the factors used by the authors to identify four categories of doctoral students: highly resourceful (with a strong propensity for research), deprived (in unfavorable socioeconomic situations), research-motivated (most likely to succeed in doctoral studies), and "deadweight" (those who only pretend working on a thesis). The findings of this study are useful for analyzing the evolution of motivations in PhD programs, as sociocultural characteristics of doctoral students in Russia have changed dramatically since 2002.
A recent study1 found the most popular motives for engaging in doctoral education to be associated with higher earning capacity, bet-
1 This study has a major limitation that renders its findings not entirely applicable to doctoral candidates, as the survey was targeted at undergraduate and graduate students, asking about their academic career intentions: "Which factors may have an impact on your decision to pursue a PhD?"
ter qualifications, career prestige or promotion, and making connections [Reznik, Chemezov 2018]. The identified pragmatic motives are classified as "erroneous", inconsistent with the primary mission of doctoral studies. "Objective" motives — associated with interest in doing research, teaching, and pursuing an individual scientific inquiry — are less widespread, which appears to undermine the sustainable development of doctoral education in Russia.
Partially contradictory findings have been obtained in some concurrent studies [Zamaraeva 2013; Shafranov-Kutsev, Yefimova, Bula-sheva 2017], which revealed that most doctoral candidates are motivated by interest in research and the opportunity for self-fulfillment and professional development. Extensive empirical data is provided on social characteristics of postgraduate students, yet the problem of motivation is glossed over without any theoretical conceptualization.
Attempts have been made in Russian literature to systematize the motives for embarking on and pursuing a PhD [Vedeneeva, Zabelina, Tsiring 2012; Kapshutar 2016; Sizykh 2014]. In particular, Yekaterina Vedeneeva and her co-authors identified three clusters of factors to describe the structure of doctoral students' motivations and values: "orientation toward achievement and self-fulfillment", "orientation toward status and comfort", and "orientation toward relationships". Marina Kapshutar suggests distinguishing between personal and social aspects of motivation, and Anastasia Sizykh explores the reasons for pursuing an academic career as a function of whether motivation comes from interest, social norm, or coercion.
A major large-scale study was carried out in 2016 on a sample of 14 universities with a special status (Project 5-100 participants and federal universities) [Bekova et al. 2017]. Having grouped the motives into academic and non-academic, the authors observe a noticeable prevalence of the former, associated with doing research at the university or a research institution (56%), teaching (48%), and doing analytical research for businesses (25%). Non-academic motives, though mentioned less often, carried considerable weight as well: 38% of the respondents believed that a PhD would help them build a career outside academia, 33% embarked on a doctoral journey for the sake of professional development, 23% were unwilling to leave academia, and 8% were interested in getting a room in the halls of residence. One in every four male PhD candidates treated doctoral study as a chance to postpone military service.
Russian studies thus show, on various arrays of data, that desire to defend a thesis and build an academic career as well interest in research and teaching are the key motives for embarking on and pursuing a Ph D. Meanwhile, non-academic motives associated with social
2 All the publications cited are based on either qualitative research methods [Sizykh 2014] or small-sample interviews [Vedeneeva, Zabelina, Tsiring 2012; Kapshutar 2016].
benefits (stipend, postponement of military service, accommodation, etc.), prestige, and possible financial gains have been found to be important for many doctoral candidates, despite their lower incidence. With some exceptions [Zamaraeva 2013; Bekova et al. 2017], all the studies mentioned above are case studies that use data obtained from individual universities or groups of universities located in the same city or region [Vedeneeva, Zabelina, Tsiring 2012; Kapshutar 2016; Ry-bakov 2018; Reznik, Chemezov 2018]. As most of them are based on interviews, interpretation and analysis of their results can be a challenge. Besides, studies exploring the motivation for pursuing a PhD rarely feature thorough theoretical analysis at the level of both methodology and interpretation, which makes it difficult to make comparisons and generalizations.
To approach the problem comprehensively, this article presents the results of an empirical mixed-methods study designed to construct a theoretically justified typology of the motives for embarking on and pursuing a PhD in Russian universities and to assess their incidence. The main theoretical framework of the study is built around self-determination theory proposed by Deci and Ryan [Deci, Ryan 19852012; Ryan, Deci 2000], which became widespread in school and higher education research [Gordeeva 2010; Maloshonok, Semenova, Terentev 2015] but has been rarely applied to doctoral education (exceptions include [Litalien, Guay 2015; Litalien, Guay, Morin 2015; Shin et al. 2018]). Available findings, however, demonstrate a high predictive capability of this theory in explaining doctoral students' dropout intentions [Litalien, Guay 2015] and satisfaction [Shin et al. 2018].
2. Deci and Ryan's Deci and Ryan's self-determination theory (SDT) of motivation posits Self-Determination three basic needs that determine human behavior: autonomy, compe-Theory tence, and relatedness [Deci, Ryan 2012]. Autonomy is based on voluntary choice, self-directed behavior, and self-control. Competence involves self-importance, self-efficacy, and enthusiasm about taking challenges and solving problems. Relatedness is understood as the need for feeling connected and accepted.
Various types of motivation regulating social behavior are identified based on this universal model. At the most basic level, distinction is made between intrinsic and extrinsic motivation. Intrinsic motivation is associated with inherent interest and enjoyment of an activity (self-regulation), whereas extrinsically motivated activities are done to obtain a reward, receive positive feedback, or avoid punishment (external regulation) [Ibid.]. Intrinsic and extrinsic types of motivation correspond to the opposite poles on the scale of autonomy.
To better understand the motivation process, Deci and Ryan also suggest distinguishing between integrated, identified, introjected, and external regulation within extrinsic motivation [Deci, Ryan 1985; 2012]. External regulation implies the lowest degree of autonomy and a com-
pletely external perceived locus of causality. With introjected regulation, behaviors are partially regulated by external norms and performed to avoid emotional discomfort (i. e. guilt) caused by failing to comply with them. Regulation through identification occurs when an individual consciously engages in an activity, understanding and accepting it as a means of achieving a personally important yet external (to the activity itself) goal. Finally, integrated regulation is the most autonomous form of extrinsic motivation, where the values determining involvement are fully assimilated to the self and brought into congruence with one's needs.
The five types of motivation identified (intrinsic and four types of extrinsic) form a continuum on the scale of autonomy, producing a distinction between autonomous vs. controlled motivation. The autonomous category includes intrinsic, identified, and integrated regulation, and the controlled one, introjected and extrinsic. Besides, Deci and Ryan identify amotivation, which has no regulation at all. Amoti-vated people go through the motions with no sense of intending to do what they are doing; they are not aware of their goals and do not seek to achieve any outcome with their actions. That is, their behavior is regulated neither extrinsically nor intrinsically. Amotivation results from feeling either that one is unable to achieve desired outcomes because of a lack of contingency, or a lack of perceived competence, or that one does not value the activity or the outcomes it would yield [Deci, Ryan 2002:17].
The model described above is regarded by the authors and their followers as universal, so it is important to assess its explanatory potential regarding specific cases and avenues of research. In this study, we are going to adapt Deci and Ryan's model to develop a typology of motives for enrolling in doctoral programs and assess its predictive capacity on a sample of PhD candidates in Russian universities.
3. Methodology The article is based on the results of a mixed-methods study carried and Data out in six Russian universities in 2018-2019. The sample included universities with a special status (three Project 5-100 participants) as well as regular ones (three universities). Two institutions were located in Moscow, and four in provinces (regions). Three were classical, one specialized in engineering and natural sciences, one in social sciences, and one in education and pedagogical sciences.
The qualitative portion of the study consisted in performing a series of semi-structured interviews with doctoral candidates (N=18) and their academic supervisors (N=24). In most cases, students and supervisors were interviewed in pairs independently of each other. Respondents were selected using the maximum variation sampling method. The structure of the sample is presented in Appendix 1. Interviews were focused on academic supervision, but some questions concerned the motives for embarking on a Ph D. In particular, aca-
demic supervisors were asked about the relatively low effectiveness of doctoral programs (high attrition rates, low thesis quality, etc.) and the distinctive features of successful and unsuccessful doctoral students.
Interview data was used to develop quantitative research instruments. In addition, using qualitative data as complementary to quantitative allowed getting a more comprehensive idea about motivation for enrolling in doctoral studies. The topic being highly sensitive, interviews with academic supervisors were of particular importance, as using candidates' self-reports alone would have implied a risk of getting a distorted picture due to social desirability bias. For instance, students may fail to mention some traditionally disapproved motives (social benefits, postponement of military service, etc.), so additional information from academic supervisors may be helpful for building a more nuanced picture.
Interviews with doctoral candidates and academic supervisors lasted 40-50 minutes; audio recording was performed in each case.
The quantitative part of the study involved an online survey of doctoral candidates, who were emailed a link to the questionnaire (participation was voluntary). The questionnaire was completed by 1,097 students, which accounts for approximately 35% of the doctoral students in the selected universities. The questionnaire, designed using EnjoySurvey3 software, included 72 questions and took about 20 minutes to complete. Some items concerned the motives for embarking on a doctoral journey—they were answered only by first-year doctoral students, who had fresher memories of making the relevant decision and choosing a specific program (N=347). The structure of the sample, broken down by mode of study, type of funding, student gender, and academic discipline is presented in Appendix 2.
4. A Typology of Motives for Embarking on a PhD and Assessment of Their Incidence
Analysis of interview transcripts allowed to identify a range of motives for enrolling in doctoral studies. Within the framework of SDT, all the motives were grouped depending on whether they were related to intrinsic motivation, extrinsic motivation, or amotivation. Within the group of extrinsic motives, the subtypes of external, introjected, identified, and integrated motivation could be distinguished. Below, we are going to dwell on each type of motivation and describe them using the interview and survey data.
4.1. Intrinsic Motiva- Intrinsic motivation [Deci, Ryan 1985] is characterized by the highest
tion: Interest in Research, Teaching, and Learning
level of autonomy and an internal perceived locus of causality associated with interest in and/or enjoyment of learning and work. Interviews revealed three groups of motives in the intrinsic motivation category related to different aspects of doctoral study: interest in research, interest in learning, and interest in teaching.
https://enjoysurvey.com/
3
4.1.1. Interest in A number of interviewees reported that their decision to embark on Research a PhD had mostly been motivated by interest in research as the core component of doctoral education. Research-motivated students regard doctoral study as inherently valuable, irrespective of the benefits it might bring (for their future career, for example). They are self-regulated and thus enjoy the highest degree of autonomy. Their narratives revolve around the concept of pure interest as a driver of their motivation:
"What were the motives? <...> my interest for inquiry and research." (doctoral candidate, male, 3rd year, chemistry)
"I actually like digging into science. I'm very interested, it's true." (doctoral candidate, female, 3rd year, jurisprudence)
Doctoral students with this type of motivation tend to mention not just an abstract interest in science but an intention to elaborate a specific research question and contribute to a particular scientific field as their motive. Entirely consumed by their topic, they truly enjoy doing research:
"I was genuinely interested in international relations, which I chose as my PhD program <...> That was when I chose economic integration to be my specific area of interest within international relations." (male, PhD degree awarded in 2011, international relations)
Some students are intrinsically driven by a desire to complete a previously initiated research project and present their findings to a broader audience, not just academic supervisor and reviewers. Eighty-seven percent of the doctoral students reported interest in and commitment to their research to be one of their motives for embarking on a PhD4.
Interview data shows that doctoral study often becomes some kind of a solace for candidates as they take comfort in doing what they truly like and enjoy. Many of the respondents contrasted doctoral studies with routine work outside academia (combining work with a PhD has been a widespread practice [Bekova et al. 2017]). Therefore, it is inherent interest that drives involvement in learning and individual research topic development.
"There is a permanent urge to develop, to write articles, to analyze what's going on, to do something, to dig deeper and expand your horizons. On the one hand, it's tough, but on the other, it gives you a new
4 Questionnaire item: "To what extent do you agree or disagree with the following statements concerning your reasons for doing a PhD? — I was interested in my research and wanted to take it further." Response options: "Strongly disagree", "Somewhat disagree", "Somewhat agree", and "Strongly agree".
flow in your life and gets you thinking." (doctoral candidate, male, 4th year, mathematics and mechanics)
Research-motivated PhD candidates approach doctoral education as a hobby and need no additional external stimuli to spend their time on learning and research, which they enjoy inherently. In their narratives, doctoral study is represented as a higher-order activity as compared to routine work:
"Well, work is work, but doing some thesis-related research in the evening or on weekends is more about enjoyment, like a hobby... I think it's pretty normal to want something more than just going to work and sitting there from 8 a. m. till 5 p. m. or from 9 a. m. till 6 p. m." (doctoral candidate, male, 4th year, mathematics and mechanics)
4.1.2. Interest in In some narratives, intrinsic motivation was manifested not in the in-Learning terest in research but in the interest in learning, "love for learning", or "self-improvement". A drive for development thus regulates internally the involvement in learning:
"It's that personal interest in self-improvement <...> Some kind of a spiritual need to move, to evolve, to be on the go." (academic supervisor, female, philology).
Some doctoral students also regard doctoral programs as an educational level, which contradicts the popular belief in academia that making PhD a stage of formal learning was a failure of education policy as there was no demand for the educational component among PhD candidates [Shestak, Shestak 2015]. According to survey data, nearly one third of the respondents (31%) embarked on a doctoral journey for the purpose of professional development5.
Since doctoral programs are less structured than Bachelor's and Master's degree programs, a higher degree of autonomy and independence is required for their successful completion [Litalien, Guay 2015]. Intrinsic motivation, which implies internal regulation and self-determined behavior, thus becomes a critical factor of degree persistence. This was emphasized by academic supervisors as they talked about the decisive role of being interested in and committed to research:
"I mean, it is vitally important for a candidate to be truly interested and deeply motivated to do that specific research." (academic supervisor, male, international relations)
5 Questionnaire item: "Why did you decide to embark on a PhD?" One of the response options: "I wanted to continue my professional development."
Out of 10 possible response options, respondents were allowed to choose as many as applied.
Self-determined and autonomous, this type of motivation is resistant to external factors that can have a negative impact on the probability of enrolling in a doctoral program. According to academic supervisors, learning-oriented motivation remains fairly widespread in spite of the decreasing prestige of academic labor and the relatively low level of faculty compensation typical of the current state of science in Russia:
"As paradoxical as it may seem, there are still people interested in research <...> You may not believe, but some people have a propensity toward scientific inquiry <...> Some really get a kick out of it. They like to live like that." (academic supervisor, male, chemistry)
4.1.3. Interest in Interest in teaching and desire to be an educator in the future repre-Teaching sent yet another type of intrinsic motivation, which was mentioned by both academic advisors and doctoral students. According to interview data, many PhD candidates have a passion for teaching, enjoy doing teaching internship projects, and willingly engage with students. For some of them, teaching at a university has been a dream:
"I realized that I loved teaching, I've been doing this for ten or eleven years now. As a freshman, I would tutor high school students, sophomores, juniors, and seniors <...> I've always enjoyed it, so I've realized that I find it interesting to teach." (doctoral candidate, male, 2nd year, economics)
"They [doctoral candidates] all dream of teaching and giving lectures <...> Well, they enjoy doing teaching internships, for example —where they can give lessons and deliver material." (academic supervisor, female, philology)
Survey results demonstrate that interest in teaching and desire to develop as an educator constitute a popular motive for embarking on a PhD (mentioned by 70%6 of the respondents), though less popular than interest in the development of research skills (91%7). This find-
6 Questionnaire item: "To what extent do you agree or disagree with the following statements concerning your reasons for doing a PhD?
— I decided to embark on a PhD to develop my teaching skills."
The following response options were available: "Strongly disagree", "Somewhat disagree", "Somewhat agree", and "Strongly agree". The indicator was estimated as a sum of percentages of the respondents who selected the options "Somewhat agree" and "Strongly agree".
7 Questionnaire item: "To what extent do you agree or disagree with the following statements concerning your reasons for doing a PhD?
— I decided to embark on a PhD to develop my research skills."
The following response options were available: "Strongly disagree", "Somewhat disagree", "Somewhat agree", and "Strongly agree". The indicator was estimated as a sum of percentages of the respondents who selected the options "Somewhat agree" and "Strongly agree".
ing contributes a lot to the debate on the goals and content of doctoral education [Gruzdev, Terentev 2017; Maloshonok, Terentev 2019]. For instance, some educators suggest that doctoral programs should be divided into university teacher training programs and researcher training programs, often questioning the very feasibility of cultivating teaching competencies in doctoral students [Maloshonok, Terentev 2019]. However, survey findings show that many candidates consider teacher training to be an important component of doctoral studies. Moreover, 65% of the respondents reported having been motivated by a desire to develop both research and teaching skills, so the idea of such tracking in doctoral education can hardly be deemed viable.
4.2. Extrinsic Motivation: PhD as an Important Step Towards Career Success, Professional Expertise, and Public Recognition
The group of motives that can be attributed to extrinsic motivation under Deci and Ryan's theory also features quiet prominently in the interview data. Extrinsically motivated candidates do not approach doctoral study as inherently valuable but rather as a means to achieve non-academic goals. This type of motivation is characterized by a lower degree of autonomy, as involvement is largely regulated externally. All the types of extrinsic motivation identified by Deci and Ryan — external, introjected, identified, and integrated — can be observed in the interview data. Let us now dwell on each type in more detail.
With external regulation, candidate engagement is determined by rewards that will be obtained upon degree completion. Such rewards may include, first of all, various social benefits of doctoral education — postponement of military service and. The opportunity to postpone military service was reported as a motive for embarking on a PhD by 29% of male doctoral candidates, and 7% of the respondents referred to a desire to get a room in the halls of residence. The high incidence of such motives indicates imperfection of the current PhD admissions system and serves as an important argument in the debate on modernizing the PhD selection process, which is blamed for the low quality of candidates admitted [Terentev, Bekova, Maloshonok 2018].
Another salient reward is the doctorate degree as a qualification that opens doors to a variety of career prospects. In this case, candidates are driven by their desire to get a prestigious and high-paying job — not by inherent interest in research, teaching, or learning. Such behavior is nonself-determined and characterized by the lowest degree of autonomy, according to Deci and Ryan's theory. The respondents pointed out that it was often not only the degree as such but also the awarding university that mattered. University name thus becomes a brand that has a certain prestige behind it and functions as a signal in the employment market:
"I wanted to complete a program at a prominent university, you know. In the end, the name of the university you graduate from also plays a role in the job market." (male, degree awarded in 2011, political science)
Some interviewees also underlined that a PhD may sometimes be wanted merely as a prefix, regardless of the program, research, and thesis content. The same type of motivation is observed in Bachelor's and Master's degree programs, where obtaining a diploma is the only factor regulating involvement in learning [Maloshonok, Semenova, Terentev 2015].
"I just needed to defend a thesis and get a degree. It's something like, you don't have to be a scientist but holding a PhD is a must." (female, degree awarded in 2017, political science)
Another type of extrinsic motivation featured in interviews with doctoral candidates and their academic supervisors may be assigned to the category of introjected motivation, where behavior is regulated by socially accepted norms, rules, values, and attitudes. The interviewees admitted that prestige and value of research and teaching still remained powerful factors driving youth into doctoral studies even if this choice contradicted candidates' salary aspirations. Besides, a doctoral degree in itself is an indicator of social status, even in the context of major shifts in the academic profession that are often criticized (see, for instance, [Senashenko 2017]).
"So far, the motives are ... some kind of prestige — prestige of teaching, of doing research maybe <...> I guess it's been a generation that shared those values—maybe the value of a PhD, of working at a university was transmitted by families. But now, it's fading away." (academic supervisor, male, sociology)
Prestige of holding a PhD is a significant factor affecting the decision to embark on a doctoral journey, almost 78% of the respondents having "strongly" or "somewhat" agreed that prestige had been among their motives.
Apart from doctoral degree diploma, competencies obtained in a doctoral program can also serve as an external regulator. By contrast with professional development — associated with "self-improvement" and "personal demand for development" (intrinsic motivation)—com-petency-oriented motivation is pragmatic, aimed at enhancing one's competitive edge in the labor market and expanding employment opportunities. In Deci and Ryan's theory, this would be identified regulation: embarking on a PhD and engaging actively in the learning process is perceived as a personally important step towards achieving an extrinsic, career-related goal. The degree of autonomy is higher here than with externally motivated behavior, as candidates consciously accept involvement as valuable, but regulation remains external, and behavior nonself-determined.
"Candidates are driven by motivation to acquire new profession-
al skills and expand their professional capacity, including employment opportunities." (academic supervisor, male, jurisprudence)
"I believe most candidates are motivated by the opportunity to enhance their qualifications and become more demanded by employers in their field." (academic supervisor, female, biology)
In some narratives, this type of identified motivation was regarded as a "social elevator" that opened additional opportunities for career promotion, especially in academia. Besides, academic supervisors pointed out that doctoral programs sometimes promoted geographic mobility of young researchers, facilitating emigration. Given the attractiveness of such a trajectory for the majority of youth, this type of motivation may exacerbate the problem of "brain drain", which is still relevant for Russian science [Yurevich, Malakhov, Aushkap 2017].
"And then, they can move to another city, or another country. This sort of mobility-oriented motivation, it's always been there." (academic supervisor, male, mathematics and mechanics)
Survey data allowed evaluating the incidence of motives associated with different career orientations. Academic tracks—desire to work at a university or research institution — were found to be the most popular trajectories. Forty-seven percent of the doctoral candidates reported having embarked on a PhD because they had believed it would help them make a teaching career at the university or another educational institution, and 53% had expected doctoral programs to help them build a research career with the university or a research institution. At the same time, substantial percentages of the respondents mentioned better career opportunities at research (28%) and non-research (29%) positions beyond academia as a motive for engaging in doctoral education.
Finally, integrated type of regulation implies, in fact, a combination of intrinsic and extrinsic motivation. Although regulation remains extrinsic to behavior, involvement in that behavior is brought into congruence with one's values, interests, needs, and feelings. In the context of this study, it can be illustrated by situations where thesis completion is interpreted as "summarizing" one's professional development (external regulation is fueled by the academic community's principles, and internal regulation comes from personal interest and zeal for self-development in a particular area). Decision to enroll in a doctoral program is therefore naturally assimilated by one's system of values and interests.
"I hope this is how they want to draw a line under a certain number of years devoted to specific research. To summarize, if you will. their professional development. I mean, well, they've been do-
ing something, and someone already knows them, and they know someone too, and they understand something, but this has been all, sort of, dispersed." (academic supervisor, male, mathematics and mechanics).
4.3. Amotivation: Going Through the Motions and the Absence of Viable Alternatives
Amotivation occurs when embarking on a PhD is not preceded by careful reflection and the candidate's behavior is not significantly influenced by any intrinsic or extrinsic regulators. Effectively, this is an absence or lack of motivation. Amotivation is fundamentally different from the categories of intrinsic and extrinsic motivation analyzed above, amotivated behavior being the least intentional and autonomous of all.
A typical context in which amotivated behavior can be observed is where there are no career alternatives to embarking on a doctoral journey. Doctoral education thus becomes a manifestation of the so-called "inertia strategy" of merely proceeding along the current track, meaning that the choice is made either mechanically or as a matter of chance. Some narratives emphasized that such situations were especially typical of particular fields:
"I graduated from the MSU Faculty of Philology, and if you are a philology graduate, doing a PhD is pretty logical because you're not drowning in job offers, you know." (male, degree awarded in 2011, philosophy)
In some cases, being nudged by one's academic supervisor, department, or another university unit appeared to be the deciding factor affecting one's final decision. If a candidate has no significant reasons against deciding to pursue a PhD, they will agree to do so. In such narratives, agency is assigned to an external agent, not the internal one, and the candidates mostly talk in the passive ("I was advised"):
"Because, well, I was advised by the faculty to pursue a doctoral degree after graduation <...> They saw potential in me and advised that I should embark on a PhD <...> At first, I had no intention of doing so." (male, degree awarded in 2011, economics)
Survey data indicates that candidates often enroll in doctoral study in the absence of viable alternatives (17%)8 or regard their choice as a
8 Questionnaire item: "To what extent do you agree or disagree with the following statements concerning your reasons for doing a PhD? — I decided to embark on a PhD because I had no other plans." The following response options were available: "Strongly disagree", "Somewhat disagree", "Somewhat agree", and "Strongly agree". The indicator was estimated as a sum of percentages of the respondents who selected the options "Somewhat agree" and "Strongly agree".
concurrence of circumstances (16%)9. As the candidate's behavior is nonintentional, nonautonomous, and not regulated extrinsically or ex-trinsically, amotivation is associated with a high risk of failure. However, interview data reveals the dynamic nature of motivation: in some cases, candidates who were amotivated at the very beginning would become involved in the process and develop intrinsic or extrinsic motives for engagement.
"I chose him as my supervisor when I was in my third year <...> I just liked the way he told about what he was doing, I got really interested. I wasn't going to do a PhD but he insisted, and then I realized I didn't want to leave, I just wanted to keep working here." (doctoral candidate, female, 4th year, chemistry)
An important factor of improving persistence and involvement of doctoral candidates with this type of motivation consists in creating an institutional and learning environment conducive to the development of a strong interest in learning and doing research, academic supervisors playing a critical role in providing such an environment.
5. Conclusion The findings of this study show that academic motives (interest in doing research, teaching, and making an academic career) remain the strongest predictors of deciding to embark on a Ph D. This is in agreement with the earlier findings of a number of studies investigating the motives for doing a PhD in Russian universities [Shafranov-Kutsev, Yefimova, Bulasheva 2017; Zamaraeva 2013; Kapshutar 2016; Beko-va et al. 2017; Mironos, Bednyi, Rybakov 2017]. At the same time, survey and interview data indicates that academic motives do not always imply the greatest degree of autonomy typical of intrinsic motivation in Deci and Ryan's terms. Doctoral education is often perceived as a way of developing competencies required for a successful academic career or as a way of enhancing one's qualifications without taking interest in the teaching and research "content" of the doctoral program.
The results also allow assessing the incidence of extrinsic (external to the PhD program) motives to enroll in doctoral study, such as postponement of military service, getting a room in the halls of residence, etc. Even though the percentage of respondents who reported such motives to have played a significant role in their decision to
9 Questionnaire item: "To what extent do you agree or disagree with the following statements concerning your reasons for doing a PhD? — My embarking on a PhD was largely a concurrence of circumstances." The following response options were available: "Strongly disagree", "Somewhat disagree", "Somewhat agree", and "Strongly agree". The indicator was estimated as a sum of percentages of the respondents who selected the options "Somewhat agree" and "Strongly agree".
do a PhD was lower than that of respondents who mentioned motives associated with autonomous regulation, it was still fairly high. In addition, it turned out that many doctoral candidates had experienced a lack or complete absence of motivation (amotivation), adopting the "inertia strategy" and embarking on a PhD without careful reflection or viable alternatives.
Just as the survey responses, the interview data indicates that motivation can change throughout the learning process, so monitoring might be advisable [Mironos, Bednyi, Rybakov 2017]. Candidates who are externally motivated or even amotivated at the very beginning often find intrinsic motives in the process and become involved. Such involvement could be promoted by environmental factors conducive to autonomy and the development of intrinsic motivation, or the transition from extrinsic to intrinsic motives. Such factors may include an engaged academic supervisor or a department/laboratory encouraging scientific inquiry and creative thinking. Besides, admission of doctoral candidates to collaborative inquiry teams working on funded research projects is also vital to develop and maintain an interest in professional research [Mironos, Bednyi, Rybakov 2017].
Quantitative data on the incidence of different types of motivation provides new findings that may contribute to the debate on the problems and prospects of doctorate in Russia (see [Shestak, Shestak 2015; Bednyi 2017; Terentev, Bekova, Maloshonok 2018; Malosho-nok, Terentev 2019]), particularly on the goals and content of doctoral programs. No substantiation was found for the popular idea of discriminating between the academic and teaching tracks and abolishing teacher training as a compulsory component of doctoral education. The findings obtained call into question the optimality of the existing practices and admission procedures in doctoral education and indicate the need to improve them. Although colleges were allowed in 2017 to consider individual attainment in the subject of major as part of admission tests, a great proportion of universities keep going by the old rules [Maloshonok, Terentev 2019]. Finally, the available data does not allow arguing absolutely for or against the current model where teaching is a critical component and the degree itself is regarded as a level of higher education. The fact that a considerable percentage of doctoral students approach the doctoral journey as an opportunity to continue their education shows that it may be not the model as such but the quality of its implementation that is the source of problems.
This study has some sampling limitations that should be taken into account in order to ensure adequate data interpretation. Both the interview and survey samples consisted of universities alone, which makes it impossible to extrapolate the findings to doctoral programs offered by institutes of the Russian Academy of Sciences and industry-specific research institutions. In addition, given that institutional differences exist even at the level of universities (see, for instance, [Bekova et al. 2017]), these findings should be regarded as a point of
departure for further research, and great caution is advised in generalizing them to doctoral education in Russia as a whole. The online questionnaire was not adapted from Deci and Ryan's model, so there are limited opportunities for theoretical interpretation of quantitative findings (hence the focus on quantitative data in the article). Besides, a research methodology based on self-report questionnaires and interviews is likely to induce social desirability bias, which is especially important in the context of the issue.
This study, therefore, should be regarded as the touchstone for theoretically grounded empirical research on the motivation to embark on and pursue a Ph D. Further research in this area may investigate the relationship between different types of motivation and the educational and research outcomes of doctoral candidates, such as publication rate, awards in competitions, participation in funded research projects, thesis completion, time-to-degree, etc. Particular attention should be given to the development of reliable instruments to measure doctoral motivation. So far, attempts have only been made outside Russia [Litalien, Guay, Morin 2015]. As for Russian studies, findings are limited to school and undergraduate education (Bachelor's, Specialist's, and Master's degrees) [Gordeeva, Sychev, Osin 2013; Semenova 2016]. Apparently, special instruments need to be designed and validated to measure doctoral students' motivational characteristics. Finally, it is important to analyze changes in their motivation and the factors driving those changes so as to elaborate specific, practical recommendations on providing conditions conducive to the development of intrinsic motivation for research and teaching among doctoral candidates.
Appendix 1. Structure of the interview sample (doctoral candidates and academic supervisors)
Characteristics N
Doctoral candidates (N=18)
Gender Male 6
Female 12
Year 1st 2
2nd 4
3rd 3
4th 3
PhD holder 6
Appendix 2. Structure of the survey sample (doctoral candidates), N=354
Characteristics %
Gender Male 55
Female 45
Mode of attendance Intramural 93
Characteristics N
Fields of research Mathematics and mechanics 3
Chemistry 3
Biology 2
Economics 2
Jurisprudence 2
Political science 2
Education 1
Philosophy 1
Philology 1
Sociology 1
Academic supervisors (N=24)
Gender Male 17
Female 7
Fields of research Economics 4
Sociology 4
Mathematics and mechanics 3
Chemistry 2
Biology 2
Education 2
Philosophy 2
Physics 1
Jurisprudence 1
Psychology 1
Political science 1
Philology 1
Characteristics %
Extramural 7
Type of funding State-funded 86
Self-funded 14
Academic disciplines Mathematics and natural sciences 16
Engineering, technology, and industrial sciences 26
Social sciences 30
Education and pedagogical sciences 7
Humanities 15
Other 7
References Bair C. R., Haworth J. G. (2004) Doctoral Student Attrition and Persistence: A Meta-Synthesis of Research. Higher Education: Handbook of Theory and Research (ed. J. C. Smart), Dordrecht: Springer, vol. XIX, pp. 481-534.
Balabanov S., Bednyi B., Kozlov Y., Maksimov G. (2003) Mnogomernaya tipologi-ya aspirantov [A Multidimensional Classification of Graduate Students]. Sociological Journal, no 3, pp. 71-85.
Bednyi B. (2017) Novaya model aspirantury: pro et contra [A New Postgraduate School Model: Pro et Contra]. Vysshee obrazovanie v Rossii/ Higher Education in Russia, no 4, pp. 5-16.
Bednyi B. (2013) Rol i struktura obrazovatelnoy podgotovki v aspiranture novo-go tipa [The Role and Structure of Educational Component in PhD Programs of a New Type]. Vysshee obrazovanie v Rossii/ Higher Education in Russia, no 12, pp. 78-89.
Bekova S., Gruzdev I., Dzhafarova Z. et al. (2017) Portret sovremennogo rossi-yskogo aspiranta [A Portrait of the Modern Russian PhD Student], Moscow: NRU HSE.
Brailsford I. (2010) Motives and Aspirations for Doctoral Study: Career, Personal, and Inter-personal Factors in the Decision to Embark on a History Ph D. International Journal of Doctoral Studies, vol. 5, no 1, pp. 16-27.
Deci E., Ryan R. (1985) Intrinsic Motivation and Self-Determination in Human Behavior. New York: Plenum.
Deci E. L., Ryan R. M. (2002) Overview of Self-Determination Theory: An Organ-ismic Dialectical Perspective. Handbook of Self-Determination Research (eds E. L. Deci, M. R. Ryan), Rochester: University of Rochester, pp. 3-33.
Deci E. L., Ryan R. M. (2012) Motivation, Personality, and Development within Embedded Social Contexts: An Overview of Self-Determination Theory. The Oxford Handbook of Human Motivation (ed. R. M. Ryan), New York, NY: Oxford University, pp. 85-107.
Gilbert R. Balatti J., Turner P., Whitehouse H. (2004) The Generic Skills Debate in Research Higher Degrees. Higher Education Research & Development, vol. 23, no 3, pp. 375-388.
Gordeeva T. (2010) Teoriya samodeterminatsii: nastoyashchee i budushchee. Part 2: Voprosy prakticheskogo primeneniya teorii [Self-Determination Theory: Present and Future. Part 2: Issues of a Practical Application of a Theory]. Psikhologicheskie Issledovaniya/Psychological Studies, no 5 Available at: http://psystudy.ru/num/2010n5-13/378-gordeeva13 (accessed 20 January 2020).
Gordeyeva T., Sychyov O., Osin E. (2013) Vnutrennyaya i bneshnyaya ucheb-naya motivatsiya studentov: ikh istochniki i vliyanie na psikhologicheskoe blagopoluchie [Inner and Outer Motivation in Students: Its Sources and Influence on Psychological Well-Being]. Voprosypsychologii, no 1, pp. 35-45
Gruzdev I., Terentev E. (2017) Dannye protiv mifov: rezultaty sotsiologicheskogo issledovaniya aspirantov vedushchikh vuzov [Data Against Myths: Evidence from the Survey of PhD Students in Leading Russian Universities]. Vysshee obrazovanie v Rossii/ Higher Education in Russia, no 7, pp. 89-97.
Halse C. (2007) Is the Doctorate in Crisis? Nagoya Journal of Higher Education, no 7, pp. 321-337.
Ivankova N. V., Stick S. L. (2007) Students' Persistence in a Distributed Doctoral Program in Educational Leadership in Higher Education: A Mixed Methods Study. Research in Higher Education, vol. 48, no 1, pp. 93-135.
Kapshutar M. (2016) Motivatsiya kak factor kachestva podgotovki aspirantov [Motivation as a Post-Graduate Training Quality Factor]. Tsennosti i smysly / Values and Meanings, no 5 (45), pp. 39-47.
Kehm B. M. (2006) Doctoral Education in Europe and North America: A Comparative Analysis. Wenner Gren International Series, no 83, pp. 67-78.
Litalien D., Guay F. (2015) Dropout Intentions in PhD Studies: A Comprehensive Model Based on Interpersonal Relationships and Motivational Resources. Contemporary Educational Psychology, no 41, pp. 218-231.
Litalien D., Guay F., Morin A. J. S. (2015) Motivation for PhD Studies: Scale Development and Validation. Learning and Individual Differences, no 41, pp. 1-13.
London J. S., Cox M. F., Ahn B., Branch S. (2014) Motivations for Pursuing an Engineering PhD and Perceptions of its Added Value: A US-Based Study. International Journal of Doctoral Studies, no 9, pp. 205-227.
Lovitts B. E. (2001) Leaving the Ivory Tower: The Causes and Consequences of Departure from Doctoral Study. Lanhem, Maryland: Rowman & Littlefield Publishers.
Maloshonok N., Semenova T., Terentev E. (2015) Uchebnaya motivatsiya studen-tov rossiyskikh vuzov: vozmozhnosti teoreticheskogo osmysleniya [Academic Motivation among Students of Russian Higher Education Establishments: Introspection]. Voprosy obrazovaniya / Educational Studies Moscow, no 3, pp. 92-121. DOI: 10.17323/1814-9545-2015-3-92-121
Maloshonok N., Terentev E. (2019) Na puti k novoy modeli aspirantury: opyt sovershenstvovaniya aspirantskikh programm v rossiyskikh vuzakh [Towards the New Model of Doctoral Education: The Experience of Enhancing Doctoral Programs in Russian Universities]. Voprosy obrazovaniya / Educational Studies Moscow, no 3, pp. 8-42. DOI: 10.17323/1814-9545-2019-3-8-42
Mironos A., Bednyi B., Rybakov N. (2017) Akademicheskie professii v spektre professionalnykh predpochteniy aspirantov [Academic Careers in the Spectrum of Postgraduate Students Professional Preferences]. Universitetskoe upravlenie: praktika i analiz/ University Management: Practice and Analysis, vol. 21, no 3, pp. 74-84.
Mosicheva I., Karavaeva E., Petrov V. (2013) Realizatsiya program aspirantury v usloviyakh deystviya FZ «Ob obrazovanii v Rossiyskoy Federatsii» [Realization of Doctoral Training Programs Based on the Federal Law «On Education in the Russian Federation»]. Vysshee obrazovanie v Rossii/ Higher Education in Russia, no 8-9, pp. 3-10.
Nerad M. (2010) Increase in PhD Production and Reform of Doctoral Education Worldwide. Research Institute for Higher Education Hiroshima University, vol. 7, pp. 769-784.
Nerad M., Evans B. (eds) (2014) Globalization and Its Impacts on the Quality of PhD Education: Forces and Forms in Doctoral Education Worldwide. Rotterdam: Sense.
Park C. (2005) New Variant PhD: The Changing Nature of the Doctorate in the UK. Journal of Higher Education Policy and Management, vol. 27, no 2, pp. 189207.
Reznik S., Chemezov I. (2018) Institut aspirantury rossiyskogo vuza: sostoyanie, problem i perspektivy razvitiya [Postgraduate Education in Russian Universities: State, Problems and Prospects of Development]. Vestnik Tomsko-go gosudarstvennogo universiteta / Tomsk State University Journal, no 430, pp. 159-168.
Ryan R. M., Deci E. L. (2000) Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemporary Educational Psychology, vol. 25, no 1, pp. 54-67.
Rybakov N. (2018) Sovremennaya model rossiyskoy aspirantury: pilotnoe issle-dovanie pervogo vypuska [A New Model of Russian Postgraduate Education: Pilot Study of the First Graduation of PhD Students]. Vysshee obrazovanie v Rossii/ Higher Education in Russia, no 7, pp. 87-95.
Semenova T. (2016) Vliyanie uchebnoy motivatsii na uspevaemost studentov: rol uchebnoy aktivnosti [The Effect of Academic Motivation on Achlievement: A Role of Academic Activity]. Vysshee obrazovanie v Rossii/ Higher Education in Russia, no 7, pp. 25-37.
Senashenko V. (2016) Problemy organizatsii aspirantury na osnove FGOS tretje-go urovnya vysshego obrazovaniya [Problems of Postgraduate Training Organization on the Basis of the Federal State Educational Standards of the
Third Level of Higher Education]. Vysshee obrazovanie v Rossii/ Higher Education in Russia, no 3, pp. 33-43.
Senashenko V. (2017) O prestizhe professii «prepodavatel vysshey shkoly», uchenykh stepeney i uchenykh zvaniy [On the Prestige of the University Teacher Profession, Postgraduate Academic Degrees and Titles]. Vysshee obrazovanie v Rossii/ Higher Education in Russia, no 2, pp. 36-44.
Sizykh A. (2014) Analiz akademicheskoy sredy kak mesta ucheby i raboty [Analysis of an Academic Environment as a Place of Studies and Work]. Voprosy obrazovaniya / Educational Studies Moscow, no 1, pp. 92-109.
Terentev E., Bekova S., Maloshonok N. (2018) Krizis rossiyskoy aspirantury: istochniki problem i vozmozhnosyi ikh preodoleniya [The Crisis of Postgraduate Studies in Russia: What Bears Problems and How to Overcome Them]. Universitetskoe upravlenie: praktika i analiz/ University Management: Practice and Analysis, vol. 22, no 3, pp. 54-66.
Shafranov-Kutsev G., Efimova G., Bulasheva A. (2017) Tendentsii i factory effek-tivnosti podgotovki aspirantov rossiyskikh vuzov v usloviyakh reformirovaniya vysshego obrazovaniya [Tendencies and Factors of Efficiency of the Training of Graduate Students of the Russian Higher Education Institutions in the Conditions of Reforming of the Higher Education]. Sotsiologicheskie issledovaniya, no 9 (400), pp. 135-144.
Shestak V., Shestak N. (2015) Aspirantura kak tretiy uroven vysshego obrazovaniya: diskursivnoe pole [Postgraduate Studies at the Third Level of Higher Education: Discursive Field]. Vysshee obrazovanie v Rossii / Higher Education in Russia, no 12, pp. 22-34.
Shin J. C., Kim S. J., Kim E., Lim H. (2018) Doctoral Students' Satisfaction in a Research-Focused Korean University: Socio-Environmental and Motivational Factors. Asia Pacific Education Review, vol. 19, no 2, pp. 159-168.
Spaulding L. S., Rockinson-Szapkiw A.J. (2012) Hearing Their Voices: Factors Doctoral Candidates Attribute to Their Persistence. International Journal of Doctoral Studies, vol. 7, no 1, pp. 199-219.
Tarvid A. (2014) Motivation to Study for PhD Degree: Case of Latvia. Procedia — Economics and Finance, no 14, pp. 585-594.
Vedeneeva E., Zabelina E., Tsiring D. (2012) Motivatsionno-tsennostnaya struk-tura i soderzhanie lichnosti aspirantov [Characteristics of Motivational Personality Sphere and System of Values in Postgraduates // Kazanskiy peda-gogicheskiyzhurnal, no 3 (93), pp. 58-67.
Wiegerova A. (2016) A Study of the Motives of Doctoral Students. Procedia — Social and Behavioral Sciences, no 217, pp. 123-131.
Yurevich M., Malahov V., Aushkap D. (2017) Plyuralizm otsenok migratsionny-kh potokov nauchnukh kadrov v Rossii [Diversity of Estimations of Scientific Migration in Russia]. Nauka. Innovatsii. Obrazovanie, no 4 (26), pp. 116-124.
Zamaraeva I. (2013) Sotsialny portret sovremennogo rossiyskogo aspiranta [The Modern Russian PhD Student: A Social Profile]. Monitoring of Public Opinion: Economic and Social Changes Journal, no 5(117), pp. 152-158.
