CC BY-NC-ND
98
Towards the New Model of Doctoral Education:
The Experience of Enhancing Doctoral Programs in Russian Universities
N. Maloshonok, E. Terentev
Received in Natalia Maloshonok
May 2019 Candidate of Sciences in Sociology, Senior Research Fellow, Director of the Center of Sociology of Higher Education, Institute of Education, National Research University Higher School of Economics. Email: nmaloshonok@hse.ru Evgeniy Terentev
Candidate of Sciences in Sociology, Senior Research Fellow, Center of Sociology of Higher Education, Institute of Education, National Research University Higher School of Economics. Email: eterentev@hse.ru
Address: 20 Myasnitskaya St, 101000 Moscow, Russian Federation.
Abstract. In the era of knowledge-based economy, improving the quality and efficiency of doctoral programs is a key aspect of ensuring economic growth and national competitiveness in the global arena. Doctoral education in Russia today is redefining its goals and organizational models in light of global challenges as well as the revised Federal Law On Education in the Russian Feder-
ation and the new Regulations on Awarding Academic Degrees. This transitional period, complicated with low completion rates and institutional problems, contributes to the urgency of devising improvement practices for doctoral education. Interviews with doctoral students and doctoral program administrators are used to analyze Russian universities' practices designed to enhance doctoral studies. Those practices are grouped in accordance with the traditionally identified aspects of doctoral education that are directly related to its success: admissions, graduate curriculum, supervision, monitoring progress, financial support, institutional climate, practices and procedures. The article also discusses the opportunities for disseminating best practices to improve doctoral education as well as the restrictions that must be taken into account.
Keywords: doctoral programs, improving doctoral education, education policy, best practices, exchange of experience.
DOI: 10.17323/1814-9545-2019-3-8-42
Achieving a critical number of innovations is key to maintaining a country's competitive power and consolidating a leading position in the global arena. This requires an array of highly qualified professionals who are not only narrowly specialized but also possess some universal competencies [Nerad 2006; 2010; Pearson, Evans, Macauley 2008;
Translated from Russian by Pearson 2005; Lee, Brennan, Green 2009]. Doctoral education is a fun-I. Zhuchkova. damental component in training skilled workforce to foster economic
and social development almost everywhere around the globe [Nerad, Heggelund 2011; Pearson, Evans, Macauley 2016], so finding and disseminating ways to improve doctoral education are vitally important.
Doctoral education in Russia is currently redefining its goals and organizational models [Maloshonok, Terentev 2019]. On the one hand, this transitional period is explained by the influence of some global trends, such as internationalization [Halse 2007; Nerad 2006; Nerad, Evans 2014] and massification of higher education [Marginson 2004; Nerad 2006], and by the spread of liberal ideas and the discourse on productivity in higher education [Olssen, Peters 2005; Zepke 2015]. On the other hand, transition has been instigated by the national education policy in training academic workforce and the adoption of two laws, the revised Law On Education, which came into force in 2013 and changed both the formal status of doctoral education and the doctoral curricula, and the revised Regulations on Awarding Academic Degrees, which came into force in 2014 and tightened the requirements for admission to doctoral studies. In addition, a number of leading Russian universities were entitled to award academic degrees of their own in 2017, which has changed significantly the rules of doctoral admission and education as well as the degree awarding procedure. That is to say, the global trends and national education policy are shaping conditions to which doctoral programs should respond by changing their curricula, student and faculty training requirements, on the one hand, while on the other such newly emerging conditions create limitations for reforms and qualitative improvements in doctoral education.
The reformation of Russian doctoral education has been discussed in scientific literature [Bedny, Rybakov, Sapunov 2017; Bedny 2017; Maloshonok, Terentev 2019]. In particular, scholars raise questions about the falling (since 2013) thesis completion rates, legal differentiation between defending a thesis and earning a certificate of the completion of a doctoral program, increased doctoral student workload, etc. [Bedny, Rybakov, Sapunov 2017; Mironos, Bedny 2016]. It should be admitted that many of the existing doctoral education issues—such as high attrition rates, challenges associated with academic writing and ensuring research productivity, financial support, and the match between doctoral graduates' skills and employers' requirements—occurred before the transition period [Balabanov, Bedny, Mironos 2007; Bedny, Mironos 2008]. However, they have grown more acute following the transformation of doctorate and tightening of doctoral student requirements and now need to be urgently addressed and tackled.
The article investigates into the practices developed at the level of universities and doctoral programs in response to the challenges mentioned above and designed to enhance the quality of doctoral training. Special attention is given to the perceptions of candidates and doctoral program administrators about the measures that could improve the quality of doctoral education and the conditions for research and thesis writing.
1. Enhancement of Issues associated with improving program quality and completion Doctoral Programs rates in doctoral education have been addressed by a number of researchers worldwide [Lipschutz 1993; Ali, Kohun, Levy 2007; Pena et al. 2010; Di Pierro 2007; 2012]. Seven aspects of doctoral studies that should be altered to increase doctoral completion rates are examined [Lipschutz 1993]: admissions, graduate curriculum, supervision, monitoring progress, financial support, institutional climate, practices and procedures.
Below, we give a short overview of the measures that could be taken to improve each of those aspects of doctoral education. Available findings mostly describe advanced systems of higher and doctoral education, such as those in the United States, Great Britain, Australia, and several European countries, so our review of best practices will be limited to the experience of those countries.
1.1. Admission In a number of countries, unlike in Russia, universities and their constituent units (faculties, departments, etc.) are free to establish their own application procedures and admission requirements. Normally, the factors considered include motivation for doctoral studies and research, academic and professional background, participation in various projects, and student readiness. The latter is often measured with dedicated standardized tests, such as Graduate Record Examinations (GRE) in the United States. A series of studies prove the GRE test valid based on a positive correlation with graduate great point average [Kuncel, Hezlett, Ones 2001], while others do not find this test to be a good predictor of doctoral student success [Moneta-Koehler et al. 2017].
Susan S. Lipschutz [Lipschutz 1993] recommends paying attention to the candidate characteristics that correlate positively with doctoral degree completion and trying to answer the following questions when making admissions decisions: is the candidate motivated enough? will they be able to excel in challenging courses without assistance? are they able to manage situations of uncertainty? how realistic are their perceptions of doctoral education and academic work? Answers to those questions could be found with the help of recommendation letters and interviews with professors and current doctoral students. Lipschutz believes that current doctoral students' evaluations of applicants may prove very useful, as current students' perceptions of the qualities necessary for successful research degree completion are based on their own significant related experience.
1.2. Doctoral Nearly in every country, doctoral education includes two fundamen-Curriculum tal elements [Peña et al. 2010]: (i) coursework, which is structured familiarly for students and is relevant to their Bachelor's or Master's academic degree experience, and (ii) thesis writing, which is perceived as a new type of activity by students. With coursework, enhancement measures are designed to increase its effectiveness in developing
professional and generic, or transferable, skills that are in demand in both academic and nonacademic labor markets [Gilbert et al. 2004; Griffiths et al. 2018]. Dedicated courses have been introduced in doctoral studies to inculcate such skills and expand employment opportunities for doctoral graduates. Besides, a lot of countries have diversified their academic tracks and now differentiate between the PhD and designated professional doctorate [Boud, Tennant 2006] to bridge the gap between doctoral education and real labor required from research degree holders [Gaff 2002]. Therefore, best practices within this aspect of doctoral education have come to involve being guided by the economy and labor market demand for graduate competencies as well as ensuring graduate curriculum flexibility and diversification in order to meet that demand.
1.3. Supervision and Low quality of doctoral supervision and/or inadequate frequency of
Monitoring Progress student-supervisor interaction have a negative impact on the doctor-
of Doctoral students al student outcomes and thesis completion [Corner, Lofstrom, Pyhalto 2017], whereas supervisor support leads to better academic progress [Martinsuo, Turkulainen 2011].
In addition, monitoring the progress of doctoral students is a critical factor of their research productivity, and the key role here should be played by the supervisor [Lipschutz 1993]. Not only should supervisors help candidates actually write a thesis (provide advice on relevant literature, assist them in designing empirical research projects, comment on the work done, etc.), but they are also expected to act as "project managers" whose functions include setting deadlines, ensuring that those deadlines are met, and providing progress and final assessment of doctoral research [Lindsay 2015]. As a business manager, the supervisor must weight all the pros and cons of alternative decisions, select the best possible path to achieve the goal in the most efficient way, and guide their student along that path, controlling their pace of progress [Vilkinas 2002].
The following is used today to enhance doctoralsupervision: supervisor performance evaluation, supervisor accreditation, introduction of dedicated supervisor development programs [Pearson, Brew 2002; McCallin, Nayar 2012; McCulloch, Loeser 2016; Lee 2018], and adoption of a workplace supervision policy allowing to bridge education and work effectively [Maguire, Prodi, Gibbs 2018]. Specialized programs for supervisor development as well as guidelines, road-maps, and regulations help supervisors determine their areas of responsibility and objectives to be achieved during doctoral supervision [Lipschutz 1993].
A large body of literature on improving doctoral supervision is devoted to matching doctoral students with supervisors [Ives, Rowley 2005; Orellana et al. 2016] and collecting and using student feedback [Marsh, Rowe, Martin 2002; Mainhard et al. 2009]. There are also studies identifying the attributes of supervisors and supervision
that have positive effects on doctoral student experience and degree completion [Grant, Hackney, Edgar 2014; Ali, Watson, Dhingra 2016; Taylor et al. 2018; Fillery-Travis, Robinson 2018].
Research findings show that supervision quality is positively related to the practices of team supervision, i. e. supervision by two or more academics [Olmos-Lopez, Sunderland 2017; Nordentof, Thom-sen, Wichmann-Hansen 2013], and mentoring [Noonan, Ballinger, Black 2007]. With the latter, scholars usually discriminate between faculty and peer mentoring programs [Holley, Caldwell 2012].
There is empirical evidence that doctoral students who had faculty mentors tend to be more employable and enjoy more education opportunities contributing to their professional socialization [Lyons, Scroggins 1990; Rose 2005; Zachary 2000], and they also demonstrate better research skills and productivity [Kram 1985; Paglis et al. 2006; Rose 2005; Terrell, Wright 1988]. On the whole, mentoring programs in doctoral education have been shown to have a positive impact on degree completion rates [Maher, Ford, Thompson 2004; Wunsch 1994].
Similar effects are observed for peer mentoring programs (where mentoring is provided by established doctoral students), which improve candidates' perceptions of learning environment safety and friendliness [Bonilla, Pickron, Tatum 1994]. Such programs also contribute to higher thesis completion rates [Dorn, Papalewis, Brow 1995].
1.4. Financial University's financial support is often approached as the key factor of Support doctoral student success [Zhou, Okahana 2016]. A number of studies reveal a significant relationship between financial support and doctoral completion [Ehrenberg, Mavros 1992; Valero 2001; Mendoza, Vil-larreal, Gunderson 2014; Ampaw, Jaeger 2012; Zhou, Okahana 2016].
The following types of financial support are identified [Gillingham, Seneca, Taussig 1991; Valero 2001]: research assistantship, teaching assistantship, and university-funded fellowship. The latter is used to attract the most talented students, while assistantships enrich doctoral students' learning experiences and promote their professional socialization and integration in the university community [Girves, Wem-merus 1988].
Assistantships are regarded as a more productive type of financial support than fellowships, as they are more conducive to overcoming academic isolation [Ibid.]. Teaching assistantship was found to be a stronger predictor of degree completion than university-funded fellowship [Bowen, Rudenstine 1992]. Research assistantships have the highest likelihood of degree completion compared to other forms of financial support [Ampaw, Jaeger 2012]. In addition, doctoral candidates who were funded primarily as research assistants are significantly more likely to take research-focused jobs in the scientific workforce after graduation, as compared to candidates who were primarily supported by fellowships [Blume-Kohout, Adhikari 2016].
1.5. Institutional Some characteristics of the doctoral program learning environment Climate may be demotivating and detrimental to research productivity. According to Lipschutz [Lipschutz 1993], this includes being underestimated by supervisor, discriminated or neglected by peers and faculty, and facing hostility and intimidation on the part of professors and supervisors. Researchers around the world investigate institutional climate as a factor of doctoral attrition [Nerad, Miller 1996]. Empirical studies have shown that positive perception of learning environment is related positively with academic achievement [MacNeil, Prater, Busch 2009], student satisfaction [Umbach, Porter 2002], and degree completion [Oseguera, Rhee 2009], ensuring a more comfortable transition to the academic career [Louis et al. 2007].
1.6. Practices and Practices and procedures include doctoral program characteristics Procedures that facilitate thesis progress and reduce time to degree [Lipschutz 1993]. A number of studies indicate that promotion of research competencies, academic writing and research paper structuring skills may have a positive influence on doctoral student success as well as time to degree completion [Brush et al. 2003; Park 2007]. With that in mind, some universities set up dedicated courses and workshops in which doctoral students learn to write literature reviews and grant applications, design research programs, select data analysis methods, write and present theses and research papers [McCallin, Nayar 2012]. Increasing the frequency of research seminars has also proved to be a powerful pedagogic practice [Brush et al. 2003], as this format allows creating a productive learning environment for knowledge sharing and constructive thesis discussion [Malfroy 2005].
A whole range of practices seek to overcome social and professional isolation of doctoral candidates. Peer writing groups, in which students get to talk about their writing, are an efficient way of mitigating such isolation [Kamler, Thomson 2006]. Learning in writing groups occurs both at an individual (during writing or reading) and collective (through discussion and peer feedback) level [Aitchison 2009].
Integration into the field-specific scientific community at a regional, national and global level can be an effective way of overcoming isolation in doctoral candidates. Through partnerships, joint projects, and reciprocal visits, students extend opportunities for developing their research competencies and working on their thesis, which has a positive impact on their research productivity [Pearson, Evans, Ma-cauley 2016].
Obviously, universities around the globe have accumulated experience in enhancing doctoral programs and increasing their effectiveness, the latter being measured by degree completion rates and research productivity indicators. The study described below was designed to identify the doctoral education enhancement practices utilized by Russian universities.
2. Research The article draws on the data collected during the research project Method and Data Doctoral Education in the Project 5-100 Universities: Current State Collection Analysis and Strategies for Development. Research under this project, ordered by the Ministry of Education and Science of the Russian Federation, was conducted in March-April 2016. Its goal was to explore the doctoral education issues faced by the Project 5-100 universities and to identify best practices for solving them. Twenty interviews with doctoral candidates and 11 with doctoral program administrators were collected for the purpose of research.
Interviews with doctoral students were conducted at 11 universities (1-3 interviews in each). The sample was designed to include a variety of fields, stages and modes of study. The breakdown by stages was the following: seven students in the first year of doctoral studies, eight in the second, and five in the third year and beyond. As for the field structure, eight respondents were pursuing doctoral programs in physics and technology, six in sociology and economics, two in mathematics, one in chemical sciences, and one in legal studies. The interview guide consisted of general questions and seven substantive modules: (i) previous educational and research experience; (ii) admission to doctoral studies; (iii) attitude towards research and motivations for earning a doctoral degree; (iv) coursework; (v) off-campus employment; (vi) supervision experiences; and (vii) career aspirations. The average length of an interview was around 60 minutes. The students were interviewed face-to-face, on Skype, and by phone.
Interviews with doctoral program administrators also covered 11 universities (one respondent in each). The interview guide involved general questions and six substantive modules: (i) the policy of attracting and selecting students to doctoral programs; (ii) the problems that the university encounters in candidate attraction and selection; (iii) the composition of doctoral students at the university; (iv) educational technology used in the university's doctoral programs; (v) mechanisms of engaging doctoral students in research projects; research conditions; (vi) research productivity and practices to improve it. The respondents were interviewed face-to-face, on Skype, and by phone. The average length of an interview was around 60 minutes.
3. Challenges of Contemporary Doctoral Education in Russia and How Leading Universities Overcome Them
Analysis of the interview transcripts along with the recent findings on doctoral education in Russia [Bedny, Rybakov, Sapunov 2017; Bed-ny 2016; 2017; Bekova et al. 2017; Gruzdev, Terentev 2017; Maloshonok, Terentev 2019] allow identifying the key "sore spots" of the faculty training system within the framework of the seven aspects of doctoral education [Lipschutz 1993].
1. Admissions: ineffective selection procedures.
2. Graduate curriculum: "blurred" boundaries between the research and pedagogical components.
3. Supervision: flawed supervisor assignment mechanisms and lack of progress monitoring.
4. Financial support: lack of effective mechanisms.
5. Institutional climate: unfavorable conditions for productive learning and research activities.
6. Practices: lack of competencies required in the academic and nonacademic labor market.
Below, we are going to dwell on each of the problems listed above, outlining their nature and the practices that Russia's leading universities use to solve them. Since the article seeks to find ways to overcome the existing problems, the titles of structural units in this article describe not problems but directions for solving them. Some of the identified issues and possible solutions are subject to discussion; by outlining them this way, we follow the respondents' opinions. The final part of the article explores the opportunities and limitations associated with introducing the practices singled out, comparing Russia's doctoral education experience to that of other countries.
3.1. Admissions: Diversify the Candidate Selection System
The new model of doctoral education suggests a more flexible system of candidate selection for all educational and research institutions. Universities have been granted more freedom in setting their admission standards and determining the format of admission tests. Because the new regulations have only been in force since 2017, it is impossible to evaluate their effectiveness yet. However, doctoral program administrators emphasized in the interviews that under the existing circumstances they often have to recruit random candidates, especially if the educational or research institution receives a lot of doctoral applications from other colleges.
"Now, we are required to increase the number of doctoral students from other institutions. Given the specific nature of our research, this is of little interest to supervisors because they are basically buying a pig in a poke—they know nothing about students' capabilities and competencies." (head of department of doctoral education and faculty evaluation)
Some universities introduce complementary personal achievement evaluation tools to get to know their applicants better and stimulate those with stronger research background. Achievements considered in the selection process include, first of all, publications and participation in scientific conferences.
"All of our applicants submit a record of their publication activity, inventions, research reports, and conference participation. This does not replace admission tests but counts as an added value and
has a certain weight in the selection process." (head of department of doctoral studies)
Besides, in some institutions, applicants to doctoral programs are asked to write an essay on the assumed thesis topic. Such essays allow admissions officers to see how deep the candidate is into the topic, evaluate the quality of research already done on the topic, and assess the candidate's writing skills, which are a critical predictor of successful thesis completion.
"Apart from admission tests, we also ask applicants to write an essay reviewing the assumed field of doctoral research, which provides a framework for the prospective thesis and features analysis of the contemporary trends in the field, research goals and objectives, and a well-grounded theoretical and applied rationale." (head of department of international doctoral programs)
3.2. Graduate The current model of doctoral education implies that graduates are Curriculum: Separate qualified as "teacher-researchers". This captures perfectly the way the Academic Tracks the educational process is structured—to embrace both tracks and teach research as well as pedagogical skills. In addition to courses in thesis-related disciplines, history and philosophy of science, academic writing and scientific communication, doctoral candidates are obliged to undergo teaching internships and take the Fundamentals of Pedagogy course. Candidates as well as doctoral program administrators believe that such goal "diffusion" is unjustified and decreases the effectiveness of doctoral programs.
"Doctoral studies should be aimed at training scientific workforce-young researchers who are willing to develop as scientists, to do something meaningful out there—but not teachers." (first-year doctoral student in physics and technology)
Judging by the interview transcripts, most doctoral candidates entered the doctoral program either to start a research career and write a thesis or to learn pedagogical skills—and the mix of the two tracks impairs performance in both. Some respondents stressed the need to separate the two components of doctoral education into independent tracks.
"There should be separate standards for teachers and researchers. Not everyone wants to teach. Some faculty members are rather reserved and want no communication with students. People like that do not need the teaching component at all. They are scientists, they will complete their degree on time, they will do everything because they are passionate about it. But there is no need mixing apples and oranges. There are the research-only type, their mission
is to advance the development of science, but they are downright incapable of teaching what they themselves have come up with." (head of department of doctoral studies)
As long as both the research and teaching components are obligatory in the existing format of doctoral education (Fundamentals of Pedagogy course and teaching internships at the bare minimum), universities resort to various tricks to allow doctoral students focus on one thing instead of spreading themselves between research and teaching practice. For instance, some institutions offer alternative teaching internship formats for candidates feeling unable or unwilling to teach.
"As for the teaching component, we certainly try to engage everyone, but if someone is uncomfortable about it, we look for alternative teaching-related experiences. They may include creating teaching methodology materials, designing laboratory tasks, integrating one's research findings into the learning process and developing relevant study guides, etc. In the end, what can we do if someone is not fond of working in the classroom?" (head of department of doctoral education)
However, an alternative point of view was also stated in the interview transcripts, advocating complementarity and equal importance of research and teaching experiences in doctoral programs. The respondents who stick to that viewpoint consider the existing situation sensible.
"A professional who can do research and get published should possess some teaching skills to assist students and disseminate their own findings." (third-year doctoral student in sociology and economics)
3.3. Supervision: Improve the Supervisor Assignment System and Introduce Team Supervision
The key role of supervision in doctoral education postulates that a systematic approach to supervisor selection or assignment must be elaborated. The interview results show that nearly one in five doctoral students experience difficulties in communicating with their supervisors, which inhibit their learning and thesis progress [Bekova et al. 2017]. Not infrequently, disharmonious supervising relationships result from random supervisor assignment, with no prior acquaintance or discussion of collaboration prospects. Consequently, such student-supervisor dyads are at a high risk of disagreements that may be generated by academic or nonacademic (such as psychological traits or communication behaviors) factors. If such disagreements surface when students are already deep into their doctoral studies and thesis, chances for degree completion will drop dramatically.
With a view to reduce the risk of mismatching doctoral students to supervisors and recruiting candidates unable to get integrated into the
local academic environment, some universities only admit their own graduates into doctoral programs. In a number of institutions, Master's degree students are encouraged to explore topics that can be elaborated later in doctoral programs. As a result, strong student-supervisor ties are formed during Master's studies.
"Many of our supervisors also supervise Master's degree students. When we see talent, we advise choosing topics to allow Master's research to evolve into a doctoral thesis later on. To prevent their efforts from going down the drain. We always keep an eye on such promising students. <...> Monitoring them through supervisors and department directors." (head of department of doctoral education and faculty evaluation)
In addition, some universities make selecting an supervisor and obtaining supervisor consent prior to application one of their admission requirements, allowing applicants and supervisors to get to know each other and assess the prospects of collaboration. This way, "pig in a poke" situations are prevented, which is particularly important when candidates are graduates from another university.
"The application procedure begins with the applicant examining the list of available supervisors on the website, selecting a desired topic, and contacting the academic directly. Next, the applicant comes for an interview, and if they are recommended for admission—we have a formal interview protocol—they proceed to admission tests." (head of department of doctoral studies)
Another major challenge in doctoral supervision has to do with the lack of tools for student progress monitoring. In the Russian model of doctoral education, thesis progress is monitored by a sole supervisor. Such concentration of supervision in the hands of one person increases the risk of failure dramatically, since the final result is largely contingent on supervisor interest and candidate perseverance.
"It all depends on the quality of your relationships with the supervisor. <...> Everyone perceives doctoral programs as a closed box: a candidate is working on something for three years, and so is their supervisor, but the outcome depends on what drives those two." (second-year doctoral student in sociology and economics)
To increase the effectiveness of student progress monitoring, a number of universities implement team supervision practices, in particular workplace supervision for candidates employed off campus. In such cases, the candidate is assigned two supervisors, one in the university and the other in the employer's organization, and they both supervise the student at the same time.
"We have joint programs with businesses—our strategic partners. <...> Doctoral students pursue internships and usually use employer's resources to conduct thesis-related experiments. They normally have two co-supervisors, one here and one at the workplace, who is most often also a professor of our university working there." (head of department of doctoral studies)
3.4. Financial Support The results of a survey conducted in 14 Russia's leading universities Mechanisms in 2016 show that insufficient financial support is a major problem for two in three doctoral students [Bekova et al. 2017]. Small scholarships push students to look for an earnings-generating employment which is often mismatched to their thesis and research activity in general. According to the survey, 90% of doctoral students are employed off campus and nearly 75% of them find it challenging to combine work and study [Ibid.:35-36]. Meanwhile, only 45% of the employed respondents have jobs that are at least partly matched to their field of study. The gravity of this problem is also reflected in the interviews with doctoral students and program administrators.
"I find it extremely difficult to engage in any research activity apart from the audits and exams, because I have to make a living. The scholarship of three thousand-odd rubles is totally inadequate." (second-year graduate student in humanities)
"Pursuing a doctoral degree has never been easy, and now it is tougher than ever. It requires an enormous amount of time, which is hard to do, as students, especially younger ones, have to earn money." (head of department of doctoral studies)
Some of the Russian universities have developed two strategies to at least mitigate, if not eliminate, the problem. The first one consists in offering on-campus employment to doctoral students and engaging them in projects administered by the research departments. Not only does this practice provide doctoral students with a certain income but it also contributes to their professional socialization, expands their research project experience, and helps them collect data for their thesis. In this model, employment and doctoral activities are not competing but complementary; besides, faculty turnover is promoted.
"This <on-campus employment of doctoral students> is a very good practice. First, they do not have to seek side jobs as they are paid by the university. Second, they contribute to university performance by writing research papers, which is encouraged by the existing policy. <...> When a first-year doctoral student is employed on campus, we can say that they are getting "hooked" from now on, as they become familiar with the community, its values, and in-
stitutional climate." (head of department of doctoral education and faculty evaluation)
The second strategy consists in elaborating dedicated funding programs for exceptionally promising candidates, which include grants and additional performance-based scholarships, as well as specialized educational programs implying high student commitment and a guaranteed extra scholarship. As a rule, admission to programs with large extra scholarships is based on a highly competitive selection process, so that only the most outstanding candidates benefit.
"In my case, embarking on a PhD meant <...> getting funds for my research projects. <...> There was a scholarship of 25,000 rubles, which was a serious contribution to my income back then." (fourth-year doctoral student in sociology and economics)
The two strategies are not mutually exclusive; in fact, they can complement each other. While being applied by some universities, they are not implemented on a massive scale despite their positive effects on time to degree completion and research skill development.
3.5. Institutional Climate: Build a Productive Learning and Research Environment
A critical aspect of doctoral program enhancement concerns creating a healthy institutional climate, which implies that students are not neglected or discriminated and have friendly relationships with peers, supervisors, and other faculty members. A negative institutional climate may demotivate doctoral students and inhibit their professional growth and research progress. The respondents did not mention institutional practices of maintaining a healthy psychological climate directly, but the importance of this parameter was obvious when students provided examples of supervisors and peers helping them tackle challenging academic tasks.
3.6. Practices and Procedures: Develop Academic Writing and Presentation Skills, Encourage Academic Mobility and Intra-University Collaborations
"Whenever I need advice on my research, I always get help and assistance. In fact, it is not only about assistance. When I was applying for a grant, I could easily come and ask for advice on what to do, how to sign documents, and even some formal issues irrelevant to research." (second-year doctoral student in physics and technology)
"I don't know about the others, but my supervisor is a jackpot. She has been so helpful. When I was having troubles with my article, re-writing it over and over, she was giving me as much psychological support as she could." (second-year doctoral student in humanities)
The Russian system of doctoral education is designed to train academics, so the existing candidate requirements include, along with
preparing a thesis, publishing two or three (depending on the field of research) articles in peer-reviewed journals and presenting one's thesis findings in at least one scientific conference. Meeting those requirements often becomes an impassable barrier to getting a degree, since a lot of candidates had no experience of writing articles for peer-reviewed journals prior to admission to doctoral programs. About half of the candidates experienced difficulties preparing and publishing articles in peer-reviewed journals from the Higher Attestation Commission's list [Bekova et al. 2017]. The gravity of this problem is reflected in the interviews with doctoral students, who underline the importance of developing academic skills during studies.
"Students should be taught academic reading and writing skills as well as critical thinking skills in the first place—that should be the focus." (second-year doctoral student in humanities)
Mitigation practices implemented by some universities mostly consist in adding dedicated courses on writing and presentation skills to doctoral curricula. Such courses (usually Science Communication, Science Popularization, Academic Writing, and others) are designed to teach doctoral students the rules of academic writing and formal presentation in Russian and foreign languages, introducing them to the publishing procedure and guidelines, the fundamentals of oral academic communication (language, logic and standards of presentation, etc.) and findings presentation, etc.
"We teach students how to present their findings in conferences, speak in public, engage in academic discussions, participate in debates, and prepare publications. In particular, we have the Science Communication course—this is a specific trend that has been a focus in Europe. We absorb this practice and try to integrate it into our doctoral programs." (head of skilled workforce training department)
"The Popularization course was also of great use. They told us how to present our inventions and skills <...>, and also about advertising, about where to go, whom to speak to, and where to find information. As part of our practical work, we learned to fill out invention applications and other research-related documents <...>, so we can already apply this knowledge further on." (second-year doctoral student in physics and technology)
"The course on academic writing was very useful. Last term, we were learning to fill out grant applications, so that those who had never done it before would see how it works, how it should be done. I had already had that experience—that was how I obtained my grant for studies in Maastricht last year. And I found out that the
course could really be useful. I came away with some additional literature from the course bibliography to use in my academic writing." (second-year doctoral student in humanities)
In Russian universities, doctoral supervisor is most often the candidate's only "entry point", so the effectiveness of supervisor-student relationships largely determines student success or failure. Such organization of doctoral studies makes candidates overly dependent on their supervisors and relationships with them. Moreover, it results in academic isolation of doctoral students, who have to stew in their own juice. With all communication being mediated by the supervisor, a candidate has no opportunity to expand their research horizons or get additional external assessment of their progress.
A good way to reduce academic isolation is to encourage academic mobility in doctoral programs, allowing candidates to network, build new professional connections, and present their findings to a broader academic community.
"We have funding for academic mobility. All doctoral candidates should go on one or two academic trips within Russia every year, whether for research purposes, or to attend a conference, or to present their thesis results." (head of department of doctoral education and faculty evaluation)
International student mobility, involving acquaintance and exchange of experience with foreign researchers, is considered the most productive type of academic mobility, according to the respondents (students as well as doctoral program administrators).
"I believe that a perfect doctoral program must involve academic mobility, a very useful feature allowing to cooperate with scholars and research teams in other countries. Lately, all major studies have been conducted by international teams, which is much more productive than being restricted to only one lab." (first-year doctoral student in physics and technology).
Another strategy to mitigate academic isolation consists in using university's own resources to promote interaction with supervisors and peers. For example, one of the universities offers special intramural grants for inter-disciplinary doctoral research.
"If you are applying for a grant, you do not necessarily have to invite researchers from your field of study. You might need, say, people with cross-disciplinary experience. For my first grant this fall, I needed someone with expertise in biology and chemistry. So what you need to do is reach out to doctoral students in other fields
and try to make connections. I believe it is a winning strategy for all." (first-year doctoral student in physics and technology)
To encourage student-faculty communication, universities often engage doctoral students in the activities of departments and research centers doing research relevant to their thesis topics. As the respondents indicate, this practice fosters professional socialization of doctoral candidates, helping them meet other professionals in their field as well as learn the academic values and standards.
"Our research departments offer positions for doctoral students so as to attract and retain young scientists at the university. <...> This allows young researchers to work with leading scholars, educators, and fellows." (head of department of international doctoral programs).
4. Opportunities and Limitations of Introducing the Practices Identified
Implementation of the practices described in the previous section involves overcoming a number of barriers, both systemic and institution-specific.
For example, there are legal restrictions on the diversification of academic tracks. To allow such diversification, institutions have to resort to circumventions, their actions sometimes being inconsistent with the unified principles of doctoral education stipulated by the federal law.
Recent changes in the Russian legislation have made it possible to lift some of the limitations. A number of faculty members interviewed in 2016 reported being restricted in setting doctoral admission requirements by law. In January 2017, the Ministry of Education and Science issued the Order "On Approving the Procedure for Admission to Doctoral Programs", which allows for considering applicants' individual attainment in the selection process, thus granting universities freedom in assigning priorities to different admission requirements. Nevertheless, with some minor exceptions, universities keep following the same old rules.
In addition, effectiveness and even possibility of implementing some of the practices depends on the institution's resources. It is obvious, in particular, that using additional sources of financial support for doctoral students is determined directly by the organization's financial status. It is also obvious that the development and implementation of dedicated courses on academic writing and oral scientific communication require not only funds but also human resources. The latter becomes an especially troubling issue in the context of massi-fication of higher education in general and doctoral programs in particular, faculty often feeling overloaded and unable to assume any extra workload.
"Increasing student workload requires attracting additional teaching workforce. <..> But where would it come from? <...> Can you imagine allocating those hours among all the teaching staff? And where do we get money to pay those teachers?" (head of department of doctoral studies)
The practices described above are not a cure-all remedy that will positively solve all the systemic problems of doctoral programs in institutional and learning environments of any type. First of all, the problems and directions for solving them are subject to debate. The most disputable practices include, for example, that of a university hiring its own doctoral students. With all the potential benefits mentioned above, this practice has negative effects, too. Probable employment with the same university after graduation may generate some typical problems of academic inbreeding, increasing academic isolation, inhibiting innovation, and undermining research productivity [Sivak, Yudkevich 2009; Yudkevich, Gorelova 2015]. It also matters to which positions doctoral students are hired and how their work is matched to their thesis. A survey conducted across the leading Russian universities shows that about 25% of doctoral students employed on campus are busy doing administrative work, and only half of those doing research and/or teaching reported their job duties being matched to their thesis research [Gruzdev, Terentev 2017]. About 40% of the doctoral students employed on campus complained about work impeding their learning progress [Ibid.:94].
Another debatable issue is the need to separate/combine the teaching and research tracks in doctoral programs [Shestak, Shestak 2015; Senashenko 2017]. To solve it, the goals of doctoral education should be defined. No agreement on this point has been reached in academia or among immediate participants of the doctoral education system—administrators, supervisors, and candidates. The above-mentioned popular opinion that the two tracks should be separated because candidates usually pursue either teaching or research goals is counterbalanced by the results of a cross-university survey of doctoral students [Bekova et al. 2017], where an essential proportion of the respondents regarded doctoral education as a tool to boost career prospects in both domains.
As for the diversification of the candidate selection system, even if the risk of recruiting candidates with nonacademic motivations and random people who are not committed to learning or building an academic career cannot be eliminated by introducing additional admission requirements (e. g. portfolio), it can still be reduced. At the same time, additional requirements may worsen inequality in admission for applicants with different backgrounds. For instance, graduates from regional universities with less advanced research and conference infrastructures will find themselves disadvantaged at the very start. Besides, making portfolio a selection criterion may result in using it to
promote favored candidates, graduates of the same university in the first place. Regardless of some positive effects, this practice may have negative consequences, similar to those discussed in relation to academic inbreeding in a broader sense [Sivak, Yudkevich 2009; Yud-kevich, Gorelova 2015].
Finally, introducing the practice of allowing students to select a supervisor prior to applying may also be fraught with some difficulties. First of all, students do not actually always have a choice. For instance, an applicant might want to explore a narrow research question, for which very few or even only one supervisor is available. Or, an applicant might need specific equipment to do their research, which only one professor or research team can provide. In cases like those, acquaintance prior to application may only be of benefit to potential supervisors who will decide whether to agree to work with a student or not. Moreover, meeting and selecting the supervisor prior to application is not even always possible. Not infrequently, an applicant will be uncertain about their research interests or willing to change their field for a doctoral degree but still unsure which topic to pick. In that case, a more effective strategy would be to provide an "orientation period" for doctoral students, during which they could elaborate on the topic of their future thesis, get to know faculty members doing research in that field, and choose an appropriate supervisor. Available findings indicate that changing fields before applying to a doctoral program is a popular trend, 21% of candidates changing their field for a related one and 6% for something totally unrelated [Bekova et al. 2017].
As we can see, the best practices that we have identified are not universal, and decisions on using them must be considered well in each specific situation. University surveys analyzing the composition of doctoral applicants (including their academic backgrounds, motivations, learning and career expectations), candidates (including doctoral program quality, supervision quality, thesis progress, etc.), supervisors, and lecturers could make an important tool for designing a reasonable doctoral program enhancement policy. Results of such surveys will allow detecting tender spots in the existing practices and devising the most effective ways of firming them.
5. Comparing Doctoral Program Enhancement Practices of Russian Universities with Those Used Abroad
The interviews with university administrators and doctoral students show that some of the practices used by foreign universities have also found application in Russian academia. Such globally implemented practices include doctoral mobility scholarships and grants which enable candidates to do academic networking within their field of research and use new connections to boost their professional development and thesis progress. The practices of encouraging communication with peers and faculty in doctoral programs within departments implemented by Russian universities are considered effective in international literature as well. The analysis performed does not allow iden-
tifying the distinctive features of such practices in Russia vs abroad or their incidence or effectiveness as a function of the national context. However, this study describes what Russian universities have been doing to overcome academic and social isolation of doctoral students. Given that those practices are perceived by the respondents (students as well as doctoral program administrators) as useful for candidates, it appears viable to disseminate them to the whole system of doctoral education in Russia.
In terms of financial support, similar trends are also observed between the doctoral program enhancement practices used in Russia and abroad. Due to the lack of empirical data on research productivity of Russian doctoral students receiving financial support, no definitive conclusion can be made yet on whether this practice is useful. However, the supporting evidence accumulated by universities in other countries [Ehrenberg, Mavros 1992; Valero 2001; Mendoza, Villar-real, Gunderson 2014; Zhou, Okahana 2016] indicates that the practice is worth disseminating among Russian universities.
As for research skill development in doctoral programs, Russian universities have adopted the international experience of providing specialized courses [McCallin, Nayar 2012] to teach research skills and competencies required for a successful academic career.
Fewer common features are observed in doctoral admissions, curriculum development, and supervision enhancement practices. It is our opinion that the potential for development in these domains remains underutilized. Russia's current laws impose constraints on universities' admission and curriculum policies, granting them very little autonomy for modifying their selection criteria, while no legal restrictions exist for supervision enhancement practices. An exception to this rule applies in the case of universities entitled to award doctoral degrees of their own. They have been using actively some of the best practices described here, in particular those concerning admissions and doctoral program development and implementation, since 2017, when they were granted the right to establish their own "rules of the game". Yet, it is too early to talk about the effectiveness of implementing those practices in Russian universities, as the first cohort of candidates enrolled after adopting the new rules will only graduate in 2020-2021. Meanwhile, the institutions that do not enjoy the privilege of establishing their own admission, learning and evaluation standards can also benefit from the ideas proposed in this article. Exchange of experience in implementing doctoral education enhancement practices at a national and cross-national level will improve thesis and degree completion rates, which may have positive effects on the scientific, economic and technology development in Russia. Such exchange could be intensified through dedicated seminars, conferences, and practical sessions for doctoral program administrators, provided that universities are transparent regarding their best practices as well as newly adopted practices and doctoral program reforms.
References Aitchison C. (2009) Writing Groups for Doctoral Education. Studies in Higher Education, vol. 34, no 8, pp. 905-916.
Ampaw F. D., Jaeger A. J. (2012) Completing the Three Stages of Doctoral Education: An Event History Analysis. Research in Higher Education, vol. 53, no 6, pp. 640-660.
Ali A., Kohun F., Levy Y. (2007) Dealing with Social Isolation to Minimize Doctoral Attrition-A Four Stage Framework. International Journal of Doctoral Studies, vol. 2, no 1, pp. 33-49.
Ali P. A., Watson R., Dhingra K. (2016) Postgraduate Research Students' and Their Supervisors' Attitudes towards Supervision. International Journal of Doctoral Studies, vol. 11, pp. 227-241.
Bednyi B. (2016) K voprosu o tseli aspirantskoy podgotovki (dissertatsiya vs kval-ifikatsiya) [On the Issue of the Goal of Postgraduate Training (Dissertation vs Qualification)]. Vysshee obrazovanie v Rossii = Higher Education in Russia, no 3, pp. 44-52.
Bednyi B. (2017) Novaya model aspirantury: pro et contra [A New Postgraduate School Model: Pro et Contra]. Vysshee obrazovanie v Rossii = Higher Education in Russia, no 4, pp. 5-16.
Bednyi B., Mironos A. (2008) Podgotovka nauchnykh kadrov v vysshey shkole. Sostoyanie i tendentsii razvitiya aspirantury [Preparing Scientific Workforce in Higher Education: Current Situation and Development Trends], Nizhny Novgorod: Lobachevsky State University of Nizhny Novgorod.
Bednyi B., Mironos A., Balabanov S. (2007) Faktory effektivnosti I kachestva podgotovki nauchnykh kadrov v aspiranture (sotsiologicheskiy analiz) [Factors of Efficiency and Quality of Training in Ph.D Programs (Sociological Analysis)]. University Management: Practice and Analysis, no 5, pp. 56-65.
Bednyi B., Rybakov N., Sapunov M. (2017) Rossiyskaya aspirantura v obrazo-vatelnom pole: mezhdistsiplinarny diskurs [Doctoral Education in Russia in the Educational Field: An Interdisciplinary Discourse]. Sotsiologicheskie issledovaniya = Sociological Studies, no 9, pp. 125-134.
Bekova S., Gruzdev I., Dzhafarova Z., Maloshonok N., Terentev E. (2017) Portret sovremennogo rossiyskogo aspiranta [The Portrait of a Modern Russian Graduate Student]. Modern Analytics of Education, iss. 7(15), Moscow: National Research University Higher School of Economics.
Blume-Kohout M.E., Adhikari D. (2016) Training the Scientific Workforce: Does Funding Mechanism Matter? Research Policy, vol. 45, no 6, pp. 1291-1303.
Bonilla J., Pickron C., Tatum T. (1994) Peer Mentoring among Graduate Students of Color: Expanding the Mentoring Relationship. New Directions for Teaching and Learning, vol. 1994, no 57, pp. 101-113.
Boud D., Tennant M. (2006) Putting Doctoral Education to Work: Challenges to Academic Practice. Higher Education Research & Development, vol. 25, no 3, pp. 293-306.
Bowen W. G., Rudenstine N. L. (1992) In Pursuit of the Ph. D. Princeton, NJ: Princeton University.
Brush C. G., Duhaime I. M., Gartner W. B., Stewart A., Katz J. A., Hitt M. A., ... Venkataraman S. (2003) Doctoral Education in the Field of Entrepreneur-ship. Journal of Management, vol. 29, no 3, pp. 309-331.
Corner S., Lofstrom E., Pyhalto K. (2017) The Relationship between Doctoral students' Perceptions of Supervision and Burnout. International Journal of Doctoral Studies, vol. 12, pp. 91-106.
Di Pierro M. (2007) Excellence in Doctoral Education: Defining Best Practices, College Student Journal, vol. 41, no 2, pp. 368-376.
Di Pierro M. (2012) Strategies for Doctoral Student Retention: Taking the Roads Less Traveled. The Journal for Quality and Participation, vol. 35, no 3, pp. 2932.
Dorn S. M., Papalewis R., Brown R. (1995) Educators Earning Their Doctorates: Doctoral Student Perceptions Regarding Cohesiveness and Persistence. Education, vol. 116, no 2, pp. 305-310.
Edwards B. (2002) Postgraduate Supervision: Is Having a PhD Enough? Paper Presented at the Australian Association for Research in Education Conference (Brisbane, 2002, December 1-5).
Ehrenberg R. G., Mavros P. G. (1992) Do Doctoral Students' Financial Support Patterns Affect Their Times-to-Degree and Completion Probabilities. National Bureau of Economic Research Working Paper No w4070.
Fillery-Travis A., Robinson L. (2018) Making the Familiar Strange-a Research Pedagogy for Practice. Studies in Higher Education, vol. 43, no 5, pp. 841853.
Gaff J. G. (2002) Preparing Future Faculty and Doctoral Education. Change: The Magazine of Higher Learning, vol. 34, no 6, pp. 63-66.
Gillingham L., Seneca J. J., Taussig M. K. (1991) The Determinants of Progress to the Doctoral Degree. Research in Higher Education, vol. 32, no 4, pp. 449468.
Gilbert R., Balatti J., Turner P., Whitehouse H. (2004) The Generic Skills Debate in Research Higher Degrees. Higher Education Research & Development, vol. 23, no 3, pp. 375-388.
Girves J. E., Wemmerus V. (1988) Developing Models of Graduate Student Degree Progress. The Journal of Higher Education, vol. 59, no 2, pp. 163-189.
Grant K., Hackney R., Edgar D. (2014) Postgraduate Research Supervision: An 'Agreed' Conceptual View of Good Practice through Derived Metaphors. International Journal of Doctoral Studies, vol. 9, pp. 43-60.
Griffiths D. A., Inman M., Rojas H., Williams K. (2018) Transitioning Student Identity and Sense of Place: Future Possibilities for Assessment and Development of Student Employability Skills. Studies in Higher Education, vol. 43, no 5, pp. 891-913.
Gruzdev I., Terentev E. (2017) Dannye protiv mifov: rezultaty sotsiologicheskogo issledovaniya aspirantov vedushchikh vuzov [Data Against Myths: Evidence from the Survey of PhD Students in Leading Russian Universities]. Vysshee obrazovanie v Rossii = Higher Education in Russia, no 7, pp. 89-97.
Halse C. (2007) Is the Doctorate in Crisis? Nagoya Journal of Higher Education, vol. 7, pp. 321-337.
Holley K. A., Caldwell M. L. (2012) The Challenges of Designing and Implementing a Doctoral Student Mentoring Program. Innovative Higher Education, vol. 37, no 3, pp. 243-253.
Ives G., Rowley G. (2005) Supervisor Selection or Allocation and Continuity of Supervision: Ph. D. Students' Progress and Outcomes. Studies in Higher Education, vol. 30, no 5, pp. 535-555.
Kamler B., Thomson P. (2006) Helping Doctoral Students Write: Pedagogies for Supervision. London: Routledge.
Kram K. E. (1985) Mentoring at Work: Developmental Relationships in Organizational Life. Glenville, IL: Scott, Foresman and Co.
Kuncel N. R., Hezlett S. A., Ones D. S. (2001) A Comprehensive Meta-Analysis of the Predictive Validity of the Graduate Record Examinations: Implications for Graduate Student Selection and Performance. Psychological Bulletin, vol. 127, no 1, pp. 162-181.
Lee A. (2018) How Can We Develop Supervisors for the Modern Doctorate? Studies in Higher Education, vol. 43, no 5, pp. 878-890.
Lee A., Brennan M., Green B. (2009) Reimagining Doctoral Education: Professional Doctorates and Beyond. Higher Education Research & Development, vol. 28, no 3, pp. 275-287.
Lindsay S. (2015) What Works for Doctoral Students in Completing Their Thesis? Teaching in Higher Education, vol. 20, no 2, pp. 183-196.
Lipschutz S. S. (1993) Enhancing Success in Doctoral Education: From Policy to Practice. New Directions for Institutional Research, vol. 1993, no 80, pp. 6980.
Louis K. S., Holdsworth J. M., Anderson M. S., Campbell E. G. (2007) Becoming a Scientist: The Effects of Work-Group Size and Organizational Climate. The Journal of Higher Education, vol. 78, no 3, pp. 311-336.
Lyons W., Scroggins D. (1990) The Mentor in Graduate Education. Studies in Higher Education, vol. 15, no 3, pp. 277-286.
Maguire K., Prodi E., Gibbs P. (2018) Minding the Gap in Doctoral Supervision for a Contemporary World: A Case from Italy. Studies in Higher Education, vol. 43, no 5, pp. 867-877.
Maher M. S., Ford M. E., Thompson C. M. (2004) Degree Progress of Women Doctoral Students: Factors that Constrain, Facilitate, and Differentiate. Review of Higher Education, vol. 27, no 3, pp. 385-408.
Mainhard T., van Der Rijst R., van Tartwijk J., Wubbels T. (2009) A Model for the Supervisor-Doctoral Student Relationship. Higher Education, vol. 58, no 3, pp. 359-373.
Maloshonok N., Terentev E. (2019) National Barriers to the Completion of Doctoral Programs at Russian Universities. Higher Education, vol. 77, iss. 2, pp. 195-211.
Malfroy J. (2005) Doctoral Supervision, Workplace Research and Changing Pedagogic Practices. Higher Education Research & Development, vol. 24, no 2, pp. 165-178.
Marginson S. (2004) National and Global Competition in Nigher Education. The Australian Educational Researcher, vol. 31, no 2, pp. 1-28.
Marsh H. W., Rowe K. J., Martin A. (2002) PhD Students' Evaluations of Research Supervision: Issues, Complexities, and Challenges in a Nationwide Australian Experiment in Benchmarking Universities. The Journal of Higher Education, vol. 73, no 3, pp. 313-348.
Martinsuo M., Turkulainen V. (2011) Personal Commitment, Support and Progress in Doctoral Studies. Studies in Higher Education, vol. 36, no 1, pp. 103120.
McCallin A., Nayar S. (2012) Postgraduate Research Supervision: A Critical Review of Current Practice. Teaching in Higher Education, vol. 17, no 1, pp. 6374.
McCulloch A., Loeser C. (2016) Does Research Degree Supervisor Training Work? The Impact of a Professional Development Induction Workshop on Supervision Practice. Higher Education Research & Development, vol. 35, no 5, pp. 968-982.
MacNeil A.J., Prater D. L., Busch S. (2009) The Effects of School Culture and Climate on Student Achievement. International Journal of Leadership in Education, vol. 12, no 1, pp. 73-84.
Mendoza P., Villarreal P., Gunderson A. (2014) Within-Year Retention among Ph. D. Students: The Effect of Debt, Assistantships, and Fellowships. Research in Higher Education, vol. 55, no 7, pp. 650-685.
Mironos A., Bednyi B. (2016) K voprosu o gosudarstvennoy itogovoy attestat-sii v aspiranture novogo tipa [On the Issue of Final State Certification in the Postgraduate School of a New Type]. University Management: Practice and Analysis, no 3, pp. 118-128.
Moneta-Koehler L., Brown A. M., Petrie K. A., Evans B. J., Chalkley R. (2017) The Limitations of the GRE in Predicting Success in Biomedical Graduate School. PloS one, vol. 12, no 1, e0166742.
Nerad M. (2006) Globalization and Its Impact on Research Education: Trends and Emerging Best Practices for the Doctorate of the Future. Quality in Postgraduate Research: Knowledge Creation in Testing Times (eds M. Ki-ley, G. Mullins), Canberra: The Center for Education Development and Academic Methods, pp. 5-12.
Nerad M. (2010) Increase in PhD Production and Reform of Doctoral Education Worldwide. Higher Education Forum, no 7, pp. 69-84.
Nerad M., Evans B. (eds) (2014) Globalization and Its Impacts on the Quality of PhD Education: Forces and Forms in Doctoral Education Worldwide. Rotterdam: Sense.
Nerad M., Heggelund M. (eds) (2011) Toward a Global PhD? Forces and Forms in Doctoral Education Worldwide. Seattle: University of Washington.
Nerad M., Miller D. S. (1996) Increasing Student Retention in Graduate and Professional Programs. New Directions for Institutional Research, no 92, pp. 6176.
Noonan M. J., Ballinger R., Black R. (2007) Peer and Faculty Mentoring in Doctoral Education: Definitions, Experiences, and Expectations // International Journal of Teaching and Learning in Higher Education, vol. 19, no 3, pp. 251262.
Nordentoft H. M., Thomsen R., Wichmann-Hansen G. (2013) Collective Academic Supervision: A Model for Participation and Learning in Higher Education. Higher Education, vol. 65, no 5, pp. 581-593.
Olmos-López P., Sunderland J. (2017) Doctoral Supervisors' and Supervisees' Responses to Co-Supervision. Journal of Further and Higher Education, vol. 41, no 6, pp. 727-740.
Olssen M., Peters M. A. (2005) Neoliberalism, Higher Education and the Knowledge Economy: From the Free Market to Knowledge Capitalism. Journal of Education Policy, vol. 20, no 3. P. 313-345.
Orellana M. L., Darder A., Pérez A., Salinas J. (2016) Improving Doctoral Success by Matching PhD Students with Supervisors. International Journal of Doctoral Studies, vol. 11, pp. 87-103.
Oseguera L., Rhee B. S. (2009) The Influence of Institutional Retention Climates on Student Persistence to Degree Completion: A Multilevel Approach. Research in Higher Education, vol. 50, no 6, pp. 546-569.
Paglis L. L., Green S. G., Bauer T. N. (2006) Does Advisor Mentoring Add Value? A Longitudinal Study of Mentoring and Doctoral Student Outcomes. Research in Higher Education, vol. 47, no 4, pp. 451-476.
Park C. (2007) Redefining the Doctorate. Discussion Paper of the Higher Education Academy https://www.lancaster.ac.uk/staff/gyaccp/RedefiningThe-Doctorate.pdf
Pearson M. (2005) Framing Research on Doctoral Education in Australia in a Global Context. Higher Education Research & Development, vol. 24, no 2, pp. 119-134.
Pearson M., Brew A. (2002) Research Training and Supervision Development. Studies in Higher Education, vol. 27, no 2, pp. 135-150.
Pearson M., Evans T., Macauley P. (2008) Growth and Diversity in Doctoral Education: Assessing the Australian Experience. Higher Education, vol. 55, no 3, pp. 357-372.
Pearson M., Evans T., Macauley P. (2016) The Diversity and Complexity of Settings and Arrangements Forming the 'Experienced Environments' for Doctoral Candidates: Some Implications for Doctoral Education. Studies in Higher Education, vol. 41, no 12, pp. 2110-2124.
Peña E. V., Jimenez y West I., Gokalp G., Fischer L., Gupton J. (2010) Exploring Effective Support Practices for Doctoral Students' Degree Completion. College Student Journal, vol. 45, no 2, pp. 310-323.
Rose G. L. (2005) Group Differences in Graduate Students' Concepts of the Ideal Mentor. Research in Higher Education, vol. 46, no 1, pp. 53-80.
Senashenko V. (2017) Aspirantura kak obrazovatelnaya programma s nauch-no-issledovatelskoy komponentoy ili nauchno-issledovatelskaya programma s obrazovatelnoy komponentoy? [Post-Graduate Study as Educational Program with Scientific Research Compo-nent or Scientific Research Program with Educational Component?] Vestnik Vysshey Shkoly = Higher School Herald, no 10, pp. 4-10.
Shestak V., Shestak N. (2015) Aspirantura kak tretiy uroven vysshego obrazovani-ya: diskursivnoe pole [Postgraduate Studies at the Third Level of Higher Education: Discursive Field]. Vysshee obrazovanie vRossii = Higher Education in Russia, no 12, pp. 22-34.
Sivak Y., Yudkevich M. (2009) Akademicheskiy inbriding: za i protiv [Academic Inbreeding: Pro and Contra]. Voprosy obrazovaniya / Educational Studies Moscow, no 1, pp. 170-187. DOI: 10.17323/1814-9545-2009-1-170-187
Taylor R. T., Vitale T., Tapoler C., Whaley K. (2018) Desirable Qualities of Modern Doctorate Advisors in the USA: A View through the Lenses of Candidates, Graduates, and Academic Advisors. Studies in Higher Education, vol. 43, no 5, pp. 854-866.
Terrell M. C., Wright D. J. (eds) (1988) From Survival to Success: Promoting Minority Student Retention. NASPA Monograph Series, No 9. Washington, D.C.: National Association of Student Personnel Administrators.
Tinto V. (1993) Leaving College: Rethinking the Causes and Cures of Student Attrition. Chicago: University of Chicago.
Umbach P.D., Porter S. R. (2002) How Do Academic Departments Impact Student Satisfaction? Understanding the Contextual Effects of Departments. Research in Higher Education, vol. 43, no 2, pp. 209-234.
Valero de Y.F. (2001) Departmental Factors Affecting Time-to-Degree and Completion Rates of Doctoral Students at One Land-Grant Research Institution. The Journal of Higher Education, vol. 72, no 3, pp. 341-367.
Vilkinas T. (2002) The PhD Process: The Supervisor as Manager. Education + Training, vol. 44, no 3, pp. 129-137.
Wunsch M. A. (1994) Developing Mentoring Programs: Major Themes and Issues. New Directions for Teaching and Learning, vol. 1994, no 57, pp. 27-34.
Yudkevich M., Gorelova O. (2015) Akademicheskiy inbriding: prichiny i posled-stviya [Academic Inbreeding: Causes and Consequences]. University Management: Practice and Analysis, no 1, pp. 73-83.
Zachary L. J. (2000) The Mentor's Guide: Facilitating Effective Learning Relationships. San Francisco: Jossey-Bass.
Zepke N. (2015) What Future for Student Engagement in Neo-Liberal Times? Higher Education, vol. 69, no 4, pp. 693-704.
Zhou E., Okahana H. (2016) The Role of Department Supports on Doctoral Completion and Time-to-Degree. Journal of College Student Retention: Research, Theory & Practice, vol. 20, no 4, pp. 511-529.
