CC BY-NC-ND
212
How University Teachers View the Digital Transformation of Higher Education
D.M. Rogozin, O.B. Solodovnikova, A.A. Ipatova
The article was submitted to the Editorial Board in December 2021
Abstract
Dmitry Rogozin — Candidate of Sociological Sciences, director of the Center for Field Studies, Institute of Social Analysis and Forecasting, Russian Presidential Academy of National Economy and Public Administration. Email: [email protected]. (Corresponding author)
Olga Solodovnikova — Candidate of Philological Sciences, senior research fellow of the Center for Field Studies, Institute of Social Analysis and Forecasting, Russian Presidential Academy of National Economy and Public Administration Email: [email protected]
Anna Ipatova — Candidate of Cultural Studies, senior research fellow of the Center for Field Studies, Institute of Social Analysis and Forecasting, Russian Presidential Academy of National Economy and Public Administration. Email: [email protected]
Address: 11 Prechistenskaya Embankment, bldg. 1, Moscow, 119034, Russia.
The absolute majority of publications about changes in higher education resulting from the COVID-19 pandemic focus on the problems faced by students. They fail to articulate the position of university faculty members who are concerned about their dwindling role as a result of the digital transformation of education. Since 2020, the Institute of Social Analysis and Forecasting at the Russian Presidential Academy of National Economy and Public Administration with the support of the Russian Ministry of Science and Higher Education has conducted a monitoring study of the attitude of university faculty members to the changes taking place in higher education. The present article is based on the results of three research waves (non-random, administrative, river sample) conducted in April 2020 (N=33,987), June-July 2020 (N=27,484) and April-May 2021 (N=26,334). An overall positive trend is observed in teacher attitudes: the peak of discontent about the introduction of distance education has passed, and the attitude to online learning has become calmer and more level-headed. Most teachers continue to express unconditional support for traditional in-person learning, however. The article takes a close look at the attitude of teachers to the digital transformation of higher education and analyzes their narratives. Teachers believe that the most promising aspect of the digital transformation of universities is the use of blended learning technologies that combine the benefits of classical and innovative teaching methods. The article identifies risk factors and further opportunities for digital innovations in higher education.
Keywords digital transformation, higher education, online survey, survey of university faculty members, administrative survey, hybrid learning model, blended learning model, distance education, distance learning.
For citing Rogozin D.M., Solodovnikova O.B., Ipatova A.A. (2022) Kak prepodavateli vu-zov vosprinimayut tsifrovuyu transformatsiyu vysshego obrazovaniya [How University Teachers View the Digital Transformation of Higher Education]. Vopro-sy obrazovaniya / Educational Studies Moscow, no 1, pp. 271-300. https://doi. org/10.17323/1814-9545-2022-1-271-300
In the spring of 2020, due to the pandemic, the total lockdown, and the uncertainty about the further spread of COVID-19, the higher education system was faced with a choice: suspend classes, dismissing students and faculty for an indefinite period, or switch entirely to a distance learning mode. The education system immediately transitioned to distance learning, and the administrative coercion to it, which offered no alternatives, inevitably created resistance [Радина, Балакина, 2021; Рогозин, 2021b]. Most university teachers reacted negatively, considering this situation an attack on academic freedom, which would lead to the imminent destruction of the entire higher education.
At the same time, for several years already, the higher education system has been undergoing a planned digital transformation, with high-tech digital solutions being introduced and curricula being modernized. While before the pandemic these processes had run in parallel, the obligatory rapid transition to distance education inextricably linked them to each other in the perception of university teachers. Educators' prejudice against the distance learning mode has become a critical part of the context in which the digital transformation of higher education has taken place. University teachers can hardly be blamed for their resistance to change: the emergency in the country, the need for rapid adjustment, the dramatically increased workload, the burden of responsibility, and administrative pressure made them hostages to the situation since in most cases these circumstances were drivers of the change taking place.
It has been more than a year and a half. Do university teachers still have the initial prejudice against the distance learning mode and the digital transformation of higher education? What are faculty attitudes towards distance education in 2021? Have university teachers found the optimal balance between distance and classroom learning? Have the current events influenced faculty attitudes towards digital transformation? How do university teachers perceive digital transformation in general? How many of them are actually opposed to digital transformation? The present study examines the main elements of the digital transformation of higher education based on the current evaluations of faculty members, who are one of the main stakeholder groups of these changes.
The digital transformation of higher education and the university teacher's role in it
The digital transformation of higher education has been a topic of discussion for at least 30 years now. One of the initial impulses for this discussion was active learning technologies, seen as a way to increase students' learning motivation. In their classic work of 1991, C. Bonwell and colleagues state that active learning using the methods of the emerging IT industry will change the role of a university teacher from just a "transmitter of knowledge" to a "facilitator" who instead transmits the teaching method [Bonwell, Eison, 1991]. Thus, already at the outset of the debate, the university teacher's role in digital transformation was one of the central issues.
In recent years, various international organizations released statements and memoranda emphasizing the priority of active learning for successful future development. They included the decisions within the Bologna Process and the European Higher Education Area [Zahavi, Friedman 2019], Partnership for 21st Century Learning [Laar et al., 2017], and various editions of the book Assessment and Teaching of 21st Century Skills [Care, Wilson, Griffin, 2018].
Further development of the discourse followed technological advances, with the gamification of education becoming its frontier topic [Subhash, Cudney, 2018]. Despite many works on the benefits of gamification and incorporation of IT technologies into the educational process, university teachers remained skeptical about dig-italization and gamification [Guerrero-Roldan, Noguera, 2018]. The most frequent areas of their concern were the anticipated changes of roles in the learning process, "time loss" for both teachers and students, a break with academic tradition, and the substitution of administering for teaching.
A compromise option combining the benefits of traditional academic education and innovative techniques was thought to be blended, or hybrid, learning [Борисова, 2019]. According to its generally accepted definition, "blended learning environments <...> combine synchronous and asynchronous activities and are situated on a continuum between face-to-face and online teaching and learning" [Graham, 2019]. In the second half of the 2010s, many studies were conducted to prove the effectiveness of blended learning in optimizing student engagement and organizing the entire learning process [Halverson, Graham, 2019; Manwaring et al., 2017; Boelens, de Wever, Voet, 2017; Boelens, Voet, de Wever, 2018]. However, the cornerstone of success remains the perception of blended learning technologies by faculty: if they are interested in using digital potential in the educational process, there is a positive change; if they are not motivated, no evidence for the effectiveness of blended learning is found.
In the autumn of 2019, just before the pandemic, Canadian researchers analyzed blended learning practices at four universities across the country, drawing on in-depth interviews with twenty fac-
ulty members from various disciplines who had experience with digital courses [Heilporn, Lakhal, Belisle, 2021]. They found out that three basic scenarios can contribute to the success of blended programs in the educational process:
• creative revision of the existing structure and pace of educational courses (blended learning depends on the dynamics of instruction and requires alternating synchronous and asynchronous classes; any needless lengthening, pauses, and so forth make it less efficient);
• providing a choice in teaching and learning activities (blended learning is effective if it gives the author of the course more autonomy and freedom in finding the most appropriate tools, study guides and approaches to instruction, and reduces the amount of strictly regimented operations);
• paying attention to the role of the university teacher and his or her interaction with the course (it is emphasized that blended learning does not reduce the importance of a teacher, but, on the contrary, increases it, requiring him or her to establish a trust-based relationship with the students, stimulating direct contact between the members of the educational group).
The result of the studies carried out so far is the understanding that the digital transformation has three independent agents/ stakeholders — society, academia (faculty) and students [Muril-lo-Zamorano et al., 2021] — and that their interests do not self-ev-idently coincide. Society can either artificially impose digitalization on universities or, on the contrary, inhibit it due to the lack of funding, necessary organizational decisions, and so forth. Students may have a very strong or, conversely, totally no demand for innovations in education. As for academia, they have personal ambitions and skepticism that affect the progress of the IT-based revolution. At the same time, the interests of these three agents should be considered in the aggregate, because their intersection is what creates an opportunity space for digital transformation at a particular point in time in a particular country.
Two common perspectives on the nature of the digital revolution, the technocratic and the humanistic, usually encourage the analysis of IT-based teaching methods in terms of public or student benefit, often ignoring the opinion of academia considered to be biased in advance and keen to preserve the status quo. It is no coincidence that the vast majority of authors examining the dramatic changes in the educational process caused by the COVID-19 pandemic focus on students' problems [Dam§a et al., 2021]. Watching the current discourse, faculty members, who were worried since the early days of the digital age about the diminishment of their impor-
tance and the redefinition of roles in the educational process, might consider their concerns legitimate: their voice is often unheard and their position not articulated.
Several early works on faculty readiness for digital transformation aimed to identify the factors contributing to the educators' positive attitude towards IT-based innovations [Buchanan, Saint-er, Saunders, 2013]. In particular, Finnish researchers outlined the "holistic concept of digital competence" required of today's teachers [Ilomaki et al., 2016]. Among the basic competences that were later referred to by other authors were the following:
• technical competence (the ability to use relevant technology and programs);
• the ability to use digital technologies in a meaningful way for working, studying, and in everyday life;
• the ability to evaluate digital technologies critically and motivation to participate in the digital culture.
While starting with the basic competence, that is, educators' ability to navigate modern technology, the authors finished with a much more interesting statement emphasizing the importance of academia's critical attitude to the IT-based revolution. In their opinion, an ambassador of any process should avoid formalism and a conciliatory stance: digital learning can only develop effectively if it receives constructive criticism from the professional community, which should be considered an integral component of the IT-based revolution in the educational environment. In addition, the authors emphasized the importance of a holistic approach to the role of the teacher, who is not merely a "function" of digital transformation but also has other academic interests and demands.
Norwegian researchers proposed a slightly different perspective on digital competence by identifying three specific levels of IT skills of faculty members [Gudmundsdottir, Hatlevik, 2018]:
• general digital competence, that is, the instrumental skills and knowledge university teachers require in order to use digital technology in their work, including mastery of relevant software;
• subject-specific digital competence, enabling faculty members to identify the specifics of teaching a particular discipline in an online format and address it creatively by developing unique online and blended courses;
• professional digital competence, ensuring the application of pedagogical skills in the unfamiliar digital environment: changing strategies and tactics of communication with students, developing a proper online assessment, and so forth.
The two models described above have much in common, as they take into account the multidimensionality of teaching, where the simple transfer of knowledge through a usual "analog" or digital channel is mediated by individual pedagogical excellence.
In 2020 the model of digital competences in higher education was very timely supplemented with a parameter formulated in the work of T. Aagaard and A. Lund [Aagaard, Lund, 2020]. The authors suggested that besides general, subject-specific and professional digital competences, transformative competence is also critical for the stability of higher education. Transformative competence refers to the ability of faculty and students to continuously reform and update their teaching and learning practices and is absolutely necessary to find ways out of extraordinary situations. Published in December 2019, the book of these authors could claim to be prophetic considering the COVID-19 challenge that followed, making transformative competence one of the most in-demand in the educational services market.
At the same time, it is evident that the adaptation of the educational process to the online delivery mode depends not only on the teacher's will and on students' willingness. The third stakeholder of digital transformation — the society represented by its regulatory institutions — also plays an important role. According to E. King and R. Boyatt, institutional culture is the key factor contributing to effective online learning [King, Boyatt, 2014], while F. Pettersson [Pettersson, 2018], based on the analysis of available sources, concluded that the positive effect of the teacher's personal digital competences is largely mediated by the organizational context of his or her activities. Both the absence of any institutional support and the excess of institutional regulation and control can be detrimental. If control over the teaching process becomes obtrusive, one of the key advantages of digital transformation — its flexibility and in-novativeness — may be lost. In their work Seamless Learning. Perspectives, Challenges and Opportunities, researchers from Singapore clearly demonstrate the importance of informal, unwritten methods developed in the process of live teacher-student communication for successful online teaching [Looi et al., 2019].
Methodology Since 2020 the Russian Presidential Academy of National Economy and Public Administration has been conducting a monitoring study of the faculty attitudes to the changes taking place in higher education. The study is based on non-random, administrative, river sample. The main administrative river sampling is organized with the support of the Russian Ministry of Science and Higher Education, invitations to participate in the survey are sent to all higher education institutions in Russia. The sampling process is controlled
locally by the universities' administrative staff. In order to control and evaluate the administratively approved responses, a second, alternative river sampling of respondents is organized through targeted advertising on social networks. For more details on the organization of the survey, see [Рогозин, 2021b]. This article is based on the results of the third wave of the study.
The first wave took place from April 10 to 15, 2020, inclusive. A total of 58,812 people participated in the survey over six consecutive days, of which 20,273 people met the eligibility criteria. Of the eligible respondents, only 6% refused to participate in the survey and 5% stopped completing the questionnaire before reaching the end. A total of 33,987 completed questionnaires were collected. The response rate, or the ratio of completed questionnaires meeting the eligibility requirements and quality criteria, to all click-throughs to the questionnaire form, was 89%.
The second research wave was conducted from June 25 to July 10, 2020. During 16 consecutive days, 42,382 people took part in the survey. 928 of them, or 2% of the sample, did not qualify for the survey; 11,680 respondents, or 28%, refused to participate right after answering the screening questions; 2,290 people, or 5%, stopped filling in the questionnaire before reaching the end. The final sample included 27,484 questionnaires. The response rate was 65%.
The third wave of the survey took place from April 23 to May 31, 2021. In 38 consecutive days, 42,272 click-throughs to the questionnaire were registered, and 32,086 people started to fill it out. A total of 26,334 questionnaires were collected that met the eligibility requirements (the respondents were teachers of higher education institutions). After controlling for errors and inaccuracies and editing the array, the final sample of 24,337 observation units was obtained. The response rate was 57%.
The consistent decrease in the response rate is, firstly, due to the large number of surveys conducted in higher education and, consequently, decreased faculty interest in participating in repetitious studies, and, secondly, due to the administrative organization of the survey, which without proper support lowers respondents' motivation to participate. Although the response rate is far from the extreme value (in opinion polls, it does not exceed 10-15%), this downward trend requires an in-depth methodological analysis.
Faculty attitudes towards the digital transformation of higher education
In 2021, the Russian Ministry of Science and Higher Education developed the Strategy for Digital Transformation of the Science and Higher Education Sector, aimed at achieving "digital maturity". One of the goals of the Strategy is to implement the target model of a digital university in all higher education institutions subordinated
to the Russian Ministry of Science and Higher Education1, which, in particular, implies the active participation of the university faculty members in the planned changes. In order to assess faculty attitudes towards them, one of the questionnaire blocks in the third wave of our study was devoted to digital transformation.
Most respondents claim to be well aware of the digital transformation processes in higher education (Table 1).
Table 1. Awareness of Digital Transformation Processes in Higher Education, % by column
Do you know, have you heard or read, or are you currently reading for j the first time about the digital transformation in higher education? ! Wave 3, spring of 2021 (N = 24 337)
Know about it in detail 28.3
Have heard or/and read something 58.2
Reading about it now for the first time 7.4
Don't know 6.1
More than a quarter of respondents indicate that they know about the current changes in detail, and 58% have heard something about them. Even if we assume that the answer option "Have heard something" is a socially approved norm for a university employee, it can be argued that it is the pandemic and recent changes in the educational process that have made the knowledge about IT-based innovations in higher education background and common. At the same time, just over 15% of respondents take a negativistic stance; the majority take a neutral wait-and-see stance, and almost a third of respondents report a positive attitude (Table 2).
Table 2. Attitudes Toward Digital Transformation, % by column
What is your overall attitude toward the digital transformation: positive, negative, or neutral? i Wave 3, spring of 2021 \(N = 24 337)
Positive 31.8
Negative 15.6
Neutral 40.9
Don't know 11.7
Only about 60% of respondents are aware that digital transformation is taking place at their university; 16% know nothing about it, and 22% cannot give a definite answer. Faculty members' evalu-
1 The Strategy for Digital Transformation of the Science and Higher Education Sector was approved by the Ministry of Science and Higher Education of the Russian Federation in 2021. https://www.minobrnauki.gov.ru/docu-ments/?ELEMENT_ID=36749
ations of the process and first results of the digital transformation in their university are somewhat more critical than their attitude toward digital transformation as such, but most tend to evaluate these results as "good" or "satisfactory" (Table 3).
Table 3. Evaluation of the Implementation of Digital transformation at the University of the Respondent, % by column
How do you evaluate the implementation of digital transformation at your higher education institution? Wave 3, spring of 2021 (N = 24 337)
Excellent 11.2
Good 39.7
Satisfactory 34.3
Unsatisfactory 4.7
Don't know 10.2
University teachers who are concurrently employed in administrative positions assess the quality of the digital transformation in their university as "excellent" or "good" somewhat more often (Table 4). However, this bias is not decisive here.
Table 4. Dependence of the Evaluation of Digital Transformation on the Respondent's Performance of Administrative Tasks, % by column
How do you evaluate the implementation of I Do you perform any administrative or managerial tasks in adj dition to teaching?
digital transformation at your higher education institution? ! Yes, I do j No, I do not i Don't know I Total
Excellent ! 11.9 j 10.4 I 7.2 I 11.2
Good i 40.5 ! 38.7 j 35.7 i 39.7
Satisfactory ! 34.8 j 33.9 j 32.0 ! 34.3
Unsatisfactory ! 4.5 ! 4.8 ! 4.0 ! 4.7
Don't know i 8.3 | 12.1 I 21.1 I 10.2
This block of the questionnaire included one multiple-choice question, namely: "What do you think the digital transformation of higher education is primarily aimed at?" Since the Strategy for Digital Transformation of the Science and Higher Education Sector was published on the website of the Ministry of Science and Higher Education on July 14, 2021, which is almost three months after the start date of the survey, the response options were not directly related to this document. Respondents were free to choose up to two answer options from the list, or they could articulate their own position by commenting on the option "Other" (Table 5).
Table 5. Assumed Goals of the Digital Transformation (up to two response options)*
Response options Absolute values The proportion of all responses (%) The proportion of all respondents (%)
Distance teaching of students, development of online education 14 507 33,9 59,6
Digital library resources, access to international databases 10 515 24,6 43,2
Automation of administrative and economic activities of the university 1876 4,4 7,7
Automation of university research activities, equipping laboratories and research centers 2139 5,0 8,8
Control over the university leadership by the Ministry 1419 3,3 5,8
Control over teaching activities 3049 7,1 12,5
A feedback system for faculty and students 3963 9,3 16,3
A digital archive, educational and regulatory university databases 2570 6,0 10,6
Other 886 2,1 3,6
Don't know 1831 4,3 7,5
Total 42 755 100,0 175,7
* Since it is a multiple-choice question, the sum of percentage proportions of the total number of respondents exceeds 100%.
All the proposed response options can be roughly divided into those gravitating toward positive, neutral, and negative scenarios of digital education development in Russia. They are selected from among the possible ones based on the results from the first wave of the survey on faculty attitudes towards digital transformation in the COVID-19 era [Рогозин, 2021b]. The small percentage of teachers who chose the option "Other" confirms the validity of the list of answer options included in the questionnaire.
The body of relatively positive evaluations is represented by the statements about the following opportunities created by the digital transformation:
• digital library resources, access to international databases;
• automation of university research activities, equipping laboratories and research centers;
• a feedback system for faculty and students;
• a digital archive, educational and regulatory university databases.
The body of neutral statements includes the following options:
• distance teaching of students, development of online education;
• automation of administrative and economic activities of the university.
The selection of the following statements suggests that the respondent expects the digital transformation of the university to generate problems for the institution:
• control over the university leadership by the Ministry;
• control over teaching activities.
In the era of the digitalization of education, the negative connotation attached to the idea of supervision by higher authorities is persistent in both Russian and international discourse.
Positive evaluations of digital transformation prevail over negative and even neutral ones. Positive statements account for more than half of all responses, neutral ones for about 45%, and negative ones for 10%. Even if we assume that we have failed to take into account all the variety of possible negative statements, forcing critical respondents to choose the option "Other", the negativist tone is still less significant than the others.
At the same time, the most promising aspects of digital education development — the development of a feedback system for faculty and students, and the technical re-equipment of laboratories and research centers — are not considered the most likely goals of the current IT-based transformation. The number of respondents who selected "a feedback system" only slightly outnumbers those who chose the negative scenario of "control over teaching activities". Apparently, this distribution of answers reflects faculty members' awareness of the ambivalence of IT-based transformation: digitalization, which enables a university teacher to better understand and feel a student in case of direct inquiry (and vice versa), similarly makes a teacher more vulnerable and transparent to bureaucratic control. Many of the respondents may not be sure which trend will shape the image of digital education in Russia to a greater extent.
Facilitating access to digital library resources and international databases, while being useful in itself, is rather an element of technical support of digital transformation than its driver. The large number of respondents who chose this as the ultimate goal of the IT-based reforms of the Russian educational environment may, therefore, be indicative of their low agency and conservative orientation towards using available resources instead of redefining practices.
Finally, the most popular statement about the goals of digital transformation associates the latter with the development of on-
line education and distance teaching of students. Since this statement can include a variety of implicit attitudes, its interpretation is problematic. In the perception of many Russians, as well as in the everyday and even academic discourse, digital, distance and online education are often considered synonymous. In reality, digital transformation, or digitalization, is much broader than simply moving the educational process to an online environment or replacing face-to-face interaction with students with distance education [Lund, Furberg, Gudmundsdottir, 2019; neTpyHeBa u gp., 2020]. Even before the pandemic, the most promising area of the digital transformation of universities was considered the use of blended learning technologies, which maximize the benefits of classical and innovative teaching methods. In those subject areas and with those students where / for whom gamification, internetization and other digital innovations ensure more effective learning, the use of innovations is appropriate; when the innovations do not seem to provide additional benefits or when they put at risk the established traditions of higher education, classical face-to-face methods should be used. The ultimate goal of blended learning is not to modernize all higher education at any cost, and especially not to leave teachers and students without face-to-face meetings, but to apply the technologies, methods, and pedagogical innovations that are most appropriate in each particular case. Whether our respondents meant such adaptation of the educational process or anticipated a decline of all face-to-face teaching modes, remains unclear. Given the observations from the first wave of the survey, the latter option is more likely.
A small proportion of respondents were not satisfied with the available response options and articulated the goal of the digital transformation in Russia in their own way. Due to the considerable size of the total sample, the number of alternative answers received allows us to analyze them in detail (N = 886 in the edited array). The option "Other" was most often chosen by the respondents who sought to highlight specific negative aspects of the IT-based modernization of education, as evidenced by the top-15 most frequently used words in the open-ended responses received (Table 6).
Some of the words above have a predominantly negative connotation, such as the words "reduction", "decrease" and "destruction". In this context, the otherwise neutral word "saving" is also seman-tically related to them. At the forefront, in addition to the neutral phrases about "digital transformation" suggested by the very structure of the question, are "reduction of quality", "destruction of education" and "breakdown".
The analysis of the most frequent answers shows that there are two answer types at the two extremes of the sample: one group is characterized by a pronounced formal approach to completing the
Table 6. Fifteen Most Frequent Words in Open-Ended Responses About Digital Transformation Goals, % by column
Position Word Absolute values The proportion of the total sum of the most frequent words (%)
1 Education 404 24.8
2 University teacher 146 8.9
3 Digital 127 7.8
4 Reduction 111 6.8
5 High 93 5.7
6 Decrease 91 5.6
7 Student 88 5.4
8 Educational 86 5.3
9 System 83 5.1
10 Saving 77 4.7
11 Quality 75 4.6
12 Higher education institution 69 4.2
13 Process 62 3.8
14 Destruction 60 3.7
15 Transformation 60 3.7
Total 1632 100.0
questionnaire, the other — by containing a highly emotional message. In the first case, the respondent often copied the formally correct answer to the question from official documents, articles, and guidance manuals, or gave a commonly used definition:
The digital transformation of education is an update of the expected educational outcomes, the content of education, the methods and organizational forms of academic work, and the evaluation of the achieved results in a rapidly evolving digital environment, aimed at radically improving the educational outcomes of each student (male, 38 years old, Candidate of Sciences, St. Petersburg).
The interaction of administration, faculty and students in organizing and managing the educational process, research, and economic activity of the university (male, 64, Doctor of Sciences, the Republic of Mordovia).
In the second case, the respondent sought not to answer the question, but to express his or her emotional attitude to the research topic in general:
Who cares what I think, my opinion does not affect anything! What do the authors of this questionnaire want to find out by asking this question? The focus and goals of the digital transformation are defined not by an average faculty member!!! (male, 44, Candidate of Sciences, Kirov Oblast)
All other responses can be arranged on the usual continuum reflecting positive to negative attitudes to the IT-based innovation of the educational environment. This reveals the factors of possible success, as well as problem areas of the innovation process.
Respondents with a moderately positive attitude to digital transformation usually mentioned one of the following three assumed goals of the current changes (or a combination of them): personalization of educational trajectories, mastering of new digital products by students, and the technological advancement of the country and higher education.
The idea behind the digital transformation of education is for everyone to achieve the required educational outcomes through the personalization of the educational process, including the use of artificial intelligence methods and virtual reality tools, the development of digital educational environments in educational institutions, providing public broadband Internet access, and working with big data (male, 39, no academic degree, Moscow).
The goal of education changes to building and developing students' ability to learn and understand the logic of finding new solutions that move science forward (female, 36, no academic degree, Khabarovsk Krai).
Equipping higher education institutions with modern digital technologies, intended to make education and educational materials more accessible to all (female, 50, Candidate of Sciences, Altai Krai).
Ideally, digitalization should lead to the algorithmization of all the main processes in the university, that is, managerial, administrative, economic, research and educational ones, followed by the emergence of information systems to support activities and then decision-making (male, 38, Candidate of Sciences, Saratov Oblast).
Some respondents avoided generally accepted answers, sharing unconventional ideas about the innovation goals.
Creation of virtual platforms where students from different regions and cities could receive a proper education. Creation of virtual on-
line universities akin to Yandex.Taxi (male, 45, Doctor of Sciences, Novosibirsk Oblast).
Certain responses suggest a likelihood of success in the digital transformation of Russian higher education. The most important factors contributing to the likelihood of success are the competence of many respondents in defining the proposed concepts, the ability to distinguish between "digitalization", "distance education" and other developments in higher education, the awareness of blended learning methods, and the willingness to adopt best practices, while critically evaluating digitalization in general. An example of the respondents' competence:
Improving the quality of education through the development of individual digital learning paths for students. Introduction of advanced AR- and VR-based learning systems and artificial intelligence systems, ensuring access to quantum computing. All the rest has nothing to do with digital transformation, being part of the usual IT systems development process (male, 67, Candidate of Sciences, Moscow).
Statements in favor of blended learning:
No education system is self-sufficient; we should therefore strive for a balanced and high-quality organization of the learning process in higher education, combining primarily classroom learning with students' independent work and employing necessary online resources and the positive energy of a living person (female, 38, Candidate of Sciences, Volgograd Oblast).
It is primarily a learning model for acquiring knowledge both face-to-face in the classroom and through online courses, the creation of a single digital space for the learner for different areas of activity (female, 50, Candidate of Sciences, Chelyabinsk Oblast).
The negative responses are represented by several common statements that describe digitalization as a process aimed at "optimizing" higher education institutions and saving public funds by reducing the faculty size, "dumbing down" students, dividing education into "high-quality intramural" and "low-quality extramural", embezzling the budget and goldbricking.
The destruction of higher education in the country and the world, the stratification of society into a large group of fooled, easily manipulated people and a small elite group having access to
quality face-to-face education (female, 45, Candidate of Sciences, Irkutsk Oblast).
The stated goal is to ultimately reduce education to a service delivery system (male, 49, Candidate of Sciences, Omsk Oblast).
The holy grail of the proponents of digitalization is to finally eliminate the university teacher as an obstacle to the market-based interaction "administration — diploma — student"; record courses on Coursera and sell the rights to them. Nobody is concerned about what will happen to the professions and knowledge increment in one generation. Maximizing profits in the short-term is what they really care about (female, 47, Candidate of Sciences, Moscow).
To save university funds. The state hopes to replace traditional education with online education to spend less money on supporting higher education institutions. In my opinion, the transition to fully online learning is unacceptable: we thereby discredit the whole idea of getting a higher education (male, 24, no academic degree, Yaroslavl Oblast).
All the reforms of recent years are aimed at imitating a response to the challenges of our time, while what actually happens is a redistribution of funds, the creation of a hierarchy of universities consisting of elite institutions and those for the "plebs", increasing control and unification of education and teaching, the supremacy of indicators, the decay of academic freedom and ethical standards in relationships of faculty members with each other and with students, replaced by loyalty to the leadership and all the innovations initiated by them (female, 71, Doctor of Sciences, Moscow).
The opinions of Russian university teachers reveal both similarities and differences with their foreign colleagues' perceptions of the digital transformation of higher education. Given the Russian context, at the fore is the problem of social justice and the stratification of society into "elite" and "masses" who are allegedly forced into distance education. Besides, Russian university teachers are concerned about traditional learning being replaced with "optimizing" online learning and are less worried about the constraints on academic freedom and the teacher's changing role in society. They place economic concerns at the forefront [Ларионова и др., 2021]. After one year of the pandemic, critical respondents point primarily to institutional problems related to digital transformation and not so much to technical or organizational ones. The reason could be that most of the urgent infrastructural challenges have been
resolved during the first year, encouraging teachers to move from criticism of their own working conditions towards considering more global issues related to educational process management.
Thus, the analysis of narratives clearly identifies the problem areas in the digital transformation of higher education as perceived by Russian faculty members. Firstly, it is the narrowing down of digital transformation to the digitalization of the educational process and the introduction and/or wider spread of digital technologies, whereby digital transformation is understood as a full or partial transition to distance learning. Secondly, it is a dilution of institutional trust between all participants in the process, and a discrepancy between the stated and actual reform goals, articulated and recognized by university teachers. Respondents especially often pointed to the mismatch between the declared goals and the real-world situation:
The question is incorrect. If it is about the declared goals in order to check the competence of the respondent, one group of answer options is appropriate. If it is about the actual situation, other options should be chosen (male, 70, Candidate of Sciences, Kamchatka Krai).
According to what is declared, [digital transformation is aimed at] all of the above. In reality, it is aimed at saving money at the expense of higher education and reducing funding and staff (male, 50, Doctor of Sciences, Moscow).
I can't know the real goals. For me personally, the advantage of digitalization in education comes down to positions 1 and 2 (woman, 60 years old, Doctor of Sciences, St. Petersburg).
Besides doubting the integrity of the digital transformation actors and pointing out the lack of transparency in their goals, the respondents also distrust their competence and ability to introduce innovations into higher education:
The system is extremely flawed. We fill out the profile with indicators that do not correspond to our real-life activities. The developers have a poor understanding of the needs of faculty and offices. For reports, we have to fill out the forms all over again (male, Doctor of Sciences, Moscow).
I believe that proponents of digital transformation do not know its exact goals, but simply follow the mainstream and the general motto of saving budget funds. All of the above goals can be achieved one way or another through the digitalization of the educational process (male, 56, Candidate of Sciences, Moscow).
I think and observe in our university that distance education is considered a way to save money on faculty. The leadership does not understand that online education should be blended and hybrid and requires lots of time and advanced qualifications (female, 53, Candidate of Sciences, Primorsky Krai).
I believe that digital transformation is inevitable in today's world. However, this process is usually implemented by people who do not understand anything about it — at least in those universities with which I cooperate (male, 39 years old, Candidate of Sciences, Moscow).
The attitudes of university faculty members to distance learning is one of the main subjects of this monitoring study, and the relevant questions in one form or another have been included in all three research waves. The peak of faculty discontent due to the introduction of distance learning has passed: their attitude to online learning has become more calm and level-headed. In a year, the proportion of negative evaluations decreased by 9.6 percentage points, the share of positive ones rose by 6 percentage points, while neutral assessments increased by 3.5 percentage points (Table 7).
Table 7. Respondents' Attitudes Toward Distance Education, % by column*
What is your overall attitude toward distance education in higher education institutions: positive, negative, or neutral? Wave 2, summer of 2020 (N = 25 386) Wave 3, spring of 2021 (N = 24 337) The difference, percentage points
Positive 20.0 j 26.0 6.0
Negative 47.0 i 37.3 -9.6
Neutral 29.2 ! 32.7 3.5
Don't know 3.9 ! 3.9 0.1
* The question was asked in the second and third research waves.
Although in 2021 the proportion of those dissatisfied with distance learning remains the highest (37.3%), more than a quarter of the respondents already have a positive attitude towards it, and about 33% take a neutral position. In the first wave of the study conducted on April 10-15, 2020, when asked about online delivery mode, 70% of those surveyed claimed that it had a negative impact on student learning; 15% believed that the quality of students' online learning did not differ from that of in-person learning, and only 2% admitted that distance learning had its advantages.
Faculty attitudes towards online learning
The change of sentiments among the faculty members confirms our hypothesis based on the first wave of the survey: the rejection of IT-based transformation was primarily due to the pandemic shock, the general uncertainty about the future characteristic of the spring of 2020, and the dramatically increased workload at the end of the academic year [Рогозин, 2021b]. Given the short-term effect of the above factors, their elimination could pave the way for a more conscious attitude toward the introduction of distance and blended learning, which is exactly what happened [Логинова, Бендрикова, Дегтярев, 2021; Магомедов, Абдусаламов, Магдилова, 2020]. The greatest willingness to change their evaluations was most likely shown by the so-called "neopessimists" of spring 2020, whose apprehension and defensive pessimism were an attempt to cope with current and future threats [Рогозин, 2021а]. Once the danger had passed or started to look less frightening, this group of respondents took a more neutral or even positive stance.
Faculty evaluations of the convenience of the distance teaching mode
If we look at more person-oriented questions, in particular, "How much do you agree with the following statement: distance teaching mode is convenient and comfortable for me personally", the overall positive trend in faculty attitudes becomes even more pronounced (Table 8). The proportion of university teachers who totally agree or rather agree that the distance teaching mode is comfortable for them grows with each measurement: while in the first wave, there were 27.9% of them, in the second wave, they were already 34.4%, and by the third wave their share reached 44.1%. Accordingly, the proportion of those who totally disagree or rather disagree that the distance teaching mode is convenient is decreasing: from 67.2% in the first wave to 62.5% in the second wave and 51.6% in the third wave.
Table 8. Measuring the Convenience of the Distance Teaching Mode for University Teachers by the Degree of Agreement With the Proposed Statement, % by column
Distance teaching mode is convenient and comfortable for me personally Wave 1, spring of 2020 (N = 30 839) Wave 2, summer of 2020 (N = 25 386) Wave 3, j spring of 2021 ; (N = 24 337) Difference between waves 2 and 1, p. p. Difference between waves 3 and 2, p. p.
Totally agree 5.1 7.3 10.9 2.2 3.6
Rather agree 22.8 27.1 33.2 4.3 6.1
Rather disagree 42.1 38.3 35.2 -3.8 -3.1
Totally disagree 25.1 24.2 16.4 -1.0 -7.8
Don't know 4.9 3.2 4.3 -1.8 1.2
In the third wave, the evaluations of the convenience of distance education for faculty and students were for the first time consistent: 41% of faculty members believed that the distance learning mode was convenient and comfortable for students, and about 53% thought that it was uncomfortable (Table 9). In the first and second waves in 2020, faculty members rated their levels of discomfort with the introduction of the distance teaching mode higher than those of students.
Table 9. Measuring the Convenience of the Distance Learning Mode for Students by the Degree of Agreement With the Proposed Statement, % by column
The distance learning mode is convenient and comfortable for students Wave 1, spring of 2020 (N = 30 839) Wave 2, summer of 2020 (N = 25 386) Wave 3, spring ! of 2021 (N = 24 337) | Difference between waves 2 and 1, p. p. Difference between waves 3 and 2, p. p.
Totally agree I 3.6 3.8 7.0 0.2 i 3.2
Rather agree 1 23.3 21.6 34.1 -1.7 1 12.5
Rather disagree 1 42.0 43.3 37.0 1.3 1 -6.3
Totally disagree i 18.0 26.2 15.9 8.3 ! -10.4
Don't know j 13.2 5.1 6.1 -8.1 j 1.0
In the second wave of the study, the evaluations of the convenience of distance learning for students deviate from the general downward trend in dissatisfaction: the respondents who reported the convenience of distance learning for students in the second wave are 1.3% less than those in the first wave (26.9% versus 25.4% of those who totally agree or rather agree), and the teachers who indicated its inconvenience, conversely, are 9.5% more (60% versus 69.5% of those who totally disagree or rather disagree). The difference in values is almost completely offset by the decrease in the share of respondents who could not answer the question: by the second wave, their proportion has declined by 8.1%. Thus, between the first and second waves, there is no positive dynamics in the evaluations of the comfort of the distance learning mode for students, but in the third wave, it is evident. The distribution of answers to this question is then for the first time close to the distribution of answers to the question about the convenience of distance teaching for faculty.
The measurements in the first and second waves were carried out during the examination period and immediately afterward. It was the first semester of distance teaching, and credit tests and examinations could be administered online for the first time. It is likely that faculty members who had been unable to confidently evaluate
the convenience of distance learning for students in the first wave made up their minds by the second wave, which took place after the examination period, when 69.5% of respondents evaluated the distance learning mode as inconvenient for students.
Order of priority Along with a decrease in the rejection of distance teaching due to of the teaching the possibility of choice, the proportion of those who reject the exmodes clusivity of face-to-face teaching is increasing. While in the spring of 2020 more than half of the interviewed university teachers claimed that physical presence was absolutely necessary for their classes, a year later the share of such answers decreased to 38.7%, or by 13.7 percentage points (Table 10).
Table 10. Priority Ranking of Face-to-Face Teaching Format According to Faculty Evaluations, % by column*
Classes in my courses are best delivered only face-to-face 1 Wave 1, Í spring of I I2020 j \(N = 30 839) j Wave 3, spring of 2021 (N = 24 337) Difference between waves 3 and 1, p. p.
Totally agree i 52.4 38.7 -13.7
Rather agree ! 35.3 34.5 -0.9
Rather disagree ; 6.4 19.0 12.6
Totally disagree \ 1.5 4.6 3.1
Don't know ! 4.3 3.2 -1.1
* The question was asked in the first and third waves.
At the same time, there is still considerable untapped potential for developing faculty positive attitudes toward digital transformation and hybrid, adaptive modes of knowledge transfer. The majority of university teachers still unconditionally prefer traditional face-to-face teaching. Only a quarter of the respondents totally or rather disagree that in-person teaching is a priori better than distance teaching.
Teachers in the arts and culture, natural science, medical science, and agriculture most often insist on the priority of face-to-face instruction. Teachers in economics and management, computer science, social science, and law are more likely to be positive about the distance delivery of their classes. Thus, the respondents have a quite utilitarian approach to evaluating the advantages of distance teaching: when it is technically more feasible and does not involve practical classes and work in creative studios and laboratories, teachers support distance mode more eagerly [Захарова, Вилкова, Егоров, 2021].
The distribution of responses to the question "What proportion of the student's total learning time can be allocated to distance learning to ensure high-quality and effective education in your courses?" is quite stable. The only noticeable changes are a decrease in the proportion of those who could not answer (by 5.2 percentage points) and an increase in the proportion of those who would devote only a quarter of the total learning time to distance learning. This means that the prevalence of uncritical tolerant attitude toward the distance learning mode is decreasing, while there is an increase in the proportion of respondents who evaluate it realistically and aim to introduce distance teaching techniques so that they occupy from a quarter to half of the time allocated to a particular discipline (Table 11). Measurements for the third research wave were made in the spring of 2021. By then distance teaching in one form or another had been implemented for almost three semesters; therefore, some courses had already been adapted to the new format, and some were delivered online for the second time. Technical and information support for these disciplines had been expanded, and many pressing organizational issues had been resolved, allowing teachers to evaluate in practice and in a more thoughtful way the possibility of working within a blended learning model. The proportion of respondents who could not answer the question predictably decreased from 9.1% to 3.9%.
Table 11. Preferred Proportion of Learning Time Allocated to Distance Learning to Ensure High-Quality Education, as Evaluated by University Teachers, % by column
What proportion of the student's total learning time can be allocated to distance learning to ensure high-quality and effective education in your courses? Wave 2, sum- i mer of 2020 (N = 25 386) Wave 3, spring of 2021 (N = 24 337) Difference between waves 3 and 2, p. p.
0% 7.8 10.3 2.5
not more than 25% 50.9 55.0 4.1
50% 22.0 23.4 1.4
more than 75% 6.9 5.5 -1.4
100% 3.3 1.9 -1.4
Don't know 9.1 3.9 -5.2
Thus, only 10.3% of faculty members (although the percentage has slightly increased compared to last year) consider distance learning in their courses unacceptable at all. The absolute majority believe that a quarter to half of the learning time can be spent online and it will not affect the effectiveness of learning.
Conclusion The results of the third wave of a monitoring study into the faculty attitudes towards the changes in higher education indicate that almost 90% of university teachers are ready to adopt the blended learning model. Even before the pandemic, it was considered the most promising option given the forthcoming digital transformation. The survey results shed light on the reason behind a mildly critical attitude to online learning of more than half of the respondents: university teachers are against a full transition to distance learning, but admit its appropriateness in some cases.
There are two major problems in university teachers' perceptions of digital transformation, which can impede its implementation. The first problem is that, in the perception of many faculty members, digital transformation is reduced to the digitalization of the learning process. The second problem is the crisis of institutional trust between the participants. The narrow view of digital transformation is largely due to the abrupt and mandatory introduction of distance education during the pandemic, and the crisis of trust results from the lack of a targeted, thoughtful and reasoned public dialogue on most of the topics of concern to teachers, such as cutting staff and the number of universities, distinguishing between digitalization and distance learning, and social inequality in education. Thus, the risk factors are at the same time the opportunities to reduce institutional distrust and to develop public dialogue and joint decision-making on further digital innovations in higher education.
The article is part of the research work conducted within the government contract of the Russian Presidential Academy of National Economy and Public Administration.
References Aagaard T., Lund A. (2020) Digital Agency in Higher Education. Transforming Teaching and Learning. New York: Routledge. Boelens R., Voet M., de Wever B. (2018) The Design of Blended Learning in Response to Student Diversity in Higher Education: Instructors' Views and Use of Differentiated Instruction in Blended Learning. Computers and Education, vol. 120, February, pp. 197-212. doi:10.1016/j.compedu.2018.02.009 Boelens R., de Wever B., Voet M. (2017) Four Key Challenges to the Design of Blended Learning: A Systematic Literature Review. Educational Research Review, vol. 22, June, pp. 1-18. doi:10.1016/j.edurev.2017.06.001 Bonwell C.C., Eison J.A. (1991) Active Learning: Creating Excitement in the Classroom. Washington, D.C.: School of Education and Human Development, George Washington University. Borisova E.V. (2019) Tsifrovaya obrazovatel'naya sreda: zadachi, vyzovy, perspek-tivy [Digital Educational Environment: Tasks, Challenges, Prospects]. Mir obra-zovaniya — obrazovanie v mire, no 4, pp. 32-37. Buchanan T., Sainter Ph., Saunders G. (2013) Factors Affecting Faculty Use of Learning Technologies: Implications for Models of Technology Adoption.
Journal of Computing in Higher Education, vol. 25, no 1, pp. 1-11. doi:10.1007/ s12528-013-9066-6
Care E., Wilson M., Griffin P. (eds) (2018) Assessment and Teaching of 21st Century Skills: Research and Applications. Singapore: Springer. doi:10.1007/978-3-319-65368-6
Dam§a C., Langford M., Uehara D., Scherer R. (2021) Teachers' Agency and Online Education in Times of Crisis. Computers in Human Behavior, vol. 121, Article no 106793. doi:10.1016/j.chb.2021.106793 Graham C.R. (2019) Current Research in Blended Learning. Handbook of Distance
Education (eds M.G. Moore, W.C. Diehl), New York: Routledge, pp. 173-188. Gudmundsdottir G.B., Hatlevik O.E. (2018) Newly Qualified Teachers' Professional Digital Competence: Implications for Teacher Education. European Journal of Teacher Education, vol. 41, no 2, pp. 1-17. doi:10.1080/02619768.2017.1416085 Guerrero-Roldan A.E., Noguera I. (2018) A Model for Aligning Assessment with Competences and Learning Activities in Online Courses. The Internet and Higher Education, vol. 38, no 1, pp. 36-46. doi:10.1016/j.iheduc.2018.04.005 Halverson L.R., Graham C.R. (2019) Learner Engagement in Blended Learning Environments: A Conceptual Framework. Online Learning, vol. 23, no 2, pp. 145178. doi:10.24059/olj.v23i2.1481 Heilporn G., Lakhal S., Belisle M. (2021) An Examination of Teachers' Strategies to Foster Student Engagement in Blended Learning in Higher Education. International Journal of Educational Technology in Higher Education, vol. 18, April. doi:10.1186/s41239-021-00260-3 Ilomaki L., Paavola S., Lakkala M., Kantosalo A. (2016) Digital Competence — an Emergent Boundary Concept for Policy and Educational Research. Education and Information Technologies, vol. 21, no 3, pp. 655-679. doi:10.1007/s10639-014-9346-4
King E., Boyatt R. (2015) Exploring Factors that Influence Adoption of e-Learning with Higher Education. British Journal of Educational Technology, vol. 46, no 6, pp. 1272-1280. doi:10.1111/bjet.12195 Laar E. van, van Deursen A.J.A.M, van Dijk J.A.G.M., de Haan J. (2017) The Relation between 21st-Century Skills and Digital Skills: A Systematic Literature Review. Computers in Human Behavior, vol. 72, March, pp. 577-588. doi:10.1016/j. chb.2017.03.010
Larionova V.A., Semenova T.V., Murzakhanova E.M., Daineko L.V. (2021) Ekonom-icheskie aspekty vynuzhdennogo perekhoda na distantsionnoe obuchenie, ili Kakuyu tsenu zaplatili vuzy za distant [Economic Aspects of Emergency Transition to Distance Education, or The Price of Going Online in Higher Education]. Voprosy obrazovaniya / Educational Studies Moscow, no 1, pp. 138-157. doi: 10.17323/1814-9545-2021-1-138-157 Loginova N.S., Bendrikova A.Yu., Degtyarev S.I. (2021) Distantsionnoe obuchenie: problem i variant ikh resheniya (na primere obobshcheniya opyta distantsion-nogo obucheniya v AGMU [Distance Learning: Problems and Solutions (Using the Example of Generalizing the Experience of Distance Learning at ASMU)]. Mezhkul'turnaya kommunikatsiya v obrazovanii i meditsine, no 3, pp. 6-19. Looi C.-K., Wong L.-H., Glahn C., Cai S. (eds) (2019) Seamless Learning. Perspectives,
Challenges and Opportunities. Singapore: Springer. Lund A., Furberg A., Gudmundsdottir G. (2019) Expanding and Embedding Digital Literacies: Transformative Agency in Education. Media and Communication, vol. 7, no 2, pp. 47-58. doi:10.17645/mac.v7i2.1880 Magomedov Sh.B., Abdusalamov R.A., Magdilova L.V. (2020) Tsifrovye instrumenty v obuchenii studentov-yuristov v usloviyakh distantsionnogo i smeshannogo obucheniya [Digital Instruments in the Training of Law Students in the Conditions of Distance and Mixed Learning]. Law Нerald of Dagestan State University, vol. 36, no 4, pp. 9-13. doi: 10.21779/2224-0241-2020-36-4-9-13
Manwaring K., Larsen R., Graham Ch.R., Henrie C.R., Halverson L.R. (2017) Investigating Student Engagement in Blended Learning Settings Using Experience Sampling and Structural Equation Modeling. The Internet and Higher Education, vol. 35, July, pp. 21-33. doi:10.1016/j.iheduc.2017.06.002 Murillo-Zamorano L.R., Sánchez J.A.L., Godoy-Caballero A.L. Muñoz C.B. (2021) Gamification and Active Learning in Higher Education: Is It Possible to Match Digital Society, Academia and Students' Interests? International Journal of Educational Technology in Higher Education, vol. 18, Article no 15. doi:10.1186/ s41239-021-00249-y
Petruneva R.M., Avdeyuk O.A., Petruneva Ju.V., Avdeyuk D.N. (2020) Problemy distantsionnogo obrazovaniya glazami prepodavateley: uroki koronavirusa [Problems of Distance Education through the Eyes of Teachers: Lessons of Coronavirus]. Primo Aspectu, no 2, pp. 65-71. doi: 10.35211/2500-2635-20202-42-65-71
Pettersson F. (2018) On the Issues of Digital Competence in Educational Contexts—a Review of Literature. Education and Information Technologies, vol. 23, no 2, pp. 1005-1021. doi:10.1007/s10639-017-9649-3 Radina N.K., Balakina Ju.V. (2021) Vyzovy obrazovaniyu v usloviyakh pandemii: ob-zor issledovaniy [Challenges for Education during the Pandemic: An Overview of Literature]. Voprosy obrazovaniya / Educational Studies Moscow, no 1, pp. 178-194. doi: 10.17323/1814-9545-2021-1-178-194 Rogozin D.M. (2021b) Distantsionnoe obuchenie v period pandemii COVID-19. Metod-ologiya administrativnogo oprosa prepodavateley i studentov vuzov [Distance Learning during the COVID-19 Pandemic: Methodology of Administrative Survey of University Teachers and Students]. Moscow: Delo. Rogozin D.M. (2021a) Predstavleniya prepodavateley vuzov o budushchem distantsionnogo obrazovaniya [The Future of Distance Learning as Perceived by Faculty Members]. Voprosy obrazovaniya / Educational Studies Moscow, no 1, pp. 31-51.doi: 10.17323/1814-9545-2021-1-31-51 Subhash S., Cudney E. A. (2018) Gamified Learning in Higher Education: A Systematic Review of the Literature. Computers in Human Behavior, vol. 87, May, pp. 192-206. doi:10.1016/j.chb.2018.05.028 Zahavi H., Friedman Y. (2019) The Bologna Process: An International Higher Education Regime. European Journal of Higher Education, vol. 9, no 4, pp. 1-17. doi:10.1080/21568235.2018.1561314 Zakharova U.S., Vilkova K.A., Egorov G.V. (2021) Etomu nevozmozhno obuchit onlain: prikladnye spetsialnosti v usloviyakh pandemii [It Can't Be Taught Online: Applied Sciences during the Pandemic]. Voprosy obrazovaniya / Educational Studies Moscow, no 1, pp. 115-137. doi: 10.17323/1814-9545-2021-1115-137
