CC BY-NC-ND
46
Employer Attractiveness of Universities:
Measurement Approaches
S. Alasheev, E. Kogan, N.Tyurina
Received in Sergey Alasheev
June 2016 Senior Researcher, Privolzhsky Branch of the Federal Institute for Education Development. Address: 37 Maslennikova Ave, 443056 Samara, Russian Federation. Email: alasheev s@mail.ru Efim Kogan
Doctor of Sciences in Mathematical Physics, Professor, Research Advisor, Privolzhsky Branch of the Federal Institute for Education Development. Address: 37 Maslennikova Ave, 443056 Samara, Russian Federation. Email: efko-gan@yandex.ru Natalya Tyurina
Manager of MIA Russia Today's Social Navigator Project. Address: 4 Zubovsky Blvd, 119021 Moscow, Russian Federation. Email: nv.tyurina@rian.ru
Abstract. The paper suggests principles and techniques for assessing uni-
versities, based on their employer attractiveness, which is measured by the demand for their products. Products of the University is 1) trained specialists (graduates), 2) research projects and technical-technological development 3) scientific results (articles in maga-zines)-each of these having their own consumers. It turns out that the level of employer attractiveness is determined by the organization of university resources: equipment and facilities, personnel, managerial structure and policies. The proposed university assessment principles provide the basis for a university ranking.
Keywords: university ranking, employer attractiveness, university product, assessment criteria, assessment indicators.
DOI: 10.17323/1814-9545-2016-4-186-205
We are grateful to the members of the Expert Education Council (Social Navigator project, MIA Russia Today) for their constructive discussions and ranking methodology suggestions, as well as to Bolotov V., Gu-zhel D. and Mayor-ov A. for their ideas concerning the development of the approach proposed in the article.
Among the basic characteristics of today's Russian higher education system (insufficient intellectual and material assets; low-efficient research and development activities; and low commitment to intellectual product commercialization [Kogan, Postalyuk 2008]), researchers emphasize the following most challenging drawbacks:
• Low commitment of universities to territorial development (as a rule, universities don't associate their own progress with economic and sociocultural development of the region and territories they supply graduates to);
• No experience of dealing with the labor market or involving businesses in the formation of staff resources, development of professional standards and student assessment criteria;
• Low association between admission mechanisms and prospects of regional labor market evolution;
• Extreme disintegration between science, education and businesses, which makes research development and personnel training divorced from economic realia and the needs of regional economies.
At the same time, universities are the key intellectual resource for regions but are insufficiently used by both executive authorities and local communities. The potential of higher education contravenes its limited participation in regional development: universities are unable to present their intellectual opportunities, while regional authorities and businesses are not ready to use them. Of course, such non-involvement of universities in the life of regions affects the educational process, which often evolves self-sufficiently, in complete isolation from real-life community problems.
The preferred focus of universities on meeting regional needs is a dominant trend in the development of higher education all over the world. In developed economies, universities shape the structure of regional labor markets to a large extent.
Factories and plants formed the kernel and pivot of cities and regions back in the industrial era. Today, their role has been passed on to universities and regional academic organizations as post-industrial "factories". Instead of being just educational or research institutions, they gradually evolve into fully-fledged participants of regional development, becoming the backbone of regional growth.
Researchers agree that higher education cannot be considered solely as a source of manpower at the level of state regional policy. This "extended" definition of university mission is explained by a number of factors. First, the academic and teaching staff of universities represent the brainpower of regional communities, which can become and is actually becoming a source of new ideas and projects, including those in legislation, consulting, and expert evaluation. Second, today's Russia mostly has regionally-oriented labor markets, so many researchers believe that internal, territory-related growth factors-human capital, historical traditions, and the cultural context of a given community—are crucial for regional economic development. Local labor markets are now assigned a much greater value than before, since the "right" educational structure of the population is one of the key long-term factors of economic growth [Belokrylova 2006; Zink-ovsky 2007; Pilyasov 2007; Chelnokova, Firsova 2013].
The increased importance of higher education for regional development and industrial growth puts the need to develop adequate university performance assessment criteria on the agenda. Widespread assessment methods based on university resources inhibit effective positioning and use of higher education potential. Assessment of resources as a goal in itself should be replaced by assessment of performance and effectiveness of higher education, where resources
should be treated as a means of achieving the desired result. Organization of university assets, including human resources, normally mirrors university effectiveness.
The role of higher education in regional and industry-specific development depends on the quality of its products. The structure of its potential and social expectations determine three types of products: (i) new knowledge as a result of research and development activities; (ii) inventions in technology, engineering and humanities; participation in regional and industry-specific programs; (iii) the training of skilled personnel and keeping the skill levels of employed professionals up-to-date.
Each of the products has groups of potential consumers: the academic community, businesses and authorities at different levels, and undergraduate candidates. The extent to which a university engages to meet consumers' needs determines how demanded its products will be. Attractiveness may be taken as a basis of integral university characteristics. We suggest assessing universities based not on their organizational structure or resources but on the outside customer's demand for their products—basically, on their performance. This being understood, we assume that a good organizational structure provides an adequate quality of products, i. e. product quality measures the quality of the producing institution. Assessment logic like this brings us much closer to determining the actual role of a university in the regional economy and in industry-specific development.
The principles of university assessment proposed in this article will be used to provide a university ranking. From now on, we will only focus on university attractiveness, i. e. the demand for university products on the part of the market segments concerned.
1. How to measure Higher education is designed to ensure production and capitalization university of skilled labor and intellectual products (research and development, attractiveness consulting, engineering support for companies and enterprises in different sectors), as well as to keep the skill levels of employed professionals up-to-date. Universities act as fully-fledged participants in education and intellectual property markets, competing for the customer whose demands determine university attractiveness.
Competitive factors can be of many different types: human capacity, material resources, and other operational conditions. We suggest analyzing only university performance, i. e. attractiveness of university products, as a competitive factor, assuming that operational conditions are always sufficient to provide the results observed. This is an integral criterion that indirectly considers the traditional factors of organizational structure and resources. Therefore, the proposed university assessment approach takes into account the following characteristics of university activities:
1) demand for university graduates (education products) in the real economy;
2) economic demand for design, development and consulting services rendered by the university;
3) the academic community's demand for the university's research products.
We assess these characteristics of university activities based on the following criteria:
• Attractiveness of university graduates in the labor market is measured by employer demand, i. e. how often graduates are employed upon a company's request, including the number of employer-sponsored education contracts.
• Attractiveness of a university's research and development products for real sector companies and enterprises is measured by the university's income from the sale of research and technology, projects and services designed to support companies and organizations' activities, as well as professional education programs.
• Academic attractiveness of research and development results asserted in scientific publications can be measured by the citation index.
To be able to use the assessment criteria, we need to introduce adequate and measurable indicators, which should meet the following requirements:
• be objective, i. e. based on external evaluations or information provided by a company whose officials are held responsible;
• be quantitative to ensure verifiability;
• be based on uniform statistical reporting procedures to ensure a valid data comparison.
• Such indicators may include:
• the percentage of graduates from full-time programs assigned to jobs;
• the proportion of income from R&D and educational services rendered to third-party organizations;
• citation indices of university authors' works.
University rankings are designed to achieve a specific objective and are for the attention of a specific consumer segment. By trying to bring various aspects of university activities together, ranking designers usually increase the number of indicators [Arzhanova et al. 2013; Zavarykina, Lopatina, Perfilyeva 2012]; as a result, indicators related to different assessment criteria eventually balance each other out in their final values. We presume that the number of indicators should be kept to a minimum, provided that they reflect the selected assess-
ment criteria—those of university product attractiveness—to the maximum possible extent.
Let us now dwell on the newly introduced criteria and indicators.
Criterion 1: Employers' demand for university graduates.
Employer's interest in university graduates is measured by businesses' requests. Most important here is not how many graduates get employed and by which means, but how actively they are demanded in the real economy. It is not employment as such that we assess, but targeted demand for professionals, which determines the "utility" of a university, i. e. the need for its products. University is regarded and perceived as a tool that can be used to enhance business productivity and contribute to territorial development.
Employers' demand is determined by the percentage of graduates assigned to jobs in the total number of university graduates from the main full-time programs.
The proposed indicator has two important characteristics. First, it independently measures and, more importantly, describes the demand for graduates through the majors offered by universities in response to businesses' requests. Second, it describes contractual relationships between the educational organization (students) and enterprises, institutions and organizations1. Job assignments are requested by organizations, which include employer-sponsored education contracts. It is assumed that graduates from additional, part-time, extramural and distance education programs are employed already.
Federal Statistical Monitoring BnO-1, paragraph 2.9 'Job Assignments for Graduates from Full-Time Education Programs Funded from Budgets of All Levels" may serve as a source of information for quantitative indicator assessment.
Criterion 2: Commercialization of a university's end product.
The commercialization degree indicator is the proportion of income from commercialization of research and development products, professional training programs for organizations, and consulting services in a university's budget. The indicator considers two sources of university revenue: (i) income from research and development at the expense of all possible sources of financing; and (ii) funds received from organizations (not individuals) for educational services provided. This indicator is measured using Federal Statistical Monitoring BnO-2, paragraph 3.1. "Distribution of Institution's Funds by Sources and Activities". This federal statistics provision allows for measuring objectively the demand for a university's research and technology products and its engagement in keeping the skill levels of employed professionals up-to-date. Basically, this group of products constitutes the ma-
1 Instructions for completing the federal statistical monitoring form (as amended by Rosstat Order No. 598 of October 2, 2014).
jor portion of a university's participation in the regional economy and territorial development.
The indicator is defined in the statistical monitoring form as "the specific weight of income from research and development and educational activity at the expense of extra-budgetary funds provided by organizations in the total amount of institution's funds actually received from budgets of all levels, extra-budgetary sources, and proprietary funds ".
Criterion 3: Academic attractiveness of a university's scientific product.
The role of scientific product is played by new knowledge as the result of scientific research. Demand for such product can be measured by the citation indices of university authors' works. Information is provided by the publication activity metrics of the Russian Science Citation Index (RSCI), chapter on "Comparing Bibliometric Indicators of Organizations": /'-index of publication activity for each university, h-index, and the total number of citations of a university's publications in the RSCI.
The RSCI gives a quite comprehensive and objective picture of the publication activity of Russian authors and scientific institutions. The RSCI database contains information on both Russian-language publications and Russian publications in foreign languages as well as journals that have English versions. In assessing publication activity and the citation impact of Russian researchers and scientific institutions, the RSCI uses information on publications of Russian authors and works that cite them contained in the Scopus international citation database, making it possible to consider not only publications in RSCI-indexed Russian periodicals but also publications of Russian authors in foreign journals2.
The advantage of the proposed approach to university performance assessment is that it does not require making direct requests to concerned institutions, thus helping to avoid subjectivity.
Having assessed the demand for university products, we can make a ranking of universities based on their attractiveness in specific industries. Such a ranking will allow prospective consumers of higher education services to understand their chances of getting an education that will make them competitive in the regional or national labor market, the expected effectiveness of attracting the university to solving development problems of companies, economic sectors and territories, and the feasibility of their hopes for new scientific research results and the training of academic researchers.
A ranking like this will provide structured information to those choosing a university in order to solve their own problems; it is designed for those who regard universities as a resource of their own success.
2 http://elibrary.ru/proiects/citation/cit index.asp
Therefore, the ranking objectives are determined to meet the requirements of the key segments of consumers of products and services provided by universities, i. e. they ought to:
• provide concerned individuals and companies with information on university opportunities for training professionals capable of working in a real competitive economy;
• inform businesses of a university's potential to solve their technology, organizational and personnel problems;
• inform the academic community and other concerned parties of a university's research productivity.
2. Popular Rankings of higher education institutions are constantly created in university Russia and on an international scale. The range of ranking criteria is rankings very wide3, since consumers interested in educational services are motivated by the most diverse factors when assessing or selecting a university. Different criteria classification and prioritization methods provide a diversity of rankings to meet the interests of different target groups.
Within the scope of assessment, rankings vary from generalized (the top universities in Russia) to consumer-customized (a ranking of Moscow universities by tuition fees).
The most well-known producers of Russian rankings include: Re-iTOR Independent Rating Agency, the Ministry of Education and Science of the Russian Federation, the Russian Rectors' Union, Business Russia Public Organization, Vladimir Potanin Foundation, Russia Today International News Agency (MIA Russia Today), Interfax, Echo of Moscow radio station, the Kommersant publishing house, Career magazine, Finance business magazine, SuperJob Internet recruiter, and others.
The Social Navigator project of MIA Russia Today (formerly RIA Novosti)4 in cooperation with the National Research University Higher School of Economics (NRU HSE) monitors around 500 higher education institutions based on the average USE (Unified State Exam) admission score. This monitoring is used to create university rankings with sub-rankings for different disciplines or university categories. The Social Navigator rankings take into account universities' reputations inherited from Soviet times, which is at odds with other local and global rankings, such as the Academic Ranking of World Universities5, the
3 The general idea of Russian and global ranking criteria is given in [Ivanova 2015].
4 RIA Novosti's Admission Quality Ranking of Russia's State Universities. http:// ria.ru/ratings academy/
5 Conducted by the Center for World-Class Universities (CWCU) of Shanghai Jiao Tong University. http://www.shanghairanking.com/ru
Times Higher Education World University Rankings6, or the QS World University Rankings7.
Forbes L/fe magazine offers a ranking of universities that produced the highest number of the top 200 Russian Forbes billionaires8. The ranking is based on the cumulative wealth of billionaire alumni.
In 2011-2013, the National Personnel Training Foundation (NPTF)9 assisted by experts from the Center for International Comparative Research (Institute of International Organizations and International Cooperation, NRU HSE) implemented a project called Development and Testing of Professional Education Institution Ranking Methodology. This multidimensional ranking methodology allows the comparison of universities based on either cumulative data (aggregates) or individual parameters and activities.
The Ministry of Education and Science of the Russian Federation has monitored the effectiveness of Russian education institutions since 201310. This Monitoring explores a series of indicators grouped into eight areas: education, research and development, international outlook, financial and economic activity, faculty salaries, employment policy, students enrolled, and additional educational institution indicators. The Monitoring is designed to provide information support for managerial decisions and has brought the notion of "ineffective university ranking" into use.
All the above mentioned rankings have the following characteristics, which are both advantages and limitations:
• The multidimensionality and comprehensiveness of the indicators is achieved by considering different aspects of university activities.
• The lack of statistical information available is compensated for by direct requests to universities subject to ranking.
• Weight coefficients of the indicators (and their distribution across the groups of ranking factors) are normally assigned based on expert opinions, whereas the methodology requires a substantial justification of weight assignment (within the model tested).
• In the absence of necessary quantitative data, interrogatory methods are used to collect information from applicants, students and graduates.
• Nearly all of the rankings include indicators of university authors'
6 One of the oldest and most prestigious rankings. https://www.timeshigher-education.com/world-university-rankings/2015/world-ranking#!/page/0/ length/25
7 A ranking by British company Quacquarelli Symonds (QS) assessing universities in 20 emerging economies of Europe and Central Asia.
8 http://www.forbes.ru/forbeslife/obrazovanie-i-karera
9 http://ranking.ntf.ru/
10 http://indicators.miccedu.ru/monitoring/
publication activity. Meanwhile, using these indicators to compare different types of higher education institutions is not entirely flawless: as we know, citation indices (h-index and other publication activity indicators) depend largely on the field of research11.
3. Using university product attractiveness indicators to create rankings
The selected university assessment indicators are used in a number of Russian and global rankings, in different combinations with other indicators. However, none of the famous rankings includes a set of indicators as a characteristic of outside consumer's demand for university products. The indicators used in rankings are more likely to describe university processes or resources than performance. Even when some indicators consider university effectiveness parameters, they are not regarded from the attractiveness perspective and do not measure the outside demand for this effectiveness. When these indicators are used in combination with process-related ones, their contribution to university assessment is leveled off in the final ranking.
3.1. Citation index
Most Russian and global rankings use bibliometric indicators of university authors' publication activity. They use research productivity indicators, e. g. the number of articles published in reputable peer-reviewed journals (The Academic Ranking of World Universities) or the number of papers per faculty member (the THE World University Rankings), as well as citation indicators, e. g. the average number of citations per article (the THE World University Rankings) or citation indices per faculty member (the THE World University Rankings), most often combining both types.
For instance, the annual Monitoring of Universities conducted by the Ministry of Education and Science of the Russian Federation uses the following bibliometric indicators: the number of citations received by articles published in the previous five years and indexed in Web of Science per 100 faculty members; the number of citations received by articles published in the previous five years and indexed in Scopus per 100 faculty members; the number of citations received by articles published in the previous five years and indexed in the RSCI per 100 faculty members; the number of university publications indexed in Web of Science per 100 faculty members; the number of university publications indexed in Scopus per 100 faculty members; the number of university publications indexed in the RSCI per 100 faculty members; and the overall number of university publications per 100 faculty members (among 62 other university performance indicators).
Popular citation indices, like h-index, have a number of disadvantages as research productivity indicators, being, for example, depend-
https://m.wikipedia.org/wiki/H-%D0%B8%D0%BD%D0%B4%D0%
B5%D0%BA%D1%81
ent on the field of research. Meanwhile, /-index is less associated with the field of research, assessing the demand for the university's (not an author's) publications in most diverse areas available. In fact, this indicator measures the number of acknowledged (highly cited) researchers in a given university.
3.2. Income from R&D and educational services rendered to third-party organizations
University ranking indicators also assess the research performance of universities quite often, which is usually externalized in the "proportion of income from R&D in the overall income of the educational institution". However, this reasonably clear indicator accretes a number of additional ones: the proportion of R&D completed using a university's own resources (with no co-contractors involved) in the university's overall income from R&D; the income from R&D (except that from budgetary funds of the Russian Federation and national science foundations) per faculty member; the number of license agreements; the proportion of funds obtained by educational institutions from using their intellectual products in a university's overall income (based on the Monitoring conducted by the Ministry of Education and Science of the Russian Federation). Each of these indicators carries a certain weight in determining the final ranking. It is no surprise that income from a university's intellectual products as such is thus compensated by and dissolved in a number of related indicators.
R&D performance indicators in popular university rankings are interpreted as "scientific achievements" but not as the demand for (return on) intellectual products. At the same time, another aspect of intellectual product attractiveness—employee retraining services for real economy enterprises—is considered in the Education Activity or Extra-Budgetary Educational Services sector, along with fee-based educational services provided to the population.
3.3. Assigning In the abovementioned rankings, assigning graduates to jobs is un-graduates to jobs derstood as engaging them in labor relations and making them part of the economy. The abundant indicators of employers' demand for university graduates feature no such familiar Federal Statistical Monitoring factor as "assigning graduates to jobs upon employers' requests". In particular, the only indicator used in the Ministry's Monitoring is "the proportion of graduates employed within one calendar year after graduation in the overall number of university graduates from the main full-time programs", calculated based on the Pension Fund data (i. e. it does not consider whether a graduate is employed in their field of study or not).
Therefore, job assignments for graduates cannot be regarded as a pronounced employer demand for university professionals.
4. Objects of Qualities assessed using the ranking criteria have different origins in ranking different fields because the specific features of a university's product
determine its market and capitalization conditions. So, to be able to correctly compare the university performance indicators from different categories, we need to group universities based on similar or related types of activities.
We suggest dividing universities into six categories:
• Classical universities;
• Engineering (technical) universities;
• Agricultural universities;
• Management universities (economics, finance, law);
• Universities in humanities (pedagogy, philology);
• Medical universities.
With a classification like this, we minimize the number of field overlaps between the categories. The proposed breakdown of universities into categories based on their major fields of study is consistent with classifying publications based on areas of research in the RSCI database: engineering, natural sciences, medicine, agriculture, social (including pedagogical) sciences, and humanities. A similar classification was used to develop and test a multidimensional ranking model under the NPTF project12.
At the same time, the proposed system of assessment criteria is essentially invariant for different types of universities: attractiveness serves as a strategic goal achieved via markets of relevant products. This observation may be used to rank universities regardless of the above classification or to introduce another classification based on other significant parameters.
The sample does not include branches (local subdivisions) of higher education institutions, military colleges, extramural and distance education institutions, theological (divinity) schools, and universities of art and culture—the criteria for assessment of their performance differ from those proposed in this article.
We also omit institutions with no necessary information available in the Federal Statistical Monitoring BnO-1 or BnO-2, as well as those not indexed in the RSCI.
5. University We present the results of using the described indicators to create uni-
rankings based on the selected assessment criteria
versity rankings based on the data obtained by MIA Russia Today with our participation in 2015. All in all, we ranked 463 universities from 80 subjects of the Russian Federation (Table 1).
The distribution of universities based on the indicator "the proportion of graduates assigned to jobs in the overall number of graduates from the main full-time programs" indicates a pretty high demand for
12 http://ranking.ntf.ru/DswMedia/larionovamarinaplenary1 24042013.pdf
Table 1. The distribution of universities based on their main fields of study
Classical universities Í 87
Engineering (technical) universities ! 140
Agricultural universities | 56
Management universities (economics, finance, law) i 61
Universities in humanities (pedagogy, philology, physical education and sports) Í 72
Medical universities | 47
Table 2. The proportion of graduates assigned to jobs across different university categories
Type of university Average (%) Min. (%) Max. (%) No. of observations
Classical universities 55.8 0 100 87
Engineering (technical) universities 67.4 0 100 140
Agricultural universities 75.6 3 100 56
Management universities (economics, finance, law) 30.0 0 100 61
Universities in humanities (pedagogy, philology, physical education and sports) 56.4 0 100 72
Medical universities 41.1 0 100 47
All universities ranked 56.9 0 100 463
university graduates in the labor market (Table 2). The proportion of graduates assigned to jobs is the highest in agricultural universities, reaching on average three quarters (75.6%) of all graduates. Graduates from management universities appear to be the least demanded by employers (about 30% on average).
The number of job assignments provided to university graduates reflects the market demand for graduates of specific higher education institutions.
Table 3 shows the distribution of universities based on the indicator "the proportion of income from commercialization of R&D products and professional training programs for organizations in a university's budget". Not unexpectedly, engineering universities demonstrate the highest proportion of income from intellectual product commercialization, which is about 16% of their budgets. Lower values are observed in medicine and humanities-8.3% and 8.4%, respectively. Agricultural universities show the lowest proportion of all.
Only a small proportion of universities (from 3% to 15% in different categories) receive no income from selling R&D products or render-
Table 3. The proportion of income from commercialization of R&D products and professional training programs for organizations in a university's budget
Type of university Average (%) Min. (%) Max. (%) No. of observations
Classical universities 10.8 0 79 87
Engineering (technical) universities 16.0 0 52 140
Agricultural universities 6.6 1 15 56
Management universities (economics, finance, law) 11.1 0 82 61
Universities in humanities (pedagogy, philology, physical education and sports) 8.4 0 57 72
Medical universities 8.3 0 22 47
All universities ranked 11.3 0 82 463
Table 4. Mean /-indices of citation of university authors' publications across university categories
Type of university Mean /'-index Median i Min. í (%) Max. (%) No. of observations
Classical universities 11.11 10 4 40 87
Engineering (technical) universities 9.85 9 4 29 140
Agricultural universities 7.57 7 4 17 56
Management universities (economics, finance, law) 8.69 8 3 24 61
Universities in humanities (pedagogy, philology, physical education and sports) 7.35 7 2 17 72
Medical universities 10.49 10 5 23 47
All universities ranked 9.33 9 2 40 463
ing educational services to organizations. Over half of the universities in each category receive a relatively low income from these activities (5-10% in the overall university budget). About 15-20% of agricultural and medical universities deal with this type of revenue once in a blue moon. Proportions of over 25% in the overall university budget are mostly demonstrated by engineering universities.
Attractiveness of university products based on "the citation index of university authors' works" is assessed using the organization's i-index. Highly cited researchers are a gauge of university research performance. The mean i-index values across university categories are presented in Table 4.
Naturally, the citation index is not a perfect instrument. Citation indices (/'-index and h-index) have specific patterns in different areas of research—an issue that has been constantly reported by researchers13.
However, citation indices of the universities subject to ranking do not differ too much depending on the university category. In our opinion, the lack of significant differences in the discussed indicator between the university categories in our sample (unlike the differences based on the fields of study) is explained by the diversity of research and development areas in university activities (the process of "univer-sitization").
The distribution of universities based on citation indices offers quite a comprehensive picture of the characteristics of the demand for scientific publications across the predetermined university categories. Most universities in agriculture and humanities show the maximum /-index values of 6-7 highly cited researchers, as compared to 8-11 in medical universities and a broader range of 6-7 to 10-11 and even 12-13 in engineering and management universities. The highest values of 8-9 to 12-13 highly cited researchers are demonstrated by classical universities.
To construct the final ranking, calculated as the sum of all indicators used, we normalize the obtained indicator values to ensure data comparability.
When creating complex rankings, it is essential to use weight coefficients indicating the priority of specific indicators. When intermediate (particular) values of ranking indicators are summed up, the weights of these indicators are considered equal. In other words, the cumulative consumer's demand for a university's products is assessed indiscriminately for all the three indicators introduced.
The complete ranking of Russian universities based on the demand for their products is available on the website of the Social Navigator project14.
Classical, engineering and agricultural universities show the closest distribution of their final indicators. However, positive extreme values reveal some universities that stand out. Universities in management, humanities and medicine are less homogeneous in their final rankings, but they also feature some exceptionally successful institutions, whose high attractiveness manifests itself in the consolidated indicator.
The proposed criteria describing the demand for relevant university activities and products have different weight coefficients in different universities. This is a result of differences in university policies shaped
13 Напрммер: http://eqworld.ipmnet.ru/ru/info/sci-edu/Polyanin IndexH 2014. html
14 http://ria.ru/abitura rus/20151215/1341953336.html
by outside requests and the extent to which they are realized. "Globalization will continue to gather pace, but what we've seen recently, as in other markets, is the growing impact of technology, which threatens many components of the traditional university" [Barber, Donnelly, Riz-vi 2013:187] as well as the areas of demand for universities.
References Arzhanova I., Larionova M., Zavarykina L. et al. (2013) Modelnaya metodologiya mnogomernogo ranzhirovaniya rossiyskikh vuzov [Model-Based Methodology of Multidimensional Ranking of Russian Universities]. Vestnik mezhdunar-odnykh organizatsiy: obrazovanie, nauka, novaya ekonomika, no 1, pp. 8-30. Barber M., Donnelly K., Rizvi S. (2013) Nakanune skhoda laviny. Vysshee obrazovanie i gryadushchaya revolyutsiya [An Avalanche Is Coming. Higher Education and the Revolution Ahead]. Voprosy obrazovaniya/ Educational Studies. Moscow, no 3, pp. 152-229. Belokrylova O. (2006) Vzaimodeystvie biznesa i vlasti v protsesse formirovaniya strategii ekonomicheskogo razvitiya regionov [Business-Government Cooperation in Developing a Regional Economic Advancement Strategy]. Prob-lemy formirovaniya gosudarstvennykh politik v Rossii [The Problems of National Policy Development in Russia], Moscow. Chelnokova O., Firsova A. (2013) Tipologiya podkhodov k analizu vliyaniya uni-versitetov na innovatsionnoe razvitie regionov [A Typology of Approaches to Analysis of University Influence on Regional Innovation Development]. Izvestiya Saratovskogo universiteta. Seriya: Ekonomika. Upravlenie. Pravo, no 13, pp. 577-583.
Ivanova A. (2015) Rossiyskiy opyt sostavleniya reytingov vuzov [The Russian Experience of University Rankings]. Vestnik ChGPU im. I. Y. Yakovleva, no 1 (85), pp. 129-136. Available at: http://vestnik.chgpu.edu.ru/upload/ docs/2015/1%2885%29 2015.pdf (accessed 10 October 2016). Kogan E., Postalyuk N. (2008) Novaya organizatsiya resursov vysshey shkoly: otvety na vyzovy ekonomiki [The New Organization of Higher Education Resources: Solutions to Economic Challenges]. Vysshee obrazovanie segod-nya, no 6, pp. 8-12.
MIA Russia Today (2015) 2015 Employer Attractiveness Ranking of Russian Universities. Available at: http://ria.ru/abitura rus/20151215/1341953336.html (accessed 10 October 2016). Pilyasov A. (2007) Rynki truda v epokhu perekhoda ot industrialnoy ekonomiki k ekonomike uslug [Labor Markets in the Era of Transition from Industrial to Service Economy]. Otechestvennye zapiski, no 3, pp. 230-258. Zavarykina L., Lopatina A., Perfileva O. (2012) Sravnitelny analiz mezhdunarod-nykh metodologiy ranzhirovaniya vysshikh uchebnykh zavedeniy [Comparative Analysis of International University Ranking Methodologies]. Vestnik mezhdunarodnykh organizatsiy, vol. 7, no 1 (36), pp. 70-121. Zinkovsky Y. (2007) Vzaimodeystvie rynka truda i regionalnoy sistemy profes-sionalnogo obrazovaniya [Relationship Between the Labor Market and Regional Vocational Education Systems]. Metodologiya upravleniya kachest-vom i ustoychivym razvitiem ekonomicheskkh sistem [The Methodology of Managing the Quality and Sustainable Development of Economic Systems] (ed. A. Babkin), St. Petersburg: Peter the Great St. Petersburg Polytechnic University.
