CC BY
11
-□ □-
Пропонуеться метод комплексного оцтювання результатiв дiяль-ностi суб'ектiв освттх середовищ, зокрема вищих навчальних закла-дiв, на основi розрахунку узагальне-ного об'ему т-симплекса, вершинами якого е оцтки дiяльностi вищих навчальних закладiв за рiзними кате-горiями. Побудовано перелш категорш та здшснено видбпр показнитв до цих категорш. Метод верифжовано в розробленш тформацшно-аналтич-нш системi. Здшснено дослидження методу на чутлив^ть до змти оцток категорш та динамшу змти комплексноI оцтки
Ключовi слова: т-симплекс, рейтинг ВНЗ, оцтка суб'екта освтньо-го середовища, детермтант Келi-
Менгера
□-□
Предлагается метод комплексной оценки результатов деятельности субъектов образовательных сред, в частности высших учебных заведений, на основе расчета обобщенного объема т-симплекса, вершинами которого являются оценки деятельности высших учебных заведений по различным категориям. Построен перечень категорий и осуществлен отбор показателей к этим категориям. Проведено исследование метода на чувствительность к изменению оценок категорий и динамику изменения комплексной оценки. Метод верифицирован вразра-ботанной информационно-аналитической системе
Ключевые слова: т-симплекс, рейтинг ВУЗа, оценка субъекта образовательной среды, детерминант Кэли-Менгера
-□ □-
UDC 005.8
|DOI: 10.15587/1729-4061.2018.126287|
THE METHOD FOR EVALUATION OF EDUCATIONAL ENVIRONMENT SUBJECTS' PERFORMANCE BASED ON THE CALCULATION OF VOLUMES OF M-SIMPLEXES
А. Kuchansky
PhD, Associate Professor Department of Cybersecurity and Computer Engineering*
E-mail: kuczanski@gmail.com Yu. Andrashko Lecturer
Department of System Analysis and Optimization Theory State Higher Educational Institution «Uzhhorod National University» Narodna sq., 3, Uzhhorod, Ukraine, 88000 А. Biloshchytskyi Doctor of Technical Sciences, Professor, Head of Department**
O. Danchenko Doctor of Technical Sciences, Associate Professor, Head of Department Department of Business Administration and Project Management University of Economics and Law "KROK" Laherna str., 30-32, Kyiv, Ukraine, 03113 0. 11 a r i o n o v PhD, Associate Professor Department of Intellectual and Information Systems***
I. Vatskel Postgraduate student** T. Honcharenko Senior Lecturer Department of Information Technologies* *Kyiv National University of Construction and Architecture Povitroflotsky ave., 31, Kyiv, Ukraine, 03037 **Department of Information Systems and Technologies*** ***Taras Shevchenko National University of Kyiv Volodymyrska str., 60, Kyiv, Ukraine, 01033
1. Introduction
The processes of radical changes in the economy and society that have arisen as a result of the global crises of 2008 and 2014 put forward new tasks for management activities in education. A significant birthrate decline in Ukraine in the late 1990s and early 2000s led to a significant decrease in the number of potential applicants, which in turn sharpened the competition between higher educational institutions (HEI) in the national market of educational services. In addition,
Ukraine is a member of the world educational process, which is why it is necessary to take into consideration that competition with foreign HEI is constantly growing. According to the research of the analytical center CEDOS, about 60 thousand Ukrainians studied at foreign higher educational institutions in the academic year of 2016-2017. According to preliminary estimates, the number of students may increase up to 68 thousand people in the academic year of 2018-2019 [1].
To ensure effective functioning of organizations in the educational sphere under modern conditions, it is necessary
©
first of all to improve the system of management of these organizations. This improvement is associated with the necessity of using modern management methodologies in the implementation of almost all kinds of activity: scientific, educational, organizational, etc.
Most leading scholars believe that the system of higher education should be self-controllable and self-regulating. Self-control of the system of higher education involves minimizing all kinds of centralized administrative influence of the state on the activity of universities, development of competitive principles in the field of higher education, that is, a transition to selective priority funding of HEI depending on their rating. Self-control of educational systems is described in more detail in [2].
The subject of educational environment (SEE) implies universities, structural units of HEI: institutes, faculties, departments, academic and teaching staff of these units, as well as groups of scientists, united by their involvement in certain projects. The known methods of SEE performance evaluation have a number of shortcomings. Specifically, the convolution method requires correct selection of weight coefficients, which can appear a difficult task. Each SEE a priori understands the directions, which have positive results, and will insist on taking these directions into account in convolution with the maximum coefficient. That is, for selection of such a system, it is necessary to reach a consensus of all SEE on the choice of weight coefficients. The ideal point method requires constant refinement of the ideal point and the use of expert evaluation, associated with manifestation of the subjective factor in comprehensive evaluation.
Development of a new method for comprehensive SEE performance evaluation, which can serve as an effective tool for implementation of control of universities and departments, is relevant. An important feature of this method must be the possibility of automation for application in HEI control systems. The known methods of SEE performance evaluation need regular intervention of the subjective factor in the calculation process, agreement of experts' opinions, and a change in the system of coefficients when changing evaluation priorities, which is difficult to automate. The merit of the study is development of the method for SEE performance evaluation, which is easy to automate, does not require involvement of experts, selection of weight coefficients, the ideal point and solution of additional problems in construction of a comprehensive score.
2. Literature review and problem statement
There are several world-wide techniques for estimation of HEI performance. The most popular among them are World University Rankings, which is described in [3]. This ranking is formed by Quacquarelli Symonds (also known as QS ranking). Only 5 best universities of Ukraine are represented in the QS ranking as of 2017, which does not enable us to use it for comparing all Ukrainian universities. But QS technique cannot be used as a whole within one country. QS ranking gives diversified evaluation of results of various activities of a University. Most of the parameters that are taken into account when calculating ranking are objective and can be obtained from open sources. Some of the indicators are subjective in nature, specifically, the reputation index. To obtain the indicators, it is necessary to conduct a
survey of a sufficient number of qualified experts, which is a complicated task.
Another rating is the Academic Ranking of World Universities or the Shanghai rating, which is described in [4], published by Shanghai Jiao Tong University. The Shanghai ranking focuses, first of all, on the results of scientific activity of a university. Only the universities, the graduates of which are Nobel or Fields prize laureates get to Shanghai rating. Such universities are limited in number, which does not make it possible to use the technique at the Shanghai ranking for comprehensive evaluation of all universities. However, the technique can be effectively used for evaluation of research and educational institutions.
The aspect of HEI being represented in the WEB-space is estimated using the methodology of Ranking WEB of Universities [5] (known also as Webometrics). Together with the international ratings of HEI performance, there are national ratings, specifically, the information educational resource osvita.ua, based on its own methodology, forms the University ranking "Top-200 Ukraine" each year [6]. Each procedure takes into account the indicators, which are characteristic only for it, in the rating calculation. For example, the academic reputation of the university and reputation of its graduates among employers are taken into consideration when compiling QS-rating. The methodology "Top-200 Ukraine" takes into consideration the volume of investments, made by private, high-tech business in startups of universities.
Papers [7, 8] show that a common feature for the above methodologies is determining performance of SEE and specifically of HEI based on several groups of indicators. In particular, each of the explored ratings takes into consideration the following groups of indicators:
- the number of scientific publications of employees of universities that are indexed in science-metric bases (Scopus, Nature, Science, etc.);
- citation indices (SCIE-Science Citation Index - Expanded, SSCI - Social Science Citation Index, etc.);
- qualitative composition of the HEI staff, including a number of professors, awards laureates, involvement of foreign lecturers and researchers, etc.
Traditional bibliometric indexes are often used for calculation of the mentioned indicators. Paper [9] describes the method of the h-index calculation. Article [10] proposed to use the so-called g-index. The drawbacks of these methods include the fact that these methods partially lose information about citations of publications. Paper [11] describes the shortcomings of h- and g-indices and proposes the use of e-index to eliminate these shortcomings. However, e-index does not fully solve the problem of loss of information on publications citing.
Study [12] contains the methods of construction of scalar and vector evaluations of scientists in terms of their research activities. The ideal point method for construction of the vector evaluation is described in [12]. However, for application of this method, it is necessary to correctly select the point, the coordinates of which are scientific performance scores of scientists, the best in terms of achieving maximum efficiency or effectiveness according to a certain criterion. It is a complicated task. The method of transition from qualitative HEI performance evaluation to quantitative evaluation was proposed in [13]. The disadvantage of this method is the need to involve experts to determine the qualitative scores.
The problem of choice of a college by students was described as a problem of multi-criteria decision making in paper [14]. The adaptive method of decision making based on the ideal point method was proposed to solve this problem. A set of indicators, which is used for evaluation of engineering colleges, was also proposed. However, the methods for finding or specifying the ideal point are not considered in the paper. Some indicators, such as convenience of location, may not be used for SEE performance evaluation. The methods for expert performance evaluation of economic schools were proposed in article [15]. The main difficulty of using the proposed methods is the need to attract a large number of competent and unbiased experts. In the paper [16], the ABC model for scientific-research performance evaluation, which is based on three indicators, determining the number of scientific and methodical works, was constructed. However, the work does not take into consideration citing of publications. This model can not be used for evaluation of other SEE activities either.
Paper [17] considered the model for prediction and evaluation of the quality level of educational institutions, which makes it possible to make transition from evaluations of HEI to prediction of development prospects taking into consideration resources available. The method for prediction of potential of scientific research directions was described in [18]. The drawback of the method is the use for forecast calculation of current average, for which a separate problem of selection of smoothing parameters arises. Paper [19] describes combined prediction methods, which take into consideration selective comparison with a model that is a fixed segment of the time series. In contrast to the method that is described in [18], this method is less sensitive to selection of parameters; however, its adaptation to the mechanism that generates the temporal series of potentials of development of scientific directions is required. Article [20] offers adaptive combined models of prediction of temporal series taking into consideration the results of identification of similarities in retrospection of these time series. In paper [21], the method of construction of fuzzy expert evaluations that can be used for the problem of prediction of potentials of scientific directions development was considered. However, a separate difficult problem in this method is selection of experts. Research [22] proposed the method for identification of scientific research directions for scientists based on cluster analysis of scientific publications, which is a preparatory stage for the problem of prediction of development of potential of research directions.
The main drawback of traditional techniques for performance evaluation of SEE and specifically HEI is overloading with lots of forms, formulas, ranking lists, etc. Specifically, in work [8], it was indicated that traditional techniques of HEI evaluation are a separate cumbersome kind of activity. In addition, the above techniques are primarily aimed at scientific performance evaluation, but it is not of less importance to take into consideration other characteristics of the university activity, such as academic, organizational, international, etc. It is essential that these characteristics could be estimated based on the objective data that can be obtained from public sources. This will allow automation of the HEI evaluation and decrease the need for involvement of experts.
3. The aim and objectives of the study
The aim of present study is to develop an efficient and flexible method for comprehensive SEE performance evaluation.
To accomplish the aim, the following tasks have been set:
- to develop a method for comprehensive SEE performance evaluation, based on calculation of volumes of m-sim-plexes, using selected indicators, which reflect the major aspects of SEE performance;
- to explore the developed method as for sensitivity to changes in overall scores in categories of indicators and the dynamics of a change in comprehensive scoring.
4. General description of the method for construction of comprehensive performance evaluation of educational environment subjects
Let K0, Kj,..., Km be the categories that reflect different aspects of activity of SEE S, specifically HEI. Each category of indicators determined a certain criterion for SEE performance evaluation. Let us designate through n^n,,...,^ the indicators that belong to category Ki, i = 0, m, where (m+1) is the number of categories, and ki is the number of indicators that belong to category Ki.
The stages of construction of a comprehensive performance score of SEE:
1. Determining indicators n1, n„..., nk, which belong to correspondent category Ki, i = 0, m. This information is derived from public sources that are presented in the Internet.
2. Finding performance score of a certain SEE S within time period T=[t0, t{), where t0 is the initial moment, t\ is the final moment. To do this, we will find numerical values of indicators of a subject for a correspondent period. We will designate through nj (5) the numerical value of indicator n of subject S for period T. Indicators nj (S) can be both absolute and relative. Some indicators, specifically the number of awards of academic and teaching staff should be normalized according to the number of all full-time teachers of HEI. Some of the indicators should be normalized by the number of university students. In general, it is a particular problem of the research and is not considered in this paper.
3. Construction of performance score of subject S by criteria Ki, for period T= [t0, tj). We will designate them through Qj (S) - performance score of subject S, found by criteria Ki, i = 0, m, for period T. The values of performance score of subject S by criteria Ki are calculated in different ways, depending on the method taken as a basis. For example, weight coefficients ra0,ro1v..,rok, such that ®j eR, j = 0,ki, are considered in the weighed score method. R is the set of real numbers. Coefficients oj, j = 0, ki reflect the importance of indicator nj during SEE performance evaluation, for which condition is satisfied
I ® j = 1.
j=0
Performance score of SEE is derived from the formula:
Qj (s)=i (s), (1)
j=0
where Qj (S) is the performance score of subject S, found by criterion Ki, i = 0,m, for period T=[t0, tj).
In the ideal point method, based on indicators nj (S), j = 0,ki, we will construct point FT(S)eRk' in (ki+1)-dimensional space, the number of dimensions of which is determined by the number of indicators, i = 0, m. We
will call ideal a certain point F* = (n0,n*,...,nk) of the (&;+1)-dimensional space, for which for any subject S from the totality of all the estimated subjects and arbitrary period T, condition is satisfied:
n > n (s ), b = 0, k.
(2)
It is necessary to find the distance between point FT and the ideal point F in order to assess scientific-research performance of subject S. Measure of proximity between the two points is determined based of some metric distance:
QT (S ) = p( FT (S ), F '
(3)
where p(FT (S),F*) is the Euclidean distance, the Minkowski distance, etc.
4. Calculation of comprehensive performance score QT (S) of subject S for period T=[t0, ti). In the method of weighed score and the ideal point method, comprehensive performance score of subject S is derived from formula:
QT (s ) = X wot (s ),
(4)
where QT (S) is the comprehensive performance score of subject S for period T=[t0, t1), wi, i = 0,m are the coefficients that reflect the importance of category Ki,
m
E w=1
i=0
Complexity of application of the method of weighed score and of the ideal point method is associated with the need to involve experts to determine coefficients oj, wi j = 0, k, i = 0, m, as well as to select the ideal point and a formula for finding the distance. In this work, the method, which has no specified shortcomings and is not dependent on the subjective opinion of people, who carry out the evaluation, was designed.
5. Analysis and the use of results SEE performance evaluation.
5. Research into features of the method for comprehensive evaluation of educational environment subjects based of calculation of volumes of m-simplexes
To evaluate SEE, a score in any category can be considered as a point in (m+1)-dimensional space. We will consider points Vi, i = 0, m, which are vertices of a m-simplex. A m-dimensional polytope, which is a convex shell of its m+1 vertices, is called m-simplex with vertices in points vi e Rm+1. That is, m-simplex is a set of points Am e Rm+1, for which the condition is satisfied:
Am =
A + l
Xe* =1
A(ei >0,i = 0,mU,(5)
where is some real numbers, e R.
That is, a0 (0-simplex) is the point in R, a1 (1-simplex) is the segment in R2, a2 (2-simplex) is a triangle in R3, a3 (3-simplex) is a tetrahedron in R4, a4 (4-simplex) is the pen-tachora in R5, etc.
It is possible to put each m-simplex in correspondence with numeric characteristic, which determines capacity of a part of the space, which is limited by the given m-simplex. We will call this characteristic a generalized volume of m-simplex and designate it through V(Am). For example, generalized volume of 0-simplex is equal to zero, V (A0 ) = 0, generalized volume of 1-simplex is equal to the length of segment [v0, vi],
F (a1 ) = V
V+v
Generalized volume of 2-simplex is the area of a triangle with vertices in points v0, vi, v2 in R3, which can be found from the Heron formula. To find generalized volume of m-simplex for an arbitrary number of points m, it is possible to use the Cayley-Menger formula. Details about the Cay-ley-Menger formula are described in [23].
Consider the values of SEE performance scores at moment T. Let each category Ki, i = 0, m be in correspondence with a point in (m+1)-dimensional space vi by rule
v0 =(QT (S ),0,0,...,0), Vi =(0,QT (S ),0,...,0),
Vm =(0,0,...,0,Qm ( S )),
(6)
where Qj (S) is the performance score of subject S, found by criterion Ki, i = 0,m, for period T=[t0, t1).
From the method of construction of points v; (formula (6)), it is obvious that the system of vectors that begin at the coordinate origin and finish in vi, is orthogonal and linearly independent. Such a system will set the Euclidean space.
If there is one category, by which a SEE score is set, as a result we will have 0-simplex as a point on the coordinate axis (Fig. 1). If there are two categories, 1-simplex as a segment between two points, which are defined based on evaluations of these categories, is constructed by rule (6). 1-simplex is shown in Fig. 2. If there are three categories, 2-simplex is constructed (Fig. 3), if there are four categories, 3-simplex is constructed (Fig. 4), if there are five, 4-simplex is constructed (Fig. 5). After that, generalized volumes of these m-simplexes are found: it will be the length of the segment for 1-simplex, the area of a triangle for 2-simplex, the volume of a tetrahedron for 3-sim-plex, at m>3 - a certain hyper-volume.
Fig. 1. Image of A0 or 0-simplex (point in R kY
Fig. 2. Image of a1 or of 1-simplex (segment in R2)
We will construct m-simplex with vertices in points Vi, i = 0,m and find its generalized volume from the Cay-ley-Menger formula. Let
' 0 1 1 1 • 1
1 0 d2 01 d2 ■ 02 " d02m
1 d2 "10 0 d2 • 12 • dl
1 d2 20 d2 21 0 • • dL
V 1 dm, dm1 dm2 • 0
(7)
Fig. 3. Image of a2 or 2-simplex (triangle in R3)
Then generalized volume of m-simplex is:
V (A m ) =
1
|(-1)m
2m (m !)2
where dij is the distance between points Vi and v-p i, j = 0, m, d2 = v2 + v2, M is the determinant of matrix Y.
Comprehensive performance score QT (S) of subject S for period T can be derived from formula:
QT (S )- V (A m ),
(9)
where V (A m) is the generalized volume of m-simplex, calculated from formulas (7), (8) with vertices in points (6).
Fig. 5. Image of a4 or of 4-simplex (pentachora in R5)
We will explore dynamics of a change in comprehensive performance score of SEE. To do this, we will find the time derivative of the generalized volume of m-simplexes:
dQT (S)_ dV (Am)
dT dT
(8) = (-1) m-1 y 2m(m!)2 ■tr y-1 d| dT, •
' 0 0 0 0 • 0 ^
0, m, 0 0 d01d01 d02d02 d0md0m
d 2 dT ' • • 0 0 d10d10 d20d20 0 d12d;2 • 0 • d1mdim d2md2m
V 0 dm 0dm 0 dm1dm 1 dm2dm 2 * 0 ,
(10)
(11)
where
V-V- + V V ■
d.. -- 11 11
V
v2+v2
and
V' =
dQj (S ) dT
dQT (5 ) -
In this case, -, i = 0, m are known and charac-
dT
terize the rate of a change in performance score of SEE of correspondent category Ki. If
Fig. 4. Image of a3 or of 3-simplex (tetrahedron in R4)
dV (A m )
dT
> 0,
comprehensive score QT (S) increases in time, in addition, the higher the value of the derivative, the quicker the increase. This means that SEE S activity has a positive direction of development. In case
dV (A m )
dT
< 0,
QJ (s ) = qJ (s )+e,
where QJ (S) is scores, changed by magnitude s>0. Point
(12)
n =(o,o,...,o,QJ (s ),o,...,o).
corresponds to score QT (S). Let us find the distance between points vi and Vj, in this case, we will consider point Vi taking into consideration a change in score QJ (S) from formula (12):
dj=#+2 ^(QJ (s ))2+(qt (s ))2 =
= № (S))2 + Q (S))2 + 2 QT (S)■ e + e2, (13)
since
e2 = o (e),
2-QT (S )■ e = O (e),
2-QT (S )■ e + e2 = O(e)
can be replaced and we can designate the left part of equality (13) through infinitely small e > 0, then (13) can be written down in the form of:
dj = № (s ))2 + (QT (s ))2
+ £,
(14)
d2 - d 2 =
ij ij\
(QJ (S))2 + (QT (S)) +ë-i(QT (S))2 + (QT (S ))■
= £. (15)
comprehensive score QT (S) decreases, the lower the value of the derivative, the quicker the decrease. In this case, activity of SEE S requires correction because it is determined by the negative tendency.
Matrix ¥ is inherently determined, so non-degenerated, i. e.
|Y|* 0,
and that means that an inverse matrix in formula (10) Y-1 exists.
Let us estimate sensitivity of the method for comprehensive SEE performance evaluation based on calculation of volumes of m-simplexes to a change in performance evaluation of subject S by criterion Ki - Qj(S), i = 0,m. To do this, we will consider a change in score Qj (S) in category Ki by a certain infinitely small magnitude s>0:
We find determinant of matrix ¥ taking into consideration a change in performance score of subject S. We will designate the correspondent matrix through Y:
y =
1
d 2
1
doi d2i
1 d2
1 dl
1 d 0 d \ •■■ d2. •■■ 0
m0 m1 mi
1
dt
d2m
(16)
where Y is the determinant of matrix Y.
We will find expansion of this determinant by elements of the i-th line:
№| = (-1)i
1 1 1 • 1
0 d10 d01 0 • •• dl • - d2 • d0m •• df 1m
4-1)0
d2
d(i+1)0
d(2-1)1
d2 "(i+1)1
d(2-1)i d(2+1)i
d(2-1)» d2
dm0 dm, • d2. mi
0 1 •■• 1 ••• 1
1 d01 • ■ ' doi • d0m
1 0 ••• dii • df 1m
+ (-1)i+1 4
1 d(2i-1)1 1 d2+1)1
1 dh 0 1
d(2-1)i d(2+1)i
d(2-1)» d(2+1)„
-1) dl
where dj is the distance with consideration of a change in
performance score of subject S.
Accordingly, modulus of the difference of squares of
distances d and d will be determined from the formula: ij ij
1
d 2
1 d(2-1)1 1 d(2+1)1
1
4
d1m-1
d2 d2
d(2+1)i "' d(2+1)(m-1)
m(m-1)
The determinant in each summands of the obtained sum will be expanded by the i-th column:
1
M = (-!)'
(-1) • d(i-i)0 "(¡-1)1
d2 d2
"(' + 1)0 "(' + 1)1
-(-1)'+
"0(,-1)
d2
a1('-1)
0
"2
a(,+1)('-1)
"1(i+1)
d2
u1('+1)
(-1)' • 1 d 1
d2 "m('-1) d2 "m(' + 1)
0 d2 01 -■ d2
d2 10 0 -■ d2 "1('-1)
d2 "('-1)0 d(2-1)1 0
d2 "('+1)0 d(W d2
dm,0 "21 -■ d2 "m('-1)
+1) - d(--1)» 0 " dp+1)„
0
(' + 1)
d2
d1('+1)
('-1)('+1) 0
m('+1)
K-1)'+"+1 • d?2 ' d(i-1)0 dM1
d2 d2
"('+1)0 "(' + 1)1
d2 d2
"(m-1)0 "(m-1)1
1(m-1) d2
" 1(m-1)
d 2 d2
"(' + 1)(m-1) d2
"m(m-1)
-("1)' +
1
d2
a0('-1)
0
d2
"MM 0
d2 d2
"(m-1)('-1) "(m-1)(' + 1)
Mflm
"(2+1)m
M-1)m
J2 J2
1 "('-1)0 "('-1)1
1 d2 d2
"(' + 1)0 "(' + 1)1
1 d2 d2
1 "(m-1)0 "(m-1)1
1
d2
d0( '-1)
0
d2
('+1)
('-1)('+1) 0
d2 d2
d(m-1)('-1) "(m-1)(' + 1)
1
d2
d0(m-1)
d2
"("-1)
d2
"(,+1)(m-1) 0
Similarly, we will expand determinant and find the difference of determinants . To do this, we will group
similar summands, and take into consideration equality (15), then:
M-M=(-1)'
0+(-1)'+
+2(-1)'+
(-1)'+
(-1)'+
1 1 - 1
d2 "10 0 d2 "1C-1)
d2 "('-1)0 d2 "('-1)1 0
d2 "('+1)0 d2 "('+1)1 d2
"m0 " dm('-1)
1 1 1
d10 d12p-1) d12p+
"(■-1)0 20 d(2-1)(
d2 "('+1)0 d(i+1)('-1) 0
d^0 dm('-1) dm,+
1 1 1
I2 10 0 d12p-1)
M+1)
"MM 0
m(' + 1)
('-1)m d2
0
d(2-1)0 d(2-1)1 - 0
d(2+1)0 d(2+1)1 ■■ d(2+1)('-1)
dm, - ■■ d2 "m('-1)
1
d1(m-1)
d(L)(m-1)
d2
"(' + %-1)
dm (m-1)
l1('+1)
0
m(' + 1)
1
d1(m-1) d(2-1)(m-1)
d(2 + 1)(m-1) d2
"m(m-1)
K-1)"
d2 d2
"('-1)0 "('-1)1
d2 d2
"('+1)0 "('+1)1
+ (-1)'
' + m + 1 — . "2 d2
E • 1 "('-1)0 "('-1)1
1 d2 d2
"('+1)0 "('+1)1
1 d(m-1)0 "(2m-1)1
1
d 2
"0('-1)
0
d2
0('+1)
"('-1)('+1)
-1('-1)('+1)
-i('+1)('-1)
d(2m-1)('-1) d(m-1)(' + 1)
1 1 1 dM " ' dl
+(-1)'
' + m + 1 — . d2 d2
E • 1 "('-1)0 "('-1)1
1 d(2+1)0 d(2+1)1
1 d2 d2
1 "(m-1)0 "(m-1)1
0('-1)
-i(' + 1)('-1)
('-1)m d2
2 (m-1)1 " ■ d(2m-1)('-1) d(2m-1)(' + 1) " ■ d(2m-1)m
1 1 1 1
d01 d02('-1) d02('+1) d02(m-1
('-1)(m-1) " 2
"(' + 1)(m-1)
0('+1)
-1('-1)('+1)
d2 d2
"(m-1)('-1) "(m-1)(' + 1)
0(m-1)
"(2i-1)(m-1) "(¡ + 1)(m-1)
0
Given that all the determinants do not contain d^, that is many calculations as in the case of application of the Gauss are some constant. That is why in our case all the summands method. If in square brackets are magnitudes O(s). As the number of summands in brackets is finite, the sum is also magnitude O(s). That is, difference contains 2m+3 summands,
of which m+2 are magnitudes O(s), and m+1 are magnitudes Y =
O(s2). Accordingly, the whole sum is magnitude O(s). And this means that proportional changes in comprehensive score correspond to small changes in scores in specific categories.
Consider the numerical methods for computation of then the determinant of matrix ¥ will be calculated from
S00 801 810 S11
V81m 8 2m
(17)
generalized volume of m-simplex. To calculate determinant D = |Y| by the numerical method, it is possible to select the Gauss method with the choice of the main element, because there are zeros on the diagonal of the matrix. Since the matrix is symmetric and is negatively determined, it is possible to use the Cholesky LDL-factorization for m>1 in order to calculate determinant D. In this case, it will require half as
formulas:
m
M = ÜL,
L ~J8H ^ L2k ,
(18) (19)
0
1
1
L=T
s»- Y LikLjk
, j < i, i, j = 0, m,
(20)
where (Lj) is the lower triangular matrix with positive
V j/i, j=0
elements on the diagonal; is the coefficient of matrix Y.
More information about application of the Cholesky method for computation of determinants of matrices can be found in [24].
In case indicators of performance of subjects of educational environment, which belong to the different categories, are equal, it is possible to use the formula that is described in [25] in order to find the volume of m-simplex:
V(Am) =-
^ ' om-1
2(m-1)
((m -1)!)2
Q m-1 m-1 Q m-1 ^ m-1
4Y Yd2d2 +2Yd2 -m4Yd4 -i
a/./ • im jm 2 y / im a / , im
i=0 j= i+1
(21)
where a is the values of indicators, which are equal or the distance from vertices of m-simplex to the coordinate origin, dj is the distances between vertices v and Vj of m-simplex.
Application of this method (21) makes it possible to find generalized volume of m-simplex at O(m2) of arithmetic operations, while the Cholesky method ((18)-(20)) has complexity of calculating O(m2). At m=1000, complexity of computation of generalized volume of m-simplex is 109 arithmetic operations that are performed by modern computing machines in less than a second. Clock rate of modern processors is over 1 GHz, which corresponds to 109 arithmetic operations that are performed by the processor in a second. Since the number of categories m for calculation of a comprehensive performance score of SEE, as a rule, does not exceed 10, the computational problem of finding generalized volume of m-simplex may be considered simple.
6. Discussion of results of comprehensive SEE performance evaluation based on the proposed method
To conduct comprehensive performance evaluation of SEE, it is necessary to select the indicators, which would characterize different aspects of activity. An important task is selection of the indicators, which are possible to obtain from public sources, specifically science-metric bases, university websites, etc.
In order to design the method for comprehensive evaluation of Ukrainian universities, 129 indicators were selected that were grouped in five categories, reflecting the main aspects of HEI activity: international activity, quality of the students' body, quality of academic and teaching staff, quality of scientific and research activity, resource provision of HEI.
Each category is divided into several subcategories for convenience. For example, Table 1 shows the indicators that belong to the category "international activity".
Information-analytical system "Database of Ukrainian scientists" was developed for verification of results of the research. The proposed above methods of finding performance scores of SEE were implemented in this system. During development of the system, the principles of development of distributed information systems, which are presented in [26], were used. Some of the components of the conceptual model of the automatic system, which is described in [27], were also used for identification of the directions of scientific studies of scientists. Five sets of indicators that are grouped in the following categories,
such as international activity, the students' body, academic and teaching staff, research activity and resource provision, were proposed. The main emphasis of information-analytical system was to find scientific and research performance score, based on information about publication activity of scientists.
Тable 1
Indicators for evaluation of results of activity by category "International activity"
Cipher Title Dimension
Ki INTERNATIONAL ACTIVITY
CK11 International communications
ni Number of international grants, scientific and educational projects and programs Units
n2 Number of agreements with foreign universities on students' studying by programs of «Double diploma» Units
n3 Number of students that acquired higher education by program «Double diploma» People
n4 Number of business trips abroad of academic and teaching staff with the purpose of carrying out scientific and teaching work, internship Units
n5 Number of international scientific and practical conferences on the problems of higher education and science, problems of relevant areas and other directions, which were conducted on the base of the subject Units
n6 Number of all-Ukrainian scientific and practical conferences on the problems of higher education and science, problems of relevant areas and other directions, which were conducted on the base of the subject Units
CK12 International exhibition, creative and sports activity
n7 Number of international exhibitions in the field of science, education, technologies, at which achievements of a subject were exhibited Units
n8 Number of awards (medals, diplomas), obtained by a subject at international exhibitions in the field of science, education, technologies, at which achievements of a subject were exhibited Units
ng Number of international art exhibitions, festivals and competitions, etc., at which achievements of a subject were represented Units
nio Number of awards (medals, diplomas of winners'), obtained by a subject at international art exhibitions, festivals and competitions, etc., at which achievements of a subject were represented Units
nil Number of all-Ukrainian national and field exhibitions, at which achievements of a subject were represented Units
ni2 Number of awards (medals, diplomas), obtained by a higher educational institution at all-Ukrainian national and field exhibitions, at which achievements of a subject were represented Units
ni3 Number of all-Ukrainian festivals, art forums, at which achievements of a subject were represented Units
ni4 Number of awards (medals, diplomas), obtained by higher educational institution all-Ukrainian festivals, art forums, at which achievements of a subject were represented Units
ni5 Number of awards, obtained by sportsmen at international sports competitions (Olympic games, World and European championships, World universities, World and European students' championships) Units
x
Performance scores of HEI were calculated based of the ideal point method, the weighed scores method, as well as the method based on calculation of volumes of m-simplexes. Based on the found comprehensive scores, the system performs ranking of universities and structural units. Table 2 shows the first 10 positions of HEI of Ukraine, sorted by descending order of the comprehensive score, calculated by the method based on calculation of volumes of m-simplexes and positions in the rating "Top-200 Ukraine". Only the rating "Top-200 Ukraine" includes all Ukrainian universities. Besides the rating "Top-200 Ukraine", there are no other comprehensive systems of evaluation of Ukrainian universities, which would assess the aspects of activity to the full. That is why we selected this rating for comparison.
Calculation of the rating "Top-200 Ukraine" and the rating, which was obtained in the "Database of scientists of Ukraine" based on calculation of volumes of m-simplexes is almost entirely based on the same indicators. However, that part of indicators differs. In addition, some indicators of the rating "Top-200 Ukraine" have no open access and are obtained from expert evaluation. Two evaluation systems are not fully isomorphic to perform comparison. However, comparison of the rating, which was calculated in "Database of scientists of Ukraine", with the known rating provided a possibility to assess adequacy of the proposed method for SEE performance evaluation.
Correlation between positions of HEI in the rating "Top-200 Ukraine" and the rating, which was drawn up based of the method of calculation of generalized volumes of m-sim-plexes, was calculated. Indicator of Pearson correlation of comprehensive scores, found by the method of calculating the volume of m-simplexes for 121 higher educational establishments of Ukraine and comprehensive evaluation of the respective higher educational establishments, found by the ranking technique "Top-200 Ukraine", is equal to 0.645201. For the first 10 positions, shown in Table 2, Pearson correlation coefficient is 0.812594. The value of correlation coefficient exceeds by 0.5, and by the Shaddock scale, this indicates existence of significant correlation between the results of comprehensive performance evaluation of a University. Thus, comparison that shown in Table 2 is possible.
Consider the example of calculation of the performance score of the Taras Shevchenko National University of Kyiv for the year of 2017 with the help of the proposed method. The scores were calculated by categories:
1. International activity - 26.131.
2. Students' body - 23.348.
3. Academic and teaching staff - 40.403.
4. Scientific and research activity - 37.666.
5. Resource provision - 49.502.
Then matrix ¥ is equal to:
Table 2
Results of comprehensive HEI performance evaluation in the system "Database of scientists of Ukraine" (first 10 positions)
No.by order Name «Top-200 Ukraine» [6] «Database of scientists of Ukraine»
Comprehensive score Position Volume of m-simplex Position
1 Taras Shevchenko National University of Kyiv 81.69805 2 135,745 1
2 National Technical University of Ukraine «Igor Sikorsky Kyiv Polytechnic Institute» 85.82174 1 97,105 2
3 Karazin Kharkiv National University 49.41333 3 93,635 3
4 National Technical University «Kharkiv Polytechnic Institute» 45.75635 4 83,845 4
5 Lviv Polytechnic National University 45.64861 5 79,785 5
6 Ivan Franko National University of Lviv 43.39769 9 68,045 6
7 Bogomolets National Medical University 44.60643 7 67,610 7
8 National Mining University of Ukraine 44.67087 6 66,305 8
9 Oles Honchar Dnipro National University 41.58334 11 57,390 9
10 National University of Kyiv-Mohyla Academy 44.55132 8 52,745 10
It should be taken into consideration that the system "Database of scientists of Ukraine" operates in the test mode, so the data on performance may be incomplete and the obtained performance scores are only introductory. The method for performance evaluation of SEE based on calculation of the volume of m-simplexes is integrated into the system "Database of scientists of Ukraine" as a separate micro-service. Results of the study can also be integrated into the system of design and vector control of SEE and allow solving the problem of calculation of coordinates of design of vector space subjects and partial solving the problem of calculating of environment resistance coefficients.
In subsequent scientific research, it is planned to give a detailed description of operation of the specified system.
1 0
1227.96 2315.23 2101.56 3133.28
1
1
1
1
1227.96 2315.23
0 2177.53 2177.53 0
1963.86 3051.13
2995.58 4082.85 3869.18
2101.56 3133.28 1963.86 2995.58 3051.13 4082.85 0 3869.18 0
7. Conclusions
The comprehensive score, derived from formula (8), is equal to:
y (A4 K i1698224014 = 135745.4074. 1 ' v 16■576
1. The method of comprehensive performance evaluation of SEE, specifically HEI, based on calculation of the volume of the m-simplex from Cayley-Menger formula was developed. This method is a self-sufficient tool for comprehensive performance evaluation of SEE due to:
- in contrast to the ideal point method, the developed method does not require selection of the point, the coordinates of which are performance scores of SEE, the best in terms of achieving maximum efficiency by some criteria. If the selected coordinates of a point are very small, as a result of evaluation, there can appear a point with coordinates larg-
er than the ideal point. And it would be contrary to definition of the ideal point. And if the determined coordinates of the ideal point are too large, the distance between the points, which are performance scores of SEE, will differ by a small magnitude, which complicates comparison;
- in contrast to the weighed score method, the developed method does not require selection of weight coefficients, that is does not require involvement of experts to calculate these coefficients.
2. It was shown that a proportional change in a comprehensive score correspond to small changes in scores in particular categories. The method of setting a tendency of SEE activity development by calculating the derivative of a comprehensive score in time was presented. Positive tendencies of development correspond to positive values of the time derivative, and negative tendencies correspond to negative values.
To verify research results, the indicators of HEI performance, which can be obtained from public sources and reflect the main aspects of HEI activity, were selected. Five sets of indicators were proposed, based of which criteria of the international activity, students' body, academic and teaching staff, scientific and research activity and resource provision were constructed. The database for storage of these indicators, which are integrated into the information-analytical system "Database of scientists of Ukraine", was developed. Correspondent performance scores can be used by rectors, deans, heads of departments and for analysis
of efficiency of functioning by various aspects of activity of subordinate subjects. Timely monitoring of dynamics of a change in performance allows, if necessary, make appropriate corrections to the strategy of a subject's functioning with the aim of improving its effectiveness.
This work explores numerical methods for calculation of generalized volume of m-simplexes from the Cayley-Menger formula based of Cholesky LDL-factorization in the general case. The method for calculation of generalized volume of the m-simplex in the case of equality of indicators was considered as well.
Comparing HEI ranking, which was constructed by the information system with a consolidated ranking of higher educational institutions of Ukraine [6], it was found that absolute difference in the positions of HEI between the ratings, composed for 121 higher educational establishments, does not exceed 4 positions. In other words, the error of the method is 3.3 %. After analyzing the obtained results, it is possible to draw a conclusion on that the ordered rating of the Ukrainian universities in general retains its structure. The method of comprehensive evaluation of HEI based on calculation of the volume of m-simplexes gives adequate results, because the score correctly displays the correlation between different parameters, which are performance scores of different HEI of Ukraine. Computational problem of m-simplex calculation for m=5 is easy, because it requires the order of 103 arithmetic operations and can be performed by a processor with a clock rate of 1 GHz over 10-3 s.
References
1. Slobodian O., Stadnyi Ye. Ukrainian students abroad: how much and why? // Analytical center CEDOS. 2016. URL: https://cedos. org.ua/uk/osvita/ukrainski-studenty-za-kordonom-skilky-ta-chomu
2. Bykov V. Yu. Competition - a market tool for innovation development of vocational education // New learning technologies. 2010. Issue 64. P. 22-36.
3. QS World Ratings. 2017. URL: https://www.topuniversities.com/qs-world-university-rankings
4. Academic Ranking of World Universities 2017. URL: http://www.shanghairanking.com/ARWU2017.html
5. Ranking Web of Universities. URL: http://www.webometrics.info/en/Europe/Ukraine
6. Rating of higher educational institutions "Top 200 Ukraine". URL: http://www.euroosvita.net/index.php/?category=1&id=4757
7. Titov L. P., Filoniuk V. A., Gorbunov V. A. Analysis of the international experience of evaluating the activities of scientists, institutions of science, priority of scientific areas // Prospects for the development of median nacture and scientometry. 2014. Issue 1. P. 21-29.
8. Holovenkin V. P. Determination of ratings of scientific and pedagogical workers as a lever for achieving the criteria of a research university // Kyiv politechnik. 2008. Issue 31. URL: http://kpi.ua/831-5
9. Hirsch J. E. An index to quantify an individual's scientific research output // Proceedings of the National Academy of Sciences. 2005. Vol. 102, Issue 46. P. 16569-16572. doi: 10.1073/pnas.0507655102
10. Egghe L. Theory and practise of the g-index // Scientometrics. 2006. Vol. 69, Issue 1. P. 131-152. doi: 10.1007/s11192-006-0144-7
11. Zhang C.-T. The e-Index, Complementing the h-Index for Excess Citations // PLoS ONE. 2009. Vol. 4, Issue 5. P. e5429. doi: 10.1371/journal.pone.0005429
12. Evaluation methods of the results of scientific research activity of scientists based on the analysis of publication citations / Bilosh-chytskyi A., Kuchansky A., Andrashko Y., Biloshchytska S., Kuzka O., Terentyev O. // Eastern-European Journal of Enterprise Technologies. 2017. Vol. 3, Issue 2 (87). P. 4-10. doi: 10.15587/1729-4061.2017.103651
13. A method to evaluate the scientific activity quality of HEIs based on a scientometric subjects presentation model / Bilosh-chytskyi A., Myronov O., Reznik R., Kuchansky A., Andrashko Yu., Paliy S., Biloshchytska S. // Eastern-European Journal of Enterprise Technologies. 2017. Vol. 6, Issue 2 (90). P. 16-22. doi: 10.15587/1729-4061.2017.118377
14. Lakshmi T. M., Venkatesan V. P., Martin A. An Identification of Better Engineering College with Conflicting Criteria using Adaptive TOPSIS // International Journal of Modern Education and Computer Science. 2016. Vol. 8, Issue 5. P. 19-31. doi: 10.5815/ ijmecs.2016.05.03
15. Sreeramana A. P., Kumar P. M. ABC Model of Research Productivity and Higher Educational Institutional Ranking // International Journal of Education and Management Engineering. 2016. Vol. 6, Issue 6. P. 74-84. doi: 10.5815/ijeme.2016.06.08
16. Baden-Fuller C., Ravazzolo F., Schweizer T. Making and Measuring Reputations // Long Range Planning. 2000. Vol. 33, Issue 5. P. 621-650. doi: 10.1016/s0024-6301(00)00064-9
17. Development of parametric model of prediction and evaluation of the quality level of educational institutions / Otradskaya T., Gogunskii V., Antoshchuk S., Kolesnikov O. // Eastern-European Journal of Enterprise Technologies. 2016. Vol. 5, Issue 3 (83). P. 12-21. doi: 10.15587/1729-4061.2016.80790
18. The Method of the Scientific Directions Potential Forecasting in Infocommunication Systems of an Assessment of the Research Activity Results / Biloshchytskyi A., Kuchansky A., Andrashko Yu., Biloshchytska S., Dubnytska A., Vatskel V. // 2017 4th International Scientific-Practical Conference Problems of Infocommunications. Science and Technology (PIC S&T). 2017. P. 69-72. doi: 10.1109/infocommst.2017.8246352
19. Kuchansky A., Biloshchytskyi A. Selective pattern matching method for time-series forecasting // Eastern-European Journal of Enterprise Technologies. 2015. Vol. 6, Issue 4 (78). P. 13-18. doi: 10.15587/1729-4061.2015.54812
20. Development of adaptive combined models for predicting time series based on similarity identification / Kuchansky A., Bilosh-chytskyi A., Andrashko Yu., Biloshchytska S., Shabala Ye., Myronov O. // Eastern-European Journal of Enterprise Technologies. 2018. Vol. 1, Issue 4 (91). P. 32-42. doi: 10.15587/1729-4061.2018.121620
21. Mulesa O., Geche F. Designing fuzzy expert methods of numeric evaluation of an object for the problems of forecasting // Eastern-European Journal of Enterprise Technologies. 2016. Vol. 3, Issue 4 (81). P. 37-43. doi: 10.15587/1729-4061.2016.70515
22. A method for the identification of scientists' research areas based on a cluster analysis of scientific publications / Biloshchytskyi A., Kuchansky A., Andrashko Yu., Biloshchytska S., Kuzka O., Shabala Ye., Lyashchenko T. // Eastern-European Journal of Enterprise Technologies. 2017. Vol. 5, Issue 2 (89). P. 4-11. doi: 10.15587/1729-4061.2017.112323
23. Michelucci D., Foufou S. Using Cayley-Menger determinants for geometric constraint solving // Proceedings of the 9th ACM symposium on Solid modeling and applications. Eurographics Association. 2004. P. 285-290.
24. Watkins D. S. Fundamentals of Matrix Computations. 2nd ed. New York: John Wiley & Sons. Inc, 2005. 620 p. doi: 10.1002/0471249718
25. Thurau C., Kersting K., Bauckhage C. Yes we can - simplex volume maximization for descriptive web-scale matrix factorization // Proceedings of the 19th ACM international conference on Information and knowledge management - CIKM '10. 2010. P. 1785-1788. doi: 10.1145/1871437.1871729
26. Morozov V., Kalnichenko O., Liubyma I. Managing projects configuration in development distributed information systems // 2017 2nd International Conference on Advanced Information and Communication Technologies (AICT). 2017. P. 154-157. doi: 10.1109/aiact.2017.8020088
27. Conceptual Model of Automatic System of Near Duplicates Detection in Electronic Documents / Biloshchytskyi A., Kuchansky A., Biloshchytska S., Dubnytska A. // 2017 14th International Conference The Experience of Designing and Application of CAD Systems in Microelectronics (CADSM). 2017. P. 381-384. doi: 10.1109/cadsm.2017.7916155
