CC BY
7
Дослиджено наявт методи i засоби оцтювання компетентностi експертiв. Проведено оцтювання компетентностi експертiв у сферi техтчного регулю-вання за встановленими критерiями. Результаты опрацьовано за допомогою спещальних методик i програмних засо-бiв з урахуванням невизначеностi даних та застосуванням методу аналтичног ieрархii. Проведено порiвняльний аналiз отриманихрезультатiв з метою оцтю-вання збiжностi i придатностi методов
Ключовi слова: компетенттсть, експерт, критери оцтювання, техтч-не регулювання, невизначетсть даних,
аналтична ieрархiя
□-□
Исследованы существующие методы и средства оценивания компетентности экспертов. Проведено оценивание компетентности экспертов в сфере технического регулирования по установленным критериям. Результаты обработаны с помощью специальных методик и программных средств с учетом неопределенности данных и применением метода аналитической иерархии. Проведен сравнительный анализ полученных результатов с целью оценивания сходимости и пригодности методов
Ключевые слова: компетентность, эксперт, критерии оценивания, техническое регулирование, неопределенность, аналитическая иерархия
UDC 389:14:621.317:354
[DOI: 10.15587/1729-4061.2017.106825|
A COMPARATIVE ANALYSIS OF THE ASSESSMENT RESULTS OF THE COMPETENCE OF TECHNICAL EXPERTS BY DIFFERENT METHODS
O. Velych ko
Doctor of Technical Sciences, Professor, Director Scientific and Production Institute of Electromagnetic Measurements State Enterprise ''All-Ukrainian State Scientific and Production Centre for Standardization, Metrology, Certification and Protection of Consumer'', (SE "Ukrmetrteststandard") Metrolohychna str., 4, Kyiv, Ukraine, 03143 E-mail: velychko@hotmail.com T. Gordiyenko Doctor of Technical Sciences, Associate Professor, Head of Department Department of standardization, conformity assessment and quality* E-mail: t_gord@hotmail.com L. Kolomiets Doctor of Technical Sciences, Professor, Rector* *Odessa State Academy of Technical Regulation and Quality Kovalska str., 15, Odessa, Ukraine, 65020
1. Introduction
For reasonable decision-making in any spheres of activity, it is necessary to lean against experience, knowledge and intuition of specialists. To that end, group expert evaluations - procedures of estimation of a certain problem based on the opinions of specialists (experts) for further decision-making are conducted. For such estimations, it is necessary foremost to correctly approach the selection of experts in the field of certain activity.
As practice shows, reduction of subjectivity, and, accordingly, increase of objectivity of the results of using the methods of group expert evaluation substantially depend on compliance with the rules of organization, preparation and realization of such evaluation. It depends, first of all, on the set procedures of expert evaluation, appointment of the responsible for the organization and realization of expert evaluation, and also formation of expert groups.
The well-known forms of collection of ideas are divided into individual, collective and mixed, each having a number of varieties. All are based on different algorithms of realization and have advantages and defects. In many cases, each of these varieties is used together, which gives quite often a greater effect and objectivity. However, mostly a questionnaire is used in practice of expert evaluation. It allows with the least effort to collect opinions of experts, though takes more time than some other kinds [1].
For reliable estimations of group expert evaluation in any field of activity, it is necessary foremost to choose the most optimal method. It is important from the large variety of expert methods to distinguish and to perfect the most prevailing for certain necessities in the certain field. To that end, it is necessary to carry out a comparative analysis for the evaluation of fitness of methods. Exactly the same methods were applied by the authors during the realization of researches on improving the efficiency of complex systems in the sphere of technical regulation, in particular in standardization and metrology [1].
©
2. Literature review and problem statement
Introduction of the legislation of the European Union (EU) and compliance with the requirements of the World Trade Organization (WTO), and also realization of institutional changes in accordance with European practice is the basis of reforms in the field of technical regulation. For an identification and further resolution of problematic issues in this field, it is expedient to widely use the group expert evaluation. This needs the involvement of highly skilled specialists.
In the field of technical regulation, there are specialists in different areas, particularly standardization, metrology, conformity assessment, certification, etc. Evaluations of experts in the field of standardization were the subject of previous researches [1]. Another not less challenging direction in the field of technical regulation is the metrology assurance (MA).
MA is defined as an establishment and application of metrology norms and rules, and also development, production and application of technical equipments necessary for the achievement of unity and required accuracy of a certain measurement. Important is reliable knowledge about the real condition of MA for measurement of a certain physical quantity. One of the useful means for solving this issue may be the group expert evaluation involving experts in metrology, i. e. highly skilled specialists-metrologists.
Expert methodologies of evaluation are based on the use of knowledge of skilled specialists - experts in the investigated subject field. For the estimation of experts, there are a number of characteristics used for the selection of both certain experts and groups of experts: coefficient of competence (CC); coefficient of concordance; coefficient of reliability of expert estimations, etc.
Most expert methods can be divided into classes of individual and collective (group) expert estimation. Each of the methods has a lot in common, and the difference is mainly that the evaluation of the investigated objects or spheres is carried out in different ways.
Resources and capabilities that can influence the choice of methods include:
- competence, experience, capabilities and possibilities of the expert evaluation group;
- limitation in relation to time and other resources of the organization;
- available budget in case if external resources will be needed.
Most widely used expert methods are quite simple and have the imperfections:
- the method of ranking - does not provide sufficient accuracy of ranking more than 15-20 objects [2-5];
- the method of direct evaluation - cannot be used in case of incomplete knowledge of an expert about the investigated properties of the object [2, 3, 6];
- the method of comparison, including two varieties:
- the method of successive comparison [5] - the most labour-intensive and complex;
- the method of pairwise comparison [5, 7-9] - quite simple in comparison with other methods, characterized by the highest level of reliability of the estimation results and allows investigating plenty of objects with great accuracy.
In [10], the method of evaluation of expert's competence on the basis of fuzzy set theory has been proposed. The disadvantages of this method are divergences between the finite
set of competences that characterize the states of the object, and characteristics suggested by a certain expert. This narrows the application scope of this approach.
No less known methods of scenario analysis: analysis of the root cause, scenario, influence on activity, cause-and-effect relations do not provide for numeral estimates [6, 11, 12]; the basic method of analytic hierarchy process [13-15] and its modifications [5, 7, 12, 16] are applicable only in case of a small number of the set of alternatives and do not give an opportunity to combine different opinions of expert groups.
For the research of complex objects or systems, it is expedient to use the method of Analytic Hierarchy Process (AHP). AHP is a mathematical tool for a systematic approach to complex problems of decision-making. It allows an interactive finding of such a variant (alternative) that best fits both with its understanding of the problem and with the requirements to its solution.
This method is widely used due to the works [13, 14], which have more fully revealed the possibilities of the procedure, and since then AHP has been actively developed and widely used in practice. It allows a clear and rational way to structure a complex problem of decision-making as a hierarchy, comparing and performing the quantitative estimation of alternative solutions. Many works on the application of expert methods describe the usage of special software. The software considerably increases the productivity of the applied methods of expert evaluation and eliminate errors in the calculations of the results.
3. The aim and objectives of the study
The research was aimed at developing the most effective method and means for the competence evaluation of experts using scientifically-reasonable criteria.
For the achievement of the aim, the following objectives were accomplished:
- to choose the most effective methods of expert evaluation for certain tasks in the field of technical regulation, which would let to take into account the level of competence of the experts involved;
- to conduct the comparative analysis of the results obtained by means of the selected methods of competence evaluation of experts, and estimate the fitness.
4. Materials and methods of research for development of methods of evaluation of expert's competence in the field of technical regulation
4. 1. A method of evaluation of expert's competence taking into account the data uncertainties
Being based on previous researches of the authors, questioning of metrology specialists for the evaluation of expert's competence (EEC), who have experience in these spheres and work on different types of measurement was carried out. Such types of measurement are covered: electrical power, electrical capacitance and inductance, phase shift angle, time and frequency, alternating-current (AC) voltage, high direct-current (DC) voltage, high AC voltage, high AC [17, 18].
For the evaluation of expert's competence in the field of technical regulation, a special questionnaire was worked out with the aim of carrying out the appropriate questioning.
In the ideal case, the available data about the experts will be the average score of all criteria of the EEC. In other cases, such data can be an exhaustive list of the total scores of all criteria of the EEC. In any case, there will be some data uncertainties concerning the experts. The range of uncertainties in these data can be limited by the use of independent methods or by checking for consistency.
For the realization of the method with taking into account characteristics of data uncertainties (DU), appropriate criteria for the score EEC in a certain field are established [18].
For the realization of this method (DU method), it is necessary to calculate [1]:
- average score xi for each of the M experts for all criteria of the EEC N (j from 1 to N):
N
x=£ xilN ; j=i
(i)
- relative average score xti for each of the M experts:
/ M
x„ = xJ £ x, ; (2)
/ i=1
- rationed average score xni for each of the M experts:
x = x Ix ■ ; (3)
ni ril rcmax '
- total standard uncertainties uci for each of the M (i-th) experts:
■=(1 - x„ )/10.
(4)
Calculation of the reference value xref and its total standard uncertainty uref for the overall evaluation of experts is carried out according to the formulas:
M - M A
=£*/ Mur
1=1 Ucl 1=1 " Ci
=* 1/
(5)
(6)
The consistency check of the data about the experts is performed on the basis of analysis and calculation of the value of x2:
x2 = £
m (x - x )2
V^ni ref '
(7)
m (x - x )2
x2 = Y K ni ref ' x2 k — 2 A-0,95(M-1)'
(8)
then the data on the formed group from M experts (minimum of 4 experts, maximum - up to 33) are recognized as agreed.
The remaining experts are ranked in descending order xyi. The selection of the most competent experts is expedi-
ent to carry out by the analysis of the results obtained with application of the Pareto principle [19].
The algorithm of evaluation of expert's competence by the DU method is given in [1]. This algorithm can be easily implemented with the use of common mathematical software packages.
The DU method is expedient to apply as a useful tool for the comparative EEC on the basis of objective data according to the set of criteria of the EEC for different spheres of activity. It allows selecting the most competent experts and rejecting (if necessary) the experts, whose data do not correspond to a certain level of the set requirements.
4. 2. A method of evaluation of expert's competence on the basis of Analytic Hierarchy Process
AHP is well-known [13] and used in case of a small number of the set of alternatives, when the decision-maker's efforts (DM) are aimed at comparing only the set of alternatives. The task for the AHP method consists in that, with the well-known general aim (or subtask) of the problem solution, N criteria of estimation of alternatives and n alternatives, it is necessary to choose the best alternative. AHP is a mathematical tool of systems approach to complex decision-making problems.
AHP is intended for the determination of the best option (among several) taking into account many criteria of different nature and allows using various criteria for comparison (quantitative, qualitative, numerical with different dimensions, etc.). Its maximum efficiency shows up during the search of solutions to complex problems that require systems approach and involvement of plenty of experts [1].
Basic phases for implementation of the AHP method are:
- to structure a task as a hierarchical structure with several levels (aims-criteria-alternatives);
- to execute pairwise comparisons of elements of every level and transform the results of comparisons in numbers by means of the special scale of relative importance (SRI);
- to calculate the weight coefficients (wi is the weight of the i-th of criterion) for the elements of every level and check up the consistency of the DM's judgements;
- to compute the quantitative indicator of the quality of each of the alternatives and determine the best alternative.
SRI is a well-organized set of gradations that are determined numerically, for expression of the results of pairwise comparisons [13-15]. This scale allows determining the relative importance of the compared elements exactly, so its division is dimensionless quantities.
The synthesis of the obtained weight coefficients is carried out by the expression:
If the value of the x2 criterion, calculated from data of the i-th expert, does not exceed a critical value x2 for the 0.95 confidence interval and the number of freedom degrees of M-1:
V. = £ wV-'j ¿j i'ji
(9)
where Sj is the index of the quality of the j-th alternative; wi is the weight of the i-th criterion of the EEC; Vji is the weight of the j-th of alternative on the i-th criterion of the EEC.
There may be errors in the compilation of pairwise comparison matrices (PCM). So one of the possible errors may be a violation of transitivity: aij>ais (aj - elements of the PCM) may not follow from azj>ajk, ajk>ais. Violations of consistency of numeral judgements are also possible: aijxajajk.
For the detection of inconsistency, the calculation of the index of consistency of comparisons according to the PCM is carried out in the following sequence [13, 14]:
x
f
u
f
2
u
- to sum up the elements of every column in the PCM;
- to multiply the sum of the elements of every column by the corresponding normalized components of the weight vector determined from the PCM;
- to sum up the resulting numbers and denote the values of the sum as 1max;
- to find the consistency index i=(1max-w)/(w-l), where n is the number of the compared elements (matrix size), 1>n for a skew-symmetric matrix;
- to calculate the mean value of the consistency index R for skew-symmetric matrices filled randomly (for n=7,R=l.32, and for n=8,R=l.41);
- to calculate the consistency ratio Cd=L/R, which must be within Cd<0.1, and if the value of T exceeds this level, the comparison is carried out again.
The consistency ratio Cd for the entire hierarchy less than 0.1 is a good result.
Pairwise comparisons set priorities of elements for some level of hierarchy in relation to one element of the next level. If there are more than two levels, then different vectors of priorities can be united in the matrix of priorities, from which one eventual vector of priorities for the lower level is determined.
For the realization of the AHP method [1], the PCM of indexes A - a table of numeral values of the results of the pairwise comparisons obtained by the SRI is built:
A =
ai1 ai2 aiN V aN1 aN2— aNN J
(10)
where N is the number of the generalized indexes.
The numeral values of the PCM elements A aj (i, j= =1, 2, ..., N) are set by an expert way or group of experts (by an expert), or directly in each specific estimation of the compared objects. The relative importance of the index i compared to the index j is expressed by a natural number from 1 to 9 or reverse number from 1 to 1/9. For i=j, a,j=1, and for i^j, a,j>0, aij=1/aij.
The normalized eigenvector A, for the PCM of indicators A is determined by the expression:
N N
Imax = E E aj ■ A,.
j=1 ¿=1
The consistency index Ic is a quantitative index (positive number) of noncontradiction of the comparison results (contradictions in comparisons arise due to subjective errors in the output data). The consistency index that is randomly generated on a scale from 1 to 9 of the back-symmetric matrix with the corresponding reciprocals of the elements is the casual consistency index Rc.
The value of the random consistency index of the output data depends on the dimension of the PCM A (L=N - for the A matrix) used and is given in [13, 14, 20]. If Q<0.1, then the initial information is considered consistent, and if Q>0.1, then the output data are considered inadmissibly distorted. In the second case, it is necessary to revise the output data with the introduction of additional or corrective information.
Next, the PCM Bk (£=1, 2, ..., M) is built for each of the compared by each generalized index:
Bk =
( bk bk bk
U11 "12 ••• U1M
bk bk bk
21 22 2M
bk bk bk V M1 M2 —UMM J
(15)
where bk is the result of pairwise comparison of the i-th and j-th object relatively the k-th index; M is the number of the compared objects.
Numeral values of the elements of the PCM Bk - bk (i, j=1, 2, ..., M) are obtained with the use of available information from those or other accessible sources. If i=j, bij=1, and if
m bij>0, = 1 bk.
The normalized eigenvectors Bk for each built PCM Bk, which determine local priorities are defined by the formula:
Bk=M n bj E M n b
j=1
j=1
(16)
a, = n n aj e n n
j=1
(11)
¿=1 V j=1
The consistency of local priorities, i. e. the quality of the output data included in the PCM A is checked, for which the consistency ratio Cd is determined by the expression:
The index k as a natural number is used to indicate the number of the indicator, to which the value with this index belongs.
The consistency of local priorities, i.e. the quality of the output data, included in the PCM Bk is checked, for which consistency ratio Ckd is determined by the formula:
Cd = IJRC,
(12)
ck = ikjRc,
(17)
where Ic is the consistency index of the output data; Rc is the casual consistency index.
The consistency ratio for the system as a whole characterizes its weighted mean value for all the comparison matrices.
The consistency index Ic of the output data included in the A matrix is determined by the formula:
where Ikc is the consistency index of the output data included in a specific PCM Bk; Rc is the random consistency index, which depends on the dimension of the matrix used (L=M - for the PCM Bk ).
The consistency index of the output data included in the PCM Bk is determined by the formula:
Ic =
1max - N N -1 ,
(13)
jk = 1L. - m
M-1
(18)
where Xmax is the largest eigenvalue for the A matrix, which, where ^m^ are the largest eigenvalues for the PCM Bk
in turn, is determined by the formula:
determined by the formula:
M
iL=£
j=1
M
£ bk ■ Bk .
i i-
i=1
ation (dispersion) of the CC for the experts was calculated by the formula:
If Cd < 0.1, then the initial data is considered consistent, and if Cdk > 0.1 - inadmissibly distorted. In the second case, it is also necessary to revise the output data with the introduction of additional or corrective information.
In the case k=1, a comparison of one expert with competence indexes, i. e. checking for accordance to the set of criteria of the EEC is carried out. Next, the global priorities Gn for each of the M compared experts determined by the formula:
G = £ Bi B', n = 1, 2, ..., M,
n s • in' ? ? ? ?
(20)
where B?,...,B'n are the components of the normalized eigenvectors of local priorities determined by using the expression (16).
The obtained values of global priorities for each expert are ranked in ascending order of the Gn values. The expert who received the maximum value of Gn is considered the most competent [20].
5. Evaluation of competence of technical experts by the methods of data uncertainties and Analytic Hierarchy Process
For the realization of the marked and described DU and AHP methods, the following criteria of the EEC for metrol-ogists and standardizers are applied [1, 20, 21]:
Ki - education and scientific level in the field of metrology (standardization);
K2 - overall experience;
K3 - experience in the field of technical regulation (metrology, standardization);
K4 - experience as expert in the field of technical regulation (metrology, standardization);
K5 - work status.
For the realization of the AHP method, the values of the PCM of A criteria with normalized eigenvectors K -vectors of priorities for the selected criteria of the EEC and weight coefficients for the selected criteria of the EEC are defined [20].
For the proposed criteria of the EEC, the largest eigenvalue of the PCM of A criteria was Xmax=5.35. Verification of consistency of the output data used for the construction of the A matrix on the obtained consistency index Ic=0.09 and consistency ratio Cd=0.07 showed that the consistency ratio satisfied the requirements of consistency (Cd<0.1). This demonstrates the consis- — tency of the set criteria of the EEC [1, 20].
For comparison of the results, obtained with the use of DU and AHP methods, for the AHP method, recalculation of the received global priorities for the experts in the CC (kAHP) was carried out by the formula:
R = kmax -kmin, (22)
where kmax is the maximum of CC (equals 1.00); kmin is the minimum of CC, obtained for a certain expert.
6. An example of competence evaluation of time and frequency measurement expert's
Competence evaluation of 21 time and frequency measurement expert's was made. Among 21 involved experts, 16 (76 %) represented the state enterprises of the technical regulation system, 5 (24 %) - other enterprises. The questionnaire data about the experts was processed by specialized software "Competence DU 1.1" and "Competence AHP 1.1" (Ukraine) intended exactly for the EEC and based on the above methods [20].
The values of the CC used for all time and frequency measurement experts are shown in Table 1 [20]. The CCs were obtained by the use of DU (kDn) and AHP (kAHP) methods described in [18].
Table 1
Coefficients of competence of time and frequency measurement experts obtained using the methods of data uncertainties and Analytic Hierarchy Process
Expert 01 02 03 04 05 06 07 08 09 10 11
kdu 0.87 1.00 0.74 0.90 0.90 0.76 0.87 0.82 0.92 0.87 0.58
kahp 0.85 1.00 0.64 0.98 0.92 0.42 0.50 0.71 0.98 0.89 0.38
Expert 12 13 14 15 16 17 18 19 20 21 -
kdu 0.84 0.90 0.90 0.79 0.87 0.95 0.79 0.97 0.63 0.97 -
kahp 0.77 0.89 0.67 0.67 0.74 0.88 0.36 0.89 0.30 0.71 -
Comparison of CC of time and frequency measurement experts obtained by the use of DU and AHP methods was carried out and the results are shown in Fig. 1.
As can be seen from Fig. 1, comparisons of CC of the experts obtained by the use of DU and AHP methods have a clear correlation. Thus, the AHP method is characterized by a greater dispersion of the CC (kAHP) values: the difference between the highest and the lowest CC is RAHP=0.70 (minimum CC - 0.30). For the DU method, the dispersion of the CC (kDn) values is Rdu=0.42 (minimum CC - 0.58), that is by 40 % less than for the AHP method.
Measurement of time and frequency
kAHP = Gni/Gmax . (21)
In the comparative analysis of the results obtained by the DU and AHP methods, vari-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -•-Methods of DU Methods of AHP
16 17 18 19 20 21
Fig. 1. Comparison of competence coefficients of time and frequency measurement expert's obtained using the methods of data uncertainties and Analytic Hierarchy Process
7. Discussion of the results of evaluation of competence of technical experts
The results of the evaluation of CC of technical experts by the use of DU and AHP methods are shown in Table 2. Within the framework of the conducted questioning, there was also a questioning about the experience in the field of technical regulation, necessary for an achievement of the expert level.
Table 2
The results obtained using the methods of data uncertainties and Analytic Hierarchy Process
No. Field of measurement (activity) Number of experts Average CC Average for expert years
kdu kahp
1 Electrical power 26 0.76 0.65 7.2
2 Electrical capacitance 14 0.88 0.79 6.7
3 Electrical inductance 14 0.80 0.64 5.8
4 Phase shift angle 11 0.82 0.64 7.4
5 Time and frequency 21 0.86 0.72 7.5
6 AC voltage 12 0.81 0.66 7.4
7 High DC voltage 14 0.84 0.70 7.6
8 High AC voltage (U) 15 0.80 0.65 6.7
9 High AC voltage (£) 16 0.82 0.66 6.6
10 High AC current 15 0.81 0.64 6.7
11 Standardization 32 0.80 0.64 6.3
Average: 17 0.81 0.67 7.0
For all types of activity, an average value is 7, and the mode - 5. The results for the DU and AHP methods show a large correlation.
With the application of the universal software Microsoft Excel (USA), the ratio of averages for the criteria used for the EEC in the field of technical regulation was estimated (Fig. 2). The results show a small dispersion of average values for the criteria of the EEC (from 5.2 to 7.9), which testifies to a good balance.
K1
K5
K2
Fig. 2. The average for the criteria of evaluation of expert's competence in the field of technical regulation
Average values of CC of technical experts were obtained using the DU method (the average for all fields is 0.81, the maximum for all fields is 0.88, the minimum for all fields is 0.76). Average maximum eigenvalues for technical ex-
perts, were obtained using the AHP method (the average for all fields is 17.6, the maximum for all fields is 32.4, the minimum for all fields is 11.1).
A general comparison of CC of metrology and standardization experts, obtained using the DU and AHP methods, shows a clear correlation between the values on the experts. Thus, the AHP method is characterized by a greater dispersion of the CC (RAHP) values, and for the DU method, the dispersion of the CC (Rdu) values is less than for the AHP method. The dispersion of the values obtained using the DU and AHP methods is shown in Table 3.
Table 3
The dispersion of the values obtained using the methods of data uncertainties and Analytic Hierarchy Process
No. Field of measurement (activity) Variation of CC values Minimum CC
Rdu Rahp kdu kahp
1 Electrical power 0.60 0.85 0.40 0.15
2 Electrical capacitance 0.36 0.67 0.64 0.33
3 Electrical inductance 0.39 0.83 0.61 0.17
4 Phase shift angle 0.46 0.81 0.54 0.19
5 Time and frequency 0.42 0.70 0.58 0.30
6 AC voltage 0.44 0.80 0.56 0.20
7 High DC voltage 0.46 0.82 0.54 0.18
8 High AC voltage (U) 0.46 0.82 0.54 0.18
9 High AC voltage (£) 0.46 0.83 0.54 0.17
10 High AC current 0.46 0.80 0.54 0.20
11 Standardization 0.57 0.88 0.43 0.12
Average: 0.46 0.80 0.54 0.20
Dispersion of the values of CC for experts in the field of technical regulation, obtained using:
- the DU method: the average for all fields is 0.46; the maximum for all fields is 0.60; the minimum for all fields is 0.36;
- the AHP method: the average for all fields is 0.80; the maximum for all fields is 0.88; the minimum for all fields is 0.67.
The results in Table 3 show a large correlation for the DU and AHP methods. Therefore, these methods are expedient to apply as a useful tool for the comparative evaluation of expert's competence on the basis of objective data according to the set of criteria for different fields (spheres) of activity. However, the AHP method, to a lesser extent, allows taking into account opinions of less competent experts than the DU method. This is evidenced by a lower CC for this method than for the DU method. Thus, the DU method allows the selection of the most competent experts and discard the experts the data on whom disagree with a certain level of the set requirements.
8. Conclusions
1. The most effective methods of expert evaluation, suitable for the evaluation of expert's competence in the field of technical regulation, are considered in detail. The DU and
AHP methods as the most suitable are selected. Using the selected methods, the evaluation of expert's competence in the field of technical regulation is conducted (time and frequency measurement) according to the set criteria. The results were processed by special methodologies, and also universal and specialized software "Competence DU 1.1" and "Competence AHP 1.1" (Ukraine).
2. The comparative analysis of the results obtained using the DU and AHP methods showed the convergence, fitness and correlation of the obtained values for the experts. The estimated ratio of the averages for the criteria used for the EEC in the field of technical regulation, showed a small dispersion of the average values for the
criteria of the EEC (from 5.2 to 7.9), which testifies to a good balance.
However, the AHP method, to a lesser extent, allows taking into account opinions of less competent experts, than the DU method. This is evidenced by a lower CC for this method than for the DU method.Therefore, these methods are expedient to apply as a useful tool for the comparative evaluation of expert's competence on the basis of objective data according to the set of criteria for different fields (spheres) of activity.Thus, the DU method allows the selection of the most competent experts and discard the experts the data on whom disagree with a certain level of the set requirements.
References
1. Velychko, O. M. Methodologies of expert's competence evaluation and group expert evaluation [Text] / O. M. Velychko, T. B. Gordiyenko, L. V. Kolomiets // Metallurgical and Mining Industry. - 2015. - Issue 2. - P. 262-271.
2. Litvak, B. G. Ekspertnye otsenki i prinyatiye resheniy [Text] / B. G. Litvak. - Moscow: Patent, 1996. - 271 p.
3. Larichev, O. I. Teoriya i metody prinyatiya resheniy [Text]: ucheb. / O. I. Larichev. - 2-oe izd., pererab. i dop. - Moscow: Logos, 2002. - 392 p.
4. Pavlov, A. N. Metody obrabotki ehkspertnoy informacii [Text]: ucheb.-metod. pos. / A. N. Pavlov, B. V. Sokolov. - Sankt-Peterburg: GUAP, 2005. - 42 p.
5. Chernysheva, T. Yu. Ierarhicheskaya model' ocenki i otbora ehkspertov [Text] / T. Yu. Chernysheva // Doklady TUSUR. Uprav-leniya, vychislitel'naya tekhnika i informatika. - 2009. - Issue 1 (19). - P. 168-173.
6. Orlov, A. I. Ehkspertnye ocenki [Text]: ucheb. pos. / A. I. Orlov. - Moscow, 2002. - 31 p.
7. Kolpakova, T. A. Opredelenie kompetentnosti ehkspertov pri prinyatii gruppovyh resheniy [Text] / T. A. Kolpakova // Radioelek-tronika, informatyka, upravlinnia. - 2011. - Issue 1. - P. 40-43.
8. Xia, M. Consistency and consensus improving methods for pairwise comparison matrices based on Abelian linearly ordered group [Text] / M. Xia, J. Chen // Fuzzy Sets and Systems. - 2015. - Vol. 266. - P. 1-32. doi: 10.1016/j.fss.2014.07.019
9. Koczkodaj, W. W. Inconsistency indicator maps on groups for pairwise comparisons [Text] / W. W. Koczkodaj, J. Szybowski, E. Wajch // International Journal of Approximate Reasoning. - 2016. - Vol. 69. - P. 81-90. doi: 10.1016/j.ijar.2015.11.007
10. Malovik, K. N. Razvitie nauchnyh osnov povysheniya kachestva ocenivaniya i prognozirovaniya resursnyh harakteristik slozhnyh ob'ektov [Text]: monografiya / K. N. Malovik. - Sevastopol': SNUYAEHiP, 2013. - 332 p.
11. Hrabovetskyi, B. Ye. Metody ekspertnykh otsinok: teoriia, metodolohiia, napriamy vykorystannia [Text] / B. Ye. Hrabovetskyi. -Vinnytsia: VNTU, 2010. - 171 p.
12. Kalinina, I. O. Vrakhuvannia kompetentnosti ekspertiv u metodakh bahatokryterialnoho analizu v zadachakh ratsionalnoho vy-boru [Text] / I. O. Kalinina, O. P. Hozhyi, H. O. Musenko // Nauk. pratsi Chornomor. derzh. univer. Kompiuterni tekhnolohii. -2013. - Vol. 191, Issue 179. - P. 116-123.
13. Saaty, T. L. The Hierarchon: A Dictionary of Hierarchies [Text] / T. L. Saaty. - Pittsburgh, Pennsylvania: RWS Publications, 1992. - 496 p.
14. Saaty, T. L. Prinyatie resheniy pri zavisimostyah i obratnyh svyazyah: Analiticheskie seti [Text] / T. L. Saaty. - Moscow: Izd-vo LKI, 2008. - 360 p.
15. Drake, P. R. Using the Analytic Hierarchy Process in Engineering Education [Text] / P. R. Drake // International Journal of Engineering Education. - 1998. - Vol. 14, Issue 3. - P. 191-196.
16. Strategic Decision Making: Applying the Analytic Hierarchy Process [Text] / R. Roy (Ed.). - London: Springer-Verlag, 2004. -172 p. doi: 10.1007/b97668
17. Velychko, O. M. The group expert evaluation of the metrological assurance of electric power measurements [Text] / O. M. Velychko, S. R. Karpenko // Journal of Physics: Conference Series. - 2016. - Vol. 772. - P. 012043. doi: 10.1088/1742-6596/772/1/012043
18. Velychko, O. Evaluation of competence of the experts in field of metrology and instrumentations [Text] / O. Velychko, T. Gordiyenko // XXI IMEKO World Congress "Measurement in research and industry". - Prague, Czech Republic, 2015.
19. Podinovskiy, V. V. Pareto-optimal'nye resheniya mnogokriterial'nyh zadach [Text] / V. V. Podinovskiy, V. D. Nogin. - Moscow: Nauka, 1982. - 256 p.
20. Velychko, O. N. Evaluation of experts competence on the measurement of electrical power using the method of analytic hierarchy [Text] / O. N. Velychko, T. B. Gordienko, S. R. Karpenko, L. V. Kolomiets // Metallurgical and Mining Industry. - 2016. -Issue 11. - P. 70-76.
21. Velychko, O. The evaluation of activity of Technical Committees of Standardization for Metrology and Measurement on national level [Text] / O. Velychko, T. Gordiyenko // Journal of Physics: Conference Series. - 2016. - Vol. 772. - P. 012007. doi: 10.1088/1742-6596/772/1/012007
