CC BY
103
UDC 621.9.004.12
Doi: 10.31772/2587-6066-2018-19-4-605-612
For citation: Frolova E. A. [Multilevel evaluation of quality interactive electronic technical manuals for aviation technology]. Siberian Journal of Science and Technology. 2018, Vol. 19, No. 4, P. 605-612. Doi: 10.31772/2587-60662018-19-4-605-612
Для цитирования: Фролова Е. А. Многоуровневая оценка качества интерактивных электронных технических руководств для авиационной техники // Сибирский журнал науки и технологий. 2018. Т. 19, № 4. С. 605-612. Doi: 10.31772/2587-6066-2018-19-4-605-612
MULTILEVEL EVALUATION OF QUALITY INTERACTIVE ELECTRONIC TECHNICAL MANUALS FOR AVIATION TECHNOLOGY
E. A. Frolova
Saint Petersburg State University of Aerospace Instrumentation 67, Bolshaya Morskaya Str., Saint Petersburg, 190000, Russian Federation E-mail: frolovaelena@mail.ru
The method of multi-level quality assessment of interactive electronic technical manuals (1ETM) for aircraft contains the main approaches, methodological and logical-mathematical tools for the implementation of multiparameter evaluation of the quality of electronic content of these manuals. The paper postulated that the goal of assessing the quality of 1ETM for aviation equipment is to obtain a consolidated conclusion on the degree of satisfaction of the need to create interactive electronic technical manuals in the conditions of vagueness of the initial expert data used for evaluation.
It is shown that the quality assessment of interactive electronic technical manuals for the operation and repair of aviation equipment is obviously linked to non-numeric or "soft" measurements, due to the fuzzy nature of the initial qualimetric data obtained from an expert. In this version, "measurement" should be understood as a manipulation in which some strictly ordered quality gradations correlate in accordance with the 1ETM for the operation and repair of aviation equipment properties. 1n the role of quantitative measurement results, not only real numbers are considered, but also other algebraic groups that necessarily have an order relation between their elements, that is, a similarity of the inequality relation between numbers.
The multi-level quality assessment of interactive electronic technical manuals for aeronautical engineering consists of the following steps: 1. Imagery of numerical vectors of values of elementary quality indicators; 2. Calculation of ranks that determine the significance of quality indicators in their compositions based on fuzzy and (or) insufficient initial data on the compositional significance of more particular indicators in the composition of more aggregated indicators: 3. Calculation of elementary indicators values and ratings of importance based on numerical vectors, for the analyzed performance of 1ETM on the operation and repair of aircraft, the values of the integral quality indicator, as the weighted arithmetic average of mathematical similar to a particular indicators.
Keywords: interactive electronic technical manuals, aviation technology, multi-level assessment, quality parameters.
МНОГОУРОВНЕВАЯ ОЦЕНКА КАЧЕСТВА ИНТЕРАКТИВНЫХ ЭЛЕКТРОННЫХ ТЕХНИЧЕСКИХ РУКОВОДСТВ ДЛЯ АВИАЦИОННОЙ ТЕХНИКИ
Е. А. Фролова
Санкт-Петербургский государственный университет аэрокосмического приборостроения Российская Федерация, 190000, г. Санкт-Петербург, ул. Большая Морская, 67 E-mail: frolovaelena@mail.ru
Метод многоуровневой оценки качества интерактивных электронных технических руководств (ИЭТР) для авиационной техники содержит основные подходы, методологические и логико-математические инструментарии осуществления многопараметрического оценивания качества электронного контента указанных руководств. Постулировано, что цель оценки качества ИЭТР для авиационной техники есть получение сводного заключения о достигнутой степени удовлетворения потребностей в создании интерактивных электронных технических руководств в условиях нечеткости начальных экспертных данных, используемых для оценки.
Показано, что оценка качества интерактивных электронных технических руководств по эксплуатации и ремонту авиационной техники, очевидно, увязана с нечисловыми или «мягкими» измерениями, в силу нечеткого характера начальных квалиметрических данных, получаемых от эксперта. В таком варианте измерение
следует понимать как манипуляцию, при которой свойствам ИЭТР по эксплуатации и ремонту авиационной техники ставятся в соответствие некоторые строго упорядоченные градации качества. В роли количественных результатов измерения рассматриваются не только действительные числа, но и другие алгебраические группы, обязательно имеющие отношение порядка между своими элементами, т. е. подобие отношения неравенства между числами.
Многоуровневая оценка качества интерактивных электронных технических руководств для авиационной техники состоит из следующих этапов: 1) построение численных векторов значений элементарных показателей качества; 2) расчет рангов, определяющих значимость показателей качества в их композициях по получаемым в ходе экспертизы нечетким и(или) недостаточным исходным данным о композиционной значимости более частных показателей в составе более сводных показателей; 3) расчет на основе численных векторов значений элементарных показателей и оценок рангов значимости для анализируемого исполнения ИЭТР по эксплуатации и ремонту авиационной техники значения интегрального показателя качества как взвешенного среднего арифметического значения математически аналогичных более частных показателей.
Ключевые слова: интерактивные электронные технические руководства, авиационная техника, многоуровневое оценивание, параметры качества.
Introduction. The aim of assessing the quality of IETM for aircraft is to obtain a consolidated opinion on the achieved degree of satisfaction of the needs for the creation of interactive electronic technical manuals, in the conditions of the fuzziness of the initial expert data used for evaluation. The assessment of the quality of the IETM on operation and repair of aviation equipment is the determination of their compliance with the requirements of a predetermined network of indicators of this quality.
Mathematical basis for setting quality indicators for multi-level assessment. Assessment of the quality of interactive electronic technical manuals (IETM) on operation and repair of aviation equipment is the determination of their compliance with the requirements of a predetermined network of indicators of this quality [1]. If the indicator is a complex nesting of other, simpler indicators (aggregated indicator), then the one-time act of analysis and evaluation of quality develops into a multi-step sequence of qualimetric evaluation. The aggregated indicators in this qualimetric evaluation will be compositionally formed from more simple indicators, resulting in the formation of hierarchical network of indicators. In such a hierarchical network at the terminal level, as a rule, there are indicators that can be directly measured or expertly evaluated, quantitatively or qualitatively. Such indicators of the terminal level of the hierarchical network of indicators of assessment of quality of IETM on operation and repair of aircraft are called elementary. Their set is denoted by {q,}. At higher levels of the hierarchical network of quality assessment of IETM on operation and repair of aviation equipment compositionally complex, called aggregated, indicators {q,j} are established, aggregating weighted composition of indicators, from the composition of elementary indicators and(or) other aggregated indicators. Root vertex of the specified hierarchical network results is an integral index Q0, that is the quality of the IETM on operation and repair of aircraft.
As part of this hierarchical network, the compositional or aggregate importance of any quality indicator qt for the assessment of the integral indicator Q0 will be different. Accordingly, for the numerical fixing of composite or aggregate indicator of the importance qt in the convolution of the nearest aggregated indicator according to the structure of the hierarchical network of indicators, the weight coefficient wm, n is entered - the local rank
of the m-th indicator in the n-th convolution, wm, n e (0,1); wm, n e R. In addition, for each aggregated indicator the normalization condition must be satisfied:
Z wm,n =1
m
Accordingly, the numerical value of the rank of each elementary or intermediate aggregated index qt in the convolution of the integral index Q0, according to the hierarchical network of indicators, is determined by the weighting factor bm* - the global rank of the m-th index in the convolution of the integral Q0. Respectively
Qo
bm =n wm,n,
qm
where bm* e (0,1); bm* e R.
The specific features of the definition of input information from the expert group for measuring or evaluating local and global ranks determines the classification basis for identifying of the corresponding particular methods within the framework of the developed method of quali-metric assessment of IETM on operation and repair of aviation equipment. The basic feature of the separation of a particular sub-method is a specific feature of obtaining expert information at the input [2; 3]. As a rule, the role of this feature includes the dimension and character of the scale for instrumental measurement or qualitative and quantitative assessment of elementary indicators, a form for taking into account the fuzziness of the input expert data, differences in the mathematical convolution of intermediate aggregated and elementary indicators into the assessment of an integrated indicator. Algorithmic-qualimetric essence of quality assessment is initially reduced to the definition of a set of properties (individual qualities) of the object being evaluated - IETM on operation and repair of aviation equipment. The values of the estimates for these properties are evaluated on a scale.
Assessment of the quality of IETM on operation and repair of aircraft is associated with non-numeric or "soft" measurements. "Measurement" should be understood as a manipulation in which the properties of the IETM on operation and repair of aircraft equipment are correlated in accordance with some strictly ordered gradations of quality. With this approach, not only real numbers are considered in the role of quantitative measurement results, but
also other algebraic groups, necessarily having an order relation between their elements, a number of specialized scales of measurement and evaluation of the quality properties.
Objectively predetermined use of soft, non-numeric measurements in assessing the quality of IETM on operation and repair of aircraft specifies the fact that particular and aggregated indicators from a single hierarchical network of quality assessment indicators can be assessed or measured on scales (nominal, ordinal, ratios, differences), accordingly, the numerical values of these measurements and estimates will be set on different algebraic groups.
According to [4-8], it was established that for the categorical features peculiar to the IETM on operation and maintenance of aviation technology quality assessment the rational form of convolution of the integral evaluation criterion would be the additive linear criterion of the normalized form:
n
Q = E .
i=1
The above-shown application of heterogeneous scales of soft measurements or evaluation of elementary indicators leads to such a nature of the data that is input to the method, that this character is understood as insufficiency or deficiency of the input information [9]. This affects the algorithm for calculating the local ranks |w,}, which play the role of accounting for the different values of the input data in the developed method of assessing the quality of IETM on operation and repair of aircraft. Based on the foregoing, the mathematical apparatus of randomized aggregated indicators [10-14] was adopted for the mathematical basis of the developed qualimetric method.
Aggregated and integral indicators of quality of IETM on operation and repair of aircraft are calculated through the normalized additive convolution of the elementary indicators included in the corresponding network of indicators [15]. Basing on this, it is possible to evaluate k alternative options for the implementation or projects of IETM on operation and repair of aircraft. It is assumed that their sets of estimates for elementary indicators are represented by numerical vectors qj) = (q^,..., qm(j)), i = 1,., m, j = 1,., k. Each such numerical vector is a multiparametric assessment of the corresponding alternative implementation of IETM on operation and repair of aircraft in the form of a family of values of elementary indicators q = (qi,..., qm). Further, it is assumed that on the community of all evaluated realizations of IETM, quantitatively described by the indicated numerical vectors of evaluations by elementary indicators, the relation of strict dominance is given:
( > q(s)))((Vi q« > q«)a(( qf > qjs))). (1)
The expression describing condition (1) should be considered as follows: IETM q(r) is superior IETM q(s) in the considered indicator when it is not less preferable by each component of the elementary index (q/r) > q/s)) and there is an elementary or more particular aggregated indicator for which the first IETM is preferable to the second ((qjr) > q/s))). The ordering of the analyzed IETM on operation and repair of aircraft will be a strict ordering
in accordance with (1). Accordingly, together with the strict ordering relationship by preference, it is necessary to introduce a non-strict order relation:
(q(r)>q(s)) ((q(r) >q(s))v(Vi q(r) = q(s))). (2)
In this case, it is possible to determine the relation of strict order through the relation of non-strict order:
(q(r) > q(s)) » ((q(r) >q(s)) A (q(r) * q(s))). (3)
When carrying out the ordering of IETM on operation and repair of aircraft using the dominance ratio, a significant complexity arises - the presence of a large number of objects of assessment q/r), q(s), which are not comparable in relation to the non-strict order (2). The probability of comparability with respect to (2) of these random vectors is determined by the expression
P{(q(r'' > q() v(qq(> q(r) )> =
1
(4)
From (4) it can be concluded that the ability to find comparable multi-criteria quality assessments of IETM on operation and repair of aviation technology decreases rapidly with an increase in the number of analyzed indicators. So, if IETM is estimated by m = 11 indicators then the probability that a pair of randomly selected electronic manuals is comparable in all indicators at once is less than one thousandth. To ensure the comparability of multiparameter quality assessments aggregated indicators are used the essence of which is to form elementary indicators q = (q1v.., qm) along a vector of some aggregated indicator Qk, which is a function Q = Q(q) = Q(qu..., qm) of a vector of elementary exponents q satisfying the condition of monotonicity in the form
V q
U) q(l) e{q : q = (qi,...,qm),qt
:[0,1]>
{q(j) >q(l)> ^ {Q(q(J))>Q(q())>.
Ji )
( j h
(5)
In turn, the most powerful aggregated indicator is an integral indicator of Q.
Synthesis and ranking of a multi-level tree of assessment indicators. In the most general form the function of synthesis or convolution of elementary indicators in the aggregated ones for IETM on the operation and repair of aviation equipment takes the form
(q;w) = (ql,.., qm; wl,..., wm) = ^ (^^jm! w ^ (qi) .
Conducting the substitution of stretched / compressed values aiqi(0), i = 1,...,m in the convolution formula allows synthesizing the following expression
Qx («1 q(0),-, «m?mo); w)=
= [I^ qW ^ = A Qx (qi0),..., qf; w). (6)
Basing on expression (6) the following conclusions can be made: if elementary indicators are measured or assessed on the scale of relations, then the multiplicative aggregated indicator is measured or estimated on the scale of relations with stretching / compression A.
Defining and marking the so-called "difference scale shift", setting the beginning of reference as qi = 0 and the end, as qi = 1, is much easier than choosing the coefficient of stretching / compression of the ratio scale. This is the main reason for choosing the additive normalized synthesizing function in the proposed method for estimating IETM on the operation and repair of aircraft. In the mathematical apparatus of the method of quality assessment of IETM on operation and repair of aviation equipment, it is precisely the additive normalized convolutions Q+(q; w) of elementary indicators that are used to calculate the aggregated and integral estimates of these electronic manuals.
An example of the synthesis of a multi-level tree of indicators for the assessment of IETM on operation and repair of aircraft is presented in fig. 1. We consider: some list of five indicators, and initial values of quality characteristics, compositionally included in the aggregated and
integral quality indicators of IETM on operation and repair of aviation equipment, as well as a set of vectors of values of elementary quality indicators of nine realizations of a specific IETM from various competing development organizations for which a noticeable difference in technological approaches, development methods, software implementation, etc. is stated. It is implied that the vectors of values of the elementary indicators of the quality of IETM was conducted according to the originally specified five basic indicators z1,..., zn. From the initial two tables in fig. 1 it follows that all elementary indicators z1,., z5 are estimated (i. e. "softly measured") on the scale of names, except for the indicator z5, which is estimated on the ordinal scale. In the framework of the example, it is obvious that in order to ensure further integral convolution, it is necessary to evaluate all the considered indicators on the ordinal scale: <Satisfies; rather satisfies than does not satisfy; Does not satisfy>.
№ Alternative development options $1 gl §! $1 gS
1 CPS 0 1 1 0 1
2 NPO «Aviora» 1 i 1 1 0
г NPO ((Mars» (1 0.5 0 0 0
А NPO «Agat» 0 0 0 0 0,5
5 KB «Ruhm» 0,5 I ä 0 0,5
6 SPIIRAN - MTB VT 0 1 1 1 0.5
7 NPP «Radar uuns» 0 1 0 0 0
8 CNPO «Leainec» 0.5 0 0.5 0.5 0.5
9 GK «Krondshtadt» 0 0 0 0 1
Fig. 1. An example of setting IETM quality indicators when synthesizing a multi-level tree of assessment indicators
Рис. 1. Пример задания показателей качества ИЭТР при синтезе многоуровневого дерева показателей оценки
Let us assume that, in the framework of the considered example, the experience of experts allowed the scaling of the values shown in fig. 1 to the indicated ordinal scale.
The obtained values of the elementary indicators y^..., y5 of the quality of the IETM have exactly the ordinal character. This means that they are not real numbers and arithmetic operations cannot be performed on them. To carry out such operations, the indicated values of the elementary exponentsy1,., y5 must be given a numerical form. In other words, it is necessary to specify the mapping x = ^(y) of the gradations of the index y to the set of real numbers R1. This mapping should preserve the order of the gradations. From the infinite family of admissible assignments of ordinal scales, on which the parameters y1,., y5 are measured, in the framework of the example a simple transformation was initially chosen, resulting in the above mentioned ordinal scale to a decreasing number of natural numbers, respectively: <3, 2, 1 >. On the basis of this conversion, the values X1,. . ., X5 of the indicators are obtained, directly estimated on the scale of real numbers R1 and rolled up into quality aggregated indicators of IETM on the operation and repair of aircraft.
Traditionally, both in the framework of the given example of estimating elementary indicators X1,. . ., X5, and in the established practice of evaluating other combinations of elementary indicators of IETM on operation and repair of aircraft, the use of linear normalization is accepted and practically justified. This allows getting the values of elementary indicators q1,., q5 in the form reduced to the (0, 1) interval, as shown in the table in the lower part of fig. 1. for the observed example. Each of the rows in the table at the bottom of fig. 1 is a multiparametric estimate of q = (q1,., q5) vector characteristics of IETM on operation and repair of aircraft of specific performance from a particular contractor.
Determining the values of the ranks that determine the significance of the IETM quality indicators in their compositions from the fuzzy and (or) insufficient initial data obtained during the examination. It consists in building a family of all possible numerical vectors of significance ranks, taking into account fuzzy (expert) data on the priority of the relevant indicators, and calculating a aggregated numerical vector of ranks of significance. This device provides for the construction of a numerical grid for m IETM quality indicators with discrete step n. Further, this grid is thinned out according to fuzzy data I from experts on the priority of certain indicators. Formally, such initial data are presented in the form of a system of inequalities. As a result of exceptions from the initial digital grid of numerical vectors that do not correspond to fuzzy (expert) data I, n numerical vectors remain, after normalization of which it becomes possible to calculate the ranks of compositional significance for m IETM quality indicators. In other words, ranks of significance are defined as the algebraic average over each of the m indicators on the set of n thinned vectors of N initial number vectors. Each rank of significance, taking into account the fuzziness of the data set for the thinning of the original digital grid, is calculated as a random variable: that is, the mathematical expectation of the value of the rank of significance is determined
N ( m,n)
V w(t)
1 N (m,n) / j i
wi = Mwi =- V wit) = —--, (7)
N (m, n) t=1 m
and the corresponding value of the variance of a random variable or standard quadratic deviation as a measure of the accuracy of this significance rank value
I 1 N(m,n) 2
*=^ iNimn) VO"' -w ] =
■y m2 (m +1) nm(m +1)
The proposed mathematical and algorithmic tools for taking into account the compositional importance of simpler indicators for assessing the quality of IETM in the composition of more complex ones using ranks of significance determined by the methods of "soft computing", make it possible to take into account any fragmentary and fuzzy expert data on the comparative weight of individual indicators in the aggregated ones. This provides the developed method of evaluation with the necessary flexibility when working with the initial heterogeneous information of qualimetric evaluation.
The procedure of ranking indicators in the compositions, as a fundamental component of the method for assessing the quality of IETM on operation and repair of aviation equipment, can be illustrated with an example for three simpler indicators (m = 3) that are combined into a composition of a more complex indicator. The ranking results are presented in the right table in fig. 2.
Analyzing the presented example of the implementation of the procedure of ranking indicators in the compositions of more complex quality indicators of IETM on operation and repair of aviation equipment, it is necessary to pay attention that the mathematical consideration of the fuzziness and noise of the input qualimetric data I makes it possible to increase the accuracy of estimates of the ranks of the importance of elementary indicators wi (I),
i = 1,., m, in aggregated and integral indicators, Q+j \ j = 1,., k. In practice, this is reflected in a decrease in the standard deviation of s(I) and S<JJ(I), as well as an increase in the reliability of the ranking of elementary and aggregated indicators in the compositions (direct to 1 the probabilities of dominance ofp(r, s; I), r, s = 1,., m and P(j, l; I), j, l = 1,., k).
The procedure for calculating the values of indicators of quality assessment IETM. The calculation of the values of indicators for assessing the quality of IETM on operation and repair of aircraft in the conditions of the primary application of the procedure for ranking indicators in compositions of more complex quality indicators has a number of specific features:
- when determining the ranks of significance on the basis of direct, estimated input data on the priorities of simpler indicators of evaluation;
- in determining the ranks of significance based on expert sampling of data consisting of examples of previously evaluated implementations of IETM;
- when determining the ranks of significance based on fuzzy initial data on the priority of elementary and aggregate indicators in the composition of the integral;
- when determining the ranks of significance based on a multi-level hierarchical network of indicators.
The structure of the procedure for calculating the values of the quality assessment indicators of IETM on operation and repair of aviation equipment is shown in generalized form in fig. 3
The considered procedure for assessing IETМ on the operation and repair of aircraft, depending on the methods of obtaining input expert data, is as follows:
a) quality assessment in determining ranks of significance based on direct, estimated input data on the priorities of simpler assessment indicators;
b) assessment of quality in determining the ranks of significance based on an expert sample of data consisting of examples of previously evaluated implementations of the IETM;
c) quality assessment in determining the ranks of significance based on fuzzy initial data on the priority of elementary and aggregated indicators in the composition of the integral;
d) quality assessment in determining the ranks of significance based on a multi-level hierarchical network of indicators.
The generalized scheme for calculating the values of the IETM quality assessment indicators, for evaluating specific implementations of these manuals, includes three main steps:
1. Construction of numerical vectors of values of elementary quality indicators.
2. Calculation of ranks that determine the significance of quality indicators in their compositions according to fuzzy and (or) insufficient initial data on the compositional significance of more particular indicators in the composition of more aggregated indicators.
3. Calculation of the integral quality indicator, as a weighted arithmetic average of mathematically analogous more particular indicators on the basis of numerical vectors of values of elementary indicators and ratings of significance for the analyzed performance of IETM on operation and repair of aviation equipment values.
Fig. 2. An example of the procedure of ranking indicators implementation in the compositions of more complex indicators of quality
Рис. 2. Пример реализации процедуры ранжирования показателей в композициях более сложных показателей качества
Fig. 3. A generalized presentation of the structure of the procedure for calculating the values of IETM quality assessment indicators
Рис. 3. Обобщенное представление структуры процедуры расчета значений показателей оценки качества ИЭТР
The described generalized scheme for calculating the values of quality assessment indicators is the essence of the developed quality assessment method of IETM on operation and repair of aircraft equipment. It allows you to generalize the previously described and disclosed procedures for scaling and randomizing ranks of the significance of quality assessment indicators for the specified IETM logically.
Conclusion. The proposed method of multilevel quality assessment of IETM on aviation technology, based on the use of a mathematical soft computing apparatus for processing primary qualimetric data obtained from experts and making it possible to draw conclusions about the achieved quality level of created IETM on the operation and repair of aviation equipment.
References
1. Frolova E. A. [Scientifically-methodical concept of quality management of interactive electronic technical
manuals on operation and repair of aviation equipment].
Voprosy radioehlektroniki. 2018, No. 10, P. 66-74 (In Russ.).
2. Michurin S. V., Semenova E. G., Frolova E. A. [Formal presentation of spatial process management quality indicators]. Conditions alternative to the development of modern economic systems management, innovation. Proceedings of the International scientific and practical conference. 2016, P. 182-192.
3. Ivakin Ya. A., Semenova E. G., Morozov S. A., Smirnova M. S. [he hierarchy of indicators for assessing the quality of software and hardware complexes of data processing and storage centers]. Vestnik Sankt-Peterburgskogo gosudarstvennogo universiteta tekhnologii i dizajna. Seriya 1. Estestvennye i tekhniches-kie nauki. 2017, No. 5, P. 65-68 (In Russ.).
4. Boem B. U. et al. Kharakteristiki kachestva pro-grammnogo obespecheniya [Software quality characteristics]. Moscow, Mir Publ., 1981, 312 p.
5. Onishchenko V. Ya. [Classification and comparative assessment of risk factors]. Bezopasnost' truda vpro-myshlennosti. 2008, No. 7(54), P. 28-37 (In Russ.).
6. Krui M., Galei D., Mark R. Osnovy risk-menedzhmenta [Fundamentals of risk management]. Moscow, Yurait Publ., 2011, 400 p.
7. Daiitbegov D. M., Kalmykov O. V., Cherepanov A. I. Programmnoe obespechenie statisticheskoi obrabotki dannykh [Software for statistical data processing]. Moscow, Finansy i statistika Publ., 1984, 211 p.
8. Semenova E. G., Frolova E. A. Strukturirovanie trebovaniy k pokazatelyam funkcionirovaniya bortovyh kompleksov [Structuring requirements for performance indicators of airborne systems]. GUAP, 2004, VINITI, 12.02.2004 244-V2004. 9 p.
9. Semenova E. G., Smirnova M. S., Tushavi V. A. Decision making support system in multi-objective issues of quality management in the field of information technology. Research Journal of Applied Sciences. 2014, Vol. 9, Iss. 12, P. 1078-1081.
10. Khovanov N. V. Metod randomizirovannykh svodnykh pokazatelei [The method of randomized summary indicators]. St. Petersburg, SPbGU Publ., 1999, 8б p.
11. Khovanov N. V., Mikhailov M. V., Khovanov N. V. Otsenka slozhnykh finansovo-ekonomicheskikh ob'ektov s ispol'zovaniem sistemy podderzhki prinyatiya reshenii ASPID-3W [Evaluation of complex financial and economic objects using the ASPID-3W decision support system]. St. Petersburg, Izdatel'stvo Sankt-Peterburgskogo Gosudarstvennogo Universiteta Publ., 2004, бЗ p.
12. Popovich, V. V., Ivakin, Y. A., Shaida, S. S. Theory of search for moving objects. Oceans Conference Record (IEEE). 2002, No. 3, P. 1319-1329.
13. Bromfield M. A., Walton T., Wright D., Rusby M. Evaluation of cots solutions to support flight operations quality assurance in business/corporate aviation. Aviation Technology, Integration, and Operations Conference. 2018, No. AIAA 2018-3197, 13 p.
14. Liao J., Zang Y., Zhou Z., Luo X. Quality evaluation method and optimization of operating parameters in crop aerial spraying technology. Nongye Gong-cheng Xuebao / Transactions of the Chinese Society of Agricultural Engineering. 2015, No. 31, P. З8-4б.
15. Michurin S. V., Michurin S. V., Smirnova M. S. [The structure of the system of requirements for a quali-metric assessment of situational spatial process management]. Voprosy radioehlektroniki. 201б, No. б, P. 33-35 (In Russ.).
Библиографические ссылки
1. Научно-методическая концепция менеджмента качества интерактивных электронных технических руководств по эксплуатации и ремонту авиационной техники / Е. А. Фролова [и др.] // Вопросы радиоэлектроники. 2018. № 10. С. бб-74.
2. Фролова Е. А., Мичурин С. В., Семенова Е. Г. Формальное представление показателей качества управления пространственными процессами // Condi-
tions alternative to the development of modern economic systems management, innovation: Proceedings of the International scientific and practical conference. 2016. С. 182-192.
3. Иерархия показателей оценки качества программно-аппаратных комплексов центров обработки и хранения данных / Я. А. Ивакин [и др.] // Вестник Санкт-Петербургского государственного университета технологии и дизайна. Сер. 1 «Естественные и технические науки». 2017. № 5. С. 65-68.
4. Характеристики качества программного обеспечения / Б.У. Боэм [и др.]. М. : Мир, 1981. 312 c.
5. Онищенко В. Я. Классификация и сравнительная оценка факторов риска // Безопасность труда в промышленности. 2008. № 7(54). С. 28-37.
6. Круи М., Галэй Д., Марк Р. Основы риск-менеджмента / пер. Н. Сологуб. М. : Юрайт, 2011. 400 с.
7. Дайитбегов Д. М., Калмыков О. В., Черепанов А. И. Программное обеспечение статистической обработки данных. М. : Финансы и статистика, 1984. 211 с.
8. Семенова Е. Г., Фролова Е. А. Структурирование требований к показателям функционирования бортовых комплексов // ГУАП. 2004. Деп. в ВИНИТИ 12.02.2004. № 244-В2004. 9 с.
9. Semenova E. G., Smirnova M. S., Tushavi V. A. Decision making support system in multi-objective issues of quality management in the field of information technology // Research Journal of Applied Sciences. 2014. Vol. 9, iss. 12, 2014, P. 1078-1081.
10. Хованов Н. В. Метод рандомизированных сводных показателей. СПб. : Изд-во СПбГУ, 1999. 86 с.
11. Хованов Н. В., Колесов Д. Н., Михайлов М. В. Оценка сложных финансово-экономических объектов с использованием системы поддержки принятия решений АСПИД-3W. СПб. : Изд-во Санкт-Петербургского государственного университета, 2004. 63 с.
12. Popovich V. V., Ivakin Y. A., Shaida S. S. Theory of search for moving objects // Oceans Conference Record (IEEE). 2002. № 3. P. 1319-1329.
13. Evaluation of cots solutions to support flight operations quality assurance in business/corporate aviation / M. A. Bromfield [et al.] // Aviation Technology, Integration, and Operations Conference. № AIAA 2018-3197. 2018. 13 p.
14. Quality evaluation method and optimization of operating parameters in crop aerial spraying technology / J. Liao [et al.] // Nongye Gongcheng Xuebao / Transactions of the Chinese Society of Agricultural Engineering. 2015. № 31. P. 38-46.
15. Мичурин С. В., Ивакин Я. А., Смирнова М. С. Структура системы требований квалиметрической оценки ситуационного управления пространственными процессами // Вопросы радиоэлектроники. 2016. № 6. С. 33-35.
© Frolova Е. А., 2018
