CC BY
12
- Экономицеские HayKU-
INTEGRATED ASSESSMENT OF INNOVATIVE SOFTWARE PROJECTS QUALITY:
A SYSTEM OF INDICATORS
S.N. Larin, Candidate of Technical Sciences, Leading Researcher I.M. Ermakova, Researcher
Central Economics and Mathematics Institute, RAS (Russia, Moscow)
DOI:10.24412/2500-1000-2022-3-3-179-184
Abstract. Currently, there is a rapid informatization of management processes in almost all industries. To manage business activities, many enterprises use modern information systems and technologies. This allows them to gain new competitive advantages. However, in the development and implementation of such information products as innovative software projects, there are few problems. Most of them are related to the definition of the composition of the requirements for the quality of innovative software projects. At the same time, the current state in the field of assessing the quality of innovative software projects indicates the existence of a number of approaches to this process. But it is not possible to develop a universal set of private indicators that make up an integral indicator for quality assessing of such projects. Therefore, the authors of the article proposed a new approach. It allows, in each specific case, to form a set ofparticu-lar indicators that is adequate to the scope of the relevant software tools as part of a specific innovative software project. Its essence is to substantiate the type of integral quality indicator based on a hierarchical network of private indicators and develop a procedure for aggregating private indicators into an integral quality indicator of a specific innovative software project.
Keywords: innovative software projects, quality assessment, hierarchical network, integrated indicator.
Introduction. Modern information systems and technologies occupy a leading position in the management systems production of many enterprises. Their usage allows enterprises to gain competitive advantages. However, introducing information systems into the activities of enterprises, it often leads to worse results compared to those that were before their application. According to statistics, only 10% of the customers completely satisfied with developed information systems. There are many reasons for this, but one of the most important is the low quality of software products and information systems [5].
The rapid growth of information volumes predetermines the complexity of the process of developing innovative software projects (ISP). It is mainly due to a large number of requirements for quality indicators and characteristics of their properties. This circumstance increases the importance of using formalized methods to determine the quality of modern ISP development.
The model for assessing the quality of software (SW) using a larger number of char-
acteristics of quality indicators describes the ISO standard in sufficient detail - ISO/IEC 25010:2011 Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models [13]. Its counterpart in Russia is GOST R ISO/IEC 250102015 [2].
Main part
The assessment of the quality of specific ISPs in modern conditions is complicated by the lack of generally accepted definitions of private indicators related to the development, usage and maintenance of these projects. Different authors offer different sets of partial indicators and their metrics, i.e., manifestations of the characteristics of certain properties. However, the main difficulty is the relation of different parameters, which often have opposite directions. Other difficulties stem from the fact that quality metrics are usually only partial characterizations of the relevant parameters. In addition, a significant part of them is determined only indirectly and is not always measurable [1, 4, 6, 8].
A brief analysis of the current state of ISP quality assessment allows us to draw the following conclusions:
1) the desired qualities of an ISP are difficult to align with the needs and priorities of the intended user;
2) there is no one common metric that could give a universally useful quality assessment of software in general, and specific types of ISPs in particular;
3) at the best case, the intended user can obtain a useful assessment by providing the quality assessment system with a complete set of checklists and priorities in accordance with GOST;
4) integrated score at the result will always be approximate, as quality assessment methods are not exhaustive [1, 11].
Thus, it is not possible to offer a universal set of private indicators and, on its basis, form an integral indicator for assessing the quality of ISP. This fact follows that each specific subject in specific area of knowledge has its own needs for certain parameters of the types of software used as part of the same ISP and its own idea of its quality. Therefore, it is impossible in practice to create a universal multilevel decomposition of ISP quality indicators into its constituent parameters.
However, the authors proposed a new approach. It allows for each specific case to effectively synthesize a set of private indicators that is adequate to the scope of the corresponding type of software as part of the same ISP. The essence of this approach is to bring together all the requirements for ISP into a single network and to make a reasonable choice on the resulting network of the analytical form of the integral quality indicator.
In accordance with the principles of the quantification procedure itself, such network will have a hierarchical structure. In turn, the presence of a hierarchical structure of indicators in each specific case will allow for an assessment of the quality by identifying the values of the indicators for the assessed ISP and their further aggregation. The formation of an ISP quality score in accordance with this approach is:
1) on the basis of a hierarchical network of partial indicators, the form of the integral indicator of quality ISP is substantiated,
2) then the procedure for aggregating private indicators into an integral quality indicator ISP is implemented.
The application of this approach provides:
1) synthesis of a hierarchical network of partial quality of ISP indicators;
2) assessment of the significance (weight) of each of the partial indicators for the corresponding integral indicator of a higher level of the hierarchy, taking into account the fuzz-iness of the initial information;
3) convolution of the hierarchical structure of private indicators into an integral ISP quality indicator, taking into account the values and weights of all private indicators.
It should be noted that the proposed approach involves the usage of tools that will ensure the automation of quality assessment. Its development is based on the Boehm quality assessment methodology and its development for the ISO 9000 standards system. The essence of this methodology is as follows:
- initially, the original set of private indicators (with weights subjectively assigned and not ordered among themselves) is evaluated by experts;
- the value of the integral indicator of ISP quality is determined by linear convolution [12].
The essence of the proposed direction for the development of the Boehm methodology is to substantiate the possibility and develop procedures for its application not only for solving the problem of quality assessment, but also for the task of identifying individual shortcomings of the evaluated ISP.
The input data for using this approach are:
- preference matrices (akij i, j, k e n) k-th
expert on a family of quality indicators;
- a set of values of evaluations of partial quality indicators {Ci} into numerical {y} and in linguistic form { y };
- set of pairwise comparison matrices ( Vij , I, i, j e n) for L decomposable group of
quality indicators [6, 8, 9].
The output data for using the proposed approach are:
- formalized description of the network of quality indicators G;
- values of quality assessment in numeric Yp and in linguistic YP forms;
- Экономицеские HayKU-
- values of quality assessments of compo-sitionally complex indicators (parameters) Ci in numeric y(Ci) and in linguistic y (Ci) form [11, 14].
With a large number of particular indicators cj,...,cm, characterizing the ISP and / or with a significant heterogeneity of these indicators, it becomes advisable to move to a hierarchical system. At each level of this system, individual indicators of this level are sequentially aggregated into group indicators of the next level. This process of increasing the level of aggregation of group indicators ends with the construction of a single summary indicator that synthesizes all individual and group indicators of previous levels. To obtain a single conclusion about the quality of ISP, it is necessary to introduce an integral indicator that will reflect the general level of its development. In traditional methods for assessing software quality, the integral indicator has the form:
Y = f(Уl(сl), У2(с2),..., Уl(cl),..., yn(cn))
(1)
where: yi(ci) - quality assessment according to the i-th elementary parameter ci;
n - number of elementary parameters;
Here, elementary parameters are understood as independent monosyllabic characteristics that do not require further decomposition. These ISP parameters are evaluated by individual, not group indicators.
Further specification of the form of the integral ISP quality indicator depends on the mathematical features of the form of the general summary quality criterion. Theoretically, the same elementary characteristic can be measured in different scales. The indicated possibility of choosing a measurement scale will make it possible to move from the original, often incomparable scales for measuring heterogeneous characteristics to their measurement in a single scale. The transition to a single scale for measuring elementary characteristics (indicators) provides:
1) the possibility of further correct aggregation of all private (elementary and group) indicators into indicators of a higher level of the hierarchy, taking into account their significance (weight);
2) measurement of all private (elementary and group) and integral quality indicators in a single scale.
At the same time, if we confine ourselves to the task of identifying the ordering of the evaluated options according to some measured characteristic, then the measurements on any of the transformed scales can be considered equivalent (the order of the gradations of the measured parameter is an invariant of all such measurements) [3, 7].
Since the choice of specific forms of quality indicators has a fairly wide variability, in order to bring the estimates of various parameters to a single (comparable) form, it is necessary to choose a specific path. A solution of the problem of determining specific forms of presentation of quality indicators requires narrowing the set of available alternatives. Let all private indicators be measured on a scale (0,9)e^, where set of real numbers:
- 0 - complete non-compliance of parameter ci with the required level;
- 5 - essential correspondence of the parameter ci with the required level;
- 9 - absolute parameter ct match with the required level.
Determining the estimates yi(c) on the segment (0,9) of the real axis allows us to conclude that they are continuous. A specific numerical representation of yi(ci) in each particular case of evaluation in the future makes it possible to significantly simplify the form of the integral indicator (1).
In (1), the form of the integrated quality indicator of ISP reflects the fact that the Boehm methodology does not take into account the importance of various indicators, i.e., private indicators and characteristics of their parameters are not ordered by the degree of their influence on the quality of the ISP generally. Therefore, in order to take into account, the values and degree of influence of private indicators and the characteristics of their parameters, aggregated into an indicator of a higher level of the hierarchy, on the value of the integral indicator, it is necessary to sort the aggregated private indicators by importance (weight).
Such an iterative multilevel process for assessing the degree of manifestation of individual complex of parameters Yim at the m-th level of decomposition of the integral quality
indicator, through the values of indicators of a lower-level m+1 of the hierarchy, is carried out in accordance with the formula:
Ym = fm(yim+1(ci),..., yKi+im+1(cK)) (2)
Then the integral indicator (1) based on
(2), when presented by operators, look like this:
Y = F 1,..., F \..„ F m(yi(ci),..., yn(cn)) (3)
At the same time, the operator:
F m : y+1 ^ ym. (4)
The difference between the original form of representation of the integral indicator (1), which is used in the Boehm methodology and its modifications, and the proposed form of its representation in the form (3) is as follows. The formulation of the integral indicator in the form (1) limits the initial information for quality assessment to a set of estimates of elementary indicators and excludes the possibility of their multilevel composition, i.e., grouping individual elementary parameters into parameters of a higher level of generality.
In other words, excluding the possibility of entering and evaluating group indicators, the representation of the integral quality indicator in the form (1) excludes the possibility of analyzing and taking into account the meaning and assessments of all intermediate actions and expert assessments. Also, excludes the possibility of an adequate analysis of the activities of experts assessing the quality of ISP.
The formulation of the integral indicator in the form (3) is based on the usage of initial information represented by a set of estimates of elementary quality indicators. It not only does not exclude, but, on the contrary, involves a multi-level grouping of both the initial elementary and group indicators derived from them. It follows that the proposed formulation of the integral indicator in the form
(3) provides adequate accounting and analysis
of all those intermediate conclusions and assessments that are formulated by experts in the process of assessing the quality of ISP.
As a basis for the development of a mathematical apparatus for assessing the quality of ISP, authors considered that an expert, having in his mind some indefinite model of the standard of such software, is able to compare individual characteristics of ISP quality with this ideal model. In other words, he estimates the magnitude and direction of the deviation of the estimated variant of the complex from the ideal model according to all its characteristics (particular indicators):
AY = Y1 - Yo (5)
The procedure for formulating estimates of the form (5) in the direction from elementary indicators to group indicators of a higher level of hierarchy is defined in efficiency theory as a procedure for implementing the principle of "embedding" indicators "bottom-up". Thus, if the elementary quality indicators are formulated in the single scale described above (0,9) e^, then the implementation of procedures of the form (5) "bottom-up" provides a correct assessment of the quality in accordance with (3).
Conclusion
The article substantiates the expediency of using individual multi-criteria expert systems to obtain reasonable assessments of the quality of a particular ISP. They include a number of private quality indicators, characteristics of their parameters, weight coefficients and metrics for performing calculations in relation to a specific ISP and the basic requirements for its operation. In addition, in the development of the Boehm methodology, a new approach is proposed to obtain reasonable estimates of the quality of a particular ISP. It consists in determining the value of the integral quality indicator of ISP using a linear convolution of a set of private indicators with subjectively assigned and not ordered among themselves weights of their qualitative characteristics obtained as a result of expert assessments.
References
1. Gorbachenko I.M. Quality assessment of software for creating testing systems // Fundamental research. 2013. № 6-4. P. 823-827. [Electronic resource]. URL: http://fundamental-research.ru/ru/article/view?id= 31642 (access mode: 20.02.2022).
2. GOST R ISO/IEC 25010-2015. Information Technology. System and software engineering. Requirements and quality assessment of systems and software (SQuaRE). Quality models of systems and software products.
3. James R. Quality Management. M.: UNITY-DANA. 2007. 637 p.
4. Zatsarinny A.A., Ionenkov Y.S. Some methodological aspects of the choice of information systems performance indicators // High availability systems. - M.: Radio engineering. 2019. №4. P. 19-26.
5. Zatsarinny A.A., Ionenkov Y.S. Evaluation of the effectiveness of information and telecommunication systems / Ed. Doctor of Engineering A.A. Zatsarinny. - M.: Voshod-A, 2020. -120 p.
6. Kotsyuba I.Y., Chunaev A.V., Shikov A.N. Methods for assessing and measuring the characteristics of information systems. Tutorial. - St. Petersburg: Saint Petersburg State University of Information Technologies, Mechanics and Optics, 2015. - 264 p.
7. Larin S.N., Zhilyakova E.V. Methods, the composition of indicators and the algorithm for conducting an expert assessment of the quality of innovative software products // Innovative science. 2015. Release. № 4-1. P. 73-76.
8. Methods and means of measurement, testing and control: a tutorial. / V.A. Oleshchuk, A.S. Vereshchagin. - Komsomolsk-on-Amur: Federal state budgetary educational institution of higher professional education « Komsomolsk-on-Amur State University», 2015. 92 p.
9. Mitikhin V.G. On the issue of solving multicriteria problems based on the method of analysis of hierarchies // Cloud of Science. 2015. T.2. №4. [Electronic resource]. URL: http://cloudofscience.ru.
10. Nesterov A. V. Expertise: General Theory of Expertise. - M.: HSE Printing House, 2014. - 261 p.
11. Sidelnikov Y.V., Saltykov S.A. The procedure for selecting the most appropriate varieties of expert methods // Management of large systems. 2010. № 30. P. 35-66.
12. Boehm B.W., Brown J.R., Kaspar H., Lipow M., MacLeod G.J., Merritt M.J. Characteristics of Software Quality, TRW Series of Software Technology, Amsterdam, North Holland, 1978. 166 p.
13. ISO/IEC 9126-1:2001. Software engineering - Software product quality - Part 1: Quality model.
14. Fitzpatrick R. Software Quality: Definitions and Strategic Issues, Staffordshire University, School of Computing Report, 1996. 34 p.
- Экономические науки-
ИНТЕГРАЛЬНАЯ ОЦЕНКА КАЧЕСТВА ИННОВАЦИОННЫХ ПРОГРАММНЫХ
ПРОЕКТОВ: СИСТЕМА ПОКАЗАТЕЛЕЙ
С.Н. Ларин, канд. техн. наук, ведущий научный сотрудник Я.М. Ермакова, научный сотрудник
Центральный экономико-математический институт РАН (Россия, г. Москва)
Аннотация. С настоящее время идет стремительная информатизация процессов управления практически во всех отраслях производства. Для управления своей хозяйственной деятельностью многие предприятия используют современные информационные системы и технологии. Это позволяет им получать новые конкурентные преимущества. Однако, при разработке и внедрении таких информационных продуктов как инновационные программные проекты возникает достаточно много проблем. Большая их часть связана с определением состава требований к качеству инновационных программных проектов. При этом современное состояние в области оценки качества инновационных программных проектов свидетельствует о наличии целого ряда подходов к этому процессу. Но разработать универсальную совокупность частных показателей, составляющих интегральный показатель оценки качества таких проектов, не представляется возможным. Поэтому авторами статьи предложен новый подход. Он позволяет в каждом конкретном случае формировать совокупность частных показателей, адекватную области применения соответствующих программных средств в составе конкретного инновационного программного проекта. Его суть заключается в обосновании вида интегрального показателя качества на основе иерархической сети частных показателей и разработке процедуры агрегирования частных показателей в интегральный показатель качества конкретного инновационного программного проекта.
Ключевые слова: инновационные программные проекты, оценка качества, иерархическая сеть, интегральный показатель.
