CC BY
25
AN INTEGRATED APPROACH TO THE CHOICE OF METHODS OF EVALUATION OF IT PROJECTS
УДК 004.942 Ivan A. Gladkov,
candidate of technical Sciences, associate Professor, Department of information systems and technologies Belgorod University of cooperation, Economics and law E-mail: [email protected]
Viktoria S. Nehotina,
candidate of economic Sciences, associate Professor, Department of information systems and technologies Belgorod University of cooperation, Economics and law E-mail: [email protected]
An integrated approach to the choice of methods of evaluation of it projects The integrated approach of a choice of methods for estimation of IT projects taking into account a set of methods, parameters and criterion of their choice depending on specific conditions of implementation of projects is presented in the article.
Keywords: multi-criteria approach, IT project, estimation of IT projects, methods, algorithm of a choice of methods.
Иван Александрович Гладков
к.т.н., доцент кафедры информационных систем и технологий Белгородского университета кооперации, экономики и права
Виктория Сергеевна Нехотина,
к.э.н., доцент кафедры информационных систем и технологий Белгородского университета кооперации, экономики и права
Эл. почта: [email protected]
КОМПЛЕКСНЫЙ ПОДХОД К ВЫБОРУ МЕТОДОВ ОЦЕНИBАНИЯ ИТ-ПРОЕКТОВ
В статье представлен комплексный подход выбора методов для проведения оценивания ИТ-проектов с учетом набора методов, параметров и критерия их выбора в зависимости от конкретных условий реализации проектов.
Ключевые слова: ИТ-проект, оценивание ИТ-проектов, методы оценивания, алгоритм выбора методов.
1. Introduction
The purpose of the present publication consists in research of modern approaches to estimation of projects in the sphere of information technologies (IT projects), identification of the problems accompanying this process and development of the approach considering a complex of the factors having impact on decision-making at the choice of methods for estimation.
Relevance of the solution of the considered task is caused by need of implementation of calculations not only at a stage of introduction of the project, but also at the subsequent stages when updating of the made decisions. Carrying out estimation isn't possible without development of the complex approach based on application of methods of modeling and modern tools.
Now methods of estimations are widely in use and allow providing reliability of the received results in a varying degree. However in the conditions of uncertainty for semi structured tasks in the presence of several alternatives IT projects it is expedient to carry out a stage-by-stage assessment with use of several methods. This approach allows increasing on the one hand reliability of the obtained data, and with another - to minimize expenses of resources for estimation due to stage-by-stage control of level of satisfaction of the person making the decision (PMD) with results of estimation.
The existing methods of estimation of projects [1-16] allow executing in a varying degree calculation of the main indicators forming a basis for support of adoption of reasonable decisions at introduction of IT projects, however traditional approaches to their choice are based on intuitive estimates of experts. Therefore the integrated approach of estimation of IT projects at decision-making at a choice of methods for carrying out estimation due to reasonable selection of the least difficult methods when ensuring the demanded level of accuracy of results is presented in article.
The offered approach is based on application of various methods (fuzzy logic, cognitive modeling, imitating modeling, the theory of games, the portfolio theory, etc.) and algorithm at their choice depending on specific initial conditions of implementation of projects taking into account criterion of decision-making.
2. Determination of parameters of selection of methods of estimation
Depending on initial conditions carrying out estimation of IT projects can be carried out with use of a certain quantity of economic-mathematical methods (Mn), thus some methods from the presented list can belong to one group of methods (Mim).
In similar conditions (when uncertainty concerning an estimation method choice takes place) there is a problem of a choice of an optimum combination of methods for carrying out calculations.
At the solution of this task it is necessary to consider parameters (entry conditions) of implementation of the project, for example:
1. Purpose of implementation of the project. At implementation of the project not quantitative, but qualitative characteristics can act as the priority (for example, as a result of introduction of the project as a main objective improvement of quality of products, reduction of time for information processing when servicing, etc.).
2. Number of alternative projects. Some methods aren't effective at an assessment of projects with a big set of alternatives; others allow considering unlimited number of options of implementation of the project.
3. Type of the project. For each type of the project it is possible to pick up the most suitable method of estimation. For example, for projects with sufficient basic data it is expedient to use the mathematical methods based on methods of the system analysis and portfolio theory, and for projects data about which are practically absent - use of the methods based on expert judgments, etc.
4. Estimation stage. In some situations at various stages of estimation (initial, specifying or finishing) it is expedient to use various approaches.
5. Quantity of factors (criteria, indicators) which are used when carrying out estimation. Sometimes there is no opportunity and/or need to use when carrying out estimation of a large number of factors (indicators).
6. Preferences of the customer when carrying out estimation.
7. Accuracy of the obtained data.
8. A stage of implementation of the project (before introduction, after introduction, at a stage of refusal of implementation of the project).
The number of parameters (Pm) and their structure (list) is defined by the person making the decision (decision-maker) or a group of persons proceeding from specific conditions of implementation of the project.
3. General scheme of the estimation procedure
For the solution of an objective it is offered to use the following scheme:
1. Using expert judgments to select methods which can be used when carrying out estimation of the project (or to carry out an assessment, using all known methods).
2. To define the list of parameters in which the decision concerning a choice of methods of estimation will be made.
3. To make a matrix of a ratio of parameters and methods of estimation (fig. 1). At installation of parameters it is expedient to use a method of the analysis of hierarchies proceeding from the established scale of measurement of the importance of parameter for each method.
4. If there are some methods, they need to be ranged. Ranging of methods can be executed proceeding from:
1) quantities of factors (indicators) which are available for estimation;
2) labor inputs of calculations;
3) preferences of experts;
4) justifications on the basis of mathematical apparatus.
5. Proceeding from a matrix to define an optimum set of methods for carrying out estimation.
6. To execute an assessment, using the algorithm shown in figure 2.
Such approach will allow to structure substantially process of carrying out estimation of IT projects and to increase validity of the made decisions with the set accuracy.
4. Ranging of methods for carrying out estimation
As it was noted earlier, basic data at making decision on methods of estimation of IT projects is the matrix of a ratio of parameters and methods of estimation (fig. 1). Proceeding from specific conditions of implementation of the project, on the basis of opinions of experts the matrix is formed. Thus, use of the method of the analysis of hierarchies (MAH), in this case, is represented reasonable as by drawing up a matrix opinions of the group of experts have to be considered. Such approach assumes existence of coherence of opinions of experts.
The solution of the task connected with ranging of methods of estimation of projects can be presented in the form of several subtasks:
1. To determine a subjective scale of measurement of methods by quality indicators.
2. To correlate qualitative gradation of a subjective scale to certain numerical values.
3. To determine "weight" of criteria concerning the purpose of implementation of the project.
4. To make hierarchical "weighing" of methods.
First of all, it is necessary to construct matrixes of pair comparisons for
Method ^ Mi M2 Мз M4 Mn
Argument j
Pi aii ai2 ai3 ai4 ain
P2
Рз
Pm
the chosen indicators about quantitative judgments. Then, to carry out normalization of the received matrixes of pair comparisons. It is for this purpose expedient to use the way based on summation of elements of every line and normalization by division of each sum into the sum of all elements.
For carrying out pair comparisons it is expedient to use the nine-mark scale of the relations offered in [17] Saati's work, and for systematization given -to apply a matrix of pair comparisons at which construction by the most important coordination of symmetric elements a,j and aji is about results of pair comparisons: if x, > xp, i.e. x, the object is more preferable than xj, for the corresponding symmetric elements of a matrix the condition a,j > ap, if objects are equivalent has to be satisfied a,j = aji.
According to MAH methodology, quantitative judgments about pair comparisons of objects by the expert in the set scale of the relations, are represented by a square matrix of pair comparisons: A = (aj), (i,j = 1, 2, ..., n).
The coordination of symmetric elements and additional calibration restrictions which are unambiguously connecting in pairs symmetric elements are set by the following rules a,j and aji.
1. If aij = a, than aji = 1 / a, a ^ 0, a e {1, 2, ..., 9}.
2. If judgments are that object Oi (alternative, criterion) on degree of "importance", from the point of view of higher top, same, as object Oj; aij = aji = 1, in particular ay = 1, for all i.
For definition of own vector with some set accuracy £ use the following iterative procedure:
Wk =
Ake eTAke'
(1)
Fig. 1. Matrix of a ratio of parameters and methods estimations of IT projects
This procedure is carried out before achievement of accuracy:
eT\Wk+l - Wk < (2)
For check of coherence of the corresponding matrix of pair comparisons it is necessary to calculate an indicator of the maximum own value which is determined by a formula:
Amax = eTAWk+\. (3)
At the following stage it is expedient to check the validity of the received
results, for this purpose to estimate coherence of each matrix of pair comparisons and in general coherence of all hierarchy.
We will understand not simply traditional requirement of transitivity, but also the requirement of numerical (cardinal) coherence as coherence. As indicators of "proximity to coherence" of a matrix of pair comparisons the coherence index (CI) and the coherence relation (CR) serve.
The index of coherence is determined by a formula:
CI = (¿max- n) / (n - 1). (4)
The coherence relation for the considered option of a matrix of the fifth order we will calculate as IS relation to mathematical expectation of the casual E(SI) index:
SR = IS / E(SI). (5)
Further it is necessary to check coherence of pair comparisons of objects concerning all indicators, to define the maximum own values, coefficients of IS and OS, and also own vectors answering to the found maximum own values of the presented matrix of pair comparisons. On the basis of numerical values of components of the received own vector it is possible to establish priority of indicators of rather leading purpose.
For the purpose of check of coherence of all hierarchy we will enter the index of coherence of hierarchy (ICH) and the relation of coherence of hierarchy (RCH). The hierarchy admits coordinated, if to the RCH < 0,10.
ICH pays off on a formula:
ICH = CIu + WT • CI1 + + WT • CI2 + ... + WTh-1 • CIh-i. (6)
where
Wi - a vector of priorities of elements of level i concerning the greatest element u;
CIu - IS of a matrix of pair comparisons of elements of the I-level concerning the greatest element (u);
CIh - a vector which components are IS of matrixes of pair comparisons of the i-go elements of level concerning the level i-go elements;
T - designation of operation of transposing of vectors.
The ratio of the consistency of the hierarchy is calculated by the formula:
RCH = ICH / ISIS, (7)
where
ISIS - an index of coherence of hierarchy at casual filling of matrixes of pair coordination.
Calculation of ISIS is carried out on a formula:
ISIS = E(SIu) + W1T • E(SI1) + (8)
+ W2T • E(SIi) + ... + + WTh-1 • E(SIh-1)
where
Wi - a vector of priorities of elements of level "I" concerning the greatest element "u";
E(SIu) - a population mean of SI of a matrix of pair comparisons of elements of the first level concerning the greatest element (u);
E(SIh) - a vector which components are population means of SI of matrixes of pair comparisons of the i-go ele-
Fig. 2. Algorithm of a choice of a method of estimation of IT projects (in a general view)
Fig. 3. Block diagram of SSDM "Effectys"
ments of level concerning the level i-go elements;
T - designation of operation of transposing of vectors.
5. Algorithm of a choice of methods
The sequence of operations on a choice of a method of estimation of IT projects taking into account criterion of decision-making (CDM) sufficiency of the received results has to present algorithm of a choice of methods of estimation in a general view.
In figure 2 the algorithm in a general view which is reduced to the general procedure consisting of consecutive carrying out calculations for each of the chosen methods and decision-making about further carrying out calculations proceeding from the analysis of "criterion of satisfaction" with the obtained data is shown. The question of definition of criteria represents the separate task demanding development of the corresponding approach to its decision.
Use of the developed algorithm will help to carry out rationally estimations of IT projects, allowing reasoning the made decisions.
6. Conclusion
Carrying out estimation with use of integrated approach demands existence of modern automation equipment for carrying out calculations. Now, the set of the systems of support of decision-making (SSDM) are developed and theoretical bases of construction and which practical results of introduction are successfully put into practice, are reflected in works of native and foreign scientists, such as: Dorozh-ny P.M., Litvak B.G., Petrovsky A.B., Rogozin O.V., Telnov Yu.F., S. Alter, M. Ginzberg, R.L. Kean., G. Kopland, D. Pauer, H. Rayf, E. Winston, D. Edwards and U. Edwards.
In works of the specified authors separate aspects problems of a choice of the optimum IT project for realization without the analysis of problem situations in which section search
of methods for estimation with the smallest expenses of resources runs are considered. Therefore the theoretical provisions presented in article are realized at development of the system of support of decision-making (SSDM) of "Effectys" which block diagram is submitted in figure 3.
The specified SSDM is realized in the form of the integrated automated system focused on the solution of tasks in semistructured conditions. The system is presented in the form of a complex of the hardware-software means allowing the decision-maker to solve the problems of intellectual character demanding semantic processing of large volumes of information which is stored in a database at estimation of efficiency of IT projects. Operability of SSDM is checked at experimental modeling. Now completion of SSDM for the purpose of its transformation into the operating model with adaptation is made for various initial conditions. Results of operation of SSDM testify to efficiency of the offered estimation methods.
_
References
1. A. Tversky and D. Kahneman. Advances in prospect theory: Cumulative representation of uncertainty. // Journal of Risk and Uncertainty, 5. -1992, pp. 297-323.
2. B. Roy. The outranking approach and the foundations of ELECTRE methods. // Theory and Decision. - 31 (1). - 1991, pp. 49-73.
3. C.A. Bana e Costa and J.C. Vansnick. The MACBETH approach: Basic ideas. In Proceedings of the International Conference on Methods and Applications of Multicriteria Decision Making. - FUCAM, Faculty Universitaires Catholiques de Mons, 1997, pp. 86-88.
4. D. Bouyssou and P. Perny. Ranking methods for valued preference relations - A characterization of a method based on leaving and entering flows. European Journal of Operational Research, 61(1-2). - 1992, pp. 186-194.
5. E. Hinloopen and P. Nijkamp. Regime-methods voor ordinal multic-riteria-analyses. Kwantitatieve Methoden. - 7(22). - 1986, pp. 61-78.
6. H. Pastijn and J. Leysen. Constructing an outranking relation with
ORESTE. // Mathematical and Computer Modelling. - №12 (10/11). - 1989, pp. 1255-1268.
7. H. Voogd. Multicriteria Evaluation for Urban and Regional Planning. PionLtd., London. - 1983. - 376 p.
8. J.C. Vansnick. On the problem of weighs in multiple criteria decision making (the noncompensatory approach). // European Journal of Operational Research. - №24. - 1986, pp. 288-294.
9. J.H.P. Paelinck. Qualitative multiple criteria analysis, environmental protection and multiregional development. // Papers of the Regional Science Association. - №36. - 1976, pp. 59-74.
10. L.R. Gardiner and R.E. Steuer. Unified interactive multiple-objective programming - An open-architecture for accommodating new procedures. // Journal of the Operational Research Society. - №45. - 1994, pp. 1456-1466.
11. M. Grabisch, T. Murofushi, M. Sugeno (eds.). Fuzzy Measures and Integrals. Theory and Applications. // Studies in Fuzziness and Soft Computing. - vol. 40. - Physica- Verlag, Heidelberg. - 2000, pp. 3-41.
12. M.Roubens. Ordinal multiattributes sorting and ordering in the
presence of interacting points of view. In D. Bouyssou, E. Jacquet-Lagreze, P. Perny, R. Slowinski, Vanderpooten D. and P. Vincke, editors. Aiding Decisions with Multiple Criteria: Essays in Honour of Bernard Roy. - Kluwer Academic Publishers, Dordrecht. -2001, pp. 229-246.
13. R.L. Keeney, H. Raiffa. Decision with Multiple Objectives: Preference and Value Tradeoffs. - Cambridge University Press, New York. - 1993. -569 p.
14. W.S.M. De Keyser and P.H.M. Peeters. ARGUS - A new multiple criteria method based on the general idea of outranking. - In M. Paruccini, editor, Applying Multiple Criteria Aid for Decision to Environmental Management. Kluwer Academic Publishers, Dordrecht. - 1994, pp. 263-278.
15. Y. Siskos and D. Yannacopou-los. UTASTAR: An ordinal regression method for building additive value functions. // Investigacao Operacion-al. - №5(1) . - 1985, pp. 39-53.
16. Saati T.L. Decision-making -the Method of the Analysis of Hierarchies. - M. : Radio and Communication, 1993. - 278th page
