CC BY
29
Multi-Criteria Choice of Antivirus Tools with Using the Ray Diagrams
Gorshkov A. V. Lokhvitskii V. A. Khomonenko A. D., Rybakova E.A.,
Emperor Alexander I Mozhaisky Military Space Academy Gorshkov V. N.,
Petersburg State Transport University St.-Petersburg, Russia Emperor Alexander I
St.-Petersburg, Russia vovan296@mail.ru Petersburg State Transport University
agorshkov23@yandex.ru St.-Petersburg, Russia khomon@mail.ru
Abstract. A multi-criteria choice of software based on ray diagrams is considered. This solves the problem of peer review and selection of antiviral drugs based on a balanced assessment of indicators of antivirals office applications, such as: performance, ergonomics and the pattern of completeness, self-defense, monitoring and others. Compared following antiviral drugs: Panda Cloud Antivirus, Kaspersky Internet Security, Norton Internet Security and Avira Free Antivirus. The representation of complex quality indicators in the form of polygons areas is considered. This significantly increases the visibility of a multidimensional comparison of antiviral tools.
Keywords: multi-criteria choice of software, antiviral drugs, ray diagrams, decision-making.
Introduction
The widespread application and continuous improvement of information technology, the availability of a large number of software products (software) on the market, and the lack of decision-makers, technical knowledge and experience to select the PP make it necessary to develop methods and means for selecting the appropriate software from a variety of analogs. This requires finding a compromise between technical features, functionality, and financial issues can be formulated as a multi-criteria decision-making problem.
In the article the model of multi-criteria selection of the best software on the example of the anti-virus tools on the user's preferences. Among the existing methods of selecting the most interesting are the methods of decision-making (pairwise comparisons Saaty, production systems with clear rules, fuzzy inference algorithms, etc.) [1-5].
Methods of software evaluation
There are the following methods for software evaluation: measurement, registration, organoleptic, settlement, expertise, sociological methods.
The measurement method is based on obtaining information using tools.
Registration method is based on obtaining information during testing and operation of the software is recorded or counted when certain events (time and number of crashes or failures, the transmission control other modules, start time, end time).
Sensory method is based on the use of information obtained from the analysis of the perception of the senses (sight, hearing) to define indicators for ease of application.
Calculation method is based on the theoretical and empirical relationships (in the early stages of development), statistical data accumulated during testing, operation and maintenance of
the software. Using the calculation method determined by the duration of the calculation, the response time, reliability, the necessary resources.
Expert method is based on determining the values of quality indicators according to experts, competent in solving this problem, on the basis of their experience and intuition. Expert method is used in cases where the problem can not be solved by any other existing method or other methods are more laborious.
Sociological methods are based on processing of special questionnaires - questionnaires.
when conducting tests to assess the software quality assurance is carried out dynamic tests on the basis of the strategy of "black box '' are the following types of tests: Stochastic testing -on a random set of test data Deterministic testing - the program on a computer using a specially selected test cases (functional, input-output). Controlled by each combination of input data and corresponding results, as well as every statement in the specification of the test program.
Testing in real time - the processing of input data with the time of their arrival, duration and priority processing, the use of resources and the dynamics of interaction with other programs.
The approaches to the choice of software
Software selection problems (PO) were considered in a relatively small number of works. In particular, see textbook [6] written by Ryzhikov Yu, offered a visual approach to the comparative evaluation of mathematical packages (MathLab, Scientific Workplace, Maple and Mathcad) for a number of indicators presented in the form of ray diagrams. This approach was developed in [7]. The report [8] on the basis of the said approach proposed method for selecting software version control systems.
The thesis Akhaev AV [9] with respect to the modules of a software system 1c accounting studies the issues of choice of software products based on ontology, fuzzy measures and choquet integral. Noted approach provides a more accurate assessment of integrated software products being compared, but rather is characterized by labor-intensive computing.
In [10] carried out a choice of the software package for simulation using fuzzy analytic hierarchy process. In [11] hierarchy analysis method is proposed to use to select one of the two ERP (Enterprise Resource Planning) systems. The article [12] on the basis of the methodology of the analysis of the operational environment is solving the problem selection software routing system. In this case, a description of software features in the rank scale.
In [13] proposed a generalization of the analytic hierarchy process for deciding when inaccurate comparisons using the Dempster-Shafer theory [14], which allows to process incomplete and inaccurate preferences. The article [15] proposes an approach to the choice of software using fuzzy inference algorithm Takagi-Sugeno the example of project management systems.
Technology ray diagrams
In many practical cases, the problem boils down to the choice of multi-criteria evaluation of individual quality indicators -without proper methodological basis and ensure system integrity. At best, the customer (the decision-maker - DMP) presented quality indicators tables for comparable options hard to take and do not give explicit preference for.
For partial aggregation rating algorithms are used additive, multiplicative or harmonic convolution maximizing the minimum and maximum efficiency, and others [1, 5]. The choice of aggregation method is difficult and decision-makers argued perceived badly. Therefore, a crucial role is played by visualization problems quantitative estimates of system properties and, in particular, aggregation of these assessments - for a final decision on choosing one of the options being compared.
one of the tools of decision-making in this paradigm is to build a radial diagrams. Radiation diagram is a set (number of properties compared objects) rays, on each of which one of the labels are deposited normalized indicators. Tags of different objects are connected by lines of different colors (structure). Educated polygons allow them with the utmost clarity to perform multidimensional mapping objects for various purposes. In particular, in [7] performed multi-criteria assessment of various build options (configure) sonar system "Vector" on the 12 parameters on the basis of ray diagrams technology [6].
In general, among the reference variants are not superior to others in all respects. Then unconsciously chosen option, limiting the large area.
The area of each figure is calculated as the sum of the areas of its constituent triangles:
M
S =1X SiSi+1sin Y i. (1)
i=1
Here, the M - number of indicators, {g_} - "Radial" sides of the triangles, i. e. partial indicators appropriate option (gM+1 = g1), and {y.} - central angles. With a symmetrical arrangement of beams all angles are equal, that allows you to compare the options by the values can be easily calculated expression
S * =XiSiSi+1.
Now let's discuss the technology with radiation diagram based on the use of formula (1).
First of all, note the specifics of the account "negative characteristics", the usefulness of which decreases with increasing index. Negative are always expensive components (negative sign of the utility) - cost, operating cost, weight, power consumption and response time, error, gyro care, the delay of work, etc. This approach is clearly excludes "areal" interpretation and thereby undermines the ideological basis of the proposed technology. To "negative" properties you can change the normalization principle: for each of them the minimum value is divided by the value
achieved in the corresponding version. There are other options for the normalization [16].
Additional option to account for the importance of performance can be realized by introducing a correction factor (index of importance of quality indicators).
Further, values of sums of (1) and S* depend on the transfer rates of the order (more precisely, by selecting pairs). This makes it possible, for each task in addition to consider the usefulness of pairwise combinations of indicators.
The effect of accounting pair combinations can be enhanced by changing the central angles - their sum 2n distribution in proportion to the amount of utility steam. Of course, in this case, a decision must be based on a complete version of the formula (1). It should be understood that minor angle change appreciably affect only sectors central squares with sharp corners.
Charting and calculation of the mentioned areas are relatively easy to automate. It is implemented in the developed one of the authors of the article [3] (Lokhvitskii VA) software Ray diagram. Perform the solution to the problem of multi-criteria selection of antiviral agents with the use of this software tool.
Comparing anti-virus software
The choice of software tools for information security [17-20] issues studied quite poorly. For example, we note [21], which provides an analysis is made of software tools for the analysis and evaluation of information security risk, which is held at a level (yes/no).
Antivirus software (antivirus) - a specialized program for the detection of computer viruses and unwanted programs in general and recovery of infected files such programs, as well as for the prevention of infecting files or operating system with malicious code.
Panda Cloud Antivirus - antivirus software with firewall features developed by Panda Security. The product was introduced in spring 2009 as a security solution with the new security model, using cloud computing. The program provides protection against viruses, Trojans, spyware, worms, adware and dialers. In November 2011 servers "Collective Intelligence" (eng. Collective Intelligence) Panda Cloud Antivirus analyzed more than 200 million files.
Kaspersky Internet Security - line of software products developed by "Kaspersky Lab" on the basis of "Kaspersky AntiVirus" for a comprehensive real-time protection for home PCs from known and new threats.
Norton Internet Security - security suite developed by Symantec. It includes antivirus, firewall, email scanner, spam filter, anti-phishing protection. Share Norton Internet Security accounted for 61 % of the market of similar software in the United States in 2007.
Avira Free Antivirus - antivirus, free for personal use. The product includes a resident monitor (which checks the processes when you try to access the files), the scanner and the program automatic or manual updates. Beginning with the ninth version has a function of detecting adware, spyware and other malware.
Generalized indicators table antivirus
In the table 1 summarizes the indicators of anti-virus and the average marks from experts.
Table 2
Normalized to 1 indicators and rating assessment complex indices of antiviruses
Table 1
Summarizes the indicators of anti-virus
Indicator Panda Cloud Antivirus Kaspersky Internet Security Norton Internet Security Avira Free Antivirus
Treatment of active infection 7,5 8,7 6,2 7,4
Speed (scanning) 1,4 5,4 7,3 3,5
Ergonomics 3,4 6,8 5,5 7,2
Real-time protection (monitoring) 4,6 8,5 7,9 8,7
Impact on work with office software 6,9 4,7 5,3 7,1
Self-defense 6,2 9,5 9,8 6,7
Virus database 3,5 7,9 5,7 8,3
False positives 4,8 5,0 6,1 4,3
Indicator Panda Cloud Antivirus Kaspersky Internet Security Norton Internet Security Avira Free Antivirus
Treatment of active infection 0,86 1,0 0,71 0,85
Speed (scanning) 0,19 0,74 1,0 0,48
Ergonomics 0,47 0,94 0,76 1,0
Real-time protection (monitoring) 0,53 0,98 0,91 1,0
Impact on work with office software 0,68 1,0 0,89 0,66
Self-defense 0,63 0,97 1,0 0,68
Virus database 0,42 0,95 0,69 1,0
False positives 0,9 0,86 0,7 1,0
S 0,9593 2,4426 1,9570 1,9573
Rating 4 1 3 2
Perform the normalization of particular indicators as previously described. The weight of the indicators will take equal to 1. Then, we calculate complex indicators of quality for each antivirus on the basis of (1) and determine ratings according to the results of calculations (table 2). As seen from table 2 the most best rating 1 has Kaspersky Internet Security.
Ray diagram built on the basis of the obtained results is shown in Fig. 1. Legends: green dotted bar - Kaspersky Internet
Security; red line - Panda Cloud Antivirus; blue hatch line - Norton Internet Security; purple spot line - Avira Free Antivirus.
Perform the calculation from the weights of partial indices W={1,5; 3,5; 2,5; 3,0; 1,0; 1,0; 2,5; 2,0}. In this case, you must
7
3
Fig. 1. Ray diagram of antivirus estimation without accounting weights of partial indicators
multiply the i-th normalized index for the corresponding ratio, calculated by the formula: ri = wi /max(^),i = 1,8.
Then we calculate composite indicators for each antivirus on the basis of (1) and determine ratings according to the results of calculations (table 3).
Table 3
Normalized indices based on weighting and rating-ratings complex indices of antiviruses
Ray diagram built on the basis of the obtained results from the weights of partial indices showed in Fig. 2. Legends: green dotted bar - Kaspersky Internet Security; red line - Panda Cloud
Antivirus; blue hatch line - Norton Internet Security; purple spot line - Avira Free Antivirus.
As you can see from the result data of table 3 (lower 2 lines), compared with the previous version (table 2) have changed the values of S, with the General conclusion about the ranking of compare antivirus tools remained unchanged. In the General case this may not be so.
Based on these results, we can conclude that if the advantage of Kaspersky Internet Security to Panda Cloud Antivirus is clearly traced (the value of the complex index of S first about 2,5-3 times greater than the second), the choice between Norton Internet Security and Avira Free Antivirus does not so obvious.
Here some private indicators are compensated by others, and keeping the weight coefficients further complicates a final decision on choosing one of the options being compared.
Conclusion
Representation of complex quality indicators in the form of polygons areas significantly increases the visibility of a multidimensional mapping compared antiviral tools (generally software), especially taking into account the weight coefficients. In this case, approximately equal areas can be selected with the figure of a "correct" form, for which the values of the partial indicators are better balanced.
Multi-criteria selection of software classes of antiviral agents on the user's preferences on the basis of the radial diagrams, in our opinion:
1) compared with the method of analysis of hierarchies Saaty and generalization based on Dempster-Shafer theory [13, 14] can reduce the complexity of the calculations;
Indicator Panda Cloud Antivirus Kaspersky Internet Security Norton Internet Security Avira Free Antivirus
Treatment of active infection 0,37 0,43 0,31 0,36
Speed (scanning) 0,19 0,74 1,0 0,47
Ergonomics 0,34 0,67 0,55 0,71
Real-time protection (monitoring) 0,45 0,84 0,78 0,86
Impact on work with office software 0,19 0,29 0,25 0,19
Self-defense 0,18 0,28 0,29 0,2
Virus database 0,3 0,68 0,49 0,71
False positives 0,51 0,49 0,4 0,57
S 0,65 2,08 1,76 1,8
Rating 4 1 3 2
7
3
Fig. 2. Ray diagram of antivirus estimation with accounting weight of the partial indicators
2) compared with the fuzzy inference algorithm using Takagi-Sugeno proposed in [15], provides greater visibility and reduced the complexity of the calculations;
3) compared with the approach proposed in [9] on the basis of ontology, fuzzy measures and Choquet Integral, achieved greater visibility resulting output.
References
1. Mikoni S. V. Mnogokriterial'nyi vybor na konechnom mnozhestve al'ternativ [Multi-criteria selection on a finite set of alternatives], St. Petersburg, Lan', 2009, 270 p.
2. Saaty Th. L. The Hierarchon: A Dictionary of Hierarchies. Pittsburgh, Pennsylvania, RWS Publ., 1992, 496 p.
3. Takagi T., Sugeno M. Fuzzy Identification of Systems and Its Applications to Modeling and Control, IEEE Trans. Syst. Man Cybern., 1985, vol. 15, no. 1, pp. 116-132.
4. Ma J., Lu J., Zhang GQ. Decider: A fuzzy multi-criteria group decision support system, Int. J. Uncertainty Fuzziness Knowledge Based Syst., 2010, no. 23, pp. 23-31.
5. Podinovskii V. V., Potapov M.A., Neliubin A. P., Podi-novskii o. V. The theory of criteria importance: locating the status and directions of further development [Teoriia vazhnosti kriteriev: sovremennoe sostoianie i napravleniia dal'neishego razvitiia], Trudy XII Vserossiiskogo soveshchaniiapo problemam upravleniia VSPU-14 [Proc. XII all-Russian conf. on control problems VCPU-14], Moscow, 2014, pp. 7697-7702.
6. Briakalov G.A., Voitsekhovskii S.V, Vorob'ev E.G, Gofman V.E, Gridin V. V., Driukov Iu. P., Zamula A. A., Zakharov A. I., Kompaniets R. I., Lipetskikh A. G., Ryzhik-ov Iu. I., Khomonenko A. D., Tsygankov V. M. Osnovy sovremen-nykh komp'iuternykh tekhnologii [The foundations of modern computer technology], the textbook, ed. A. D. Khomonenko. St. Petersburg, KORONA print, 2005, 672 p.
7. Ryzhikov Y. I., Alekseev A. V., Lokhvitskii V.A. Navy problem and ray diagram [Flotskie problemy i luchevye dia-grammy], Sbornik trudov tret'ei mezhdunarodnoi nauchno-prak-ticheskoi konferentsii "Imitatsionnoe i kompleksnoe modeliro-vanie morskoi tekhniki i morskikh transportnykh system " "IKM MTMTS2015", St. Petersburg, 2015, pp. 121-125.
8. Gorshkov A. V., Khomonenko A. D. Method for choice software of versions control system [Metod vybora pro-gram-mnogo obespecheniia sistem upravleniia versiiami. Analiz i prog-nozirovanie sistem upravleniia v pro-myshlennosti i na transporte], Trudy XVI mezhdunarodnoi konferentsii molodykh uche-nykh, studentov i aspirantov, 21-23 Apr. 2015, St. Petersburg, St. Petersburg, FGBOU VPO PGUPS, 2016, pp. 125-132.
9. Akhaev A. V. Metodika, modeli i algoritmy vybora pro-grammnykh produktov na osnove ontologii i nechetkoi mery [Method, models and algorithms for software choice using ontologies and fuzzy measure], Tomsk, 2014, 19 p.
10. Azadeh A., Shirkouhi S. N., Rezaie K. A robust decision-making methodology for evaluation and selection of simulation software package, Int. J. Adv. Manuf. Technol., 2010, Vol. 47, pp. 381-393.
11. Karaarslan N., Gundogar E. An application for modular capability-based ERP software selection using AHP method, Int. J. Adv. Manuf. Technol., 2009, Vol. 42, pp. 1025-1033.
12. Smirlis Y. G., Zeimpekis V., Kaimakamis G. Data envelopment analysis models to support the selection of vehicle routing software for city logistics operations, Oper. Res., 2012, Vol. 12, pp. 399-420.
13. Utkin L. V., Simanova N. V. A generalization of the analytic hierarchy process for decision with inaccurate comparisons using the theory of Dempster - Shafer [Obobshchenie metoda analiza ierarkhii dlia priniatiia resheniia pri netochnykh sravne-niiakh s ispol'zovaniem teorii Dempstera - Sheifera], Izvestiia OrelGTU. Seriia "Informatsionnye sistemy i tekhnologii " [Iz-vestiya OrelGTU. Series "Information systems and technologies"], 2007, no. 4/268 (535), pp. 223-227.
14. Dempster A. P. Upper and Lower Probabilities Induced by a Multi-Valued Mapping, Ann. Math. Stat., 1967, no. 38, pp. 325-339.
15. Titov A. I., Khomonenko A. D. Choice of software using algorithm Takagi-Sugeno on example of projects management system [Vybor programmnogo obespecheniia s pomoshch'iu algoritma Takagi-Sugeno na primere sistem upravleniia proek-tami], Nauchno-tekhnicheskie vedomosti Sankt-Peterburgskogo gosudar-stvennogopolitekhnicheskogo universiteta. Informatika. Telekommunikatsii. Upravlenie. [St. Petersburg State Polytechnical Univ. J. Computer Sci. Telecommunication and Control Systems], 2016, no. 1 (236), pp. 41-52.
16. Khovanov N. V. Analiz i sintez pokazatelei pri in-format-sionnom defitsite [Analysis and synthesis of parameters under information deficiency], St. Petersburg, Publ. house St. Petersburg Univ., 1996, 196 p.
17. Kornienko A. A., Eremeev M. A., Kustov V. N., Glu-kharev M. L., Iakovlev V. V., Diasamidze S. V., Borodulin M. E. Informatsionnaia bezopasnost' i zashchita informatsii na zheleznodorozhnom transporte. Chast ' 2. Programmno-appa-ratnye sredstva obespecheniia informatsionnoi bezopasnosti na zheleznodorozhnom transporte. Uchebnoe posobie. [Information security and information security on railway transport. P. 2. Hardware-software means of information security on railway transport. Textbook. Ser. Higher professional education], Moscow, Educational-methodical center on education on railway transport, 2014, 448 p.
18. Gindin S. I., Khomonenko A. D., Iakovlev V. V., Mat-veev S. V. Model evaluation of efficiency distributed data processing including the spending of ensuring the information security [Model' otsenivaniia operativnosti raspredelennoi obrabotki dannykh s uchetom zatrat na obespechenie informat-sionnoi bezopasnosti], Problemy informatsionnoi bezopasnosti. Komp'iuternye sistemy [Information Security Problems. Computer Systems], 2013, no. 4, pp. 59-67.
19. Kornienko A. A., Polyanichko M. A. Methodology of Conflict Detection and Resolution in Cyber Attacks Protection Software on Railway Transport, Intellectual Technologies on Transport, 2015, no. 1, pp. 18-21.
20. Markov A. S. Model estimation and planning of testing of software in the information security requirements [Modeli otsen-ki i planirovaniia ispytanii programmnykh sredstv po trebovani-iam bezopasnosti informatsii], Vestnik Moskovskogo gosudarst-vennogo tekhnicheskogo universiteta im. N. E. Baumana. Seriia: Priborostroenie [Bulletin of Moscow state technical Univ. N. Uh. Bauman. Series: Instrument Eng.], 2011, no. SPEC, pp. 90-103.
21. Baranova E. K. Chernova M. V. Comparative analysis of software tools for the analysis and evaluation of information security risk [Sravnitel'nyi analiz programmnogo instrumentariia dlia analiza i otsenki riskov informatsionnoi bezopasnosti], Problemy informatsionnoi bezopasnosti. Komp'iuternye sistemy [Information Security Problems. Computer Systems], 2014, no. 4, pp. 160-168.
Многокритериальный выбор антивирусных средств с помощью лучевых диаграмм
Горшков А. В. Петербургский государственный университет путей сообщения Императора Александра I Санкт-Петербург, Россия agorshkov23@yandex.ru
Лохвицкий В. А. Военно-космическая академия имени А. Ф. Можайского Санкт-Петербург, Россия vovan296@mail.ru
Хомоненко А. Д., Рыбакова Е. А., Горшков В. Н.
Петербургский государственный университет путей сообщения Императора Александра I Санкт-Петербург, Россия khomon@mail.ru
Аннотация. Рассматривается многокритериальный выбор программных продуктов на основе лучевых диаграмм. При этом решается задача экспертной оценки и выбора антивирусных средств на основе взвешенной оценки таких показателей антивирусных средств офисных приложений, как быстродействие, эргономичность, полнота вирусной базы, самозащита, мониторинг и др. Выполнено сравнение следующих антивирусных средств: Panda Cloud Antivirus, Kaspersky Internet Security, Norton Internet Security и Avira Free Antivirus. Рассмотрено представление комплексных показателей качества в виде многоугольников областей, что заметно увеличивает наглядность многоаспектного сопоставления сравниваемых антивирусных средств.
Ключевые слова: многокритериальный выбор программных средств, антивирусные средства, лучевые диаграммы, принятие решений.
Литература
1. Микони С. В. Многокритериальный выбор на конечном множестве альтернатив / С. В. Микони. - СПб. : Лань, 2009. - 270 с.
2. Saaty Th. L. The Hierarchon : A Dictionary of Hierarchies / Th. L. Saaty. - Pittsburgh, Pennsylvania : RWS Publ., 1992. - 496 p.
3. Takagi T. Fuzzy Identification of Systems and Its Applications to Modeling and Control / T. Takagi, M. Sugeno // IEEE Trans. Syst. Man Cybern. - 1985. - Vol. 15, no. 1. - P. 116-132.
4. Ma J. Decider : A fuzzy multi-criteria group decision support system / J. Ma, J. Lu, GQ. Zhang // Int. J. Uncertainty Fuzziness Knowledge Based Syst. - 2010. - N 23. - P. 23-31.
5. Подиновский В. В. Теория важности критериев : современное состояние и направления дальнейшего развития / В. В. Подиновский, М. А. Потапов, А. П. Нелюбин, О. В. Подиновский // Тр. XII Всерос. совещания по проблемам управления ВСПУ-14. - М., 2014. - С. 7697-7702.
6. Брякалов Г. А. Основы современных компьютерных технологий : учеб. / Г. А. Брякалов, С. В. Войцеховский, Е. Г. Воробьев и др. ; под ред. А. Д. Хомоненко. - СПб. : КОРОНА принт, 2005. - 672 с.
7. Рыжиков Ю. И. Флотские проблемы и лучевые диаграммы / Ю. И. Рыжиков, А. В. Алексеев, В. А. Лохвицкий // Сб. тр. III междунар. науч.-практич. конф. «Имитационное и комплексное моделирование морской техники и морских
транспортных систем» - «ИКМ МТМТС 2015». - СПб., 2015. -С. 121-125.
8. Горшков А. В. Метод выбора программного обеспечения систем управления версиями / А. В. Горшков, А. Д. Хо-моненко // Анализ и прогнозирование систем управления в промышленности и на транспорте : тр. XVI междунар. конф. молодых ученых, студентов и аспирантов, СПб., 21-23 апр. 2015 г. - СПб. : ФГБОУ ВПО ПГУПС, 2016. - С. 125-132.
9. Ахаев А. В. Методика, модели и алгоритмы выбора программных продуктов на основе онтологии и нечеткой меры : автореф. ... канд. техн. наук / А. В. Ахаев. - Томск, 2014. - 19 с.
10. Azadeh A. A robust decision-making methodology for evaluation and selection of simulation software package / A. Aza-deh, S. N. Shirkouhi, K. Rezaie // Int. J. Adv. Manuf. Technol. -2010. - Vol. 47. - P. 381-393.
11. Karaarslan N. An application for modular capability-based ERP software selection using AHP method / N. Karaarslan, E. Gundogar // Int. J. Adv. Manuf. Technol. - 2009. - Vol. 42. -P. 1025-1033.
12. Smirlis Y. G. Data envelopment analysis models to support the selection of vehicle routing software for city logis-tics operations / Y. G. Smirlis, V. Zeimpekis, G. Kaimakamis // Oper. Res. - 2012. - Vol. 12. - P. 399-420.
13. Уткин Л. В. Обобщение метода анализа иерархий для принятия решения при неточных сравнениях с использованием теории Демпстера - Шейфера / Л. В. Уткин, Н. В. Си-манова // Изв. ОрелГТУ. Серия «Информационные системы и технологии». - 2007. - № 4 / 268 (535). - С. 223-227.
14. Dempster A. P. Upper and Lower Probabilities Induced by a Multi-Valued Mapping / A. P. Dempster // Ann. Math. Stat. -1967. - No. 38. - P. 325-339.
15. Титов А. И. Выбор программного обеспечения с помощью алгоритма Такаги - Сугено на примере систем управления проектами / А. И. Титов, А. Д. Хомоненко // Науч.-техн. ведомости Санкт-Петербург. гос. политех. ун-та. Информатика. Телекоммуникации. Управление. - 2016. - № 1 (236). -С. 41-52.
16. Хованов Н. В. Анализ и синтез показателей при информационном дефиците / Н. В. Хованов. - СПб. : Изд-во Санкт-Петербург. ун-та, 1996. - 196 с.
17. Корниенко А. А. Информационная безопасность и защита информации на железнодорожном транспорте. Ч. 2. Программно-аппаратные средства обеспечения информа-
ционной безопасности на железнодорожном транспорте : учеб. пособие / А. А. Корниенко, М. А. Еремеев, В. Н. Кустов и др. - М. : УМЦ по образованию на ж.-д. транспорте, 2014. -448 с. (Сер. Высшее профессиональное образование.)
18. Гиндин С. И. Модель оценивания оперативности распределенной обработки данных с учетом затрат на обеспечение информационной безопасности / С. И. Гиндин, А. Д. Хомоненко, В. В. Яковлев, С. В. Матвеев // Проблемы информационной безопасности. Компьютерные системы. -2013. - № 4. - С. 59-67.
19. Kornienko A. A. Methodology of Conflict Detection and Resolution in Cyber Attacks Protection Software on Railway
Transport / A. A. Kornienko, M. A. Polyanichko // Интеллектуальные технологии на транспорте. - 2015. - № 1. -C. 18-21.
20. Марков А. С. Модели оценки и планирования испытаний программных средств по требованиям безопасности информации // Вестн. Москов. гос. тех. ун-та им. Н. Э. Баумана. - 2011. - SPEC. - С. 90-103. (Сер. Приборостроение.)
21. Баранова Е. К. Сравнительный анализ программного инструментария для анализа и оценки рисков информационной безопасности / Е. К. Баранова, М. В. Чернова // Проблемы информационной безопасности. Компьютерные системы. - 2014. - № 4. - С. 160-168.
