CC BY
130
УДК 004.891.2
DOI: 10.25559/SITITO.14.201802.397-407
MODELING AND ANALYSIS OF FEATURES OF TEAM PLAY STRATEGIES IN ESPORTS APPLICATIONS
Alexander M. Kadan, Le Li, Tsiango Chen
Yanka Kupala State University of Grodno, Grodno, Belarus
МОДЕЛИРОВАНИЕ И АНАЛИЗ ОСОБЕННОСТЕЙ ПРИМЕНЕНИЯ КОМАНДНЫХ СТРАТЕГИЙ В ПРИЛОЖЕНИЯХ КИБЕРСПОРТА
А.М. Кадан, Лэ Ли, Цзяньго Чень
Гродненский государственный университет им. Янки Купалы, г. Гродно, Беларусь
Kadan A.M., Li Le, Chen Tsiango, 2018
Keywords
Cybersport; eSports; eSports game; first-person shooter game; Counter Strike;
Counter-Strike: Global offensive; CS: GO; strategy games; team play; team play strategy; data analysis; machine learning.
Ключевые слова
Киберспорт; eSports; командный шутер от первого лица;
Counter-Strike: Global offensive; CS:GO; стратегия игры; командная игра; командная стратегия; анализ данных; машинное обучение.
Abstract
The perfect combination of sports and information technologies, implemented in video and computer games, has made eSports unusually popular, rapidly developing and meeting the challenges of the modern world. The scale of the competitions and the amount of financial investments in eSports have increased rapidly. In this regard, special attention is given to activities related to the preparation of sports teams for competitions. Of particular interest for coaches is the study of the best competitive practices, strategies and tactics of both teams of rivals and their individual players.
The article proposes and confirms the hypothesis that the methods of training eSports sportsmen based on mathematical approaches to data analysis are much more substantive and justified, which formally confirm the advantages of various team tactics and allow drawing conclusions about the prospects for using various methods of conducting the game.
As the main source of data for analyzing the strategies used by the teams, the demo files of the saved games of the Counter Strike tournaments were used. Based on them, a dataset was created, including games' metadata, data on game situations related to the behavior of individual players - movement, use of equipment, results of interaction with the enemy, as well as additional computed features of team play related to the level of game activity, the use of open space, traps, group character and speed of movement. Based on the methods of machine learning, recommendations are made on important features of priority team strategies that assume high / low values of these features in case of victory or defeat of the respective team. Also, was made a study of the features that affect the victory of individual teams with antagonistic gaming interests.
Аннотация
Идеальное сочетание спортивных и информационных технологий, реализованное в видео- и компьютерных играх, сделало киберспорт необыкновенно популярным, стремительно развивающимся и отвечающим вызовам современного мира. Стремительными темпами увеличились как масштабы проводимых соревнований, так и объемы финансовых вложений в киберспорт. В связи с этим особую привлекательность получают виды деятельности, связанные с подготовкой спортивных команд к соревнованиям. Особый интерес для тренеров представляет изучение лучшей соревновательной практики, стратегий и тактик как команд соперников, так и отдельных их игроков.
В статье предложена и подтверждена гипотеза, что гораздо более содержательны и обоснованы методы подготовки киберспортсменов, основанные на математических подходах к анализу дан-
Alexander M. Kadan, Candidate of Technical Sciences, Associate Professor, Department of Mathematics and Informatics, Yanka Kupala State University of Grodno (22 Ozheshko Str., Grodno 230023, Belarus), ORCID: http://orcid.org/0000-0003-3701-8100, kadan@mf.grsu.by
Le Li, Master's degree student of Engineering and Technology, Department of Mathematics and Informatics, Yanka Kupala State University of Grodno (22 Ozheshko Str., Grodno 230023, Belarus), ORCID: http://orcid.org/0000-0003-3402-1646, nenelovesg@gmail.com
Tsiango Chen, Master's degree student of Engineering and Technology, Department of Mathematics and Informatics, Yanka Kupala State University of Grodno (22 Ozheshko Str., Grodno 230023, Belarus), ORCID: http://orcid.org/0000-0003-4100-3809, peerlessjian@gmail.com
Vol. 14, no 2. 2018 ISSN 2411-1473 sitito.cs.msu.ru
Modern Information Technologies and IT-Education
ных, которые формально подтверждают преимущества тех или иных командных тактик и позволяют сделать выводы о перспективах использования тех или иных методов ведения игры. В качестве основного источника данных для анализа стратегий, используемых командами кибер-спортсменов, были использованы демо-файлы сохраненных игр турниров по Counter Strike. На их основе был сформирован датасет, включающий метаданные игры, данные об игровых ситуациях, связанных с поведением отдельных игроков - движением, использованием экипировки, результатах взаимодействия с противником, а также дополнительно рассчитанные признаки командной игры, связанные с уровнем игровой активности, использованием открытого пространства, ловушек, группового характера и скорости перемещения. На основе методов машинного обучения сделаны рекомендации о важных признаках приоритетных командных стратегий, которые предполагают высокие/низкие значения этих признаков в случае победы или поражения соответствующей команды. Также было проведено исследование признаков, влияющих на победу отдельных команд, имеющих антогонистические игровые интересы.
Introduction
eSports (also referred to as cybersport, e-sports, electronic sports, pro gaming) are a form of competitive activity and special practice of preparing for competitions on the basis of computer and / or video games. The cyber game provides an environment interaction management facilities, ensuring equal conditions of competitions with a person or team to team1.
Modern man is no longer satisfied with the traditional way of leisure and entertainment and strives for their new forms. Electronic sports meet the needs of modern man because of extraordinariness and attractiveness, effective implementation of incredible stories in a high degree. The perfect combination of sports and information technologies, implemented in video and computer games, has made eSports unusually popular, rapidly developing and meeting the challenges of the modern world.
Technologies of electronic sports deeply penetrated all spheres of modern society, such as entertainment, sports, software, telecommunications, computer equipment production, information technology, finance [1]. Electronic sports in some countries, such as South Korea, Japan, Sweden, the United States, have become an important sector of the national economy and have a financial turnover of tens of billions of dollars [2-5].
eSports as a sport recognized in Russia. So on June 7, 2016, the order of the Ministry of Sport of the Russian Federation on the inclusion of eSports in the register of official sport disciplines of the Russian Federation was published2. On January 6, 2017, the Ministry of Justice of the Republic of Belarus registered the Republican Public Association "Belarusian Federation of eSports"3.
All eSports disciplines are divided into several main classes, distinguished by the properties of game spaces, models, the game task and the developing gaming skills of eSports players: first-person shooters, real-time strategies, auto- and avia-simulators, team role-playing games with tactical and strategic game elements and other [6-8].
Features of special training for eSports competitions
Due to the fact that the prize funds of the eSports competitions can reach several million dollars, the activities related to the training of sports teams for competitions received particularly attraction. Of particular interest to coaches of eSports teams is the study of the features of human immersion in the game [9 12], the psychology of human behavior in competitive situations of digital reality [13 19], human-machine interaction in extreme situations [20 22], the best competitive practice, strategy and tactics both teams of rivals, and their individual players [23 26].
Features of individual training for eSports competitions Traditionally, the training of eSports players is reduced to the formation of technical skills of playing the game [20] and psychological stability [14, 15, 16].
The trainers of the eSports teams have developed special methods with which they can accurately determine the degree of the current training of the eSports player and the level of his immersion in the game [9]. It is necessary to monitor each of the player closely to not miss those moments that need special attention and training. For example, the instructor noticed that the player incorrectly places his hands on the keyboard or the mouse is not properly held, or he has problems with the correct choice of the strategic position in the game, and so on. Based on this information, the training course can be adjusted, which makes it possible to work as efficiently as possible and not teach what players already know or, conversely, to focus on what is not given to them.
Forming team play skills
First-person shooter assumes not only a high level of individual training of cyber sportsmen, but a competent and coordinated game in the team is equally important. Teaching skills and tactics of the
1 Charter of the Russian eSports Federation / Russian eSports Federation [Электронный ресурс]. URL: https://resf.ru/about/documentation/1/ (дата обращения: 20.05.2018).
2 Official Internet portal of legal information [Электронный ресурс]. URL: http://publication.pravo.gov.ru/Document/View/ 0001201606070022 ?index=0&rangeSize=1 (дата обращения: 20.05.2018).
3 V Belarusi poyavilas' federaciya kibersporta // Informational portal TUT.BY [Электронный ресурс]. URL: https://42.tut.by/526631 (дата обращения: 20.05.2018).
Современные информационные технологии и ИТ-образование
Том 14 № 2 (2018) ISSN 2411-1473 sitito.cs.msu.ru
team play is a priority in the training of high-level teams. It includes a whole complex of exercises and tasks aimed to develop key skills, starting from the basic exercises and ending with teaching the tactical guidance of the team.
The main goal of the coach is to create one strong team from the group of individual talented players. Working with the eSports players, it is important to realize that the coach has a great influence on their worldview [25, 26].
At the same time, the perception of the best game practice by the coaches of eSports teams is subjective, formed as a result of personal observations, communication with colleagues, and bears the imprint of his preferences and worldview. The experience of using approaches based on the use of mathematical methods and formal conclusions to the organization of the training process are not known to authors. At high-level eSports tournaments, all games records are saved and available for later review. Therefore, it is possible to consider the task of analyzing such stored data not only to learn the qualities of individual players, but, first of all, to explore the strategy of the team play, to determine the game style of the team. And also to find the critical points of the game, on which the final result depends, and to predict the most likely behavior of the opposing team. Taking into account the above, the aim of this work is to develop methods for modeling and analyzing the features of team play strategies in eSports applications (using the example of Counter Strike) and developing recommendations based on them that help achieve high eSports results.
Brief description of the game Counter Strike
Counter-Strike: Global Offensive (CS:GO) - is a multiplayer firstperson shooter computer game developed by Valve Corporation and Hidden Path Entertainment [6-8]. It is devoted to the confrontation of terrorists (terrorist forces) and counter-terrorists (counter-terrorist forces, in the Russian localization of Global Offensive - Special Forces). The task for the terrorist team is to destroy the counter-terrorist team and explode the bomb in a certain place (on the game maps there are usually two this places). The task for the counter-terrorist team is to destroy the terrorist team and prevent the bomb exploding. In more detail, we can note the following:
1. The game involves 2 teams of 5 players each.
2. The match takes place on one of the official game maps.
3. The game consists of 2 periods. The first period lasts 15 rounds (for example, it can end with the score 9-6). The second period ends if the team wins in 16 rounds (for example, 16-13)
4. Each team spends one period for defense (for the team Counter-Terrorist Force), and the second period - for the attack (for the team of terrorists).
5. If after two periods the teams have an equal number of won rounds (15-15), 2 additional periods of 3 rounds are appointed.
6. At the beginning of each period on the financial account of each team is 800 USD. At the beginning of each additional period on the account of each team is 10 000 USD.
Players have access to 5 levels of equipment: the first - for primary weapons (Shotguns, Sub-Machine Guns, Rifles and Machine Guns); the second for the secondary weapon (Pistol); the third - for melee weapons (Knives and Tasers); the fourth for grenades; the fifth - for a bomb (it can only be carried by terrorists).
In Counter-Strike, one of the aspects of the game is the amount of money on the player's account. This money can be used to buy weapons and equipment. It is possible to shop only in the so-called buy-zone, presence at which is symbolized by corresponding icon. The presence of a limited cash account makes serious adjustments to the tactics of the game. Good and careful planning of the monetary policy in the match can substantially bring the team closer to winning.
Retrieving data for analysis
Information about all large and small official tournaments since the release of the Counter Strike (2013) is available on the official website of HLTV4. It also contains information on upcoming matches, team's line-up, statistics, etc. Also through this site, you can get access practically to all records of the games played in tournaments on Counter Strike.
Replays are stored there in the form of "demo-files" or "demos", which have the extension "*.dem" and are generated by the game server each time the game ends. Demo files are binary files of special structure, in which all the events of the completed game are recorded. The size of the demo file depends on the complexity and duration of the game and usually reaches several hundred megabytes. The demo file consists of events associated with the so-called "tick" (tickrate is unit frequency of sending and receiving data from the server). During the game, the game machine data is exchanged with the server. How often this exchange takes place depends on the tickrate. Each "tick" server and client update the data. The state of the game zone, the coordinates of the players, the data about the shoots, the wound and death of the players and other details of the game process are updated. In official games, the tickrate equals 128, i.e. there are 128 "ticks" per second.
To extract data from demos, special programs ("demo-parsers") can be used. They allow you to provide access to the structure of the demo file and organize the extraction of information about interesting events from it. In this paper, the #DemoInfo5 parser was used, because it allows a simple modification and initially has most of the necessary functions.
Data structures used in the analysis
Based on the capabilities of the demo parser #DemoInfo and its SDK, the structure of the demo file can be characterized as follows. During the match, there is a huge number of events. The event includes not only the murder / death of the player, but also the movement of players, shots, etc. Each event is tied to a tick. Ticks are counted throughout the whole round and by the end their number, as a rule, reaches several hundred thousand. During each event, complete information is collected about the player (active weapon, health indicator, position on the map, etc.). As a result, the demo file contains the entire set of events and access to them is done only by "tick". And to get the right information it is enough to find the necessary "tick" and select the necessary information from the provided in the demo file.
The data that is extracted from the demo file and used for further analysis will be stored in the form of several log-files represented in the format of csv-files.
For analysis purposes, we will use files that contain:
4 CS:GO News & Coverage // HLTV.org [Электронный ресурс]. URL: https://www.hltv.org/ (дата обращения: 20.05.2018).
5 #DemoInfo [Электронный ресурс]. URL: https://github.com/StatsHelix/demoinfo (дата обращения: 20.05.2018).
Vol. 14, no 2. 2018 ISSN 2411-1473 sitito.cs.msu.ru
Modern Information Technologies and IT-Education
1. Data on the movement and gaze direction of the players.
2. Data on the events of "murder / death" ("fighter / victim").
3. Data on the events "shots".
4. Data on the events "loss of health" ("shot / damage").
5. Data on the events "use of grenades".
6. Metainformation about the rounds of the game.
These files are generated from the demo file using the #DemoInfo parser. The parser code was previously modified.
Receiving data related to the features of the team play
In addition to the above "primary" data, which can be directly extracted from the demo file, various non-elementary features (attributes) of game situations are also interesting. They can be obtained only as a result of calculations on the "primary" data.
This, in particular, features such as the size of the controlled space (the size of the zone visible to the player at various moments in the game); average speed of moving players at different stages of the game; the type and direction of hits into the player by enemy and a number of other features related to both the individual game behavior of the eSports player, and the features of the team play. Such features of the game can be used during the training process, for example, when examining samples of the best game practice or for
щ щ < ■
W0&
In particular, we used the percentage of playing time, during which the team's players were in the zone of small (blue color on heat map), medium (yellow) and high (red) visibility as features of the team play influencing the victory of a team.
The degree of aggressiveness of the game at various stages In CS:GO, there are two types of game - aggressive, which is characterized by fast movement on the game map, and calm. Aggressive game is considered the preferred option. The advantage of such game is that the team constantly carries out active movements and does not use any ambushes, but act actively. This allows you to put the enemy in a dead end and always have a few seconds of advantage. Calm game involves slow moving on the map, using all possible tricks and loopholes for enter the enemy's rear. It is desirable to use this variant of the game as part of the group in order to ensure the proper level of protection.
As one of the features of the game the degree of aggressiveness of
comparing your team's performance with the indicators of a potential opponent's team.
In this paper, both "primary" data and computed game features will be used to determine among them such features that have a significant impact on the achievement of victory. Briefly discuss the specifics of some of them.
Estimating the average size of the controlled space for the team The CS:GO game is played on the official map chosen before the start of the competition. The player, being at a certain point of such a map, has a limited controlled space for observation and impact on the enemy using available weapons. In fact, the player chooses the position in the game is largely due to the fact that it can help the player to inflict more damage on the enemy and less damage to himself. Thus, the search and the occupation of positions that are characterized by a larger visibility zone on the map (or the same as the larger area of the player-controlled space) will probably be one of the factors that significantly affect the result of the game.
In this paper, the official "de_mirage" map was chosen as the map for analysis. The analysis methods for other maps are similar. The graphic representation of the map "de_mirage", an example of the zone of visibility (controlled zone) for a pre-selected point, and the heat map of the controlled zones of the map are shown in Figure 1.
the team at various stages of the round of the game was considered. The degree of aggressiveness of the game was calculated on the basis of the average value of the movement of players on time intervals of a given size (for example, on intervals of 1000 ticks, which is 12.8 seconds).
Based on the change in the coordinates of each player during the round, the value of his movement at the given time interval was calculated. This allowed us to draw conclusions about the team's activity at various stages of the game.
Graphs 2a) and 2b) in Figure 2 represent the amount of movement of teams on the final interval during one round of the game. The third graph represents the difference in movement for the two teams. Unfortunately, the magnitude of the aggressiveness of the game can not uniquely predict the victory: in this round Team 1 won, although the game of Team 2 was much more aggressive. Thus, the nature of the dependence of the result of the game on the level of aggressiveness of the game should be studied in more detail.
a) the original map b) an example of the controlled zone c) the heat map of the controlled zones
Fig. 1. Example of a game map and evaluation of its controlled zones (zones of visibility) а) оригинальная карта б) пример зоны видимости c) тепловая карта зон видимости
Рис. 1. Пример игровой карты и оценки величины ее зон видимости (контролируемого пространства)
Современные информационные технологии и ИТ-образование
Том 14 № 2 (2018) ISSN 2411-1473 sitito.cs.msu.ru
Alexander M. Kadan, Le Li, Tsiango Chen
Big Data and applications
401
a) the movement of the 1st team
b) the movement of the 2nd team
c) the difference in the movement
Fig. 2. Example of visualization of data about the aggressiveness of team players
а) перемещение 1-й команды
б) перемещение 2-й команды
в) разность перемещений команд
Рис. 2. Пример визуализации данных о перемещении игроков команды
Determining the nature of the group movement of players
Для командного передвижения игроков традиционно
рассматриваются два варианта:
For the team movement of players two options are traditionally considered:
1. Divide, one go straight, and the other gets around the enemy from the rear.
2. Go together in the chosen direction.
Most players choose the first option, and are often mistaken. Players are sure that if they surround the enemy, they will win the round, but this is a misconception. The fact is that when the players are divided, the strength of their team becomes weaker. For example, a good player who can neutralize the enemy with one shot, separates from the partner and goes around. As a result, the group has already lost a strong team member. Therefore, there is a steady opinion that it is best to stay together and try to act together. The work also considered the assumption that prolonged group movement will achieve a positive result of the game. Movements of
groups of 2 and 3 players were considered.
To calculate the percentage of the group movement of the team, the game round was divided into n intervals, where n = (game round length) / 1000). For each of the possible combinations of 5 players on 2 (or of 5 on 3, in the case of group of 3 players), the percentage of group movement at each of the n intervals was calculated as follows. If the distance between all the players in the group for less than half the interval was less than a certain constant (for example, 100), then their movement was considered as the group movement within the interval.
Figure 3 graphically shows the share of group movement of the both teams players. The data was obtained from the game, which ended in the victory of the terrorist team. The graphs show that the share of the group movement of the terrorist team is greater than of the counter-terrorist team. Therefore, we can assume that a high share of group movement can have positive impact on the result of the game.
original value
Counter-terrorist team Terrorist team
10 15
Interval number
sorted value
07
0.6 OS 0.4
0.3 ^ 0 2 0-1
0.0 -I
x Counter-terrorist team Terrorist team
e#** x x * *
X X * * X
xxx
-Г"
10
Interval number
-Г"
15
20
Fig. 3. Graphical representation of the share of group movement Рис. 3. Графическое представление процента группового движения
Vol. 14, no 2. 2018 ISSN 2411-1473 sitito.cs.msu.ru
Modern Information Technologies and IT-Education
The degree of activity of the game at various stages The activity of eSports players, determined by the team play tactics, can vary depending on the stage of the game. We've divide the game into three stages and analyze the level of activity of players on each of them. First and last intervals in 1/10 of the total round time will be defined as "the beginning of the game" and the "the end of the game", the remaining 8/10 - as "the middle of the game". The percentage of active movement for each team at each stage is calculated in the same way as in the case of determining the level of aggressiveness of the game. To calculate the numerical values of the team's game activity at various stages of the game, we will divide the length of the game round into intervals of 1000 ticks. For each interval,
we find the average amount of movement for all active players of the team. If the average movement is greater than a certain constant (for example, 1000), then we assume that the team was actively moved at this interval. The ratio of the number of such intervals to the total number of intervals in the game stage will be considered as the percentage of team activity at this stage.
Figure 4 shows the activity of Team1 and Team2 teams at the initial stage of the game, in the middle of the game and at the final stage of the game. For team Team1 percent activity is 71%, 67%, 57% for the beginning, middle and end of the game, respectively. For team Team2 these indicators are equal 85%, 64%, 71%, respectively.
Fig. 4. Example of visualization of movement activity Рис. 4. Пример визуализации активности движения
Analysis of damage done to the enemy
In first-person shooter games and, in particular, in CS:GO, a lot of events related to shooting and using of grenades. In addition to the events of the player's complete neutralization (murder), intermediate events are important, involving partial damage to enemy players. In this regard, various situations of hitting the enemy were analyzed, such as hitting in front, left, right, behind.
The specified situations can be recognized on the basis of the data of the main log-file «shot-hits». Using information about the attacker's shot direction and the victim's gaze direction, we can calculate at what angle relative to the position of the victim the shot was made. If the angle between the indicated directions is in the range from - n / 4 to n / 4 radians - then the shot that caused the damage was made from behind; in the range from 3n / 4 to 5n / 4 - in the front; in the range from n / 4 to 3n / 4 - on the right; and left in the remaining cases. Using this approach, you can find the number of hits of each type in each round of the game. Such data allow us to conclude that players use traps and a secret location. In the presence of «frontal» hits, obviously, there is a duel «face to face» (which in games happens rarely). If there are «on the side» hits, and even more «behind» - we can conclude the use of tactic, which involves the use of ambushes, traps and detours of the enemy, was used.
Evaluation of game situations using machine learning methods
Formation of a dataset for analysis
To form the dataset, the data extracted directly from the game's demo file and the data calculated on the basis of game situations associated with individual game rounds were used. The dataset was used to make decisions based on machine learning methods [27, 28]. During the formation of the dataset, 50 saved demo files were processed. All demo files were records of games played on the same «de_
mirage» map. The total count of records is 1536. The row of the dataset includes 61 features. Features can be divided into three categories: the metadata of the round; the data of the terrorist team game; the data of the counter-terrorist team game. The target value associated with the row is the type of the winning team - terrorist team or counter-terrorist team.
The metadata of the round contains 15 features and includes, in particular, the number of the round; duration of the round in ticks; the number of points scored by the terrorist team; the number of points scored by the counter-terrorist team; the number of surviving players in the terrorist team; the number of surviving players in the counter-terrorist team; initial sum of money of terrorist team; initial sum of money of counter-terrorist team; initial equipment of terrorist team; initial equipment of counter-terrorist team; whether a bomb was planted; whether the bomb was defused.
The data included in the dataset for terrorist and counter-terrorist teams is approximately the same. For each team, 23 features are included in the dataset. In particular, this is: the start time of the movement; average speed; percentage of active game time; the percentage of active game time at the k-th stage (k = 1,2,3 - start, middle, end of the game); percentage of time of group movement (2 players); percentage of time of group movement (3 players); the average size of the zone of visibility per game; the percentage of the k-th level of visibility (k = 1,2,3 - small, medium, high visibility); the number of shots; the number of hits; the number of hits in front / back / left / right; the number of flash / smoke / HE grenades used; percentage of effectively used grenades.
Prediction of the game result
To predict the game result based on the built-in dataset, we conducted an experiment. In the experiment, four methods of machine learning were used: the method of decision trees, the method of a random forest, the method of k-nearest neighbors, and the method of princi-
Современные информационные технологии и ИТ-образование
Том 14 № 2 (2018) ISSN 2411-1473 sitito.cs.msu.ru
pal component analysis.
The software was developed on the basis of the library of machine learning scikit-learn of Python 3 [27, 28]. When learning the model, 60% of the data from the original data set was used as training data. The remaining 40% of the data was used as test data. The results of predicting the victory of an arbitrary team based on the features of the dataset are presented in Table 1. A high percentage of correct predictions indicates the correct choice of the features of the dataset.
Table 1. Percentage of successful predictions of round results Таблица 1. Процент успешного предсказания результата раунда
Method Percentage of correct predictions
Decision trees 97,9 %
Random forest 99,7 %
k-nearest neighbors 97,9 %
Principal component analysis 97,1 %
In Figure 5, the red dots correspond to the average values (normalized to 1) of the features at which the victory of the terrorist team was achieved. The blue line connects the average (normalized to 1) values of the same features, at which the victory of the counter-terrorist team was achieved.
As can be seen in Figure 5, the probability of a victory for the terrorist team is higher the more the value of the features "4, Planting the bomb", "10, Percentage of active movement at the final stage of the round", "17, Number of shots", "18, Number of hits" "19, Number of hits to the opponent from behind", "22, Number of smoke grenades used", "23, Number of used flash grenades", "24, Number of HE grenades used", "25, Percentage of grenade use efficiency". Conversely, the probability of victory for terrorists team is higher, the lower the value of the features "5, Defusing the bomb", "14, Percentage of time in low-level visibility zone", "15, Percentage of time in the middle-level visibility zone" etc.
Factors affecting the result of the game and recommendations for teams
Prediction the result of the game and recommendations for the terrorist team
Let's try to answer the question, what features affect the achievement of a successful result of game for the terrorist team (as well as the counter-terrorists team).
We will assume that success (victory) does not depend on the game of the enemy, but only on the game quality of the team itself. To do this, we drop all the columns from the dataset in which are the features associated with the enemy team. As a result, further work is done with 25 columns of the original dataset.
On an abbreviated version of the dataset, the accuracy of the prediction with the decision tree method is reduced to 88.06%. In addition, the decision tree method allows you to assess the importance of single game features. Based on the data of the decision tree method, it can be argued that the following features (sorted in order of decreasing importance) are most important for assessing the quality of the game of the terrorist team: "Planting the bomb" (36.66%); "Defusing the bomb" (18.90%); "The number of hits" (11.35%); "The number of shots" (6.35%); "Level of equipment" (4.14%); "Duration of the round" (4.02%); "Percentage of effective use of grenades" (2.92%); "The number of used HE grenades" (2.63%); "The amount of money the team in the early rounds" (2.50%); "Percentage of the presence in the zone of low-level visibility" (2.46%).
On the same basis, we can conclude that the following features do not play a significant role in predicting the result of the game (the percentage of their influence on the result is approximately equal to 0.00): "Percentage of active movement at the initial stage"; "Percentage of active movement in the middle of the game"; "Percentage of the playing time of the movement by the group of 2"; "Percentage of the playing time of the movement by the group of 3"; "Percentage of time in the high-level visibility zone"; "Percentage of time in the zone of middle-level visibility".
In order to formulate recommendations, it is important to find the answer to how the most important features affect the victory of the team - with increasing or decreasing the values of such features. To do this, we find the mean values of the influencing features in case of victory and their average values in case of defeat of the terrorist team.
0 12 3 4 5 6 7
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Features of team play
[(0, 'Winner'), (1, 'LenRound'), (2, 'T_StartMoney'), (3, 'T_EquipValue'),
(4, 'BombPlanted'), (5, 'BombDefused'), (6, 'T_AvgSpeed'), (7, 'T_PerActiveMove'), (8, 'T_PerActiveMoveStarf), (9, 'T_PerActiveMoveMid'), (10, 'T_PerActiveMoveEnd'), (11, 'T_PerGroupMove2'), (12, 'T_Per-
GroupMove3'), (13, 'T_AvgViewArea'), (14, 'T_PerViewAreaLess'), (15, 'T_PerViewAreaMid'), (16, 'T_Per-ViewAreaMuch'), (17, 'T_NumShots'), (18, ,T_NumHits'), (19, ,T_NumHitsBack'), (20, ,T_NumHitsLeft'), (21, ,T_NumHitsFront'), (22, ,T_NumSmokeGrenade'), (23, ,T_NumFlashGrenade'), (24, ,T_ NumFireGrenade'), (25, ,T_PerEffectiveGrenades')] Fig. 5. Dependence of the victory of the terrorist team on the values of game features
Рис. 5. Зависимость победы команды Террористов от значений атрибутов игры
Prediction the result of the game and recommendations for the counter-terrorist team
Let's also try to answer the same question for another team - what factors influence the prediction of the result of the game for the counter-terrorist team.
We also assume that the success of the team does not depend on the play of the enemy, but only on the level of the play of the team itself. To do this, we drop all the columns from the dataset containing the features associated with the terrorist team. As a result, further work is done with 25 columns of the original dataset. The accuracy of the prediction with the decision tree method is reduced to 83.89%. Based on the data of the decision trees method, it can be argued that in order to assess the quality of the play of the
Vol. 14, no 2. 2018 ISSN 2411-1473 sitito.cs.msu.ru
Modern Information Technologies and IT-Education
counter-terrorist team, the most important are the following features (sorted in order of decreasing importance): "Planting the bomb" (36.66%); "Defusing the bomb" (18.90%); "Level of equipment" (11.63%); "The number of hits" (8.52%); "The amount of money the team in the early round" (3.92%); "Percentage of the presence in the zone of middle-level visibility" (2.79%); "Percentage of time of active movement" (2.70%); "The duration of the round" (2.38%); "The number of hits to the enemy on the left" (2.13%). You can also draw a conclusion (based on the data of the dataset used) that the following features are almost irrelevant for predicting the result of the game (the percentage of their effect on the result is approximately equal to 0.00): "Average speed of movement"; "Percentage of active movement at any stage of the round"; "Percentage of the playing time of movement by the group"; "Percentage of time in the zone of high-level visibility"; "Number of smoke grenades used"; "Number of flash grenades used".
As before, we will determine how the most important features affect the victory of the team - with increasing or decreasing the values of such features. To do this, we will find the average values of the influencing features in case of victory and their average values in case of defeat of the terrorist team.
In Figure 6, the blue dots correspond to the average values (normalized to 1) of the features at which the counter-terrorist team's victory was achieved. The red line connects the average (normalized to 1) values of the same features at which the victory of the terrorist team was achieved. As can be seen in Figure 6, the probability of the victory of the counter-terrorist team is greater, the higher the value of the features "3, Level of Equipment", "5, Defusing the bomb", "6, Average speed of movement", "8, Percentage of active movement at the initial stage of the round", "15, Percentage of average time in the zone of visibility", "17, Number of shots", "18, Number of hits", "19, Number of hits from the back", "22, Number of smoke grenades used", "23, Number of flash grenades used", "24, Number of HE grenades used", "25, Percentage of grenade use efficiency".
• • Counter-terrorist team — Terrorist team
• /
♦ * • • # • ф • /
• • / l\ * /
•
0 1 2 3 4 5 6 7
3 10 II 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Features of team play
[(0, 'Winner'), (1, 'LenRound'), (2, 'CT_StartMoney'), (3, 'CT_EquipValue'), (4, 'BombPlanted'), (5, 'BombDefused'), (6, 'CT_AvgSpeed'), (7, 'CT_PerActiveMove'), (8, 'CT_PerActiveMoveStart'), (9, 'CT_PerActiveMoveMid'), (10, 'CT_PerActiveMoveEnd'), (11, 'CT_PerGroupMove2'), (12, 'CT_PerGroupMove3'), (13, 'CT_AvgViewArea'), (14, 'CT_PerViewAreaLess'), (15, 'CT_PerViewAreaMid'), (16, 'CT_PerViewAreaMuch'), (17, 'CT_NumShots'), (18, 'CT_NumHits'), (19, 'CT_NumHitsBackf), (20, 'CT_NumHit-sLeff), (21, 'CT_NumHitsFront'), (22, 'CT_NumSmokeGrenade'), (23, 'CT_NumFlash-Grenade'), (24, 'CT_NumFireGrenade'), (25, 'CT_PerEffectiveGrenades')] Fig. 6. Dependence of the victory of the counter-terrorist team on the values of game features
Рис. 6. Зависимость победы команды Спецназа от значений атрибутов игры
Conversely, the probability of a counter-terrorist team victory is higher, the lower the value of the features "4, Planting the bomb", "9, Percentage of time in low-level visibility zone", "16, Percentage of time in high-level visibility zone", etc.
Universally important game features
Comparison of the graphs in Figures 5 and 6 confirms that, whether it be a terrorist team or a counter-terrorist team, there are some common features of the game that are important for victory. In particular, the higher the values of the features "The amount of money the team in the early round", "Percentage of active movement time", "Number of shots", "Number of hits", "Number of grenades used", "Percentage of time in the zone of middle-range visibility", etc. , the greater the probability of winning any of the teams.
Conclusion
In this article, the authors examined and analyzed, based on methods of data analysis, the advantages of team strategies used in the game practice and during the training process in the preparing of eSports players team.
As the main data source for analyzing the strategies used by eSports players teams, the demo files of the saved games of the Counter Strike tournaments were used. Methods for selecting and extracting data for analysis have been developed. Based on them, the dataset was created, including metadata of games, data on game situations related to the behavior of eSports players teams movement, use of equipment, results of interaction with the enemy. Additionally, team game features related to the level of game activity, control of open space, the use of traps, the speed of movement and the group character of the game were calculated.
Based on the methods of machine learning, important features of team strategies were found. It was also determined which values, high or low, should have these features in case of victory or defeat of the team. Also, the study of features that influence the victory of teams with antagonistic gaming interests was conducted. We hope that the results of the work will be interesting to coaches and players of eSports teams, as well as specialists in data analysis. The events associated with first-person shooter games, in particular, with the game Counter Strike, are very diverse. This allows us to make new assumptions about the nature of personal and team strategies used, to put forward new hypotheses, to form new datasets for analysis and to conduct their research. Of course, it is interesting to conduct such an analysis on a training set of data of a much larger volume, as well as using data obtained on other maps. And also to perform the search for new game features that significantly affect the result of this or that team.
Perhaps all this will allow finding, justifying and applying new schemes and tactics in the training process that will increase the chances of winning.
Acknowledgements
The work was supported by the company "Vizor Games" (Minsk), the leading developer of computer games for social networks.
Современные информационные технологии и ИТ-образование
Том 14 № 2 (2018) ISSN 2411-1473 sitito.cs.msu.ru
References [14]
[1] Freeman G. Multiplayer Online Games: Origins, Players, and Social Dynamics. New York: A K Peters/CRC Press, 2018. 178 [15] p.
[2] Chang T-S., Ku C-Y., Fu H-P. Grey theory analysis of online population and online game industry revenue in Taiwan. Technological Forecasting and Social Change. 2013; 80(1):175-185. [16] DOI: 10.1016/j.techfore.2012.06.009
[3] Chee F. Understanding Korean experiences of online game hype, identity, and the menace of the "Wang-tta". Proceedings of the 2005 DiGRA International Conference: Chang- [17] ing Views: Worlds in Play. 2005, vol. 3, 12 pages. Available
at: http://www.digra.org/wp-content/uploads/digital-li-brary/06278.12446.pdf (accessed 20.05.2018).
[4] Boyle E.A., Connolly T.M., Hainey T., Boyle J.M. Engagement in [18] digital entertainment games: A systematic review. Computers
in Human Behavior. 2012; 28(3):771-780. DOI: 10.1016/j. chb.2011.11.020
[5] Yamamoto K., McArthur V. Digital economies and trading in counter strike global offensive: How virtual items are valued [19] to real world currencies in an online barter-free market. Proceedings of 2015 IEEE Games Entertainment Media Conference (GEM). Toronto, ON, Canada, 2015. pp. 1-6. DOI: 10.1109/ [20] GEM.2015.7377220
[6] Jansz J., Tanis M. Appeal of Playing Online First Person Shooter Games. CyberPsychology & Behavior. 2007; 10(1):133-136. [21] DOI: 10.1089/cpb.2006.9981
[7] Grimshaw M., Charlton J., Jagger R. First-person shooters: Immersion and attention. Eludamos. Journal for Computer Game [22] Culture. 2011; 5(1):29-44. Available at: http://www.eluda-mos.org/index.php/eludamos/article/view/vol5no1-3/ html3 (accessed 20.05.2018).
[8] Nacke L., Lindley C.A. Flow and immersion in first-person [23] shooters: measuring the player's gameplay experience. Proceedings of the 2008 Conference on Future Play: Research, Play, Share (Future Play '08). ACM, New York, NY, USA, 2008. pp. 81-88. DOI:10.1145/1496984.1496998
[9] Cheng M.-T., She H.-C., Annetta L.A. Game immersion experience: its hierarchical structure and impact on game-based [24] science learning. Journal of Computer Assisted Learning. 2015; 31(3): 232-253. DOI: 10.1111/jcal.12066
[10] Jennett C.I. Investigating Real World Dissociation in Computer Game Immersion: Manipulating Sense of Progression and Measuring Awareness of Distracters. A thesis submitted in [25] partial fulfilment of the requirements for the degree of Doctor of Philosophy. UCL Interaction Centre, Department of Psychology, 2009. 241 p.
[11] Cairns P., Cox A., Nordin A.I. Immersion in Digital Games: Re- [26] view of Gaming Experience Research. Chapter 12. Handbook
of Digital Games (eds M.C. Angelides, H. Agius), Wiley-IEEE Press, 2014. pp. 337-361. DOI: 10.1002/9781118796443. ch12
[12] Reddy G.S.H. Empirical Investigation on Measurement of [27] Game Immersion using Real World Dissociation Factor. Thesis
no: MSCS-2016-13, 2016. 85 p. [28]
[13] Snyder W.M., Wenger E. Our World as a Learning System: A Communities-of-Practice Approach. Blackmore C. (eds) Social Learning Systems and Communities of Practice. Springer, London, 2010. pp. 107-124. DOI: 10.1007/978-1-84996-133-2_7
Pargman D., Jakobsson P. Do you believe in magic? Computer games in everyday life. European Journal of Cultural Studies. 2008; 11(2):225-243. DOI: 10.1177/1367549407088335 Pargman D., Jakobsson P. The magic is gone: a critical examination of the gaming situation. Gaming realities: a challenge for digital culture: Athens, 6-8 October 2006. M. Santorineos (Ed.) Athens: Fournos, 2006. pp. 15-22. Jakobsson P. E-sport, cyberatleter och professionalitet. Unga och nätverkskulturer: mellan moralpanik och teknikroman-tik. S. Lindgren, T. Sandgren (Ed.) Stockholm: Ungdoms-styrelsen, 2007. pp. 61-80.
Suárez L., Thio C.F., Singh S. Why people play massively multiplayer online games? International Journal of e-Education, e-Business, e-Management and e-Learning. 2013; 3(1):7-12. DOI: 10.7763/IJEEEE.2013.V3.184
Kim Y., Lee J., Kim G. Extending "out of the body" saltation to 2D mobile tactile interaction. Proceedings of the 10th Asia Pacific Conference on Computer Human Interaction (APCHI '12). ACM, New York, NY, USA, 2012. pp. 67-74. DOI:10.1145/2350046.2350064
Hudson M., Cairns P. The effects of winning and losing on social presence in team-based digital games. Computers in Human Behavior. 2016; 60:1-12. DOI: 10.1016/j.chb.2016.02.001 Chen Y-C., Chen P.S., Hwang J-J., Korba L., Song R., Yee G. An analysis of online gaming crime characteristics. Internet Research. 2005; 15(3):246-261. DOI: 10.1108/10662240510602672 Hollingdale J., Greitemeyer T. The Effect of Online Violent Video Games on Levels of Aggression. PLOS ONE. 2014; 9(11):e111790. DOI: 10.1371/journal.pone.0111790 Christou G. A comparison between experienced and inexperienced video game players' perceptions. Human-centric Computing and Information Sciences. 2013; 3:15. DOI: 10.1186/2192-1962-3-15
Rambusch J., Jakobsson P., Pargman D. Exploring E-sports: A Case Study of Gameplay in Counter-strike. Proceedings of the 2007 DiGRA International Conference: Situated Play. Tokyo: University of Tokyo, 2007. pp. 157-164. Available at: http://www.digra.org/wp-content/uploads/digital-li-brary/07313.16293.pdf (accessed 20.05.2018). Calvillo-Gámez E.H., Cairns P., Cox A.L. Assessing the core elements of the gaming experience. Chapter 3. Game User Experience Evaluation. R. Bernhaupt (Ed.) Springer International Publishing, 2015. pp. 37-62. DOI: 10.1007/978-3-31915985-0
Windleharth T.W., Jett J., Schmalz M., Lee J. H. Full Steam Ahead: A Conceptual Analysis of User-Supplied Tags on Steam. Cataloging & Classification Quarterly. 2016; 54(7):418-441. DOI: 10.1080/01639374.2016.1190951 Olshefski E.G. Game-changing event definition and detection in an eSportss corpus. Proceedings of the 3rd Workshop on EVENTS at the NAACL-HLT 2015. New York, USA, 2015. pp. 77-81. Available at: http://aclweb.org/anthology/W15-08 (accessed 20.05.2018).
Coelho L.P., Richert W. Building Machine Learning Systems with Python. Second Edition. Packt Publishing, 2015. 305 p. VanderPlas J. Python Data Science Handbook: Essential Tools for Working with Data. O'Reilly Media, 2017. 548 p.
Submitted 20.05.2018; revised 15.06.2018; published online 30.06.2018.
Vol. 14, no 2. 2018 ISSN 2411-1473 sitito.cs.msu.ru
Modern Information Technologies and IT-Education
Список использованных источников
[1] Freeman G. Multiplayer Online Games: Origins, Players, and Social Dynamics. New York: A K Peters/CRC Press, 2018. 178 p.
[2] Chang T-S, Ku C-Y., Fu H-P. Grey theory analysis of online population and online game industry revenue in Taiwan // Technological Forecasting and Social Change. 2013. Vol. 80, issue 1. Pp. 175-185. DOI: 10.1016/j.techfore.2012.06.009
[3] Chee F. Understanding Korean experiences of online game hype, identity, and the menace of the "Wang-tta" // Proceedings of the 2005 DiGRA International Conference: Changing Views: Worlds in Play. 2005. Vol. 3, 12 pages. URL: http://www.digra.org/wp-content/uploads/digital-li-brary/06278.12446.pdf (дата обращения: 20.05.2018).
[4] Boyle E.A., Connolly T.M., Hainey T., Boyle J.M. Engagement in digital entertainment games: A systematic review // Computers in Human Behavior. 2012. Vol. 28, issue 3. Pp. 771-780. DOI: 10.1016/j.chb.2011.11.020
[5] Yamamoto K., McArthur V Digital economies and trading in counter strike global offensive: How virtual items are valued to real world currencies in an online barter-free market // Proceedings of 2015 IEEE Games Entertainment Media Conference (GEM). Toronto, ON, Canada, 2015. Pp. 1-6. DOI: 10.1109/GEM.2015.7377220
[6] Jansz J., Tanis M. Appeal of Playing Online First Person Shooter Games // CyberPsychology & Behavior. 2007. Vol. 10, issue 1. Pp. 133-136. DOI: 10.1089/cpb.2006.9981
[7] Grimshaw M., Charlton J., Jagger R. First-person shooters: Immersion and attention // Eludamos. Journal for Computer Game Culture. 2011. Vol. 5, issue 1. Pp. 29-44. URL: http:// www.eludamos.org/index.php/eludamos/article/view/vol-5no1-3/html3 (дата обращения: 20.05.2018).
[8] Nacke L., Lindley C.A. Flow and immersion in first-person shooters: measuring the player's gameplay experience // Proceedings of the 2008 Conference on Future Play: Research, Play, Share (Future Play '08). ACM, New York, NY, USA, 2008. Pp. 81-88. DOl:10.1145/1496984.1496998
[9] Cheng M.-T., She H.-C., Annetta L.A. Game immersion experience: its hierarchical structure and impact on game-based science learning // Journal of Computer Assisted Learning. 2015. Vol. 31, issue 3. Pp. 232-253. DOI: 10.1111/jcal.12066
[10] Jennett C.I. Investigating Real World Dissociation in Computer Game Immersion: Manipulating Sense of Progression and Measuring Awareness of Distracters. A thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy UCL Interaction Centre, Department of Psychology, 2009. 241 p.
[11] Cairns P., Cox A., Nordin A.I. Immersion in Digital Games: Review of Gaming Experience Research. Chapter 12. / M.C. Angelides, H. Agius // Handbook of Digital Games, Wiley-IEEE Press, 2014. Pp. 337-361. DOI: 10.1002/9781118796443.ch12
[12] Reddy G.S.H. Empirical Investigation on Measurement of Game Immersion using Real World Dissociation Factor. Thesis no: MSCS-2016-13, 2016. 85 p.
[13] Snyder W.M., Wenger E. Our World as a Learning System: A Communities-of-Practice Approach / C. Blackmore // Social Learning Systems and Communities of Practice. Springer, London, 2010. Pp. 107-124. DOI: 10.1007/978-1-84996-133-2_7
[14] Pargman D., Jakobsson P. Do you believe in magic? Computer games in everyday life // European Journal of Cul-
tural Studies. 2008. Vol. 11, issue 2. Pp. 225-243. DOI: 10.1177/1367549407088335
[15] Pargman D., Jakobsson P. The magic is gone: a critical examination of the gaming situation. / M. Santorineos // Gaming realities: a challenge for digital culture: Athens, 6-8 October 2006 Athens: Fournos, 2006. pp. 15-22.
[16] Jakobsson P. E-sport, cyberatleter och professionalitet / S. Lindgren, T. Sandgren // Unga och nätverkskulturer: mellan moralpanik och teknikromantik. Stockholm: Ungdoms-styrelsen, 2007. Pp. 61-80.
[17] Suárez L., Thio C.F., Singh S. Why people play massively multiplayer online games? // International Journal of e-Education, e-Business, e-Management and e-Learning. 2013. Vol. 3, issue 1. Pp. 7-12. DOI: 10.7763/1JEEEE.2013.V3.184
[18] Kim Y., Lee J., Kim G. Extending "out of the body" saltation to 2D mobile tactile interaction // Proceedings of the 10th Asia Pacific Conference on Computer Human Interaction (APCH1 '12). ACM, New York, NY, USA, 2012. Pp. 67-74. DO1:10.1145/2350046.2350064
[19] Hudson M., Cairns P. The effects of winning and losing on social presence in team-based digital games // Computers in Human Behavior. 2016. Vol. 60. Pp. 1-12. DOI: 10.1016/j. chb.2016.02.001
[20] Chen Y-C, Chen P.S., Hwang J-J, Korba L, Song R, Yee G. An analysis of online gaming crime characteristics. Internet Research. 2005. Vol. 15, issue 3. Pp. 246-261. DOI: 10.1108/10662240510602672
[21] Hollingdale J., Greitemeyer T. The Effect of Online Violent Video Games on Levels of Aggression // PLOS ONE. 2014. Vol. 9, issue 11. Pp. e111790. DO1: 10.1371/journal.pone.0111790
[22] Christou G. A comparison between experienced and inexperienced video game players' perceptions // Human-centric Computing and Information Sciences. 2013. Vol. 3, pp. 15. DO1: 10.1186/2192-1962-3-15
[23] Rambusch J., Jakobsson P., Pargman D. Exploring E-sports: A Case Study of Gameplay in Counter-strike // Proceedings of the 2007 DiGRA International Conference: Situated Play. Tokyo: University of Tokyo, 2007. pp. 157-164. URL: http://www.digra.org/wp-content/uploads/digital-li-brary/07313.16293.pdf (дата обращения: 20.05.2018).
[24] Calvillo-Gámez E.H., Cairns P., Cox A.L. Assessing the core elements of the gaming experience. Chapter 3. / R. Bernhaupt // Game User Experience Evaluation Springer International Publishing, 2015. Pp. 37-62. DOI: 10.1007/978-3-319-15985-0
[25] Windleharth T.W., Jett J., Schmalz M., Lee J. H. Full Steam Ahead: A Conceptual Analysis of User-Supplied Tags on Steam // Cataloging & Classification Quarterly. 2016. Vol. 54, issue 7. Pp. 418-441. DOI: 10.1080/01639374.2016.1190951
[26] Olshefski E.G. Game-changing event definition and detection in an eSportss corpus // Proceedings of the 3rd Workshop on EVENTS at the NAACL-HLT 2015. New York, USA, 2015. Pp. 77-81. URL: http://aclweb.org/anthology/W15-08 (дата обращения: 20.05.2018).
[27] Coelho L.P., Richert W. Building Machine Learning Systems with Python. Second Edition. Packt Publishing, 2015. 305 p.
[28] VanderPlasJ. Python Data Science Handbook: Essential Tools for Working with Data. O'Reilly Media, 2017. 548 p.
Поступила 20.05.2018; принята в печать 15.06.2018; опубликована онлайн 30.06.2018.
Современные информационные технологии и ИТ-образование
Том 14 № 2 (2018) ISSN 2411-1473 sitito.cs.msu.ru
Кадан Александр Михайлович, кандидат технических наук, доцент, факультет математики и информатики, Гродненский государственный университет им. Янки Купалы (230023, Беларусь, г. Гродно, ул. Ожешко, 22), ORCID: http://orcid.org/0000-0003-3701-8100, kadan@mf.grsu.by Ли Лэ, магистрант техники и технологии, факультет математики и информатики, Гродненский государственный университет им. Янки Купалы (230023, Беларусь, г. Гродно, ул. Ожешко, 22), ORCID: http://orcid.org/0000-0003-3402-1646, nenelovesg@gmail.com Чень Цзяньго, магистрант техники и технологии, факультет математики и информатики, Гродненский государственный университет им. Янки Купалы (230023, Беларусь, г. Гродно, ул. Ожешко, 22), ORCID: http://orcid.org/0000-0003-4100-3809, peerlessjian@gmail.com
© ®
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted reuse, distribution, and reproduction in any medium provided the original work is properly cited.
Vol. 14, no 2. 2018 ISSN 2411-1473 sitito.cs.msu.ru
Modern Information Technologies and IT-Education
