CC BY
79
KEY MOBILE NETWORK PERFORMANCE INDICATORS CHOICE FOR CUSTOMER EXPERIENCE ESTIMATION
Sergey V. Arzhantsev,
Moscow Technical University of Communications and Informatics (MTUCI), Postgraduate student, Moscow, Russia, [email protected]
Keywords: mobile network, Telco, QoE, CSI, quality, KPI
A typical task for Telco is to assess the quality of network operation and network services. In recent years, the need to move from the network and network elements technical KPIs analysis in the direction of estimation the quality of subscriber services. Because customer satisfaction with the service is the subjective value of - QoE, the correct display matching his expectations is the customer satisfaction index, formed on the basis of processing quantitative interviews - CSI. This problem is actual in the context of growing subscriber base, variety of equip-ment types produced by different vendors, as well as a significant amount of custom services which are different between the main market segments: B2C and B2X.
Tis article tells about existing methodologies for the technical KPI estimation from the client's perspective: on the side of the operator, at the subscriber side and the third side. It examples and shows the characteristics and these methods use cases. Outlined the main KPIs for basic services of mobile network subscriber (voice service, data service, VAS), such as the successful connection rate, the success rate of the delivery, the miss of drop rate in the provision of a service. The algorithm for processing the formation of CSI and technical KPI for the subsequent evaluation of QoE in order to make operational and technical solutions for the implementation of services with a network point of view has been represented. The paper marks and justifies the importance of statistical methods in technical and technological statistical processing. The applicability of statistical methods implemented by separate functional unit with respect to network management systems is justified by the presence of a lot of equipment ven-dors, which are under the management of a communication network systems, such as Huawei, Nokia, Ericsson, Cisco Systems, Alcatel-Lucent, ZTE.
The results of the investigation could be tested on the Telco network to check the correctness of the pro-posed algorithm and its modifications. It is important to note that this approach does not depend on the specifics of the data sources that must be taken into account in the automation implementation process, the need for which is also pointed in this article.
Для цитирования:
Аржанцев С.В. Основные показатели выбора мобильных сетей с точки зрения клиента // T-Comm: Телекоммуникации и транспорт. 2016. - Том 10. - №6. - С. 69-72.
For citation:
Arzhantsev S.V. Key mobile network performance indicators choice for customer experience estimation. T-Comm. 2016. Vol. 10. No.6, рр. 69-72.
7T>
A common problem of the continuous network lifecycle {the basic steps are planning, development and maintenance) is a quality control. This usually meant the network quality control. Since a network is a technical complex consisting of hardware and software components sets (switching equipment, communication lines, service platforms, radio acecss equipment etc.) the subsystems performance quality was estimated separately. Moreover a Lypiea! estimation handled with subsystem elements with an averaging (for instance, an average routers ports utilization in the domen) or by the elements (for example license capacity of a mobile switching center, central processing unit utilization etc.).
The general approach to the problem is collecting a dataset for the statistical measurement of elementary events (usually characterized with counters) or Key Performance Indicators -KPIs formed on their basis. It is necessary to note that the last ones are the calculated values and depend on an equipment vendor. The vendor either describes calculating KPIs implemented in equipment (¡1 is an ordinary practice for decentralized management networks, packet switching based datacom networks for instance) or gives recommendations for user including formulas and counter sets to calculate the KPIs (the usual situation for centralized management networks as mobile networks).
This assessment approach - by KPIs - could be used to estimate network and service performance from the network perspective as the system providing an end user opportunity to use the service. The basic KPI set probably has an ability to assess the service in the case of some degenerate problem (e.g. the service speed of a wired broadband channel to the end B2B client organized with the leased line). But the primary trends in ICT [1] (exponential traffic growth, M2M connections percentage increase, visualization, an equipment unification and etc.) and a significant increase in performance of client terminals makes the approach based on network performance KPIs analytics a by performance inapplicable on the highly competitive market. The concepts specific for the model - QoS and SLA - stored as an integral pan of the network performance assessment, but to the foreground moves the experience evaluation problem of using the client service from the end user perspective - QoE.
The problem of QoE evaluation is especially interesting to solve applying to the convergent mobile Telco network (B2C and B2B segment) which erases the border between technological radio access networks (GSM/UMTS/LTE/WiFi/WiMAX) working with different bands and based on different vendors equipment to form the network (the primary vendors today are Huawei, NSN, Ericsson, Cisco, Alcatel-Lucent, ZTE) and the great variety of user equipment (Apple, Samsung, Huawei, Lenovo, Xiaomi, Sony etc.). The problem could be seem not really actual for rural, where it can be solved by average values of trafflc/speed/availability analysis but is topical for dense city areas with tricky human-made relief.
The problem has some ways to solve:
• Network-wide calculation of network KPIs depending on a service. Traditionally the basic mobile network services are voice/data communication and VAS including SMS/MMS. This approach has limitations especially, for an example after full-functional implementing of IMS on Telco network the concept should be revised significantly since the signal traffic will be pass the unified signaling platform with packet switching on transport layer. From the other hand the problem should be re-
solved by using PCF and PCRF. The advantage of the method is it's unambiguous in terms of network problems localization.
• The most hard service (e.g. video). The essential disadvantage of the method is a heterogeneity of the service named 'video' by users. It could be streaming video (live broadcasting), prebuffered video (TV on demand service) or even OTT-integrated video (Skype video chat). Certainly from the user perspective there could be no difference between lis 'video' types because he didn't get the fully service, but from the operational network quality problem solv ing there are three diverse services. The method also doesn't take into account the (still) basic mobile network service which is a voice communication but also VAS. The advantage of this method is its clarity in performance, unambiguous measurements and extensive experience in the implementation of fixed networks. Also advantage of this method is its relatively simple modeling for the study.
• Technological (not billing) customer CDR logs analysis and its comparison with the network performance statistics. It's the most objective approach from subscriber point of view since it shows the real service experience of user. The great issue of the method is the data value. Another one is the absence of switching subsystem generated CPRs in the case of tail GUL-RAN events (meaning GERAN/UTRAN/cUTRAN of GSM/UMTS/LTE respectively)
• Systems of the CEM class which are actively testing and implementing on Telco networks for the last couple years by software developers are the complex decisions consolidating data from network management and subscriber statistics systems. The main now day issues for them are the architecture complexity and the poor llexibility and functionality.
• Another one way to solve the problem is software probes (apps) installation in user equipment in form of a third-party soft. Some Telcos tested the decisions but the main problem for the mass use is the fact that it is first of all the client-side software installing in user equipment. But since there is a permanent synchronization process with the server side we see very tangible UE battery decrease, it makes negative user experience and leads to removing or turning off the application. One more else minus is the method doesn't handle with mobile phones (not smartphones).
There are more interesting approaches to the problem. An example is a joint project of Ericsson and Indonesian operator XL Asiata, aimed at improv ing the socialization of means of Internet access for users to F-'acebook [3]. But such an approach is not uniform. But in this document, which is marked with typical complexity for estimating QoE, that may be supplemented with:
• The problem of small sample (temporal and territorial - that is, the sample is usually limited in time and location - in conjunction with the network hardware or geographic area). This is usually due to a large array of data you want to store.
• The amount of error of geolocation. Thus, in the experiment conducted by Ericsson binding technical means of verification of KPI statistics and user application receives an error of timing, resulting in an error occurred geolocation.
• Overlay Top level applications - OTT - hard to emulate in the (pseudo) online with a reconciliation of network statistics.
The disadvantage of all of these methods is that after studying the statistics, we will see only an estimate, conducted by studying the statistics, hut do not reflect real customer satisfaction of our services. To display the real customer satisfaction our
services is supposed to use the questionnaire |2] and the formation mechanism of the index of customer satisfaction - Customer Satisfaction Index - CSI (method of forming and collecting information for the formation of CSI deserves separate consideration and will not be shown in this paper).
Since we have determined that the basic services tor the evaluation of quality in terms of end-user mobile communications network services are voice, data and VAS (SMS / MMS as the base), we can create the following list of higher-level KPI in terms of subscriber;
1. Voice communication service
1.1 Call setup success rate (for all sub-systems: the radio, switching, databases domain and B-user side)
1.2 Call drop rate (should be splittcd for A/B-side)
1.3 Speech quality - could be applied by the procedure described in the recommendations of the 3GPP TS 45.008 on 22 December 2014
2. Data communication service
2.1 Average speed per subscriber
2.2 Sessions setup success rate
2.3 Session drop rate
3. SMS/MMS
3.1 Number of SMS successfully delivered with the first time
3.2 Number of MMS delivered from with the first time
3.3 Percentage of successfully delivered confirmation of SMS delivery
3.4 Percentage of successfully delivered confirmation of MMS delivery
An average delivery time for the 3"1 KPls group incorrectly used as a KPI because of the lack of information on subscriber availability in the network (for example, a subscriber B forcibly transferred the phone in airplane mode).
Selected characteristics correspond with the recommendations of 3GPP and ETS1, in particular the ETSI TS 132 410 V12.0.0 (2014-10), the relevant recommendations of the 3GPP TS 32.410 version 12.0.0 Release 12 and ETSI TS 132 455 VI 1.0.0 (20 ¡2-10) - for GSM and UMTS, the relevant recommendations of the 3GPP TS 32.455 version 11.0.0 Release 11 -for LTE [4, 5]. On the basis of these recommendations in the future we need to pick up a set of technical KPI, which correspond with the data of a higher-level indicators reflecting customer experience.
So wc can offer a rational algorithm for the study of customer experience through CSI based on the technical capabilities of service provider and staff (from the staff required advanced knowledge of database theory, statistical analysis, mobile technology and the basics of marketing). This algorithm is shown in Figure I below.
I QoE
I
I
CSI
Díü WJtfi ] 1
depmdngof —> Am data Basic KPI 4* Statistic^ araire «i- tai taa part CSI interview resufcs
1 1 1
Figuru 1. QoE and CSI and their comparison
At the same time, the parameter value of CSI is not objective. It is a subjective evaluation, as will be useless if a significant change in the formation CSI techniques will have a place or ir-
regular/non-representative samples. But over time, by the formation of the optimal points of contact with the customer (for example in accordance with the following services: voice, data, VAS) Telco can monitor the impact of technical indicators of a customer satisfaction.
As a result Of the technical statistics processing and its comparison with CSI data by statistical analysis tools is required to take a number of technical (and organization if necessary) measures in Telco work to correct the KPls in the formation of the next CS! sample. Schematically, this continuous cycle corresponds to the concept of PDCA in the framework of the ideas of lean manufacturing: continuous improvement processes within the goals and flexible adaptation to changes. On the other hand, such an analysis with feedback designed to help locate problem areas in the technological network operator.
Optional task for the future in the monitoring of customer satisfaction the technical aspects of the services is the task of extrapolating CSI changes in conditions or the introduction of new services. The resulting process is schematically depicted in Figure 2 below.
I.Wliindthrr ИВ*» и«» iFonniht ÍÍCiííbrfjn^r.
UffCOWfttOKrt! etmxw jod ur\ir[ tf'QûErfCSi
Figure 2, The process of working with CSI and QoF.
Thus to estimate the customer satisfaction index (to make the comparison between CSI and QoE in fact) should be done the following steps:
1. Expertly choose the KPls and their components.
2. Estimate the tools and sources to get the data (based on the technical and organizational capacity).
3. Form the CSI mythology and make representative sampling.
4. Use statistical analysis instruments to compare QoE and CSI for significance.
5. Plan and implement organizational and technical arrangements for improving QoE / CSI based on the analysis.
6. Close the loop and automate the process.
This approach allows us flexibly organize and dynamically monitor the technical performance indicators and compare it with the expectations of the subscriber regarding the proposed network operator service.
References
!. Arzhantsev S. V, Actual methodological and regulatory aspects of telecommunications operator quality assessment / T-Comm. 2015, Vol 9. No. 4, pp. 31-35. (in Russianj.
2. Erohina LI.. Pctrileleeva T.A., Skornichenko NN. Quality customer service ofcellular communication, 2010.
3. Measuring and improving network performance. Ericsson White paper. Uen 284 23-3245. Ericsson Corp. - October 2014,
4. Digital cellular telecommunications system (Phase 2+): Universal Mobile Telecommunications System (UMTS); LTE; Telecommunication management; Key Performance Indicators (KPI) for UMTS and GSM (3GPP TS 32.410 version 12.0.0 Release 12).
5. LTE; Telecommunication management; Key Performance Indicators (KPI) for the Evolved Packet Core (EPC); Definitions (3GPP TS 32.455 version 11.0.0 Release II).
T-Comm Vol.10. #6-2016
PUBLICATIONS IN ENGLISH
ОСНОВНЫЕ ПОКАЗАТЕЛИ ВЫБОРА МОБИЛЬНЫХ СЕТЕЙ С ТОЧКИ ЗРЕНИЯ КЛИЕНТА
Аржанцев Сергей Владимирович, ФГОБУ ВО МТУСИ, аспирант кафедры МТС, Москва, Россия,
Аннотация
Типовой задачей для оператора связи является оценка качества работы сети и сетевых сервисов. В последнее время стала очевидна необходимость ухода от анализа технических KPI по сети и сетевым элементам в сторону оценки качества абонентских сервисов. Так как удовлетворённость клиента сервисом есть субъективная величина - QoE, то корректным отображением соответствия его ожиданий от качества сервиса является индекс удовлетворенности клиента, формируемый на базе обработки количественных опросов - CSI. Данная задача актуальна в условиях растущей абонентской базы, разнообразия типов оборудования, выпускаемых различными производителями, а также значительного количества пользовательских сервисов, различных между основными сегментами рынка: B2C и B2X. Рассмотрены существующие методики оценки технических KPI с точки зрения клиента: на стороне оператора, на стороне абонента и на третьей стороне. Приведены примеры и особенности использования данных методов. Обозначены основные KPI для базовых сервисов абонента мобильной сети связи (голосовой сервис, сервис передачи данных, VAS), такие как успешность установления соединения, успешность доставки, отсутствие обрывов в процессе предоставления сервиса. Описан алгоритм обработки формирования CSI и технических KPI для последующей оценки QoE с целью принятия операционно-технических решений по реализации сервисов с сетевой точки зрения. Отмечена и обоснована важность статистических методов обработки технической и нетехнической статистики. Применимость статистических методов, реализованных отдельным функциональным блоком относительно систем управления сетью, обоснована большим количеством производителей оборудования, под управлением систем которых находятся сети связи, таких как Huawei, Nokia, Ericsson, Cisco Systems, Alcatel-Lucent, ZTE.
Результаты работы возможно апробировать на сети оператора связи для проверки корректности предложенного алгоритма и его модификации. Важно отметить, что данный подход не должен зависеть от специфики источников данных, что необходимо учесть при реализации автоматизации процесса, необходимость которой также отмечена в данной статье.
Ключевые слова: мобильная сеть, Оператор связи, QoE, CSI, качество, KPI. Литература
1. Аржанцев С.В. Актуальные методологические и нормативные аспекты оценки качества для оператора связи // T-Comm: Телекоммуникации и транспорт. - 2015. - Том 9. - №4. - С. 31-35.
2. Ерохина Л.И., Пантелеева Т.А., Скорниченко Н.Н. Обеспечение качества обслуживания потребителей услуг сотовой связи. -Тольятти: Поволжский гос. ун-т сервиса, 2010.
3. Measuring and improving network performance. Ericsson White paper. Uen 284 23-3245. Ericsson Corp. - October 2014.
4. Digital cellular telecommunications system (Phase 2+); Universal Mobile Telecommu-nications System (UMTS); LTE; Telecommunication management; Key Performance Indicators (KPI) for UMTS and GSM (3GPP TS 32.410 version 12.0.0 Release 12).
5. LTE; Telecommunication management; Key Performance Indicators (KPI) for the Evolved Packet Core (EPC); Definitions (3GPP TS
32.455 version 11.0.0 Release 11).
m
T-Comm ^м 10. #6-2016
