CC BY-NC-ND
167
Specifics of the integration of Business Intelligence and Big Data technologies in the processes of economic analysis
Stanislav Mitrovic
University of Novi Sad, Republic of Serbia; Doctoral Student of the Faculty of Economics Lomonosov Moscow State University
Address: 1, Leninskie Gory, Moscow, 119991, Russian Federation E-mail: [email protected]
Abstract
The volume of data used for economic analysis of the activities of organizations is growing every year. Despite the fact that all information required for economic analysis is available from various sources, such data are very often useless for analysis from the point of view of their economic potential.
The purpose of this study is to outline a foundation for integrating Business Intelligence and Big Data into economic analysis processes. The theoretical and methodological basis of this study is provided by scientific research, methodological and practical developments of domestic and foreign authors on the application of IT solutions in economic analysis.
According to the results of the research, modern information technologies, in particular, the Business Intelligence and Big Data systems have considerably changed the possibilities for improving economic analysis and reducing decision-making time. From the methodological point of view, many aspects of integration of BI and Big Data solutions and their implementation in the economic analysis processes in Russia's companies remain insufficiently developed. The foreign market of modern information technologies for business analytics has a longer history and is being developed more rapidly.
The main conclusions of the study indicate that modern organizations operating on a highly competitive market should understand that the accumulation of Big Data does not always lead to the expected business benefits. In this context, the conclusion is that a modern company should not set as its goal to process all the available data in order to improve the quality of its economic analysis. It is more significant to use the entire volume of data for segmentation, which allows effective construction of a large number of models for small clusters, solving specific problems of economic analysis based on the application of modern IT systems.
Key words: economic analysis, modern information technologies, Business Intelligence, Big Data, Data Lake, business analytics, Big Data analytics.
Citation: Mitrovic S. (2017) Specifics of the integration of Business Intelligence and Big Data technologies in the processes of economic analysis. Business Informatics, no. 4 (42), pp. 40—46. DOI: 10.17323/1998-0663.2017.4.40.46.
Introduction
In contrast to the Russian market, the foreign market of modern information technologies, in particular, of Business Intelligence (BI) systems and Big Data used to analyze economic information, has a longer history and is developing more rapidly. According to an IDC forecast (March 2017), worldwide profits from the Big Data and Business Analytics market will reach $150.8 billion by the end of 2017, which is 12.4% more than the previous year. It is expected that commercial purchases of the hardware, software, and services related to Business Analytics of Big Data will show an average Compound Annual Growth Rate (CAGR1) of 11.9% until 2020, when the profits will exceed $210 billion [1].
The largest volume of investments into Big Data business analytics technologies is expected in 2017 in the fields of banking, continuous production, federal (central) governmental authorities, as well as in the field of professional services. It is predicted that all these industries combined will spend a total of $72.4 billion by the end of 2017 for the development of Big Data tools. These industries will remain the largest investors into Big Data in the year 2020 as well, when their investments will total $101.5 billion. The highest growth rate of expenditures will be observed in the banking industry (CAGR of 13.3%), health services, insurance, operations with securities, investment management and telecommunications (each with CAGR of 12.8%) [2].
1. Application of Business Intelligence systems and Big Data for the improvement of economic analysis in Russia and worldwide
According to Gartner Research, Big Data business analytics makes it possible to control in-house information in modern banking struc-
tures better and to expose signs of fraud more promptly than was possible before, thereby ensuring financial safety. As follows from this study, economic analysis of Big Data helps us to "solve almost all key banking problems more efficiently, including client acquisition, improvement of services, clients' credit score evaluation, and so forth. Moreover, these technologies help banks to comply with the regulations issued by the authorities by increasing the speed and quality of report preparation, making the economic analysis of data deeper and broader, helping to prevent fraud and money laundering" [3].
BDA+BI technologies are mostly used in the banking industry to analyze client databases, including the clients' behavior, preferences and expectations. For example, PNC Bank (USA) performs a behavioral analysis of their clients' activities on different websites (primarily their online accounts in social networks), processes the information about the purchases made by clients and about their lifestyle, and uses these data to offer flexible interest rates. Another example is Commonwealth Bank of Australia (CBA), which analyses all transactions of their clients and complements this research with analysis of the clients' behavior in social networks. The combination and analysis of these data flows have allowed the bank to significantly reduce its credit default rate [4].
There is also positive experience in Russia. For instance, the Ural Bank for Reconstruction and Development launched a project for Big Data processing using BI technologies several years ago. This project used the client database to create personalized credit offers, investment tools, and other services. Within just one year of using the innovative IT solutions, the bank managed to increase its retail loan portfolio by more than 50% [4].
1 Compound Annual Growth Rate (CAGR) is the average rate of growth of investments over a period greater than one year
Even though insurance companies are among the leaders of the segment and are interested in applying Big Data technologies and Business Intelligence to economic analysis, Bravura Solutions and Financial Services Council agencies conclude that only a small percentage of insurance companies compared to the banking industry has started to move in this direction [5]. According to a poll, 67% of insurance companies believe that they only have limited access to their clients' data. Respondents think that these data are sufficient to personalize the interaction with the clients, but they cannot be considered Big Data and be used to predict the clients' behavior. However, about 30% of foreign insurance companies that participated in the poll already use Big Data and BI technologies for economic analysis to predict their clients' needs and create personalized offers, whereas 23.7% of the companies did not even consider such innovational solutions yet [5].
Active development of the technologies of Big Data analysis in the retail sector — another leader of this segment — goes in different directions. Let us list the possibilities of practical application that can be considered as positive experience for Russian conditions. For instance, in worldwide practice, the brick-and-mortar retail industry uses the economic analysis of Big Data with BI technologies to analyze customers' behavior, plan their trips inside the store, group and place the goods according to customers' preferences, and plan acquisitions. As a result, sales increase. In the online retail sector, the economic analysis of Big Data — even an analysis using the most basic set of features of these technologies — can serve as the basis for the store algorithm itself: the customers receive suggestions based on their previous choices and personal preferences. This information can be obtained, for example, from social networks, prior purchases, and orders from online stores. In all cases, the intelligent economic analysis of Big
Data helps to reduce overheads, increase the customers' loyalty, nurture their interest, and involve broader audiences [6].
Experts forecast that under present conditions, investments into analytical software for intelligent analysis of Big Data will continue to grow, reaching more than $70 billion in 2020. The main part of this amount will be spent on processing consumer surveys, tools of reporting and analysis, and means of data storage management [5]. It is worth noting that the world market demand for technologies and services in the field of Big Data and business analytics is stimulated by the increase in data volumes, development of new technologies, and a cultural shift towards making decisions related to information [2]. According to the data published by RAC, such a fast increase in the use of business analytics technology related to Big Data and the development of the new methods in this field can be explained by the fact that this segment achieved a new level of development compared to Business Intelligence technology. This new level allows the companies to use an extended range of applications (with built-in BI functions) and does not require the specialists responsible for economic analysis to have any additional specialized skills, whereas the similar segment of the CRM systems requires a different methodological approach and level of skills of the personnel [5].
Currently, the technologies of Big Data processing are reaching a higher level of development worldwide compared to the practices observed in Russia. According to the latest version of the "Cycle of technologies maturity" [7] published by Gartner, which estimates the level of methodology maturity and implementation, Big Data have already passed "the peak of excessive expectations, which they had in 2016, and moved into the stage of "disillusioning", i.e. the stage of active prototyping and development of the technology. According to expert estimates,
Big Data will reach their final stage of maturity in 5—10 years. Simultaneously, the analysis of data that combines all approaches to Big Data including Business Intelligence technologies has also reached its peak [7].
2. Prospects of interaction of Business Intelligence systems and Big Data in economic analysis
Even though the most investments into the technology of Big Data business analysis are made in the USA, other world players show higher activity in this area than before. According to IDC's data, 45% of the market of hardware, software, and services related to Big Data business analysis belong to European, Middle Eastern, African, and Asian (not counting Japanese) companies [1].
Demand is growing for analytical tools that can perform a complicated forecasting analysis in the field of Big Data. At the same time, the availability and range of standard solutions for Big Data business analysis are increasing. These solutions are already available from such major companies as Microsoft, IBM, Teradata, Oracle and SAP. This promotes a higher quality of products developed in this market.
78% of the companies around the world are certain that the business analysis of Big Data will be profitable and lead to various advantages within the next 1—3 years. Worldwide, 70% of major companies on the market not only use their Big Data for business analytics, but also purchase data from external sources (it is expected that this percentage will reach 100% by 2019) [1]. It is also expected that more and more companies will profit from their data by selling them or by offering some valuable content based on their data. We should note that for Russia, this is still only a distant perspective. However, there is a new type of offers — "data as a service" (from the Google search engine and others). For many companies worldwide, including Russian
companies — and small businesses as well — this will make it easier to enter the Big Data market in the near future.
According to a forecast by IDC, investments into Big Data business analysis systems convenient for all users will become a must for all companies by 2018 [1]. It is also expected that the budgets for cheaper financial and organizational cloud solutions in the field of Big Data business analysis will grow three times as fast as those for localized solutions.
Based on the opinion of Gartner's experts, we can conclude that the Big Data and BI markets are at the last stage of a long-term transition from systematic reporting (IT approach) to the means of automated analysis (business approach). As a result, the modern platforms of business analytics and Big Data analysis (BI&BDA) will need to comply with new requirements for availability, flexibility and depth of analytic understanding [7]. This promotes the development of simpler, more flexible, and quicker solutions of business analytics, capable of expanding the business opportunities using a deep understanding of various data sources. This is why cloud business analytics and Big Data storage (for example, in the Amazon cloud) is one of the priority directions in the foreign technologies. This opportunity is important for many companies, especially for small businesses, and may be recommended as a promising trial or working version for Russia's companies. The advantage of this version is that the company does not need to spend a lot of money — which often is the reason why business analytics is not affordable for small businesses — buy or lease server space for the Big Data storage, considering that the company does not need this equipment on a permanent basis.
So-called streaming analytics, which makes real-time data analysis possible, is another promising tool for Big Data business analysis. We should note that unlike the Russian market, the world market tends to divide the field
of Big Data into multiple distinct areas (for example, storage, processing, data filtering, real-time on-demand models, data visualization, deep learning, etc.) that solve narrower specific problems of economic analysis. The development of ready-to-use solutions for small and medium businesses which work as independent applications or as SaaS or BDaS models (Big Data as Service) is closely connected with the aforementioned subdivision [8].
When we talk about the connection between the two innovation technologies (Big Data and BI) in economic analysis, we have to mention that they are not developing in parallel and are not subordinate to each other (as some experts believe). Since 2016, the development of the Big Data business analysis technology is closely integrated with the development of the Business Intelligence technology. This creates a two-level model of economic analysis. The first level is the "traditional" Big Data analytics, where huge volumes of data are processed in the offline mode (not in real time). A new, second level provides the possibility of economic analysis of relatively big volumes of data in real time thanks to the in-memory technologies which are included today in most BI solutions. For today's Russian business, this level opens up new possibilities as a consequence of economic analysis of the data accumulated by a company. This is connected, in particular, with the appearance and application of such phenomenon as "data lakes," which are the next evolutionary step to improve the business analytic processes "on the fly" and so influence events as they occur in real time. Let us briefly characterize this innovative phenomenon from the point of view of its application to economic analysis and give several methodological recommendations in this field for modern Russian companies.
A data lake is a repository that can store massive volumes of raw data. These data should be connected with the company's economic ana-
lytical system to perform economic analysis of the company's activities. Data lakes allow one to analyze various incomparable data in their original format. This is a huge advantage for today's organizations since no big expenses are necessary for storage and processing, and the data from the "lake" are accessible for business analytics purposes to any user within the company at any time [9].
Economic data are collected and stored in the "lake" because it is unknown beforehand how they could be used for business development. However, the very popular opinion shared by many organizations that "lakes" could replace traditional data storage facilities is erroneous. Lakes should provide organizations with new analytic possibilities and optimize the expenses for data storage and processing during economic analysis. In this context, many modern companies in the field of Big Data economic analysis face the question of whether their organizational activities and financial expenses for the creation of data lakes will be justified, especially if the company either already has or is creating a data storage for the BI technologies integration to process Big Data. According to Teradata researchers' estimates [9], data lakes can be used for more affordable data storage, and these technologies, when integrated, exert a stronger synergistic effect on economic analysis. The main distinction from the traditional storage is that a "lake" can grow almost indefinitely, keeping the expenses low, and can provide instant access to any data for all users. Moreover, while data in a traditional storage are structured, "lakes" provide cheap storage for all types of data, i.e. data from social networks or various devices, including data in audio and video formats [9].
At the same time, companies should be aware that it may be risky to accumulate "lakes" with the aim of improving the results of economic analysis within the company. For example, company employees tend to store any kind of data in the "lake" without any quality con-
trol. Data security and access control are rarely maintained. Data from the "lake" can be effectively used primarily by business analytics specialists, however, they can be accessed by any business users within the company. Lastly, the main danger is that these "lakes" — if they are not sorted and cleaned periodically — could be transformed with time into "swamps", making the data unusable even for the simplest processes of business analytics [10]. We have to note that the concept of Big Data assumes that no company data should be discarded, especially if they can be stored at low cost. Any data should be initially considered to be "smart data". If some data are discarded right away, there will be no way of knowing if they were useful for business purposes. At the same time, we have to keep in mind that it is rather difficult to determine the potential usefulness of such data. Hence, an organization does not necessarily have to create and keep such a "lake" on its own. Outsourcing is already a common practice, and different companies propose appropriate tools — in particular, the "cloud" tools mentioned above.
Conclusions
Analyzing the problem under consideration, we have to conclude that from the methodological point of view, many aspects of the solutions integration and implementation into the processes of economic analysis in the field of BI and Big Data are still not perfectly developed. One of the main problems is to find algorithms and methodological principles as well as factors and conditions that would allow modern companies to perform economic analysis of Big Data using the capabilities of Business Intelligence with maximum effectiveness for business, as well as the advantages that could give the company a competitive edge. As demonstrated by the analysis performed, the methodological aspects of this question are covered in theoretical and technical business literature much less than the aspects of storage, analyti-
cal processing of data, and Big Data management technology.
The main conclusion of this research is that the modern market is very competitive, and companies should understand that it is not necessarily profitable to collect and store Big Data. As with any project aimed at the optimization of business activity, when starting a project for applying Business Intelligence to process Big Data, companies should begin with solving the issue of data storage and system-atization, as well as the pertinence of the data for certain business goals. The company data analysts need to ask themselves the following questions: "what should the ideal results look like?", "what is the measure of success?", and "what types of information are more useful for achieving an ideal result?". When answered, these questions may or may not lead to the conclusion that Big Data storage is necessary. As an alternative, a cloud solution may be chosen for the storage and processing of Big Data to help the company make strategic managerial decisions.
It is natural to draw another important conclusion: despite the concept of BI & Big Data, a modern company should not try to process all available data in the hope of increasing the quality of the results of economic analysis. It should be kept in mind that Big Data economic analysis that uses Business Intelligence and other modern IT tools is most effective when it is used to build predictive models. The precision of such models depends on the quality of the data sample. Hence, the real challenge of Big Data integrated with BI in economic analysis is not to utilize all available data to create predictive models, as this will not increase the models' accuracy. The more important thing is to use all available data to segment and cluster the data. This will provide an opportunity to effectively create many models for small clusters while solving practical problems of economic analysis based on modern information technologies. ■
References
1. Tadviser (2017) Bol'shie dannye (BigData): mirovoy rynok [Big Data: world market]. Available at: http:// www.tadviser.ru/index.php/CraTbfl:Eo.romHe_flaHHbie_(Big_Data)_MHpoBOH_pbfflOK (accessed 14 August 2017) (in Russian).
2. IDC (2015) Double-digit growth forecast for the worldwide Big Data and business analytics market through 2020 led by banking and manufacturing investments, according to IDC. Available at: http://www.idc.com/ getdoc.jsp?containerId=prUS41826116 (accessed 14 July 2017).
3. CNews (2014) Infografika: Chto mogut bol'shie dannye v bankakh?[Infographics: What Bid Data can do in banks?]. Available at: http://www.cnews.ru/articles/infografika_chto_mogut_bolshie_dannye (accessed 14 July 2017) (in Russian).
4. CNews (2014) Kak krupneyshie banki ispol'zuyut bol'shie dannye [How the largest banks use Big Data]. Available at: http://www.cnews.ru/articles/kak_krupnejshie_banki_ispolzuyut_bolshie (accessed 14 July 2017) (in Russian).
5. Tadviser (2017) Teoriya ipraktika Bol'shikh dannykh v otraslyakh [Theory and practice of Big Data in industries]. Available at: http://wwwtadviser.ru/index.php/OTaTbfl:TeopHfl_H_npaKTHKa_Eo.bmHX_ flaHHbix_B_OTpac.HX#cite_note-4 (accessed 18 August 2017) (in Russian).
6. CNews (2014) Infografika: Kak roznitsa ispol'zuet bol'shie dannye [Infographics: How retailers use Big Data]. Available at: http://www.cnews.ru/articles/infografika_kak_roznitsa_ispolzuet (accessed 13 July 2017) (in Russian).
7. Gartner (2016) Gartner hype cycles 2016: Major trends and emerging technologies. Available at: https:// www.gartner.com/webinar/3377422 (accessed 18 August 2017).
8. CNews (2016) Biznes-analitika i bol'shie dannye v Rossii 2016 [Business analytics and Big Data in Russia 2016]. Available at: http://www.cnews.ru/reviews/bi_bigdata_2016 (accessed 14 August 2017)
(in Russian).
9. Teradata (2017) Insights and tools from Big Data and analytics innovators. Available at: http://bigdata. teradata.com/ (accessed 16 August 2017).
10. CNews (2016) Kak razvernut'«ozero dannykh»: 5sovetov [How to rollout a data lake: 5 recommendations]. Available at: http://www.cnews.ru/articles/kak_razvernut_ozero_dannyh_5_sovetov (accessed 16 August 2017) (in Russian).
