CC BY
34
УДК 331.103.6 + 001.895
THE EFFECTIVENESS OF CROWDSOURCING IN AN
ORGANIZATION (EXAMPLE OF JSC "SBERBANK OF RUSSIA")
R. A. Dolzhenko, A. Yurkova Altai State University (Barnaul, Russia)
The major objective of economic reforms in Russia is to create innovative market economy. Innovative way of any business is based primarily on the intellectual and creative potential of employees, and whole society. Open this potential allows a crowdsourcing; it is a new technology for the domestic market.
This article discusses the possibility of evaluating the effectiveness of using crowdsourcing in an organization. The essence of crowdsourcing is studied. Crowdsourcing classifications are defined. A scheme of interaction between participants crowdsourcing project in the organization is highlighted, key roles of participants are described. Components that make up the effect of using this technology in company are highlighted. The performance and effectiveness of crowdsourcing activities are compared. Indicators to assess the effectiveness of crowdsourcing in the organization are highlighted. Identify the most effective ways to improve the quality of this technology.
The article analyzes the efficiency of using the crowdsourcing on example of JSC "Sberbank of Russia". The costs and benefits of using this technology in the bank in 2012 are compared. The effectiveness of two crowdsourcing projects to finalize documents in bank is studied. It is shown that the effectiveness of crowdsourcing depends largely on engagement participants in this process.
Keywords: crowdsourcing, crowdsourcing platform, teamwork with the ideas, performance of crowdsourcing, effectiveness of crowdsourcing.
ЭФФЕКТИВНОСТЬ КРАУДСОРСИНГА В ОРГАНИЗАЦИИ (НА ПРИМЕРЕ ПАО «СБЕРБАНК РОССИИ»)
Р. А. Долженко, А. Юркова Алтайский государственный университет (Барнаул, Россия)
Основной целью экономических реформ в России является создание инновационной рыночной экономики. Инновационный путь любого предприятия основан, прежде всего, на интеллектуальном и творческом потенциале сотрудников и общества в целом. Открыть этот потенциал позволяет краудсорсинг; это новая технология для внутреннего рынка.
Обсуждается возможность оценки эффективности использования краудсорсинга в организации, рассматривается суть краудсорсинга, приведены его классификации. Также обозначена схема взаимодействия участников проекта краудсорсинга в организации, описываются ключевые роли участников, выделяются компоненты, влияющие на использование этой технологии в компании, сравниваются производительность и эффективность деятельности краудсорсинга. Выделяются индикаторы для оценки эффективности краудсорсинга в организации. Определены наиболее эффективные способы повышения качества этой технологии.
Анализируется эффективность использования краудсорсинга на примере ПАО «Сбербанк России». Сравниваются затраты и преимущества использования этой технологии в банке в 2012 году. Изучается эффективность двух проектов краудсорсинга для доработки документов в банке. Показано, что эффективность краудсорсинга во многом зависит от участников участия в этом процессе.
Ключевые слова: краудсорсинг, краудсорсинговая платформа, совместная работа с идеями, производительность краудсорсинга, эффективность краудсорсинга.
Introduction
Currently, all over the world, there is an increasing interest in possibilities of collective network activity, enlargement of expert community, inclusion of new participants having their own view of the problem to be solved. One of such possibilities is a crowdsourcing, an instrument that was first described by J. Howe (J. Howe, 2006). However, many companies refuse to use it because they can't evaluate the economic effectiveness of crowdsourcing.
But the key advantage of crowdsourcing is related to the economic effect that it can bring to the company. The possibilities of its use are not limited to the participants' creative activities on an organizational problem-solving. Accordingly, it is necessary that the usage of crowdsourcing is premeditated, calculated and effective for all participants of economic relations.
Only one work is dedicated to the issue of evaluating the effectiveness of using crowdsourcing in its direct form (Huberman, 2009). The author attempted to consider the effectiveness of crowdsourcing in an organization empirically with the help of mathematical calculations. A number of articles consider the specificity of data analysis of collaborative crowdsourcing platforms modeling the activity of crowdsourcing participants (Horton, 2010; Ignatov, 2014) but such approaches for evaluating the technology effectiveness, despite their methodological significance do not have any practical effect for the organization that considers the possibility of its use in practice. Our work is essentially a study evaluating the effectiveness of crowdsourcing in an organization.
1. The essence of crowdsourcing and the main issues of its use in an organization
First of all, it is necessary to define the essence of crowdsourcing. From the point of view of organization, crowdsourcing can be defined as the use of intellect and experience of a large number of customers, employees, interested public to find new ideas for improving products, processes, services and/or examination of important decisions and documents of the organization.
A crowdsourcing project, in turn, is a project aimed at solving the problems of different complexity as a result of collective intellectual activity of employees, partners, customers and external experts.
The customer of crowdsourcing gets an opportunity to use the concentrated potential of mass community of people united by a common idea in the framework of using this technology. Taking into account all the advantages of crowdsourcing highlighted by various authors (Raykar, 2010; Marjanovic, 2012); its use has the following advantages in comparison with other standard instruments for solving intellectual problems for businesses and society:
• Remote teamwork with the ideas by using the Internet. A team of talented people from around the world can be involved in this job;
• Crowdsourcing affords the opportunity to test new ideas, products, services by the end consumers which are its participants;
• Practice shows that crowdsourcing is characterized by high effectiveness due to the quickness, relatively low cost and innovative solutions;
• A community of crowdsourcing participants, selects and improves suggestions by itself;
• Identification of the best experts among the participants by the participants themselves is possible on the basis of rating analysis.
At the same time, crowdsourcing can be used within the organization in simplified form. The staff of the firm may be got involved in the work. In this case it scores the following local advantages for the organization:
• Crowdsourcing affords each employee an opportunity to effect the decisions made and processes conducted;
• It allows defining the optimal solution or reduces the probability of making the wrong decision and therefore loss of time and financial resources because participants' combined knowledge and experience are incomparably higher than any of an expert and working group;
• Internal crowdsourcing greatly speeds up the conversion process because it directly depends on the involvement of employees and customers in the change process.
The following types of Crowdsourcing can be highlighted with regard to the use of crowdsourcing for the needs of the organization:
1) Depending on the content and timing, the following types of crowdsourcing are defined (Table 1).
Table 1
Crowdsourcing classification depending on the content and timing of the projects
Permanent (open for more than 2-3 months) Temporary (open for less than 2-3 months)
Suggestions gathering to improve key problems of the customer Crowdsourcing of the internal do-cumnts Contests (logo, name etc.) Strategic projects for the organization
Source: author's work
2) Depending on the audience: External crowdsourcing is conducted with the participation of employees but also customers, partners and external community in the Internet.
Internal crowdsourcing — only employees of the organization can participate.
3) Depending on the limitation of participation:
Mass crowdsourcing — a maximum number of participants can be invited in the project, their membership is not personified.
Opened crowdsourcing — a certain target audience is actively invited in the project but the access to other participants is not limited.
Closed crowdsourcing — a limited number of participants can be, they are strictly personified and carefully selected according to certain criteria.
To maximize the quality of crowdsourcing, first of all, it is necessary to attract as many people as possible, and secondly, it is important that their skills, abilities, work experience are the most relevant in regard to the issues of the crowdsourcing project. As a rule, the optimal number of participants in the project is several thousand people. Thus, crowdsourcing may "operate" in simple cases (with the involvement of a wide circle specialists) and in cases of necessity to solve complex problems that require specific knowledge and skills (by selecting such experts that have necessary knowledge and skills).
After reviewing the key concepts of crowdsourcing, some aspects of evaluating the effectiveness of the use of this technology should be considered.
2. Some aspects of evaluating the effectiveness of crowdsourcing in an organization
The effectiveness can be understood as "a relative effect, a performance of the process, operation, project defined as the ratio of the effect, result in costs, expenses, caused contributed its preparation" (Raizberg, 1998). Thus effectiveness can be evaluated through a comparison of the obtained costs and results.
Since the two notions "effectiveness" and "performance" are often mixed up, it is necessary to distinguish between them. As the performance is a result of how its objects were achieved, and the effectiveness is described as a characteristic of process activity, the highlight of performance in relation to crowdsourcing is necessary because the quality of this technology depends on the achieved objects and goals by the participants and not on the generation of ideas and their amount.
Graphical comparison of "effectiveness' and "performance" with regard to crowdsourcing is given in Figure 1.
Source: author's work
Figure 1. Effectiveness and performance of crowdsourcing using
Thus, if the results of the specific crowdsourcing project are assessed from the point of view of the achieved goals and objects by the participants — it is advisable to use the notion — "performance". In case of evaluating the quality of the instrument itself in comparison with possible costs and results, is necessary to use the notion — "effectiveness". In our work we will use the effectiveness of crowdsourcing.
Four main ways used in crowdsourcing can be determined: knowledge, creativity, voting, and funding. The most important way determining its effectiveness — the creative potential of the participants which effectiveness reflects on the platform through the quality of suggestions and ideas accepted for implementation by the customer. Full evaluating of the effectiveness of the crowdsourcing platform will evaluate the usage of all four aspects for meeting the customer's goals.
Researcher G. G. Azgaldov believe that any result of the company's activity, "is characterized necessarily and sufficiently by two most common properties — quantity (total amount) and quality (one such unit)" (Azgaldov, 1996). Thus, effectiveness of crowdsourcing can be determined by 2 factors: the number of suggestions (total amount of ideas on the project) and the quantity of suggestions (the number of ideas accepted for implementation).
We believe that it is necessary to use such key constructs of the ratio of final (output) and input variables of effectiveness as resource productivity and resource intensity to assess the effectiveness of crowdsourcing. It is known that:
Resource productivity = 1ntput defines the result
Output
obtained from the unit of input;
Resource intensity =
Intput
denotes the specific
Output
quantity of costs per unit of the achieved result.
Thus the resource productivity and resource intensity of crowdsourcing can be calculated as follows:
Resource productivity of crowdsourcing = (1)
Total
Resource intensity of crowdsourcing =
Totalk Out,
(2)
where:
Outks — ideas approved by the customer for implementation (ideas at output of the project);
Totalks — suggested ideas during the project (total amount of ideas).
Conversion of the index of effective ideas accepted by the customer, to the index of financial result from their implementation raises some difficulties as various projects timely for the organization propose completely different financial content.
Besides, it is quite difficult to evaluate the financial question of crowdsourcing effectiveness first of all because the largest domestic organizations have just started to use this technology and secondly it still remains the know-how, so as a rule, companies that actively use crowdsourcing do not reveal the results that have been achieved on crowdsourcing platforms.
Currently, the possibility of combining such technologies as crowdsourcing and artificial intelligence (machine learning) is being worked at that will maximize the effectiveness of crowdsourcing in future (E. Kamar, 2012) but complicate its assessment.
3. Practice of crowdsoursing using and evaluation of its effectiveness
Let us evaluate the effectiveness of crowdsourcing using by example of JSC "Sberbank of Russia", an organization which is the first one among domestic companies started to use this teamwork technology.
So we need to set the results of crowdsourcing activities, determine the costs and compare them.
Sberbank uses two complete platforms for implementation of crowdsourcing projects in its crowdsourcing activities (Wikivote, Witology).
There is an information in open source that according to Wikivote! data where the crowdsourcing projects of Sberbank were conducted, in 2012 the economic effect of implementation in this financial organization of the best ideas obtained during the crowdsourcing was more than 13 million rubles (Crowdsourcing in JSC "Sberbank of Russia").
Since 2013 the documents dealing with Sberbank activities undergo crowdsourcing evaluation with participation of staff from the whole country. As of 01.06.2014, 21701 of participants have been registered in crowdsourcing of Sberbank. Altogether they have sent 17800 of the authors' ideas, written 26 511 of
comments to the discussed standard acts, marked 226124 of grades for ideas (Ideas of Sberbank).
The second element of the evaluating system of the effectiveness of crowdsourcing is the cost for its implementation.
They consist of the following components:
• Cost for platform (creation and further support or rent);
• Cost for managers' salary (moderators and facilitators) of the crowdsourcing;
• Cost for remuneration of the crowdsourcing participants and winners.
Approximate costs for the full implementation of the crowdsourcing project in the organization are given in Table 2.
Table 2
Costs for project implementation in the framework of internal crowdsourcing
Servics Types of projects of internal crowdsourcing
Type 1 Type 2 Type 3
IT platform (per year) $ 787 ths. $ 35 ths.
Technical and organizational support + methodology $ 150 ths. (per project) $ 35 ths. (per month)
Total $ 3.65 mln. (18 projects per years) $ 450 ths. (number of projects is not limited)
Source: author's work
Cost for the system support including the managers' salary are also given in Table 2 with regard to crowdsourcing projects implemented on the third party platforms of partner-companies.
Salary costs for the specialists responsible for the implementation of crowdsourcing in Sberbank can be neglected because this activity is a part of their standard functionality which includes a range of work within the corporate innovation system of the bank.
The following costs were made for the remuneration of the crowdsourcing winners in Sberbank in 2012 (Table 3).
Table 3
Estimated conditional costs for the remuneration of the crowdsourcing winners in Sberbank in 2012
TOTAL FOR 2012
Projects number Total cost for remuneration, $
Projects organized on the third party technological platform "WITOLOGY" 11 77000
Projects organized on the third party technological platform "WIKIVOTE" 17 68000
Total 28 145000
Source: author's work
Thus the costs for crowdsourcing using in JSC "Sberbank of Russia" could be from $ 450 000 to $ 3650000 (including the cost for project support) depending on the project type + costs for the remuneration of the crowdsourcing winners in the amount of $145000.
According to the bank data, the costs for platforms creation for crowdsourcing amounted to $ 35000. Job cost in the framework of crowdsourcing on the outside platforms provided by partner-companies is approximately $ 35 000 per month that corresponds to our approximate costs under condition of Type 3 chosen by the bank.
The analysis of practical use of crowdsourcing in this organization allowed identification some problems that reduce its effectiveness. Some of them are the following:
1. Ineffective gathering of ideas: a large number of incoming ideas, their low quality. 86000 ideas were registered during crowdsourcing of Sberbank, 20% of them are under consideration.
2. Ineffective selection and approval procedures: 100% of ideas need to be examined, long examination periods, high labor costs, ineffective procedure of "filtration". 52% of ideas were rejected, only 10% of ideas were considered in time in Sberbank among 10400 ideas approved for the implementation.
3. Inefficient use of crowdsourcing platforms: low interest in business, dissatisfaction of employees. 7500 ideas were implemented by Sberbank, 80% of them were under consideration for more than 1 year.
New approaches of selecting and evaluating the ideas are necessary in order to improve qualitatively the working models with ideas in the framework of crowdsourcing:
• Business and/or customers should set a problem for solving;
• Work on solutions should be organized in teams;
• New principles of motivation for the authors of ideas should be well-organized — conversion from participant's motivation to team's motivation;
• Expert professional community should be formed according to types of activities;
• Effective communication should be provided between the participants while searching the solution;
• Selection of ideas should be performed by the community's participants;
• "Filtration" of ideas should be performed by means of rating (with the help of participants' voting);
• Efforts should be concentrated on the best solutions;
• The number of active participants should be maximum.
The practice of crowdsourcing using in Sber-bank shows that the following elements are necessary to build an effective crowdsourcing platform:
the Internet platform, rating system of the participants and their suggestions, "filtration" of the participants before including into the project, "filtration" of the suggestions in order to find spam (criteria of inefficient operation), off-topic discussions, guide for the beginners of crowdsourcing, system for setting up (a) problem(s) for the participants, remuneration system for the project leaders, system of project results reporting, facilitators work on the platform, organized security system and privacy of content, retrieval system of suggestions according to different criteria, etc.
The comparison of the results of a number of crowdsourcing projects of Sberbank leads to the conclusion that the activity of crowdsourcing participants largely depends on the topic of the problem and participants' involvement in this process. The more popular it is among the participants, the greater results will be achieved on the platform. Similar studies of foreign researches showed that there is no correlation between the quality of crowdsourcing and remuneration system for participants (Yuen, 2011). Additional remuneration for those wishing to work within the framework of crowdsourcing leads to an increase in the number of participants, but does not increase the quality of their suggestions.
Having analyzed the practice of using crowdsourc-ing in an organization, it should be noted that while evaluating the effectiveness of crowdsourcing, first of all it is necessary to assess the effectiveness of the proposed during the project ideas that should be implemented with this technology but not the use of this technology in general.
The cost of implementing the crowdsourcing platform and its support is relatively low as well as participants' remuneration suggested the best from the point of view of the customer ideas. If the participants of crowdsourcing are permanent, then it could be considered as a kind of virtual organization unit which assumes standard evaluating methods of the effectiveness of its activities.
Conclusion
The economic effect of crowdsourcing use in an organization expressed in financial performance is difficult to evaluate because there are a great number of possible ideas and limitless creative abilities of crowd-sourcing participants. One person may give a limited number of ideas, crowdsourcing as a mechanism of collective intelligence can provide the customer with an unlimited number of ideas.
The practice of using crowdsourcing shows that it has great potential. It can be stated to some degree that in the potential, the form of intellectual activity that can be outsourced to organizations, may also be transmitted to crowdsourcing. It's not time to realize this ability yet.
REFERENCES
1. Azgaldov G. (1996). On account of quality category when calculating the comparative effectiveness of objects. Economics and Mathematical Methods, 32 (3), 66-84.
2. Crowdsourcing in JSC "Sberbank of Russia". (2013). Retrieved from. URL: http://www.slideshare.net/ wikivote/ss-19546252
3. Endovitskii D. A., Lyubushin I. E., Babicheva B. C. (2013). Resource economic analysis: theory, methodology, practice // Economic analysis: theory and practice, 2013. № 38, P. 2-8.
4. Gafforov E. B., Merkushova N. I. (2013). Analysis of use crowdsourcing in innovative business models // Economic analysis: theory and practice. 2013. № 40. P. 2-11.
5. Horton J. J., & Chilton L. B. (2010). The labor economics of paid crowdsourcing. In Proceedings of the 11th ACM conference on Electronic commerce (pp. 209-218). ACM.
6. Howe J. (2006). The rise of crowdsourcing. Wired magazine, 14 (6), 1-4.
7. Huberman B. A., Romero D. M., & Wu F. (2009). Crowdsourcing, attention and productivity. Journal of Information Science.
8. Ideas of Sberbank. (2013). Retrieved from http://idea.sberbank21.ru/
9. Ignatov D. I., Kaminskaya A. Y., Konstantinova N., Konstantinov A. V. (2014). Recommender system for crowdsourcing platform Witology, in: Proceedings of The 2014 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2014, 11-14 August 2014 Warsaw, Poland: IEEE Computer Society Conference Publishing Services (CPS), 2014. P. 327-335.
10. Kamar E., Hacker S., Horvitz E. (2012). Combining Human and Machine Intelligence in Large-scale Crowdsourcing // AAMAS "12 Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems — Volume 1, p. 467-474
11. Marjanovic S., Fry C., & Chataway J. (2012). Crowdsourcing based business models: In search of evidence for innovation 2.0. Science and Public Policy, scs009
12. Raizberg, B. (Ed.). (1998). Modern Dictionary of Economics. Moscow: INFRA-M.
13. Raykar V. C., Yu S., Zhao L. H., Valadez G. H., Florin C., Bogoni L., & Moy L. (2010). Learning from crowds. The Journal of Machine Learning Research, 11, 1297-1322.
14. Yuen M. C., King I., & Leung K. S. (2011, October). A survey of crowdsourcing systems. In Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE Third Inernational Conference on Social Computing (SocialCom), 2011 IEEE Third International Conference on (p. 766-773). IEEE.
