Научная статья на тему 'Challenges and critical success factors for e-procurement adoption in Ethiopia'

Challenges and critical success factors for e-procurement adoption in Ethiopia Текст научной статьи по специальности «Экономика и бизнес»

CC BY
2208
487
i Надоели баннеры? Вы всегда можете отключить рекламу.
Ключевые слова
Challenges / e-procurement / procurement / Ethiopia

Аннотация научной статьи по экономике и бизнесу, автор научной работы — Shiferaw-Mitiku-Tebeka, Addis Yessuf

To ensure public procurement process fair, transparent, efficient and ethical, the Ethiopian Government implemented e-procurement adoption in selected pilot public organizations. This study intended to identify the major perceived challenges and critical success factors for e-procurement implementation and to ascertain strategies to mitigating the existing perceived challenges for e-procurement implementation in nine federal level organizations IN Ethiopia, selected as pilot scaled-down e-procurement implementation sites. Interview and structured questionnaire were used to collect primary data from purposively selected top managements and experts with directorate position from Finance, Procurement and ICT department from a total of 27and 54 employees, respectively and carried out analysis with 89 percent response rate. The result revealed that, man-power retention, inconsistent and disruptive infrastructure, integration with the legacy system, top management and employees’ commitment and attitude, supplier integration, security fear, weak and inconsistent support, poor monitoring and evaluation practices were found as the major challenges. While, training given to employees on how to use e-procurement tools and best procurement practices, the existence of change management programs for users on implementation of e-procurement through effective consultations and the high skill of procurement employees with IT perspective were found to be the major critical success factor.

i Надоели баннеры? Вы всегда можете отключить рекламу.
iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.
i Надоели баннеры? Вы всегда можете отключить рекламу.

Текст научной работы на тему «Challenges and critical success factors for e-procurement adoption in Ethiopia»

DOI 10.18551/rjoas.2019-02.13

CHALLENGES AND CRITICAL SUCCESS FACTORS FOR E-PROCUREMENT

ADOPTION IN ETHIOPIA

Shiferaw-Mitiku-Tebeka, Assistant Professor Addis Yessuf, Lecturer Addis Ababa University, College of Business and Economics, School of Commerce,

Addis Ababa, Ethiopia *E-mail: [email protected]

ABSTRACT

To ensure public procurement process fair, transparent, efficient and ethical, the Ethiopian Government implemented e-procurement adoption in selected pilot public organizations. This study intended to identify the major perceived challenges and critical success factors for e-procurement implementation and to ascertain strategies to mitigating the existing perceived challenges for e-procurement implementation in nine federal level organizations IN Ethiopia, selected as pilot scaled-down e-procurement implementation sites. Interview and structured questionnaire were used to collect primary data from purposively selected top managements and experts with directorate position from Finance, Procurement and ICT department from a total of 27and 54 employees, respectively and carried out analysis with 89 percent response rate. The result revealed that, man-power retention, inconsistent and disruptive infrastructure, integration with the legacy system, top management and employees' commitment and attitude, supplier integration, security fear, weak and inconsistent support, poor monitoring and evaluation practices were found as the major challenges. While, training given to employees on how to use e-procurement tools and best procurement practices, the existence of change management programs for users on implementation of e-procurement through effective consultations and the high skill of procurement employees with IT perspective were found to be the major critical success factor.

KEY WORDS

Challenges, e-procurement, procurement, Ethiopia.

The key competitive priorities for the 21st century in the Supply Chain Management is to embrace the emerging internet-based information and communication technologies such as e-procurement, to transform traditional procurement to continually find solution to reduce costs, increase efficiency, and to reduce the longest lead time across the supply chain actors. Procurement now is seen as a core player in supply chain focus on time to market, product quality based competition, cost efficiency, inventory management, and customer uncertainty (Monczla and Morgan, 2000). The practice of e-procurement is changing the way businesses purchase goods, hence, it make purchasing activities more effective in terms of both time and cost being responsive and ensuring efficiency. Nowadays, most of private and public owned organizations, and manufacturing and services rendering companies procure products and services using electronic data interchange and the internet, the application of e-procurement is inevitable. However, the implementation of e-procurement is not a guarantee for success for every organization since; some of the companies all over the world implement e-Procurement and succeed while some companies failed.

According to the FDRE Public Procurement and Property Administration Agency (2014), expenditure on public procurement takes the lion share from the annual Government budget; 64 percent of the annual budget, 14 percent of the GDP of Ethiopia. Considering its share in the economy, the implementation of e-procurement in Ethiopia seeks to support the country's economic development by ensuring efficiency and effectiveness in the execution of public procurement, to serve as a policy instrument in terms of encouraging and enhancing the capacity of local producers small and micro-enterprises, suppliers, contractors and consultants, to build trust between government and donors and increase the flow of official

development assistance through bilateral and multilateral channels, to create a conducive environment for various partnership arrangements between the government and the private sector, to reduce trade and non trade barriers within the framework of enhancing the competition policy and to protect the environment by encouraging the purchase of environmental friendly goods only.

In order to achieve transparency and modernization in public procurement, the Ethiopian Federal Government Procurement and Property Administration Proclamation No. 649/2009 and article 31 emphasizes on Electronic Procurement and it gives mandate to the Ministry to authorize the use of electronic means as a method of procurement. In order to implement this:1) the Agency shall conduct a study and submit proposal on a system of conducting procurement by means of electronic exchange of information which is appropriate to the level of development of the country; 2) the Agency shall ensure that public bodies, suppliers and supervising entities develop the capacity required to implement the system; 3) upon due consideration of the proposed system of effecting procurement by means of electronic exchange of information and where he is satisfied that the overall system and capacity of public bodies and suppliers allows the carrying out of procurement through electronic exchange of information, the Minister may authorize the implementation of the electronic system in all or certain public procurement proceedings by establishing the appropriate framework for the operation of the proposed electronic system (Federal Negarit Gazeta September, 2009).

In this regard, considering its vitality, the Ethiopian government took the initiative to implement e-procurement at selected pilot government organizations. However, there are limited empirical studies in the literature on the challenges of implementing e-procurement in the country. Such a study will help the country to develop as well as refine the e-procurement policies, strategies, and procedures to implement e-procurement at national level, in all sectors of the economy. Hence, this study sought to ascertain the major perceived challenges and critical success factors exhibited during the pilot stage of e-procurement implementation in Ethiopia.

Problem statement. The benefits of e-procurement implementation in the public sector where much of the GDP is spent will only be gained if the challenges, that will hinder the pre-implementation and post- implementations milestones, are addressed well.

M.V. Jooste and C. de W. van Schoor (2003) put forward that countries implemented e-procurement like South Africa has very unique problems in implementation of e-procurement, which attributed to limited and monopolized supply base, limited bandwidth, social responsibilities, and e-procurement affordability. In this regard, the attempt by the Ethiopian government to implement e-procurement solutions in a few governmental organizations is met with various challenges such as failing to retain properly trained procurement officers, e-procurement solutions that are chosen to best fit the procurement process in Ethiopia, less supplier participation, limited top management supports, lack of employee and top management commitment, poor infrastructure and suppliers integration are few among others based on the informal pilot preliminary interview made with a total of fifteen employees, who attended Integrated Financial Management Information Systems implementation training from the Addis Ababa University. This calls for the intervention of research and policy options for proper e-procurement implementation in public organizations at national level. Hence, the challenges do come in various forms and categories. Accordingly, identifying the challenges and critical success factors will help the implementation of e-procurement solutions to be successful and ultimately to reap the benefits of those solutions at large.

Research questions. This study intended to answer the following specific research questions:

• What are the perceived critical success factors for effective implementation of e-procurement in Ethiopia?

• What are the major perceived challenges hindering the proper e-procurement implementation in Ethiopia?

• To what extent the selected pilot organizations have adopted e-procurement and realized its benefits?

The general objective of this study is identifying the major critical success factors and challenges exhibited in the pilot implementation of e-procurement solution in Ethiopia.

Specific objectives:

• To identify the perceived critical success factors for the implementation of e-procurement in Ethiopia;

• To identify the major perceived challenges for proper e-procurement implementation in Ethiopia;

• To ascertain the extent to which the selected pilot organizations have adopted e-procurement and realized its benefits.

E-procurement implementation. A pilot implementation of a software solution named Integrated Financial Management Information Systems (IFMIS) in seven to ten governmental organizations in an attempt to test the integration of finance, procurement and inventory processes.

LITERATURE REVIEW

The ever increasing trend of e-business practices in the late 1990's led to the development of new opportunities related to procurement like e-procurement; spend management, outsourcing and joint product design (Lancioni, Smith, and Oliva, 2000). The advent of the internet as a business systems platform has been a channel for major changes in the operation; the way organizations and governments operate and change the status of organizational procurement, which attributed to the adoption of Information Technologies. This section covers the review of related literatures mainly empirical literature and conceptual framework of the study where the study is based on and the identified literature gap, for which the present study will contribute to the existing literature.

There are studies undertaken by different scholars related to identifying the critical success factors for e-procurement adoption from different countries perspective. However, to the knowledge of the researchers, no prior study has been conducted from Ethiopia perspective as adoption of e-procurement solution is at the pilot stage of its implementation in Ethiopia. Hence, major critical success factors were drawn from the exiting body of literature.

Empirical reviews on critical success factors for e-procurement adoption. Deployment of technology alone does not ensure the success of technological intervention and a public sector e-Procurement initiative depends on users and buyers making use of the new process and system. The solution must attract end users to view e-Procurement as the preferred means by which to purchase goods and services (KPMG, 2001). The success of the e-procurement solution/project also depends on communication to the users (Birks, Bond, & Radford, 2001). According to the CGEC (2002), the two major obstacles to increasing support among users are their level of technological awareness and acceptance, and their willingness to change long-established internal business processes. As the implementation process develops, periodic user satisfaction surveys may identify the possible need for additional training (OSD, 2001).

With an empirical research conducted by Gunasekaran and Ngai (2008) indicated that, there are several factors that are critical to the success for any e-procurement initiative either in the public sector, which need to be addressed for successful implementation of e-procurement in an organization Viz., the users acceptance of new systems of information, the quality of information that would be obtained, trust in the new systems, perceived risks, skills that staffs have and the training they would require, support from the top management at the organization, benefits that would be obtained from implementing the new system and continuous assessment of the benefits, benchmarking and compliance to best practices and factual selection of e-procurement solution.

Similarly, Mose (2012) conducted a study on the impact of e-procurement on the operations of Kenya Commercial Bank, noted five critical factors that had the greatest impact

on e-procurement namely., user acceptance of e-procurement systems, reliability of information technology and supplier performance, top management and employees' commitment to success of adoption, monitoring the performance of e-procurement systems and senior management support to e-procurement implementation process. The same study identified that, resistance to change by employees, luck of management support to the new changes, existence of old information technology systems and equipment; and lack of board approval to using e-procurement system were found to be the major challenges for e-procurement adoption. However, the aforementioned critical success factors were sub-categorized and split into different component to determine the critical success factors for implementation of e-procurement system. For example, a study conducted by Paniand Kar (2011) stated that training and capacity building of staff in procurement practices is a critical factor for successful e-procurement implementation. The staff needs to be well trained and equipped so as to be well conversant with the e-procurement systems. Considering that the success of e-procurement is dependent on the users who are part of the implementation process. These two sub-components considered as a critical success factor for e-procurement implementation are regarded as top management and employees' commitment in a study conducted by Mose (2012). Similarly, Stenning and Associates (2003) highlighted the need for transactions between different systems to be exchanged in secure ways with absolute assurances regarding the identities of the buyers and suppliers. In order to encourage buyers and suppliers to engage in e-Procurement, it is critical that both parties have complete confidence and trust in the underlying security infrastructure. While, in a study conducted by Mose (2012), this component was considered as the reliability of Information Technology and infrastructure for e-procurement adoption. This indicates, despite the different views exhibited among and between different scholars in determining the critical success factors for e-procurement adoption, their major concern area found to be similar. Hence, to avoid redundancy of the different variables considered by different researchers, the different critical success factors extracted from the existing literature were merged in to five major variables on the basis of their degree of similarity as described below under the conceptual framework of the study.

Figure 1 - Conceptual framework of the study (Source: Self depicted, and partially adopted from

Mose, 2012)

Based on the empirical reviews made above, the conceptual framework of the present study considered five major critical success factors which were considered in several studies undertaken in different countries viz., top management support and employee commitment, reliability of information technology and infrastructure, reliability of legal factors, reliability of user acceptance (buyer and supplier) performance and the reliability of monitoring and evaluation systems for e-procurement adoption (see fig. 1). However, prior to the adoption of the conceptual framework and distribution of the data collection instrument (questionnaire) content validity was checked with experts with trust area of logistics and supply chain management. Accordingly, the experts' reflections were incorporated in the data collection instrument.

Identified Literature gap. To the knowledge of the researchers, no prior study has been conducted as far as the study target area is concerned; hence, the present study will bridge the existing literature gap to the Ethiopian e-procurement related literature perspective. Additionally, this study will give an insight to policy makers and practitioners in the area to pay due attention to those perceived critical success factors, which impact e-procurement implementation.

MATERIALS AND METHODS OF RESEARCH

Description of the study area. Governmental organizations namely, the FDRE Ministry of Education, FDRE Ministry of Finance and Development, FDRE Ministry of Health, National Planning Commission, FDRE Public Procurement and Disposal Services, FDRE Public Procurement & Property Administration Agency, Ethiopian Road Authority, the Integrated Financial Management Information Systems (IFMIS) project office and the Addis Ababa University were selected for the study, which were selected as pilot e-procurement implementation sites. These are the hotbeds of the implementation where, the success or the failure of the implementation is going to have serious implications on the future of e-procurement in Ethiopia.

Study population, Sample frame and Method of data collection. Since e-procurement brings a platform for experts from different departments, hence, structured questionnaires were used to collect primary data from purposively selected employees in the department of Finance, Procurement and ICT of the selected organizations.

In order to triangulate the findings of the data collected using questionnaire, interviewees were purposively selected from Finance, Procurement and ICT Directorate/ experts those who use the e-procurement system specially, leading, implementing and supporting the IFMIS solution in their respective organizations

Sample size determination technique. A study by Sarah Elsie Baker and Rosalind Edwards [n.d], on quantifying the sample size required for qualitative interviews concluded that, the number of people required to make an adequate sample for a qualitative research project can vary from one to a hundred or more. However, considering the length of time this type of research often takes, the difficulty of gaining entrée to even the most mundane group or setting, the difficulty in transcribing thousands of hours of interviews, and the "publish or perish" world in which we live, the best bet is to consider in the broad range of between a dozen and 60, with 30 being the mean.

Hence, this study is qualitative in nature, hence unstructured interview was used as the main data collection instrument, generally accepted rule of thumb dictates that 5-10 people are appropriate as a sample size for a interviewing. A total of sample size of 27 employees were purposively selected from the nine selected public organizations to undertaking interview with top managements, with directorate position and experts in the functional areas of Finance, Procurement and ICT. Besides, questionnaire was employed using census method, as there has been high employee turnover exhibited on those selected organizations, implemented the e-procurement systems. Hence, primary data were collected using unstructured in-depth-interview and questionnaire to elicit the major challenges they face and they think should be addressed in their e-procurement implementations and critical success factors to be considered at the full implementation stage of e-procurement.

A total of 54 questionnaires were distributed to employees involved in the aforementioned three functional areas (a minimum of two employees were drawn from each functional areas of the nine selected organizations) and 44 valid questionnaires were collected and retained for final analysis with 81.5% response rate. It is possible that, the response rate was affected by the fact that the survey was only relevant to employees using e-procurement solution in the functional areas of procurement, finance and ICT and the selected organizations were experiencing high employee turnover in the selected functional areas.

Method of data analysis. As the research is a qualitative one, non-parametric statistical analysis was used. Challenges exhibited in e-procurement implementation were descriptively analyzed using content analysis approach and simple statistical tools like mean, frequency and percentage; and Critical success factor analysis was made using non- parametric test mainly Kendall's W. Coefficient of Concordance model, to measure association with the N blocks representing N independent judges, each one assigning ranks to the same set of K applicants (Kendall, M. G., and Babington-Smith, B., 1939). Kendall's W measures the extent to which the N judges agree on their rankings of the K respondents. Kendall's W bears a close relationship to Friedman's test; Kendall's W is in fact a Scaled version of Friedman's test statistic:

W= TF / N (K-1) (1)

The scaling ensures that W=1 if there is perfect agreement among the N judges in terms of how they rank the K applicants. On the other hand, if there is perfect disagreement among the N judges w = 0. The fact that the judges don't agree implies that they don't rank the K applicants in the same order. So each applicant will fare well at the hands of some judges and poorly at the hands of others. Under perfect disagreement, each applicant are fare the same overall and will thereby produce an identical value for Rj. This common value of Rj will be R, and as a consequence, W=0. Thus, the inferential statistical test-Kendall's W Test (coefficient of concordance) was fitted to determine if there is any significant difference in the various rankings of variables to be considered in measuring the critical success factors for e-procurement implementation in Ethiopia.

RESULTS AND DISCUSSION

This section presents the demographic profile of the respondents, critical success factors for e-procurement adoption, major roles (benefits) of e-procurement adoption, level of e-procurement adoption and challenges for e-procurement adoption in the selected public organizations in Ethiopia, implemented e-procurement solution called IFMIS.

Table 1 - Reliability Statistics of the model

Items Cronbach's Alpha N of Items Number of cases Excludeda Cases

CSF 1 : Top Management Support and employee Commitment for e-procurement adoption .897 9 44 0

CSF 2: Reliability of Information Technology and infrastructure for e-procurement adoption ..875 9 43 1

CSF 3: Reliability of Legal factors in e-procurement adoption .933 6 44 0

CSF 4: Reliability User Acceptance (Buyer and supplier) performance of e-Procurement adoption .905 7 44 0

CSF 5 : Reliability of Monitoring and Evaluation Systems for e-procurement adoption .906 4 44 0

Extent of Perceived benefits derived from e-procurement adoption .915 11 36 8

Challenges hindering the adoption of e-procurement .956 20 31 13

Level of e-procurement adoption .983 20 32 12

Over all Reliability Statistics of the model .934 86 24 20

a. Lstwise deletion based on all variables in the procedure.

In order to distinguish those variables (questions) which reflect different underlying variables (like CSF for e-procurement adoption, challenges of e-procurement adoption and the level of e-procurement adoption) the Principal Component analysis test was computed and then the overall reliability test was checked using Cronbach's Alpha. Accordingly, the

reliability statistics of the model indicates, Cronbach's Alpha is 0.934, which indicates a high level of internal consistency is secured, with an overall reliability of coefficients for a set of variables of 86 items.

Table 2 - Respondents' Level of education

What is your highest level of education? Frequency Percent

Diploma 4 9.1

n/n Bachelor Degree 28 63.6

Masters Degree and above 12 27.3

Total 44 100.0

Source: Own survey, December 2017.

The findings from Table 3 indicated that more than half of (63.6%) the respondents were Bachelor Degree holders, 27.3% had a Graduate Degree and 9.1% of the respondent obtained diplomas. Therefore, the respondents had attained the necessary education and knowledge to implement e-procurement in the selected public entities in Ethiopia.

A list of five success factors in the implementation of e-procurement in selected public organizations in Ethiopia namely; the FDRE Ministry of Education, FDRE Ministry of Finance and Development, FDRE Ministry of Health, National Planning Commission, FDRE Public Procurement and Disposal Services, FDRE Public Procurement & Property Administration Agency, Ethiopian Road Authority, the Integrated Financial Management Information Systems (IFMIS) project office and the Addis Ababa University were identified and respondents were requested to indicate the extent to which they agreed or disagreed with each factor. A five point Likert scale was used where: 1= strongly disagree, 2= disagree, 3= moderately agree, 4= agree, and 5= strongly agree. From the responses, Rank of Kendall's Coefficient of Concordance model and descriptive measures of central dispersion: mean and frequency were fitted to analyzing the data (Table 2).

Mose (2012) did a research on the impact of e-procurement on the operations of Kenya Commercial Bank (KCB). The study noted five critical factors that had the greatest impact on e-procurement. These five factors are user acceptance of e-procurement systems, reliability of information technology and supplier performance, top management and employees' commitment to success of adoption, monitoring the performance of e-procurement systems and senior management support to e-procurement implementation process.

Accordingly, the results revealed that, among the five critical success factors the reliability of Information Technology and infrastructure was ranked first being the key critical success factors for e-procurement implementation in Ethiopia.

The provision of reliable monitoring and evaluation systems ranked as the second most critical success factor and the reliability of user acceptance (buyer and supplier) for implementation of e-procurement ranked third and the fourth and fifth critical success factors for e-procurement implementation were found to be the top management support and employee commitment; and the availability of reliable legal environment for e-procurement adoption with Kendall's Coefficient of Concordance of 0.035, indicating respondents level of agreement over the 54 items were not consistent in their ranking.

The reliability of Information Technology and infrastructure was ranked as the first most critical success factors with a mean value of 3.08, being a key to implementation of e-procurement solution. The reliability of Information Technology and infrastructure as a key critical success factor attributed to the functional integration of the e-procurement system with other business processes (such as finance and inventory or IFRS, and ERP), the supply of high band width internet connection for e-procurement implementation, the high usability of the system to suppliers having features like friendly interfaces, document uploads, integration with invoice systems, etc), the compatibility of the e-procurement implementation systems with the existing hard ware and software followed by and the availability of security and authentication measures considering the sensitivity of data and legal nature of accountable documents with mean value of 3.48, 3.27, 3.23, 3.16, 3.16 and 3.05, respectively.

Table 3 - Critical success factors for e-procurement adoption

CRITICAL SUCCESS FACTORS Attributes of CSF for e-procurement adoption Strongly Disagree Moderately Agree Strongly Mean Rank of Kendall's w.

disagree agree agree Coefficient of Concordance

There is staff training/capacity building on how to use e-procurement tools and best procurement practices. 5 12 10 6 11 3.14 21.0

There is change management programs for users on implementation of e-procurement through effective consultations 9 9 8 11 7 2.95 18.8

Factor 1: Top Management Support and employee Commitment for e-procurement adoption (Mean of Mean=2.80) There is provision of adequate resources necessary for e-procurement implementation 10 12 6 13 3 2.70 18.4

There is top management leadership to e-procurement implementation 6 15 10 9 4 2.77 17.9

Employees' commitment to the implementation of e-procurement is high 11 6 14 9 4 2.75 17.5

There is risk management practices (contingency approach to practice e-procurement) 7 9 15 12 1 2.80 17.6

Senior managers organizes a training on e-procurement adoption 10 13 11 8 2 2.52 16.1

Top management gives due care about e-procurement and willing to invest on e-procurement adoption 11 9 9 12 3 2.70 16.1

The skill of procurement employees with IT perspective is high 10 8 8 14 4 2.86 18.8

The security and authentication measures are in place, considering the sensitivity of data and legal nature of accountable documents. 7 7 11 15 4 3.05 19.5

Availability of a reliable internet service to facilitate the e-procurement implementation process is not a problem 14 7 6 12 5 2.70 17.5

Availability of a reliable ICT infrastructure within my organization to facilitate the e-procurement implementation process is not a problem 8 11 8 14 3 2.84 18.7

Factor 2: Reliability of Information Technology and infrastructure for e-procurement adoption (Mean of Mean=3.08) e-procurement implementation systems are compatible with existing hard ware and software 5 5 15 16 3 3.16 21.0

The functional integration of the e-procurement system with other business processes such as finance and inventory or IFRS, ERP is strong 5 3 7 24 5 3.48 22.5

Usability of the system to suppliers is high having features like friendly interfaces, document uploads, integration with invoice systems, etc) 8 6 7 19 4 3.16 19.7

There is no inconsistency of internet connection and power outages hindering the successful adoption of e-procurement 13 5 7 14 5 2.84 16.9

The e-procurement implementation demands high band width on the internet connection 7 2 14 14 7 3.27 20.4

Reliable information is available on the website in the adoption of e-procurement 7 7 8 13 9 3.23 19.8

Factor 3: Reliability of Legal factors in e-procurement adoption (Mean of Mean=2.61) There is documented e-procurement policy manual within the organization to guide the implementation process 16 7 6 12 3 2.52 16.2

There is legal and administrative procedure for e-procurement adoption 15 5 11 10 3 2.57 15.9

There is e-procurement operations instruction 13 9 8 12 2 2.57 16.0

There is legal framework that incentivizes suppliers to use e-procurement 16 7 8 11 2 2.45 15.0

iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.

There is changing manual procedures, policy, and legal framework in favor of e-procurement 12 8 6 12 6 2.82 18.0

There is up-to-dated procurement information in e-procurement implementation 10 10 10 10 4 2.73 16.6

Factor 4: Reliability user acceptance (Buyer and Supplier) performance of e-Procurement adoption (Mean of Mean=2.83) Training is offered to suppliers on e-procurement adoption 13 7 8 13 3 2.68 16.2

The system involves most of stakeholders mainly buyers & suppliers in e-procurement adoption 12 8 9 12 3 2.68 16.2

Employees are willing to use e-procurement system 6 6 15 11 6 3.11 19.3

Legal traders (suppliers)are willing to use e-procurement 6 8 20 8 2 2.82 17.3

New processes are designed for automation of e-procurement 7 9 16 10 2 2.80 17.8

Both buyers and suppliers are willing and ready to make e-procurement adoption succeed 6 9 16 10 3 2.89 18.1

Both buyers and suppliers have developed trust on the system (e-procurement) 10 6 13 10 5 2.86 17.5

Factor 5 : Reliability of Monitoring and Evaluation Systems for e-procurement adoption (Mean of Mean=2.90) Existence of a project management team to spearhead, monitor and evaluate progress of e-procurement implementation is in place 9 10 13 9 3 2.70 16.4

There is practices of improving the performance of e-procurement adoption through continuous evaluation and feed back 8 8 16 7 5 2.84 17.3

Compliance with rules and regulations is ensured 5 10 15 10 4 2.95 18.4

Observation of procurement guidelines is being practiced and monitored 6 6 15 12 5 3.09 19.6

Kendall's Wa =0.035, Chi-Square =51.601, df = 34, Asymp. Sig.= 0.027 N=44, a = Kendall's Coefficient of Concordance. Source: Own Survey, December 2017.

The finding proved that, the implementation of e-procurement solution is dependent on the features of the e-procurement solution itself to accommodate the users' (buyers and sellers) service requirements and the access to high bandwidth internet service in a consistent basis and the security and authentication measures that can guarantee safe transactions of a user.

The finding is consistent with a study conducted by Naveed Sarwar (2017) in Assessing e-procurement success factors in telecommunication organizations in Pakistan affirmed that, Top management support, Transparency of a system and availability of ICT Infrastructure were found as the top three critical success factors for e-procurement adoption.

The respondents also moderately agreed that, provision of reliable of monitoring and evaluation systems necessary for implementation of e procurement ranked as the second key critical success factors to the implementation of e-procurement with a mean of 2.90. The result was mainly attributed to the compliance of e-procurement implementation with rules and regulations with a mean value of 3.09 followed by the availability of the practices of observation of procurement guidelines with a mean value of 2.95.

The reliability of user acceptance (buyer and supplier) for implementation of e-procurement ranked third with a mean of 2.83, considering that buyer and supplier are major stakeholders in the procurement process to successfully implement e-procurement. The reliability of user acceptance is attributed to willingness of employees to use e-procurement system, willingness of legal traders (suppliers) to use e-procurement, willingness and readiness of both buyers and suppliers to make e-procurement adoption succeed and the trust developed by both buyers and suppliers on the system (e-procurement) with a mean value of 3.11, 2.82, 2.89 and 2.86, respectively.

Top management support and employee commitment for e-procurement adoption was found to be the fourth critical success factors with a mean of 2.80. Among the major variables attributed for top management support and employee commitment are the availability of staff training on how to use e-procurement tools and best procurement practices with mean of 3.14, followed by the availability of change management programs for users on implementation of e-procurement through effective consultations and the high skill of procurement employees with IT perspective with a mean of 2.95 and 2.86, respectively.

A study conducted by Pani and Kar (2011) claimed that, training and capacity building of staff in procurement practices is a critical factor for successful e-procurement implementation. The staffs need to be well trained and equipped so as to be well conversant with the e-procurement systems. Hence, considering that the success of e-procurement is dependent on the users who are part of the implementation process.

Similarly, the finding is consistent with a study conducted by Kevin George Obat(2016) on critical success factors in the implementation of e-procurement in public entities in Kenya found that, staff training on use of e-procurement tools and best procurement practices, the provision of adequate resources necessary for implementation of e-procurement systems and top management leadership and commitment to e-procurement implementation were the top three key critical success factor for e-procurement adoption

The availability reliable legal factors in e-procurement adoption ranked as fifth critical success factor with a mean of 2.61, being the least among the given factors determining the successful implementation of e-procurement in Ethiopia. The availability of changing manual procedures, policy, and legal framework in favor of e-procurement implementation, the up-to-datedness of procurement information during the e-procurement implementation, the availability of legal and administrative procedure for e-procurement adoption ; and the availability of e-procurement operations instruction were found to be the major sub-factors attributed to the availability of reliable legal factors in e-procurement adoption as the fifth least critical success factor with mean of 2.82, 2.73, 2.57, and 2.57, respectively.

Respondents were asked to rank the extent to which e-procurement adoption benefited the public organization where they are working (Adoption of e-procurement benefited to a great extent "5: to a no extent "1").

Table 4 - Benefits of e-procurement adoption

Major roles (benefits) derived from the adoption of e-procurement solution N Missing To no extent To a little extent Moderate extent Great extent Very great extent Mean Kendals Mean Rank, N=36

To what extent does the e-procurement implementation bring transparency between buyers and suppliers? 44 0 3 8 9 9 15 3.57 5.56

To what extent does the e-procurement implementation reduce error in procurement transaction? 44 0 2 3 11 18 10 3.70 5.83

To what extent does the e-procurement implementation increase transaction speed? 44 0 2 1 16 11 14 3.77 6.68

To what extent does the e-procurement implementation bring standardized buying? 42 2 1 5 4 17 15 3.95 6.54

To what extent does the e-procurement implementation increase productivity? 40 4 1 3 13 9 14 3.80 6.58

To what extent does the e-procurement implementation eliminate paper work? 43 1 7 3 8 12 13 3.49 5.54

To what extent does the e-procurement implementation bring process efficiencies? 42 2 2 6 10 12 12 3.62 6.08

To what extent does the e-procurement implementation ease the process like common process, standardization, visibility and efficient invoice settlement 41 3 1 10 5 12 13 3.63 6.32

To what extent does the e-procurement implementation improve the roles of supply chain actors like knowledge sharing, more value-added activity, and productivity 43 1 1 8 12 11 11 3.53 5.83

To what extent does the e-procurement implementation improve supplier management and selection, reduce supplier number and improve supplier integration? 43 1 5 7 11 10 10 3.30 5.33

To what extent does the e-procurement implementation improve control and compliances 42 2 1 11 5 17 8 3.55 5.69

Source: Own Survey, December 2017.

The result revealed that, public organizations implemented e-procurement solution affirmed that the adoption of e-procurement solution led them to exhibit transparency between buyers and suppliers as the major benefit followed by reduce error in procurement transaction, increase transaction speed, bring standardized buying, increase productivity, eliminate paper work, bring process efficiencies, ease the process like common process, standardization, visibility and efficient invoice settlement, improve the roles of supply chain actors (like knowledge sharing, more value-added activity, and productivity), improve supplier management and selection, reduce supplier number and improve supplier integration and improve control and compliances to the level of between moderate to a great extent. However, the existing system doesn't let supplier to participate.

This indicates that the adoption of e-procurement served major expected benefits that can be derived out of the e-procurement solution. A good procurement system should be guided by the principles of good governance and values such as transparency, accountability and integrity (Wittig, 2003). More to that, a study conducted by Gunasekaran & Ngai ( 2008), affirmed that the quality of information that would obtained, trust in the new systems, perceived risks, skills that staff have and the training they would require, support from the top management at the organization, benefits that would be obtained from implementing the new system and continuous assessment of the benefits, benchmarking and compliance to best practices and factual selection of e-procurement solution

Respondents were asked to indicate the extent to which their organization has adopted those key activities in the procurement function through e-procurement system .A five point Likert scale was used to measure the level of e-procurement adoption where: 1= Not available; 2 = adopted to small extent; 3 = adapted to a moderate extent; 4= adapted to great extent 5= adapted to very great extent. From the responses, a descriptive measure of central dispersion mean was fitted to analyzing the data.

The finding revealed that, the level of e-procurement adoption for most of the procurement processes expected to be handled between small to a moderate extent. The

level of adoption of e-procurement practices mainly advertising tenders online was calculated a mean value of 2.66, registering suppliers and manage suppliers information accounted a mean of 2.62, suppliers are able to access the bid information with mean of 2.59, existence of functioning website ranked below moderate level of adaption with mean value of 2.53 followed by call for proposals done through company website, the system is capable of handling bid award management, short listing of suppliers online, Company staffs make requisitions online the system is capable of integrating with the internal and external stakeholders' system with a mean value of 2.45, 2.38, 2.32, 2.32, and 2.30, respectively. While, the level of e-procurement adoption specifically, bid opening and closing features and online submission of bids by supplier were not available on the existing e-procurement adoption as the current implementation doesn't register suppliers online.

Table 5 - Level of e-procurement adoption

E-Procurement adoption levels (Using a five point Likert scale: 1= Not available; 2 = adopted to small extent; 3 = adopted to a moderate extent; 4= adopted to great extent 5= adopted to very great extent, hence, Minimum =1 and Maximum=5) N Missing Mean

Advertising tenders online 38 6 2.66

Online submission of proposals 37 7 2.05

Short listing of suppliers online 37 7 2.32

Company staffs make requisitions online 37 7 2.32

Call for proposals done through company website 38 6 2.45

Existence of functioning website 36 8 2.53

Posting item specifications on company website 36 8 2.17

Procurement Process (availing bids documents and bid information online 37 7 2.19

Suppliers are able to access the bid information 37 7 2.59

Registering suppliers and manage Suppliers information 34 10 2.62

The system is capable of playing bid opening and closing features 36 8 1.89

Suppliers are able to submit their bids online 36 8 1.94

Suppliers are able to trace the online status of their application 37 7 2.00

The system is capable of evaluating the bids received 37 7 2.05

The system is capable of ranking the bid result 36 8 2.11

The system is capable of handling Bid award management 37 7 2.38

The system is capable of contract management with the supplier 37 7 2.08

The system is capable of monitoring and follow-up the progress of the suppliers to meet their deadline (notification) 37 7 2.00

The system is capable of auction management system 37 7 2.14

The system is capable of integrating with the internal and external stakeholders' system (Bank, Inventory Management, Finance Department) 37 7 2.30

Source: Own Survey, December, 2017.

From the findings in table 5 above, public organizations in Ethiopia, implemented e-procurement solution below a moderate extent with a mean < 3 for the different types of e-procurement related activities as listed in the table above like advertising tenders online, online submission of proposals, short listing of suppliers online, company staffs make requisitions online, call for proposals done through company website, existence of functioning website, posting item specifications on company website, procurement process (bids documents and bid information online), suppliers are able to access the bid information, registering suppliers and manage suppliers information, suppliers are able to submit their bids online, suppliers are able to trace the online status of their application, the system is capable of evaluating the bids received, capability of the system to play bid opening and closing features and evaluating the bids received, ranking the bid result, handling Bid award management, contract management with the supplier and monitoring and follow-up the progress of the suppliers to meet their deadline, auction management system and integrating with the internal and external stakeholders' system.

The result contradicts against with the observations by Croom and Brandon-Jones (2004) who assert that e-procurement systems enable organization to carry out individual or all stages of the procurement process such as searching for suppliers, sourcing, negotiation, ordering and posting of purchase review using internet-based (integrated) information and communication technologies. However, this may be attributed to challenges like usage of decades-old equipment and parts whose documentation is paper-based and lacks the digital

format necessary for e-Procurement systems as noted by Moore (2003). Hence, this study proved that, despite the major benefits realized through the adoption of e-procurement solution, most of the public organizations involved with pilot implementation were failing to utilize the solution to full-fledged extent by reaching every supply chain actors through electronic transaction.

Respondents were asked to rank challenges hindering the adoption of e-procurement with a five point liker scale where, 1 =Not challenging factor, 2 = Least challenging factor, 3= moderately challenging factor, 4= Challenging, 5 = Very challenging factor.

Table 6 - Challenges hindering the adoption of e-procurement

Challenges to e-procurement adoption N Missing Not challenging factor Least challenging factor Moderately challenging factor Challenging Very challenging factor Mean

Making equipment compatible is expensive 37 7 9 7 3 12 6 2.97

Lack of regular use by employees 37 7 7 6 10 7 6 3.35

e-procurement adoption is not cost effective approach 36 8 3 7 11 10 8 2.97

Lack of finances to adopt e-procurement 39 5 2 5 12 7 13 3.33

Old IT equipment that needs overhaul 39 5 6 3 8 8 14 3.62

Resistance to change 39 5 5 3 7 11 12 3.54

Lack of e-procurement implementation capacity by small suppliers 38 6 7 0 12 5 14 3.58

Lack of internet access by small suppliers 38 6 5 1 16 7 10 3.50

Lack of company board approval 39 5 1 8 12 8 10 3.41

Lack of managerial support 39 5 2 8 9 8 13 3.46

Availability of consistent follow up and support to employees' training 39 5 3 7 7 13 9 3.33

Inefficient infrastructure framework to facilitate procurement transactions (making payments such as bid guarantees, CPOs, etc on the suppliers' side using the e-procurement system and making payments to the supplier on the buyers' side) 39 5 3 7 7 13 9 3.46

Insufficient top Management Support for e-procurement adoption 40 4 4 7 9 8 12 3.43

Less employee Commitment for e-procurement adoption 40 4 3 9 7 12 9 3.38

Poor IT infrastructure for e-procurement adoption 40 4 4 8 6 7 15 3.53

Unavailability of legal frameworks for e-procurement adoption 39 5 3 7 12 4 13 3.44

Resistance to adopt e-Procurement from buyers' side 38 6 0 8 11 8 11 3.58

Resistance to adopt e-Procurement from suppliers' side 38 6 1 8 7 8 14 3.68

Poor monitoring and evaluation Systems for e-procurement adoption 38 6 3 6 13 7 9 3.34

Source: Own Survey, December 2017.

Among the major challenges for the adoption of e-procurement in the selected public organizations were found to be resistance to make use of e-procurement on the suppliers' side with a mean of 3.68, old IT infrastructure that needs overhaul with a mean of 3.62, resistance to adopt e-procurement adoption from buyers' side (3.58), lack of e-procurement implementation capacity by small suppliers with a mean of 3.58 followed by resistance to change with a mean of 3.54 and poor IT infrastructure for e-procurement adoption with a mean of 3.53. In addition to the data collected using questionnaire, in-depth unstructured interviews were made with a total of 27 employees of the selected organizations with Directorate and senior expert position with functional areas of procurement, finance and ICT. Add to this, experts from MOFED, which controls and facilitates the adoption of the e-procurement (IFMIS), were also interviewed. Accordingly, the following are found to be the major challenges hindering the adoption of the e-procurement in Ethiopia.

Manpower retention: High turnover rate in departments of procurement, finance and inventory is a real threat pointed out in all of the pilot organizations. It has been mentioned that, this is attributed to the small salary scheme of the government. The Ethiopian government's failure to give special attention to the trained and skilled officers of the aforementioned departments had led to the high turnover rate that left many organizations

not to make full use of the pilot implementation. The e-procurement implementation software system works in assigning accounts to different employees in different departments. Therefore, if a trained employee with a privilege leaves that department, the particular work of the department won't be carried out unless someone else with similar access privilege is hired and trained. Thus, where there is no enough manpower to work on the system, the system wouldn't be used completely. The only option available will be falling back to the manual system and integration to the IFMIS that piles unnecessary work overhead to employees.

Inconsistent and disruptive infrastructure: It has been noted that the inconsistent internet connection throughout a year has proved itself to be a nightmare for any internet based implementation such as this pilot implementation. A number of days pass without doing anything on the system while there are urgent procurements, inventories and financial activities to be conducted. This is infuriating department directors and discouraging employees as they have to go back to the manual system and integrate back to the pilot implementation of IFMIS taking lots of their valuable work hours. Power outages and slow internet connections do contribute a lot for the poor performance of the infrastructure. Many of the interviewees raised their concerns on how the full implementation of e-procurement system roll out would ever be successful with the ever inconsistent and poorly performing infrastructure of the country.

Top management commitment and attitude: Top managers do still favor paper works than digital systems. Many of them wouldn't want to believe what the pilot implementation can accomplish and they only consider legitimate and correct what is done on papers. This attitude brings a negative impact on their commitments towards trainings of employees, incentives and supports to the e-procurement adoption. In a few occasions, it has been pointed out that legal framework is compulsory for the top management to take the system seriously. Integration with the legacy system: It is not easy to add works done via the manual system due to different system problems. Previous works can't be recorded as easy as possible. A few operations in the system are not in line with the working of Ethiopian financial operations which calls for customizations.

Employee capacity, commitment and attitude: Some employees don't want to leave the comfort zone of the manual system where they have been working on for years. Commitment to trainings and working on IFMIS are affected directly by the attitude towards a new system where they feel the new system introduces new practices which are not in line with their preferred old way of conducting tasks. Resistance to the pilot system stems from this fear and anxiety. In a few departments, capacity of employees to make use of a digital system is so limited and that calls for consistent capacity building efforts to be well planned and conducted.

Supplier integration and supplier culture: Interviewees opened about their fear of the fact that suppliers would be suspicion of conducting their works online. A few even mentioned how suppliers are reluctant in getting registered in the supplier list of the system. They have the tendency of considering the software system as something that spies on behalf of the government. If a full e-procurement implementation is to be rolled out where suppliers are going to participate, the attitude of suppliers to work on a digital environment should be worked on and legal frameworks that would enhance the integration of suppliers to the digital systems should be in place and enforced well.

Inconsistency of reforms made on procurement solution systems by the Government: This pilot adoption is regarded as one of the futile attempts on the side of the Ethiopian government to introduce a new system or practice in governmental offices. Various inconsistent efforts that ended up the way they were supposed to be paved the way for this particular adoption effort to be considered as one of them. Even those who are enjoying the pilot adoption feared the end of this attempt.

Security fear: Officers and directors do fear about the access their usernames and passwords by other actors so that approvals that they don't know about will be conducted on behalf of them. This fear lets them not to commit them towards the full adoption of the system and stay suspicious of the e-procurement adoption. However, an intensive interview

iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.

made with ICT experts and Directorate across the selected organizations on the same regard, they proved that those ICT support experts have the privilege to reset the passwords of any user in the organization whenever the user is short of accessing their account but not approving any financial transaction. Hence, strict tracing and tracking mechanism is required to strike the balance between the privilege given to the ICT support experts and the fear exhibited on those users/decision makers across e-procurement implementing organizations

Support, monitoring and evaluation: Timely support from those in charge is one of the specially noted subjects during the interviews. Without the timely and proper support, days will go on without working on the system.

Partial implementation and lack of mobile integration: The pilot implementation named IFMIS has a focus on financial procurement affairs that accommodates often the final results of procurement; it doesn't cover the bid processes of procurement. Many employees fail to see its operational and managerial benefits due to this incomplete implementation of e-procurement system. Since the paperwork is also together with the sample implementation, a few employees fail to see the benefits of the sample e-procurement implementation. If the system had been workable on smart phones, it would have been more usable, appealing and trust worthy to users as pointed out in one particular instance.

Similarly, a study conducted in Mauritania by Nazima B. and Vani R., (2017), found that, inadequacies in government policies and legislation can affect the uptake of an ICT system. The lack of standards in the development of e-procurement system, lack of comprehensive technology development and standards to implement the system especially in developing countries often, lack of human resource with technical expertise or knowledge of e-procurement systems as well as lack of financial capacity to risk such as absorb e-Procurement system failure.

CONCLUSION

The study concluded that the five major critical success factors for e-procurement implementation in Ethiopia are the reliability of Information Technology infrastructure followed by provision of reliable of monitoring and evaluation systems, reliability of user acceptance (buyer and supplier), top management support and employee commitment; and the availability of reliable legal environment for e-procurement adoption. While, major perceived challenges exhibited during the adoption of e-procurement solution in Ethiopia as identified by top level managers (interviewees) of public organization are manpower retention, inconsistent and disruptive infrastructure, top management commitment and attitude, integration with the legacy system, employee capacity, commitment and attitude, supplier integration and supplier culture, inconsistency of reforms made on procurement solution systems by the government, security fear, support, monitoring and evaluation; and partial implementation and lack of mobile integration. Add to this, data collected using questionnaire on the same regard, revealed that major challenges hindering the smooth implementation of e-procurement solutions are found to be old IT equipment that needs overhaul, resistance to change, lack of e-procurement implementation capacity by small suppliers, lack of internet access by small suppliers, poor IT infrastructure for e-procurement adoption, resistance to adopt e-Procurement from buyers' side and resistance to adopt e-Procurement from suppliers' side. Hence, government should develop a legal framework and motivational scheme to create interest on legal traders (suppliers) and enforce them to actively play in the e-procurement adoption role as they are the major actors to realize the major benefits (fair public procurement process, transparent, efficient and ethical) that can be derived from e-procurement adoption. The study is limited in that; no attempt was made to examine the relationship between the use of the various e-procurement applications and business outcomes. Instead it was limited to an application called IFMIS. Hence, future research could examine the relationship between the use of integrative e-procurement applications and business outcomes such as increased communication and integration (vertical and horizontal and forward and backward), operational agility, improved product/service design, and overall supply chain performance.

REFERENCES

1. Birks, C., Bond, S. & Radford, M. (2001). Guide to e-Procurement in the Public Sector: Cutting through the Hype. London, UK: Office of Government Commerce, HMSO.

2. Consortium for Global Electronic Commerce (CGEC) (2002, October). Measuring and Improving Value of E-procurement Initiatives. Madison, WI: University of Wisconsin-Madison, Consortium for Global Electronic Commerce.

3. Croom, S. & Brandon-Jones, A. (2004). "E-Procurement: Key issues in e-Procurement implementation and operation in the public sector", 13th International Purchasing & Supply Education & Research Association (IPSERA) Conference, April 4-7, Italy.

4. Federal Negarit Gazeta of the Federal Democratic Republic of Ethiopia. (2009). Proclamation No. 649/2009. The Ethiopian Federal Government Procurement and Property Administration Proclamation. p. 4858. No.60 ADDIS ABABA September, 2009.

5. Gunasekaran, A., & Ngai, E. W. (2008). Adoption of e-procurement in Hong Kong: an empirical research. International Journal of Production Economics, 113(1), 159- 175.

6. James Mauti Mose (2013). The critical success factors and challenges in e-procurement adoption among large scale manufacturing firms in Nairobi, Kenya. European Scientific Journal May 2013 edition vol.9, No.13 ISSN: 1857 - 7881 (Print) e - ISSN 1857- 7431.

7. Kendall, M. G. (1939). The problem of rankings. Ann. Math. Static, 10: 275-287.

8. Kevin George Obat (2016). On critical success factors in the implementation of e-procurement in public entities in Kisumu County, Kenya. Master Thesis Submitted To School Of Business, University Of Nairobi.

9. KPMG (2001). University of California Office of the President System- Wide E-Procurement Assessment and Strategy Recommendation. KPMG Consulting.

10. M.V. Jooste and C. de W. van Schoor. (2003). A framework for the implementation of e-procurement. Department of Industrial and Systems Engineering. University of Pretoria, South Africa.The south african journal of industrial engineering. Vol 14, No 2.[Available] Online: http://sajie.journals.ac.za/pub/article/view/262 [15 October 2017].

11. Monczka (2000). What will happen and what you should know: purchasing, 124(1), 78-83

12. Moore, J. (April 1, 2007). E-Procurement Pays Its Own Way: Strategic Sourcing and e-Procurement Transform Purchasing in DeKalb County, GA." FCW.COM

13. Mose, W. N. (2012). Impact of Electronic Procurement on The Operations of Kenya Commercial Bank (Doctoral dissertation, University of Nairobi).

14. Nazima B., 2017. The Challenges and Benefits of Adopting E-procurement - Reference to Mauritian Economy. International Journal of Management and Applied Science, 3(5).

15. Naveed S. (2017). Assessing e-procurement success factors in telecommunication organizations. Thesis submitted to the Capital University, Islamabad.

16. Operational Services Division (OSD) (2001). Critical Success Factors and Metrics: Enhanced Comm-Pass Initiative. Boston, MA: Commonwealth of Massachusetts.

17. Pani, A. K., &Kar, A. K. (2011, January). A study to compare relative importance of criteria for supplier evaluation in e-procurement. In System Sciences (HICSS), 2011 44th Hawaii International Conference on (pp. 1-8). IEEE.

18. Pani, A.K. (2011). A study to compare relative importance of criteria for supplier evaluation in e-procurement. In System Sciences, 44th Hawaii Int. Conf. (pp. 1-8). IEEE.

19. Sarah Elsie Baker and Rosalind Edwards [n.d]. How many qualitative interviews is enough? Expert voices and early career reflections on sampling and cases in qualitative research.National Centre for Research Methods Review Paper [Available] Online: http://blog.soton.ac.uk/dissertation/files/2013/09/how_many_interviews.pdf [10/Oct 2017].

20. Stenning & Associates Pty Ltd (S&A) (2003). Final Report: Evaluation and Review of the e-Procurement Pilot Project, Version 1.2, Hobart, Australia.

21. The FDRE Public Procurement & Property Administration Agency (2014).The Ethiopian procurement system. The FDRE Public Procurement & Property Administration Agency.

i Надоели баннеры? Вы всегда можете отключить рекламу.