ANALYZING FINANCIAL PERFORMANCES AND EFFICIENCY OF THE RETAIL FOOD IN SERBIA BY USING THE AHP – TOPSIS METHOD Текст научной статьи по специальности «Экономика и бизнес»

ANALYZING FINANCIAL PERFORMANCES AND EFFICIENCY OF THE RETAIL FOOD IN SERBIA BY USING THE AHP - TOPSIS

METHOD

Radojko Lukic1, Dragana Vojteski Kljenak2, Slavica Andelic3 *Corresponding author E-mail: [email protected]

ARTICLEINFO ABSTRACT

The aim and purpose of this paper is to point out to the quality of financial performance and efficiency of food retailers in Serbia, as well as measures for improvement in the future, based on theoretical knowledge and empirical analysis using AHP-TOPSIS methods. The problem of analyzing the financial performance of all companies, which includes trading companies, is very topical, significant and complex. Consequently, mathematical methods and models have lately been increasingly used. With this insight in mind, this paper investigates the financial performance and efficiency of food retailers in Serbia using AHP and TOPSIS methods. Of all the observed optimization criteria (cost of goods sold, operating costs, gross margin and net profit), the most significant was the cost of goods sold. The most efficient food retailer in Serbia is Aman. The Mercator-S Company is inefficient. In order to improve the efficiency of food retailers in Serbia, it is necessary to apply the Western business models (private brand, multichannel sales, organic food sales and others), the concepts of strategic management accounting and to strengthen the digitalization of business.

© 2020 EA. All rights reserved.

Introduction

Given its importance, this paper's research subject is to measure the efficacy of food

retailers in Serbia, such as analysis in Western literature (Danielle at el., 2019; Jacob at el.,

1 Radojko Lukic, Ph.D., Full Professor,University of Belgrade Faculty of Economics, Kamenica 6, Belgrade Republic of Serbia, Phone: +381 11 3021 112, E-mail: rlukic@ekof. bg.ac.rs ,ORCID ID (https://orcid.org/0000-0001-6529-0297)

2 Dragana Vojteski Kljenak, Ph.D., Associate Professor, Faculty of Business Economics and Entrepreneurship, Mitropolita Petra 8, Belgrade, Republic of Serbia, Phone: +381 11 2762 194, E-mail: [email protected], ORCID ID (https://orcid.org/0000-0002-5707-1585)

3 Slavica Andelic, Ph.D., Associate Professor, Faculty of Business Economics and Entrepreneurship, Mitropolita Petra 8, Belgrade, Republic of Serbia, Phone: +381 11 2762 194, E-mail, [email protected], ORCID ID (https://orcid.org/0000-0003-4021-0822)

Original Article

Received: 06 November 2019

Accepted: 04 March 2020

doi:10.5937/ekoPolj2001055L

UDC 658.14/.17:330.342]:339.372 (497.11)

Keywords:

efficiency, retail food, Serbia, AHP, TOPSIS

JEL: L81, M31, M41, O32

2019; Jian at el., 2019; Kalpana at el., 2019; Yonggan at el., 2019). The aim and purpose of this research is to determine the factual state based on which adequate measures for improvement in the future will be suggested.

As is well known, a very rich body of literature is devoted to the general issue of applying the DEA analysis in evaluating company performance (Hwang, 1981; Hwang, 1995; Andersen, 1993; Yousefi, 2010; Li, 2014, 2017; Tsolas, 2015). It is also richer in terms of application in the retail sector (Bhargava, 1998; Karan, 2008; Keener, 2013; Kingyens, 2012; Konuk, 2018; Lau, 2013; Manini, 2018; Martini 2017; Pang, 2013; Parades , 2014; Rogova, 2018; Simbolon, 2017; Trejo, 2017; Zaernyuk, 2016; Ufuncu, 2018; Urbonaviciute, 2019). Recently, in the Western literature, in addition to the DEA methods, AHP methods have been used (Chang, 1996) as well as the TOPSIS method. As far as literature in Serbia is concerned, it is, to our knowledge, very poor in this regard, only in some works the AHP and DEA methods are partially applied (Lukic, 2011a, b; 2018, 2019; Lukic, 2018; Popovic, 2018) , which is not the case with the TOPSIS method. This void should be filled to a certain extent by this work, which should reflect its scientific and professional contribution among other things.

From the very nature of the problem addressed in this paper, the basic hypothesis of the research itself arises: knowledge of the current financial situation of food retailers in Serbia is a prerequisite for improvement in the future. This can be easily achieved by taking adequate measures.

The research of the given hypothesis in this paper is based on the application of the AHP and TOPSIS methods. For the sake of comprehensiveness, ratio analysis and statistical analysis are here used to a certain extent.

For the purpose of investigating the treated problem in this paper, empirical data were taken from the Business Registers Agency of the Republic of Serbia. They are "manufactured" in accordance with relevant international standards and there are no restrictions on their global comparability. This completely refers to the data obtained in this research.

Materials and methods

The Analytical Hierarchy Process Method (AHP) is a multi-criteria decision making recommended by Thomas Saaty in the 1970s (Saaty, 1970; Saaty 1980; Saaty, 2001; Saaty, 2008). It is used to solve complex structural hierarchical problems of decisionmaking and weighting coefficients (ponders) for each criterion (Harker, 1987; Hanie, 2016; Stojanovic, 2016). Figure 1 shows an example of a hierarchy in AHP.

Figure 1. An example of hierarchy in the AHP

The analytical hierarchical process method is based on the following axioms (Saaty, 1986; Harker, 1987; Alphonce, 1997):

The reciprocity axiom: If element A is n times more significant than element B, then element B is 1/n times more significant than element A.

The homogeneity axiom: Comparison makes sense only if the elements are comparable.

The dependency Axiom: It is allowed to compare between a group of elements of one level with respect to higher elements, i.e. lower-level comparisons depend on higherlevel elements.

The expectations axiom: Any change in the structure of the hierarchy requires an estimate of priorities in the new hierarchy.

Each comparison of the two elements of the hierarchy (model) is made using the Saaty's value scale (Table 1.).

Table 1. Saaty's value scale

Importance intensity Definition Explanation

1 Equal importance The two elements are of identical importance with respect to the goal.

3 Weak dominance Experience or reasoning slightly favors one element over another.

5 Strong dominance Experience or reasoning favors one element a lot more than the other.

7 Demonstrated dominance The dominance of one element is confirmed in practice.

9 Absolute dominance The highest degree dominance

2,4,6,8 Intermediate values A compromise or further division is needed.

Source: Saaty, 2008

The analytically hierarchical process (AHP) method proceeds through the following steps (Saaty, 2001; Saaty, 2008; Hanie, 2016; Stojanovic, 2016):

Step 1: Forming a pairwise comparison matrix

Step 2: Normalization of the pairwise comparison matrix

a,

Y" fi

,i,j = 1, ...,n

Step 3: Determining relative importance, i.e. weight vectors

-,i,j = 1, ...,n

(2)

(3)

Consistency index - CI is a measure of deviation of n from X and can be represented

J max r

by the following formula:

If CI <0.1, the estimated values of the coefficients a are consistent and the deviation of

' v

Xmax from n is negligible. This means, in other words, that the AHP method accepts an inconsistency of less than 10%.

Using the consistency index, the consistency ratio CR = CI / RI can be calculated, with RI being a random index. Table 2 gives random consistency indices.

Table 2. Random consistency indices

Matrix size

(a number of 1 2 3 4 5 6 7 8 9 10

criteria) (n)

RI 0 0 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49

Source: Hanie, 2016

The TOPSIS (Technique for Order Preference by Similarity to Ideal Solution)

method has been used very successfully in evaluating companies' financial performance (Ufuncu et al., 2018). It is a multi-criteria decision-making technique first developed and applied by Hwang and Yoon (1981). (Hwang, 1981; Hwang, 1995). According to this method, alternatives are determined by their distances from the ideal solution. The goal is to choose the optimal alternative that is closest to the ideal solution, i.e., farthest from the negative ideal solution (Young, 1994). A positive ideal solution maximizes utility, that is, minimizes costs (in relation to a given problem). In contrast, a negative ideal solution maximizes costs, i.e. minimizes utility (Yousefi 2010; Wang 2007). 58 http://ea.bg.ac.rs

The TOPSIS method consists of 6 steps (Ufuncu et al., 2018). Step 1: Create an Initial Matrix

The initial matrix A shown with "m" denotes the alternative number and with "n" the

•j

number of criteria:

Step 2: Formation of the Weighted Normalized Decision Matrix

The normalized decision matrix (R.; i = 1,..., m; j = 1,..., n) is determined by the

equation (14) with the elements of the matrix A.:.

t = 1,2,3, ...,m

j = 1,2,3.....n

In the equation (6) the weight measure "j" is represented by W The weighted normalized decision matrix (Vj; • = 1,..., m; j = 1,..., n) was determined using the equation (6) with the elements of the normalized matrix:

Step 3: Determine the Positive and Negative Ideal Solution

The value of the positive-ideal solution (A+) and negative-ideal solution (A-) is determined from the value of the weighted normalized matrix (Vj). A+ is better and A- is a worse performance score.

The value of the positive-ideal solution (A+) and the negative-ideal solution (A-) is determined as follows (equation (7) and (8) respectively)

where j is related to the benefit criterion, and j' is related to the cost criterion.

Step 4: Determine special measures (i.e. distance of alternatives from the ideal and negative-ideal solution)

The distance from the positive-ideal solution (S+) and the negative-ideal solution (S:) for each alternative according to the given criterion is determined using equations (9) and (10).

Step 5: Determine the coefficient of relative closeness to the ideal solution

Specific measures of positive-ideal solution (S+) and negative-ideal solution (S-) were used to determine the relative closeness to the ideal solution (C+) for each decision point. C + represents the relative closeness to the ideal solution and takes a value in the range 0 < C. + < 1. "C + " = 1 indicates the relative closeness to the positive-ideal solution. "C+" = 0 indicates relative closeness to the negative-ideal solution.

The relative closeness to the ideal solution (C.+; i = 1,..., m; j = 1,..., n) was determined using the equation (11):

sr

(11)

c? = -S

t

Step 6: Sort the alternatives according to relative superiority

Determining the relative superiority of the score represents the company's realized performance. High scores correlate with better performance. The results can be used to determine company's rankings within the industry (Ufuncu et al., 2018).

Results and discussion

Table 3 shows some financial indicators of the performance of selective major food retailers in Serbia for 2018.

Table 3. Some indicators of financial performance of selective food retailers in Serbia, 2018

Gross margin/Sales (%) Operating costs/Sales (%) Net profit/Sales (%)

Delhaize Serbia 30% 27% 3%

Mercator-S 21% 23% -2%

Gross margin/Sales (%) Operating costs/Sales (%) Net profit/Sales (%)

DIS trade 10% 9% 1%

Aman 17% 14% 2%

Univerexport 23% 23% 0%

Note: Author's calculation Source: Agency for Business Registers of the Republic of Serbia

There are therefore significant differences in the gross margin rate between observed food retailers. They range from 10% (DIS stores) to 30% (Delhaize Serbia). These differences are in their own way reflected in their efficiency, since operating expenses (business expenses) are covered by gross margin as the difference between the sale and the purchase value of goods sold and the rest is net profit. Table 4 presents the statistics of input / output data as criteria used to evaluate the effectiveness of selective food retailers in Serbia for 2018 using the AHP and TOPSIS methods.

Table 4. Data input / output statistics

(I) Purchase value of goods sold (in millions of RSD) (I) Operating costs (in millions of RSD) (O) Gross margin (in millions of RSD) (O) Net profit (in millions of RSD)

Delhaize Serbia 70666 27157 29822 2665

Mercator-S 65054 19376 17714 -1662

DIS trade 17490 1840 1879 39

Aman 14256 2451 2871 420

Univerexport 14236 4309 4339 30

Statistics of Input / Output Data

Max 70666 27157 29822 2665

Min 14236 1840 1879 -1662

Average 36340.4 11026.6 11325 298.4

SD 25823.9 10324.4 10885.4 1385.5

Correlations

Purchase value of goods sold Pearson Correlation 1 979** .952* .185

S i g . (2-tailed) .004 .013 .766

N 5 5 5 5

(I) Purchase value of goods sold (in millions of RSD) (I) Operating costs (in millions of RSD) (O) Gross margin (in millions of RSD) (O) Net profit (in millions of RSD)

Operating costs Pearson Correlation 979** 1 .993** .349

S i g . (2-tailed) .004 .001 .565

N 5 5 5 5

Gross margin Pearson Correlation .952* .993** 1 .458

S i g . (2-tailed) .013 .001 .438

N 5 5 5 5

Net profit Pearson Correlation .185 .349 .458 1

S i g . (2-tailed) .766 .565 .438

N 5 5 5 5

**. Correlation is significant at the 0.01 level (2-tailed). *. Correlation is significant at the 0.05 level (2-tailed).

Source: Agency for Business Registers of the Republic of Serbia. Author's calculation of input / output statistics using DEA - Solver and SPSS software

In the further presentations of the problem, we will examine the effectiveness of selective food retailers in Serbia for 2018 using the AHP/TOPSIS method (Table 5). Weighting criteria were determined using the AHP method (CR =1.9%). In this particular case, therefore, in the order of importance of the criteria, the situation is as follows: purchase value of goods sold, operating expenses, gross margin and net profit. This is quite understandable given the fact that the cost of goods sold represents the most important investment in the trade sector. Operating costs are very significant in commerce and most of them relate to employee earnings. Employee earnings affect employees' motivation to achieve the best possible sales. Gross margin yield from inventories is a significant indicator of trade performance. And the return on net sales is also a significant indicator of trade performance. For these reasons, the relevant optimization criteria were chosen in this paper. The optimization criteria in this case are designated as: C1 - cost of goods sold, C2 - operating costs, C3 - gross margin and C4 - net profit. The alternatives are: A1 - Delhaize Serbia, A2 - Mercator-S, A3 - DIS store, A1 - Aman and A5 - Univerexport. (Data was processed using AHP Online System -AHP-OS, and ARASSoftware.xlsx.)

Table 5 shows the initial decision matrix.

Table 5. The initial decision matrix

weights of criteria 0.533 0.321 0.082 0.064

kind of criteria -1 -1 1 1

C1 C2 C3 C4

Delhaize Serbia 70666 27157 29822 2665

Mercator-S 65054 19376 17714 -1662

DIS trade 17490 1840 1879 39

Aman 14256 2451 2871 420

Univerexport 14236 4309 4339 30

MAX 70666 27157 29822 2665

MIN 14236 1840 1879 -1662

0-0ptimal Value 14236 1840 29822 2665

Table 6 shows the normalized decision matrix.

Table 6. The normalized decision matrix

weights of criteria 0.533 0.321 0.082 0.064

kind of criteria -1 -1 1 1

C1 C2 C3 C4

0-0ptimal Value 0.2362 0.2994 0.3450 0.4580

Delhaize Serbia 0.0476 0.0203 0.3450 0.4580

Mercator-S 0.0517 0.0284 0.2049 0

DIS trade 0.1923 0.2994 0.0217 0.0067

Aman 0.2359 0.2247 0.0332 0.0722

Univerexport 0.2362 0.1278 0.0502 0.0052

Table 7 shows the weighted normalized decision matrix.

Table 7. Weighted normalized decision matrix

C1 C2 C3 C4

0-0ptimal Value 0.1259 0.0961 0.0283 0.0293

Delhaize Serbia 0.0254 0.0065 0.0283 0.0293

Mercator-S 0.0276 0.0091 0.0168 0

DIS trade 0.1025 0.0961 0.0018 0.0004

Aman 0.1257 0.0721 0.0027 0.0046

Univerexport 0.1259 0.0410 0.0041 0.0003

Table 8 and Figure 2 show the ranked decision matrix.

Table 8. Ranked decision matrix

S K K Ranking

O-Optimal Value 0.2796 1 1

Delhaize Serbia 0.0895 0.3200 0.3200 4

Mercator-S 0.0535 0.1913 0.1913 5

DIS trade 0.2008 0.7181 0.7181 2

Aman 0.2052 0.7340 0.7340 1

Univerexport 0.1714 0.6130 0.6130 3

We can therefore conclude that Aman is an efficient food retailer in Serbia. It is followed by the following companies respectively: DIS store, Univerexport, and Delhaize Serbia. Inefficient is, understood broadly, the Mercator-S Company.

Conclusions

The research conducted in this paper in the context of measuring the efficiency of food retailers in Serbia using the AHP method has shown that the importance of certain criteria is as follows: purchase value of goods sold, operating costs, gross margin and net profit. The cost of goods sold and operating expenses are inputs and gross margin and net profit are outputs. The goal is to maximize yields with given resources. In our opinion, the given optimization criteria, given the nature of the business of food retailers, are very significant.

The results of the research in this paper that used the TOPSIS method show that an efficient food retailer in Serbia is Aman which is followed by the following companies respectively: DIS store, Univerexport, and Delhaize Serbia. An inefficient food retailer in Serbia is the Mercator-S Company.

In order to increase the efficiency of food retailers in Serbia in the future, it is necessary, in principle, to apply new business models which are in line with the Western model (private brand, multichannel sales, organic food sales and others), and concepts of modern strategic management accounting. Likewise, business digitalization needs to be improved.

Conflict of interests

The authors declare no conflict of interest.

References

1. Alphonce C. B. (1997). Application ofthe Analytic Hierarchy Process in Agriculture in Developing Countries. Agricultural Systems, 53, 97-112.

2. Andersen, P., & Petersen, N. C. (1993). A procedure for ranking efficient units in data envelopment analysis. Management Science, 39(10), 1261-1264.

3. Bhargava, M., Dubelaar, C. and Scott, T. (1998). Predicting bankruptcy in the retail sector: an examination of the validity of key measures of performance. Journal of Retailing and Services, 5(6), 105-117.

4. Chang, D.Y.(1996). Aplikation of the extent analysis method on fuzy AHP. European journal of operational research, 95(3), 649-655.

5. Danielle Ufer, David L. Ortega, Christopher A. Wolf (2019). Economic foundations for the use of biotechnology to improve farm animal welfare. Trends in Food Science & Technology, 91, 129-139.

6. Harker P. T. and Vargas L.G.(1987). The Theory of Ratio Scale Estimation: Saaty's Analytic Hierarchy Process. Management Science,. 33(11), 1383-1403.

7. Hanie, M., Boutkhoum,O., Tikniouine, A. and Agouti, T. (2016). Application of an integrated multi-criteria decision making AHP-TOPSIS methodology for ETL software selection. Springer Plus, 5(263), 1-17.

8. Hwang C.L., Yoon K.S. (1981). Multiple attribute decision making: methods and applications. Berlin: Springer.

9. Hwang, C. L., Yoon, K.P. (1995). Multiple Attribute Decision Making: An Introduction. Paperback / Sage Pubns.

10. Jacob Tizhe Liberty, Jalal Dehghannya, Michael O. Ngadi (2019). Effective strategies for reduction of oil content in deep-fat fried foods: A review. Trends in Food Science & Technology, 92, 172-183.

11. Jian Ju, Xueqi Chen, Yunfei Xie, Hang Yu, ... Weirong Yao (2019). Application of essential oil as a sustained release preparation in food packaging. Trends in Food Science & Technology, 92, 22-32.

12. S. Kalpana,S.R.Priyadarshini,M.MariaLeena,J.A.Moses,C.Anandharamakrishnan (2019). Intelligent packaging: Trends and applications in food systems. Trends in Food Science & Technology 93, 145-157.

13. Karan, M.B., Ulucan, A. and Kaya, M. (2008). Estimation of credit risk of retail stores by using their payment history: A combined logistic regression and multi-dea. 5th Internation Scientific Conference Business and Management 2008, 16-17 May 2008, Vilniua, Lithuania, 222- 227.

14. Keener, M.H. (August 2013). Predicting The Financial Failure Of Retail Companies In The United States. Journal of Business & Economics Research, 11(8), 373-380.

15. Kingyens, Angela Tsui-Yin Tran (2012). Bankruptcy prediction of companies in the retail - apparel industry using data envelopment analysis. Degree of Doctor of Philosophy Graduate Department of Chemical Engineering and Applied Chemistry University of Toronto.

16. Konuk, F. (2018). Financial and Performance Analysis of Food Companies: Application of Topsis and DEA. MANAS Journal of Social Studies, 7(3), 381-390.

17. Lau, K.H. (2013). Measuring distribution efficiency of a retail network through data envelopment analysis. Int.J.Production Economics, 146, 598-611.

18. Li, Z., Crook, J. and Andreeva, (2017). Dynamic prediction of financial distress using Malmquist DEA. Expert systems With Applications, 80, 94-106.

19. Li, Z., Crook, J. and Andreeva, G. (2014). Chinese Companies Distress Prediction: An Application of Data Envelopment Analysis. Journal of the Operational Research Society, 65, 466-479.

20. Lukic, R. (2011a). Evaluation of Retail Business Performance. Belgrade: Faculty of Economics [in Serbia: Evaluacijaposlovnih performansi u maloprodaji].

21. Lukic, R. (2011b). Estimates of economic performance of organic food retail trade.

Economic research, 24(3), 157-169.

22. Lukic, R. (2018). The Analysis of the Operative Profit Margin of Trade Companies in Serbia. Review of International Comparative Management, 19(9), 458-478.

23. Lukic, R., Lalic, S., Suceska, A., Hanic, A. and Bugarcic, M. (2018). Carbon diohide emissions in retail food. Economics of Agriculture, 65(2), 859-874.

24. Lukic, R. (2018). Analysis of Return on Investment in Trade of Serbia: Current State and Prospects. Proceedings of the Matica Srpska for Social Sciences, 168 (4/2018), 777-803 [in Serbia: Анализа приноса на инвестицще трговине Србще: Актуелно стаае и перспективе. ЗборникМатице српске за друштвене науке].

25. Lukic, R., Lalic, S., Suceska, A., Hanic, A. and Bugarcic, M. (2018). Carbon diohide emissions in retail food. Economics of Agriculture, 65(2), 859-874.

26. Lukic, R. (2019). Efficiency analysis of trading companies in Serbia. Proceedings of the Faculty of Economics Brcko, 13 (1), 15-27 [in Serbia: Analiza efikasnosti trgovinskih preduzeca u Srbiji. Zbornik radova Ekonomskog fakulteta Brcko].

27. Manini, R. and Amat, O. (2018). Credit scoring for the supermarket and retailing industry: Analysis and application proposal. Economics Working Paper Series, Working Paper No. 1614, Universitat Pompeu Fabra, Barcelona, Department of Economics and Business, 1-14.

28. Martino, G. et al. (2017). Supply Chain Risk Assessment in the Fashion retail Industry: An Analytic Network Process Approach. International Journal of Applies research, 12(2), 140-154.

29. Pang, J. and Kogel, M (2013). Retail Bankruptcy Prediction. American Journal of Economics and Business Administration, 5(1), 29-46.

30. Paradi, J.C., Wilson, D. and Yang, X.P. (2014). Data Envelopment Analysis of Corporate Failure for Non-Manufacturing Firms Using a Slacks-Based Measure.

Journal of Service Science and Management, 7, 277-290.

31. Popovic, S, Jankovic, I, & Stojanovic, Z., (2018). The importance of bank credits for agricultural financing in Serbia. Economics of Agriculture, 65 (1), 65-80.

32. Rogova, E. and Blinova, A. (2018). The Technical Efficiency of Russian Retail Companies: An Empirical Analysis. Zesz. Nauk. UEK, 5 (977), 171-185.

33. Saaty, T.L. (1970). Optimization in integers and Related Extremal Problems. New York: MCGraw-Hill.

34. Saaty, T.L. and Vargas, L.G. (2001). Models, Methods, Concepts & Applications of the Analytic Hierarchy Process. International Series in Operations Research and Management Science, Springer.

35. Saaty T.L.(1980). The analytic hierarchy process. New York: McGraw-Hill.

36. Saaty, T.L. (2008). Decision Making With The Analytic Hierarchy Process. Int J Serv Sci, 1(1), 83-98.

37. Saaty, T.L. (1986). Axiomatic foundation of the Analytic Hierarchy Process.

Management Science, 32(7), 841-855 .

38. Stojanovic, M. and Rogodic, D. (2016). Evaluating E-commerce Sites in Serbia Using AHP-TOPSIS Methods. Proceedings University of Synergy, 99-104. [in Serbia: Ocenjivanje sajtova za E-kupovinu u Srbiji upotrebom AHP-TOPSIS metoda. Zbornik radova univerziteta sinergija, 99-104].

39. Simbolon, R. and Elviani S. (2017). Bankruptcy Analysis Using Altman Z-score Model in Retail Trading Company Listed in Indonesia Stock Exchange. Proceedings of The 7th Annual International Conference (AIC) Syiah Kuala University and The 6th International Conference on Multidisciplinary Research (ICMR) in conjunction with the International Conference on Electrical Engineering and Informatics (ICELTICs) 2017, October 18-20, 2017, Banda Aceh, Indonesia, 273-279.

40. Trejo García , J. C., Martínez García, M. A. and Venegas Martínez, F. (2017). Credit risk management at retail in Mexico: An econometric improvement in the selection of variables and changes in their characteristics. Contaduría y Administración, 62, 399-418.

41. Tsolas, I.E. (2015). Firm credit risk evaluation: a series two-stage DEA modeling framework. Ann Oper Res, 233, 483-500.

42. Üfüncü, T., Akyüz, K. C., Akyüz, Í., Bayram, B. £., Ve Ersen, N. (2018). Evaluation Of Financial Performance Of Paper Companies Traded At BIST With TOPSIS Method. Kastamonu University Journal Of ForestryFaculty, 18(1), 92-98.

43. Urbonaviciüté, K. and Maknickiené, N. (2019). Investigation of digital retail companies financial performance using multiple criteria decision analysis.

Economics and Management/ Ekonomika ir vadyba, 11, 1-9.

44. Yousefi, A. ve Hadi-Vencheh, A. (2010). An integrated group decision making model and its evaluation by DEA for automobile industry. ExpertSystems with Applications, 37(8), 543-556.

45. Yonggan Zhao, Min Zhang, Sakamon Devahastin, Yaping Liu (2019. Progresses on processing methods of umami substances: A review . Trends in Food Science & Technology, 93, 125-135.

46. Wang, Y. J. and Lee, H. S. (2007). GeneralizingTOPSIS for fuzzy multiple-criteria group decision-making. Computers and Mathematics with Applications, 53, 17621772.

47. Zaernyuk, V.M. et al. (2016). Solving the Problem of Credit Defaults in Retail Sector. European Research Studies, XIX (2), 205-217.