CC BY
39
DOI https://doi.org/10.18551/rjoas.2018-12.04
INTERACTION EFFECT OF INTOLERANCE OF AMBIGUITY, PERSPECTIVE AND TYPE OF BALANCED SCORECARD MEASURES ON PERFORMANCE EVALUATION
Handoyo Sarwo Edy1,2,3*, Thoyib Armanu4, Ratnawati Kusuma4, Andarwati4
1Student, Doctoral Degree Program, Faculty of Economics and Business, University of Brawijaya, Indonesia 2Management Department, Faculty of Economics and Business, University of Tarumanagara, Indonesia 3Accounting Department, Faculty of Economics and Communication, Bina Nusantara University, Indonesia 4Faculty of Economics and Business, University of Brawijaya, Indonesia *E-mail: sarwoh@fe.untar. ac.id
ABSTRACT
Performance evaluation is defined as key activities to ensure whether organizational goals can be realized or not. The character of the evaluator and evaluation object are the determining factors for evaluating performance evaluation. The purpose of this research was to determine the interaction effects of intolerance of ambiguity evaluator with the balanced scorecard perspective, intolerance of ambiguity evaluator with the type of balanced scorecard measures, as well as perspective with the type of balanced scorecard measures on the results of performance evaluation. The approach used in this research was quantitative, the type of research was experimental using the factorial design of 2x2x2. The research was conducted at two private universities that implemented the balanced scorecard in Jakarta. 128 respondents of the research consisted of head of study programs, deans and rectorates. The analytical method used was a three-way variant analysis that was processed using SPSS version 24 software. The results show that the interaction between intolerance of ambiguity evaluator and balanced sorecard perspective has an effect on the results of performance evaluation, the interaction of intolerance of ambiguity with the type of balanced scorecard measures has an effect on the results of performance evaluation, and the interaction between perspective with the type of balanced scorecard measures has an effect on the results of performance evaluation. These findings provide implications for performance evaluation systems that can influence organizational behavior to achieve organizational goals.
KEY WORDS
Interaction effect, intolerance of ambiguity, perspective, size, performance evaluation.
Accreditation results owned by universities can be an indicator of their qualities. When deciding on college choices, prospective students will see the accreditation assesment of institutions and study programs. Likewise, graduate recruiters will recruit graduates from universities by looking at the accreditation of institutions and study programs. In the face of competition, universities need to have the best actration at national and international levels. To have such accreditation, universities can use balanced scorecard (BSC) as a method to translate their vision and mission into performance measures that can be controlled and realized.
Since being introduced by Kaplan and Norton in 1996, the BSC has been widely used in profit-oriented and non-profit oriented institutions. Several foreign and domestic universities have implemented BSC to measure their performance. Large universities in the USA such as the University of California Berkeley, the University of California San Diego, Cornell University, the University of Tennessee, and so on have for some time used this BSC concept to measure the performance of their organizations (Indrajit and Djokopranoto, 2006: 144). Similarly, several universities in Jakarta, Indonesia has also implemented BSC such as Tarumanagara University (Pramono, 2009), Bina Nusantara University (Polla, 2007).
A survey conducted by Anand, Sahay and Saha (2005) found that 45.28% of respondents had used BSC as a performance management tool.
In order to obtain accreditation, the role of the evaluator of the internal performance of universities and accreditation institutions is to determine the level of accreditation obtained. Before the accreditation of study programs or institutions is carried out by an accreditation institution, the performance evaluator from within the university can be the initial determinant to determine the level of accreditation. The results of performance evaluation are influenced by evaluation context, object of evaluation, and evaluator (Gelderman, 1998).
The BSC has major strengths in unique measures in every perspective (Kaplan and Norton, 1992). However, Lipe and Salterio (2000) proved in their research that managers ignore one of the key characteristics of BSC, namely the placement of unique sizes for the strategic objectives of business units in the evaluation of performance evaluation. If the manager assesses performance with a common size only, it will create a bias for the usefulness of the bSc, which is caused by the amount of implementation costs including management time that is not comparable with the usefulness of BSC or in short, the BSC does not cause any benefits (Kaplan and Norton, 1996).
In line with the findings of Lipe and Salterio (2000), Ittner, Larcker and Meyer (2003) in their research proved that there is a subjectivity in determining the relative weight of measures used in evaluation because there is a tendency to bias from a financial perspective for common performance measures. Lipe and Salterio (2000) stated that there are human cognitive tendencies and other factors that result in performance evaluation which is not aligned with the strategies and objectives articulated in the BSC of the company. These other factors are the availability of third party guarantees (Libby, Salterio and Webb, 2004), the BSC evaluation approach (Roberts, Albright and Hibbets, 2004) and manager's knowledge of BSC (Dilla and Steinbart, 2005).
The results of research conducted by Ittner, Larcker and Meyer(2003) revealed various performance measures that are charged to the BSC bonus plan in most service companies. Subjectivity occurs in the BSC plan that allows managers to reduce the balance in giving bonuses by placing a portion of the weight on financial measures. Financial performadnce is the main determinant of bonuses. They argued that psychology-based explanations are more relevant than economic-based explanations in explaining company performance measurement practices.
The results of research conducted by Cardinaels and Veen-Dirks (2010) proved that when evaluating using BSC, evaluators tend to pay more attention to financial measures than non-financial measures. Furthermore, the results of Bone and Sholihin's (2012) research indicate that when evaluation is conducted by using BSC, the evaluators tend to give more attention to both financial and non-financial perspectives in common size than unique size. Gelderman (1998) stated that there are several characteristics of evaluators of performance measurement systems, such as locus of control and ambiguity tolerance. With the various characteristics of evaluators, the results of the performance evaluation system can vary. DeBusk, Killough and Brown (2010) argued that an aspect of individual differences that has been tested in accounting research since 1970 are intolerance of ambiguity.
Budner (1962) supported the possible relationship between intolerance of ambiguity (IOA) and the use of information. Individuals who are intolerant of ambiguity will be more likely to seek additional information to reduce the threat of ambiguity from existing data or the absence of overall data (DeBusk, Killough and Brown, 2010). Dermer (1973) stated that there is a possibility that cognitive characteristics of information users can influence perceptions about important information and therefore can influence how information influences behavior.
Evaluators who are not tolerant of ambiguity need a lot of information in doing their work compared to evaluators who are tolerant of ambiguity (Gelderman, 1998). DeBusk, Killough and Brown (2010) suggested that intolerance of ambiguity is related to individual self-confidence. Liedtka, Church and Ray (2008) stated that individuals who are intolerant of ambiguity reject ambiguous information. Banker, Chang and Pizzini (2004) found that performance evaluation is influenced by measures related to strategy rather than measures
that are not related to strategy when evaluators are provided with detailed information about the strategy of the business unit.
This research follows up on what Lipe and Salterio (2000), DeBusk, Killough and Brown (2010), Cardinaels and Veen-Dirks (2010), and Bone and Sholihin (2012) have done to identify possible biases in performance evaluation. Lipe and Salterio (2000) found that when performing performance evaluation using BSC, evaluators are more likely to pay attention to common measures than unique measures. While research from Cardinaels and Veen-Dirks (2010) found that when evaluation is conducted by using BSC, evaluators tend to pay more attention to financial measures than non-financial measures. Furthermore, the results of Bone and Sholihin's (2012) research showed that evaluators use both common financial measures and common non-financial measures in evaluating their performance. Bone and Sholihin (2012) research results indicated that when evaluation using BSC is conducted, evaluators tend to pay more attention to common financial measures than unique financial measures as well as common non-financial measures versus unique non-financial measures.
The results of this research are contrary to the theory proposed by Kaplan and Norton (1992) as the inventor of the BSC, namely that the BSC does not only focus on financial measures but also non-financial measures so as to provide a more comprehensive view of a company to act in accordance with its long-term goals and have the main strength on unique measures in each perspective. If the manager assesses performance with common measures only, it creates a bias for the usefulness of the BSC because the amount of implementation costs including management time is not comparable to the usefulness of the BSC. Rather, the BSC does not cause any benefit (Kaplan and Norton, 1996).
The gap between the BSC theory and the findings of some of the results of the study may be explained by using DeBusk, Killough and Brown (2010) findings, which stated that evaluators who are not tolerant of ambiguity will be more likely to seek additional information to reduce the threat of ambiguity from existing data (DeBusk, Killough and Brown (2010). The existence of a gap between the results of some of these studies with BSC theory can be expected because the research is only at the stage of object evaluation, namely the perspective and type of BSC measures related to performance evaluation. IOA performance evaluators are treated as control variables.
This research limits the combined effect of IOA, perspective and type of BSC measures on the results of performance evaluation, including: the effect of IOA interaction with the BSC perspective on the results of performance evaluation, the effect of IOA interaction with the type of BSC measures on the results of performance evaluation, the effect of perspective interaction with type of BSC measures on the results of performance evaluation. This is in line with the opinion of Gelderman (1998) who stated that the performance measurement system is determined by the interaction between two variables and the three variables, namely the evaluation context, the evaluation object and the evaluator as well as each variable. The results of studies which revealed the effect of interaction between independent variables on the dependent variables were not found. Whereas in practice, performance evaluation activities involve the interaction of several independent variables on the dependent variable. The performance measurement system design is determined by what is economically desired by the organization. The system is the basis for building hypotheses that relate between IOA (evaluator), perspective and type of BSC measures (object of evaluation) with performance evaluation.
HYPOTHESIS DEVELOPMENT
The Effect of Interaction between IOA with BSC Perspective on Performance Evaluation. Differences in the characteristics of evaluators of organizational performance have various effects in providing performance evaluation. When making decisions, evaluators who are intolerant of ambiguity will tend to seek additional information to reduce the threat of ambiguity from existing data or the absence of overall data (DeBusk, Killough and Brown, 2010). Information in the BSC can be grouped related to financial and non-
financial perspectives (Kaplan and Norton (1992)). Gelderman (1998) stated that the characteristics of evaluators, evaluation objects and evaluation context and interaction of the three affect the performance evaluation system. Its tolerance for ambiguity with evaluation object, namely the BSC perspective has an effect on the results of organizational performance evaluation. Thus, based on this, hypothesis can be stated as follows:
Ha1: There is an effect of IOA interaction with BSC perspective on performance evaluation.
The Effect of Interaction between IOA with Type of Measures on Performance Evaluation. When making decisions, performance evaluators that are not tolerant of ambiguity tend to seek additional information to reduce the threat of ambiguity from existing data or the absence of overall data (DeBusk, Killough and Brown, 2010). The strength of the BSC is on unique performance measures at each perspective. If the performance evaluator uses only common performance measures, there are wasted resources and the BSC that does not provide benefits (Kaplan and Norton, 1996). Gelderman (1998) stated that the characteristics of the evaluators, evaluation object and evaluation context and the interaction of all three affect the organizational performance evaluation system. Thus the interaction between the characteristics of the evaluator (the level of tolerance for ambiguity) with evaluation object (type of BSC measures) has an effect on the results of organizational performance evaluation. Based on this, the following hypothesis can be stated:
Ha2: There is an effect of IOA interaction with the type of BSC measures on performance evaluation.
The Effect of Interaction between Perspective with Type of BSC Measures on Performance Evaluation. Kaplan and Norton (1996) argued that: "If it can measure it, you can manage it". The statement is the rationale for measuring all activities grouped into four perspectives in the BSC, namely finance, customers, internal business processes, growth and learning. Types of performance measures can be classified as common and unique. Performance measures for qualitative activities tend to be common statements that give rise to subjective assessment potential while quantitative performance measures are detailed and more objective. The BSC has the main strength in unique measures in every perspective. If the manager assesses performance with a common measure only, it will create a bias for the usefulness of the BSC. This is due to the large amount of implementation costs including management time that is not comparable to the usefulness of the BSC. Rather, the BSC does not cause any benefits (Kaplan and Norton, 1996).
Object evaluation affects the performance evaluation system (Gelderman, 1998). BSC has a financial and non-financial perspective with a common and unique type of measures. If the evaluator is provided with an evaluation object in the form of financial perspective information compared to a non-financial perspective, the results of the performance evaluation can be different. Likewise, if common performance measures is provided compared to unique performance measures, the results of the performance evaluation can be different. Thus the perspective and type of BSC measures have an influence on performance evaluation. Based on this thought, there is an effect between perspective interaction and type of BSC measures on performance evaluation. Based on the description, the following hypothesis can be formulated.
Ha3: There is an effect of interaction between IOA, perspective and type of BSC measures on performance evaluation.
METHODS OF RESEARCH
This research used quantitative approach. The type of the research was experimental using factorial design. With factorial design, it allowed researchers to test the effects of two or more manipulations at the same time on the dependent variable (Sekaran and Bougie, 2013: 195). This research used a factorial design of 2x2x2, as shown in Table 1. Independent variables which were attribute variables (controlled) were IOA with two high and low treatment levels. Manipulated independent variables were experimental variables, namely perspective variables with two levels of treatment, namely financial and non-financial,
and variable types with two levels of treatment, namely common and unique. The dependent variable was performance evaluation.
Table 1 - Research Experiment Design
IOA Perspective
Financial Non Financial
Common Unique Common Unique
High Group A Group B Group C Group D
Low Group E Group F Group G Group H
Source: Processed by authors.
To find out IOA performance evaluators, the researchers used the Budner scale. According to DeBusk, Killough and Brown (2010), the Budner scale was most often used to determine the level of intolerance of ambiguity because it had the best level of validity and reliability compared to other scales. The score on Budner's scale was determined by giving a value to each participant's response item. Statement items with answer choices were provided in the form of a 1 to 7 Likert scale. If the statement item was answered by the participant by crossing the number "1", then the score for the item statement was 7, if the number "2" was selected, then the score for the statement item was 6, and so on. The higher score indicated the higher tolerant of ambiguity or lower IOA.
The other two independent variables were perspective and type of measures. The BSC perspective was grouped into two, namely financial and non-financial. Details for non-financial perspectives included customer perspective, internal business processes, learning and growth. Types of performance measures could be divided into common and unique measures. Performance measures that were common in nature, gave orientation as an outcome while performance measures that were unique in nature, contained a combination of outcome-oriented measures and outcome triggers (Cardinels and Venn-Dirks, 2010). The dependent variable of this research was organizational performance evaluation. Measurement of performance evaluation was done by giving a performance assessment score. Then the scores of each group were compared.
The IOA instrument was adopted from the Budner scale in DeBusk, Killough and Brown (2010). The BSC perspective instrument was sourced from Kaplan and Norton (1996). Instruments of type of BSC was adopted from Cardinels and Venn-Dirks (2010), which was adjusted to the accreditation assessment guidelines of the Indonesian National Accreditation Board. Performance evaluation instruments were adopted from Bone and Solihin (2012).
Research location was at two private universities located in Jakarta. Data collection was done in even semester of 2016/2017 academic year up to odd semester of 2017/2018 academic year. The research population was all heads in two Jakarta private universities that applied BSC and offerred strata study programs 1. To find out about private universities that implemented BSC, it was done by using a google search engine search and was confirmed through interviews with the heads of study programs. Respondents who became participants in the research were 128 people. Gay and Diehl (1992: 146) argued that to be representative and the results can be derived, the sample must be as large as possible and for experimental research, the minimum sample was 15 subjects per group. The sample determination method used was random sampling and data collection was conducted using questionnaires.
The experimental steps began by asking participants to fill out personal data and read the case to do the experimental work. The experimental assignment contained Budner's questionnaire and the case of study program performance evaluation. In Budner's questionnaire, participants were asked to fill in the answers according to their description. Participants were asked to cross the numbers listed under the statement which consisted of 16 items. There were numbers 1 through 7 that described the participant's response to each statement. Number 1 represented "strongly disagree" and number 7 represented "strongly agree". Furthermore, participants were grouped into 8 and asked to work on a case of study program. In that case, participants were described as faculty/high school heads who evaluated division performance (study programs). Response to performance evaluation was
measured on a scale of 0-100. The score range category was as follows: 0-50 = bad, 51-75 = enough, 76-90 = good, 91-100 = very good.
Tasks for each group were the same. It's just that for groups A, C, E, G, got common performance measures, while groups B, D, F, H got unique performance measures. Group A had the characteristics as performance evaluators who were not tolerant of ambiguity was provided with common performance measures for financial perspectives while group C was provided with common performance measures for non-financial perspectives. Group E as performance evaluators who were tolerant of ambiguity was provided with common performance measures for financial perspectives while group G was provided with common performance measures for non-financial perspectives. Group B had the characteristics as performance evaluators who were not tolerant of ambiguity was provided with unique performance measures for financial perspectives while group D was provided with unique performance measures for non-financial perspectives. Group F as performance evaluators who were tolerant of ambiguity was provided with unique performance measures for financial perspectives while group H was provided unique performance measures for non-financial perspectives.
After participants evaluated the performance of the study program, a manipulation check was conducted. If the participant provided performance evaluation (score) and opinion, then the performance evaluation (score) was used. If the participants only assessed one, then they were considered a failure as a participant. If the participant passed the evaluation phase, then a manipulation check was conducted to find out the participants' understanding of their position as evaluators. The IBM SPSS Statistics 24 program was used to process data using the three-way ANOVA method.
RESULTS AND DISCUSSION
Based on the data processing for the intolerance of ambiguity questionnaire and 4 sets of completed performance evaluation cases, the distribution of the number of research participants in eight cells in Table 1 shows group A: 17, group B: 17, group C: 17, group D: 16, group E: 15, group F: 15, group G: 15, and group H: 16 participants. Thus the total of research participants were 128 people, ranging from the head of the study programs to the head of the rectors.
From the answers to the performance evaluation scores given by each participant, descriptive statistical processing tailored to the purpose of the research was carried out and the results of the mean performance evaluation can be seen in Table 2. The effect of interaction between IOA*Perspectives in descriptive statistics showed that the interaction of evaluators who had low IOA that were provided by financial perspectives provided the highest mean score of performance evaluation, namely 80.5667 followed by low IOA with non-financial perspectives, high IOA with financial perspectives, and the lowest was high IOA with non-financial perspectives with a mean performance evaluation score of 65.6061.
Table 2 - Mean and Standard Deviation of Performance Evaluation Value Based on IOA, Perspective
and Type of BSC measures
Descriptive Statistics; Dependent Variable: Performance evaluation
IOA Perspective Type of measures Mean Std. Deviation N
Low Financial 80.5667 7.95974 30
Non-Financial 76.0968 5.81008 31
High Financial 73.4706 4.84446 34
Non-Financial 65.6061 5.08023 33
Low Unique 83.2903 6.01772 31
Common 73.1333 4.13341 30
High Unique 71.7879 6.05061 33
Common 67.4706 5.90960 34
Financial Unique 81.1250 7.37367 32
Common 72.4688 4.13470 32
- Non-Financial Unique Common 73.5938 67.7813 7.57696 6.41940 32 32
Source: SPSS Output.
Interaction of IOA*Type of measures in Table 2 shows 4 mean scores for performance evaluation. It turned out that evaluators who had low IOA who were provided with unique measures information gave a highest mean score for performance evaluation, then followed by low IOA with common measures, high IOA with unique measures and the lowest were evaluators with high IOA who were provided with common measures by giving a mean score of performance evaluation of 67.4706.
Interaction of Perspective*Type of measures in Table 2 shows the financial perspective with unique measures was given the highest performance evaluation score of 81.1250 by the performance evaluator. The order of mean score of the next evaluation performance was a non financial perspective with unique measures, a financial perspective with common measures, and the lowest mean score of performance evaluation was a non-financial perspective with common measures information with a score of 67.7813.
Of the three interactions, the greatest mean score of interaction of performance evaluation was a low IOA with unique measures with a mean score of 83.2903. On the contrary, the lowest mean score of interaction of performance evaluation was a high IOA with a non-financial perspective with a score of 65.6061.
Standard deviation indicates risk, namely the possibility of not obtaining the mean score of performance evaluation. The higher the standard deviation, the more risky in the sense that the higher the probability of not getting the mean score of performance evaluation. When viewed from the standard deviation, the lowest was the performance evaluator who had a low IOA who were provided with common measures information with a standard deviation of 4.13341. Conversely, evaluators who had low IOA who were provided financial perspective information had the highest standard deviation of 7.95974.
After the IOA questionnaire passed the validity and reliability test and the performance evaluation data passed the normality and homogeneity test, data processing was carried out. The results can be seen in Table 3. From the table, it showed that the interaction of IOA*Perspective produced Sig. 0.044 which was lower than 0.05, which meant that Ha1 was accepted. Thus there was an effect of interaction between IOA evaluators and the BSC perspective in determining the performance evaluation score. Next was interaction of IOA*Type of BSC measures which showed Sig. 0.00, which was smaller than 0.05, it meant that Ha2 was accepted. There was an effect of interaction between IOA evaluator with the type of BSC measures in determining performance evaluation scores. Table 3 also showed the interaction of Perspective*Type of measures with Sig. 0.032, which was lower than 0.05. It meant that Ha3 was accepted. This proves that evaluators were influenced by the information provided in the form of perspective interaction with the type of BSC measures when the evaluators provided a performance evaluation score.
Table 3 - ANOVA Test Results
Tests of Between-Subjects Effects; Dependent Variable: Performance evaluation
Source Type III Sum of Squares Df Mean Square F Sig.
Corrected Model 5821.170a 7 831.596 41.836 .000
Intercept 697417.845 1 697417.845 35085.472 .000
IOA * Perspective 82.715 1 82.715 4.161 .044
IOA * Type of size 295.532 1 295.532 14.868 .000
Perspective * Type of size 93.189 1 93.189 4.688 .032
Error 2385.322 120 19.878
Total 704259.000 128
Corrected Total 8206.492 127
a. R Squared = .709 (Adjusted R Squared = .692)
To determine the direction of the interaction effect between the independent variables on the dependent variable, it can be seen on the interaction plot. The data of mean of performance evaluation shown in Table 2 was used as a basis for making plots presented in Figure 1. Based on Figure 1, the lower the intolerance of ambiguity, the performance evaluators increasingly provide higher performance evaluation scores. Effects of interaction between IOA*Perspective showed that performance evaluators who had low intolerance of ambiguity were provided with information on the type of BSC measures with unique
measures, gave the highest performance evaluation scores, while performance evaluators who had low intolerance of ambiguity were provided with information on the type of BSC measures with common measures gave the lowest score of performance evaluation. Performance evaluators who had low intolerance of ambiguity with financial perspective information provided a higher performance evaluation score than those provided by non-financial information.
90
80
70
60
50
Ï5 40 t^ > O LU 30
<u o c ro
E c
o .2 <D 5
CL
CO <U
20 10 0
Figure
Financial Perspective
Non Financial Perspective Unique Measure
■Common Measure
Low IOA High IOA
1 - Interaction Plots of IOA*Perspective and IOA*Type of measures
85 80
e c
a
mn f 75
S %
CL
LU
a e
Unique Measure Common Measure
70 65 60
Financial Perspective Non Financial Perspective Figure 2 - Interaction Plots of Perspective*Type of measures
To determine the effect of interaction between Perspective*Type of BSC measures, it can be seen on Figure 2. Based on the figure, the direction of interaction of financial perspectives with unique measure had a highest effect on the mean score of performance evaluation compared to the interaction of financial perspectives with common measure. Interaction between financial perspectives with unique and common measures had a higher effect on performance evaluation scores than interaction between non-financial perspectives with unique and common measures.
Paying attention to Table 1 and Figures 1 and 2, three combinations of interactions, namely IOA*Perspective, IOA*Type of measures, and Perspective*Type of measures showed that the interaction between low IOA and unique measure had a highest effect on the mean score of performance evaluation, followed financial perspective with unique measure, IOA with financial perspective. The interaction between high iOa and non-financial perspective gives the lowest effect in determining the mean score for performance evaluation.
DISCUSSION OF RESULTS
The Effect of Interaction between IOA*BSC Perspective on Performance Evaluation. The results of the research in Table 3 prove that the interaction of IOA*BSC perspective had an effect on performance evaluation. Based on Figure 1, the direction of the interaction effects for the financial and non-financial perspective lines from the top right to the bottom
right. This shows that the direction of the effect of interaction between IOA*BSC Perspective, namely the interaction of low IOA with financial perspective had the highest effect on performance evaluation followed by the interaction of low IOA with non-financial perspectives, high IOA with financial perspectives, and the lowest effect of interaction in performance evaluation was interaction between high IOA with a non-financial perspective.
Budner (1962) supported the possible relationship between intolerance of ambiguity (IOA) and the use of information. Dermer (1973) stated the possibility that cognitive characteristics of information users (evaluators) can influence perceptions about important information and therefore can influence how information affects behavior. Liedtka, Church and Ray (2008) stated that evaluators who are not tolerant of ambiguity reject ambiguous information. DeBusk, Killough and Brown, 2010 stated that evaluators who are not tolerant of ambiguity (high IOA) when making decisions will be more likely to seek additional information to reduce the threat of ambiguity. In the BSC, organizational performance is assessed from four perspectives, namely finance, customers, internal business processes, and learning and growth (Kaplan and Norton, 1992). In this research the last three perspectives are grouped as non-financial perspectives. Gelderman (1998) stated that the interaction between evaluator characteristics, evaluation object and evaluation context influence the performance evaluation system. Thus the results of this research supported the opinions of the authors and researchers. Interaction of characteristics of evaluators in the form of IOA evaluators with evaluation object in the form of BSC perspectives had an effect on performance evaluation.
The Effect of Interaction between IOA*Type of BSC measures on Performance Evaluation. Table 3 showed that IOA*Type of BSC measures had an effect on performance evaluation. Figure 1 gave the direction of the interaction effect for the line of unique measures and common measures, namely from the top right to the bottom right. The figure showed the direction of the effect of interaction between IOA* Type of BSC measures was interaction between a low IOA with a unique measures that has the highest performance evaluation followed by the interaction of low IOA with common measures, high IOA with unique measures, and the lowest interaction effects in performance evaluation was the interaction between a high IOA with the common measures.
To reduce the negative impact of ambiguity, evaluators who had a high IOA required additional information when evaluating performance (DeBusk, Killough and Brown, 2010). If additional information obtained was inadequate, the evaluator would provide a bad evaluation results and vice versa. Types of performance measures in the BSC can be distinguished in the form of unique and common performance measures (Kaplan and Norton, 1992). In the BSC, all measures were relevant to the strategy (Kaplan and Norton, 1996). Performance measures information that were common in nature, provided a common orientation as an outcome, While performance measures that were unique, contained a combination of outcome-oriented measures and outcome triggers (Cardinels and Venn-Dirks, 2010). The results of this research were in line with the opinions of some of these experts and supported the opinion of Gelderman (1998) that the interaction between the characteristics of evaluators in the form of IOA evaluators and evaluation object, namely the type of BSC measures, had an effect on performance evaluation.
The Effect of Interaction between Perspective*Type of BSC measures on Performance Evaluation. Table 3 showed that the interaction between Perspective*Type of BSC measures had an effect on performance evaluation. Figure 2 clarified the direction of interaction effects for unique and common measures lines, namely from the top right to the bottom right. This shows the direction of the effect of interaction between Perspective*Type of BSC measures is the interaction of financial perspectives with unique measures that have the highest effect on performance evaluation followed by the interaction of financial perspectives with common measures, non-financial perspective with unique measures, and the lowest effect of interaction in performance evaluation was the interaction of non-financial perspectives with common measures. The results of this research clarified the research position of Ittner, Larcker and Meyer (2003) which proved that there was a subjectivity in determining the relative weight of measures used in evaluation because there was a tendency to bias from a
financial perspective for common performance measures. Likewise, the results of this research clarified the position of the results of Bone and Sholihin (2012), namely, when evaluating using BSC, evaluators paid more attention to the common measures of financial perspective than unique measures of financial perspective and common measures of non-financial perspective compared to unique measures of financial perspective.
According to Kaplan and Norton (1996), the BSC asked managers to view business from four important perspectives, namely financial perspective, customer perspective, internal business process perspective, and learning and growth perspective. Financial measures in the BSC indicated the results of actions taken by the company and the BSC complemented financial measures with operational (non-financial) measures on customer satisfaction, internal processes and innovation, and the development of company activities (Hoque, 2006). If the performance evaluator used only common performance measures, there were wasted resources and the BSC did not provide benefits (Kaplan and Norton, 1996). Unique performance measures, containing a combination of outcome-oriented measures and outcome triggers (Cardinels and Venn-Dirks, 2010). The results of this research proved to support the opinions of these experts and supported the opinion of Gelderman (1998) that the interaction between perspective and type of measurement which was evaluation object had an effect on performance evaluation.
CONCLUSION
Based on the description, the answers to the research problem formulation can be stated as follows:
There is an effect of interaction between the IOA evaluator and the BSC perspective on the results of performance evaluation. The interaction of differences in the characteristics of performance evaluators is in the form of low and high tolerance levels; with the difference in information provided to performance evaluators in the form of financial and non-financial perspectives in the BSC empirically has an effect on differences in performance evaluation results. Evaluators who have low intolerance of ambiguity when provided with financial perspective information provide the highest evaluation results and evaluators who have high intolerance of ambiguity when provided with non-financial perspective information provide the lowest evaluation results.
There is an effect of interaction between the IOA evaluator and the type of BSC measures on the results of performance evaluation. The interaction of differences in the characteristics of performance evaluators is in the form of low and high tolerance levels, with the difference in information provided to performance evaluators in the form of unique measures and common measures in the BSC empirically has an effect on differences in performance evaluation results. Evaluators who have low IOA when provided with unique measures information provide the highest evaluation results and evaluators who have high IOA when provided with common measures information provide the lowest evaluation results.
There is an effect of interaction between perspective and type of BSC measures on the results of performance evaluation. Interaction from different perspectives is in financial and non-financial forms, with differences in information provided to performance evaluators in the form of unique measures and common measures in the BSC empirically have an effect on the difference in performance evaluation results. Evaluators who provide financial perspective information with unique measures provide the highest evaluation results and evaluators who provide non-financial perspective information with common measures provide the lowest evaluation results.
Based on these conclusions, the results of this research have practical and academic implications so that there are some suggestions that can be proposed.
For organizational management that applies BSC, when evaluating performance, it is necessary to consider the interaction between evaluators, namely the IOA and evaluation object, namely the perspective and type of BSC measures. Performance evaluators who has low IOA which are provided with unique information type provides the highest performance
evaluation results. If this is done, it will make it easier for the organization to have good performance and can influence organizational behavior that tends not to work extra hard. Conversely, if the performance evaluators chosen have a high IOA, a non-financial perspective will be provided which will result in a low performance evaluation. If this is done, it will make the organization more difficult to have high performance and can influence organizational behavior which tends to be demanded to work extra hard. Which choice is the best depends on the performance evaluation system that is in accordance with the goals of the organization.
This research did not reveal the interaction between IOA*Perspective*Type of measures. For academic purposes, the next research can be done to find out the best effect of interaction between the three independent variables on performance evaluation. Furthermore, the characteristics of evaluators can also be developed again such as locus of control, value orientation and others. Likewise, research participants can also be grouped based on the homogeneity of the position, for example only the heads of the study programs. Thus if this is done in the next research, it will be able to increase its contribution to the organizational performance evaluation system.
REFERENCES
1. Anand, M., Sahay, B. S., & Saha, S. (2005). Balanced Scorecard in Indian Companies. Vikalpa, 30(2), 11-26. doi:10.1177/0256090920050202
2. Bone, H., & Solihin, M. (2012). Pengaruh Perspektif dan Jenis Ukuran dalam Balanced Scorecard terhadap Evaluasi Kinerja. Ekuitas: Jurnal Ekonomi dan Keuangan, 16(December), 4th ser., 457-472.
3. Budner, N. Y. (1962). Intolerance of ambiguity as a personality variablel Journal of Personality, 30(1), 29-50. doi:10.1111/j.1467-6494.1962.tb02303.x.
4. Cardinaels, E., & Veen-Dirks, P. M. (2010). Financial versus non-financial information: The impact of information organization and presentation in a Balanced Scorecard. Accounting, Organizations and Society, 35(6), 565-578. doi:10.1016/j.aos.2010.05.003
5. Dermer, J. (1973). Cognitive Characteristics and The Perceived Importance of Information. The Accounting Review, 48, 4th ser., 511-519.
6. Dilla, W. N., & Steinbart, P. J. (2005). Relative Weighting of Common and Unique Balanced Scorecard Measures by Knowledgeable Decision Makers. Behavioral Research in Accounting, 17(1), 43-53. doi:10.2308/bria.2005.17.1.43
7. Gay, L.R. & Diehl, P.L. (1992). Research Methods for Business and Management. New York: Macmillan.
8. Gelderman, M. (1998). Usage of Performance Measurement and Evaluation Systems: the Impact of Evaluator Characteristics. Retrieved July 13, 2011, from http://proc.isecon.org
9. Hoque, Z. (2006). Strategic Management Accounting: Concepts, Processes and Issues. London: Pearson Prentice Hall.
10. Indrajit, R. E., & Djokopranoto, R. (2006). Manajemen Perguruan Tinggi Modern. Yogyakarta, Indonesia: Penerbit Andi.
11. Ittner, C. D., Larcker, D. F., & Meyer, M. W. (2003). Subjectivity and the Weighting of Performance Measures: Evidence from a Balanced Scorecard. The Accounting Review, 78(3), 725-758. doi:10.2308/accr.2003.78.3.725
12. Kaplan, R. S., & Norton, D. P. (1992). The Balanced Scorecard-Measures that Drive Performance. . Harvard Business Review, 71, 1st ser., 71-79.
13. (1996). The Balanced Scorecard. Boston, MA: Harvard Business School Press.
14. Libby, T., Salterio, S. E., & Webb, A. (2004). The Balanced Scorecard: The Effects of Assurance and Process Accountability on Managerial Judgment. The Accounting Review, 79(4), 1075-1094. doi:10.2308/accr.2004.79.4.1075
15. Lipe, M. G., & Salterio, S. E. (2000). The Balanced Scorecard: Judgmental Effects of Common and Unique Performance Measures. The Accounting Review, 75(3), 283-298. doi:10.2308/accr.2000.75.3.283
16. Polla, G. (2007). Annual Report Jan-Dec 2007 Binus University. Retrieved February 16, 2017, from http://cms.binus.edu/datapage/file/BINUSAnnualReport2007-Final-Summary.pdf
17. Pramono. (2009). Analisis Strategi Pengembangan Sumber Daya Manusia dan Penerimaan Teknologi Informasi terhadap Pencapaian Kinerja Perguruan Tinggi: Suatu Model Pendekatan Balance Scorecard pada Universitas Tarumanagara. Jurnal Pendidikan, 1, Tahun XXVIII.
18. Roberts, M. L., Albright, T. L., & Hibbets, A. R. (2004). Debiasing Balanced Scorecard Evaluations. Behavioral Research in Accounting, 16(1), 75-88. doi:10.2308/bria.2004.16.1.75
19. Sekaran, U, & Bougie, R. (2013). Research Methods for Business, Six Edditions. United Kingdom: John Woley & Sons Ltd.
