CC BY
73
Perspectives of Science & Education
International Scientific Electronic Journal ISSN 2307-2334 (Online)
Available: https://pnojournal.wordpress.com/2022-2/22-02/ Accepted: 29 December 2021 Published: 30 April 2022
M. Fatchurahman, M. A. Setiawan, K. Karyanti
Evaluation of the CSE-UCLA model on the performance of school counselor in Indonesia
The problem and the aim of the study. Evaluation is an attempt to review whether the program follows the objectives and achieves the expected goals. The evaluation of the UCLA CSE model was chosen because this model is very suitable for evaluating ongoing Education programs. This study aims to find out how the performance of counselors in schools using the UCLA SCE evaluation model.
Research methods. This research is evaluation research with five stages: the assessment system, program planning, implementation program, Program Improvement, and Certification Program. The research subjects were 180 people divided into 150 students and 30 counselors from state high school 1 Palangkaraya, state high school 1 Kuala Kapuas, state high school 1 Sampit, state high school 1 Katingan, State Senior High School 1 Pangkalan Lada. The research instruments used in this study were interviews, questionnaires, observation, and documentation. This research instrument was used to obtain data related to the performance of counselors in Indonesian schools. Data analysis was carried out in-depth by analyzing the percentage of the questionnaire results that the research subject had filled in.
Results. The results showed that the guidance and counseling services had been running in a good category. The analysis results show that the evaluation of the UCLA CSE model carried out shows that every aspect gets a score with good criteria. The assessment aspect gets a score (72.40), the planning program (69.20), program implementation (70.00), improvement program (70.16), and program certification 73.20 if. The average evaluation result is (70.99) good category. The research findings show that the results of the very good category have not been found, so school counselors need to improve the quality of service to achieve the very good category. Excellent service will help school counselors gain the trust of students and parents.
Conclusion. Evaluation of the program using the UCLA CSE model conducted in several schools in Indonesia shows that the school's guidance and counseling service program has been running in a good category. Support from the system needs to be strengthened to recognize guidance and counseling in schools. School counselors need to socialize guidance and counseling services so that students become more confident and confident in the existence of guidance and counseling. The government also needs to provide a strategic position of guidance and counseling in the world of education so that the existence of school counselors is increasingly recognized and has an important role in the world of education.
Keywords: evaluation, CSE-UCLA model, school counselor performance, service program
For Reference:
Fatchurahman, M., Setiawan, M. A., & Karyanti, K. (2022). Evaluation of the CSE-UCLA model on the performance of school counselor in Indonesia. Perspektivy nauki i obrazovania - Perspectives of Science and Education, 56 (2), 561-572. doi: 10.32744/pse.2022.2.33
_Introduction
NESCO is a United Nations organization engaged in education and culture that has
v___) proclaimed four pillars of education now and in the future, which need to be developed
by all educational institutions, especially formal education institutions. The four pillars are: (1) learning to know, (2) learning to do, (3) learning to be, and (4) learning to live together. The application of these four pillars is very important in facing the era of globalization and the industrial era 4.0. It is necessary to cultivate an attitude of mutual understanding between races, ethnicities, and religions not to cause various conflicts that originate from these things. The applied education must also follow the community's needs or the needs of the area where the education is held so that the elements of local content that are developed must be following the local area's needs. Creating a good quality of education requires cooperation from various parties. One element of education that plays a role in shaping students is guidance and counseling. A Guidance and Counseling teacher or school counselor carries out his duties as a social worker to realize quality education. Without counseling guidance in schools, the education process will not run optimally. Therefore, school guidance and counseling are the main foundations for informing and producing quality education. Guidance and counseling teachers in schools are basically: (1) as facilitators who bridge to achieve maximum learning outcomes; (2) helping students identify themselves in determining their future career and planning the goals of these students in the future; (3) overcome students' personal, social, learning, and career problems.
Guidance and counseling are familiar in education, from the most basic to the highest level. Guidance and counseling are one part of the system that supports the education process in schools to run well. Guidance and counseling is a service aimed at assisting a person or group in reaching an understanding or solving a problem [1; 2]. To become a guidance and counseling teacher requires special qualifications that must be met; in this case, the special qualifications for counseling guidance teachers are clearly stated in government regulations regarding the qualification standards for S1 Counseling Guidance. The regulation is Government Regulation No. 27 of 2008, which explains a counselor's competencies. The competence and qualifications of this counselor will provide a clear picture of the counselor's performance in the future. Performance is influenced by two factors: from within oneself and outside [3]. The performance will measure how far the counseling guidance teacher can provide quality services to students. This quality is clearly described in government regulation No. 27 of 2008, where counselors must have competence, personality, professional, social and pedagogic.
In general, the performance of counseling guidance teachers can be seen from providing guidance services to students. Performance is a form of organizational orientation or achievement. Performance cannot be separated from leadership, where leadership is one of the determinants of a more positive environment [4]. The direction of a leader will form the spirit of leadership an loyalty and lead to the performance of subordinates [5]. Leaders can direct subordinates to have measurable performance [6].
Performance can achieve a predetermined program individually and in groups [7]. The performance of the guidance and counseling teacher can be observed by paying attention to the services provided; good service will indicate that the guidance and counseling teacher has a good performance. Besides that, the performance of the Guidance Counseling teacher
can also be observed by paying attention to the existing management. Management will act as a bridge so that the structured program can be implemented properly.
Counselors have a broad role not only in the scope of the school. Extension workers can be involved with the community environment that needs cooperation because extension workers bridge the community [8]. Performance cannot be separated from the evaluation process, where evaluation is usually carried out systematically and logically in a system [9]. The system plays a role in achieving the expected goals [10]. Evaluation cannot be separated from the evaluator, but the evaluator has a limited role [11]. Evaluation is part of a system [12; 13]. Evaluation is one of the important things to do. Evaluation is aimed at finding deficiencies and correcting existing deficiencies [14]. This concept is reinforced by article reviews [15]. The evaluation will create quality activities, so it can be understood that evaluation is an important part that must be done. The results of previous studies show that evaluation is a measure of satisfaction [16]. Several studies also consider that evaluation is vital and must be carried out. Evaluation provides a clear picture of the purpose of a system [17]. Evaluation provides clear policy directions[18]. Evaluation will guide so that goal are achieved effectively [19; 20]. Evaluation directs the movement of knowledge development [21]. Evaluation is the main component to achieving success [22].
Findings in several schools in Indonesia show that the performance of guidance and counseling teachers is still questionable. There are still many guidance and counseling teachers who do not meet the qualification standards, namely guidance and counseling undergraduate graduates; in the field, some guidance and counseling teachers are subject teachers who double as guidance and counseling teachers. This condition will certainly greatly affect the performance of a guidance and counseling teacher at school. Another requirement to become a counselor is to have a professional certificate, but most guidance and counseling teachers do not fully have the professional certificate in question. Counseling services will not meet the standards if the scientific background is not appropriate, then the direction of counseling services will certainly be unclear. Seeing the condition of counseling services in schools in Indonesia, an evaluation process is needed to see the extent of the services provided by counselors in schools. One alternative evaluation that can be done is by using the evaluation model of the Center for the Study of Evaluation-University of California in Los Angeles (CSE-UCLA). Alkin introduced the CSE-UCLA model by emphasizing five main aspects, namely (1) the assessment system is an evaluation component that is used to see the initial conditions of the program being evaluated, (2) program planning is an evaluation component that is used to see things that might be included to meet program needs, (3) program implementation is an evaluation component that is used to see the form of implementation of activities carried out in order to introduce the program to its users, (4) program improvement is an evaluation component that is used to see an overview of program performance/functions, and certification program (5) (is an evaluation component used to see an overview of the benefits and value of the program [23]. The basis for the researcher uses the CSE-UCLA model because this model is very suitable for evaluating ongoing education programs. This opinion aligns with the concept put forward by pen research [24], which states that the CSE-UCLA model is suitable for assessing the educational process [25]. Based on the problems above, researchers are interested in finding out how the performance of guidance and counseling teachers in planning and implementing guidance and counseling service programs through the CSE-UCLA model in schools in Indonesia.
_Materials and methods
The type of this research is program evaluation research with the evaluation design used is CSE-UCLA. This study aims to measure the success of a particular program, product, or activity. This research aims to assess an activity program's benefits, usefulness, contribution, and feasibility from a particular unit/institution. Epistemologically, collecting data uses an objective approach because it uses instruments in interviews, documentation, observations, and questionnaires. This model has five stages, namely: (1) System Assessment, (2) Program Planning, (3) Program Implementation, (4) Improvement Programs, (5) Certification Program.
Table 1
UCLA CSE evaluation research model
System assessment Vision, mission, goals, legality, strategy, and support for Guidance and Counseling program services in schools
CSE UCLA Program Planning Teacher readiness and ability, team readiness, organizational structure, infrastructure, and funding for counseling guidance services
Program Implementation Quantity and quality of Guidance and Counseling program services
Program Improvement Guidance and Counseling Services both classical, individual, and group
Program Certification Quality of tangibility, reliability, responsibility, assurance, and empathy
Research subject
This study's data sources are: (1) Respondents were counselors and students (2). The research subjects were taken from several schools in Indonesia, namely in senior high schools in Indonesia, namely state high school 1 Palangkaraya, state high school 1 Kuala Kapuas, state high school 1 Sampit, state high school one Katingan, State Senior High School 1 Pangkalan Lada. Overall, the subjects of this study amounted to 180 people, with the distribution of 150 students and 30 counselors.
Research data collection
Data collection in this study combines several data collection tools to obtain data objectively. The research data obtained were dug up with several instruments, namely interview instruments, documentation, observations, and questionnaires [26]. This research will use qualitative and quantitative methods with data sources using the school environment. Interviews were used to analyze the achievement of the guidance and counseling program in schools. The document in question is data on counseling service programs, especially services that counselors in schools have carried out. Questionnaires were used to explore data obtained from students regarding the guidance and counseling service program implemented in schools. The focus of monitoring and evaluation based on the stages of monitoring and evaluation of CSE-UCLA is stated as follows: (1). Need assessment includes the stages of focusing on determining the problem, (2). Assessment of Program Planning data collected during the assessment stage is intended for decision-making objectives that have been formulated (3). Formative evaluation focuses attention on program implementation. This assessment is directly related to implementing activities, guidance activities, and media use (4): Summative Evaluation Assessment, the collection of all data about the results and impacts of the program. Through summative evaluation, it is
hoped that the objectives that have been implemented will be known and, if not, how the efforts will be carried out and find the cause.
Research data analysis
Descriptive analysis is a statistic that functions to describe or provide an overview of the object under study through sample or population data as it is, without analyzing and making conclusions that apply to the public [27]. Analysis of data on the implementation of guidance and counseling service programs in senior high schools in Indonesia by looking at the components of the Assessment System, Program Planning, Program Implementation, Program Improvement, Program Certification using quantitative descriptive analysis measuring tools. For the constraints found in implementing this program using a qualitative descriptive analysis. The steps involved in evaluating this research can be explained as follows: 1) Primary data analysis is carried out by analyzing the data obtained from the results of filling out questionnaires from users of the guidance and counseling service program, namely students, including a) Calculating the average score obtained for each evaluation and convert it to classification and category using a scale of five based on the ideal average (Mi) and ideal standard deviation (SDi) namely System Assessment, Program Planning, Program Implementation, Program Improvement, Program Certification using quantitative descriptive with theoretical ideal criteria which are grouped into five categories on a five scale.
Research results
The results of the analysis of the CSE-UCLA evaluation in several Muhammadiyah secondary schools can be obtained from the data on each aspect, namely as follows: Components of the assessment system.
Table 2
System assessment
No Evaluation aspect Percentage Category
A1 Strategy to meet the needs of Guidance and Counseling services 88,00 Very good
A2 Media Guidance and Counseling services provided 86,00 Very good
A3 Competency of counselors in providing counseling guidance services 45,00 Less
A4 School support for Guidance and Counseling program services 60,00 Enough
A5 Community support for Guidance and Counseling services in schools 83,00 Very good
Average 72,00 Good
The results from table 1 above are the aspect of the assessment system consisting of five items with very good scores and the highest score. The evaluation aspect of number A1 with a score of 88.00%, the lowest score on A3 with 45.00%, and A4 with a score of 45.00%. The score of 60.00%. The low score on A1 is because there are still guidance and counseling teachers with non-linear educational backgrounds, and the school also does not recruit guidance and counseling teachers according to the field of science due to limited budgets from local governments. The second aspect is program planning with the following description.
Table 3
Program planning
No Evaluation aspect Percentage Category
B1 The readiness of the counselee in receiving guidance and counseling services 88,00 Very good
B2 Needs assessment according to field analysis 67,00 Good
B3 Supporting documents and media for the guidance and counseling service program is available 66,00 Good
B4 Counselors can assess programs and develop counseling service programs 58,00 Enough
B5 Availability of space in providing guidance and counseling services 67,00 Good
Average 69,2 Good
The results of the analysis of table two show that the program planning aspect on item B1 has the highest score of 88.00%, while the lowest score is on item B4 with a score of 58.00%. This condition is caused by the fact that most of the existing counselors come from educational backgrounds that are not yet appropriate, so that they do not understand how to carry out needs assessments and develop guidance and counseling service programs in schools.
Table 4
Program implementation
No Evaluation aspect Percentage Category
C1 The quantity of regular and continuous service delivery for participants 88,00 Very good
C2 Information on the availability of services provided by guidance and counseling teachers for students 57,00 Enough
C3 Introduction of guidance and counseling services in schools 60,00 Enough
C4 Introduction of guidance and counseling services in schools 68,00 Good
C5 Socialization of the provision of guidance and counseling services for students 67,00 Good
C6 Guide to introduction and understanding of guidance and counseling services 80,00 Very good
Average 70,00 Good
Table 3 analysis of program implementation got a low score on items C2 (57%) and C3 (60%), while the highest score was on item C1 (88%). The analysis found that not all schools have information about the existence of guidance and counseling services; the understanding in students is assumed that special guidance and counseling teachers are to deal with children with problems. The services in schools are only in the scope of guidance and counseling without any further understanding specifically for students.
Table 5
Improvement program
No Evaluation aspect Percentage Category
D1 Implementation of the preparation and implementation of an integrated guidance and counseling service program with a counseling guidance laboratory 80,00 Very good
D2 The program format follows the provisions 81,00 Enough
D3 Availability of support in improving guidance and counseling services 58,00 Enough
D4 Counselors participate in training activities for providing guidance and counseling services 78,00 Good
D5 Counselors take part in Guidance and Counseling program preparation training 67,00 Good
D6 Availability of service delivery offerings in different contexts 57,00 Enough
average 70,16 Good
Analysis of the aspect of the improvement program obtained data that there is only one item that gets very good criteria, namely item number D1. At the same time, the lowest item is item number D6, with 57. The analysis results found that the offer of guidance and counseling services was not delivered comprehensively in schools so that the assumption circulating was that a special counselor for handling children with problems and children who were naughty and often violated stipulations in school.
Table 6
Certification program
No Evaluation aspect Percentage Category
E1 The effectiveness of the Guidance and counseling service program 70 Very good
E2 The condition of the place where the service is provided is adequate and representative 88 Very good
E3 Adequate service room ventilation 89 Very good
E4 Adequate information on the place of service delivery 87 Very good
E5 Service materials are available and up to date 75 Good
E6 Guidance and counseling service guidelines are adequate 70 Good
E7 Counselors are easy to find When the counselee needs service assistance 60 Enough
E8 Service providers following the required competencies 60 Enough
E9 There are criticisms and suggestions for the implementation of the Guidance and counseling service program 55 Enough
E10 Some facilities support guidance and counseling program services 78 Good
Average 73,2 Good
Analysis of aspects of the certification program can be explained that in items e2, e3, e4, the criteria for scoring are very good. Aspects of the certification program for service facilities and facilities already exist in schools, but the utilization of services is not appropriate so that the impact on the quality of programs and services is less than optimal. The lowest score from the certification program aspect is at point number E7, E8, E9 with a sufficient score category. Overall, the CSE-UCLA analysis can be understood in the following table.
Table 7
Results of the UCLA CSE evaluation analysis
Evaluation component Score Criteria
System assessment 72,40 Good
Program planning 69,20 Good
Program implementation 70,00 Good
Program improvement 70,16 Good
Program certification 73,20 Good
Average 70,99 Good
The analysis of the table above shows that the CSE-UCLA analysis in the guidance and counseling service program in schools, when viewed in each aspect, has gone well. Overall, the UCLA CSE evaluation analysis results showed a good category. Implementation of guidance and counseling service programs in Indonesia are categorized into good groups. The results of the analysis of each aspect of the evaluation of the UCLA CSE model in the aspect of system assessment scores (72.0), program planning (69.20), program implementation (70.00), program improvement (70.16), and program certification (70.20). All aspects, if averaged, then get a score (70.99). Improvements are still needed in order to achieve the very good criteria. Good achievements are also due to the covid-19 pandemic, so counseling guidance teachers need adjustments in providing services to students.
Discussion
Guidance and counseling are one part of the education system. As a system, counseling guidance has a continuous flow and series to form a system intended to produce something. The expected goal of guidance and counseling in an education system is to create mentally healthy students to achieve educational goals well. An evaluation process can be carried out to see how far the goals can be achieved or not. Experts developed various evaluation forms to see the level of success of a program. One of the evaluation models that can be used to see the education program being implemented is the UCLA CSE evaluation model developed by Alkin [28].
Program evaluation using the UCLA CSE model begins with a Needs Assessment. The interviews conducted with counselors showed that student problems were explored through problem-solving tools, reports from the teacher council or homeroom teacher, and reports from classmates. From the data on students' various problems, the counselor analyzes which problems are the most dominant and have a big influence on students, so they need to get direct treatment.
The next stage is Program planning: This stage is intended to collect data directly related to guidance and counseling services and meet the needs identified in the first stage. The guidance and counseling service process is carefully evaluated at the planning stage to determine whether a lesson plan has been developed based on the needs analysis results. The program implementation stage follows the counselee's needs, who require guidance and counseling services. The improvement stage aims to see the extent of the effect of the implementation that has been carried out. Does the service provide a better change effect or just the opposite? The final stage of evaluating the UCLA CSE model is the certification program. The certification stage is when a review is carried out to acknowledge the plan's suitability, following the stated objectives.
The findings of this study are in line with research conducted by Cetin which said that evaluation had a better impact on educational change [29]. The evaluation assesses the quality of student learning outcomes to be more measurable [30]. Evaluation provides a clear picture of the sustainability of an implemented program [31]. Evaluation is part of a system that functions to provide a clear picture of the achievements of a program
being implemented [32]. This study shows that evaluation is one of the important components of a program. Evaluation will help someone in analyzing the success of a program being implemented. The questionnaire data distributed to students and school counselors shows that the evaluation aspect gets various scores. Aspects of system assessment value (720%), program planning aspects (69.20%), program implementation aspects (70.00%), program improvement aspects (70.16%) and program certification aspects (70, 20%). Each aspect meets the criteria with a good category in all aspects. From all the results of the analysis of each aspect, there are still a few components that reach the very good criteria. In the assessment system, indicators with poor scores are still found. The analysis results require special attention so that the counselor's performance can be maintained properly. Various collaborations are needed in a system to achieve the goals properly.
Several obstacles are found in carrying out evaluation research using the CSE-UCLA model, including the following. (1) On the component of the assessment system, the support system is still very weak. In several schools in Indonesia, the school authorities consider that guidance and counseling are still not very important. Subject teachers replace many guidance and counseling teachers. Even some schools do not have guidance and counseling teachers due to limited budgets from the Foundation for providing education services. Aspects of program planning found the problem of Guidance and counseling teachers being less able to assess services because Guidance and Counseling teachers were not from Guidance and Counseling graduates, so they did not know how to do good assessments of students. The implementation component was found to be constrained by the lack of socialization of guidance and counseling services due to a lack of skills in service management, thus requiring support from other parties, especially subject teachers. The improvement program found deficiencies, namely the service format that was not suitable and the absence of adequate support made the service unable to run properly. The certification component was found to be an obstacle that counselors have a lot of work and lack of input and suggestions for developing guidance and counseling services in schools so that services tend to be monotonous and not by the required competencies. Recommendations for handling existing problems are needed, namely by improving a better system so that guidance and counseling can be more optimal in providing services.
Every organization has general and specific goals, short-term and long-term goals, which will be realized by using various existing resources or production factors. Managers will not achieve their goals optimally if resources or production factors are not carried out with the right process. Management plays a very important role because management is "the process of planning, organizing, leading and controlling organizational efforts and the process of using all organizational resources to achieve predetermined organizational goals. Evaluation is an important component in realizing the sustainability of a program. evaluasi akan melihat sejauh mana Pendidikan dapat berjalan dengan baik [33]. Performance evaluation of school counselors with the UCLA CSE model provides a clear picture so that the program that has been designed can be implemented properly. The evaluation of the UCLA CSE model guides school counselors to synergize with the education system. Currently, the challenges of education are getting bigger. School counselors as part of the education system must be able to synergize with other components to achieve quality education goals and follow the laws of the Republic of Indonesia.
_Conclusion
Evaluation of the program using the UCLA CSE model conducted in several schools in Indonesia shows that the school's guidance and counseling service program has been running in a good category. The aspect of system assessment scores (72.0), program planning (69.20), program implementation (70.00), program improvement (70.16), and program certification (70.20). Every aspect of the evaluation shows a score with good criteria. The planning aspect got a good score but a smaller score than the other aspects, 69.20. These results indicate that there is a need to improve the guidance and counseling service program in order to get very good criteria. Cooperation from the government is needed so that the program can run well. Several things can be done to improve the quality of service programs, namely by recruiting guidance teachers following the required educational qualifications. Support from the system needs to be strengthened to recognize guidance and counseling in schools. In-depth training is needed to assess students' needs so that the prepared programs are right on target. The socialization of guidance and counseling services in schools also needs to be improved to become more familiar with guidance and counseling. Socialization will help reduce the perception of guidance and counseling teachers as school police. The performance of guidance and counseling teachers also needs to be improved so that the prepared programs can be carried out properly. The results of this study indicate that evaluation has a major impact on a program. Evaluation provides an overview of what must be addressed and maintained. This study indicates that the education system, especially in guidance and counseling services, requires strong system support from the government and related educational institutions. Strong system support will result in maximum guidance and counseling services in educational institutions.
REFERENCES
1. McCaie F. T, Narrative interventions in post-modern guidance and career counseling. British Journal of Guidance & Counselling, 2020, vol. 48, no. 6, pp. 870-873. DOI: 10.1080/03069885.2020.1780415
2. Rahim M., & Hulukati W, Development of Handbooks of Guidance and Counseling to Enhance Elementary School Teachers' Competence in Cultivating Students' Creativity. European Journal of Educational Research, 2021, vol. 10, no. 2, pp. 657-670. DOI: 10.12973/eu-jer.10.2.657.
3. James L. Gibson et al, Organisasi: perilaku, struktur, dan proses, 2014, 2nd ed. Surabaya, Binarupa Aksara (In Indonesia).
4. Velarde J. M., Ghani M. F. A. & Adams D., Leadership practices in a conducive multicultural learning environment: insights from national secondary school teachers. Malaysian online journal of educational management (MOJEM), 2020, vol. 8, no. 3, pp. 48-63.
5. Widtayakornbundit S. & Phinaitrup B. A, Mediating Effects Of Relationship Quality On The Correlations Between Instructional Leadership And Loyalty. Malaysian Online Journal of Education, 2014, vol. 2, no. 1, pp. 48-61.
6. Duong M. Q, Perceptions of faculty members on the leadership capacity of their students at Vietnam National University Ho Chi Minh City. International Journal of Education and Practice, 2020, vol. 8, no. 2, pp. 356-364. DOI: 10.18488/journal.61.2020.82.356.364.
7. Nugroho F. A, Persepsi Siswa Terhadap Kinerja Guru Bimbingan dan Konseling yang Bersertifikasi Pendidik. PSIKOPEDAGOGIA Jurnal Bimbingan dan Konseling, 2014, vol. 3, no. 2, p. 98-107, DOI: 10.12928/psikopedagogia. v3i2.4470 (In Indonesia)
8. Fatchurahman M. & Asep Solikin, Revitalisasi fasilitas bimbingan dan konseling di sekolah, 2018, 1st ed. Malang: CV. IRDH International Research and Development for Human Beings (In Indonesia).
9. Makiva M., Ile I., & Fagbadebo O. M., Evaluating transformation progress of historically disadvantaged South Africans: Programme perspective on the downstream petroleum industry. African Evaluation Journal, 2019, vol. 7, no. 1, pp. 1-12, DOI: 10.4102/aej.v7i1.373.
10. Warinda E., Evaluating operationalisation of integrated monitoring and evaluation system in Kisumu County: Implications for policy makers. African Evaluation Journal, 2019, vol. 7, no. 1, pp. 1-12, DOI: 10.4102/aej.v7i1.385.
11. Mbava N. P., & Dahler-Larsen P., Evaluation in African contexts: The promises of participatory approaches in theory-based evaluations. African Evaluation Journal, 2019, vol. 7, no. 1, pp. 1-9, doi: 10.4102/aej.v7i1.383.
12. Martinaitis Z., Christenko A., & Krauciuniene L., Evaluation systems: How do they frame, generate and use evidence?, Evaluation, 2019, vol. 25, no. 1, pp. 46-61. DOI: 10.1177/1356389018802135.
13. Ilyasin M., Exploring excellency-based curriculum for indonesian primary schools in relation to the social community environment. Journal of Social Studies Education Research, 2019, vol. 10, no. 4, pp. 246-265.
14. Bahri, Sumaryana A., Karnaesih E., & Karlina N., The implementation of the allocation and distribution of school operational assistance program for compulsory education units in Kuningan regency, West Java province. Journal of Social Studies Education Research, 2019, vol. 10, no. 1, pp. 178-192.
15. Coldwell M., Reconsidering context: Six underlying features of context to improve learning from evaluation. Evaluation, 2019, vol. 25, no. 1, pp. 99-117. DOI: 10.1177/1356389018803234.
16. Yudong T., Aman M. S., Hooi L. B., & Siswantoyo, Satisfaction evaluation model of high-level athletes management system in universities of Sichuan Province of China. Cakrawala Pendidikan, 2020, vol. 39, no. 1, pp. 26-38. DOI: 10.21831/cp.v39i1.24559.
17. Goldman I., et al., Evaluation2 - Evaluating the national evaluation system in South Africa: What has been achieved in the first 5 years? African Evaluation Journal, 2019, vol. 7, no. 1, pp. 1-11. DOI: 10.4102/aej.v7i1.400.
18. Pabari M., Amisi M. M., David-Gnahoui E., Bedu-Addo D., & Goldman I., Evidence-informed policy and practice: The role and potential of civil society. African Evaluation Journal, 2020, vol. 8, no. 1, p. 1-11. DOI: 10.4102/aej.v8i1.470.
19. Turner R. E., & Cromhout P. B., Case study used to guide the development of effectiveness principles for principles-focused evaluation. African Evaluation Journal, 2020, vol. 8, no. 1, pp. 1-6. DOI: 10.4102/AEJ.V8I1.416.
20. Hsu Y. Y., & Lin C. H., Evaluating the effectiveness of a preservice teacher technology training module incorporating SQD strategies. International Journal of Educational Technology in Higher Education, 2020, vol. 17, no. 1, pp. 17-31, DOI: 10.1186/s41239-020-00205-2.
21. Harper L. M., & Dickson R., Using developmental evaluation principles to build capacity for knowledge mobilisation in health and social care. Evaluation, 2019, vol. 25, no. 3, pp. 330-348. DOI: 10.1177/1356389019840058.
22. Supriyadi E., Indro H. Y., Priyanto E., & Surwi F., Students' evaluation on teaching in vocational and technical schools. International Journal of Instruction, 2020, vol. 13, no. 2, pp. 621-636. DOI: 10.29333/iji.2020.13242a.
23. Suyasa P. W. A., & Kurniawan P. S., Pemberdayaan Model CSE-UCLA Dalam Pelaksanaan Evaluasi Program Blended Learning di SMA Negeri 1 Ubud. WACANA AKADEMIKA: Majalah Ilmiah Kependidikan, 2018, vol. 2, no. 2, p. 137162. DOI: 10.30738/wa.v2i2.2627 (In Indonesia).
24. Andriani S., Evaluasi Cse-Ucla Pada Studi Proses Pembelajaran Matematika. Al-Jabar:Jurnal Pendidikan Matematika, 2015, vol. 6, no. 2, pp. 167-176. DOI: 10.24042/ajpm.v6i2.46 (In Indonesia).
25. Naibaho L., Online Learning Evaluation during Covid-19 using CSE-UCLA Evaluation Model at English Education Department Universitas Kristen Indonesia, Budapest International Research and Critics Institute. (BIRCI-Journal): Humanities and Social Sciences, 2021, vol. 4, no. 2, pp. 1987-1997. DOI: 10.33258/birci.v4i2.1887.
26. Fatchurahman M., Setiawan M. A., & Karyanti K., The development of group healing storytelling model in multicultural counselling services in Indonesian schools: Examination of disciplinary cases. The Education and science journal, 2021, vol. 23, no. 4, pp. 157-180. DOI: 10.17853/1994-5639-2021-4-157-180.
27. Fatchurahman M., Setiawan M. A., Syarif D. F. T., & Efasanty M., Focused Solution Group Counseling as a Solution to Improve Career Choice Decision-making Abilities. Universal Journal of Educational Research, 2020, vol. 8, no. 12, pp. 6635-6640. DOI: 10.13189/ujer.2020.081227.
28. Putu Semadi I. G., Dantes N., & Sri Mertasari N. M., Studi Evaluatif Berbasis Model Cse-Ucla Tentang Pelaksanaan Program Pendidikan Bintara Di Sekolah Polisi Negara Polda Bali. Jurnal Penelitian dan Evaluasi Pendidikan Indonesia, 2019, vol. 9, no. 2, pp. 89-105. DOI: 10.23887/jpepi.v9i2.2899 (in Indonesia).
29. Qetin e., & Gundogdu K., The evaluation of the 7th grade english curriculum according to the eisner's educational connoisseurship and criticism model. Elementary Education Online, 2020, vol. 19, no. 3, pp. 1242-1266. DOI: 10.17051/ilkonline.2020.728032.
30. i§gi C., Kirmizi F. S., & Akkaya N., Evaluation of creative writing products according to content and some variables. Elementary Education Online, 2020, vol. 19, no. 2, pp. 718-732. DOI: 10.17051/ilkonline.2020.693207.
31. Ishola F., & Cekan J., Evaluating the sustainability of health programmes: A literature review. African Evaluation Journal, 2019, vol. 7, no. 1, pp. 1-7. DOI: 10.4102/aej.v7i1.369.
32. Bilgiler S., Ara§tirmalari E., §ahin M., Yildiz D. G. O., & Uman R. D., An Evaluation of the Theses on Social Studies Education in Turkey Turkiye' deki Sosyal Bilgiler Egitimi Tezleri Uzerine Bir Degerlendirme Sosyal ya§ama uyum, hayat boyu devam eden bir suregtir. Bu sureg , bireyin hem bireyi hayata hazirlamayi, ona hay, 2011, vol. 2, no. 2, pp. 96-121.
33. Chikova, O. A., Shakhnovich, I. S., & Popp, I. A., Evaluation of the effectiveness of competitive educational activities: analysis of the experience of using the methodology of structural equations modeling. Perspektivy nauki i obrazovania - Perspectives of Science and Education, 2021, vol. 52, no. 4, pp 448-460. DOI: 10.32744/ pse.2021.4.30.
Information about the authors
Mohammad Fatchurahman
(Indonesia, Central Kalimantan) Department of Guidance and Counseling, Faculty of Teacher Training and Education University Muhammadiyah of Palangkaraya E-mail: mfatchurahman789@gmail.com ORCID ID: 0000-0002-5747-2290 Scopus Author ID: 57222012193
Muhammad Andi Setiawan
(Indonesia, Central Kalimantan) Department of Guidance and Counseling, Faculty of Teacher Training and Education University Muhammadiyah of Palangkaraya E-mail: andisetiawan@umpr.ac.id ORCID ID: 0000-0001-7678-4057 Scopus Author ID: 57223855977
Karyanti Karyanti
(Indonesia, Central Kalimantan) Department of Guidance and Counseling, Faculty of Teacher Training and Education University Muhammadiyah of Palangkaraya E-mail: karyanti982@gmail.com ORCID ID: 0000-0002-8517-5146 Scopus Author ID: 57223842055
