CC BY
4
Original scientific paper UDC:
37.091.26
Received: November 09, 2023. 37.091.279.7
Revised: February 01, 2024. 37.015.3
Accepted: February 23, 2024. d 10.23947/2334-8496-2024-12-1-57-76
Check for updates
Constructivist Feedback-Based Assessment Method as Key for Effective Teaching and Learning: The Development and Impact on Mechanical Engineering Students' Adaptive Capacity, Decision Making, Problem
Solving and Creativity Skills
Waskito Waskito1" , Rizky Erna Wulansari1 , Rifelino Rifelino1 , Aprilla Fortuna1 , Abel Nyamapfene2 , Siti 'Afiat JaliP
1Faculty of Engineering, Universitas Negeri Padang, Indonesia e-mail: waskito@ft.unp.ac.id, rizkyema@ft.unp.ac.id, rifelino@ft.unp.ac.id, apnllafortuna@studentunp.acjd institute of Education & Faculty of Engineering, University College London, London, United Kingdom,
e-mail: a.nyamapfene@ucl.ac.uk 3Faculty of Technical and Vocational Education from University Tun Hussein Onn Malaysia, Malaysia
e-mail: siti.'afiat.jalil@moe.edu.my
Abstract: Educators must conduct assessments in their learning; it determines students' weaknesses in the teaching material they follow during learning. Unfortunately, the implementation of assessment by educators was not optimal, and the weakness was that the existing assessment method was only fixated on assessing students without providing feedback on the assessment. At the same time, this feedback was essential for students in learning, which can help learners assess performances that cannot be seen and felt by themselves, as well as a tool to motivate students, notification or information, and reinforcement. Therefore, this research aimed to develop a Constructivist Feedback-Based Assessment Method for learning assessment. The method used in this research was Research and Development (R&D). After development, the Constructivist Feedback-Based Assessment Method for learning assessment will be implemented to see its effect on students' adaptive capacity, decision-making, problem-solving, and creativity skills. Independent sample t-test and linear regression analysis were used as data analysis techniques describing the impact of the assessment on those skills. The results showed that the Constructivist Feedback-Based Assessment Method has five stages: preparing the assessment material, diagnostic assessment, assessment for learning, assessment of learning, and reflection. It effectively affects students' skills, such as adaptive capacity, decision-making, problem-solving, and creativity. It can be concluded that the Constructivist Feedback-Based Assessment Method can improve students' adaptive capacity, decision-making, problem-solving, and creativity. Novelty in this research was the existence of constructivism integrated into feedback-based assessment, which the existing assessment has not highlighted the constructivist side of assessment.
Keywords: assessment, feedback, constructivist, teaching and learning, vocational education.
Introduction
Implementing assessment in learning significantly contributes to educators' professional development and learners' learning (Bragg et al., 2021; Hennessy et al., 2022). Assessment serves to provide feedback to learners. Based on this, educators can analyze the information, comment on it, and use it to check and regulate learning. Assessment, if implemented correctly, can improve the quality of learning. The function of assessment is not to give a rating but to see where the learners' mastery is and what they have not mastered (Wetzel et al., 2020). Based on the survey results, using assessment as an evaluation has not been maximally implemented in learning. Too many lecturers/educators still do not utilize this evaluation function to improve the process and quality of teaching and learning. It is not even an exaggeration to say that lecturers/educators very rarely develop evaluations to obtain information about what has been planned for an interaction. It can be seen that assessment is not carried out optimally,
'Corresponding author: waskito@ft.unp.ac.id
fTS (?) I © 2024 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.ora/licenses/bv/4.0/).
where tests are usually given only at the time of Mid Test or Final Test (Jalinus et al., 2023). If assessment is not really implemented properly, then educators cannot know the extent of students' understanding, so that learning that occurs is not effective.
Therefore, the solution to this problem is to develop an assessment method. Although the assessment method has existed before and is applied in schools, the weakness of the existing assessment method is that the assessment method is fixated on assessing students only, without providing feedback on the assessment. At the same time, this feedback is essential for students in learning, which can help learners assess performances that cannot be seen and felt by themselves (Hattie, 2008), as well as a tool to motivate learners, notification or information, reinforcement, and motivation (Sims et al., 2023). The advantage of this feedback-based assessment method is that it will obtain information about the pattern of achieving learning objectives. To meet the learning objectives set, diagnostic information for each learner can more effectively help learners know which parts of the topic they still have not mastered so that these learners can quickly learn the lesson topics that have not been mastered (Waskito et al., 2022). The assessment results from using this feedback-based assessment method provide diagnostic information from each score obtained by each learner, called individual diagnostic information, and information on groups of learners or group diagnostic information (Duan et al., 2020).
According to (Fuchs and Fuchs, 1986), many meta-analysis studies on learning quality improvement factors. Based on the results, assessment methods that include feedback are the most influential factor in improving learning quality. Based on previous research, the assessment that includes feedback positively impacts many learner behaviors, especially those related to students' skills. In the literature, assessment methods rank at the top of the list of studies, compared to teaching strategies and techniques to improve learners' academic achievement. Relevant meta-analysis studies also show that assessments that include feedback significantly impact learner success (Karaman, 2021; Phelps, 2019; Wisniewski et al., 2020; Yan et al., 2023). At the same time, constructivism emphasizes students' active role in constructing their knowledge through interaction with learning materials and social experiences. This approach also positively impacts assessment methods (Mohammed et al., 2020).
However, there seems to be an empirical gap in previous research. There is a lack of rigorous research in the previous literature. Constructivist integrated feedback on assessment in enhancing students' ability, which has not been explored, seems essential and worthy of investigation. Empirical investigation of these issues is necessary because assessment is one of the learning elements that can provide better learning quality if combined with feedback and constructivism. In addition, previous research has focused on combining assessment with feedback only, as well as focusing only on improving learning outcomes (Prasetya, Fajri, et al., 2023; Prasetya, Syahri, et al., 2023; Waskito et al., 2023). Therefore, this research aims to develop a constructivist feedback-based assessment method as a key to effective learning in improving the quality of learning in vocational schools and to see the impact of a constructivist feedback-based assessment method on students' adaptive capacity, decision-making, problem-solving, and creativity.
Materials and Methods
Research Design
The method used in this research was Research and Development (R&D). The development model in this study refers to the Research and Development model based on Borg and Gall (Aka, 2019) (Figure 1).
The steps of this research are as follows: 1) identifying problems and analyzing the needs for the development of constructivist feedback-based assessment methods; this problem identification activity is carried out using a survey method and analysis questionnaire; 2) designing and developing constructivist feedback-based assessment methods and its procedures, this development stage was carried out using the explanatory sequential design method in designing constructivist feedback-based assessment method, and; 3) expended trials the constructivist feedback-based assessment method to students using the quasi-experiment method, to see the impact on adaptive capacity, decision making, problem-solving and student creativity. In the expended trials stage, the group of students who were given the constructivist
feedback-based assessment method treatment was called the experimental group, where the treatment was given for 12 weeks. The first week was given a pre-test, weeks 2 to 11 were given treatment, and the last week was given a post-test.
Output and Indicator of Achievement
Process
Start
v Obtained the theoretical basis of the development of constructivist feedback-based assessment methods, willingness and feasibility, and problems faced by educators and students in conducting assessments.
Problem Identification and Need Analysis Method: Survey Instrument: Need analysis questionnaire —»
J,
Designing the constructivist feedback- based assessment methods Method: explanatory sequential design
—» constructivist feedback-based assessment methods
Expert Judgment and Revision Method: focus group discussion
Comments and suggestions of constructivist feedback-based assessment methods from expert before expended trials implementing
Pilot study Metode: survey Instrument: Need analysis, adaptive capacity, décision making, problem solving dan creativity skîII questionnaires Data analysis results of pilot study; validity and reliability of questionnaires
Expended Trials Method: quasi experiment Instrument: adaptive capacity, décision making, problem solving dan creativity skill questionnaires Effectiveness' data analysis results of constructivist feedback-based assessment methods and its impact on students' adaptive capacity, decision making, problem solving dan creativity skill
.
Stop
J
Figure 1. Research Design
Research Participants
Research respondents in the needs analysis activities amounted to 20 Vocational High School, and respondents in this trial activity were 121 mechanical engineering students of Vocational High School who were divided into two groups, namely 61 in the experimental group and 60 in the control group.
Research Instruments
This questionnaire was adapted from the rubric for assessing work-relevant skills developed by (Sánchez-Ramírez et al., 2022), consisting of a series of closed questions with answers to be rated on a scale of 1 to 5 to investigate their skills. The pilot study aims to determine the validity and reliability of the research questionnaire before it is retrieved in an expanded trial, and this pilot study was conducted on 60 vocational high school students. Validity data analysis was carried out using the Intraclass Correlation Coefficient (ICC); if the rater consistency is 0.500, it is classified as valid (Su et al., 2023). It can be concluded that the agreement between raters is very strong, and each rater has a pretty good consistency. The reliability of the questionnaire was tested using Cronbach's alpha to analyze the suitability of the research questions (Surucu and Maslakci, 2020). Cronbach's alpha has been widely used in studies in the field of science education to see if the questionnaire is reliable (Baidal-Bustamante et al., 2023).
Table 1. Research Instrument Indicators and Pilot Study Analysis Results
Variables and Indicators Answer Intraclass Correlation Coefficient (ICC) Cronbach's alpha Coefficient
Adaptive capacity has three evaluation indicators: adapting and accepting change, contributing to change, and encouraging and handling change. Rubric assessment Likert scale - strongly agree (5), to strongly disagree (1) 0.890 0.578
Decision Making has two indicators of evaluation objectives: decisionmaking with criteria when alternatives are proposed and choosing the most appropriate option to anticipate the consequences. Rubric assessment Likert scale - very often (5), to never (1) 0.696 0.616
Problem Solving has three indicators of evaluation objectives: identifying problems, analyzing and solving problems, preventing problems, and overcoming complex problems. Rubric assessment Likert scale - strongly agree (5), to strongly disagree (1) 0.821 0.801
Creativity has two evaluation indicators, namely, generating ideas and creating original ideas for specific purposes Rubric assessment Likert scale - strongly agree (5), to strongly disagree (1) 0.601 0.615
Table 2. Hypothesis Development
Variable Hypothesis Data Analysis Technique
Independent Dependent Null Hypothesis (H0) Hypothesis Alternatives (Ha)
Adaptive Capacity There is no difference in the Adaptive Capacity ability of experimental and control group students after applying the Constructivist Feedback-Based Assessment Method. There is a difference in the Adaptive Capacity ability of experimental and control group students after applying the Constructivist Feedback-Based Assessment Method.
Decision Making There is no difference in the decision-making ability of experimental and control group students after applying the Constructivist Feedback-Based Assessment Method. There is a difference in the decision-making ability of experimental and control group students after applying the Constructivist Feedback-Based Assessment Method.
Constructivist Feedback-Based Assessment ProblemSolving There is no difference in the problemsolving ability of experimental and control group students after applying the Constructivist Feedback-Based Assessment Method. There is a difference in the problemsolving ability of experimental and control group students after applying the Constructivist Feedback-Based Assessment Method. Independent sample t-test
Method Creativity There is no difference in the creativity ability of experimental and control group students after applying the Constructivist Feedback-Based Assessment Method. There is a difference in the creativity ability of experimental and control group students after applying the Constructivist Feedback-Based Assessment Method.
Adaptive Capacity, Decision Making, Problem Solving and Creativity The constructivist feedback-based assessment method does not impact students' adaptive capacity, decision-making, problem-solving, and creativity in the experimental group. The constructivist feedback-based assessment method impacts students' adaptive capacity, decision-making, problem-solving, and creativity in the experimental group.
Data Analysis Technique and Hypothesis Development
This research was analyzed quantitatively using the methods of percentage, average, standard deviation, and parametric statistics. Hypotheses and data analysis techniques regarding the impact of the Constructivist Feedback-Based Assessment Method on students' adaptive capacity, decision-making, problem-solving, and creativity skills are presented in Table 2.
Results and Discussions
Needs Analysis and Development of Constructivist Feedback-Based
Assessment Method
Need analysis is an analysis conducted to examine a phenomenon of the needs of a program (Mubai et al., 2020). Respondents used in filling out this needs analysis questionnaire are teachers of Vocational High Schools majoring in mechanical engineering in Padang City. The needs analysis is carried out to identify the learning evaluation that has been carried out at this time. This analysis is carried out as a consideration for developing constructivist feedback-based assessment methods in Vocational Schools. The results of observations made at several vocational schools in Padang City stated that teachers had implemented evaluation or assessment in every lesson. However, the assessment teachers implement is still unstructured; sometimes, teachers give formative tests, and sometimes, they do not. Thus, teachers have not found the right way to implement assessment in learning. Teachers expect a structured assessment method that teachers can guide. The assessment plays an essential role in learning. A well-implemented assessment will improve the quality of education.
The needs analysis data shows a gap between the current assessment and the mean expectation of assessment implementation. It means that teachers expect an assessment method they can guide in a structured and easy-to-understand manner. It does not mean that teachers at school have not implemented the assessment system, but that the assessment has not been implemented optimally due to confusion in implementing the assessment method. Therefore, teachers expect an innovative assessment method to improve their education and learning levels. Teachers in vocational high schools are very open-minded to change, innovation, and the times, which is why they are also open-minded to this new assessment method that will be developed. Literate teachers always try to adapt to the environment and changing times. Assessment that is guided and maximally implemented can give students a perfect understanding of the material because teachers can design learning based on the assessment given, and the feedback given in this method can result in students being active in learning because there is more discussion about what has been understood, what will be done to improve understanding.
The implementation of assessment can analyze the extent of learners' mastery and what they have not mastered. Even in the literature, this assessment is ranked at the top in improving the quality of learning (Phelps, 2019). It means that by improving the quality of learning, students' learning outcomes will also increase (Hartmann, 2019). So, this research is expected to contribute to the science and related literature on constructivist feedback-based assessment methods, and the correct implementation of assessment will be implemented in learning. The assessment method has existed before and is implemented in schools. However, the weakness of the existing assessment method is that the assessment method is fixated on assessing students only without providing feedback on the assessment. This feedback is essential for students in learning, which can help learners assess performances that cannot be seen and felt by themselves (Jalinus et al., 2023), as well as a tool to motivate learners, notification or information, reinforcement, and motivation (Hattie, 2008).
This constructivist feedback assessment method was developed by (Middleton et al., 2023), which uses tutor input and the '5 Keys' indicators of academic value, namely: (i) internal locus of control; (ii) understanding the class; (iii) forward-thinking; (iv) improvement-focused, and (v) action-oriented. However, the weakness of the (Middleton et al., 2023) assessment method is that they do not emphasize dialogic interaction. Thus, the novelty of this research is to develop a constructivist feedback-based assessment method, which aims to achieve positive outcomes by providing a person with comments or suggestions that are useful for their learning or future and constructing students' knowledge. This constructive feedback focuses on the work rather than being a personally damaging attack on the individual. So, the results can
be faster processes, improved behavior, identifying weaknesses, or providing new perspectives. The following constructive feedback-based assessment methods have been developed.
Preparing the assessment material
of learning
The teacher gives the task after the
Figure 2. Constructivist Feedback-Based Assessment Method
In this constructivist feedback-based assessment method, there are five steps that the teacher must take: Preparing the assessment material; at this stage, the teacher prepares the assessment material before the learning begins. Educators often forget to prepare assessment materials because they are busy preparing them in the form of syllabuses, lesson plans, and teaching materials before learning begins, so the preparation of assessment materials is often neglected. When teachers implement this assessment method, this first stage will remind teachers to prepare assessment materials that will be implemented in learning. This activity is carried out before learning begins. Diagnostic Assessment: this second stage is conducted before the learning begins; at this stage, teachers can provide a series of written questions (multiple choice or short answer) to assess students' current knowledge base or current views on the topic/issue to be studied in the subject. Teachers can use this diagnostic assessment to analyze students' initial abilities. Thus, the teacher can design the proper learning procedure according to the student's initial abilities visible through the diagnostic assessment.
Assessment for Learning This third stage is carried out during the learning process; what is meant during the learning process is that the teacher can give this test when the subject matter is completed in one day, for example, daily assignments. The technique used in this stage is formative assessment. The test given can be multiple choice, essay, or even case. By this stage, students can construct their knowledge based on the given cases by teachers. After this stage, the teacher provides feedback to students about their learning understanding and how to improve learning understanding; the technique used in this feedback is face-to-face/direct comment. Assessment of Learning: an assessment conducted after all learning has been completed; the technique used in this assessment is a summative test and can be given at the end of the semester. This assessment is used to determine future learning goals and pathways for students. After this assessment stage, the teacher also provides feedback written on notes. The teacher can use the results of this assessment to see if the students' abilities have reached
I
the goals and standards of learning. Reflection: The last stage is reflection; at this stage, students reflect on themselves about what they have learned, what they understand, and what they have not understood. This reflection stage can be done at the end of the semester.
Impact of Students' Skills Toward the Implementation of Constructivist
Feedback-Based Assessment Method
Implementing the constructivist feedback-based assessment method in learning activities can effectively develop students' adaptive capacity, decision-making, problem-solving, and creativity. This is in line with the findings of (Gulikers et al., 2013), which state that Feedback-Based Assessment contributes to developing students (Furtak et al., 2008) and added that students' creativity and problem-solving skills can be improved through Feedback-Based Assessment.
The Impact on Adaptive Capacity
The data analysis results on the impact of the Constructivist Feedback-Based Assessment Method on the adaptive capacity instrument showed that the "strongly agree" score was the highest percentage among all items (Table 3). The item "Students can provide several alternative solutions to solve problems" achieved the highest score (M=67.8, SD=84.9), with 62.3% of students at the very adaptable level and 24.6% at the adaptable level. Furthermore, "Students can control changes and support my friends" received the lowest score of the five items (M = 64.8, SD = 62.6); 41% of students are competent, and 45.9% can control changes.
Table 3. Data Analysis of Adaptive Capacity Rubric on Control Group
Item Strongly Agree Agree Neutral Disagree Strongly Disagree M SD
Post-test
Students are adaptable and work with different groups 20 (33.3%) 0 (0%) 0 (0%) 11 (18,3%) 30 (50%) 30,5 47,5
Students can accept change as a challenge 15 (25%) 1 (1.6%) 0 (0%) 18 (30%) 27 (45%) 28,8 34,8
Students can provide several alternative solutions to solve the problem 16 (26.67%) 1 (1.6%) 1 (1.6%) 18 (30%) 25 (41.67%) 30,8 36,2
Students can control change and support friends 10 (16.67%) 0 (0%) 0 (0%) 15 (25%) 36 (60%) 20,0 24,5
Students can work towards a goal 15 (25%) 0 (0%) 1 (1.6%) 10 (16.67%) 35 (58.3%) 24,5 34,8
Pre-test
Students are adaptable and work with different groups 0 (0%) 0 (0%) 1 (1.6%) 27 (45%) 33 (55%) 18 24,5
Students can accept change as a challenge 0 (0%) 0 (0%) 0 (0%) 27 (45%) 34 (56.6%) 17,6 25,1
Students can provide several alternative solutions to solve the problem 0 (0%) 0 (0%) 1 (1.6%) 24 (40%) 36 (60%) 17,4 22,9
Students can control change and support friends 0 (0%) 0 (0%) 2 (3.3%) 27 (45%) 32 (53.3%) 18,4 23,9
Students can work towards a goal 0 (0%) 0 (0%) 1 (1.6%) 29 (48.3%) 31 (51.7%) 18,4 25,7
Based on the data analysis in Table 3, it can be concluded that the control group that did not implement the Constructivist Feedback-Based Assessment Method of learning has not improved students' adaptive capacity, which can be seen as the most of highest scores in the 'never.' Data analysis of the
adaptive capacity rubric on an experimental group can be seen in Table 4.
Table 4. Data Analysis of Adaptive Capacity Rubric on Experimental Group
Item Strongly Agree Agree Neutral Disagree Strongly Disagree M SD
Post-test
Students are adaptable and work with different groups 29 (47.5%) 15 (24.6%) 10 (16.4%) 7 (11,5%) 0 (0%) 62.3 58.4
Students can accept change as a challenge 32 (52.5%) 20 (32.8%) 9 (14.8%) 0 (0%) 0 (0%) 66.8 70.5
Students can provide several alternative solutions to solve the problem 38 (62.3%) 15 (24.6%) 5 (8.2%) 3 (4.9%) 0 (0%) 67.8 84.9
Students can control change and support friends 25 (41%) 28 (45.9%) 6 (9.8%) 2 (3.3%) 0 (0%) 64.8 62.6
Students can work towards a goal 30 (49.2%) 31 (50.8%) 0 (0%) 0 (0%) 0 (0%) 68.5 79.8
Pre-test
Students are adaptable and work with different groups 1 (1,6%) 1 (1,6%) 10 (16.4%) 26 (42.6%) 23 (37.7%) 22,8 19,8
Students can accept change as a challenge 0 (0%) 0 (0%) 11 (18%) 26 (42.6%) 24 (39.3%) 21,8 22,3
Students can provide several alternative solutions to solve the problem 0 (0%) 1 (1,6%) 8 (13.1%) 26 (42.6%) 26 (42.6%) 21,2 20,8
Students can control change and support friends 0 (0%) 0 (0%) 6 (9.8%) 28 (45.9%) 27 (44.3%) 20,2 23,2
Students can work towards a goal 0 (0%) 0 (0%) 9 (14.7%) 25 (40.9%) 27 (44.3%) 20,8 21,2
Based on the data analysis in Table 3, it can be concluded that implementing the Constructivist Feedback-Based Assessment Method of learning has improved students' adaptive capacity, which can be seen in the highest scores. The significant differences between the pre-test and post-test analysis of the control and experimental groups can be seen in Table 5 in detail.
Table 5. Analysis of Adaptive Capacity T-test Results
Paired Sample T-test
Observations Groups N Mean Differences t df P
Pretest-Posttest analysis of adaptive capacity instrument Experimental Group 61 21.6 6.961 60 0.000
Control Group 60 7.48 0.758 59 0.412
Independent Sample T-test
Post-test comparison analysis of adaptive capacity instrument M t df P
Experimental Group 61 64.23 5.864 120 0.000
Control Group 60 49.68
Table 5 showed a significant difference between the experimental and control groups on adaptive capacity (df=120, t=5.864, pvalue=, p<0.05). It indicates that the experimental group had a higher mean adaptive capacity at the post-test than the control group. Therefore, the impact of the treatment was already evident in the post-test scores after the implementation of the Constructivist Feedback-Based Assessment Method in the experimental group. It can be seen in the P-value; if the P-value> 0.05, then
the null hypothesis is rejected. Based on the data above, it can be interpreted that H0 is rejected, which means there is a significant difference between the experimental and control groups on the post-test score. The results of this study indicate that the Constructivist Feedback-Based Assessment Method can improve adaptive capacity. Alt et al., (2023) found that through the Constructivist Feedback-Based Assessment Method, students can get feedback from the teacher that will affect logical thinking and reflective thinking and provide explanations. This increase in logical thinking can improve students' adaptive capacity (Arsovic and Stefanovic, 2020; Handrayani et al., 2023; Wulansari and Nabawi, 2021).
The Impact on Decision-Making
The results on the impact of the Constructivist Feedback-Based Assessment Method on Decision-Making showed that the score of "very often" was the highest percentage among all items (Table 5). The item "Students can make the right decision when trying to choose between various alternative solutions to a problem" achieved the highest score (M=57.4, SD=105.6), with 80.3% of students very often and 13.1% of students often discussing with the teacher. Moreover, "Students do not make decisions based on emotional factors" got the lowest score of the four items (M = 54.4, SD = 77.9), namely 60.7% of students very often and 27.9% of students often do not make decisions based on emotional factors.
Table 6. Data Analysis of Decision-Making Rubric on Control Group
Item Very often Often Sometimes Rare Never M SD
Post-test
Students discuss with the teacher to make a decision 20 (33.3%) 0 (0%) 3 (5%) 0 (0%) 37 (61.7%) 29,2 42,4
Students do not make decisions based on emotional factors 23 (38.3%) 0 (0%) 0 (0%) 0 (0%) 37 (61.7%) 30,4 49,9
Students can discuss the consequences of various alternative decisions 15 (25%) 0 (0%) 7 (11.7%) 5 (8.3%) 34 (56.7%) 28 29,2
Students can make informed decisions when choosing between various alternative solutions to a problem. 18 (30%) 0 (0%) 4 (6.7%) 3 (5%) 36 (60%) 28,8 36,8
Pre-test
Students discuss with the teacher to make a decision 0 (0%) 0 (0%) 7 (11.7%) 12 (20%) 41 (68.3%) 17,2 17,5
Students do not make decisions based on emotional factors 0 (0%) 0 (0%) 6 (10%) 12 (20%) 42 (42%) 16,8 17,7
Students can discuss the consequences of various alternative decisions 0 (0%) 0 (0%) 2 (3.3%) 20 (33.3%) 38 (63.3%) 16,8 20,4
Students can make informed decisions when choosing between various alternative solutions to a problem. 0 (0%) 0 (0%) 6 (10%) 19 (31.7%) 35 (58.3%) 18,2 18,3
Based on the data analysis in Table 6, it can be concluded that the control group that did not implement the Constructivist Feedback-Based Assessment Method of learning did not improve students' decision-making, which can be seen as the most of highest scores in the 'never.' Data analysis of the decision-making rubric on the experimental group can be seen in Table 7.
Table 7. Data Analysis of Decision-Making Rubric on Experimental Group
Item Very often Often Sometimes Rare Never M SD
Post-test
Students discuss with the teacher to make a decision 38 (62.3%) 17 (27.9%) 3 (4.9%) 0 (0%) 3 (4.9%) 54 81.0
Students do not make decisions based on emotional factors 37 (60.7%) 17 (27.9%) 5 (8.2%) 2 (3.3%) 0 (0%) 54.4 77.9
Students can discuss the consequences of various alternative decisions 48 (78.7%) 3 (4.9%) 7 (11.5%) 3 (4.9%) 0 (0%) 55.8 103.3
Students can make informed decisions when choosing between various alternative solutions to a problem. 49 (80.3%) 8 (13.1%) 2 (3.3%) 2 (3.3%) 0 (0%) 57.4 105.6
Pre-test
Students discuss with the teacher to make a decision 0 (0%) 1 (1.6%) 9 (14.8%) 12 (19.7%) 39 (63.9%) 18,8 16,4
Students do not make decisions based on emotional factors 0 (0%) 0 (0%) 4 (6.6%) 22 (36.1%) 36 (59%) 18,4 20,5
Students can discuss the consequences of various alternative decisions 0 (0%) 0 (0%) 7 (11.5%) 7 (11.5%) 47 (77%) 16,4 19,4
Students can make informed decisions when choosing between various alternative solutions to a problem. 0 (0%) 0 (0%) 5 (8.2%) 10 (16.4%) 46 (75.4%) 16,2 18,9
Based on the data analysis in Table 7, it can be concluded that implementing the Constructivist Feedback-Based Assessment Method of learning has improved students' decision-making skills. The significant differences between the pre-test and post-test analysis of the control and experimental groups can be seen in Table 8 in detail.
Table 8. Analysis of Decision-Making T-test Results
Paired Sample T-test
Observations Groups N Mean Differences t df P
Pretest-Posttest analysis of adaptive capacity instrument Experimental Group 61 2.11 6.872 60 0.001
Control Group 60 6.16 0.976 59 0.633
Independent Sample T-test
Post-test comparison analysis of adaptive capacity instrument M t df P
Experimental Group 61 51.05 6.375 120 0.000
Control Group 60 38.29
Table 8 showed a significant difference in decision-making skills between the experimental and control groups (df=120, t=6.375, p-value=, p<0.05). It shows that the experimental group has a higher mean decision-making score in the post-test than the control group. Therefore, the impact of the treatment was already evident in the post-test scores after the implementation of the Constructivist Feedback-Based Assessment Method in the experimental group. It can be seen that at the P-value > 0.05, the null hypothesis is rejected. Based on the data above, it can be interpreted that H0 is rejected, which means there is a
difference in the decision-making skills of experimental and control group students after implementing the Constructivist Feedback-Based Assessment Method. This is in line with research conducted by (Fazel and Ali, 2022), which explains that feedback gives students an overview of their decisions. They can see the positive and negative aspects of their decisions. By realizing the consequences of the decisions, students become more cautious and consider their choices better. It also helps students understand how to solve problems arising from their decisions (Torres et al., 2020). They can see the impact of their decisions on the situation and learn how to overcome the problems that arise (Teras et al., 2020).
The Impact on Problem-Solving
The data analysis result of the impact of the Constructivist Feedback-Based Assessment Method on problem-solving showed that the "strongly agree" score was the highest percentage among all items (Table 7). The item "Students can make good decisions in solving problems to achieve goals" achieved the highest score (M=57.8, SD=103.2), with 78.7% of students at the very capable level and 16.4% of students at the capable level of making good decisions. Furthermore, "Students can consider a variety of solutions" got the lowest score of the four items (M = 55.4, SD = 79.7); 60.7% of students are capable, and 32.8% can consider different solutions.
Table 9.Data Analysis of Problem-Solving Rubric on Control Group
Item Strongly Agree Agree Neutral Disagree Strongly Disagree M SD
Post-test
Students can identify problems that arise 28 (46.7%) 1 (1.7%) 5 (8.3%) 10 (16.7%) 15 (25%) 38,8 56,9
Students can make good decisions in solving problems to achieve goals 19 (31.7%) 0 (0%) 3 (5%) 13 (21.7%) 25 (41.7%) 31 37,4
Students can solve complex problems well 16 (26.7%) 1 (1.7%) 3 (5%) 17 (28.3%) 23 (38.3%) 30 30,3
Students can consider a range of different solutions. 23 (38.3%) 0 (0%) 4 (6.7%) 10 (16.7%) 23 (38.3%) 34 46,1
Pre-test
Students can identify problems that arise 0 (0%) 0 (0%) 4 (6.7%) 9 (15%) 47 (78.3%) 15,4 19,3
Students can make good decisions in solving problems to achieve goals 0 (0%) 0 (0%) 0 (0%) 12 (19.7%) 48 (80%) 14,4 21,5
Students can solve complex problems well 0 (0%) 0 (0%) 2 (3.3%) 12 (19.7%) 46 (76.7%) 15,2 19,8
Students can consider a range of different solutions. 0 (0%) 0 (0%) 2 (3.3%) 13 (21.7%) 45 (75%) 15,4 19,7
Based on the data analysis in Table 9, it can be concluded that the control group that did not implement the Constructivist Feedback-Based Assessment Method of learning did not improve students' problem-solving, which can be seen as the highest scores in the 'never.' Data analysis of the problemsolving rubric on the experimental group can be seen in Table 10.
Table 10. Data Analysis of Problem-Solving Rubric on Experimental Group
Item Strongly Agree Agree Neutral Disagree Strongly Disagree M SD
Post-test
Students can identify problems that arise 39 (63.9%) 20 (32.8%) 0 (0%) 2 (3.3%) 0 (0%) 55.8 85.0
Students can make good decisions in solving problems to achieve goals 48 (78.7%) 10 (16.4%) 3 (5%) 0 (0%) 0 (0%) 57.8 103.2
Students can solve complex problems well 38 (62.3%) 16 (26.2%) 3 (5%) 4 (6.6%) 0 (0%) 54.2 80.1
Students can consider a range of different solutions. 37 (60.7%) 20 (32.8%) 4 (6.7%) 0 (0%) 0 (0%) 55.4 79.7
Pre-test
Students can identify problems that arise 0 (0%) 0 (0%) 10 (16.4%) 2 (3.3%) 49 (80.3%) 16,6 22,0
Students can make good decisions in solving problems to achieve goals 0 (0%) 0 (0%) 4 (6.7%) 12 (19.7%) 45 (73.8%) 16,2 18,9
Students can solve complex problems well 0 (0%) 0 (0%) 3 (5%) 10 (16.4%) 48 (78.7%) 15,4 20,0
Students can consider a range of different solutions. 0 (0%) 0 (0%) 3 (5%) 11 (18%) 47 (77%) 15,6 19,7
Based on the data analysis in Table 10, it can be concluded that implementing the Constructivist Feedback-Based Assessment Method of learning has improved students' problem-solving skills. The significant differences between the pre-test and post-test analysis of the control and experimental groups can be seen in Table 11 in detail.
Table 11. Analysis of Problem-Solving T-test Results
Paired Sample T-test
Observations Groups N Mean Differences t df P
Pretest-Posttest analysis of adaptive capacity instrument Experimental Group 61 2.74 6.286 60 0.001
Control Group 60 4.82 0.885 59 0.736
Independent Sample T-test
Post-test comparison analysis of adaptive capacity instrument M t df P
Experimental Group 61 57.42 5.582 120 0.001
Control Group 60 36.01
Table 11 showed a significant difference in problem-solving ability between the experimental and control groups (df=120, t=5.582, p-value=, p<0.05). It showed that the experimental group had a higher mean problem-solving score in the post-test than the control group. Therefore, the impact of the treatment was already apparent in the post-test scores after the implementation of the Constructivist Feedback-Based Assessment Method in the experimental group. It can be seen in the P-value> 0.05 that the null hypothesis is rejected. Based on the data above, it can be interpreted that H0 is rejected, which means there are differences in the problem-solving skills of experimental and control group students after implementing the Constructivist Feedback-Based Assessment Method. The feedback assessment method has a significant impact on students' problem-solving skills. By providing excellent and directed feedback, students can better develop their problem-solving skills (Mubai et al., 2020). Menurut (Kardoyo
et al., 2020) also explains that students learn to evaluate their proposed solutions through feedback. They can see the advantages and disadvantages of their solutions and understand the criteria for good evaluation (Lacey and Minnis, 2020). These skills are essential in choosing the best solution from various possible alternatives. Also, feedback gives students an idea of their approach to the problem. They become more aware of the strategies they choose and realize if there are weaknesses in their approach (Taherdoost, 2022).
The Impact on Creativity
The analysis of the impact of the Constructivist Feedback-Based Assessment Method on creativity showed that the score of "strongly agree" was the highest percentage among all items (Table 9). The item "Students can analyze their ideas to optimize results" achieved the highest score (M=57.2, SD=100.7), with 77% of students at the very capable level and 16.4% at the capable level of analyzing ideas. Moreover, "Students are curious and interested in learning new things" received the lowest score of the four items (M=54.6, SD=76), where 59% were very curious, and 29.5% were curious and interested in learning new things.
Table 12. Data Analysis of Creativity Rubric on Control Group
Item Strongly Agree Agree Neutral Disagree Strongly Disagree M SD
Post-test
Students are interested in learning new things 28 (46.7%) 1 (1.7%) 6 (10%) 5 (8.3%) 20 (33.3%) 38,4 57,2
Students can analyze their ideas to optimize results 18 (30%) 1 (1.7%) 4 (6.7%) 18 (30%) 30 (50%) 34,4 33,7
Students can create new ideas based on limited information 17 (28.3%) 16 (26.2%) 1 (1.7%) 13 (21.7%) 27 (45%) 29,8 32,6
Students efficiently implement their new ideas into a project. 17 (28.3%) 1 (1.7%) 17 (28.3%) 6 (10%) 19 (31,7%) 34,2 33,5
Pre-test
Students are interested in learning new things 0 (0%) 0 (0%) 3 10 47 15,2 19,6
Students can analyze their ideas to optimize results 0 (0%) 0 (0%) 10 16 34 19,2 17,6
Students can create new ideas based on limited information 0 (0%) 0 (0%) 2 15 43 15,8 19,6
Students efficiently implement their new ideas into a project. 0 (0%) 0 (0%) 4 13 43 16,2 18,4
Based on the data analysis in Table 12, it can be concluded that the control group that did not implement the Constructivist Feedback-Based Assessment Method of learning has not improved students' creativity, which can be seen as the most of highest scores being in the 'never.' Data analysis of the creativity rubric on the experimental group can be seen in Table 13.
Table 13. Data Analysis of Creativity Rubric on Experimental Group
Item Strongly Agree Agree Neutral Disagree Strongly Disagree M SD
Post-test
Students are interested in learning new things 36 (59.0%) 18 (29.5%) 7 (11.5%) 0 (0%) 0 (0%) 54.6 76.0
Students can analyze their ideas to optimize results 47 (77.0%) 10 (16.4%) 3 (4.9%) 1 (1.6%) 0 (0%) 57.2 100.7
Students can create new ideas based on limited information 39 (63.9%) 20 (32.8%) 0 (0%) 2 (3.3%) 0 (0%) 55.8 85.0
Students efficiently implement their new ideas into a project. 36 (59.0%) 10 (16.4%) 15 (24.6%) 5 (8.2%) 0 (0%) 55 72.5
Pre-test
Students are interested in learning new things 0 (0%) 0 (0%) 6 (9.8%) 12 (19.7%) 43 (70.5%) 17 18,1
Students can analyze their ideas to optimize results 0 (0%) 0 (0%) 1 (1.6%) 15 (24.6%) 45 (73.8%) 15,6 20,7
Students can create new ideas based on limited information 0 (0%) 0 (0%) 2 (3.3%) 17 (27.9%) 42 (68.9%) 16,4 20,1
Students efficiently implement their new ideas into a project. 0 (0%) 0 (0%) 5 (8.2%) 11 (18%) 41 (67.2%) 15,6 17,1
Based on the data analysis in Table 13, it can be concluded that implementing the Constructivist Feedback-Based Assessment Method of learning has improved students' creativity ability. The significant differences between the pre-test and post-test analysis of the control and experimental groups can be seen in Table 14 in detail.
Table 14. Analysis of Creativity T-test Results
Paired Sample T-test
Observations Groups N Mean Differences t df P
Pretest-Posttest analysis of adaptive capacity instrument Experimental Group 61 3.76 7.719 60 0.000
Control Group 60 6.29 0.674 59 0.736
Independent Sample T-test
Post-test comparison analysis of adaptive capacity instrument M t df P
Experimental Group 61 56.11 6.264 120 0.000
Control Group 60 37.07
Table 14 showed a significant difference in creativity skills between the experimental and control groups (df=120, t=6.261, p-value=, p<0.05). It showed that the experimental group had a higher average creativity value in the post-test than the control group. Therefore, the impact of the treatment was already apparent in the post-test scores after the implementation of the Constructivist Feedback-Based Assessment Method in the experimental group. It can be seen in the P-value > 0.05 that the null hypothesis is rejected. Based on the data, it can be interpreted that H0 is rejected, which means there are differences in the creativity skills of experimental and control group students after implementing the Constructivist Feedback-Based Assessment Method. This is in line with research conducted by (Jawad et al., 2021), which states that implementing feedback assessment methods significantly impacts the
development of students' creativity abilities. Feedback given to students can stimulate the exploration of new ideas. By understanding the positive aspects of their creative ideas, students become more motivated to try different approaches and expand the boundaries of their creativity (Balakrishnan, 2022); when students get positive feedback and feel supported in their creative ideas, they feel more ownership of that creativity. This sense of ownership increases the motivation to continue developing and expressing their creativity (Cai et al., 2020). Feedback assessment methods can expand and enrich students' creativity skills by providing constructive and supportive feedback. This is not only beneficial in the context of formal education but also prepares students to face creative challenges in daily life and the future (Fortuna et al., 2023; Sansi et al., 2023).
Linear Regression Analysis
Linear regression analysis is a statistical method to understand a study's relationship between two or more variables. In linear regression, the main objective is understanding how the dependent variable (y) relates to one or more independent variables (x). This relationship can be linear, which means that constant changes in the independent variables can explain changes in the dependent variable. In this study, there are four independent variables, namely adaptive capacity (X1), decision-making (X2), problem-solving (X3), and creativity (X4), and one dependent variable, namely Constructivist Feedback-Based Assessment Method (Y). So, the linear regression equation is as follows.
y = a + bx1 + bx2 + bx3 + bx4 + £ (1)
Here, y is the dependent variable, x is the independent variable, a is the intercept (the value of y when x = 0), b is the regression coefficient (shows how much change is expected when increasing by one unit), and £ is the prediction error. Before conducting a linear regression analysis, the analytical prerequisite tests of normality (Figure 3a) and linearity (Figure 3b) must be performed first.
(a3) <b3) (a4) (b4)
Figure 3. Results of QQ-Plots Normality (a) and Scatter Plots Linearity (b) of research variables
Figure 3 above shows that the distribution of research data was normal and linear. Based on the results of the Saphiro-Wilk analysis, the adaptive capacity variable (Figure 3a1) [p > 0.05, W = 0.76], the decision-making variable (Figure 3a2) [p > 0.05, W = 0.92], the problem-solving variable (Figure 3a3) [p > 0.05, W = 0.81], and the creativity variable (Figure 3a4) [p > 0.05, W = 0.79] indicate that the research data are normal. In addition, the scatter plot illustration on each variable shows that the observation
points are evenly distributed, indicating that the linearity assumption has been met. Therefore, it can be concluded that this research data is eligible for parametric analysis, namely linear regression analysis (Table 11), because it meets the analysis requirements test.
Table 15. Linear Regression Analysis Results
B t Sig. r2
Constant 16.577
Adaptive Capacity 0.740 9.166 0.000 0.71
Decision-Making 0.766 8.460 0.000 0.75
Problem-Solving 0.806 9.376 0.001 0.82
Creativity 0.788 9.643 0.000 0.77
Sample 121
P-Value 0.000
F Value 18.784
Regression Equation y = 16.577 + 0.740x1 + 0.766x2 + 0.806x3 + 0.788x4 + e
In this Linear Regression analysis, the pre-test was used to compare adaptive capacity, decisionmaking, problem-solving, and creativity abilities between the experimental and control groups after being given a pre-test before and post-test after treatment. The Linear Regression Analysis presented in Table 15 showed that there is a significant difference between the experimental group and the control group after implementing the Constructivist Feedback-Based Assessment Method treatment (F (l, 119) = 18.784, p < 0.05). There was a significant difference between the experimental group students and the control group in the ability of adaptive capacity (p < 0.05), decision-making (p < 0.05), problem-solving (p < 0.05), and creativity (p < 0.05), where the experimental group students have better abilities than the control group (Figure 4) after the implementation of Constructivist Feedback-Based Assessment Method. Table 15 also showed that 71% adaptive capacity, 75% decision-making, 82% problem-solving, and 77% creativity influence Constructivist Feedback-Based Assessment Method.
Figure 4. Comparison of Experimental and Control Class Skill Percentages
This study's results align with (van der Kleij, 2019), which explains that feedback assessment provides positive results on student skills. Implementing constructivist feedback-based assessment methods allows learners to know what and why they will learn to become active participants in the passive learning process. When introducing a new topic, learners must share the objectives they need to get good results and notes (Oliveira et al., 2021). From the beginning of learning, learners are responsible for their learning, allowing each to create their knowledge of the subject, cooperate with their peers and educators, expand their framework, and move towards better knowledge and understanding of complex subjects (Falloon, 2020). One of the benefits of sharing learning objectives with learners is that they will be given tasks that match the learning objectives. According to (Ibarra-saiz et al., 2020), effective assessment is applied by providing feedback during learning to regulate the teaching and learning process to improve learners' achievement. According to (Tang et al., 2020), assessment can be considered a valid and essential part of integrating teaching and assessment. Assessments inform educators about whether learners have learned, and they have qualifying indicators of how educators should plan subsequent lessons (Firestone and Donaldson, 2019). There are four main components to assessment (Morales, 2022): (i) clarifying learning objectives and success criteria; (ii) improving the quality of inquiry/dialogue; (iii) improving the quality of marking/feedback/recording; and (iv) using self and peer assessment.
One of the key elements of assessment is asking questions (Guangul et al., 2020). Educators can use one-third of their teaching time to ask learners questions (Alt et al., (2023)). Asking questions in the form of a feedback-based assessment is essential for gaining information about learners' learning and understanding. This goal can be achieved if questions are active and influential in determining and constructing the learner's depth of knowledge (Mohammed et al., 2020). Feedback is at the heart of this assessment method (Waskito et al., 2023). The impact of this assessment method arises from the power of feedback given to learners about their learning (Brown, 2019). According to Shute [49], feedback is information sent to learners that enables or encourages them to regulate thoughts or behaviors to improve learning. According to (Brown, 2019), feedback provided through assessment significantly benefits learners' motivation, helps learners improve the quality of learning, strengthens learners' memory, and gives learners a profile of learning.
Conclusions
This study developed the Constructivist Feedback-Based Assessment Method. It looked at the effectiveness of implementing the Constructivist Feedback-Based Assessment Method on students' adaptive capacity, decision-making, problem-solving, and creativity. The results show that the Constructivist Feedback-Based Assessment Method effectively improves students' adaptive capacity, decision-making, problem-solving, and creativity. It also proves that a suitable assessment method can affect students' skills. This research contributes to existing knowledge, especially on assessment in learning, where this research provides a constructive assessment method that can be used as a consideration for teachers to apply to learning. The limitation of this research is that the constructivist feedback-based assessment method that has been developed has only been applied to see the students' adaptive capacity, decisionmaking, problem-solving, and creativity. It is hoped that this constructivist feedback-based assessment method can be applied for future research to see its effectiveness on other students' skills and abilities.
Acknowledgments
The author would like to thank Lembaga Penelitian dan Pengabdian Masyarakat Universitas Negeri Padang for funding this work with contract number 1045/UN35.13/LT/2022.
Conflict of interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Author Contributions
Conceptualization, W.W., R.E.W.; Methodology, W.W., R.R., R.E.W.; Formal analysis, R.E.W., A.F.; Validation, A.N., writing—original draft preparation: W.W, R.E.W.; writing—review and editing, R.E.W.,
A.F., S.'A.J.; Visualization: W.W., R.R., A.F., A.N. All authors have read and agreed to the published version of the manuscript.
References
Aka, K. A. (2019). Integration Borg & Gall (1983) and Lee & Owen (2004) models as an alternative model of design-based research of interactive multimedia in elementary school. Journal of Physics: Conference Series, 1318(1), 1-9. https:// doi.org/10.1088/1742-6596/1318/1/012022 Alt, D., Naamati-Schneider, L., & Weishut, D. J. N. (2023). Competency-based learning and formative assessment feedback as precursors of college students' soft skills acquisition. Studies in Higher Education, 48(12), 1901-1917. https://doi.or g/10.1080/03075079.2023.2217203 Arsovic, B., & Stefanovic, N. (2020). E-learning based on the adaptive learning model: case study in Serbia. Sadhana -
Academy Proceedings in Engineering Sciences, 45(1), 1-13. https://doi.org/10.1007/s12046-020-01499-8 Baidal-Bustamante, E., Mora, C., & Alvarez-Alvarado, M. S. (2023). STEAM Project-Based Learning Approach to Enhance Teaching-Learning Process in the Topic of Pascal's Principle. IEEE Transactions on Education, 1-10. https://doi. org/10.1109/TE.2023.3283850
Balakrishnan, B. (2022). Exploring the impact of design thinking tool among design undergraduates: a study on creative skills and motivation to think creatively. International Journal of Technology and Design Education, 32(3), 1799-1812. https:// doi.org/10.1007/s10798-021-09652-y Bragg, L. A., Walsh, C., & Heyeres, M. (2021). Successful design and delivery of online professional development for teachers: A systematic review of the literature. Computers & Education, 166, 104158. https://doi.org/10.1016/j. compedu.2021.104158
Brown, G. T. L. (2019). Is Assessment for Learning Really Assessment? Frontiers in Education, 4(June), 1-7. https://doi. org/10.3389/feduc.2019.00064
Cáceres, M., Nussbaum, M., González, F., & Gardulski, V. (2021). Is more detailed feedback better for problem-solving? Interactive Learning Environments, 29(7), 1189-1210. https://doi.org/10.1080/10494820.2019.1619595
Cai, W., Khapova, S., Bossink, B., Lysova, E., & Yuan, J. (2020). Optimizing employee creativity in the digital era: Uncovering the interactional effects of abilities, motivations, and opportunities. International Journal of Environmental Research and Public Health, 17(3), 1-19. https://doi.org/10.3390/ijerph17031038 Duan, R., Kawahara, T., Dantsuji, M., & Nanjo, H. (2020). Cross-lingual transfer learning of non-native acoustic modeling for pronunciation error detection and diagnosis. IEEE/ACM Transactions on Audio Speech and Language Processing, 28(c), 391-401. https://doi.org/10.1109/TASLP.2019.2955858 Falloon, G. (2020). From digital literacy to digital competence: the teacher digital competency (TDC) framework. Educational
Technology Research and Development, 68(5), 2449-2472. https://doi.org/10.1007/s11423-020-09767-4 Fazel, I., & Ali, A. M. (2022). EAP teachers' knowledge and use of learning-oriented assessment: A cross-contextual study. System, 104, 102685. https://doi.org/10.1016Zj.system.2021.102685
Firestone, W. A., & Donaldson, M. L. (2019). Teacher evaluation as data use: what recent research suggests. Educational Assessment, Evaluation and Accountability, 31(3), 289-314. https://doi.org/10.1007/s11092-019-09300-z
Fortuna, A., Waskito, W., Purwantono, P., Kurniawan, A., Andriani, W., & Alimin, M. (2023). Designing Learning Media Using Augmented Reality for Engineering Mechanics Course. Journal of Engineering Researcher and Lecturer, 2(1), 18-27. https://doi.org/10.58712/jerel.v2i1.20 Fuchs, L. S., & Fuchs, D. (1986). Effects of Systematic Formative Evaluation: A Meta-Analysis. Exceptional Children, 53(3),
199-208. https://doi.org/10.1177/001440298605300301 Furtak, E. M., Ruiz-Primo, M. A., Shemwell, J. T., Ayala, C. C., Brandon, P. R., Shavelson, R. J., & Yin, Y. (2008). On the fidelity of implementing embedded formative assessments and its relation to student learning. Applied Measurement in Education, 21(4), 360-389. https://doi.org/10.1080/08957340802347852 Guangul, F. M., Suhail, A. H., Khalit, M. I., & Khidhir, B. A. (2020). Challenges of remote assessment in higher education in the context of COVID-19: a case study of Middle East College. Educational Assessment, Evaluation and Accountability, 32(4), 519-535. https://doi.org/10.1007/s11092-020-09340-w
Gulikers, J. T. M., Biemans, H. J. A., Wesselink, R., & van der Wel, M. (2013). Aligning formative and summative assessments: A collaborative action research challenging teacher conceptions. Studies in Educational Evaluation, 39(2), 116-124. https://doi.org/10.1016/j.stueduc.2013.03.001 Handrayani, D., Rahmadani, K., Baqi, F. A., & Kassymova, G. K. (2023). Education Transformation in Era 4.0: The Effect of Learning Facilities on Student Learning Outcomes. Journal of Computer-Based Instructional Media, 1(1), 34-43. https://doi.org/10.58712/jcim.v1i1.106
Hartmann, M. (2019). Development of circle learning media to improve student learning outcomes. Journal of Physics: Conference Series, 1-7. https://doi.org/10.1088/1742-6596/1321/2/022099
Hattie, J. (2008). A Synthesis of Over 800 Meta-Analyses Relating to Achievement. In Visible Learning (pp. 1-392). https://doi. org/10.4324/9780203887332
Hennessy, S., D'Angelo, S., McIntyre, N., Koomar, S., Kreimeia, A., Cao, L., Brugha, M., & Zubairi, A. (2022). Technology Use for Teacher Professional Development in Low- and Middle-Income Countries: A systematic review. Computers and Education Open, 3(\), 100080. https://doi.org/10.1016/j.caeo.2022.100080 Ibarra-sáiz, M. S., Rodríguez-gómez, G., & Boud, D. (2020). Developing student competence through peer assessment: the role of feedback, self-regulation and evaluative judgement. Higher Education, 80(1), 137-156. https://doi.org/10.1007/ s10734-019-00469-2
Jalinus, N., Wulansari, R. E., Heong, Y. M., & Kiong, T. T. (2023). Teaching activities for supporting students' 4cs skills development in vocational education. Journal of Engineering Researcher and Lecturer, 2(2), 28-37. https://doi. org/10.58712/jerel.v2i2.95
Jawad, L. F., Majeed, B. H., & Alrikabi, H. T. S. (2021). The Impact of Teaching by Using STEM Approach in The Development of Creative Thinking and Mathematical Achievement Among the Students of The Fourth Scientific Class. International Journal of Interactive Mobile Technologies, 15(13), 172-188. https://doi.org/10.3991/ijim.v15i13.24185 Karaman, P. (2021). The Impact of Self-assessment on Academic Performance: A Meta-analysis Study. International Journal
of Research in Education and Science, 7(4), 1151-1166. https://doi.org/10.46328/ijres.2344 Kardoyo, Nurkhin, A., Muhsin, & Pramusinto, H. (2020). Problem-based learning strategy: Its impact on students' critical and creative thinking skills. European Journal of Educational Research, 9(3), 1141-1150. https://doi.org/10.12973/EU-JER.9.3.1141
Lacey, R. E., & Minnis, H. (2020). Practitioner Review: Twenty years of research with adverse childhood experience scores - Advantages, disadvantages and applications to practice. Journal of Child Psychology and Psychiatry and Allied Disciplines, 61(2), 116-130. https://doi.org/10.1111/jcpp.13135 Middleton, T., ahmed Shafi, A., Millican, R., & Templeton, S. (2023). Developing effective assessment feedback: academic buoyancy and the relational dimensions of feedback. Teaching in Higher Education, 28(1), 118-135. https://doi.org/10 .1080/13562517.2020.1777397
Mohammed, Husam, S., & Kinyo, L. (2020). The role of constructivism in the enhancement of social studies education. Journal
of Critical Reviews, 7(7), 249-256. https://www.scirp.org/reference/referencespapers?referenceid=3017074 Morales, J. (2022). The Evaluation of Teacher Performance in Higher Education. International Journal of Science and Society,
4(3), 140-150. https://doi.org/10.54783/ijsoc.v4i3.507 Mubai, A., Rukun, K., Giatman, M., & Edidas, E. (2020). Needs Analysis in Learning Media Development Based on Augmented Reality (AR) for Computer Network Installation Courses. Jurnal Pendidikan Teknologi Kejuruan, 3(1), 31-35. https:// doi.org/10.24036/jptk.v3i1.3723
Oliveira, G., Grenha Teixeira, J., Torres, A., & Morais, C. (2021). An exploratory study on the emergency remote education experience of higher education students and teachers during the COVID-19 pandemic. British Journal of Educational Technology, 52(4), 1357-1376. https://doi.org/10.1111/bjet.13112 Phelps, R. P. (2019). Test Frequency, Stakes, and Feedback in Student Achievement: A Meta-Analysis. Evaluation Review,
43(3-4), 111-151. https://doi.org/10.1177/0193841X19865628 Prasetya, F., Fajri, B. R., Wulansari, R. E., Primawati, & Fortuna, A. (2023). Virtual Reality Adventures as an Effort to Improve the Quality of Welding Technology Learning During a Pandemic. International Journal of Online and Biomedical Engineering (IJOE), 19(2), 4-22. https://doi.org/10.3991/ijoe.v19i02.35447
Prasetya, F., Syahri, B., Fajri, B. R., Wulansari, R. E., & Fortuna, A. (2023). Utilizing Virtual Laboratory to Improve CNC Distance Learning of Vocational Students at Higher Education. TEM Journal, 12(3), 1506-1518. https://doi.org/10.18421/ TEM123-31
Sánchez-Ramírez, J. M., Íñigo-Mendoza, V., Marcano, B., & Romero-García, C. (2022). Design and Validation of an Assessment Rubric of Relevant Competencies for Employability. Journal of Technology and Science Education, 12(2), 426-439. https://doi.org/10.3926/jotse.1397 Sansi, A. S., Rini, F., Mary, T., & Kiong, T. T. (2023). The Development of Android-based Computer and Basic Network Learning
Media. Journal of Computer-Based Instructional Media, 1(2), 1-13. https://doi.org/10.58712/jcim.v1i2.19 Sims, W. A., King, K. R., Preast, J. L., Burns, M. K., & Panameño, S. (2023). Are School-Based Problem-Solving Teams Effective? A Meta-Analysis of Student- and Systems-Level Effects. Journal of Educational and Psychological Consultation, 33(4), 1-25. https://doi.org/10.1080/10474412.2023.2232785
Su, F., Yang, X., Yin, J., Shen, Y., & Tan, L. (2023). Validity of Using Pathological Response as a Surrogate for Overall Survival in Neoadjuvant Studies for Esophageal Cancer: A Systematic Review and Meta-analysis. Annals of Surgical Oncology, 30(12), 7461-7471. https://doi.org/10.1245/s10434-023-13778-9 Surucu, L., & Maslakci, A. (2020). Validity and Realibity In Quantitative Research. Business & Management Studies: An International Journal, 8(3), 2694-2726. https://doi.org/10.15295/bmij.v8i3.1540
Taherdoost, H. (2022). What are Different Research Approaches? Comprehensive Review of Qualitative, Quantitative, and Mixed Method Research, Their Applications, Types, and Limitations. Journal of Management Science & Engineering Research, 5(1), 53-63. https://doi.org/10.30564/jmser.v5i1.4538 Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical
studies. Computers and Education, 148, 103798. https://doi.org/10.1016Zj.compedu.2019.103798 Teras, M., Suoranta, J., Teras, H., & Curcher, M. (2020). Post-Covid-19 Education and Education Technology 'Solutionism': a
Seller's Market. Postdigital Science and Education, 2(3), 863-878. https://doi.org/10.1007/s42438-020-00164-x Torres, J. T., Higheagle Strong, Z., & Adesope, O. O. (2020). Reflection through assessment: A systematic narrative review of teacher feedback and student self-perception. Studies in Educational Evaluation, 64, 100814. https://doi.org/10.1016/j. stueduc.2019.100814
van der Kleij, F. M. (2019). Comparison of teacher and student perceptions of formative assessment feedback practices and association with individual student characteristics. Teaching and Teacher Education, 85, 175-189. https://doi. org/10.1016/j.tate.2019.06.010
Waskito, Irzal, Wulansari, R. E., & Ya, K. Z. (2022). The Adventure of Formative Assessment with Active Feedback in The Vocational Learning : The Empirical Effect for Increasing Students ' Achievement. Journal of Technical Education and Training, 14(1), 54-62. https://doi.org/10.30880/jtet.2022.14.01.005 Waskito, Wulansari, R. E., Syahri, B., Erizon, N., Purwantono, Yufrizal, & Kiong, T. T. (2023). Countenance Evaluation of Virtual Reality (VR) Implementation in Machining Technology Courses. Journal of Applied Engineering and Technological Science, 4(2), 825-836. https://doi.org/10.37385/jaets.v4i2.1917
Wetzel, M., Zufferey, S., & Gygax, P. (2020). Second language acquisition and the mastery of discourse connectives: Assessing the factors that hinder l2-learners from mastering french connectives. Languages, 5(3), 1-26. https://doi.org/10.3390/ languages5030035
Wisniewski, B., Zierer, K., & Hattie, J. (2020). The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback
Research. Frontiers in Psychology, 10, 1-14. https://doi.org/10.3389/fpsyg.2019.03087 Wulansari, R. E., & Nabawi, R. A. (2021). Efforts to Improve Problem Solving Skills and Critical Thinking Skills Through Problem-Based Integrated Computer Assisted Instruction (CAI) in Vocational Education. Jurnal Pendidikan Teknologi Kejuruan, 4(4), 111-117. https://doi.org/10.24036/jptk.v4i2.21123 Yan, Z., Wang, X., Boud, D., & Lao, H. (2023). The effect of self-assessment on academic performance and the role of explicitness: a meta-analysis. Assessment and Evaluation in Higher Education, 48(1), 1-15. https://doi.org/10.1080/0 2602938.2021.2012644
