CC BY
35
УДК 37.036.5
DOI: 10.17853/1994-5639-2022-3-104-125
DEVELOPMENT OF HIGH ORDER THINKING SKILLS IN INDONESIAN TEACHERS
J. Surjanti1, A. F. Prakoso*2, R. Y. Kurniawan3, N. C. Sakti4, E. I. Nurlaili5
Universitas Negeri Surabaya, Surabaya, Indonesia.
E-mail: 'junsurjanti@unesa.ac.id; 2albrianprakoso@unesa.ac.id; 3rizakurniauan@unesa.ac.id;
4noridacanda@unesa.ac.id; 5ekanurlaili@unesa.ac.id
*Corresponding author
Abstract. Introduction. The 21st century teachers need be able to create students who are creative and have critical thinking skills which are possibly obtained by implementing high order thinking skills (HOTS) in learning activities.
Aim. This study aims to examine whether teacher professional knowledge (TPK) can improve teacher's critical thinking (CT) and whether HOTS literacy (HL) and teacher professional development (TPD) can mediate the relationship between TPK and CT.
Methodology and research methods. This study is explanatory research with 724 teachers serves as respondents. The analysis was carried out using WarpPLS and data were collected through questionnaire.
Results and scientific novelty. The research results demonstrate that TPK affect CT directly and indirectly. HL and TPD intermediate the relationship of TPK on CT.
Practical significance. This study provides important information to relevant stakeholders, such as the Ministry of education, universities, and schools which explains or proposes factors that can improve teachers' ability in developing critical thinking or higher-order thinking based questions. This information needs to be immediately known so stakeholders or policy makers can design appropriate regulations or policy that can develop teacher capacity and education curriculum.
Keywords: Teacher Profesional Development/TPD, Teacher Professional Knowledge/ TPK, Long Time Teaching/LTT, Experience Teacher/ET, HOTS Literacy/HL, Source Tracking/ ST, HOTS Indicator/HI, Critical Thinking/CT.
Acknowledgements. This research was supported by Universitas Negeri Surabaya (Surabaya, Indonesia).
For citation: Surjanti J., Prakoso A. F., Kurniawan R. Y., Sakti N. C., Nurlaili E. I. Development of high order thinking skills in Indonesian teachers. The Education and Science Journal. 2022; 24 (3): 104-125. DOI: 10.17853/1994-5639-2022-3-104-125
ФОРМИРОВАНИЕ НАВЫКОВ МЫШЛЕНИЯ ВЫСОКОГО ПОРЯДКА (HOTS) У ПРЕПОДАВАТЕЛЕЙ В ИНДОНЕЗИИ
Дж. Сурджанти1, А. Ф. Пракосо2, Р. Ю. Курниаван3, Н. К. Сакти4, Э. И. Нурлайли5
Государственный университет Сурабая, Сурабая, Индонезия.
E-mail: 'junsurjanti@unesa.ac.id; 2albrianprakoso@unesa.ac.id; 3rizakurniawan@unesa.
ac.id; 4noridacanda@unesa.ac.id; 5ekanurlaili@unesa.ac.id
Аннотация. Введение. Учителя XXI века должны быть в состоянии воспитать творческих учеников, которые мыслят критически, за счет развития навыков мышления высокого порядка (HOTS) в процессе обучения.
Цель. Настоящее исследование направлено на изучение того, могут ли профессиональные знания учителей улучшить критическое мышление и могут ли грамотность в применении HOTS и профессиональное развитие учителей опосредовать их взаимосвязь.
Методология и методы исследования. Данное исследование носит пояснительный характер при общей выборке 724 учителей. Анализ проводился с использованием WarpPLS, а данные собирались с помощью анкет.
Результаты и научная новизна. Результаты исследования показали, что профессиональные знания учителей влияют на критическое мышление прямо и косвенно. Грамотность в применении HOTS и профессиональное развитие учителей являются промежуточными звеньями между профессиональными знаниями учителей и критическим мышлением.
Практическая значимость. Настоящее исследование предоставляет важную информацию соответствующим заинтересованным сторонам, таким как Министерство образования, университеты и школы, относительно того, какие факторы могут улучшить способности учителей при подготовке вопросов, направленных на критическое мышление (мышление высокого порядка). Эти факторы должны быть немедленно обнаружены и изучены заинтересованными сторонами, чтобы поддержать регулирование развития потенциала учителей, а также текущий процесс разработки учебных программ.
Ключевые слова: профессиональное развитие учителей, профессиональные знания учителей, долгосрочное обучение, опытный учитель, грамотность в применении HOTS, отслеживание источников, индикатор HOTS, критическое мышление.
Благодарности. Исследование было поддержано Universitas Negeri Surabaya (Сурабая, Индонезия).
Для цитирования: Сурджанти Дж., Пракосо А. Ф., Курниаван Р. Ю., Сак-ти Н. К., Нурлайли Э. И. Формирование навыков мышления высокого порядка (HOTS) у преподавателей в Индонезии // Образование и наука. 2022. Т. 24, № 3. С. 104-125. DOI: 10.17853/1994-5639-2022-3-104-125
Introduction
Critical thinking skills are ideal thinking [1]. The 21st century teachers must have the ability to instill high order thinking (HOTS) in students in order to be able to apply, analyse, synthesise and evaluate information processing [2]. Higher order thinking skills (HOTS) are one of the important aspects in education to provide metacognitive skills [3]. Therefore, educators began to be interested in teaching thinking skills with information and teaching content developed on metacognitive skills, which were defined as meta-learning [4]. Me-ta-teaching strategies can help mediate metacognitive skills [5] and the key to metacognitive thinking as a key element in the transfer of learning [6].
HOTS can foster critical thinking skills if evaluated with the right evaluation technique [7]. It is necessary to evaluate critical thinking questions to measure the application of HOTS learning. The HOTS assessment can improve students' critical thinking skills [8]. Hence, the evaluation should be adapted to current needs [9]. Teachers are required to use certain assessment techniques or explore problems by involving students for assessment [10]. The professional educators must always provide feedback on the learning process to improve the quality of learning [11].
There were still discrepancies between the assessment design and the importance of the final assessment [12]. However, Willingham [7] also found that the implementation of HOTS is still not being implemented, which is disappointing. The problem of the low ability to compose HOTS questions was found in HOTS seminar participants, it was proven that teachers had difficulty in preparing HOTS questions. Caution is needed in assessing to maximise learning [13]. Learning with HOTS includes higher-order thinking skills, it is necessary for the creativity of teachers to master HOTS evaluation techniques [2]. The phenomenon of inequality between teachers' abilities in adjusting the evaluation of HOTS. It is interesting to study about the antecedents that influence critical thinking.
A coherent and systematic professional development for teachers is essential [14]. Previous research shows that certification programmes for professional development are important in Indonesia, currently there are certified and non-certified teachers [15]. Moreover, non-certified teachers have heavy responsibilities with limited welfare, financial limitations and no social support [16]. It is important to observe the gap between certified and non-certified teachers in the preparation of HOTS questions.
Teachers must have sufficient experience to develop evaluation questions that can foster critical thinking [17]. Teaching experience is able to predict the ability of teachers [18]. Even though it takes a long time to develop critical think-
ing skills [19]. Allard & Doecke stated that novice teachers and experienced teachers have different professional knowledge statuses [20]. The teacher must function as a facilitator, who mediates learning to be able to trigger students' meta-cognitive thinking [19]. However, teachers with less than 1 year of experience tended to be higher than 1-5 years [18]. It is important to research the experience of professional teachers [21].
Critical thinking can be derived through HOTS-based learning indicators [22]; hence, it is important that learning should be designed using good performance indicator criteria [23]. According to Albitz [24], teachers are responsible for having information literacy and critical thinking skills in learning because critical thinking is regarded as a deliberate metacognitive process to generate conclusions based on logical arguments and problem solutions [25]. In critical thinking, there are components that include skills in analysing arguments, making conclusions using inductive or deductive reasoning, assessing or evaluating, and making decisions or solving problems. Many studies discuss the importance of teacher literacy because literacy is assumed to be an aid for critical thinking [19-21]. Teacher literacy can be measured by their ability to trace sources and their ability to reduce critical thinking indicators on HOTS-based questions. However, from the results of preliminary interviews, facts showed that many teachers did not have prior understanding whether on how to make HOTS-based questions or on how to apply their knowledge in making questions by applying critical thinking skills. Referring to the phenomena and initial data obtained, this study aims to analyse the relationship between teacher professional development (TPD), teacher professional knowledge (TPK), HOTS literacy, and critical thinking in HOTS-based questions making.
Literature Review
1. Critical Thinking
HOTS-based learning offers an alternative and open search for critical thinking which has been developed from Bloom [27]. This idea is supported by many researchers who generally state that critical thinking is an important feature for problem-solving and decision-making [6, 7, 25, 27, 29]. Paul & Elder [28] stated that a critical thinker can 1) ask vital questions and problems, formulate them clearly and precisely, as well as collect and assess relevant information using abstract ideas to interpret it effectively; 2) reach reasonable conclusions and solutions and test them against relevant criteria and standards; 3) think openly in alternative systems of thinking, recognise and assess them based on assumptions, implications, and practical consequences; as well as 4) communicate effectively with others in finding solutions of complex problems.
Critical thinking skill will affect lives in the future because it serves as the basis of the ability to solve problems as well as make a decision, good judgment, and analysis [29]. Critical thinking skills allow individuals to broaden their perspectives and navigate their decisions in their lives. In higher education, critical thinking skills consider an important skill [1]. According to Goddu [4], critical thinkers are people who can move beyond "typical" thinking models to advance their ways of thinking. These people usually 1) acknowledge personal limitations; 2) look at the problem as a challenge; 3) understand their goals; 4) use evidence to make judgments; 5) are interested in other people's ideas; 6) challenge/doubt extreme views; 7) think before acting; 8) avoid emotionalism; 9) keep open-minded; and 10) engage in active listening. Critical thinking is reasonable and reflective thinking that focuses on deciding what to believe or do. This definition is able to describe the essence of this term [30].
Surface-based performance appraisal (actual performance appraisal) seems to be the best way to assess a person's critical thinking disposition
[32]. When a person is assessed using performance-based assessments on the surface, the assessment will focus on their performance and the things that should be done in connection with that performance [31], including critical thinking. Critical thinking is a skill and disposition of individual development
[33]. This dimension includes higher-order thinking which refers to critical, logical, reflective, metacognitive, and creative patterns [34]. According to Norris & Ennis [35], in measuring the level of critical thinking, some tests still contain content that cannot truly describe the appropriate level of critical thinking. In line with that, Ennis [36] explains that the term 'critical thinking' is used only to describe reasonable reasons used when doing or analysing something. However, this definition has not explained the level of effectiveness of thinking and metacognitive understanding through deliberate reflective assessment, thereby increasing the possibility of drawing logical conclusions to solve a problem [25]. In learning activities, critical thinking becomes very important because it allows individuals to gain a more complex understanding of the information they encounter and promotes good decision making and problem solving in real world applications.
The critical thinking is a metacognitive process that, through deliberate reflective judgment, increases the likelihood of generating a logical conclusion for an argument or a solution to a problem [32]. Instruction in critical thinking becomes very important because it allows individuals to gain a more complex understanding of the information they encounter and promote good decision making and problem solving in real world applications. Higher order thinking skills (HOTS) assessment instrument consists of C4 (analysis), C5 (evaluation), and C6 (creation) [37]. The research by Ennis [28], based on Bloom's theory,
states that there are aspects in critical thinking, namely: 1) having an open mind and paying attention to alternatives; 2) have a desire to get good information; 3) able to assess the credibility of sources; 4) able to identify conclusions, reasons, and assumptions; 5) able to ask appropriate clarifying questions; 6) able to assess the quality of the argument, including reasons, assumptions, evidence, and level of support for conclusions; 7) able to develop and maintain a position that makes sense well and does justice to the challenges; 8) able to formulate a reasonable hypothesis; 9) able to plan and conduct experiments well; 10) able to define terms according to context; 11) able to draw careful conclusions; and 12) able to integrate all items in the list. Accordingly, this study is purposely to develop critical thinking indicators developed by Ennis for teachers in compiling HOTS-based questions regarding to the empirical data obtained.
2. Teacher Professional Development/TPD
Teacher professional development refers to the development of teacher content, pedagogy, and technology used for future development [35]. Likewise, according to Soti [36], teacher professional development contributes to overcoming problems caused by students' poor outcomes and increasing student engagement. Teacher professional development is provided during the implementation of action plans reflecting on student evaluations [37] as supported by a study conducted by Romero & Vasilopoulos [38] on the professional development programmes in China. In addition, Gupta & Lee [39] believe that professional development is intended to improve teacher knowledge and student learning in order to systematically improve student achievement in basic literacy. Meanwhile, according to Liew, Tan, & Ganapathy [40] higher-order thinking skills (HOTS) use information and communication technology (ICT) in teacher training to instill and develop critical thinking skills and professionalism. This study purposely intends to prove that certification programmes for teacher professional development programme in Indonesia possibly affect teachers' ability to cultivate critical thinking and high order thinking skill.
3. Teacher Professional Knowledge/TPK
Previous research [41] found that mentors' knowledge of practice-oriented teaching emerged from their professional experience. Professional knowledge for conceptualising knowledge is needed for teaching practice [21]. Therefore, from a professional perspective, teachers are considered knowledgeable and professional when they get certification about their abilities [42]. To encourage teacher reflection on professional knowledge, Zanin-Yost & Freie mention the need for proof-of-concept of graphic assessments/TPACK instruments [43]. This study responds to research conducted by Tsang [44], who found a relationship
between career goals and teaching experience mediated by the economic environment. This is in accordance with Cooke's research [48], which states that development training can be useful feedback for the development and improvement of lessons. This result is also the basis for this research to reveal the relationship between professionalism, development, and teacher knowledge through the practice of professional development (training) and teaching. Based on this thought, the researcher developed indicators of teaching experience developed by the researcher through length of teaching, participating in teacher training.
4. HOTS literacy
Efforts to search or search for sources of information and literacy make a person able to cultivate critical thinking skills [46]. This is in accordance with Potzsch's opinion, who states that technology has a role that can be used comprehensively to stimulate knowledge, competence, and constructive skills in teaching and learning [47]. In addition, Cooke [48] mentions that literacy skills can be grown from the habit of reading and searching for available information, both manual and digital. This habit can then grow critical thinking skills [48]. Higher order thinking skills (HOTS) underlie various types of use of information and communication technology (ICT) in learning activities [40]. This situation then presents a challenge where teachers are asked to integrate ICT to promote learning. Phenomena like these are the pioneers of the emergence of HOTS-based learning [40]. In addition, the development of education and demands from the government require teachers to adjust their mindset and abilities in order to produce graduates (students) with high thinking abilities so that they can face the demands of existing developments. The ability to think high makes a person able to think critically about the object (learning) so that they can develop themselves to the demands given [49].
The curriculum that is now often used is structured on the basis of authentic assessment [23]. According to Jan Muller in designing the curriculum, there are four arrangements that must be met or what is often referred to as the "Authentic Assessment Toolbox", namely: 1) standard goals and objectives (which describe what learners should know and do); 2) authentic tasks (which is administered and designed to assess a learner's ability to apply knowledge and skills driven by real-world challenge standards); 3) criteria for indicators of good performance on assignments; and 4) rubric grading scales (which are used to assess student performance). In addition, Jampel et al. [53] state that in learning, evaluation is an activity to collect, process, and analyse data so that it can function as a source of information and recommendations in decision making. This opinion is supported by Chakraborty & Das [9], who state that an
evaluation is necessary but the evaluation should be carried out by reducing the limitations of closed questions so that they can actually be used as a measure of learner ability. In Bloom's taxonomy, it is stated that higher order thinking skills can be measured [22] and evaluated [32] as well as analysed quantitatively and qualitatively [35]. Accordingly, the researcher argues that evaluation techniques can be used to measure critical and high-level thinking skills in learners.
This study aims to measure the evaluation of the suitability of learning indicators and Bloom's taxonomy indicators developed by teachers against predetermined indicators. As explained by Tanujaya, Mumu, & Margono, students with higher order thinking skills tend to be more successful in their academic performance [3]. According to them, higher order thinking skills can be seen from three things, namely: 1) critical thinking skills; 2) creative thinking skills; and 3) systematic thinking skills [3]. This supports Zohar's opinion, which states that HOTS learning or the so-called teaching of higher order thinking skills can be developed through Bloom's taxonomy [2] through the level of application, analysis, synthesis and evaluation activities in processing information. Brookhart states that test questions and other assessments are for analysing, reasoning, problem solving, and creative thinking [5]. This is because in life, critical and high-level thinking skills are needed to ensure their success in the future.
Methods
1. Population and Sample
This study is explanatory research, which aims to determine the relationship among TPK (teacher professional knowledge), TPD (teacher professional development), HL (HOTS literacy), and CT (critical thinking) during HOTS teaching preparation. The population was economics teachers at junior and senior high schools located in East Java, Indonesia. Sampling was taken using sample random technique considering the voluntary participation of the respondents to answer/respond the questionnaires. Questionnaires had been distributed from January to April 2021 and successfully obtained 724 respondents.-
Based on the WarpPLS SEM requirement, sample should be provided at least five times of the indicators used in the analyses [54]. As this study used 9 indicators, the minimum sample to meet the standard for WarpPLS SEM are 45 (9x5). From the data obtained, this study successfully involved 724 participants/respondents; thus, the number of samples used considers as more than appropriate for the analysis.
2. Data Collection Method
The instrument of this study is online-based questionnaires consisting of several questions developed from 4 variable and 9 indicators (Table 1). Questionnaires were distributed to economics teachers who were included in the population set by the researcher through link spread in the social media. The data obtained from the questionnaires were analysed using the Warp Partial Least Square (WarpPLS) Structural Equation Model (SEM). This technique was employed in order to predict the overall picture of models containing the relationship of all factors as well as their collinear relation. The analyses consist of measurement model and structural model approaches.
Table 1
Variable and Indicator
Variable Indicator
Teacher professional knowledge/ TPK (X1) a. experience/teaching length (X1.1) b. experience/amount of attending training (X1.2)
Teacher professional development/ TPD (Y1) a. certification or non-certification (Y1.1)
HOTS literacy (Y2) a. news/source adoption capacity (Y2.1) b. suitability of questions with question indicators (Y2.2)
Critical thinking in the preparation of HOTS (Y3) a. basic classification (elementary clarification) (Y3.1) b. giving reasons for a decision (the basic for the decision) (Y3.2) c. further classification (advanced clarification) (Y3.3) d. allegation and integration (supposition and integration) (Y3.4)
3. Research Hypotheses
1. H1: teacher professional knowledge (TPK) is suspected to significantly
influence teacher professional development.
2. H2: teacher professional knowledge (TPK) is suspected to significantly
influence HOTS literacy (HL) in HOTS questions making.
3. H3: teacher professional knowledge (TPK) is suspected to significantly
influence critical thinking (CT) in HOTS questions making.
4. H4: HOTS literacy (HL) is suspected to significantly influence critical
thinking (CT) in HOTS questions making.
5. H5: teacher professional development (TPD) is suspected to significantly
critical thinking (CT) in HOTS questions making.
6. Hб : tvacher profovsional knowledge (TPK) is suspeoted to significontly influonce critical thinking (CT)im loacTer professlsnaldevelopment (TPD) mediated by HOTS literacy (HL).
Fig. 1. Research framework Results and Discussion
1. Results
This study was conducted among 724 teachers and economic olympiad coaches, who accompanied their students to take part in the economic olympiad in East Java and were willing to participate in this study by filling the online questionnaires. This study purposedly find out an empirical evidence of the hypoth-esesthroughWarpPLS.According to the WarpPLSanalysis, it is foundout that:
1. H1 is accepted, meaning that teacher professional knowledge signif-
icantly influenced teacher professional development (TPK influenced TPD) .
2. H2 is accepted, meaning that teacher professional knowledge signifi-
cantly influenced HOTS literacy in questions making (TPK influenced HL).
3. H3 is accepted, meaning that teacher professional knowledge signifi-
cantly influenced critical thinking in HOTS questions making (TPK influenced CT).
4. H4 is accepted, HOTS literacy significantly influenced critical thinking
in questions making (HL influenced CT) .
5. H5 is rejected, meaning that teacher professional development did not
significantly influence critical thinking in HOTS questions making (TPD did not influence CT).
6. H6 is accepted, meaning that teacher professional knowledge signifi-
cantly influenced critical thinking in HOTS questions making me-
diated by teacher professional development and HOTS literacy in questions making.
R2 = 0.12 Rz = 005
Fig. 2. Hypothesis model
Measurement model
Measurement model test is to test the discriminant validity and convergent validity. Firstly, in discriminatory validity, it is to determine the degree to which two similar concepts are conceptually different. In addition, an empirical test is conducted to re-examine the relationship among measures. Although the relationship based on a summation scale is similarly correlated, they were actually and conceptually different. From the discriminant validity test, the results show that the summation scale is quite different from other similar concepts [55]. Therefore, it indicates that the data used in the analyses were sufficient and met the requirement set (Table 2).
Table 2
Discriminant validity
Variables X1 Y1 Y2 Y3
Teacher professional knowledge (X1) 0.847
Teacher professional development (Y1) 1.000
HOTS literacy (Y2) 0.871
Critical thinking in HOTS questions making (Y3) 0.833
Secondly, the convergent validity assesses the extent to which two measures of the same concept are correlated. Here, researchers determine to find out the alternative measures of a concept and correlate them with a summation scale. The high correlation results indicate the intended concept measurement scale and explain the standard used in the analysis [55]. According to Hair Jr. et al., when the results of values are above 1.0 or below -1.0, they are declared to meet a reasonable range and possibly serve as an important indicator of problems in the model [5]. Based on the results of the convergent validity test, all values measured in the study have met the appropriate range or passed the convergent validity test (Table 3).
Table 3
Convergent validity
Variables Construct item Standardised loading Cronbach's alpha AVE Composite reliability
Teacher professional X1.1 0.847 0.608 0.847 0.836
knowledge X1.2 0.847
(X1)
Teacher professional Y1.1 1.000 1.000 0.871 1.000
development
(Y1)
HOTS literacy Y2.1 0.871 0.681 1.000 0.863
(Y2) Y2.2 0.871
Critical thinking Y3.1 0.829 0.814 0.802 0.878
in HOTS questions Y3.2 0.833
making Y3.3 0.770
(Y3) Y3.4 0.774
Structural Model Test
The results of hypothesis testing to describe the empirical relationship between teacher professional knowledge (TPK), teacher professional development (TPD), HOTS literacy (HL), and critical thinking (CT) in HOTS questions making. From data analysis, firstly, it was found that there is a significant influence on teacher professional knowledge to teacher professional development with a coefficient of 0.607 and a value of p < 0.001. This result means that H1 is accepted and indicates the increase of teacher professional knowledge after participating in teacher professional development programme (TPD). Secondly, there is a significant influence on teacher professional knowledge (TPK) on HOTS literacy in HOTS questions making proved by its coefficient of 0.342 and p-value of < 0.001. This result shows that H2 is accepted and implies that the higher teacher professional knowledge is, the higher the literacy in HOTS questions making (HOTS literacy) becomes.
Table 4
Hypothesis testing results (direct impact)
Hypothesis Path coefficient P-value Information
H1: teacher professional knowledge ^ teacher professional development 0.607 < 0.001 Supported
H2: Teacher professional knowledge ^ HOTS literacy 0.342 < 0.001 Supported
H3: teacher professional knowledge ^ critical thinking in HOTS questions making 0.072 0.025 Supported
H4: HOTS literacy ^ critical thinking in HOTS questions making 0.195 < 0.001 Supported
H5: teacher professional development ^ critical thinking in HOTS questions making 0.026 0.240 Not Supported
Thirdly, it is also shown that teacher professional knowledge has significant effect on critical thinking in HOTS questions making proved by the coefficient value is higher than p-value (0.072 > 0.025) meaning that H3 is accepted. This result indicates that teacher professional knowledge has significant and positive effect on critical thinking in HOTS questions making. Fourthly, there is a significant and positive relationship between HOTS literacy and critical thinking in questions making. This statement is proved by the analysis in which shows that coefficient has higher value than p (0.195 > 0.001) meaning that H3 is accepted. These results imply that HOTS literacy in questions making positively affects critical thinking in HOTS questions making. However, there is an evidence that teacher professional development did not significantly influence critical thinking in HOTS questions making proved by coefficient of 0.026 and p-value of 0.240. This result means that H5 is rejected indicating that teacher professional development did not significantly affect teachers' critical thinking especially in HOTS question making.
The teacher professional development and HOTS literacy can serve as mediating variable for teacher professional knowledge and critical thinking in HOTS questions making (Table 5). This is proved by the coefficient value higher than its p-value (0.082 > 0.013). This result indicates that teacher professional development and HOTS literacy were able to mediate teacher professional knowledge to significantly influence critical thinking in HOTS question making meaning that H6 is accepted.
Table 5
Hypothesis testing results (indirect impact)
Hypothesis Mediating variable Path coefficient p-value Information
H6: teacher professional knowledge ^ critical thinking in HOTs questions making teacher professional development ^ HOTS Literacy in questions making 0.082 0.013 Supported
2. Discussion
The Influence of Teacher Professional Knowledge on Teacher Professional Development
The result did not support Hong, Lin & Jiar [18] who mention that teachers' experiences determine their ability. On the contrary, this study shows that teachers, despite their less teaching duration, are better in preparing HOTS-based teaching (especially making questions) than those who have longer teaching experiences and own teacher certification (as proof for teacher professional development programme). However, it cannot be denied that attending teacher training as part of teacher professional development programme is able to improve and develop teacher professional knowledge and experiences. Thus, this can serve as a proof to strengthen evidence and reference for the appropriate model in designing teacher professional development programme.
The Influence of Teacher Professional Knowledge on HOTS literacy
The result show that teacher professional knowledge significantly influenced HOTS literacy. This result proves that knowledge on teacher training is practice-oriented and emerges from the teacher professional experience [44]. This statement means that teachers' experience came from their teaching duration (the length of teaching) and the numbers of training attended to increase their HOTS literacy. Therefore, it is necessary to encourage training activities to continue to improve HOTS literacy owned by teachers. The results of this study also support the findings of Watson A., Miller T., Patty T. [48], who stated that the teachers in training were very interested in the initial experience and that the feedback was beneficial to them in developing and improving the learning process. Hence, it is possible for teachers to improve ones' experiences or skills by joining/attending training or other development programme. Moreover, they possibly obtained various knowledge to equip themselves in order to carry out better learning process or handle classroom activity through HOTS literacy.
Moreover, the result of this study proves that teacher professional knowledge can lead to knowledge conceptualisation as this conceptualisation requires the experiences coming from the teaching practices [21]. It turns out that based on teaching experience and following training to build professional knowledge and encourage to apply the HOTS literacy conceptualisation developed and owned in teaching practice that teachers are considered from a professional perspective from the lens of their knowledge and professional views, in-service teachers are different from pre-service ones [45], especially those who pay attention to their specific themes.
The Influence of Teacher Professional Knowledge on Critical Thinking
From the results of this study, it was found that teacher professional knowledge had a positive effect on critical thinking. This supports teachers, who have had longer teaching experience, to attend trainings as well as in developing critical thinking abilities, as stated by Tinto V., Angelo T. A., Gray P. J., about teachers who must have enough experience to compile evaluation questions that can foster critical thinking [17].
Teacher professional knowledge which is measured based on teaching experience and participating in training is learning through real experience in the field, so it is natural that the higher teacher professional knowledge can improve critical thinking abilities. Baten mentions the importance in remembering and understanding information obtained in order to improve one's critical thinking abilities as well as high order thinking skills [5].
The Effect of HOTS Literacy on Critical Thinking
Based on the results of data analysis, this study found that HOTS literacy possessed by teachers had a positive effect on critical thinking. It is proved by the value of p is less than its coefficient (0.001 < 0.195) (Table 4). In addition, the results of data processing show that HOTS literacy has the most dominant influence on the critical thinking ability of teachers compared to other variables studied (Tables 4, 5). This indicates that to maximise critical thinking skills, teachers need to apply the conceptualisation of HOTS literacy in their teaching activities. These results support the statement of Buckley [30], which states that performance-based assessment on the surface (actual performance appraisal) is an appropriate form or way to assess critical thinking disposition. From this opinion, several other studies have shown that through the application of performance-based assessment, teachers as the assessed individuals can focus on their performance as well as conceptualise HOT literacy in carrying out teaching and learning activities.
The Influence of Teacher Professional Development on Critical Thinking
However, there is evidence that teacher professional development has no effect on critical thinking. This indicates that a teacher, who has high teacher professional development, does not guarantee that they will have good critical thinking skills. The results of this study do not support the findings of Surjanti J. et al. [16], who state that non-certified teachers have heavier responsibilities with limited welfare, financial limitations and no social support. In this study, the opposite was found, meaning that even with limited welfare, financial limitations and limited social support for non-certified teachers or in this case being declared to have low teacher professional development, they still have the opportunity to have good critical thinking skills.
Even based on the findings of Wai Kit et al. [43], who state that higher order thinking skills (HOTS) can use information and communication technology (ICT) which are possible to be owned by certified teachers with high teacher professional development, it is not proven to be able to improve the ability of teachers to grow critical thinking abilities.
Moderating Effect of Teacher Professional Development and HOTS literacy
The results showed that teacher professional development and HOTS literacy significantly moderated the relationship between teacher professional knowledge and critical thinking. It is proved by the analysis in which p-value is less than its coefficient (0.013 < 0.082) (Table 5). This result shows that teacher professional development (TPD) and HOTS literacy (HL) play a significant and positive role in moderating the relationship between teacher professional knowledge (TPK) and critical thinking (CT). Although the teacher professional knowledge did not directly show a significant increase in improving critical thinking skills, but through teacher professional development (TPD) and HOTS literacy (HL) as a moderator, teacher professional knowledge is provably capable to improve critical thinking skills. Albeit no direct effect found in this study referring to teacher professional development (through teacher certification) on their critical thinking skills, this situation did not thwart the relationship model of TPD, TPK, HL, and CT. This is because even without certification (a proof for teacher professional development programme), teachers with professional knowledge and HOTS literacy remain capable in developing and improving their critical thinking skills.
This finding proves that teacher professional development (TPD) and HOTS literacy (HL) can be used to improve teacher's critical thinking skills. Teachers with critical thinking skills, especially in HOTS questions making, can certainly be
considered to achieve their professional function. This is similar to Masek & Yamin [19], who agree that teachers as facilitators can mediate learning by encouraging students' meta-cognitive thinking in which leads to their ability to develop critical thinking. In addition, Tanujaya, Mumu and Margono state that the teacher's goal is to print or educate students to have high and critical thinking skills [3].
In this study, it was found that teacher professional development (TPD) and HOTS literacy (HL) play significant roles in which serve variables that mediate the improvement of teacher competence. In addition, the results of this study indicate the importance of institutional support in improving and developing teacher competencies. This is because institutional supports can be a pivotal factor in increasing teacher professional knowledge because they can provide various opportunities, which are capable to improve teachers' HOTS literacy and critical thinking; thus, causing them (teachers) to develop their professionalism (teacher professional development). This study found that the level of teacher professional knowledge (TPK) certainly affects their level of critical thinking. Supports for teacher professional development (TPD) and HOTS literacy (HL) further improve teachers' critical thinking skills. Nevertheless, this study succeeded in explaining the role of teacher professional development and HOTS literacy in providing a positive and significant impact on teacher professional knowledge on critical thinking as mentioned in previous studies.
Conclusion
This study proves that out of six hypotheses made, five are proved to be accepted, while one is rejected. In this study, the results show that teacher professional knowledge supports teacher professional development, HOTS literacy, and critical thinking. Yet, teacher professional development has not been able to support critical thinking. In addition, HOTS literacy was able to mediate the relationship between teacher professional knowledge and critical thinking. Accordingly, these findings indicate the importance of improving and developing teacher professionalism and HOTS literacy in order to foster critical thinking in teachers.
References
1. Ennis R. H. Critical thinking: A streamlined conception. US: Palgrave Macmillan; 2015. 31-47 p.
2. Tan S. Y., Halili S. H. Effective teaching of Higher-Order Thinking (HOT) in education. The Online Journal of Distance Education and e-Learning. 2015; 3 (2): 41-47.
3. Tanujaya B., Mumu J., Margono G. The relationship between Higher Order Thinking skills and academic performance of student in mathematics instruction. International Education Studies. 2017; 10 (11): 78-78. DOI: 10.5539/ies.v10n11p78
4. Goddu G. C. Critical thinking by Alec Fisher. Cambridge: Cambridge University Press; 2001. 1-249 p.
5. Baten C. E. Your classroom. Journal of Education. 1918; 88: 495-496. DOI: 10.1177/002205741808801819
6. Fisher R. Teaching thinking and creativity Developing creative minds and creative futures Thinking. Early Child Development and Care. 1998; 141: 1-15.
7. Possin K. Why teach critical thinking: Commentary on "Why Not Teach Critical Thinking" by B. Hamby. In: OSSA Conference Archive [Internet]; 2016. University of Windsor; 2016 [cited 2021 July 09]. 9 p. Available from: https://www.researchgate.net/ publication/332304331_Why_Teach_Critical_Thinking_Commentary_on_Why_Not_Teach_ Critical_Thinkmg_by_B_Hamby
8. Widana I. W. Higher Order Thinking skills assessment (HOTs). Jisae: Journal of Indonesian Student Assesment and Evaluation. 2017; 3 (1): 32-44. DOI: 10.21009/jisae.031.04
9. Kumar Chakraborty U., Das S. Automatic free text answer evaluation using knowledge network. International Journal of Computer Applications. 2015; 117 (3): 5-8. DOI: 10.5120/20532-2876
10. Banta T. W. Teaching that assessment course! Assessment Update. 2015; 27 (3): 3-14. DOI: 10.1002/au.30020
11. Mursalin E. T. Development of authentic assesment tools in physics science learning. Journal of Physics: Conference Series. 2018; 1028 (1): 65-72. DOI: 10.1088/17426596/1028/1/012200
12. Bell E., Allen R., Brennan P. Assessment of higher order thinking skills: A discussion of the data from the 2001 random sampling exercise and a workshop for teachers. Australia: Queensland Board of Senior Secondary School Studies; 2001. 41 p.
13. Gobbo B. K., Shmulsky S., Bower M., Manual S. Strategies for teaching STEM subjects to college students with autism spectrum disorder. Journal of College Science Teaching. 2018; 047 (06): 12-18. DOI: 10.2505/4/jcst18_047_06_12
14. Paesani K. Teacher professional development and online instruction: Cultivating coherence and sustainability. Foreign Language Annals. 2020; 53 (2): 292-297. DOI: 10.1111/ flan.12468
15. Valiandes S., Neophytou L. Teachers' professional development for differentiated instruction in mixed-ability classrooms: Investigating the impact of a development program on teachers' professional learning and on students' achievement. Teacher Development. 2018; 22 (1): 123-138.
16. Surjanti J., Wijayati D. T., Hadi H. K., Muafi. The relationship pattern of social support, work stress, and work satisfaction; a context of non-certified primary education teacher in Indonesia. In: 31st International Business Information Management Association Conference: Innovation Management and Education Excellence through Vision 2020, IBIMA 2018 [Internet]; 2018 25-26 Apr; Milan, Italy. Milan, Italy: International Business Information Management Association, IBIMA; 2018 [cited 2021 July 09]; p. 489-495. Available from: https://www. scopus.com/record/display.uri?eid=2-s2.0-85060785995&origin=inward&txGid=484b3563e9 7a0ec95801cf7f9b5b77c9
17. Tinto V., Angelo T. A., Gray P. J. Assessment about the impact of learning communities on students? Assessment Update. 2000; 12 (2): 10-13.
18. Lin S. H. P., Jiar Y. K. Relationships between LINUS teachers' knowledge of basic language constructs, teaching experience and perceived teaching abilities. Universal Journal of Educational Research. 2018; 6 (9): 1962-1973. DOI: 10.13189/ujer.2018.060915
19. Masek A., Yamin S. The effect of problem based learning on critical thinking ability: A theoretical and empirical review alias. International Review of Social Sciences and Humanities. 2011; 2 (1): 215-221.
20. Allard A., Doecke B. Professional knowledge and standards-based reforms: Learning from the experiences of early career teachers. English Teaching: Practice and Critique. 2014; 13 (1): 39-54.
21. Hordern J. Teaching, teacher formation, and specialised professional practice. European Journal of Teacher Education. 2015; 38 (4): 431-444. DOI: 10.1080/02619768.2015.1056910
22. Szabo Z., Schwartz J. Learning methods for teacher education: the use of online discussions to improve critical thinking. Technology, Pedagogy and Education. 2011; 20 (1): 79-94. DOI: 10.1080/1475939X.2010.534866
23. Denisa K. Authentic assesment in context of ESP. In: International Scientific and Practical Conference "WORLD SCIENCE". Proceedings of the 2nd International Scientific and Practical Conference "Innovative Technologies in Science"; 2016 Feb 25-26; Dubai, UAE. № 3 (7), Vol. 4. Ajman, UAE: Scientific and Educational Consulting Group "WORLD Science"; 2016. p. 24-28.
24. Albitz R. S. The what and who of information literacy and critical thinking in higher education. Portal: Libraries and the Academy. 2007; 7 (1): 97-109.
25. Dwyer C. P., Hogan M. J., Stewart I. An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity. 2014; 12: 43-52.
26. Lai E. R. Critical thinking: A literature review research report [Internet]. 2011 [cited 2021 July 09]. Available from: http://www.pearsonassessments.com/research
27. Ennis R. H. Critical thinking dispositions: Their nature and assessability. Informal Logic. 1996; 18 (2): 165-182. DOI: 10.22329/il.v18i2.2378
28. Ennis R. H. The nature of critical thinking: An Outline of critical thinking dispositions and abilities. University of Illinois. 2011; 2 (4): 1-8.
29. Elder L., Paul R. Critical thinking: The nuts and bolts of education. Optometric Education. 2008; 33: 88-91.
30. Buckley P. Introduction to critical thinking. In: Doyle E., Buckley P., Carrol C. (eds.). lnnovative Business School Teaching. New York: Routledge; 2014. p. 105-108. DOI: 10.4324/9780203113684
31. Jacob C. Critical thinking in the chemistry classroom and beyond. Journal of Chemical Education. 2004; 81 (8): 1216-1223. DOI: 10.1021/ED081P1216
32. Ennis R. H. The nature of critical thinking: Outlines of general critical thinking dispositions and abilities [Internet]. University of Illinois. 2011 [cited 2021 July 09]; 2 (4): 1-8. Available from: http://criticalthinking.net/wp-content/uploads/2018/01/The-Nature-of-Critical-Thinking.pdf
33. Davies M. A Model of critical thinking in higher education. 2015; 30 151-186. DOI: 10.1007/978-3-319-12835-1_2
34. King F. J., Goodson L., Rohani F. Higher Order Thinking skills. Florida: Florida State University; 1998. 176 p.
35. Sormunen C., Chalupa M. Critical thinking skills research: Developing evaluation techniques. Journal of Education for Business. 2010; 69 (3): 172-177. DOI: 10.1080/08832323.1994.10117679
36. Pithers R. T., Soden R. Critical thinking in education: A review. Educational Research. 2010; 42 (3): 237-249. DOI: 10.1080/001318800440579
37. Serevina V., Sari Y. P., Maynastiti D. Developing high order thinking skills (HOTS) assessment instrument for fluid static at senior high school. Journal of Physics: Conference Series. 2019; 1185 (1). DOI: 10.1088/1742-6596/1185/1/012034
38. Chai C. S. Teacher professional development for Science, Technology, Engineering and Mathematics (STEM) education: A review from the perspectives of Technological Pedagogical Content (TPACK). Asia-Pacific Education Researcher. 2019; 28 (1): 5-13. DOI: 10.1007/S40299-018-0400-7
39. Soti O. F. Professional development in Food and Textiles Technology in Samoa: How can we improve learning in the classroom? Pacific-Asian Education Journal. 2019; 31: 89-100.
40. Bozku§ K., Bayrak C. The application of the dynamic teacher professional development through experimental action research. International Electronic Journal of Elementary Education. 2019; 11 (4): 335-352.
41. Romero G., Vasilopoulos G. From rural China to Canada: Communities of practice to support a teacher professional development study program abroad. The Electronic Journal for English as a Second Language [Internet]. 2020 [cited 2021 July 09]; 23 (4). Available from: https://files.eric.ed.gov/fulltext/EJ1242662.pdf
42. Gupta A., Lee G. L. The effects of a site-based teacher professional development program on student learning. International Electronic Journal of Elementary Education. 2020; 12 (5): 417-428. DOI: 10.26822/iejee.2020562132
43. Wai Kit L., Debbita Ai Lin T., Ganapathy M. ESL Teachers' HOT pedagogical practices via ICT: A case study. Pertanika Journal of Social Sciences & Humanities. 2020; 28 (S2): 321-335.
44. Clarke M., Killeavy M., Moloney A. The genesis of mentors' professional and personal knowledge about teaching: Perspectives from the Republic of Ireland. European Journal of Teacher Education. 2013; 36 (3): 364-375. DOI: 10.1080/02619768.2012.755513
45. Dreher A., Kuntze S. Teachers' professional knowledge and noticing: The case of multiple representations in the mathematics classroom. Educational Studies in Mathematics. 2015; 88 (1): 89-114. DOI: 10.1007/s10649-014-9577-8
46. Krauskopf K., Foulger T. S., Williams M. K. Prompting teachers' reflection of their professional knowledge. A proof-of-concept study of the Graphic Assessment of TPACK Instrument Karsten. Teacher Development. 2018; 22 (2): 153-174.
47. Tsang K. K. The purpose of teaching and teaching experience: A preliminary analysis of a qualitative research in progress. Alberta Journal of Educational Research. 2015; 61 (1): 103-105.
48. Watson A., Miller T., Patty T. Peer collaboration in an early field teaching experience: A replicable procedure for pre-service teacher trainers. Education. 2011; 131 (4): 798-798.
49. Zanin-Yost A., Freie C. Voices and choices: Critical information/media literacy and behrman's practices. Reference Librarian. 2020; 61 (2): 133-153. DOI: 10.1080/02763877.2020.1755765
50. Potzsch H. Critical digital literacy: Technology in education beyond issues of user competence and labour-market qualifications. TripleC. 2019; 17 (2): 221-240. DOI: 10.31269/ triplec.v17i2.1093
51. Cooke N. A. Critical library instruction as a pedagogical tool. Communications in Information Literacy. 2020; 14 (1): 86-96. DOI: 10.15760/comminfolit.2020.14.1.7
52. Goodsett M. Best practices for teaching and assessing critical thinking in information literacy online learning objects. The Journal of Academic Librarianship. 2020; 46 (5). DOI: /10.1016/j.acalib.2020.102163
53. Jampel I. N., Lasmawan I. W., Ardana I. M., Ariawan I. P. W., Sugiarta I. M., Divayana D. G. H. Evaluation of learning programs and computer certification at course institute in bali using cse-ucla based on saw simulation model. Journal of Theoretical and Applied Information Technology. 2017; 95 (24): 6934-6949.
54. Solimun, Fernandes A. A. R., Nuijannah. Metode Statistika Multivariat Pemodelan Persamaan Struktural (SEM) Pendekatan WarpPLS. Malang: Universitas Brawijaya Press; 2017. 190 p. (In Indonesian)
55. Hair Jr. F. J., Sarstedt M., Hopkins L., G. Kuppelwieser V. Partial least squares structural equation modeling (PLS-SEM). European Business Review. 2014; 26 (2): 106-121. DOI: 10.1108/EBR-10-2013-0128
Information about the authors:
Jun Surjanti - Professor, Economics Education Study Programme, Universitas Negeri Surabaya; ORCID 0000-0003-0305-6985; Surabaya, Indonesia. E-mail: junsurjanti@unesa. ac.id
Albrian Fiky Prakoso - Assistant Professor, Economics Education Study Programme, Universitas Negeri Surabaya; ORCID 0000-0002-7594-0924; Surabaya, Indonesia. E-mail: al-brianprakoso@unesa.ac.id
Riza Yonisa Kurniawan - Assistant Professor, Economics Education Study Programme, Universitas Negeri Surabaya; ORCID 0000-0003-2419-210X; Surabaya, Indonesia. E-mail: rizakurniawan@unesa.ac.id
Norida Canda Sakti - Associate Professor, Economics Education Study Programme, Universitas Negeri Surabaya; ORCID 0000-0002-5805-2466; Surabaya, Indonesia. E-mail: noridacanda@unesa.ac.id
Eka Indah Nurlaili - Assistant Professor, Economics Education Study Programme, Universitas Negeri Surabaya; ORCID 0000-0002-3932-5895; Surabaya, Indonesia. E-mail: eka-nurlaili@unesa.ac.id
Conflict of interest statement. The authors declare that there is no conflict of interest.
Received 22.10.2021; revised 27.01.2022; accepted for publication 09.02.2022.
The authors have read and approved the final manuscript.
Информация об авторах:
Сурджанти Джун - профессор учебной программы экономического образования Университета Негери Сурабая; ORCID 0000-0003-0305-6985; Сурабая, Индонезия. E-mail: junsurjanti@unesa.ac.id
Пракосо Альбриан Фики - старший преподаватель учебной программы экономического образования Университета Негери Сурабая; ORCID 0000-0002-7594-0924; Сурабая, Индонезия. E-mail: albrianprakoso@unesa.ac.id
Курниаван Риза Иониса - старший преподаватель учебной программы экономического образования Университета Негери Сурабая; ORCID 0000-0003-2419-210X; Сурабая, Индонезия. E-mail: rizakurniawan@unesa.ac.id
Сакти Норида Канда - доцент учебной программы экономического образования Университета Негери Сурабая; ORCID 0000-0002-5805-2466; Сурабая, Индонезия. E-mail: noridacandai@unesa.ac.id
Нурлайли Эка Инда - старший преподаватель программы экономического образования Университета Негери Сурабая; ORCID 0000-0002-3932-5895; Сурабая, Индонезия. E-mail: ekanurlaili@unesa.ac.id
Информация о конфликте интересов. Авторы заявляют об отсутствии конфликта интересов.
Статья поступила в редакцию 22.10.2021; поступила после рецензирования 27.01.2022; принята к публикации 09.02.2022.
Авторы прочитали и одобрили окончательный вариант рукописи.
