https://doi.org/10.17323/jle.2022.12537
Exploring the Impact of Process-Genre Approach on Learners' Academic Writing and Higher Order Thinking Skills
Dian Eka Chandra Wardhana ®
University of Bengkulu
ABSTRACT
Background. The study of using process-genre approach that was used to increase the writing competence had been conducted by many researchers. However, the contirbution of this learning approach for HOTs aspects has not been done by many researchers, especially on learning writing Indonesian
Purpose. Many studies only focused on one research approach, consequently, the finding was not quite comprehensive. Besides, to examine the effect of process-genre approach on academic writing competence and HOTs, this research also aimed to explore attitude and students' expectation on their learning experience by using this approach. Methods. The research used a sequential, mixed-method explanatory approach. Two grades were randomly chosen to enroll in the experimental class and a monitoring class of up to 52 students. The individual is a member of the Indonesian language education department who is currently taking the writing 1 course at the Bengkulu University, Indonesia. There are two instruments used in this analysis, the writing test & HOTs test for Quantitative statistics and semi-structured qualitative data interviews. The data from the writing test and the HOTs were quantitatively analysed for the paired sample t-test, the stand-alone sample t-test and the MANOVA, while the interview data were analysed using thematic analysis techniques. Results and Implications. The results have found that process-genre approach had a substantial positive effect on scholarly writing and student HOTs. Besides, thematic research also reveals that there are favorable views and expectations of students regarding the influence of the process-genre approach towards academic writing and student HOTs. This finding is iexpected to enrich knowledge about how students could enhance their writing ability and HOTs by using process-genre approach.
Citation: Wardhana, D. E. C. (2022). Exploring the impact of process-genre approach on learners' academic writing and higher order thinking skills. Journal of Language and Education, 8(2), 140-153. https://doi. org/10.17323/jle.2022.12537
Correspondence:
Dian Eka Chandra Wardhana, dec.wardhana@unib.ac.id
Received: June 08, 2021 Accepted: June 23, 2022 Published: June 30, 2022
KEYWORDS:
academic writing, higher-order thinking skills, mixed methods, process-genre approach
@ ®
INTRODUCTION
The Higher Order Thinking Skills (HOTs) are basic skills that students must possess in addition to their academic achievement (Marshall & Horton, 2011; Tajudin & Chinnappan, 2016). In other words, these skills can assist students in constructing knowledge and various information. HOTs are also considered an important aspect of education and teaching because these skills The ability, speed, and efficacy of student learning can be impaired (Alrawili et al., 2020). The HOTs in the continuum of education and learning are therefore required in re-
sponse to the demands of evolution. of education (Ramadhan et al., 2019) and produces qualified human resources that following the demand of the industrial world. (Heong et al., 2012; Misrom et al., 2020).
One way to develop HOTs for students is through learning academic writing. This is based on the fact that by using thinking skills a person can generate ideas critically and analytically in writing activities (Singh et al., 2018). Besides, with writing activities, students are invited to be creative which can encourage them to a higher taxonomic level of thinking ap-
plying, analyzing, synthesizing, and evaluating (Singh et al., 2020). Besides, with writing activities, students able to express their ideas and then organize the information through the writing they produce (Helaluddin et al., 2020).
Various studies on HOTs through the infusion approach in learning to write have been carried out by many researchers. Studies on student perspectives in learning writing to improve HOTs were carried out by Ganapathy & Kaur (2014). The results of this study stated that students felt involved in active learning, experienced independence in learning, developing writing skills, and personal skills. The study of the instructor's perspective states that they have used various strategies that can help learners increase HOTs through real-life situations and problem-solving skills (Singh et al., 2020). Research & development in learning to write by integrating HOTs was also carried out by several researchers to create valid, practical, and effective learning products in the form of syllabus, modules, and textbooks (Lustyantie et al., 2019; Singh et al., 2018). Besides, another study revealed that HOTs questions were able to improve the descriptive text writing skills of grade 8 junior high school students (Sianturi et al., 2020).
From these studies, there is one thing that has been missed by researchers/teachers an in-depth exploration effort of learning to write using the best approach in maximizing student HOTs the process-genre approach. This approach is the most advanced of several existing approaches. The genre method approach includes an understanding of vocabulary, meaning & aims, and language skills (Badger & White, 2000; Tesfie, 2017; Yanghee & Jiyoung, 2005) that can be used to improve student HOTs. The purpose of this thesis is therefore to fill in previous study gaps by using the following research querys: (1) How successful is the genre approach in improving student writing skills and HOTs relative to the traditional approach? (2) What attitudes and expectations do students have of the process genre approach to learning to write?
LITERATURE REVIEW
Higher-Order Thinking Skill
The word higher-order thinking skills (HOTs) has been commonly used in numerous paper papers, but it is not a straightforward term to describe (Schraw et al., 2011). It should be remembered, though, that while the definition is difficult to describe, it is reasonably straightforward to list some of its features (Resnick, 1987). HOTs are non-algorithmic, appear to be dynamic, and often include problems of complexity. Simply put, HOTs are characterized as the willingness of students to think at a higher level so that they can assess, examine and develop inventions to solve current problems (Ichsan et al., 2019)
Based on Bloom's taxonomy and the updated model (Krath-wohl, 2002; Leighton, 2011), the implementation, review,
assessment, and improvement of skills are core elements at the HOTs stage. Brookhart (2010) suggests that HOTs include logical thought skills, reflective logic, and metacogni-tive skills. And the imaginative thought. In line with Brokhart, Martinez & McGrath (2014) claimed that thinking skills are closely linked to HOTs and deeper learning. Moreover, the topic of HOTs usually gives preference to the insertion of triggers in contextual contexts (Abdurahman et al., 2019; Tiruneh et al., 2017).
Concerning the current knowledge age, HOTs can be defined as the highest level in the cognitive process hierarchy. With HOTs, students will face problems dominated by multiple types of information and with comparatively small processing time (Nourdad et al., 2018)In other words, HOTs concentrate on improving students' knowledge to be able to interpret accurately, assess current knowledge and construct something different (Chinedu & Kamin, 2015). HOTs will appear when students receive new information, store it in memory, relate existing awareness and product details to accomplish priorities or overcome difficult problems (Nourdad et al., 2018).
Academic Writing
Based on its name, academic writing is of course related to writing activities carried out in an academic environment. There are several criteria used in classifying types of writing into groups of academic writing. Zainurrahman (2011) describes several criteria for academic writing: (1) academic writing is written by components of the academic community, (2) scientifically, and (3) conventional in nature or restrained by various rules. In addition to these four criteria, there are other criteria used in categorizing academic writing containing definition, description, classification, causality, comparison, contrast, and argument (Murray & Hughes, 2008).
Oshima & Hogue (2006) describes that academic writing is the type of writing required at the college level which is different from personal writing, literature, journalism, and business. These differences are identified in the specific audience, the tone or tones, and the purpose of the writing. Several types of academic writing are differentiated into several genres, narrative, procedure, hortatory, expository, observation, and implantation (Mahsun, 2014). Indrawati & Subadiyono (2018) added that academic writing can be divided into several forms, such as books & book reports, translations, essays, research articles, conference papers, academic journal articles, theses & dissertations, and abstracts.
Process-Genre Approach
In general, there are three approaches used to learn how to write a product based approach, a process-based approach, and a genre-based approach. All three approaches are judged by experts to have their respective drawbacks, so
it is important to merge the three approaches. The term for combining the three approaches is the process-genre approach. This approach is a writing activity that adopts a process model and genre theory which is intended as a solution to the various limitations of the three approaches (Agibuay, 2017; Babalola & Litinin, 2012).
Several stages must be passed in the process of learning to write using the process- genre approach. Jarunthawatchai (2010) mentions several stages in this approach: (1) Construction of the context, (2) examination of the text of the model, (3) collective construction of the text, (4) individual construction of the text, and (5) contemplation on prose. Another stage model in the process-genre approach is also proposed by Zhao (2017) ) by simplifying the stages into just four parts analysis of model texts, independent writing, revising, and editing or proofreading. On the other hand, there is a stage model that includes six stages: (1) exploration of intent and functions, (2) language scaffolding, (3) modeling, (4) mutual/ joint text construction, (5) individual text construction, and (6) description, free conversation, Q&A, & assignment (Du, 2015). In this study, the design of the writing learning approach used was the Jarunthawatchai process genre-based approach which included five stages as mentioned above.
METHODS
Research Design
This thesis employs a sequential mix-method approach to data collection and interpretation to address a variety of research questions (Ebadi & Rahimi, 2017; Riazi & Candlin, 2014). The goal of this approach is to test the feasibility of a genre-based approach to developing student writing skills & HOTs and to examine their expectations and attitudes towards learning. The use of mixed-method methodology helps researchers to carry out in-depth experiments utilising two or three stages of study or data collection at the same sample (Bakla, 2020; Greene et al., 2005; Kamalodeen & Jameson-Charles, 2016; Kazazoglu, 2020). Moreover, mixing approaches are useful for exploring inconsistencies and increasing efficiency by cultivating a holistic viewpoint (Creswell, 1994; Creswell & Plano-Clark, 2007; Luo & Dappen, 2005). In other words, this study uses qualitative findings to help explain, elaborate, and clarify quantitative findings (Creswell et al., 2003).
The use of mixed-method methodology helps researchers to carry out in-depth experiments using two or three steps to evaluate or extract data from the same sample (Bakla, 2020; Greene et al., 2005; Kamalodeen & Jameson-Charles, 2016; Kazazoglu, 2020).
Participants
Two classes from the first semester of the Indonesian Language Education Program at the University of Bengkulu, Indonesia, were chosen as participants in this research. One
class was randomly chosen to act as an experimental group and as a monitoring group. Both of the participants are students aged 19 n.d. 52 students are taking composition lessons (26 experimental group & 26 control group).
Data Collection
The dataset was grouped into two aspects of the collection of quantitative and qualitative data. Quantitative data collection was carried out using active evaluations in the form of essay writing tests (pretest & post-test) and HOTs tests in the form of multiple open-ended questions. In addition, qualitative data collection used approaches in the form of semi-structured interviews. More in-depth, the instruments in data collection were described in the following section.
Instruments
Performance Test
The type of test used in data collection to see students' academic writing skills is a writing test using Indonesian. This writing test is carried out at the beginning before the learning treatment (pretest) and also afterward (post-testing) granted to experimental and control groups. Student writing results. These are then assessed using the academic writing assessment rubric using assessment rubric developed by Oshima & Hogue (2006). Some of the aspects assessed are: (a) format aspect/maximum score of 5, (b) mechanics/maximum score of 5, (c) content/maximum score of 20, (d) organization/maximum score of 45, and (e) sentence structure & grammar/maximum score 25.
To reduce subjectivity and assessment bias, the researchers measured the consistency of the rubric score, known as inter-rater reliability. That is, it is not only researchers who are in charge of assessing student writing but also involving one of the experienced Indonesian language course teachers (Landis & Koch, 1977; Panou, 2013). Measurement of inter-rater reliability is done by selecting 10 writings randomly from both pretest and post-test (Alwaleedi, 2018). The measurement results show that the acceptable consistency of the two assessors is 0.90 which is obtained using the Pearson product-moment correlation coefficient with the SPSS version 23 application.
Higher-Order Thinking Skill Test
Another instrument used in this study is the HOTs performance test. This test measures three levels of student HOTs which include the ability to analyze, evaluate, and create open-ended questions (Salem, 2018). This test consists of three main parts: (1) the first part measures the level of student performance in the analysis task, (2) the second part measures the level of students' ability to evaluate and assess evidence and arguments in their writing, and (3) the third part measures the level of the ability of students to create something new or unusual in supporting their writ-
ing. The overall score on the HOTs performance test is 75 points with 25 points each.
Semi-Structured Interview
Subsequent data collection was carried out using semi-structured interviewing methods to examine student expectations and perspectives towards the process-genre approach to develop their academic writing skills and HOTs. Half-structured interviews were chosen by researchers because they have the potential to explore deeper and allow researchers to add several questions as a process of clarification and elaboration (Guthrie, 2019). Besides, the questions in the interview were obtained from the interview instruments used by Ebadi & Rahimi(2017) dan Pham & Usaha (2016). Twenty six students were willing to be interviewed were asked by the researcher to fill out the willingness sheet. Interviews were conducted directly for approximately 30-45 minutes for each student. All conversations in the interview were recorded using a digital recorder by asking for participant consent. The language used for the interview was Indonesian to make easy the process of getting the information from students. Interview was conducted in a closed room to avoid distractions during the interview process (Lee et al., 2019).
Data Analysis Quantitative Analysis
The data obtained from the pre-test and post-test were then quantitatively evaluated using the SPSS 23 application. In this analysis, the researcher estimated the mean value, the standard deviation, the t-test, and the one-way MANO-VA Multivariate Analysis. Paired-t samples were run to see whether there was a substantial improvement in HOTs and student academic writing using the process-generic methodology in the experimental community and traditional approaches in the control group. Besides, the examiner also conducted a one-way MANOVA test to monitor the effect of covariates on the post-test of both classes.
Qualitative Analysis
The data collected from the interviews were analysed using a thematic approach. This study was used to identify the most influential trends in student attitudes and activities regarding the impact of the genre approach on the production of academic writing skills and HOTs. Some experts suggest that thematic research often helps to analyse the viewpoints of different students, highlight parallels and differences, and produce surprising insights (Braun & Clarke, 2006; King, 2004; Nowell et al., 2017). Thus, the interviews were transcribed and coded based on open thematic coding to find the most crucial key variables on student impressions (Liu & Sadler, 2003).
Besides, in determining the accuracy of student responses to interview questions (credibility), researchers used the mem-
ber checking technique (Morse et al., 2002). This technique is an opportunity for participants to examine and agree on certain aspects of the interpretation of the data they have provided. During the interview process, participants were given a transcript of the interview and then asked to verify it by editing, clarifying, elaborating, and even if possible, they deleted their words from the transcript (Carlson, 2010).
Procedure
As previously reviewed, this report incorporates Quantitative and contextual approaches to data analysis and analysis. For this cause, in general procedures for this study can be illustrated in the following chart.
In the early stages, the researcher carried out a writing test (pretest) for all participants, both the experimental class and the governing class. This is the researcher/lecturer presented pictures/photos on written themes such as education, health, law, and other recent issues. In this pretest, the lecturer allocates 60 minutes for students to write essays with a minimum number of 500 words.
Furthermore, the intervention phase was carried out by the researcher by applying the genre process approach in learning to write to the experimental group. On the other hand, the control group was approached using a traditional approach that involved multiple steps to clarify the definition of the text, to present illustrations, to identify examples of essay texts and to write separately. On the other hand, referring to the opinion of Jarunatawatchai (2010) there are five stages in the genre process approach, (1) building context, (2) analyzing sample texts, (3) writing texts together, (4) writing essays independently, and (5) reflection.
After the intervention process in learning academic writing is complete, the researcher conducts a writing test which is commonly called a post-test. Similar to the pretest, this post-test was given to both groups with duration of 60 minutes with a minimum of 500 words. Finally, the interview phase was carried out by a special researcher on group participants who were superior to the HOTs level they achieved by seeing the increase in HOTs scores from pretest to posttest. This interview was conducted to help describe and explain known quantitative research findings in the pretest and post-test processes.
RESULTS
Quantitative Analysis Academic Writing
The first results to be quantitatively analysed were student pre-test and post-test scores in all schools. The researcher performed two paired t-test experiments to determine the impact of the process-genre approach on the development of writing skills. Table 1 below displays the overall disparity
Figure 1
Researchflow
Figure 2
Learning steps with the genre process approach
between the experimental community's pre-test and posttest scholarly writing and the control class. From the data seen in the table, it can be observed that the overall posttest score of the two groups is higher than the pre-test average.
Table 2 is also the outcome of the study using the t-test matched samples to determine the discrepancy between the pre-test and the post-test in both grades. The table indicates that there is a difference in the pre-test and post-test academic writing of the two groups. In other words, there
Table 1
Descriptive statistics, analyzing the number of differences in the average score of academic writing between the pretest & posttest for the two classes
Mean
N
Std. Deviation
Std. Error mean
Writing Ex-pre Writing Ex-post Writing Co-pre Writing Co-post
56.5000 73.5000 57.5769 64.6923
26 26 26 26
7.28972 9.68401 8.09530 8.26931
1.42963 1.89919 1.58762 1.62175
Table 2
Paired samples t-test, investigating the differences between the pretest and post-test, wrote over the academics of the two groups
Paired Differences
95% confidence interval of
Mean Std. devia" Std. Error the difference t df Sig.
tion mean --(2-tailed)
Lower Upper
Exp -17.00000 5.97997 1.17277 -19.41536 -14.58464 -14.496 25 .000
Con -7.11538 4.01325 .78706 -8.73637 -5.49440 -9.040 25 .000
was a wide difference between the pre-test and post-test academic writing of the two schools. This difference is the consequence of the use of a solution for the process-genre in the experimental class and of conventional methods in the control class.
Table 3 below indicates the amount of mean gaps between the two classes of pre-tested students in their academic writing results. From this table, it can be seen that the pre-
test scores of student academic writing in the control and experimental classes are nearly equal. In comparison, the variations between the scholarly writing pretests in the two classes are still related to Table 4 below. Table 4 indicates that there is no substantial difference in the original evaluation in the trial and control classes. In other words, the pretest outcomes of student academic writing in both classes were identical before being approached with a process-genre approach and traditional approaches.
Table 3
Descriptive figures, investigating the number of mean variations between the two classes.
Groups N Mean Std. deviation Std. error mean
Experimental (pre) 26 56.5000 7.28972 1.42963
Control (pre) 26 57.5769 8.09530 1.58762
Table 4
Independent samples t-test, analysing the discrepancies between the pre-test classes.
t-test for equality of means
Levene's test for equal- 95% confidence interval of
ity of variances t df Sig. Mean dif- Std. error differ- the difference
--(2-tailed) ference ence -
f Sig. Lower Upper
Writing (pre) .663 .419 .504 50 .616 1.07692 2.13644 -3.21424 5.36809
In comparison, the following table 5 identifies the number of differences in the post-test performance in student academic writing in experimental and control institutions. From this table it can be concluded that the average student writing score in the experimental category is greater than the average student writing score in the control group.
Table 6 displays the outcome of the discrepancy between the two groups' post-test evaluations, which turned out to be better than the management group score. The findings of the data analysis found that there were substantial variations between the two groups. This means that the genre-based approach to academic writing outcomes of
Table 5
Descriptive data, investigating the amount of variations in the average post-test between the two classes
Groups N Mean Std. deviation Std. error mean
Experimental (post) Control (post) 26 26 73.5000 64.6923 9.68401 8.26931 1.89919 1.62175
Table 6
Independent samples of t-test, discrepancies between post-tests of the two classes
t-test for equality of means
Levene's test for equali- 95% confidence interval of
ty of variances t df Sig. Mean Std. error the difference
(2-tailed) difference difference
F Sig. Lower Upper
Writing (post) .252 .618 -3.527 50 .001 -8.80769 2.49740 -13.82386 -3.79153
students outperforms academic writing results using traditional approaches. Thus, the use of the genre-based approach is more successful in improving student scholarly writing than the traditional approach in the control class.
Higher-Order Thinking Skills
Apart from studying the impact of the process-genre approach on students' academic writing, this study also explores the influence of this approach on their HOTs. For this second target, the researcher ran a paired-samples
t-test on the HOTs test outcome findings using open-ended questions in both groups. Findings of the evaluation can be found in Tables 7 and 8 below.
Table 7 displays the results of the pre-test and post-test analysis of the average discrepancy between student HOTs in both grades. The table indicates that the average student HOT score in the post-test is higher than the pre-test score, both in the experimental class and in the control class. Table 8 reveals the difference between pre-test and post-test student HOTs in both schools, showing that there is a major contrast between
Table 7
Descriptive numbers, the amount of mean variations between the pre-test and the post-test of the two classes
Mean N Std. deviation Std. error mean
Eva-Pre-Ex 16.4615 26 2.55433 .50095
Eva-Post-Ex 21.9231 26 2.34810 .46051
Eva-Pre-Cont 17.2692 26 1.88108 .36891
Eva-Post-Cont 20.5769 26 2.87295 .56343
Ana-Pre-Ex 16.7308 26 1.21845 .23896
Ana-Post-Ex 22.1154 26 1.63284 .33741
Ana-Pre-Cont 16.0000 26 1.72047 .32023
Ana-Post-Cont 19.5000 26 3.02324 .59291
Cre-Pre-Ex 16.9615 26 2.47355 .48510
Cre-Post-Ex 21.5385 26 2.37033 .46486
Cre-Pre-Cont 17.4615 26 1.88108 .36891
Cre-Post-Cont 17.7692 26 1.98611 .38951
Table 8
Paired samples t-test, differences between the pretest and post-test of the two groups
Paired differences
Mean Std. deviation Std.Error mean 95% confidence interval of the difference Lower Upper t df Sig. (2-tailed)
Eva (Ex) -4.65385 3.28563 .64437 -5.98094 -3.32675 -7.222 25 .000
Ana (Ex) -5.38462 1.57675 .30923 -6.02148 -4.74775 -17.413 25 .000
Cre (Ex) -4.57692 1.17211 .22987 -5.05035 -4.10350 -19.911 25 .000
Eva (Con) -4.11538 3.20408 .62837 -5.40954 -2.82123 -6.549 25 .000
Ana(Con) -3.50000 3.04959 .59807 -4.73176 -2.26824 -5.852 25 .000
Cre (Con) -.30769 2.70953 .53138 -1.40209 .78671 -.579 25 .568
the pre-test and the post-test. This indicates that both tables suggest that the solution to the genre mechanism and the traditional method will substantially develop student HOTs.
In comparison, Table 9 above is the research outcome of the one-way MANOVA test, which seeks to assess the disparity in
Table 9
Tests of Between-subjects Effects
student post-test scores on three dimensions of HOTs in both classes (experiment & control). The inter-subject impact test in the table above indicates that the relationship between the genre processes approaching HOTs gave a score of 7,334 with a significance of 0,001. This shows that the unvariate approach to the process-genre affects student HOTs in all three aspects.
Source Dependent Type III Sum df Mean F Sig. Noncent. Observed
Variable of Squares Square Parameter Powerb
Corrected Model Post_control 104.333a 2
Post_ex 4.487c 2
Intercept Post_control 29000.205 1
Post_ex 37269.55 1
HOTs Post_control 104.333 2
Post_ex 4.487 2
Error Post_control 533.462 75
Post_ex 344.962 75
Total Post_control 29638.000 78
Post_ex 37619.000 78
Correct Total Post_control 637.795 77
Post_ex 349.449 77
52.167 7.334 .001 14.668 .930
2.244 .488 .616 .976 .127
29000.205 4.077E3 .000 4077.174 1.000
37269.551 8.103E3 .000 8102.980 1.000
52.167 7.334 .001 14.668 .930
2.244 .488 .616 .976 .127
7.113 4.599
Table 10.
Categories of students' perceptions and behavior towards the influence of the genre process approach on their HOTs
No_Categories_Themes
1. Benefits of the genre process approach
2. Excess steps in the process genre approach
3. New insights in writing
4. Positive attitude towards the genre process approach
Writing skills
HOTS upgrade
Text analysis phase
Collaborative writing phase
Reflective phase
Text structure
Current issues
Qualitative Analysis
al phases is considered to provide knowledge that can then be applied in writing activities. RP-01 states that:
Centered on the second research question, the goal of this review, is to explore student perceptions and attitudes towards the process-genre approach in order to improve their academic writing skills & HOTs. Data derived from interviews is qualitatively analysed using a thematic analysis. Generally speaking, the findings of the analysis can be found in the table below.
All students interviewed from the experimental group showed positive responses about learning to write with the process-genre approach. The students felt that the process-genre approach provided benefits not only in terms of improving academic writing skills but also for increasing their HOTs. The process-genre approach that applies sever-
For me, this approach provides tips in writing such as how to compose interesting writing.
In addition to writing skills, students also feel that their HOTs are honed and developed in learning activities using this process-genre approach. Efforts to find valid and credible sources of information, provide solutions to problems, and analyze a phenomenon are activities that hone student HOTs. This is supported by the RP-21 and RP-10 statements which state:
When I write about the problem of unemployment, the solutions that I will offer in my writing must be relevant. So I have to read various sources so that my writing is more interesting.
I think writing trains me to come up with different creative ideas. For example when I write about the learning method using video as the best method then I have to describe its advantages over other methods in my essay.
Regarding the phases in the process-genre approach, the students gave a positive response and found this approach much more effective. One activity phase that was appreciated was the writing phase together. According to them, the discussion and brainstorming is able to generate new ideas about the theme of writing. The RP-07 & RP-22 claim that:
I feel that many ideas come up when discussing with friends. In fact, when my friends share their opinions, other ideas automatically pop into my head.
When discussing with friends, I am able to provide their opinions and suggestions. I think this appro0ach is the best way of developing writing themes.
In addition to the collaborative writing phase, students also gave positive responses to other phases the text analysis and reflective phases. Students assessed the text analysis phase to help them recognize the structure of the essay text that they had not known so far. According to their view, the writing was just an activity of describing ideas from thought to written form without understanding that a text also has a systematic.
Like RP-14 which reported:
I thought a free essay text without any ties was like writing a short story or a novel. Now, I just found out that an essay text has a standard structure such as the opening paragraph, the body of the essay, and the closing paragraph.
Students also appreciated the last phase in the process-genre approach which was considered to have provided an opportunity for them to perfect the text they produced. The reflective phase is the final phase in writing by discussing the results of the writing with the lecturer. In this phase, students get corrections and suggestions in the form of using word choices (diction), sub-themes that must be developed or which must be ignored, punctuation, and so on. For example, the RP-12 says:
I really like discussing with the lecturery. Ifeel that my writing is more perfect after receiving feedback and suggestions from my lecturer.
Besides, students considered that the process-genre approach was also able to help them master various current issues. When they choose an issue that they raise as the theme of their writing, of course, they need more accurate information about that issue. For that reason, students try to find various references on the issue, both searching through the internet and reading books, newspapers, magazines, and others. One of them is the RP-6 which claims that:
I feel like learning to write will make me accustomed to finding the right source of writing. Example when I write about the
practice of bullying then I have to look for literature on that theme.
Finally, based on the findings of the student interviews, it can be inferred that there are optimistic attitudes towards the introduction of the genre solution. Any students felt that their writing was stronger in terms of consistency for this approach. The standard of the writing meant was not only limited error in the linguistic element, but also in the choice of ideas and the organisation of the writing process. RP-26 reported the following:
I feel confident in my writing after receiving suggestions and corrections from friends and lecturers. I find it helpful to use the right words choice according to the context.
DISCUSSION
HOTs becomes one of the educational objectives besides academic achievement. The integration of HOTs into learning writing has a essential role for teachers or lecturers in higher education. The goal of this research is to examine the effects of the process-genre approach on the production of writing and higher-level thinking skills for students. Selain itu, penelitian kali ini juga bertujuan untuk mengeksplorasi bagaimana persepektif mahasiswa terhadap pembelajaran menulis tersebut.
The results of this study indicate that the process-genre approach has a positive effect on the development of HOTs and student writing skills. It can be indicated by improvement of HOTS and students' writing with comparing pretest and posttest scores. This is consistent with the conclusions of the report published by Ganapathy & Kaur (2014) which states that HOTs-based writing learning can facilitate students' writing skills and interests. For this reason, learning to write like this must be implanted explicitly in classroom learning.
In relation to writing competence, several researchers have conducted studies relating to the use of a process-genre approach to writing instruction. Several researches suggest that this approach has a substantial influence on the learning success of students in order to improve the standard of the writing they produce (Huang & Zhang, 2019; Janenop-pakarn & Thongrin, 2020). On the other hand, research findings also reinforce that this combination approach can improve writing skills, both writing essays, reports, and other academic writing (Assaggaf, 2016; Alabere & Shapii, 2019; Maolida & Salsabila, 2019; Sarala et al., 2015).
Based on the findings of the quantitative and qualitative analysis, the HOTs required for writing activities. This is focused on the idea that students would go beyond their comprehension of certain content to apply their critical reasoning as well (Singh et al., 2020). The learning of writing process also supports students' desire to think critically and creatively differently (Lustyantie et al., 2019). This means
that students can provide various possible responses or views on a problem or the same theme. Based on facts, academic writing has strong relation with HOTs' ability.
In general, this finding proved that the process-genre approach gave positive impact on improvement of students' writing ability and HOTs. The process-genre approach as a hybrid approach is considered capable by experts in developing students' writing skills. Learning to write is the most difficult stage of language learning which requires effective methods, one of which is by combining several approaches (Bin-Hady et al., 2020; Kondal, 2020; Rusinovci, 2022). That is, the process-genre approach is a synthesis of two approaches that help students master writing skills from various aspects, including aspects of organization, vocabulary, grammar, and mechanics (Agesta & Cahyono, 2017; Hoa, 2022; Xu & Li, 2018).
One of the phases appreciated by the students in the process-genre approach was the writing phase together. This phase is considered capable of generating higher-order thinking skills by providing input and views to each other. This kind of social interaction contributes to the formation of a learning community, maintaining space to encourage HOTs through the creation of a shared knowledge process (Wang, 2020). In other words, teacher and peer interactions have a positive impact on activities that encourage student HOTs (Gailea et al., 2021).
The role of the teacher in the process-genre process approach must indeed be a priority. When students can write their ideas into writing, the task of the lecturer is how to provide input and directions to the writing without the tendency to change or divert to other ideas. In other words, the function of teacher as facilitator would have an effect on students' independence in learning (Ramon-Casas et al. 2018; Toprak & Yucel, 2020). The process-genre approach has advantages for the teaching role which includes various roles, such as assistant, evaluator, and examiner (Yuniar et al., 2019). Feedback during learning also helps provide a clear picture of the process, context, and goals of writing (Winarti et al., 2021).
For further research, I recommend to enrich the scope of language competence and not only focus on writing ability. Besides, the next research could emphasize on learning outcome and examine the effectiveness of critical thinking, creative thinking, self-efficacy and so on. The limitation of the research namely the participants of interview section were only limited to experiment group. It meant that this research just focused on exploring students' perception about learning the process-genre approach and ignore traditional approach. To achieve comprehensible qualitative finding, the researcher needed to involve participant from control group.
Based on background elaboration, this research was tried to fill the gap about incomprehensible learning writing. The Majority of previous studies only used one research design, thus the finding was limited. This research employed mixed method design to gain real research finding that can be viewed
from many perspectives. By using this design, the quantitative finding that is generalizing can be supported by using proofs from qualitative data. It means that the different data analysis could strengthen and support valid and reliable of the research finding.
CONCLUSION
HOTs is one of the competences in education. One of the ways to enhances students' GOTS ability is to integrate it into learning, especially in academic writing course. In gaining the target, the lecturer or teacher could use the process-genre approach that is special method in learning writing by combining process approach and process-genre approach. This study gives emperical evidence about the use of the process-genre approach which has positive impact on students' writing ability ehancement and HOTs. Moreover, another finding showed that students had positive attitude on the process-genre approach. It means that the qualitative finding supports the previous emperical evidence.
The use of the process-genre approach in learning writing was appropriate. This form of hybrid approach is a blend of the process and genre approaches aimed at mitigating the limitations of both approaches. The characteristic of this approach dould stimulate students' ability in thinking that produces good writing. Besides, the various stages of learning in this approach are considered capable of encouraging and stimulating student HOTs such as analyzing, evaluating, and creative skills.
Concerning writing activities, of course, students need a level of thinking that is more than just putting ideas into writing. There needs to be an understanding of the problems that they raise as writing topics. Students must also be able to analyse and evaluate the various evidence and arguments put forward in writing. In the end, the highest level thinking ability is the ability to create which must be expressed by students in providing solutions, the best alternative options, and creating something new in overcoming the problems discussed in their writing.
The further research about using the process-genre approach in improving students writing ability or HOTS needs to be done comprehensively. This will be used to generalize the finding and get more deep description about this topic. The next research must consider other aspects: (1) participants; (2) Target of language skills; (3) learning of other language skills. Furthermore, it is recommended that the research involves all participants in interview session to obtain accurate data, enlarges the target, such as creative thinking, critical thinking and self-efficacy, and considers language aspects, such as reading, listening and speaking.
DECLARATION OF COMPETING INTEREST
None declared ■
REFERENCES
Abdurahman, A., Setyaningsih, C. A., & Jalmo, T. (2019). Implementing multiple representation based worksheet to develop critical thinking skill. Journal of Turkish Science Education, 16(1), 138-155. https://doi.org/10.12973/tused.10271a
Agesta, S., & Cahyono, B. Y. (2017). Effect of process-genre approach on the Indonesial efl writing achievement across personality. International Journal of English Language and Linguistics Research, 5(3), 39-48.
Agibuay, R. M. (2017). Descriptive writing using the process-genre approach. Asia Pasific Journal of Contemporary Education and Communication Technology, 3(1), 272-282.
Alabere, R. A., & Shapii, A. (2019). The effects of process-genre approach on academic writing. Journal of English Educators Society, 4(2), 89-98. https://doi.org/10.21070/jees.v4i2.2598
Alrawili, K. S., Osman, K., & Almuntasheri, S. (2020). Effect of scaffolding strategies on higher-order thinking skills in science classroom. Journal of Baltic Science Education, 19(5), 718-729. https://doi.org/10.33225/jbse/20.19.718
Alwaleedi, M. A. (2018). Collaborative writing in arabic as a second language (asl) classrooms: A mixed-method study [Unpublished doctoral dissertation]. The University of Queensland. https://doi.org/10.1080/07908318.2018.1521422
Assagaff, H. T. (2016). A process genre approach to teaching report writing to Arab EFL computer science students. International Journal of English Linguistics, 6(6), 8-18. https://doi.org/10.5539/ijel.v6n6p8
Babalola, & Litinin, H. A. (2012). Effects of process-genre based approach on the written english performance of computer science students in a Nigerian polytechnic. Journal of Education and Practice, 3(6), 1-7. https://www.iiste.org/Journals/index. php/JEP/article/view/1630
Badger, R.,& White, G. (2000). A process genre ppproach to teaching writing. ELT Journal, 54(2), 153-160. https://doi.org/10.1093/ elt/54.2.153
Bakla, A. (2020). A mixed-methods study of feedback modes in efl writing. Language Learning and Technology, 24(1), 107-128. https://doi.org/10.125/44712
Bin-Hady, W. R. A., Nasser, A. N. A., & Al-Kadi, A. (2020). A pre-experimental study on a process-genre approach for teaching essay writing. Journal of Language and Education, 6(4), 44-54. https://doi.org/10.17323/jle.2020.10347
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101. https://doi. org/10.1191/1478088706qp063oa
Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. ASCD Member Book.
Carlson, J. A. (2010). Avoiding traps in member checking. The Qualitative Reportualitative Report, 15(5), 1102-1113. https://doi. org/10.46743/2160-3715/2010.1332
Chinedu, C. C., & Kamin, Y. (2015). Strategies for improving higher order thinking skills in teaching and learning of design and technology education. Journal of Technical Education and Training, 7(2), 35-43. https://publisher.uthm.edu.my/ojs/index. php/JTET/article/view/1081
Creswell, J. W. (1994). Research design: Qualitative and quantitative approaches. Sage Publications.
Creswell, J. W., Clark, P., Gutmann, V. L., & Hanson, W. (2003). Advanced mixed methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook on mixed methods in the behavioral and social sciences (pp. 209-240). Sage.
Creswell, J. W., & Plano-Clark, V. L. (2007). Designing and conducting mixed methods research. Sage Publications.
Du, B. (2015). Raising process-genre awareness: A proposal of esp writing lesson plan. Creative Education, 6(6), 631-639. http:// dx.doi.org/10.4236/ce.2015.66062
Ebadi, S., & Rahimi, M. (2017). Exploring the impact of online peer-editing using google docs on efl learners' academic writing skills: A mixed methods study. Computer Assisted Language Learning, 787-815. https://doi.org/10.1080/09588221.2017.136 3056
Gailea, N., Syafrizal, S., & Prabowo, E. (2021). Collaboration of online learning and high order thinking to develop writing and speaking skill. Psychology and Education, 58(2), 8680-8688. https://doi.org/10.17762/pae.v58i2.3518
Ganapathy, M., Kaur, M., Singh, M., Kaur, S., & Kit, L. W. A. I. (2017). Promoting higher order thinking skills via teaching practices. 3L: The Southeast Asian Journal of English Language Studies, 23(1), 75-85. http://doi.org/10.17576/3L-2017-2301-06
Gao, J. (2007). Teaching writing in Chinese universities: Find an eclectic approach. The Asian EFL Journal, 50(2).
Greene, J. C., Krayder, H., & Mayer, E. (2005). Combining qualitative and quantitative methods in social inquiry. In B. Somekh & C. Lewin (Eds.), Research methods in the social sciences (pp. 275-282). Sage Publishing.
Guthrie, K. H. (2019). "Nothing is ever easy": Parent perceptions of intensity in their gifted adolescent children. The Qualitative Report, 24(8), 2080-2101. https://doi.org/10.46743/2160-3715/2019.3598
Helaluddin, H., Ahmad, M. A., & Anshari, A. (2020). Need analysis of academic writing teaching model based on process-genre approach: What do they really need? Universal Journal of Educational Research, 8(10), 4728-4735. https://doi.org/10.13189/ ujer.2020.081043
Heong, Y. M., Yunos, J. M., Othman, W., Hassan, R., Kiong, T. T., & Mohamad, M. M. (2012). The needs analysis of learning higher order thinking skills for generating ideas. Procedia-Social and Behavioral Sciences, 59, 197-203. https://doi. org/10.1016/j.sbspro.2012.09.265
Hoa, T. M. (2022). Impacts of process-genre approaches on EFL sophomores' writing performance, writing self-efficacy, writing autonomy. Journal of Language and Education, 8(1), 181-195. https://doi.org/10.17323/jle.2022.12165
Huang, Y., & Zhang, L. J. (2019). Does a process-genre approach help improve students' argumentative writing in English as a foreign language? Findings from an intervention study. Reading & Writing Quarterly, 36(4), 339-364. https://doi.org/10.108 0/10573569.2019.1649223
Ichsan, I. Z., Sigit, D. V., Miarsyah, M., Ali, A., Arif, W. P., & Prayitno, T. A. (2019). Hots-aep: Higher order thinking skills from elementary to master students in environmental learning. European Journal of Educational Research, 8(4), 935-942. https:// doi.org/10.12973/eu-jer.8.4.935
Indrawati, S., & Subadiyono. (2018). Process genre based approach model in learning academic writing skills. International Seminar and Annual Meeting BKS-PTN Wilayah Barat (pp. 658-662). Universitas Sriwijaya.
Janenoppakarn, C., & Thongrin, S. (2020). Enhancing medical and health science students' writing development through a modified process-genre approach. The Asian EFL Journal, 24(2), 99-135.
Jarunthawatchai, W. (2010). A Proces-genre approach to teaching second language writing: Theoretical perspective and implementation in a thai university setting. University of Southampton. https://doi.org/10.1109/fie.2016.7757408
Kamalodeen, V. J., & Jameson-Charles, M. (2016). A mixed methods research approach to exploring teacher participation in an online social networking website. International Journal of Qualitative Methods, January-De, 1-14. https://doi. org/10.1177/1609406915624578
Kazazoglu, S. (2020). Is printed-text the best choice? A mixed-method case study on reading comprehension. Journal of Language and Linguistic Studies, 16(1), 458-473. https://doi.org/10.17263/JLLS.712879
King, N. (2004). Using templates in the thematic analysis of text. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 257-270). Sage.
Kondal, B. (2020). Impact of the process genre approach on the management students' business writing skills. ELS Journal on Interdisciplinary Studies in Humanities, 3(2), 297-307. https://doi.org/10.34050/els-jish.v3i2.9727
Krathwohl, D. R. (2002). A revision of bloom's taxonomy: An overview. Theory into Practice, 41(4), 212-218. https://doi. org/10.1207/s15430421tip4104_2
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorial data. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310
Lee, K., Ang, C., & Dipolog-ubanan, G. (2019). "My first year in the university": Students' expectations, perceptions, and experiences. Journal of Social Science Research, 14, 3134-3145. https://doi.org/10.24297/jssr.v14i0.8006
Leighton, J. P. (2011). A cognitive model for the assessments of higher order thinking skills. Information Age Publisher.
Liu, J., & Sadler, R. W. (2003). The effect and affect of peer review in electronic versus traditional modes on L2 writing. Journal of English for Academic Purposes, 2(3). 193-227. https://doi.org/10.1016/S1475-1585(03)00025-0
Luo, M., & Dappen, L. (2005). Mix-methods design for objective-based evaluation of a magnet school assistance project. Evaluation and Program Planning, 28(1), 109-118. https://doi.org/10.1016/j.evalprogplan.2003.12.001
Lustyantie, N., Arung, F., & Fitriany, D. A. (2019). Re-designing the higher order thinking skills (hots) approach for academic writing students. Research, Society and Development, 8(10), 1-19. https://doi.org/10.33448/rsd-v8i10.1420
Mahsun, M. (2014). Teks dalam pembelajaran bahasa Indonesia kurikulum 2013 [Text in learning Indonesian curriculum 2013]. Raja Grafindo.
Maolida, E. H., & Salsabila, V. A. (2019). Integrating process-genre approach with project IBUNKA to improve students' English writing. English Review: Journal of English Education, 8(1), 91-100. https://doi.org/10.25134/erjee.v8i1.2057
Marshall, J. C., & Horton, R. M. (2011). The relationship of teacher-facilitated, inquiry-based instruction to student higher-order thinking. School Science and Mathematics, 111(3), 93-101. https://doi.org/10.1111/j.1949-8594.2010.00066.x
Martinez, M., & McGrath, D. (2014). Technology alone won't transform teacher to facilitator. Phi Delta Kappan, 96(1), 41-45. https://doi.org/10.1177/0031721714547861
Morse, J. M., Barret, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative research. InternationalJournal of Qualitative Methods, 1(2), 1-9. https://doi.org/10.1177%2F160940690200100202
Murray, N., & Hughes, G. (2008). Writing up your universsity assigment and research project. McGraw Hills.
Nordin, S. M., & Mohammad, N. (2006). The best two approaches: Process/genre based approach to teaching writing. The English Teacher, 35(1), 75-85.
Nourdad, N., Masoudi, S., & Rahimali, P. (2018). The effect of higher order thinking skill instruction on efl reading ability. InternationalJournal of Applied Linguistics and English Literature, 7(3), 231-237. https://doi.org/10.7575/aiac.ijaleLv.7n.3p.231
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16, 1-13. https://doi.org/10.1177/1609406917733847
Oshima, A., & Hogue, A. (2006). Writing academic english. Pearson.
Panou, D. (2013). L2 writing assessment in the Greek school of foreign language. Journal of Language Teaching & Research, 4(4), 649-654. https://doi.org/10.4304/jltr.44.649-654
Pham, V. P. H. (2021). The effects of lecturer's model e-comments on graduate students' peer e-comments and writing revision. Computer Assisted Language Learning, 34(3), 324-357. https://doi.org/10.1080/09588221.2019.1609521
Pham, V. P. H., & Usaha, S. (2016). Blog based peer response for L2 writing revision. Computer Assisted Language Learning, 29(4), 724-748. https://doi.org/10.1080/09588221.2015.1026355
Ramadhan, S., Mardapi, D., Prasetyo, Z. K., & Utomo, H. B. (2019). The development of an instrument to measure the higher order thinking skill in physics. European Journal of Educational Research, 8(3), 743-751. https://doi.org/10.12973/eu-jer.83.743
Ramon-Casas, M., Nuno, N., Pons, F., & Cunillera, T. (2018). The different impact of structured peer-assessment task in relation to university undergraduates' initial writing skills. Assessment & Evaluation in Higher Education, 44(5), 653-663. https://doi. org/10.1080/02602938.2018.1525337
Razak, R. A., & Lee, C. M. (2012). Promoting higher order thinking in Chinese language literary text via online social network. Journal of Education and Practice, 3(16), 199-210. https://www.iiste.org/Journals/index.php/JEP/article/view/3716/3765
Resnick, L. (1987). Education and learning to think. National Academy Press.
Riazi, A. M., & Candlin, C. N. (2014). Mix-methods research in language teaching and learning: Opportunities, issues and challenges. Language Teaching, 47(2), 135-173. https://doi.org/10.1017/S0261444813000505
Rusinovci, X. (2022) An amalgamation of the process and genre approaches to teaching writing. International Journal of Social Sciences: Current and Future Research Trends, 13(1), 73-81.
Salem, A. A. M. S. (2018). Engaging esp university students in flipped classrooms for developing functional writing skills, hots, and eliminating writer's block. English Language Teaching, 11(12), 177-198. https://doi.org/10.5539/elt.v11n12p177
Sarala, T., Fauziah, B. I., & Rahim, B. S. A. (2015). Role of model essays in developing students writing skills in malaysian schools: A review of literature. Mediterranian Journal of Social Sciences, 6(2), 56-61. http://dx.doi.org/10.5901/mjss.2015.v6n2s1p56
Schraw, G. M. T., McCrudden, S., Lehman, L., & Hoffman, B. (2011). An overview of thinking skills in assessment of higher order thinking skills. Information Age Publisher.
Sianturi, L. P. D., Silalahi, D. E., & Purba, C. N. (2020). Improving students' writing ability based on higher order thinking skills (hots) questions at 8th grade in smp swasta kartika 1-4 pematangsiantar. Journal of English Teaching as a Foreign Language, 6(2), 1-17.
Singh, C. K.S., Gopal, R., Ong, E. T., Singh, T. S. M., Mostafa, N. A., & Singh, R. K. A. (2020). ESL teachers' strategies to foster higher-order thinking skills to teach writing. Malaysian Journal of Learning and Instruction, 17(2), 195-226. https://doi. org/10.32890/mjli2020.17.2.7
Singh, C. K. S., Singh, R. K. A., Singh, T. S. M., Mostafa, N. A., & Mohtar, T. M. T. (2018). Developing a higher order thinking skills module for weak ESL learners. English Language Teaching, 11(7), 86-100. https://doi.org/10.5539/elt.v11n7p86
Singh, R. K. A., Singh, C. K. S., Mostafa, N. A., & Singh, T. S. M. (2018). A review of research on the use of higher order thinking skills to teach writing. International Journal of English Linguistics, 8(1), 86-93. https://doi.org/10.5539/ijel.v8n1p86
Tajudin, N. M., & Chinnappan, M. (2016). The link between higher order thinking skills, representation, and concepts in enhancing timss tasks. International Journal of Instruction, 9(2), 199-214. https://doi.org/10.12973/iji.2016.9214a
Tesfie, A. (2017). Teachers' cognition on process genre approach and practice of teaching writing skills in efl context. English for Spesific Purposes World, 54(19), 1-17.
Tiruneh, D. T., Cock, M. De, & Elen, J. (2017). Designing learning environments for critical thinking: Examining effective instructional approaches. InternationalJournal of Science and Mathematics Education, 16, 1065-1089. https://doi.org/10.1007/ s10763-017-9829-z
Toprak, Z., Yucel, V. (2020). A peculiar practice of academic writing: Epidemic writing in the Turkish graduate education. Cogent Education, 7(1), 1-15. https://doi.org/10.10.8012331186x.2020.1774098
Wang, L. (2022). Effects of regulation on interaction pattern in web-based collaborative writing activity. Computer Assisted Language Learning, 35(1-2), 1-35. https://doi.org/10.1080/09588221.2019.1667831
Winarti, W., Cahyono, B. Y., Mukminatien, N., & Khoiri, N. E. (2021). Collaborative writing using process writing approach: The effect of group size and personality types. InternationalJournal of Instruction, 14(4), 391-410. https://doi.org/10.29333/ iji.2021.14423a
Xu, W., & Li, X. (2018). Teaching academic writing through a process-genre approach: A pedagogical exploration of an EAP program in China. The Electronic Journal for English as a Second Language, 22(2), 1-21. http://www.tesl-ej.org/wordpress/ issues/volume22/ej86/ej86a1/
Yan, G. (2005). A process genre model for taeching writing. English Teaching Forum, 43(3), 18-26.
Yanghee, K., & Jiyoung, K. (2005). Teaching Korean university writing class: Balancing the process and genre approach. Asian EFL Journal, 7(2), 1-15.
Zainurrahman, Z. (2011). Menulis: Dari teori hinggapraktik [Writing: From theory to practice]. Alfabeta.
Zhao, S. (2017). On the application of the process-genre approach to teaching english writing. 2017 3rd Annual International Conference on Modern Education and Social Science (pp. 495-499). DEStech Publication.
Yuniar, R. F., Widiati, U., & Astuti, U. P. (2019). The effect of using Wattpad on process-genre approach towards writing achievement in tertiary level. Jurnal Pendidikan: Teori, Penelitian, dan Pengembangan, 4(7), 897-905. http://dx.doi.org/10.17977/ jptpp.v4i7.12631
Zohar, A., & Schwartzer, N. (2005). Assessing teachers' pedagogical knowledge in the context of teaching higher-order thinking. InternationalJournal of Science Education, 27(13), 1595-1620. https://doi.org/10.1080/09500690500186592