CC BY-ND
17
https://doi.org/10.17323/jle.2023.12399
Effectiveness of cooperative grouping in developing reading skills of university level EFL learners
Jagoda Topalov ®
University of Novi Sad, Novi Sad, Serbia
ABSTRACT
Background. Collaborative Strategic Reading (CSR) is an instructional method in which students work together in a cooperative framework, jointly construct a model of text and come to its potential meaning through discussions.
Purpose. A quasi-experimental pretest-posttest non-equivalent control group research design was used to examine the effects of cooperative grouping within the framework of CSR, with the aim of determining whether cooperative grouping is effective in developing EFL students' reading skills.
Method. The response variables included the students' scores on questions testing Vocabulary, Factual Information, Prose Summary, Sentence Simplification, Reference Question and Insert text, whereas the explanatory variable was group membership (+/- cooperative), measured across three testing times (the beginning, middle and the end of the experimental intervention). Results. The results indicate that the students exposed to CSR within cooperative groups significantly developed those reading skills which focus on the comprehension of global information - prose summary, insert text and reference question. A possible explanation is that, in order to answer these questions, readers must approach the text in a holistic manner and focus on its main ideas, which seems to be facilitated by discussions in heterogeneous teams and negotiations of meaning resulting from those discussions.
Conclusion. The main pedagogical implication of the results concerns the need for introducing cooperative grouping as an alternative to a typical university-level foreign language classroom, allowing teachers to organize an effective, interactive context for reading academic texts in English.
KEYWORDS
EFL reading skills, EFL reading instruction, cooperative learning, Collaborative Strategic Reading, university students, quasi-experimental design
Citation: Topalov J. (2023). Effectiveness of cooperative grouping in developing reading skills of university level EFL learners. Journal of Language and Education, 9(1), 158-171. https://doi. org/10.17323/jle.2023.12399
Correspondence:
Jagoda Topalov, jagoda.topalov@ff.uns.ac.rs
Received: May 11, 2021 Accepted: September 01, 2022 Published: March 31, 2023
INTRODUCTION
In the academic context of tertiary education one of the most useful and most employed skills is reading, whereas reading in English, with its growing significance as a lingua franca, places a set of new, highly demanding challenges on EFL learners. The expectations placed on university students to be able to read academic texts in English are increasing, which is why curriculum designers and practitioners alike are tasked with putting forward most effective ways in
which EFL reading instruction can be offered. The urgency of adopting an improved approach to reading instruction is, in the Serbian educational context, strikingly apparent in the results of the Programme for International Student Assessment (PISA)3 for 2018, according to which Serbian students score below the OECD average on the reading proficiency measure, with only 3% of students able to "comprehend lengthy texts, deal with concepts that are abstract or counterintuitive, and establish distinctions between fact and opinion, based on
1 PISA (Programme for International Student Assessment). (2018). PISA 2018 results. https://
nces.ed.gov/surveys/pisa/pisa2018/index.asp#/
implicit cues pertaining to the content or source of the information".4 Having recognized the need to bring reading instruction up-to-date, the policy-makers from the Serbian Ministry of Education, Science and Technological Development in their Framework of the National Curriculum of the Republic of Serbia5 underline that cooperative learning is one of the methods which leads to higher achievement and better learning gains, in contrast with individual and competitive learning. In addition to improved learning gains, cooperative learning also answers to the challenges of the changing labour market in which "employers across the world give increasingly higher priority to behavioural skills than to technical competencies" (Cunningham et al., 2010). Teamwork, collaboration and sociability are ranked high in UNESCO's framework for transversal competencies6 and a timely integration of these inter-personal skills in education would ensure that the students are adequately prepared for the world of work and are equipped to live meaningful, sustainable, and responsible lives in a world that is rapidly becoming interconnected.
Despite the potential benefits of cooperative learning, the organization and dynamics of the teaching process and the implementation of different teaching approaches and strategies in the Serbian educational context, as evidenced by PISA scores, in most cases remain very static and rigid (Milic, 2016). As students are rarely exposed to cooperative framework and seldom taught how to take part in it, previous research shows that, even though they have moderately positive attitudes towards group work and team activities, they exhibit considerably low willingness to participate in it (Topalov et al., 2015; Topalov et al., 2017). In an educational context in which the individual method is promoted and practiced, this study aims at investigating the extent to which structured, heterogeneous cooperative grouping is an effective method in developing reading skills of young adult EFL learners. For this purpose, drawing on the social interdependence theory (Johnson & Johnson, 2005; 2015), a quasi-experimental pretest-posttest non-equivalent control group research design was implemented to examine whether there was a statistically significant change in the students' scores on the reading comprehension measure between the students who were exposed to reading instruction within cooperative groups and those who were taught reading in a traditional classroom setting.
Student Interaction in the Foreign Language Classroom
Recent decades saw a growing interest of researchers and educators in learning through interaction and cooperation,
in response to the weaknesses of the teacher-centred classroom. Although the traditional method allows the teachers to more easily manage all aspects of classroom organization, individual differences among students are marginalized, the teaching materials are aimed at the average student, while both gifted and struggling students are side-lined, and the responsibility for achieving lesson aims is largely assumed by the teacher (Estes et al., 2014).
Cooperative learning appeared as one of the responses to the observed need for a shift in the nature of authority and responsibility in the classroom. Social interdependence theory (Johnson & Johnson, 2005; 2015) provides a theoretical basis for the investigation of the effects of cooperative work and has, thus far, been validated by hundreds of research studies (Johnson and Johnson (2008) offer a highly informative overview of studies investigating cooperative learning procedures). As it accounts for how the outcomes of social interaction are affected both by individual actions of those who participate in interaction, as well as by the actions of other participants, it defines cooperative learning as a "pedagogical strategy in which small groups of students work together to achieve a common goal, with each individual responsible for their own learning as well as the success of their peers" (Davies et al., 2013, p. 564). Its aim is to contribute to the development of cognitive and social skills through selected techniques, wherein the learning process is rigorously structured and the tasks used are clearly defined and regulated; the students are responsible both to themselves and to the group; the group represents the core of all classroom interactions, with the teacher's main task being to provide support (Johnson et al., 2014).
Johnson and Johnson (2009) indicate the following four basic principles of cooperative learning: (a) positive interdependence, according to which the success of an individual is linked to the success of the rest of the group in such a way that an individual can only achieve the goal if the other members of the group reach that goal as well; it is promoted by sharing goals, means for achieving those goals and setting boundaries between group members to determine who is interdependent with whom; (b) individual and group accountability, with individual responsibility being reflected in the results of individual assessment and testing, and group accountability in group assessment; (c) promotive interaction, which takes place when group members support each other's efforts taken towards the accomplishment of the group's goals by providing mutual help and assistance, exchanging needed resources, effectively communicating, exerting mutual influence and trust, and constructively managing conflict; and (d) appropriate use of social skills, which
4 Schleicher, A. (2018). PISA 2018. Insights and interpretations. OECD. http://www.oecd.org/pisa/PISA%202018%20Insights%20and%20In-terpretations%20FINAL%20PDF.pdf
5 ONKRS (Okvir nacionalnog kurikuluma Republike Srbije). (2015). https://obrazovanje.org/rs/uploaded/dokumenta/Okvir-nacional-nog-kurikuluma_Osnove-ucenja-i-nastave.pdf
6 UNESCO. (2015). Transversal competencies in education: Policy and practice. UNESCO. http://unesdoc.unesco.org/imag-es/0023/002319/231907E.pdf
include personal and small-group skills needed for effective cooperation; in order to coordinate efforts for the purpose of accomplishing shared goals team members must get to know and trust each other, communicate clearly and accurately, support each other, and constructively resolve any conflicts (Johnson & Johnson, 2016).
In examining foreign language learning contexts, this framework has been shown to be effective in overall foreign language acquisition (Law, 2011; Wei & Tang, 2015), including English language fluency (Alrayah, 2018), productive English skills (Fen, 2011), oral proficiency (Namaziandost et al., 2020) and motivation (Ning & Hornby, 2014). Rather than simply motivating individuals to exert greater effort, positive interdependence as part of cooperative dynamics facilitates the development of new insights and the use of higher-level reasoning through promotive interaction (Johnson & Johnson, 2008).
Cooperative Framework of Reading Instruction
Turning to the pedagogy of reading, Collaborative Strategic Reading - CSR is a combination of cooperative learning and modified reciprocal teaching (Bremer et al., 2002) and is based on a socio-cognitive theory of reading that emphasizes the importance of social context in the cognitive development of reading skills (Klingner et al., 2012). In this approach to comprehension, the reader actively decodes the text, uses adequate prior knowledge, applies the cognitive resources at his or her disposal, and develops understanding of the text through structured social interaction. It is an approach to teaching comprehension strategies that places students, their cognition and ability to self-regulate at the centre of the reading process. CSR is an instructional method in which students work together in a cooperative framework, jointly construct a model of text, and come to its potential meaning through discussions (Vaughn et al., 2011). In this learning context, students achieve cognitive development that they would not otherwise be able to achieve individually through the help and support of their micro-community, i.e. the team within which they are reading. By working together with their peers on the construction of meaning, they internalize cognitive strategic knowledge through dialogue within small groups (Fan, 2015).
CSR is conducted over two phases. In the first phase, students are exposed to reading strategies, while in the second phase, cooperative teams are formed in which students continue to apply the strategic framework. The principles of modified reciprocal teaching are reflected in the first phase of the reading approach, which uses a combination of reading strategies. Within CSR, the teacher exposes students to groups of strategies that include: 1) initial review of the text, 2) monitoring comprehension, 3) summarizing parts of the text/getting the main idea, and 4) final evaluation. During the initial review of the text, the students should apply
two strategies: first, they make predictions about the text, based on the title, subtitle, pictures, etc., and then activate their previous knowledge of the topic. During reading, the students monitor comprehension and are aware of the moment when there is a breakdown in understanding. Within CSR, students are instructed to use several strategies, including identifying contextual information that would help them understand an unknown word or phrase, extending the context to a sentence that precedes or follows the sentence with the unknown word, and parsing the word and identifying roots, prefixes and suffixes. In applying the strategy of summarizing, the students focus on the most important ideas in a particular section of the text and ignore less important information. After reading, during the final evaluation, students are instructed to formulate questions and provide answers, as well as to summarize the entire text in order to test comprehension. Cooperative elements are evident in the second phase, in the structured roles that students receive when interacting, as well as in the structured materials designed to foster positive interdependence and individual and group accountability. When working with students in cooperative roles, the teacher supervises groups, facilitates in the comprehension of unfamiliar words, demonstrates how strategies and cooperative learning techniques are used, and provides assistance (Klingner et al., 2012).
To date, a number of empirical studies examined the effectiveness of CSR in reading instruction, focusing primarily on reading instruction in the first language, with participants of different ages, as well as with students with learning difficulties (Boardman et al., 2016; Kim et al., 2006; Klingner & Vaughn, 2000; Vaughn et al., 2011). Several studies have examined the impact of collaborative strategic framework on the development of foreign language. For instance, in a study with students at the A2 level of CEFR, CSR positively affected the students' reading comprehension scores, with significant positive effects on the identification of the main idea and supporting details (Fan, 2010). CSR also positively impacted students' ability to deal with vocabulary-related comprehension breakdowns (Karabuga & Kaya, 2013), their learning habits and attitudes (Mendieta et al., 2015) and their willingness to participate in cooperative reading tasks (Zoghi et al., 2010). In a study by Topalov and Radic-Bojanic (2016) CSR had positive effects on university students' affinity towards reading texts in the English language, on the degree in which they read English texts and on their belief that they are able to complete academic reading tasks. However, no study, to the best of our knowledge, has focused on the following two aspects investigated in this research: firstly, the cooperative component of CSR alone has not been investigated in such a way that the strategic input was administered to both conditions, while only the factor of cooperative work/individual work differed between the experimental and the control groups, as is the case in this research; secondly, this study examines the context of reading in a foreign language with young adults learning EFL at university, who are at a higher level of foreign language pro-
ficiency and who, therefore, have functional independence in language use and are able to read complex academic texts that require readers to activate higher cognitive levels of text processing, as well as procedural and conditional knowledge of reading strategies (studies conducted thus far in an EFL context have examined students at lower levels of proficiency, or did not include information on the participants' level of English). In practical terms, the rationale for this study is, thus, twofold. On the one hand, in teasing apart the relative contribution of the cooperative component to the success CSR has in developing EFL reading skills, as attested by previous research (Fan, 2010; Karabuga & Kaya, 2013; Zoghi et al., 2010), in addition to providing its empirical validation, this study will examine the potential benefits of introducing the cooperative component into an educational setting that is still largely oriented towards individual work. On the other hand, by using a sample of students at a higher level of EFL proficiency, it will investigate the effectiveness of CSR beyond the level of simple textual comprehension and will focus on its potential usefulness in achieving a holistic understanding of complex texts.
The goal of this study is, therefore, to determine whether the cooperative component significantly contributes to the development of EFL reading skills of university level students. In view of this goal, this study aimed at answering the following research question: Is there a statistically significant change in the students' scores on the reading comprehension measure over time (8 months) between the students who are exposed to reading instruction within cooperative groups and those who are taught reading in a traditional classroom setting?
METHOD
Design
A quasi-experimental pretest-posttest non-equivalent control group research design was applied in this study to examine the effects of cooperative grouping within CSR on the development of EFL reading skills operationalized as the score on the TOEFL reading comprehension test of university students.
The within-subjects factor in the study was Testing Time, with data collected at three different points: at baseline, after four months and after eight months. Between-sub-jects factor was Group, which included two levels: the Experimental Group followed the strategic framework of CSR within cooperative groups (+cooperative), while the Control Group followed the same strategic framework through individual work (-cooperative) (see Table 1). This intervention setup effectively eliminated the potential for contamination between experimental and control conditions, since both groups received the same strategic framework.
Participants
This study used single stage sampling. The sampling procedure consisted of non-random, convenience sampling, as intact classes were readily accessible (Gall et al., 2007). In terms of the validity of this sampling method, when each control classroom is compared to treatment classrooms within the same school, it eliminates school-based differ-
Table 1
The Design of the Experiment
Collection of de- Independent Collection of Independent Collection of de-
pendent variable dependent variable pendent
variables manipulation variables manipulation variables
Experimental Group
Baseline test
+ cooperative
Progress test
+ cooperative continued
Post-test
Control Group
Baseline test
■ cooperative
Progress test
- cooperative continued
Post-test
Input:
A combination of reading strategies as part of 1) initial review of the text, 2) monitoring comprehension, 3) summarizing parts of the text/getting the main idea, and 4) final evaluation, in a +/- cooperative setting
ences in those comparisons and provides an opportunity to replicate program impacts across multiple sites (Hitchcock et al., 2009). This further ensures that reading instruction techniques will likely not vary if both classrooms are from the same school, so that minimizing such differences in non-experimental factors will improve power and quality of the results and reduce the necessary sample size. A total of 50 first- and second-year students from the Faculty of Philosophy, University of Novi Sad participated in this study. They were all students from various humanities and social science departments (including the Departments of Psychology, Pedagogy, Philosophy, History, Media Studies, Serbian Literature, Comparative Literature and Serbian Philological Studies), who were taking a course in the English language as a requirement of their respective study program. On average, the participants were 20.1 years old, with the ages ranging between 18 and 22. They were all at B2 level of the Common European Framework of Reference - CEFR attested by the results of Quick Pen and Paper Test.7 In the context of the present study this means that students were expected to have intermediate knowledge of the language in order to be able to understand main ideas of complex texts on both concrete and abstract topics and to adjust their reading to the purpose and the nature of the text (Jones, 2002). The participants in this study attended two courses in English as a foreign language (English B2.1 and English B2.2) during two consecutive semesters, as part of a larger cohort of 81 students who were also placed at B2 level. This, in turn, means that roughly two thirds of all students at B2 level took part in this study.
Of the total number of participants, 24 were in the Experimental Group (11 male and 13 female students) and 26 in the Control Group (12 male and 14 female students). Since group equivalence could not be assumed, the gender structure of the sample was internally controlled for appropriateness with a chi-square test. The results indicate there is no statistically significant difference between the two groups (x2(1, 50)=.001, p>.01). Furthermore, in order to determine whether groups differed prior to the beginning of the experimental input, all participating students were pretested on reading level including its subcomponents (Vocabulary, Factual Information, Prose Summary, Sentence Simplification, Insert Test, Reference Question). No statistically significant differences in the reading level between the Experimental and the Control Groups were found on questions testing the students' knowledge of Vocabulary (mean difference=-0.11, t=-.427, p>.01), Factual Information (mean difference=0.13, t=.1.431, p>.01), Prose Summary (mean difference=-0.12, t=-1.089, p>.01), Sentence Simplification (x2(1, 50)=2.889, p>.01), Insert Test (x2(1, 50)=.774, p>.01), Reference Question (x2(1, 50)=1.035, p>.01), or on their overall reading comprehension score (t=.031, p>.01).
Classroom Intervention
Both groups had 90-minute classes two times a week for the duration of eleven weeks in the first semester and twelve weeks in the second, with the same teacher. Furthermore, both groups were exposed to the same set of reading strategies within the CSR framework, as was elaborated previously in the Introduction (a combination of reading strategies as part of a) initial review of the text, b) monitoring comprehension, c) summarizing parts of the text/getting the main idea, and d) final evaluation). The students in the Control Group applied the strategies with the guidance of the teacher. In this approach, the teacher taught the strategies by encouraging students to form predictions about the text, by asking questions with which the students' previous knowledge was activated, by asking them to come up with the gist of a paragraph and key words, and by giving the students the task of writing short summaries and formulating questions. On the other hand, the manner in which this strategic input was used and practiced with the Experimental Group was modified as classes progressed, from a general discussion about strategies, over teacher modelling, to students' application in reading, first with scaffolded assistance and then, finally, without it (Reynolds, 2017).
During the second stage, which began two weeks into the experiment, heterogeneous cooperative teams of four were formed in the Experimental Group. The grouping took into account the results of the baseline reading comprehension test, so that, whenever possible, groups consisted of both students with higher and lower levels of reading proficiency, and were mutually balanced. Each team member was given a specific role (leader, gist expert, unknown words expert, reporter, cf. Klingner et al., 2012) and these roles alternated in regular three-week intervals so that every student could experience different tasks and responsibilities associated with a specific role. The materials were highly structured, and included learning logs and role cards, all with the purpose of fostering positive interdependence and individual and group accountability.
At this point it is also necessary to acknowledge that the factor of the teacher may potentially be considered a confounding factor, one whose potential influences would be eliminated had the sample included more than one of intervention and control groups each. Although, due to practical reasons, it was impossible for the sample to include more groups in the same school during the same time period, as none were available, it nonetheless does not put at ease doubts arising from this experimental set-up. In order to mitigate this possible threat to the validity of the study, the data will be investigated using statistical tests that are designed to increase the power of the study (a Repeated
7 Oxford University Press/University of Cambridge/Association of Language Testers in Europe (2006). Quick Placement Test: Paper and Pen Test. Oxford University Press.
Measures ANOVA in place of two One-Way ANOVAs) and the discussion of the results will proceed cautiously.
Instruments
In order to determine the students' proficiency in the English language, the Quick Pen and Paper Placement Test was administered prior to the commencement of the experiment. The test consists of 60 questions with multiple-choice answers, arranged in order of increased proficiency. The test is able to identify the test-taker's level of proficiency relative to the six proficiency levels outlined by CEFR.
The students' reading skills were tested by means of parallel forms of TOEFL IBT Reading Comprehension Test8. The section of TOEFL that assesses EFL reading skills is designed to replicate the types of tasks that university students face when reading in an academic context (Jamieson et al., 1999). The following types of questions were included (Cohen & Upton, 2006):
(1) Vocabulary - measured the ability to understand the meaning of particular words and phrases in context (3 items, 1 point each);
(2) Factual Information - examined the ability to find the answer to a question about an important fact explicitly written in the text (3 items, 1 point each);
(3) Prose Summary - assessed the ability to understand the main ideas in the text, the relative importance of information and to distinguish main ideas from minor information or the ideas that are not in the text (1 item graded on a 3-point scale);
(4) Sentence Simplification - examined the ability to recognize main information in a longer, complex sentence consciously ignoring minor details and elaborations (1 item, 1 point);
(5) Insert Test - assessed the ability to find lexical, grammatical, and logical connections in the sequence of sentences by inserting a new sentence into the most appropriate position in the passage that was already read (1 item, 1 point);
(6) Reference Question - evaluated the ability to recognize the links between anaphoric words and their antecedents or postcedents in the text (1 item, 1 point).
The following TOEFL tests were used in chronological order of testing: Early Cinema, The Expression of Emotions and Artisans and Industrialization.
Procedure
Parallel forms of the reading comprehension test were administered at three different testing times: at baseline, four months into the intervention and upon the completion of the intervention, eight months from the beginning. The baseline test was administered during the first week of classes (beginning of October), following which the initial results testing for group equivalence were computed, allowing the experimental treatment to begin in the second week of semester. The second test was administered during the last week of first semester, after eleven weeks of classes (end of December), whereas the final test was administered after another twelve weeks of classes, during the last week in the second semester (end of May). Students from both groups completed the test individually during a 45-minute session. While there was no missing data, the data from a total of three participants were lost due to attrition (one participant only completed the initial test, while two more completed the initial and the second test, but not the final test). The data from these participants were excluded prior to conducting all relevant statistical tests.
Analysis
The collected data were analysed using descriptive and inferential statistical tests. The data were checked for normality, following which a series of Repeated Measures (RM) ANOVAs and Cochran's Q tests were applied, relative to the nature of data.
The data were analysed using SPSS v.20 statistics software.
RESULTS
Following the presentation of the results of descriptive tests, the results in this section will be given in order of the inferential tests that were applied in the analysis of data. First subsection will show the results of RM ANOVAs performed on the continuous variables of Vocabulary, Factual Information, Prose Summary and Total Reading Score. The second subsection will present the results of Cochran's Q tests, which is considered to be an alternative to the RM ANOVA test,9 performed on the dichotomous variables of Sentence Simplification, Reference Question and Insert Text.
Table 2 shows the descriptive statistics for the continuous variables of Vocabulary, Factual Information, Prose Summary and Total Reading Score.
In examining the distribution of data, the results indicate that the data is either moderately skewed (values between -1 and -/ or between +/ and +1) or approximately symmet-
ETS - Educational Testing Service. (2009J. The official guide to TOEFL® Test. McGraw-Hill.
Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. Routledge.
Table 2
Descriptive Statistics for Vocabulary, Factual Information, Prose Summary and Overall Reading Score across Three Testing Times
Variable Group Mean SD Skewness Std.Error of Skewness Kurtosis Std. Error of Kurtosis
Vocabulary Experimental Control 2.13 2.24 0.69 0.70 -.547 -.368 .536 .501 -.584 -.764 1.038 .972
Baseline Factual infor- Experimental 2.44 0.63 -.227 .536 -1.516 1.038
mation Control 2.31 0.80 -.346 .501 -1.008 .972
£ Prose sum- Experimental 1.67 0.30 -.210 .536 -.472 1.038
s e 1- mary Control 1.79 0.37 -1.464 .501 .652 .972
Total reading Experimental Control 8.28 8.27 1.74 1.53 -.583 .169 .536 .501 -.293 -.329 1.038 .972
01 -C c Vocabulary Experimental Control 2.84 2.40 0.32 0.56 -1.461 -.339 .536 .501 .137 -1.050 1.038 .972
c CD ^ eg Factual infor- Experimental 2.21 0.66 -.482 .536 -.188 1.038
2 -E c C c mation Control 2.11 0.61 -.235 .501 -.218 .972
!= !D Prose sum- Experimental 2.31 0.87 -1.397 .564 1.816 1.091
CM mary Control 1.65 0.34 -.413 .501 -.870 .972
m 01 h- Total reading Experimental Control 9.23 8.64 2.17 1.33 -.368 -.511 .536 .501 -1.100 .025 1.038 .972
01 -C c Vocabulary Experimental Control 2.50 2.33 0.51 0.58 -.121 -.128 .536 .501 -2.267 -.537 1.038 .972
b o Factual infor- Experimental 2.06 1.00 -.920 .536 .080 1.038
JÊ .E c C c mation Control 2.14 0.79 -.272 .501 -1.312 .972
!= OJ 00 n Prose sum- Experimental 1.75 0.55 -2.399 .536 5.675 1.038
mary Control 0.59 0.87 .984 .550 -.916 1.063
m 01 1- Total reading Experimental Control 8.97 7.05 1.88 2.06 -1.083 -.222 .536 .501 1.272 -.385 1.038 .972
ric (values between -/ and +/,) and that the excess kurtosis in the tested variables shows both positive results, indicating more outliers than normality, and negative, indicating fewer outliers. The kurtosis values are between -1.513 and 1.816, which is considered acceptable in proving normal univariate distribution, for all except the following two variables: Vocabulary and Prose Summary for which the data were collected during the third testing time. This will be addressed in the following section.
For the binary variables of Sentence Simplification, Reference Question and Insert Test, percentages of correct answers are shown in Table 3.
The following sections outline the results of statistical analyses conducted with the aim of answering if there is a statistically significant change in the students' scores on the read-
ing comprehension measure over time (8 months) between the Experimental and the Control Group.
Vocabulary, Factual Information and Prose Summary
A series of mixed within-between repeated measures ANO-VAs (RM ANOVAs) were performed to compare the scores of the students from the Experimental and the Control Group across three different sets of questions (Vocabulary, Factual Information and Prose Summary) used to measure EFL reading level at three different points in time (baseline, after 4 months, after 8 months). As previously stated, a repeated measures ANOVA was chosen in place of two one-way ANOVAs in an attempt to increase the power of the study; furthermore, even though all the dependent variables were drawn from the same TOEFL test, possibly suggesting that
Table 3
Percentages of Correct Answers for Sentence Simplification, Reference Question and Insert Text across Three Testing Times
Group
Experimental Control
Sentence Simplification - Test 1 79.0% 76.2%
Sentence Simplification - Test 2 92.4% 90.2%
Sentence Simplification - Test 3 83.3% 82.4%
Reference Question - Test 1 61.1% 71.4%
Reference Question - Test 2 83.3% 73.5%
Reference Question - Test 3 94.4% 82.4%
Insert Text - Test 1 61.8% 69.5%
Insert Text - Test 2 95.2% 61.1%
Insert Text - Test 3 88.9% 76.2%
MANOVA would be a more appropriate statistical test, no significant correlations were found between the dependent variables, providing further confirmation of the appropriateness of a repeated measures ANOVA. The author is aware of the risk of inflating Type I error by choosing this statistical test and will proceed cautiously with the interpretation of the results. Bonferroni post-hocs were performed for with-in-subjects simple effects and t-tests, as planned comparisons, for between-group simple effects. A preliminary testing of the model assumptions for RM ANOVA was conducted to check for normality and homogeneity of variances, including sphericity; no serious violations were discovered in all cases except with the variables of Vocabulary and Prose Summary for which the data were collected during the third testing time. The results revealed that, although the sphericity assumption was met, the data for these variables violated the assumptions of homoscedasticity and normality of distribution, which prompted a transformation of data, using a log base 10 transformation.10 Following this, the transformed data were checked and the results indicated that the data met the necessary assumptions.
The results of the RM ANOVA showed that there was a statistically significant main effect of Testing Time (F=3.587, p<.05, np2=.08) and of Group (F=4.106, p<.05, qp2=.10) on the students' results on the Vocabulary questions. Bonferroni post-hoc analysis was implemented to isolate the statistically significant mean differences for Testing Time, as a with-in-group variable. The results reveal that the Experimental Group made statistically significant progress at second testing time when compared to the first (p<.01). Furthermore, in order to make post-hoc comparisons between conditions and determine simple effects, three independent samples t-tests were conducted for each testing time. The results reveal that the Experimental Group outperformed the Control
Group during the second testing time (p<.01). In contrast, the results showed that there was no statistically significant effect of the interaction between Testing Time and Group. The effect sizes for both main effects, reported above as partial eta-squared, were considered medium (Cohen, 1988).
With respect to the tasks testing the variable of Factual Information, both main effects and the interaction effect were not statistically significant, with participants from the Experimental and Control Group showing similar scores during three different testing times.
Additionally, in testing the effect of the experimental treatment on the dependent variable of Prose Summary, a statistically significant main effect was found for Testing Time (F=15.693, p<.001, np2=.284) and Group (F=25.976, p<.001, np2=.456), as well as for the interaction between Testing Time and Group (F=9.001, p<.001, np2=.352). The Bonferroni post-hoc analysis yielded several statistically significant mean differences: firstly, the Experimental Group made progress on the second test compared to the first one (p<.01) and, secondly, the Control Group performed statistically significantly worse on the third test compared to both the first (p<.001) and the second test (p<.001). Pairwise comparisons between groups revealed that the Experimental Group outperformed the Control on both the second (p<.01) and the third test (p<.001). The reported effect sizes for the main effects and the interaction effect (partial eta-squared) are considered large (Cohen, 1988).
Sentence Simplification, Reference Question and Insert Text
In contrast to the continuous dependent variables analysed above, the dependent variables of Sentence Simplification,
10 Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. Routledge.
Insert Text and Reference Question were binary variables, which prompted the use of Cochran's Q, as an alternative to the RM ANOVA test.
When investigating within-group differences for Sentence Simplification, Cochran's Q test indicated that no statistically significant results were obtained for either the Experimental or the Control Group. The Experimental Group's scores on the Reference Question task, on the other hand, yielded a statistically significant result (Q=9.222, p<.01). A pairwise post-hoc Dunn test revealed that the difference was statistical both for the second test and the third test when compared to the first test (p<.01). As SPSS does not return the results for effect sizes when conducting Cochran's Q test, the chance-corrected measure of effect size was separately calculated following Berry, Johnston and Mielke (2018). The analysis returned a value of R=.121, indicating approximately 12% within-group agreement above what is expected by chance. The scores of the Control Group showed no statistically significant variation with respect to Testing Time. Likewise, in testing the variable of Insert Text, statistically significant results were found in the Experimental Group (Q=11.455, p<.01), but not in the Control Group. A pairwise post-hoc Dunn test indicated that the Experimental Group achieved a statistically significantly better result on the second and the third test compared to the first test (p<.01). The value of the chance-corrected measure of effect size was R=.213, indicating approximately 21% within-group agreement above what is expected by chance.
In order to test between-group differences with the dichot-omous variables of Sentence Simplification, Insert Text and Reference Question, a series of Mann-Whitney U tests were performed, with the results revealing that prior to the beginning of the intervention there were no statistically significant differences between the groups. Furthermore, no statistically significant differences between the groups were found 4 months into the experiment, as well as upon the completion of the experiment for the tasks of Sentence Simplification and Reference Question. In contrast, the difference between the Control Group's and the Experimental Group's scores for the variable of Insert Text was discovered to be statistically significant on the second test, in favour of the Experimental Group (IM03.500, p<.001).
Overall Reading Comprehension
Finally, the participants' overall scores on the reading comprehension measure were analysed by means of an RM ANOVA test. The data were first checked for normality and homogeneity of variances, including sphericity, with no violations observed. The results of RM ANOVA reveal that there was a statistically significant main effect for Group (F=4.668, p<.05, np2=.11) and a statistically significant interaction between Testing Time and Group (F=3.516, p<.05, qp2=.08), suggesting that the effect of time depends on whether the participants performed reading tasks in cooperative groups
or individually. Specifically, post-hoc testing revealed that the Experimental Group overall performed statistically significantly better than the Control Group on the second test (p<.01) and the third test (p<.01). The effect sizes for the main effect and the interaction effect, indicated above as partial eta-squared, were considered medium. No statistically significant simple effects were discovered for within-subject differences.
DISCUSSION
The current study investigated the effectiveness of the cooperative component within Collaborative Strategic Reading -CSR framework (Klingner et al., 2012) in improving EFL reading skills of young adult learners who are learning English in a university setting. The study aimed at answering whether there was a statistically significant change in the students' scores on the reading comprehension measure over time (8 months) between the Experimental and the Control Group. The findings indicate that the students exposed to reading instruction in cooperative teams performed better than the students in a teacher-led classroom with respect to a number of reading tasks, including on the overall reading scores at the end of the experimental treatment.
Specifically, in answering Vocabulary questions the students in the Experimental Group scored better in the middle of the experiment when compared to the initial test, but not on the final test, which leads to the conclusion that progress in the second semester cannot be confirmed. The Experimental Group also outperformed the Control Group on the second test, but not on the final test, prompting the need to altogether re-examine the approaches to teaching comprehension of explicit textual information, as neither cooperative teamwork, nor teacher-led instruction seem to be beneficial. In the cooperative approach, members of the team decode the text together and deal with breakdowns in comprehension by applying the prescribed strategies outlined in the Methods section. The drawback is that, without a dictionary confirmation or the involvement of the teacher, the students have no way of knowing if their guess is correct, or if they should even doubt it. This would require both metacognitive and metalinguistic awareness, which is an unrealistic expectation for many learners (Soto et al., 2020). On the other hand, in the teacher-led approach, the teacher can ensure that the students reach the correct meaning of the word; however, the teacher is unable to monitor for comprehension breakdowns in every single student, since here, as well, we are relying on the students' metacognitive awareness that they will be able to recognize when they do not understand what they are reading. At this point it is also necessary to address a limitation of this study that is generally related to vocabulary questions. Namely, even though TOEFL test preparation guidelines state that the topics of the texts are general and, thus, do not favour experts in a particular scientific or professional field, or persons with
specific prior knowledge of the topic of the text, it is possible that the topic of emotions presented from a socio-cultural and psychological perspective on the second test was nevertheless closer to students than the labour union theme presented from a political-historical perspective on the third test. Readers with adequate prior knowledge of the subject area of the text are able to create complex and correct text models (Karimi, 2018), with prior knowledge being significantly associated with vocabulary knowledge and success in understanding the text (Bernhardt, 2011; Martínez, 2022). Furthermore, a number of authors have qualms as to what is actually measured by vocabulary tasks, with the results of certain studies indicating that in about a fifth of cases what is actually measured is the reader's ability to make correct inferences (Kremmel & Schmitt, 2016;Schmitt, 1999). In light of the current study, the primary objective of vocabulary tasks was not to test whether the respondents know the meaning of a particular word, but to verify that they understand the meaning of the word in the context of the sentence or the text in which the word appears and that in the case of polysemous words they can decide which meaning is adequate in a given context. In other words, context plays a very important role in every act of text comprehension, which raises the issue of differentiating between the learner's knowledge of a word in context and the learner's skill to understand the word due to contextual clues. This is a question that is beyond the scope of this paper and that is yet to be answered.
With respect to the tasks testing the variable of Factual Information, which examined the reader's ability to find the answer to a question about an important fact explicitly written in the text, no statistically significant main effects were found, with participants from the Experimental and Control Group showing similar scores during three different testing times. The results were already relatively high for both groups, possibly suggesting positive effects of transfer of their L1 reading skills (Sparks et al., 2009). Furthermore, with the initial mean scores already high, it is rather improbable that the students would be able to make significant progress regardless of the framework of instruction they were exposed to.
In testing the variable of Prose Summary, the results indicate that the students from the Experimental Group achieved statistically significantly higher results than the students from the Control Group both on the second test and on the final test. Since summary is one of the strategies that is explicitly taught within the strategic input to which both groups were exposed, the results in favour of the Experimental Group suggest that the cooperative framework offers a more beneficial context for the acquisition and implementation of the strategy than the teacher-centred approach. Within the latter approach, students are essentially free to decide whether and to what extent they will use their cognitive capacities and apply the strategy, whereas the teacher receives confirmation that the student is participating only when
the student is called upon to give the answer in front of the whole class. On the other hand, students working together in smaller groups have to participate more actively in the implementation of this strategy, since they are responsible for their participation not only to the teacher, but also to other members of the group. Furthermore, a comparative analysis of the results of the Control Group at three testing times indicates that the Control Group performed statistically significantly worse on the third test compared to both the first and the second test. A possible explanation for such an unexpected result may be found in the very dynamics of practicing writing summaries within the traditional framework. Specifically, the teacher exposed both groups to the strategy, after which both groups practiced writing a summary each time they read a text in English. In the Experimental Group, each of the student teams would read their abstract, after which the teacher and the other students would comment on whether the summary was correct, whether it adequately conveyed the main idea of the text, its key terms, etc. In the Control Group, on the other hand, students practiced this strategy individually, after which a relatively small number of students read the summary in front of the class and received feedback from the teacher, so that the teacher could not check whether all the summaries in the Control Group were correct, or if all the students were paying attention to comments that would help them solve the task properly. Another explanation for this result may lie in the formulation of the task itself. In the first and second tests, where no statistically significant differences were found in the results, students had to choose three of the six sentences offered that best express the main idea of the text. This task was somewhat more complex on the third test. The students first had to select from among seven sentences five that best represent the ideas in the text and then place them in the appropriate category. Although the change in the task is minimal, it is still sufficient to indicate a significant deficiency in the way students in the Control Group approached the understanding of the text and its key parts, suggesting that the knowledge of solving the summary task remained at the declarative level (cf. Anderson, 2013) as the students failed to generalize that knowledge and come up with ways to apply it in a modified context.
With respect to the task of Sentence Simplification, since the results indicate that students exposed to either of the treatments did not make statistically significant gains in their scores, it is impossible to draw conclusions. Neither the traditional nor the cooperative approach can be connected with improvement in skills required to successfully complete this task, leaving open to speculation which instructional treatment may be beneficial in this respect.
It was further found that the statistically significant differences in scores on the tasks of Insert Text and Reference Question were found with the Experimental Group's scores on the second and third tests compared to the initial test, suggesting that the benefits of working in cooperative
teams in terms of finding lexical, grammatical, and logical connections in the sequence of sentences and recognizing the links between anaphoric words and their antecedents or postcedents are visible after a relatively short period of time (four months after the beginning of the experiment) and that longer exposure under these conditions does not contribute to further development of knowledge and skills. Furthermore, as both Insert Text and Reference Question assess the reader's ability to focus on the macrostructure of the text, it appears that cooperative framework can be associated with higher gains in scores on the so-called global tasks, i.e. tasks that require learners to view the text as a whole and make inferences based on linguistic input (Becker, 2016). This may be due to the dynamics of group work in cooperative teams, in which the students, in order to accomplish tasks, must make their thinking process explicitly known to the rest of the group, provide arguments for their vantage points and, as a group, negotiate the meaning of the text. This, in turn, may lead to a more developed awareness of the text as a whole, of its main ideas and key issues, which is in line with Goldman and associates (2016).
Finally, it was determined that the Experimental Group significantly outperformed the Control Group on the overall reading measure, both during second and during third testing session, but also that no statistically significant differences were found when the groups' scores were compared individually across different testing times. These results suggest that the cooperative framework was more beneficial than the teacher-centred instruction in developing the students' reading skills. As no within-group differences were discovered, the results further suggest that the strategic framework itself cannot be associated with any gains in reading skills with young adult EFL learners.
CONCLUSION
The main focus of this study was the investigation of the effectiveness of the cooperative component of the Collaborative Strategic Reading framework in developing EFL learners' reading skills. The results offer a mixed pattern of findings, with the most conclusive differences between students exposed to different experimental treatments established with respect to the reading comprehension questions that mostly focus on global information - Prose Summary and, to an extent, Insert Text and Reference Question. In correctly answering these questions, readers must approach the text in a holistic manner and focus on its main ideas, which seems to be facilitated by discussions in structured hetero-
geneous teams and negotiations of meaning resulting from those discussions.
The findings in this report are subject to several limitations, among which is certainly the instrument used to assess the students' reading proficiency. The issues concerning questions that test the readers' knowledge of vocabulary have already been discussed in the previous section and it has been noted that in one fifth of these tasks the skill of drawing conclusions based on contextual information is measured instead of the knowledge of vocabulary. Also, given the importance of prior knowledge in developing reading comprehension, it is possible that the topics of the texts favoured certain readers, putting others at a disadvantage. Furthermore, the study used a convenience sample of small size that in all likelihood reduced the statistical power of the results. With a small sample size, caution must be applied as the findings might not be directly transferable to other similar classroom contexts.
Notwithstanding these limitations, the present study hopefully adds to the ever-growing body of research on cooperative learning particularly by emphasizing the practical implications that follow from its results. A cooperative learning environment can provide an alternative to a typical university-level foreign language classroom, which mainly includes large groups of students and in which, due to group size constraints, the teacher uses the traditional, lecture-style method most of the time and tries to establish and maintain control over all aspects of classroom organization. Cooperative teams within CSR allow teachers to organize an effective, interactive context for reading academic texts in English, within which they can more easily identify the weaker and stronger sides of individual students and provide assistance at the right time to those who need it. The research findings also highlight the importance of feedback in the development of metacognitive awareness when reading. In the traditional teaching method, relatively few students receive feedback from teachers about their work, as opposed to cooperative teams, in which students receive feedback on a continuous basis, both from their teacher and, to a large extent, from other members of the group, which helps them form a clearer picture of their current knowledge and skills.
DECLARATION OF COMPETITING INTEREST
None declared.
REFERENCES
Alrayah, H. (2018). The effectiveness of cooperative learning activities in enhancing EFL learners' fluency. English Language Teaching, 11(4), 21-31. https://doi.org/10.5539/elt.v11n4p21
Anderson, J. (2013). Rules of the mind (e-book ed.). Psychology Press. https://doi.org/10.4324/9781315806938
Becker, A. (2016). L2 students' performance on listening comprehension items targeting local and global information. Journal of English for Academic Purposes, 24, 1-13. https://doi.org/10.1016/jjeap.2016.07.004
Bernhardt, E. B. (2011). Understanding advanced second-language reading. Routledge.
Berry, K., Johnston, J., & Mielke, P. (2018). The measurement of association. A permutation statistical approach. Springer. https:// doi.org/10.1007/978-3-319-98926-6_2
Boardman, A. G., Vaughn, S., Buckley, P., Reutebuch, C., Roberts, G., & Klingner, J. (2016). Collaborative strategic reading for students with learning disabilities in upper elementary classrooms. Exceptional Children, 82(4), 409-427. https://doi. org/10.1177/0014402915625067
Bremer, C., Vaughn, S., Clapper, A., & Kim, A. (2002). Collaborative Strategic Reading (CSR): Improving secondary students' reading comprehension skills. In Research to practice brief (vol. 1, pp. 2-9). National Center on Secondary Education and Transition.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Cohen, A. & Upton, T. (2006). Strategies in responding to the New TOEFL reading tasks (TOEFL Monograph No. MS-33). Educational Testing Service. https://doi.org/10.1002/j.2333-8504.2006.tb02012.x
Cunningham, W., Sanchez-Puerta, M. L., & Wuerml, A. (Eds.), (2010). Active labor market programs for youth. A framework to guide youth employment interventions. World Bank. http://hdl.handle.net/10986/11690
Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educational Technology Research and Development, 61, 563-580. https://doi. org/10.1007/s11423-013-9305-6
Estes. M. D., Ingram, R., & Liu, J. C. (2014). A review of flipped classroom research, practice, and technologies. International HETL Review, 4. https://www.hetl.org/feature-articles/a-review-of-flipped-classroom-research-practice-and-tech-nologies
Fan, Y. (2010). The effect of comprehension strategy instruction on EFL learners' reading comprehension. Asian Social Science, 6(8), 19-29. https://doi.org/10.5539/ass.v6n8p19
Fan, Y. (2015). Fostering learner autonomy through a socio-cognitive model of reading comprehension instruction. British Journal of Education, Society & Behavioural Science, 9(2), 105-117. https://doi.org/10.9734/BJESBS/2015/18161
Fen, G. (2011). Theme-based group teaching of college oral English: Endorsed by students in China EFL context. English Language Teaching, 4(1), 33-41. https://doi.org/10.5539/elt.v4n1p33
Gall, M., Gall, J., & Borg, W. (2007). Educational Research: An Introduction (8th ed.). Allyn & Bacon.
Goldman, S. R., Britt, M. A., Brown, W., Cribb, G., George, M., Greenleaf, C., Lee, C. D., & Shanahan, C. (2016). Disciplinary literacies and learning to read for understanding: A conceptual framework for disciplinary literacy. Educational Psychologist, 51(2), 219-246. http://dx.doi.org/10.1080/00461520.2016.1168741
Hitchcock, J., Kurki, A., Dimino, J., Wilkins, C., & Gersten, R. (2009). Evaluating the collaborative strategic reading intervention: An overview of randomized controlled trial options. Practical Assessment, Research & Evaluation, 14, Article 2. https://doi. org/10.7275/sakx-4c50
Jamieson, J., Jones, S., Kirsch, I., Mosenthal, P., & Taylor, C. (1999). TOEFL 2000 framework: A working paper (TOEFL Monograph No. MS-16). Educational Testing Service.
Johnson, D., & Johnson, R. (2005). New developments in social interdependence theory. Genetic, Social, and General Psychology Monographs, 131(4), 285-358. https://doi.org/10.3200/MONO.131.4.285-358
Johnson, D., & Johnson, R. (2008). Social interdependence theory and cooperative learning: The teacher's role. In R. M. Gillies, A. F. Ashman, & J. Terwel (Eds.), The teacher's role in implementing cooperative learning in the classroom (pp. 9-37). Springer. https://doi.org/10.1007/978-0-387-70892-8_1
Johnson, D., & Johnson, R. (2009). An educational psychology success story: Social interdependence theory and cooperative learning. Educational Researcher, 38(5), 365-379. https://doi.org/10.3102/0013189X09339057
Johnson, D. W., & Johnson, R. T. (2015). Theoretical approaches to cooperative learning. In R. M. Gillies (Ed.), Collaborative learning: Developments in research and practice (pp. 17-46). Nova Science.
Johnson, D. W., & Johnson, R. (2016). Cooperative learning and teaching citizenship in democracies. International Journal of Educational Research, 76, 162-177. https://doi.org/10.1016Zj.ijer.2015.11.009
Johnson, D., Johnson, R., & Smith, K. (2014). Cooperative learning: Improving university instruction by basing practice on validated theory. Journal on Excellence in University Teaching, 25(4), 1-26.
Jones, N. (2002). Relating the ALTE framework to the Common European Framework of Reference. In Council of Europe (Eds.), Case studies on the use of the Common European Framework of Reference (pp. 167-183). Cambridge University Press.
Karabuga, F. & Kaya E. S. (2013). Collaborative strategic reading practice with adult EFL learners: A collaborative and reflective approach to reading. Procedia - Social and Behavioral Sciences, 106, 621-630. https://doi.org/10.1016/j.sbspro.2013.12.071
Karimi, M. N. (2018). The mediated/unmediated contributions of language proficiency and prior knowledge to L2 multiple-texts comprehension: A structural equation modelling analysis. Applied Linguistics, 39(6), 912-932. https://doi.org/10.1093/ap-plin/amw059
Kim, A., Vaughn, S., Klingner, J., Woodruff, A., Klein Reutebuch, C., & Kouzekanani, K. (2006). Improving the reading comprehension of middle school students with disabilities through computer-assisted collaborative strategic reading. Remedial and Special Education, 27(4), 235-249. https://doi.org/10.1177/07419325060270040401
Klingner, J., & Vaughn, S. (2000). The helping behaviors of fifth graders while using collaborative strategic reading during ESL content classes. TESOL Quarterly, 34(1), 69-98. https://doi.org/10.2307/3588097
Klingner, J. K., Vaughn, S., Boardman, A., & Swanson, E. (2012). Now we get it! Boosting comprehension with collaborative strategic reading. Jossey-Bass.
Kremmel, B., & Schmitt, N. (2016). Interpreting vocabulary test scores: What do various item formats tell us about learners' ability to employ words? Language Assessment Quarterly, 13(4), 377-392. https://doi.org/10.1080/15434303.2016.1237516
Law, Y. K. (2011). The effects of cooperative learning on enhancing Hong-Kong fifth graders' achievement goals, autonomous motivation and reading proficiency. Journal of Research in Reading, 34(4), 402-425. https://doi.org/10.1111/j.1467-9817.2010.01445.x
Martínez, A. C. L. (2022). The effect of perceived interest and prior knowledge on L2 reading comprehension via several assessment methods in a higher education context. European Journal of Language and Literature, 8(2), 45-52. https://doi. org/10.26417/ejls-2019.v5i1-194
Mendieta, J., Múnera, L., Olmos, T., Onatra, C., Pérez, P., & Rojas, E. (2015). Fostering reading comprehension and self-directed learning in a collaborative strategic reading (CSR) setting. Íkala, revista de lenguaje y cultura, 20(1), 15-42. https://doi. org/10.17533/udea.ikala.v20n1a02
Milic, S. (2016). Analiza interkulturalnih kompetencija u nacionalnom okviru kurikuluma Republike Srbije i poredenje sa nacion-alnim okvirima kurikuluma referentnih zemalja [An analysis of intercultural competences in the Framework of the national curriculum of the Republic of Serbia and its comparison with the national frameworks of referent countries]. In D. Petrovic & T. Jokic (Eds.), Interkulturalno obrazovanje u Srbiji - regulativni okvir, stanje i mogucnosti za razvoj (pp. 45-67). Centar za obrazovne politike.
Namaziandost, E., Homayouni, M., & Rahmani, P. (2020). The impact of cooperative learning approach on the development of EFL learners' speaking fluency. Cogent Arts & Humanities, 7(1), 1780811. https://doi.org/10.1080/23311983.2020.1780811
Ning, H., & Hornby, G. (2014). The impact of cooperative learning on tertiary EFL learners' motivation. Educational Review, 66(1), 108-124. https://doi.org/10.1080/00131911.2013.853169
Reynolds, D. (2017). Interactional scaffolding for reading comprehension: A systematic review. Literacy Research: Theory, Method, and Practice, 66(1), 135-156. https://doi.org/10.1177/2381336917718820
Schmitt, N. (1999). The relationship between TOEFL vocabulary items and meaning, association, collocation, and word-class knowledge. Language Testing, 16, 189-216. https://doi.org/10.1177/026553229901600204
Soto, C., Gutierrez de Blume, A. P., Carrasco Bernal, M. A., & Contreras Castro, M. A. (2020). The role of meta-cognitive cues on the comprehension of proficient and poor readers. Journal of Research in Reading, 43(3), 272-289. https://doi.org/10.1111/1467-9817.12303
Sparks, R., Patton, J., Ganschow, L., & Humbach, N. (2009). Long-term crosslinguistic transfer of skills from L1 to L2. Language Learning, 59(1), 203-243. https://doi.org/10.1111/j.1467-9922.2009.00504.x
Topalov, J, Bjelica, M., & Krombholc, V. (2015). Stavovi studenata prema kolaboraciji u univerzitetskoj nastavi engleskog jezika. [Students' attitudes towards collaboration in the university level EFL instruction]. In B. Radic-Bojanic (Ed.), Strani jezici na Filozofskom fakultetu: primenjenolingvisticka istrazivanja (pp. 107-126). Filozofski fakultet.
Topalov, J., & Radic-Bojanic, B. (2016). Collaborative strategic reading instruction as a determinant of developed affective variables. In S. Guduric & M. Stefanovic (Eds.), Jezici i kulture u vremenu iprostoru (pp. 507-517). Filozofski fakultet.
Topalov, J., Radic-Bojanic, B., & Bruner, D. (2017). Thai and Serbian student attitudes regarding collaborative tasks in the EFL classroom. In B. Vujin & M. Radin-Sabados (Eds.), English studies today: Words and visions (p. 445-461). Filozofski fakultet.
Vaughn, S., Klingner, J., Swanson, E. A., Boardman, A. G., Roberts, G., Mohammed, S., & Still-Spisak, S. J. (2011). Efficacy of collaborative strategic reading with middle school students. American Educational Research Journal, 48, 938-964. https://doi. org/10.3102/0002831211410305
Wei, P., & Tang, Y. (2015). Cooperative learning in English class of Chinese junior high school. Creative Education, 6(03), 397. https://doi.org/10.4236/ce.2015.63039
Zoghi, M., Mustapha, R., & Maasum, T. N. R. B. T. M. (2010). Collaborative strategic reading with university EFL learners. Journal of College Reading and Learning, 41(1), 67-94. https://doi.org/10.1080/10790195.2010.10850336
