CC BY-ND
291
National Research University Higher School of Economics Journal of Language & Education Volume 5, Issue 4, 2019
Rashtchi, M., & Mazraehno, M. R. T. (2019). Exploring Iranian EFL Learners' Listening Skills via TED Talks: Does Medium Make a Difference? Journal of Language and Education, 5(4), 81-97. doi: https://doi.org/10.17323/ jle.2019.9691
Exploring Iranian EFL Learners' Listening Skills via TED Talks: Does Medium Make a Difference?
Mojgan Rashtchi
Islamic Azad University
Correspondence concerning this article should be addressed to Mojgan Rashtchi, TEFL Department, Faculty of Foreign Languages, Islamic Azad University, North Tehran Branch, 1651153311.
E-mail: mojgan.rashtchi@gmail.com
Mohammad Reza Tollabi Mazraehno
Islamic Azad University
Correspondence concerning this article should be addressed to Mohammad Reza Tollabi Mazraehno, TEFL Department, Faculty of Foreign Languages, Islamic Azad University, North Tehran Branch, 1651153311.
E-mail: Mr.Tollabi@gmail.com
This quasi-experimental study examined whether TED (Technology, Entertainment, Design) talks could improve the listening skills of Iranian EFL learners. The study also explored whether the different media of instruction could have differing impacts on the students' learning. Sixty intermediate level male learners aged between 18 and 20 in three intact classes (n1=n2=n3=20) were the participants selected through convenience sampling. The groups were randomly assigned to two experimental groups and one control group after ensuring that they were at the same level of language proficiency. One of the experimental groups watched TED talks via mobile devices, another group watched them through laptops, and the control group practiced listening through DVDs and CDs of the course textbook (American File 2) for ten sessions. It is worth mentioning that the mobile group watched the videos outside of the class, whereas the laptop and control groups practiced listening in the class. Every session, the participants answered some comprehension questions to enable the researchers to measure their development. The repeated measures ANOVA showed the improvement of the listening skills of all participants in the three groups during the treatment. The results of the one-way ANOVA and post hoc comparison revealed statistically significant differences between the mobile group watching the TED talks and the control group, but no differences were found between the mobile group and the laptop group or between the control and the laptop groups. The researchers inferred that the observed difference was due to using TED talks via mobile devices in the study. They could also conclude that TED talks are useful sources of practicing listening skills. The study shows the efficacy of self-directed learning via mobile devices and has implications for teachers and practitioners who are seeking ways to extend language learning beyond the confinements of the classroom.
Keywords: computer-assisted language learning; EFL learners; listening skills; mobile learning; technology; TED talk
Introduction
Listening is a multifaceted process and plays a crucial role in the development of second language competence. However, the skill does not appear to be studied as much as other language skills, partly because of the assumption that merely exposing students to spoken language is sufficient for the improvement of listening comprehension.1 Teachers find the listening skill a challenging task in the classroom, partly because they believe there is not much to do regarding its development (Goh, 2014). Traditionally, employing audiotapes followed by a set of comprehension questions was the only way to teach listening. However, the advancement of technology and its application in language teaching pedagogy has suggested that researchers have examined alternative
1 Carter, R., & Nunan, D. (Eds.), (2001). The Cambridge guide to teaching English to speakers of other languages. Cambridge, UK: Cambridge University Press.
Research article
This article is published under the Creative Commons Attribution 4.0 International License.
ways for teaching the skill. The use of computers and multimedia technologies not only has facilitated teaching the listening skill (Vanderplank, 2010) but has also fostered individualized instruction (Tanner & Landon, 2009) and self-directed learning (Joo, Kim, & Cho, 2008; Lee & Lee, 2006). Technology in the classroom can accelerate learning by providing a more productive source of language input necessary for second/foreign language learning (Krashen, 1994).
Among various English learning sources, video files are the most popular form oftraining listening comprehension. Researchers have indicated that simultaneous audio and visual input could be beneficial for foreign language learners (Rashtchi & Afzali, 2011; Seo, 2002). Therefore, many studies related to EFL learning have employed video files as learning materials rather than merely audio or texts (e.g., Chapple & Curtis, 2000; Vanderplank, 2010; Williams & Thorne, 2000) and have reported the usefulness of videos over audio materials (Sulaiman, Muhammad, Ganapathy, Khairuddin, & Othman, 2017) in teaching listening. One reason for such efficacy is that video films establish a relationship between visual and auditory channels in listening comprehension (Hoven, 1999) and can give a more realistic picture of the new language and its culture in the classroom. Besides the linguistic features, such applications include paralinguistic characteristics such as body language, gestures, prosody, and the like, which convey meaning to the learners (Fidelman, 1997). Multimedia environments also allow the addition of visual and auditory information to a written text to improve comprehension. With the use of information in multiple presentation modes, comprehension can happen in textual form, visual form, auditory form, or any combination of these presentation modes (Schnotz & Grzondziel 1996). Meskill (1996) views the opportunity of multimedia users to process combined media (text, sound, and video) simultaneously, a popular trend in software design in general and learning products in particular. The role of images in developing listening comprehension has been the topic of various studies (e.g., Guichon & McLornan, 2008; Markham, Peter, & McCarthy, 2001; Sempleski, 2003). Movies have also been implemented to improve language learners' listening skills (e.g., Oiu, 2017; Safranj, 2015).
TED (Technology, Entertainment, and Design) utilizes an alternative form of media, including a website and YouTube channel, to broadcast its conference and spread ideas. TED talks are short speeches by speakers who belong to different social groups and communities who intend to share their pioneering experiences to motivate listeners and enhance their knowledge. Due to the exciting and groundbreaking topics of TED talks, they can be an excellent source for improving EFL/ESL learners' listening skills.
In the quest for finding a different way of teaching listening, the researchers of the present study decided to examine the usefulness of TED talk videos for enhancing the listening ability of Iranian EFL learners. Therefore, the participants of two experimental groups were exposed to TED talks via two technological environments -mobile devices and laptops. The mobile group watched TED talk videos outside of the class, whereas the laptop group was exposed to the videos in the class. The researchers compared the two experimental groups with a control group that used the audio-visual materials of their textbook (American File 2) to practice listening in the class. The researchers followed two objectives; the first objective was examining whether TED talks could help develop the listening skills of learners, and the second one was discovering whether the medium through which the learners received the materials had a significant role in enhancing their listening comprehension. Since EFL learners in Iran have limited exposure to English, the researchers assumed that TED talks could help them encounter real-life language. It is worth mentioning that in EFL situations, the quality and quantity of language input received from classroom instruction is constrained by factors such as time limitations, the absence of sufficient exposure of both teachers and learners to real-life language, and students' lack of interest to listen to artificial materials. One feature of TED talks is that the topics are thought-provoking and encouraging, which makes them more appealing to the learners' senses.
Literature Review
The listening process, as Goh (2014) puts forth, is both active and complex. It is active because it involves the extraction of meaning from the input, which is usually incomplete since listeners are not able to capture every word they hear, and it is complex because the construction of meaning while listening is related to an individual's cognitive processes. Besides, some inherent problems in measuring the learners' improvement (unlike speaking and writing) and manipulation of input (unlike reading) (Lynch, 2011) make the skill a more demanding activity that so many teachers feel unqualified to teach it (Field, 2008). Listening, as the primary source of input which can facilitate language learning, has been the cornerstone of many theories of SLA (Rost, 2011). It has also been shown to have a significant role in the acquisition of pronunciation (Trofimovich,
Lightbown, Halter, & Song, 2009), the improvement of vocabulary (Vidal, 2003), and grammar (Rashtchi, Nourozi Khiabani, & Roumiani, 2012). Studies have revealed its vital role in learning to interact in the language (Berninger& Abbott, 2010; Dickinson, Golinkoff, & Hirsh-Pasek, 2010; Kim, Wagner, & Lopez, 2012).
On the other hand, the authenticity of input during listening activities has received additional attention (Rost, 2011). Employing effectual input can lead to the emergence of speaking (Krashen & Terrell, 1983; Rost, 2011) as well as other language skills (Postovsky, 1974; Winitz, 1981). Authentic materials can also make the skill more accessible to learners and teachable for educators (Field, 2008; Takaesu, 2013; Vandergrift & Goh, 2012).
A focus on the authenticity of input has led researchers to recognize TED talks as useful for practicing the teaching of listening. One condition for authenticity, as in the TED talks, is the production of language by native speakers (Hwang, 2005; Morrison, 1989; Rogers & Medley, 1988). Therefore, implementing them allows students to have direct contact with the type of input that reflects actual communication in the target language (Breen, 1985). TED talks also contain the features which Field (2008) believes real conversations possess, such as prosody, false starts, hesitations, and filler devices.
The use of TED talks has been the subject of a number of studies. For example, Takaesu (2013) used TED talks as extensive material and found that they were useful for enhancing college students' listening skills. Ahluwalia (2018), based on her findings, argued that TED talks could promote the participants' listening skills, communication skills, and self-confidence. Likewise, Gagen-Lanning (2015) found that after receiving training on the use of metacognitive strategies, the participants could use TED talk videos to improve their listening skills in a self-directed learning environment.
In the present study, as stated above, TED talks were presented via two types of media in two groups; that is, laptops and mobile devices, which can relate the study to computer-assisted language learning (CALL) and mobile learning (m-learning) technologies. The use of technology in second or foreign language classrooms is not new (Dudeney & Hockly, 2007), and several studies provide evidence that the use of different forms of technology could be attractive for language learners and can help them achieve their goals. For example, emphasized the role of e-mails, the internet, and satellite and cable TV in enhancing the learning of young learners as they could create fun and increase attention when used efficiently.2 Rashtchi and Hajihassani (2010) showed the usefulness of using weblogs in teaching reading. Many studies have verified the efficacy of CALL in learning pronunciation (Rahnavard & Mashhadi Heidar, 2017), reading (Marzban, 2011); speaking (James, 1996; Rashtchi & Khoshnevisan, 2009), vocabulary (Rashtchi & Aghili, 2014), motivation (Genc & Adyin, 2010), and autonomy (Zarei & Hashemipour, 2015).
Equally, mobile technology has attracted a great deal of attention from educators regarding integrating both conventional and innovative ways of teaching and learning into the curriculum as well as presenting adaptability and utility throughout a wide range of educational learning activities in diverse learning areas. Like other subject areas, language teachers have widely applied mobile gadgets in designing language courses to enable learners to gain access to resources and benefit from various educational experiences (Kukulska-Hulme & Traxler, 2005). Various definitions of mobile learning found in the literature refer to the advantage of making learning ubiquitous (e.g., Dye, Odingo, & Solstad, 2003; Hwang, Tsai, & Yang, 2008) which, according to Thomas (2005) can give "flexibility to learners in terms of community, autonomy, locationality, and relationality (p. 2). Denk, Wever, and Belfin (2007) suggest that mobile learning can lead to autonomous and individual learning.
The researchers of the present quasi-experimental study adopted a non-equivalent pretest-posttest control group design to investigate whether the use of TED talks via different media could enhance the listening ability of Iranian EFL learners. The following research questions gave direction to the study:
R01:Do the study groups (control, mobile, laptop) similarly benefit from the type of listening activities they experience during the treatment?
R02:Does the control group's performance on the listening tests show improvement from the first to the tenth session?
R03:Does the mobile group's performance on the listening tests show improvement from the first to the tenth session?
2 Brewster, J., Ellis, G., & Girard, D. (2004). The primary English teacher's guide. London: Penguin English Guides.
RO4:Does the laptop group's performance on the listening tests show improvement from the first to the tenth session?
RO5:Does employing TED Talks have any impact on the listening comprehension ability of Iranian EFL learners?
Materials and Methods
Participants
Based on convenience sampling, 60 Iranian male learners in three intact classes were selected from a language institute in Tehran. Their age ranged between 18 and 20 years old, their mother tongue was Persian, and they had learned English as a foreign language in Iran. Their proficiency level was intermediate according to the placement test that they took at the onset of the study.
Instruments
The first instrument was a general language proficiency test adopted from the Preliminary English Test (PET). The 60-item test consisted of listening, reading, and writing sections and was used as the pretest to ensure that there was no significant difference among the groups before the study. Two experienced teachers confirmed the content validity of the test, and Cronbach's alpha indicated that the test was reliable (r= 0.81).
Another instrument was a series of comprehension questions in multiple-choice format (Appendix A) related to the TED talk videos. The researchers prepared most of the questions, but some were taken from www.ed.ted.com. The tests measured the participants' listening improvement in each session. The reason for repeated measurement was that the type of materials used for instruction was very different in the groups, and the researchers needed to ensure that the control group was also benefitting from the treatment.
The third instrument was a 68-item listening posttest, which was adopted from the Longman Preparation Course for the TOEFL iBT test (Philips, 2015). Since the materials used in the three study groups were different, a general listening test was prepared and piloted. The assumption was that after practicing and receiving instructions, students' listening ability might improve. Therefore, the researchers were cautious not to prepare a test that was too easy for the participants and piloted the test with 30 upper-intermediate learners whose demographic characteristics (age, gender, education) were similar to those of the participants of the study. The reliability of the test computed via Cronbach's alpha was 0.82.
Materials
All groups studied American File 2 (Latham-Koenig, Oxenden, & Seligson, 2013) as their textbook. Additionally, the control group received the audio and video files of the textbook as the listening material. For the mobile and laptop groups, the researchers selected ten TED talks (Appendix B) to examine their impact on the participants' listening skills. The criterion for selecting the TED talks was to have a variety of topics that could interest learners with different preferences.
Design and Variables
The current quasi-experimental study followed a nonequivalent control group pretest-posttest design. The listening skills were the dependent variable of the study, and the use of different materials (i.e., TED talks and textbook materials) were the independent variables of the study. The use of mobiles out of the class, laptops in the class, and CDs and DVDs were the moderator variables.
Experimental Groups. First, the researchers administered the general English proficiency test to verify that the groups were at the same level of language proficiency. Afterwards, the three classes were randomly assigned to experimental group one (mobile group, n=20), experimental group two (laptop group, n=20), and the control group (n=20). The treatment continued for ten sessions, three times a week, and each session was 90 minutes.
In each session, approximately 45 minutes were allocated to practicing listening. After the treatment, the participants in the three groups sat for the listening posttest.
Mobile Group. The members of this group installed the TED application on their mobiles. One day before the class, the teacher sent the learners a short message (SMS) informing them which TED video they had to download. They were asked to watch (and listen to) the TED talk as many times as they liked and take notes during the activity. In class, the learners, using their notes, discussed the content of the talk with their classmates, and negotiated the problems they had regarding understanding the talk. The group did not watch the videos in the class for two reasons. The first reason was that the researchers were interested in examining the impact of an out-of-class activity. The second was to encourage the participants to perform the required tasks, as they would not be able to participate in the class discussion and do the comprehension tests otherwise. In the class, the teacher answered their questions or explained the parts they had not understood.
Additionally, the teacher asked the learners about the number of times, where, and when they watched the videos. As reported, they usually watched the videos two or three times before the class. The important point was that they were pleased that they could pause the videos when they were interrupted for personal reasons. Then they took the reading comprehension test. The time allocated for class discussions and answering the questions was about 10 minutes (shorter than the other groups to limit learning to the out-of-class activities). Afterwards, they studied their textbook (American File 2); however, the teacher did not play the textbook's class DVDs or audio CDs during the treatment to keep a balance in the duration of exposure to listening materials in the three groups.
Laptop Group. This group watched the TED Talk videos on their laptops in the classroom. First, the participants watched the videos to have a general idea about the content of the talk. Then, during the second viewing, the learners were free to take notes, play the videos back, and to look up the meaning of words. Next, the teacher administered the comprehension test related to the lecture. They were free to use their notes while answering the test. As the final step, the participants watched the talk and compared their responses to the test with what they heard; however, they were not allowed to change their responses. This procedure usually took about 45 minutes, and the remaining time was allotted to teaching the textbook. Similar to the mobile group, the teacher did not play the DVDs and audio CDs from the book during the treatment. The participants of this group were satisfied with watching TED talks on their laptops and did the tests enthusiastically and energetically.
Control Group. The participants in the control group had the same textbook as the experimental groups. However, the teacher played the DVDs and audio CDs from the book in the class. The learners studied the book, did the listening tasks, and followed the activities manipulated by the teacher after listening to the audio CDs and watching the DVDs. The teacher answered their questions and tried to clarify the vague points. This group did not have access to the TED Talk application. The DVDs were displayed on the video projector screen, and a CD player was used to play audio CDs.
Results
As mentioned above, the researchers administered a general proficiency test at the outset of the study. Table 1 shows the descriptive statistics obtained from the test. The skewness ratio of 0.49 (falling within ±1.96, obtained from dividing skewness statistic by standard) signifies the normal distribution of the scores. Table 1
Descriptive Statistics of the Groups
N Minimum Maximum Mean Std. Deviation Skewness
Statistic Statistic Std. Error
Groups Valid N (listwise) 60 60 49.00 58.00 52.5333 2.26594 .154 .309
As Table 2 shows, the mean of the control group (M= 52.5, SD= 1.98), the mobile group (M= 53.25, SD= 2.69), and the laptop group (M=51.85, SD=1.92) were very close to each other.
Table 2
Descriptive Statistics of Each Group
95% Confidence Interval for Mean
N Mean Std. Deviation Std. Error Lower Bound Upper Bound Minimum Maximum
Control 20 52.5000 1.98680 .44426 51.5701 53.4299 49.00 56.00
Mobile 20 53.2500 2.69258 .60208 51.9898 54.5102 49.00 58.00
Laptop 20 51.8500 1.92696 .43088 50.9482 52.7518 49.00 56.00
Total 60 52.5333 2.26594 .29253 51.9480 53.1187 49.00 58.00
The Levene statistic [F (2, 57) = 2.054, p= .138] showed that the assumption of the homogeneity of variances was met, and one-way analysis of variance (ANOVA) could be run. As shown in Table 3, there was no statistically significant difference between the groups [F (2, 57) =1.975, p=0.148] regarding language proficiency at the onset of the study.
Table 3
One-way ANOVA Between the Means of the Groups, Proficiency Test
Sum of Squares df Mean Square F Sig.
Between Groups 19.633 2 9.817 1.975 .148
Within Groups 283.300 57 4.970
Total 302.933 59
To answer the first research question of the study, which compared the degree of improvement of the groups' listening skills during the treatment, a one-way repeated-measures ANOVA was performed. Table 4 shows the means and standard deviations obtained from the students' performances on the comprehension tests. As can be seen, the average scores of students in the groups increased during the treatment. However, the slope of the increase in the scores of the mobile group is higher than the other groups, and the increase in the means of the control group shows the least improvement.
Table 4
Descriptive Statistics of the Groups on Reading Comprehension Questions
Sessions Control Mobile Laptop
Mean Std Mean Std Mean Std
First 4.65 1.93 6.30 2.54 5.20 1.91
Second 3.05 0.89 4.75 1.21 5.20 1.40
Third 3.80 0.89 5.15 1.04 5.30 1.87
Fourth 4.45 1.57 6.85 1.93 6.15 1.73
Fifth 5.35 1.57 7.25 1.68 5.90 1.71
Sixth 5.50 1.24 7.35 1.50 5.70 1.26
Seventh 4.85 1.09 6.95 1.85 5.45 1.47
Eighth 5.50 0.95 8.15 1.53 5.95 1.73
Ninth 5.60 0.68 8.80 1.06 6.55 1.36
Tenth 5.60 0.68 9.10 0.79 7.15 1.23
As Table 5 shows, the results of the skewness ratios in the groups were between ±1.96, which indicates a normal distribution of the scores and justifies utilizing parametric tests.
Table 5
Skewness Ratios of the Groups' Scores on the Reading Comprehension Tests
Session Control Skewness Mobile Skewness Laptop Skewness
First 0.51 -0.01 0.49
Second 0.90 -0.06 0.89
Third -0.06 1.23 0.21
Fourth 0.60 0.48 0.22
Fifth 0.17 -0.22 0.66
Sixth -0.37 -0.57 0.81
Seventh 0.05 0.25 -0.11
Eighth 0.00 0.01 -0.59
Ninth -0.40 -0.45 -0.89
tenth 0.71 -0.19 -0.51
For comparing the performances of the groups on the ten tests, a repeated-measures ANOVA was conducted. The significant results of the Mauchly's test (p<0.05) showed that the sphericity assumption was violated, and thus the Huynh-Feldt test was used to interpret the results of the F tests on the intra-group principal effect (time) and the interactive effect (time x group). As Table 6 shows, the effects of time, group, and time x group on students' scores were significant (p<0.01). Significant time indicates that there was a significant difference between students' mean scores in at least two sessions. The significance of the group's effect also shows that the mean scores of the students in at least two groups had a significant difference. The significance of the interactive effect of time x groups also reveals that the process of change in the students' average marks over ten sessions was different between the groups.
Table 6
Repeated Measures ANOVA for the Groups
S.O. V SS df MS F p Eta squared Observed power
time 462.58 6.55 70.67 36.64 0.000 0.391 1
Time x group 107.17 13.09 8.19 4.24 0.000 0.130 1
Error 719.55 373.12 1.93
Group 498.49 2.00 249.25 27.47 0.000 0.491 1
Error 517.21 57.00 9.07
Note: Statistically significant at 0.01 level
Figure 1 shows the process of change in the participants' scores during the treatment sessions in the three groups.
Figure 1. Groups' average test scores during treatment sessions.
Table 7 presents the comparison of the means of the groups. As the results show, there are significant differences between the groups. The highest score is for the mobile group (M=7.07) and then the laptop group (M= 5.86); that is, both groups showed significant differences from the control group. Therefore, the researchers concluded that the use of TED talks via mobiles and laptops was more effective than the audio and video files used by the control group.
Table 7
Bonferroni's / Spot-Test Results or Measuring the Difference between the Three Groups
Groups Mean* SEM
Control 4.84 0.213
Mobile 7.07 0.213
Laptop 5.86 0.213
Note: * Averages with at least one common name do not have a significant difference (p <0.05)
Since the interaction of time x group was significant and also to answer the second, third, and fourth research questions, four repeated-measures ANOVAs were conducted for each group separately. The results of the test for the control group showed that the effect of time was significant (Table 8). Therefore, the mean scores of the students during the different sessions had changed significantly. Considering the guidelines proposed by Cohen (1988), the eta squared =0.37 shows a large effect size, which means that 37% of the change in the dependent variable (listening skill) was due to the treatment.
Table 8
Repeated Measures ANOVA for the Control Group
S.O.V. SS Df MS F p Eta squared Observed power
Time 135.21 9 15.02 12.51 0.000 0.397 1
Error 205.30 171 1.20
Note: Statistically significant at 0.01 level
The results of the comparison between the means of the control group via Bonferroni's post hoc test showed that in the second and third sessions, the mean scores of the students decreased significantly compared to the first session while from the fourth to the tenth sessions, there was a significant increase in the learners' mean scores (Table 9).
Table 9
Bonferroni's Post Hoc Test to Compare Means in Different Treatment Sessions, Control Group
Control
Groups Mean* SEM
First 4.65abc 0.431
Second 3.05d 0.198
Third 3.8dc 0.200
Fourth 4.45bc 0.352
Fifth 5.35ab 0.350
Sixth 5.5ab 0.276
Seventh 4.85abc 0.244
Eighth 5.5ab 0.212
Ninth 5.6a 0.152
Tenth 5.6a 0.152
Note: * Averages with at least one common name do not have a significant difference (p <0.05)
The results of the repeated measures ANOVA for the mobile group (Table 10) showed that the effect of time was significant (p <0.05). Therefore, the mean scores of the students during the different sessions had changed significantly. The magnitude of the differences in the means was very large (eta squared=0.55).
Table 10
Repeated Measures ANOVA for Mobile Group
S.O.V. SS df MS F P Eta squared Observed power
Time 362.31 9 40.26 23.26 0.000 0.550 1
Error 296.00 171 1.73
Note: Statistically significant at 0.01 level
The mean comparison for the mobile group (Table 11) showed that in the second session, the average score of the students decreased significantly compared to the first session. While from the third to the tenth session, gradually, there was an increase in the students' average scores.
Table 11
Bonferroni's Post Hoc Test to Compare Means in Different Treatment Sessions, Mobile Group
Mobile
Groups Mean* SEM
First 6.3dc 0.567
Second 4.75e 0.270
Third 5.15de 0.233
Fourth 6.85bc 0.431
Fifth 7.25bc 0.376
Sixth 7.35bc 0.335
Seventh 6.95bc 0.413
Eighth 8.15ab 0.342
Ninth 8.8a 0.236
tenth 9.1a 0.176
Note: Averages with at least one common name do not have a significant difference (p<0.05)
The results of the repeated measures ANOVA for the laptop group (Table 12) showed that the effect of time was significant (p<0.05), and the mean scores of students during the different sessions had changed significantly. Similar to the laptop group, the magnitude of the differences in the means was very large (eta squared=0.55).
Table 12
Repeated Measures ANOVA for the Laptop Group
S.O.V. SS df MS F p Eta squared Observed power
Time 362.31 9 40.26 23.26 0.000 0.55 1
Error 296.00 171 1.73
Statistically significant at 0.01 level
The means of the laptop group obtained from administering the reading comprehension test shows the steadiness of the scores and an increase in the fourth and last sessions.
Table 13
Bonferroni's Post Hoc Test to Compare Means in Different Treatment Sessions, Laptop Group
Groups Laptop
Mean* SEM
First 5.2c 0.427
Second 5.2c 0.313
Third 5.3c 0.417
Fourth 6.15abc 0.386
Fifth 5.9bc 0.383
Sixth 5.7bc 0.282
Seventh 5.45bc 0.328
Eighth 5.95bc 0.387
Ninth 6.55ab 0.303
tenth 7.15a 0.274
Note: Averages with at least one common name do not have a significant difference (p <0.05)
A one-way ANOVA was performed to answer the fifth research question and examine whether there was a statistically significant difference between the posttest results of the experimental and control groups. As Table 14 signifies, there is a significant difference between the performance of the three groups in the listening posttest [F (2, 57) =8.20, p=.001].
Table 14
ANOVA for Pretest and Posttest Scores
Test S.O.V SS df MS F p
Between group 1447.03 2 723.52 8.20 0.001
Pretest
Within group 5031.90 57 88.28
Posttest Between group Within group 372.1 3106.75 2 57 186.050 54.504 3.413 0.040
Tukey's post hoc test examined where the difference between the groups lay. As shown in Table 15, the mobile group outperformed the control group. However, there is no statistically significant difference between the mobile group and the laptop group and the laptop group and the control group (p> 0.05).
Table 15
Tukey's Post Hoc Comparison Between the Groups
Dependent Variable Level Group (i) Group (j) Mean deviation SEM p
Control Mobile 6.1 2.33 0.03*
Post-test Control Laptop 3.05 2.33 0.393
Mobile Laptop -3.05 2.33 0.398
Discussion
The statistical analysis conducted to answer the first research question of the study showed a significant difference between the three groups of the study in favor of the group that received TED talks via mobile phones. The laptop group also had a significantly better performance than the control group. This finding, consistent with previous studies, shows the importance of technology in language learning (e.g., Brett, 1995; Fidelman,
1997; Warschauer & Healey, 1998). It also finds support from those scholars who underscored the importance of ubiquitous learning (e.g., Boyinbod & Akintola, 2008; Casey, 2005; Jones & Jo, 2004; Yahya, Ahmad, & Abd Jalil, 2010). The fact that the participants in the mobile group could select where, when, and how many times to watch the TED talks, as they asserted, could have an impact on boosting their listening skills. The researchers of the present study assume that when the participants could have the possibility to listen when they were ready, they could better focus on the talks and were cognitively more prepared to learn.
The lack of facing time constraints could also have affected their learning. Additionally, it is worth mentioning that the fact that the mobile group watched the talks outside the classroom and answered the tests in the classroom could have encouraged them to keep the information in their memory for a more extended period. Adjusting the learning to personal preferences and personality characteristics (Kukulska-Hulme,2009; Tanner & Landon, 2009) could be another reason for the success of the participants in the mobile group. Similarly, the present study is in line with previous studies that accentuated the role of self-directed learning in learners' achievement and satisfaction (e.g., Joo, Kim, & Cho, 2008; Lee & Lee, 2006; Lai, Shum, & Tian, 2016).
The results of the repeated measures ANOVAs for answering the second, third, and fourth research questions revealed that the treatment in the three groups was useful and could result in the development of the participants' listening skills. These results, in general, showed the developmental stages the participants followed in the groups. However, by scrutinizing the process of improvement reflected in the scores obtained from the comprehension tests, we can see that the mobile group experienced a more considerable increase in its mean scores than the laptop and control groups. It is noteworthy that the control group had the smallest amount of improvement. These results lead the researchers to conclude that mobile learning was more effective than other types of instruction. The use of TED talks could also be responsible for the higher mean scores in the mobile and laptop groups.
The results of the one-way ANOVA, conducted for answering the fifth research question, showed a significant difference among the groups. However, contrary to the results obtained from the classroom performances of the groups on the comprehension tests, no statistically significant difference was found between the control group and the laptop group, although the materials were different in the two groups. In view of the results, two possibilities arise. First, it could be due to the length of tests; that is to say, ten items could not adequately measure learners' listening skills. Second, using laptops, as stated by the participants and verified by the teacher, was fascinating and could encourage learners to approach the tests more enthusiastically.
Moreover, the lack of a statistically significant difference between the laptop and mobile groups urged the researchers to infer that the significant difference observed in the posttest was not due to employing TED talks. In other words, the lack of a statistically significant difference between the control and laptop groups neutralizes the impact of medium and material.
On the other hand, the nonexistence of a significant difference between the mobile group and laptop group cancels out the effectiveness of the materials enabling the researchers to emphasize the efficacy of using technology for teaching listening skills. However, it can be concluded that the significant difference between the mobile group and the control group was due to the use of mobiles, TED talks, and ubiquitous learning combined. Thus, the researchers deduced that the use of mobile was the primary reason for the findings. The fact that mobile learning, unlike computer-based learning, offers learners the flexibility to choose the best time and location for learning (Keegan, 2005), provides them with the opportunity to more deeply engage in the process of learning, and as Traxler (2011) puts forth, enables them to "enhance, extend and enrich the concept and activity of learning" (p. 6).
Several researchers have supported the potential of mobile devices in language learning (e.g., Godwin-Jones, 2004; Kadyte, 2004; Tan & Liu, 2004), claiming that it can enhance students' motivation and encourage their sense of responsibility (Savill-Smith & Kent, 2003), and can meet their needs in accordance with their lifestyles (Kukulska-Hulme, 2006). Another reason for the efficacy of mobile learning over classroom-based technologies is that it provides individuals with diverse occasions to learn rather than restraining them in pre-determined situations (Traxler, 2011).
Conclusion
The findings of this study suggest mobile phones have positive effects on the listening comprehension of EFL learners, although it does not reject the role of laptops or audio tracks and videos in improving their listening skills. Moreover, the findings contribute to the role of using technology in developing listening skills. Accordingly, the study supports the role of multimedia, CALL, and MALL in boosting learning. However, the integration of technology as a learning tool cannot blur the role of the teacher in the classroom and only plays a supplementary role in the process of learning. The findings of this study should be of interest to English teachers who are in search of methods for improving EFL learners' listening comprehension knowledge. This experiment suggests that TED talks can be used as authentic materials in the listening classes and is consistent with research on self-directed learning. The researchers suggest more research with both male and female learners utilizing the TED application for improving listening skills. Qualitative studies that focus on learners' mental processes could be illuminating as well.
Conflict of Interest
The authors declare that they have no conflict of interest.
References
Ahluwalia, G. (2018). Students' perceptions on the use of TED talks for English language learning. Language in India, 18(12), 80-86.
Berninger, V. W., & Abbott, R. D. (2010). Listening comprehension, oral expression, reading comprehension, and written expression: Related yet unique language systems in Grades 1, 3, 5, and 7. Journal of Educational Psychology, 102, 635-651. https://doi.org/10.1037/a0019319 Boyinbode, O.K., & Akintola, K.G. (2008). A sensor-based framework for ubiquitous learning in Nigeria.
International Journal of Computer Science and Network Security, 8(11),401-405. Breen, M. P. (1985). Authenticity in the language classroom. Applied Linguistics, 6(1), 60-70. Brett, P. (2011). Students' experiences and engagement with SMS for learning in higher education. Innovations in
Education and Teaching International, 48, 137-147. https://doi.org/10.1080/14703297.2011.564008 Casey, D. (2005). u-Learning = e-Learning + m-Learning. Proceedings of world conference on e-learningin corporate,
government, healthcare, and higher education (pp. 2864-2871). Chesapeake, VA: AACE. Chapple, L., & Curtis, A. (2000). Content-based instruction in Hong Kong: Student responses to film. System, 28, 419-423.
Denk, M., Weber, M., & Belfin, R. (2007). Mobile learning-challenges and potentials. International Journal of
Mobile Learning and Organisation Archive, 1(2), 122-139. Dickinson, D., Golinkoff, R. M., & Hirsh-Pasek, K. (2010). Speaking out for language: Why language is central for learning development. Educational Researcher, 29, 305-310. https://doi.org/10.3102/0013189X10370204 Dudeney, G., Hockly, N. (2007). How to teach English with technology. Essex, UK: Pearson. Dye, A., K'Odingo, J. A., & Solstad, B. (2003). Mobile education - A glance at the future. Retrieved from
https://pdfs.semanticscholar.org/ Fidelman, C. G. (1997). Extending the language curriculum with enabling technologies: Nonverbal communication and interactive video. In K.A. Murphy-Judy (Ed.), NEXUS: The convergence of language teaching and research using technology (pp. 28-41). Durham, NC: CALICO. Field, J. (2008). Listening in the language classroom. Cambridge, UK: Cambridge University Press. Genc, G., & Adyin, S. (2010). Students' motivation towards computer use in EFL learning. Paper presented at the Annual International Educational Technology Conference (IETC) (Istanbul, Turkey, Apr 26-28, 2010). Retrieved from https://eric.ed.gov/?id=ED511166 Gagen-Lanning, K. (2015). The effects of metacognitive strategy training on ESL learners' self-directed use of TED Talk videos for second language listening (Unpublished master's thesis). Iowa State University, Iowa, IA. Retrieved from https://lib.dr.iastate.edu/etd/14355
Godwin-Jones, B. (2004). Emerging technologies language in action: From Webquest to virtual realities. Language
Learning and Technology, 8(3), 9-14. Goh, C. C. M. (2014). Second language listening comprehension: Process and Pedagogy. In M. Celce-Murcia, D. M. Brinton, & M. A. Snow (Eds.). Teaching English as a second or foreign language (4th ed., pp. 72-89). Boston, MA: National Geographic Learning. Guichon, N., & McLornan, S. (2008). The effects of multimodality on l2 learners: Implications for call resource
design. System, 36(1), 85-93. https://doi.org/10.10Wj.system.2007.11.005 Hoven, D. (1999). A model for listening and viewing comprehension in multimedia environments. Language
Learning & Technology, 3 (1), 88-103. Hwang, C. C. (2005). Effective EFL education through popular authentic materials. Asian EFL Journal, 7(1), 90101.
Hwang, G.J., Tsai, C.C. & Yang, S.J.H. (2008). Criteria, strategies and research issues of context-aware ubiquitous
learning. Journal of Educational Technology & Society, 11(2), 81-91. James, R. (1996). CALL and the speaking skill. System, 24(1), 15-21. https://doi.org/10.1016/0346-251X(95)00050-T Jones, V., & Jo, J. H. (2004). Ubiquitous learning environment: An adaptive teaching system using ubiquitous technology. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds)., Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 468-474). Perth, WA: ASCILITE Joo, Y. J., Kim, N. Y., & Cho, H. K. (2008). Relationship between self-efficacy, online task value and self-regulated learning, and satisfaction and achievement in cyber education. The Journal of Educational Information and Media, 14, 115-135.
Kadyte, V. (2004). Learning can happen anywhere: A mobile system for language learning. In J. Attwell & J. Savill- Smith (Eds.), MLearn: Learning with mobile devices, research and development (pp. 1-63). London, UK: Learning and Skills Development Agency. Keegan, D. (Ed.). (2005). Theoretical principles of distance education. London, UK: Routledge. Kiernan, P. J., & Aizawa, K. (2004). Cell phones in task based learning - Are cell phones useful language learning
tools? ReCALL, 16(1), 71-84. Retrieved from https://www.learntechlib.org/p/66277/ Kim, Y. -S., Wagner, R., & Lopez, D. (2012). Developmental relations between reading fluency and reading comprehension: A longitudinal study from grade 1 to grade 2. Journal of Experimental Child Psychology, 113(1), 93-111. https://doi.org/10.10Wj.jecp.2012.03.002 Krashen, S. (1994). The pleasure hypothesis. Georgetown University Round Table on Languages and Linguistics.
Washington, DC: Georgetown University Press. Krashen, S., & Terrell, T.D. (1983). The natural approach: Language acquisition in the classroom. New York, NY: Pergamon Press.
Kukulska-Hulme, A. (2006). Mobile language learning now and in the future. In P. Svensson (Ed.), From vision to
practice: Language learning and IT (pp. 295-310). Harnosand, Switzerland: Swedish Net University. Kukulska-Hulme, A. (2009). Will mobile learning change language learning? ReCALL, 21(2), 157-165. https://
doi.org/10.1017/S0958344009000202 Kukulska-Hulme, A. &Traxler. J. (2005). Mobile teaching and learning. In A. Kukulska-Hulme & J. Traxler (Eds.). Mobile learning a handbook for educators and trainers (pp. 25-44). London, UK: Routledge. https://doi. org/10.4324/9780203003428 Lai, C., Shum, M., & Tian, Y. (2016). Enhancing learners' self-directed use of technology for language learning: The effectiveness of an online training platform. Computer Assisted Language Learning, 29(1), 40-60. https:// doi.org/10.1080/09588221.2014.889714 Latham-Koenig, C., Oxenden, C., & Seligson, P. (2013). American file 2 (2nd ed.). New York, NY: Oxford University Press.
Lee, J. K., & Lee, W. K. (2006). Self-directed learning and e-learning environment satisfaction: Comparison analysis by self-regulated efficacy. Journal of the Korean Operations Research and Management Science Society, 31(3), 127-143.
Lynch, T. (2011). Academic listening in the 21st century: Reviewing a decade of research. Journal of English For
Academic Purposes, 10, 79-88. Retrieved from http://eric.ed.gov/?id=EJ926672 Markham, P., Peter, L. A., & McCarthy, T. J. (2001). The effects of native language vs. target language captions on
foreign language students' DVD video comprehension. Foreign Language Annals, 34(5), 439-445. Marzban, A. (2011). Improvement of reading comprehension through computer-assisted language learning in Iranian intermediate EFL students. Procedia Computer Science, 3, 3-10. https://doi. org/10.1016/j.procs.2010.12.003 Meskill, C. (1996). Listening skills development through multimedia. Journal of Educational and Hypermedia,
5(2), 179-201.
Morrison, B. (1989). Using news broadcasts for authentic listening comprehension. ELT Journal, 43, 14-18.
Phillips, D. (2015). Longman preparation course for the TOEFL iBT test. New York, NY: Pearson.
Postovsky, V. A. (1974). Effects of delay in oral practice at the beginning of second language learning. The Modern
Language Journal, 58(5/6), 229-239. Qiu, J. (2017). The effect of English movies on college English listening teaching. Theory and Practice in Language " Studies, 7(11), 1011-1024. https://doi.org/10.17507/tpls.0711.10
Rahnavard, F., & Mashhadi Heidar, D. (2017). The impact of computer-assisted language learning (call) /web-based instruction on improving EFL learners' pronunciation ability. International Journal of Research in English Education, 2(7), 49-57. https://doi.org/10.18869/acadpub.ijree.2.1.49 Rashtchi, M., & Afzali, M. (2011). Spoken grammar awareness raising: Does it affect the listening ability of Iranian
EFL learners? Studies in Second Language Learning and Teaching, 1(4), 515-531. 10.14746/ssllt.2011.1.4.4 Rashtchi, M., & Aghili, H. (2014). Computerized input enhancement versus computer-assisted glosses: Do they affect vocabulary recall and retention? Theory and Practice in Language Studies, 4(8), 1665-1674. https://doi. org/10.4304/tpls.4.8.1665-1674 Rashtchi, M., & Hajihassani, H. (2010). Blog-assisted language learning: A possibility in teaching reading to
Iranian EFL learners. International Journal of Language Studies, 4(4), 245-262. Rashtchi, M., & Khoshnevisan, B. (2009). Audiotaped dialogue journal: A technique to improve speaking skill of
Iranian EFL learners. The Journal of Applied Linguistics, 1(3), 164-176. Rashtchi, M., Nourozi Khiabani, M., & Roumiani, N. (2012). The effect of listening to audiotaped journals on
Iranian EFL learners' grammar knowledge. Advances in Asian Social Sciences, 4(1), 764-768. Rogers, C. V., & Medley, F. W. (1988). Language with a purpose: Using authentic materials in the foreign language
classroom. Foreign Language Annals, 21(5), 467-478. Rost, M. (2011). Teaching and researching listening (2nd ed.). Harlow, UK: Pearson.
Safranj, J. (2015). Advancing listening comprehension through movies. Procedia Social and Behavioral Sciences,
191, 169 - 173. https://doi.org/10.1016Zj.sbspro.2015.04.513 Savill-Smith, C., & Kent, P. (2003). The use of palmtop computers for learning: A review of the literature. London, UK: Learning and Skills Development Agency. Retrieved from http://www.m-learning.org/docs/the_use_of_palmtop_computers_for_ learning_sept03.pdf Schnotz, W., & Grzondziel, H. (1996, April). Knowledge acquisition with static and animated pictures in computer-based learning. Paper presented at the Annual Meeting of American Educational Research Association (New York).
Sempleski, S. (2003). Integrating video into the classroom curriculum. Selected Papers from the 12th International
Symposium on English Teaching. Taipei, Taiwan: Crane. Seo, K. (2002). The effect of visuals on listening comprehension: A study of Japanese learners' listening strategies.
International Journal of Listening, 15, 57-81. Sulaiman, N., Muhammad, A. M., Ganapathy, N. N. D. F., Khairuddin, Z., & Othman, S. (2017). A comparison of students' performances using audio only and video media methods. English Language Teaching, 10(70), 210215. https://doi.org/10.5539/elt.v10n7p210 Takaesu, A. (2013). TED talks as an extensive listening resource for EAP students. Language Education in Asia, 4
(2),150-162. http://dx.doi.org/10.5746/LEiA/13/V4/I2/A05/Takaesu Tan, T., & Liu, T. (2004). The mobile interactive learning environment (MOBILE) and a case study for assisting elementary school English learning. Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies ICALT(pp. 530- 534). Washington, DC: IEEE. Tanner, M. W., & Landon, M. M. (2009). The effects of computer-assisted pronunciation readings on ESL learners' use of pausing, stress, intonation, and overall comprehensibility. Language Learning & Technology, 13 (3), 5165.
Thomas, S. (2005). Pervasive, persuasive eLearning: Modeling the pervasive learning space. In J. C. Richards (Ed.), Proceedings of the 3rd international conference on pervasive computing and communications workshops (pERCOMW'05) (pp. 332-336). Kauai Island, Hawai'i: IEEE Computer Society. https://doi.org/10.1109/ PERCOMW.2005.63
Traxler, J. (2011). Introduction. In J. Traxler & J. Wishart (Eds.), Making mobile learning work: Case studies of
practice (pp. 4-12). Bristol, UK: ESCalate. Trofimovich, P., Lightbown, P. M., Halter, R., & Song, H. (2009). Comprehension based practice: The development
of pronunciation in a listening and reading program. Studies in Second Language Acquisition, 31, 609-639. Vandergrift, L., & Goh, C. (2012). Teaching and learning second language listening: Metacognition in action. New York, NY: Routledge.
Vanderplank, R. (2010). Déjà vu? A decade of research on language laboratories, television and video in language learning. Language Teaching, 43 (1), 1-37. https://doi.org/10.1017/S0261444809990267
Vidal, K. (2003). Academic listening: A source of vocabulary acquisition? Applied Linguistics, 24, 56-89.
Warschauer, M., & Healey, D. (1998). Computers and language learning: An overview. Language Teaching, 31, 57-71.
Williams, H., & Thorne, D. (2000). The value of Teletext subtitling as a medium for language learning. System, 28, 217-228.
Winitz, H. (Ed.). (1981). The comprehension approach to foreign language instruction. Rowley, Mass: Newbury House.
Yahya, S., Ahmad, E. A., & Abd Jalil, K. (2010). The definition and characteristics of ubiquitous learning: A discussion. International Journal of Education and Development using Information and Communication Technology, 6(1), 117-127.
Zarei, A. A., & Hashemipour, M. (2015). The effect of computer-assisted language instruction on improving EFL learners' autonomy and motivation. Journal of Applied Linguistics, 1(1), 40-58.
Appendix A
Comprehension Questions Let's Talk About Dying Peter Saul
1. About how many of Peter Saul's cells died while you watched this talk (assuming you watched it straight through without stopping)?
a) One million cells c) Two thousand cells
b) One hundred million cells d) It's a video of a talk, so technically none of his cells died
while I watched
2. According to the speaker, the death rate for males in Australia has...........................
a) halved b) doubled c) decreased d) increased
3. What surprised Peter Saul during his interaction with Jim and Kathleen?
a) Jim had such a severe case of pneumonia that he could not talk.
b) Kathleen wanted everything possible to be done to help Jim.
c) Jim and Kathleen had never discussed the possibility of Jim's becoming sick or dying.
d) Jim had lived to 49.
4. Which of the following is the most common of the four ways to go?
a) frailty b) sudden death c) organ failure d) terminal illness
5. As we know that we are all going to die, what is really important?
a) when we die b) how we die c) where we die d) all of the items
6. What was the overwhelming reaction to the respecting Patient Choices program at John Hunter Hospital?
a) Patients and their families were uncomfortable with the idea of discussing death.
b) Patients and their families appreciated the program and thought it should be normal hospital practice.
c) Patients and their families were so enthusiastic about the program that they continued the dialogue even after funding ran out.
d) Patients and their families voiced frustration at the hospital's inability to fulfill their wishes.
7. Longevity means...............................
a) more old age, not more youth c) more youth, not old age
b) more age, not more youth d) more young age, not more youth
8. What's the "big idea" concerning the end of life care Peter Saul discussed?
a) The ICU should develop better methods to help the increasingly frail live longer.
b) The medical industry should make faster advances in life-saving technologies.
c) Euthanasia should become a culturally-accepted and government-funded program.
d) The process of dying should be reclaimed from the current medicalized model.
9. The survey of nursing home residents in the Newcastle area demonstrated that..................of them had a plan
about what to do when their hearts stopped beating.
a) One in a hundred
b) Five hundred
c) Hundred
d) One million
10. What does he think about euthanasia?
a) It's a game b) It's a sideshow c) It's a hear beat d) It's dying process
^Questions 1,3, 4, 6, 8 are taken from www.ed.ted.com.
Appendix B
List and Order of Presentation of TED Talks
1. Global power shifts- Joseph Nye (part one)
2. Global power shifts- Joseph Nye (part two)
3. Mosquito, malaria, and education - Bill Gates
4. How Curiosity got us to Mars - Bobak Ferdowsi
5. Forget shopping. Soon you'll download your new clothes - Danit Peleg
6. What adults can learn from kids - Adora Svitak
7. Your body language shapes who you are - Amy Cuddy
8. How to control someone else's arm with your brain - Greg Gage
9. Making a car for blind drivers - Dennis Hong
10. Let's talk about dying - Peter Saul
