CC BY-ND
150
National Research University Higher School of Economics Journal of Language & Education Volume 6, Issue 2, 2020
Nikitina, L., Furuoka, F., & Kamaruddin, N. (2020). Language Attitudes and L2 Motivation of Korean Language Learners in Malaysia. Journal of Language and Education, 6(2), 132-146. https://doi.org/10.17323/jle.2020.10716
Language Attitudes and L2 Motivation of Korean Language Learners in Malaysia
Larisa Nikitina, Fumitaka Furuoka, Nurliana Kamaruddin
University of Malaya
Correspondence concerning this article should be addressed to Larisa Nikitina, Faculty of Languages and Linguistics, University of Malaya, 50603 Kuala Lumpur, Malaysia. Email: larisa.nikitina@gmail.com
This study examined relationships between language attitudes and L2 motivation of learners of Korean as a Foreign Language (KFL) in a large public university in Malaysia. It employed the socio-educational model of L2 motivation and focused on the relationship between the language learners' attitudes toward speakers of the target language and their motivation to learn Korean. A systematic statistical analysis was performed to analyse the data collected from 19 (N=19) students. A robust statistical procedure adopted in this study allowed some worthwhile insights into the language attitudes-L2 motivation nexus. The findings indicated that there existed a statistically significant relationship between the language learners' instrumental orientation and their attitudes toward the speakers of Korean language.
Keywords: language learning motivation, language attitudes, Korean as a Foreign Language, Malaysia
Introduction
The importance of psychological factors, such as attitudes and motivation, in the process of learning a second or a foreign language (L2 language) has been recognized since the 1940s (Gardner, 2001). Extensive and flourishing research on language learning motivation (L2 motivation) in the field of applied and educational linguistics stems from the pioneering studies by Robert Gardner and Wallace Lambert (1959, 1972), whose original question was whether "someone could really learn a second language if they did not like the group who spoke the language" (Gardner, 2001, p. 1). Consequent studies by Gardner and Lambert as well as by other researchers have firmly established the importance of language attitudes and variety of social factors that give rise to these attitudes (Houghton 2010; Nikitina, 2019). Over time, Gardner and Lambert's research on language attitudes and L2 motivation has evolved into the influential socio-educational theory of L2 motivation (Gardner, 1985).
The socio-psychological perspective introduced by Gardner and Lambert (1959, 1972) has shaped the vectors of L2 motivation research for several decades and the ensuing studies offer ample support to the existence of complex linkages between L2 motivation, social factors and personally held attitudes (Castellotti and Moore, 2002; Csizer and Kormos, 2008; Dornyei and Csizer, 2002; Dornyei, Csizer and Nemeth, 2006; Gardner, 1968; Kormos, Kiddle and Csizer, 2011; Wlodkowsky, 2008). Collectively, the social, attitudinal and emotional factors that are present in the process of learning an additional language are known as 'language attitudes'. Originally, research on such attitudes was limited to language learners' perceptions of the target language community and target language itself (Gardner, 1985). More recently, the concept of language attitudes has expanded to include stereotypes and mental images that language learners hold about the target language countries, their cultures and people (Nikitina, 2015, 2019, 2020).
It should be noted that despite the availability of newer theoretical models of L2 motivation (e.g., Dornyei, 2005; Ushioda, 2009) the Gardnerian socio-educational model and the concept of integrativeness have retained their theoretical and explanatory values, especially in the contexts where a distinct L2 community is present and easily identifiable (Gearing and Roger, 2019). In the socio-educational model of L2 motivation, language attitudes, including the attitudes toward the L2 speaking communities, are subsumed under the constructs of
Research Article
This article is published under the Creative Commons Attribution 4.0 International License.
'integrativeness' or 'integrative orientation'. 'Orientation' can be defined as a goal or clear internalized purpose for learning an L2 language, while the wider construct of L2 motivation incorporates the directed efforts to learn the L2 (Gardner, 1985; Gardner and MacIntyre, 1991, 1993). In the socio-educational model there are two motivational orientations, namely, the 'instrumental orientation' and the 'integrative orientation'. The former pertains to practical goals for learning an L2 language (e.g., having better employment opportunities, pursuing future studies, getting financial rewards) while the latter concerns genuine positive interest in and feelings toward an L2 community and its cultures (Gardner, 1985). Earlier studies have demonstrated that there is a strong correlation between the integrative orientation and language attitudes, which produced a direct impact on the L2 motivation (Gardner, 1985; Gardner and MacIntyre, 1991; Masgoret and Gardner, 2003).
Against such a backdrop, the current study aims to examine the relationship between Malaysian language learners' attitudes toward speakers of the Korean language and these learners' L2 motivation. Hence, this study addresses the problem at the root of the socio-educational theory, namely, language learners' attitudes toward members of the L2 speaking community and their L2 motivation. The research question this study seeks to answer is: What is the relationship between the language learners' attitudes toward speakers of the Korean language and their L2 motivation?
The current study hopes to contribute to the literature on L2 motivation in three ways. Firstly, from a theoretical perspective, it aims to offer empirical support to the existence of a linkage between language learners' attitudes toward speakers of a target language and their motivation to learn the target language. Gardner's (1985) socio-educational theory and the attitude theory by Eagly and Chaiken (1998) provided a theoretical foundation to the proposition that individually-held language attitudes, including the attitudes toward speakers of a target language, would have associations with L2 motivation.
Secondly, research on language attitudes among learners of the Korean language that incorporates the language learners' stereotypes of the target language country (e.g., Nikitina and Furuoka, 2019) is still in a nascent stage. By the same token, studies that have conducted systematic statistical analyses of L2 motivation and language attitudes in the context of teaching and learning the Korean language (e.g., Nikitina, 2020) remain scarce. The current study addresses this gap. This research direction could be particularly timely because the Korean language as a university subject is becoming increasingly popular all over the world. In Malaysia, for example, Korean language programs are offered in all major universities. Therefore, it would be worthwhile to examine whether there exists a linkage between language attitudes and L2 motivation among Malaysian tertiary learners of the Korean language.
Thirdly, this study hopes to make some methodological contributions. It adopts a robust statistical procedure that combines bootstrapping and quantile regression analysis (Nikitina and Furuoka, 2018; Nikitina, Paidi and Furuoka, 2019). In view that quantitative applied linguistics and L2 research lacks frameworks for a systematic implementation of non-parametric statistics, as noted by Norris, Ross and Schoonen (2015), and considering that statistical reasoning behind a chosen method is rarely explained, this article offers a step-by-step explanation of the phases in the statistical analysis. The importance of describing novel procedures in linguistics research in as much detail as possible so that other researchers can use or refine the method was noted by Clark (1973).
This article has the following structure: subsequent to this introductory part Section 2 gives an overview of the socio-cultural background to this study. Section 3 presents research studies on Korean cultural influence in Malaysia; it also reviews pertinent empirical studies on L2 motivation among learners of Korean. The section then proceeds to highlight some methodological issues in studies on L2 motivation and language attitudes. Section 4 describes the methodology adopted in the present study and Section 5 presents the empirical findings. Section 6 offers a discussion of the findings and some concluding remarks.
Background to the Study
Malaysia has long-standing good ties with both the Koreas. However, it was not until recent decades that Korean culture has begun making an impact on the Malaysian society. This can be attributed to the increasing
popularity of the Korean entertainment scene around the world, which has been known as the Korean Wave (Hallyu). The term Hallyu was first used in the late 1990s by the Chinese media to describe a growing popularity of Korean pop-culture in China (S. Lee, 2015). Although Hallyu has since become associated with the ubiquitous presence of all things Korean, including Korean food and Korean goods, the term is still predominantly used to refer to the global spread of Korean pop culture.
The utility of the Korean Wave has been leveraged by the South Korean government as part of its public diplomacy efforts and as a means of soft power. The role of the nation-state, in this case the government, in crafting cultural policies specifically tailored to enhance the influence and attractiveness of Korean culture has been important in promoting the popularity of Korean culture abroad (Jin, 2014). The global prominence of Korean pop culture is reflected in the mental images that people around the world associate with Korea (Nikitina and Furuoka, 2019). According to the survey "Top 10 Images Associated with Korea", the second most popular image after "Korean food" was "Korean pop music and artists" (K-pop) (Korea Foundation, 2019).
In Malaysia, Korean popular culture used to be secondary to the hugely influential Japanese pop-culture and entertainment industry. For example, Korean music artists such as BoA and SES became popular due to their foray into the Japanese music industry. Moreover, the Korean government's cultural policy was focused on Malaysia's Chinese speaking population and the wider audience in this multicultural country was not the main target. It was not until the release in 2002 of the Korean drama Winter Sonata that the Korean Wave acquired a tangible presence in Malaysia and began cutting across the racial lines of the country's population. The success of this TV drama was followed with other hits, such as Autumn in My Heart and A Jewel in the Palace, all of which contributed to an ever increasing popularity of Korean dramas among the Malaysian audience.
Following this initial success in Asia in the late 1990s and the early 2000s, the seemingly sudden boom of South Korean pop-culture beyond the Asian region was enabled by the spread of the social media. In 2008, South Korean singer Psy became a global sensation with his music video Gangnam Style. The video swiftly became one of the most watched videos on YouTube, solidifying the global appeal and presence of Korean entertainment online. As noted by Jin and Yoon (2014), this swelling of the Korean Wave or Hallyu 2.0 has contributed to the second wave of Korean culture popularity and transformed what was previously seen as a niche interest to the mainstream acceptance. Apparently, the popularity of Korean pop-culture in Malaysia and the Southeast Asian region in general has stimulated interest toward all things Korean. This includes the growing consumption of Korean products, the inbound tourism to Korea and the introduction of Korean language courses in universities. Notwithstanding an immense popularity of Korean culture in Malaysia, no systematic empirical research has been done on Korean language learners' attitudes toward the target language country, its cultures and people. Therefore, it could be illuminating to examine whether popularity of Korean culture in Malaysia is reflected in the positive language attitudes and strong L2 motivation among Malaysian tertiary learners of Korean as a Foreign Language (KFL).
Literature Review
Empirical Studies on Korean Cultural Presence in Malaysia
Several studies have explored whether and how popularity of Korean pop-culture in Malaysia has influenced various aspects of social life and cultural mores in the country. Cho (2010) argued that the immense popularity of Korean culture among Malaysians from all walks of life translated, among other things, into the proliferation of Korean language courses and culture study programs in the country's institutions of higher learning. Findings reported by Teh and Goh (2014) resonate with this proposition. The researchers examined the influence of watching South Korean TV dramas on outbound Malaysian tourism to South Korea. They discovered that younger Malaysians, especially young women, were keen to visit Korea due to their interest in Korean TV dramas, pop music and entertainment programs.
In another study among Malaysian youths, Nikitina and Furuoka (2019) explored mental images of Korea, its cultures and people that tertiary learners of KFL bring into the language classroom. The findings revealed that the students had rich and diversified images of the target language country. Interestingly, almost half of all mental images (N=333) provided by the respondents pertained to Korean popular culture, various cultural
products and Korean food. In contrast, the images referring to Korean history, economy, political life and technological advancements were scarce, while the images concerning Korean 'high culture' or 'big C' culture were noticeably lacking. It should be noted that the respondents generated a substantial number of very positive references to Korean people and mentioned popular in Malaysia artists and entertainers. Comparing these findings with previous studies (e.g., Byon, 2007; Tangalycheva, 2015) Nikitina and Furuoka concluded that some images of Korea emerged as transnational 'mega stereotypes' shared by people in various countries. As an example, in each of the three studies the respondents made several very similar references to Korean people and their character traits, including 'group-oriented people' and 'hard working people'. Also, Nikitina and Furuoka's findings indicated that the students had overwhelmingly positive mental images of and attitudes toward Korea and its people.
It could be plausible to suggest that having such positive attitudes or being a fan of a pop singer, of a movie actor or even of a particular TV program would establish some positively charged emotional links to the target language culture and L2 speaking community as a whole. Such emotional attachments are at the core of the integrative orientation. As pointed out by Cho (2010), Korean entertainment industry has crafted a highly favourable image of Korea and its people amongst Malaysians. By extension, as we propose, interest in Korean TV dramas and pop-culture would not only enhance the language learners' familiarity with and interest in Korea and its cultures but also stimulate their desire to master the Korean language, so that they could understand conversations in their favourite TV dramas or follow the lyrics of their favourite songs. The present empirical study among learners of KFL aims to gain some insights into the plausibility of this tentative proposition.
Studies on L2 Motivation among Learners of Korean
Of particular interest for the current study is the prior research, either qualitative or quantitative, that included non-heritage learners of Korean and adopted a Gardnerian perspective on L2 motivation (i.e., subsumed the instrumental and integrative orientations among the research concepts). The instrumental and integrative orientations are considered in this review of literature in the widest sense: the former pertains to clearly internalized practical purposes for learning Korean while the latter concerns language learners' interest in the target culture, their attitudes toward the target language (TL) country and their willingness and interest to interact with members of the TL community.
One of the earliest available studies on L2 motivation among tertiary learners of the Korean language was conducted by Yang (2003). In that study, a larger sample (N=341) of students learning one of three East Asian languages (i.e., Chinese, Japanese or Korean) in USA universities included 135 (n = 135) learners of Korean, of which 26 percent were non-heritage learners. Yang conducted a questionnaire survey and analyzed the data using a number of statistical tests. The research instrument consisted of several subscales that assessed the nature of the students' L2 motivation, such as the integrative, instrumental, heritage-related, travel, interest and school-related reasons for learning Korean. The findings indicated that the learners of Korean were, overall, more strongly motivated compared to the learners of Chinese and Japanese languages. Also, the non-heritage Korean language learners were highly motivated by their interest in the target language and the integrative motivation was the dominant driving force for these students.
In recent years there has been a drastic increase in studies that focus on learners of Korean as a Foreign Language (KFL). These studies were done in a broad range of geographical contexts including Australia (Fraschini and Caruso, 2019; Shin, Ko and Rue, 2016), China (Z. N. Lee, 2015), Japan (S.Y. Lee, 2018; Yoshida et al., 2014), Malaysia (Nikitina, 2020), Singapore (Chan and Chi, 2010), South Korea (Gearing and Roger, 2019; Son and Jeon, 2011) and the USA (Hur and Choi, 2015; Jee, 2015; I. H. Lee, 2018). Theoretical perspectives adopted by the researchers included Gardner's (1985) socio-educational model of L2 motivation that distinguishes between integrative and instrumental orientations (e.g., Hur and Choi, 2015; S. Y. Lee, 2018, Nikitina, 2020; Yang, 2003), the L2 motivational self-system (L2MSS) proposed by Dornyei (2009) that mainly focuses on the language learners' vision of their future L2 selves (e.g., Fraschini and Caruso, 2019) or a combination of theoretical frameworks (e.g., Gearing and Roger, 2018).
Studies done in various countries have distinguished integrativeness as a ubiquitous and prominent feature within the Korean language learners' L2 motivation. Especially, the interest in Korean popular culture has been
a key determinant of the students' choice to enroll in the Korean language courses and a critical factor that maintained their L2 motivation. To give some examples, S. Y. Lee (2018), who conducted a study among Japanese learners of Korean, discovered that the students were predominantly motivated by their interest in Korean culture and had a strong intention to continue their Korean language program. American non-heritage Korean language learners in a study by Hur and Choi (2015) were overwhelmingly motivated by the integrative factors, such as the interest to "learn more about Korean culture/people/ history" (mentioned by 78% of these respondents) and by the desire to communicate with members of the target language speaking community (67%). In a similar way, I.H. Lee (2018) found out that indirect contacts with Korean popular culture had enhanced the American students' L2 motivation to learn the Korean language.
The salience of the integrative orientation does not preclude a tangible presence of the instrumentally-oriented motives. For example, Jee (2015) found out that American college students who participated in her study were in equal degrees motivated by instrumental (e.g., "studying Korean is important to me because I'll need it for my future career") and integrative (e.g., "studying Korean is important to me because it will allow me to be more at ease with fellow Koreans who speak Korean") motives. Furthermore, non-heritage learners in Hur and Choi's (2015) study stated among reasons to learn Korean the need to fulfil the university foreign language requirement (35%) and raise their GPA (13%); some respondents intended to use Korean in future employment (33%) or had plans to pursue further studies in Korea (33%). In a similar way, Chan and Chi (2010), who explored the structure of L2 motivation among learners of KFL (N = 80) in a Singapore university concluded that, based on the findings from the exploratory factor analysis (EFA), the main factors that motivated the students were "pop culture," "career," "achievement," "academic exchange" and "foreign languages and cultures".
Interesting insights into L2 motivation of Korean language learners can be gained from a O-study by Fraschini and Caruso (2019). The researchers discovered that the majority (n = 17) of their 39 (N = 39) respondents could form a group of learners who visualized their future selves using Korean language for enjoyment and leisure (e.g., consuming cultural products, having short sojourns in Korea). Also, a considerable number of the respondents (n = 11) envisioned employing their Korean language skills for various instrumental purposes (e.g., making short business trips to Korea, working as a language instructor, being employed by a Korean company). Moreover, another four (n = 4) respondents had very clear instrumentally-oriented motives; these students considered the knowledge of the Korean language as important for obtaining a high status job in the future, for acquiring a greater social respect or for gaining a competitive edge in the international job market. The O methodology employed by Fraschini and Caruso has highlighted the fact that L2 motivation is an amalgam of various motives within each individual student's language learning agenda.
The missing elements in the earlier studies of L2 motivation were the endogenous mental images that language learners have of target language countries and the attitudes embedded in these images (see Nikitina, 2019). It is only very recently that researchers have begun exploring Korean language learners' stereotypes of and attitudes toward Korea (Nikitina, 2020; Nikitina and Furuoka, 2019). Nikitina (2020) included learners of Korean in her study of relationships between country stereotypes and L2 motivation among tertiary learners of Asian languages in a Malaysian university. The findings were somewhat counter-intuitive: although the students held overwhelmingly positive mental images of and attitudes toward Korea, no statistically significant positive relationship was detected to exist between the learners' attitudes and any of the L2 motivation dimensions (i.e., "effort", "perseverance", "instrumental orientation", "integrative orientation"). This result might be due to some methodological constraints, which the present study addressed. To be more specific, firstly, this study employed a global measure of attitudes known in psychology research as a 'thermometer-type scale' and, secondly, it applied a newly-introduced robust statistical procedure (Nikitina and Furuoka, 2018) that is suitable for small sample research. The following subsection touches upon some pertinent methodological issues.
Methodological Issues in Research on Language Learners' Attitudes and L2 Motivation
Methodologists and applied linguistics researchers have noted that classroom-based research in L2 settings tends to have a small sample size, which leads to the problem of non-normal distribution of error terms (Larson-Hall, 2012; Larson-Hall and Herrington, 2010). One of the most straightforward and popular approaches to dealing with this problem is bootstrapping (Stuckler et al., 2009). In educational and applied linguistics research the bootstrapping method is often employed to enable the structural equation model analysis (e.g., Gallagher, 2013; Gu and Cheung, 2016; Hessel, 2015). For example, in her study of German university students'
ideal L2 self-image as a speaker of English, Hessel (2015) used the bootstrapping procedure to compute the confidence intervals. In a similar way, Gu and Cheung (2016) applied bootstrapping to estimate the confidence intervals in the links between ethnic minorities schoolchildren's views of an ideal L2 self, their acculturation into the host community, the influence of the heritage culture and the childrens' intended effort to learn the mainstream language. The bootstrapping procedure was also used by Gallagher (2013) who examined the relationship between willingness to communicate in the second language (L2WTC) among international students in the UK and their cross-cultural adaptation.
However, the bootstrapping method is not a panacea for all methodological hurdles faced by researchers. One of its main limitations is the tendency to proliferate outliers (Bai et al., 2016). This means that a proportion of the outlying cases in the bootstrapped data could be greater than in the original dataset (Salibian-Barrera and Zamar, 2002). To overcome this shortcoming Nikitina and Furuoka (2018) suggested combining bootstrapping with quantile regression (OR) analysis. In the OR analysis, the median value or the 0.5-quantile value is often selected for the conditional variable. In the presence of outliers, a quantile would be preferable for the conditional variable because the percentile is an order-statistic that more robust to outliers (Chen and Chalhoub-Deville, 2014). The ensuing section explains the current study's methodology.
Methodology
Scope of the Study, Participants and Research Instrument
This study is a part of a larger investigation of the relationships between language attitudes and L2 motivation among learners of East Asian languages. Due to space constraints, the qualitative findings on the students' mental images of Korea are not discussed in the current article. The article reports only the quantitative analysis results. The data were collected among KFL learners (N = 19) at a large Malaysian university. The age of the participants ranged between 20 and 24 years old (M = 21.79; Mdn = 22.00; SD = 1.08). There were more female (n = 15 or 78.9%) than male students in the sample. All of the participants majored in the Social Sciences and Humanities disciplines.
Each participant was given a photocopied form of the study's questionnaire which contained one open-ended question, a thermometer-type scale ranging from 0 °C to 100 C that assessed the students' general attitudes toward the TL speakers1 and 16 items on L2 motivation with attached 5-point Likert-type scales ranging from "strongly disagree" to "strongly agree". Also, the students were asked to provide demographic information, such as their gender, age and academic major.
Study's Variables and Research Hypothesis
There were four variables in this study, namely: (1) language learners' attitudes toward the TL speakers (ALS), (2) general L2 motivation (GEM), (3) instrumental orientation (ISO) and (4) integrative orientation (ITO). Based on the definition of an attitude by Eagly and Chaiken (1998), this study operationalized language attitudes as a general evaluative reaction of the language learners toward speakers of the target language. These attitudes were measure on the thermometer-type scale.
Among the L2 motivation variables, General motivation (GEM) measured the effort that the students were willing to expend to learn the TL (6 items; Cronbach's a = .816). The Instrumental orientation (ISO) assessed the language learners' perceptions of the TL utility and their intention to use the TL for pragmatic purposes (5 items; Cronbach's a = .785) while the Integrative orientation (ITO) measured the language learners' intention to learn the TL in order to communicate with the TL speakers and understand their ways of life (5 items; Cronbach's a = .819).
Based on the premise that "motivation binds emotion to action" (Wlodkowsky, 2008, p.2), this study assumed that the students' attitudes toward speakers of Korean (ALS) would relate to the three L2 motivation variables, namely, GEM, ISO, and ITO. In a function form, this relationship can be expressed as:
1 Thermometer-type scales are widely employed in the behavioural sciences research to assess general attitudes held by people (Maio and Haddock, 2010; Spencer-Rodgers, 2001). In the current study, the end points of the scale were graded as 0 °C (for an "extremely unfavourable" attitude) and 100 °C (for an "extremely favourable" attitude); 5 °C intervals were set between the two extreme points of the scale.
ALS = f (GEM, ISO, ITO) (1)
This function proposes that the language learners' attitudes toward the TL speakers (ALS) would have a positive (+) relationship with the L2 motivation measures (i.e., GEM, ISO, and ITO). In other words, this function proposes that more motivated language learners would have more positive attitudes toward the TL speakers.
In quantitative applied linguistics research, the hypothesis testing usually entails assessing the probability value (p-value). However, as noted by researchers and methodologists (Cumming, 2012; Larson-Hall, 2012), the p-value endorses a false dichotomy between 'statistically significant' and 'statistically non-significant' results. Besides, due to the over-reliance on the p-value little attention has been accorded to the distribution of the population parameters. To overcome this problem, the current study reports the confidence intervals (CIs) rather than the p-values. Besides, the findings reported as CIs are more informative compared to p-values (Cumming, 2012). In the current study, the alpha level was set at 10 percent (a = .10) and the null hypothesis was rejected when zero fell outside the 90 percent CI. Though setting the alpha level at 5 percent (a = .05) is the prevalent practice, there are compelling arguments in favour of increasing it to 10 percent in the Social Sciences and Humanities research (Larson-Hall, 2010).
Data analysis
A robust statistical procedure in the current study observed the sequence of steps proposed by Nikitina and Furuoka (2018) and Nikitina et al. (2019), namely: (1) the ordinary least squares (OLS) method examined the relationship between the variables; (2) the normality test (e.g., Jarque-Bera test) assessed normality of the error terms; (3) the bootstrap method estimated the standard errors in the OLS analysis; (4) the influence statistic (e.g. Hat Matrix test) were used to detect the outliers;2 (5) the quantile regression (OR) examined the relationships between the dependent and independent variables; (6) the bootstrap method estimated standard errors in the OS analysis. Various statistical software packages, such as EViews, Stata, Matlab and R allow conducting the bootstrap analysis and implementing the OR procedure. Statistical analyses in the present study for the most part were implemented with the Ox program. The Jarque-Bera test and the Hat Matrix test were performed using EViews.
It should be noted that the choice and sequence of analytical steps in a study that performs statistical analyses would be based on the results of the prior normality tests and influence statistics. However, in order to enhance the awareness for the need of a proper procedure, the initial OLS analysis in this study was performed without examining the assumptions for the feasibility of this statistical procedure.
Empirical Findings
Findings from the first step of the analysis—the Ordinary Least Squares (OLS) method—are reported in Table 1. The results show that the slope coefficient for the variable "General motivation" was -0.276. The 90 percent CI was [-15.416, 14.863], the 95 percent CI was [-18.681, 18.128] and the 99 percent CI was [-25.728, 25.175]. Based on these findings, the null hypothesis of zero coefficient could not be rejected because all three CIs included zero. In the case of the "Instrumental orientation" variable, the slope coefficient was 15.816, the 90 percent CI was [-3.291, 35.013], the 95 percent CI was [-7.421, 39.143] and the 99 percent CI was [-16.336, 48.058]. Since zero was included in all CIs, the null hypothesis could not be rejected for all three significance levels. As to the "Integrative orientation" variable, the slope coefficient was 11.892, the 90 percent CI was [-6.403, 30.188], the 95 percent CI was [-10.848, 34.173] and the 99 percent CI was [-18.864, 42.649]. These findings indicated that the null hypothesis could not be rejected for all three significance levels. In other words, the OLS analysis did not detect a statistically significant linkage between the learners' general attitudes toward speakers of the Korean language, their general L2 motivation, the instrumental orientation and the integrative orientation.
2 The Hat Matrix test was employed due to its ability to give a visual representation of the findings on the outliers. Visual displays are highly recommended for reporting quantitative findings in applied linguistics and L2 research (Larson-Hall and Plonsky, 2015) as they enable researchers to make better-informed conclusions concerning the presence of outliers.
The absence of the language attitudes-L2 motivation linkage is not only counter-intuitive but it is also at odds with the influential socio-educational model of L2 motivation (Gardner, 1985).
Table 1
Findings from the OLS analysis (Dependent variable: Attitudes toward the TL speakers [ALS]), Number of observations = 19)
Coefficient Standard Error t-ratio Confidence Intervals (CI)
GEM -0.276 8.636 -0.032 90 percent CI
(General motivation) [-15.416, 14.863]
95 percent CI [-18.681, 18.128]
99 percent CI [-25.728, 25.175]
ISO 15.861 10.925 1.451 90 percent CI
(Instrumental orientation) [-3.291, 35.013]
95 percent CI [-7.421, 39.143] 99 percent CI [-16.326, 48.058]
ITO 11.892 10.436 1.139 90 percent CI
(Integrative orientation) [-6.403, 30.188]
95 percent CI [-10.348, 34.173]
99 percent CI [-18.864, 42.649]
Constant -44.602 34.246 -1.136 90 percent CI
[-113.400, 24.196]
95 percent CI [-128.238, 39.031]
99 percent CI [-160.260, 71.056]
However, as stated in Subsection 4.3, the OLS analysis was performed without a prior check of the assumptions for the feasibility of this statistical procedure. In other words, the OLS method should not be used if these assumptions are violated. A lack of information concerning assumptions for a statistical test is a tenacious feature of quantitative L2 research (see Cunnings, 2012). Therefore, in the next step, the residual distribution from the OLS analysis was assessed. Figure 1 depicts the histogram of the estimated residuals and the normal distribution plot.
A visual inspection of the figure reveals that the residual distribution was negatively skewed and peaked around zero. Besides, according to the findings from the Jarque-Bera (JB) test (see Table 2), the mean value was 0.001 and the median value was 2.354. The fact that the median was greater than the mean indicates that the residuals were negatively skewed. The skewness value of -1.255 reveals that the tail on the left side of the probability function was longer. The kurtosis value of 4.847 indicates that the probability density function had a higher-than-normal peak (i.e., kurtosis > 3). In other words, the distribution was leptokurtic.
In addition, the Jarque-Bera statistic was 7.696 (see Table 2). Based on these results, the null hypothesis of normal distribution was rejected. The non-normal distribution of error terms violated the basic assumption for regression analysis. Therefore, the findings from the OLS test performed earlier in this study could not be considered reliable. This is because the statistical estimations of CIs must be based on the normality assumption for the error terms. In sum, the findings from the residual distribution analysis indicated that the OLS analysis was not an appropriate procedure because the error terms in the OLS analysis were not normally distributed.
Due to the violation of the normality assumption, the bootstrap method was performed in the next step to estimate the standard errors. Table 3 shows the findings from the OLS analysis with bootstrapped standard errors. The bootstrapped standard error for the variable "General motivation" was 11.502, the 90 percent CI was [-20.440, 19.888], the 95 percent CI was [-24.788, 24.236] and the 99 percent CI was [-34.175, 33.622].
Figure 1
Residual distribution and normal distribution plot
Table 2
Findings from the normality analysis
Mean Median
Standard Deviation
0.001 2.354 19.106
Skewness Kurtosis
Jarque-Bera statistic
-1.255 7.696 7.696**
Note: ** indicates significance at the 5 percent level\
Table 3
Findings from bootstrap OLS analysis (Dependent variable: Attitudes toward the TL speakers [ALS], Number of observations = 19)
Coefficient Standard Error
t-ratio
Confidence Intervals (CI)
GEM
(General motivation)
ISO
(Instrumental orientation)
ITO
(Integrative orientation)
Constant
-0.276
11.502
-0.024
15.861
11.863
1.343
11.892
14.099
0.843
-44.602
42.730
-1.043
90 percent CI [-20.440, 19.888]
95 percent CI [-24.788, 24.236]
99 percent CI [-34.175, 33.622]
90 percent CI [-4.830, 36.552] 95 percent CI [-9.292, 41.014] 99 percent CI [-18.923, 50.645]
90 percent CI [-12.823, 36.604]
95 percent CI [-18.135, 41.438]
99 percent CI [-29.658, 53.443]
90 percent CI -119.508, 30.303]
95 percent CI -135.660, 46.455]
99 percent CI -170.527, 81.723]
Based on the findings presented in Table 3, the null hypothesis of zero coefficient for the first independent variable could not be rejected for all three significance levels. For the "Instrumental orientation" variable, the bootstrapped standard error was 11.863, the 90 percent CI was [-4.830, 36.552], the 95 percent CI was [-9.292, 41.014] and the 99 percent CI was [-18.923, 50.645]. Again, since zero was included in each CI, the null hypothesis
could not be rejected for all three significance levels. Finally, for the "Integrative orientation" variable, the bootstrapped standard error was 14.699, the 90 percent CI was [-12.823, 36.609], the 95 percent CI was [-18.135, 41.438] and the 99 percent CI was [-29.658, 53.443]. In a similar way, the null hypothesis could not be rejected for all three significance levels. This means that the bootstrapped OLS analysis failed to detect statistically significant linkages between the language learners' attitudes toward speakers of the target language, their general L2 motivation, the instrumental orientation and the integrative orientation.
Considering that the bootstrap method has a tendency to proliferate outliers (Bai et al., 2016; Salibian-Barrera and Zamar 2002), we performed the Hat Matrix test in order to check the presence of outlying cases. As Figure 2 shows, the dataset had one prominent outlier (case #9). Based on this finding, the results from the OLS analysis with bootstrapped standard errors in the preceding step could not be considered reliable.
Figure 2
Findings on outliers from the influence statistics
Influence Statistics
Hat Matrix
■8-i-
2 r~4~n '"V ^kT ^T TT 18 '
Therefore, the study proceeded with the OR analysis. The median value or the 0.5-quantile value was chosen for the conditional variable. Table 4 that reports the OR results shows that the slope coefficient for the variable "General motivation" was -4.051, the 90 percent CI was [-16.827, 8.723], the 95 percent CI was [-19.582, 11.478] and the 99 percent CI was [-25.529, 17.425]. Hence, the null hypothesis of zero coefficient for the first independent variable "people" could not be rejected. In the case of the variable "Instrumental orientation", the slope coefficient was 30.237, the 90 percent CI was [14.075, 46.398], the 95 percent CI was [10.590, 49.883] and the 99 percent CI was [3.067, 57.406]. Based on these results, the null hypothesis could be rejected at the 1 percent level of significance because all three CIs did not include zero. As to the "Integrative orientation" variable, the slope coefficient was 5.402, the 90 percent CI was [-10.035, 20.841], the 95 percent CI was [-13.364, 24.170] and the 99 percent CI was [-20.557, 31.356]. These findings indicated that the null hypothesis could not be rejected for all three significance levels. In short, the OR analysis was able to detect a statistically significant linkage between the language learners' attitudes toward speakers of the target language and the instrumental orientation.
In the final step, the bootstrap method estimated standard errors in the OR regression. This 'integrated' approach that combines the OR analysis and bootstrap procedure is robust against a non-normal distribution of error terms and outliers. The findings are reported in Table 5. As can be seen from the table, the bootstrapped standard error for the variable "General motivation" was 13.586, the 90 percent CI was [-27.851, 19.747], the 95 percent CI was [-32.983, 24.879] and the 99 percent CI was [-44.062, 35.958]. Based on these results, the null hypothesis of zero coefficient for the first independent variable could not be rejected for all three significance levels. Next, the bootstrapped standard error for the variable "Instrumental orientation" was 15.626, the 90 percent CI was [2.844, 57.629], the 95 percent CI was [-3.062, 65.536] and the 99 percent CI was [-15.813, 76.787]. Therefore, the null hypothesis could be rejected at the 10 percent level of significance because zero was not included only in the 90 percent CI. In the case of the "Integrative orientation", the bootstrapped standard error was 16.993, the 90 percent CI was [-24.887, 25.192], the 95 percent CI was [-30.810, 41.616] and the 99 percent CI was [-44.678, 55.483].
Table 4
Findings from quantile regression (QR) analysis (Dependent variable: Attitudes toward the TL speakers [ALS], Number of observations = 19)
Coefficient Standard Error t-ratio Confidence Intervals (CI)
GEM -4.051 7.287 -0.555 90 percent CI
(General motivation) [-16.827, 8.723]
95 percent CI
[-19.582, 11.478]
99 percent CI
[-25.529, 17.425]
ISO 30.237*** 9.219 3.279 90 percent CI
(Instrumental orientation) [14.075, 46.398]
95 percent CI
[10.590, 49.883]
99 percent CI
[3.067, 57.406]
ITO 5.402 8.806 0.613 90 percent CI
(Integrative orientation) [-10.035, 20.841]
95 percent CI
[-13.364, 24.170]
99 percent CI
[-20.557, 31.356]
Constant -59.288* 33.117 -1.790 90 percent CI
[-117.343, -1.234]
95 percent CI
[-129.861, 11.283]
99 percent CI
[-156.884, 38.306]
Notes: *** indicates statistical significance at the 1 percent level, * indicates statistical significance at the 10 percent level.
Table 5
Findings from bootstrap quantile regression analysis (Dependent variable: Attitudes toward the TL speakers [ALS],
Number of observations = 19)
Coefficient Standard Error t-ratio Confidence Intervals (CI)
GEM -4.051 13.586 -0.298 90 percent CI
(General motivation) [-27.851, 19.747]
95 percent CI
[-32.983, 24.879]
99 percent CI
[-44.062, 35.958]
ISO 30.237* 15.626 1.995 90 percent CI
(Instrumental orientation) [2.844, 57.629]
95 percent CI
[-3.062, 65.536]
99 percent CI
[-15.863, 76.787]
ITO 5.402 16.993 0.317 90 percent CI
(Integrative orientation) [-24.887, 25.192]
95 percent CI
[-30.810, 41.616]
99 percent CI
[-44.678, 55.483]
Constant -59.288 58.673 -1.010 90 percent CI
[-162.143, 43.565]
95 percent CI
[-184.332, 68.740]
99 percent CI
[-232.199, 113.626]
Notes: * indicates statistical significance at the 10 percent level
Based on the results presented in Table 5, the null hypothesis could not be rejected for all three significance levels. Thus, the bootstrapped OR analysis confirmed the findings from the OR analysis of the existence of a
statistically significant linkage between the language learners' attitudes toward speakers of Korean language and instrumental orientation.
In sum, at the initial stage of the analysis the findings from the OLS regression—a 'classical' statistical test that was performed without checking the assumptions—failed to detect a statistically significant relationship between the language attitudes, the integrative orientation and the general L2 motivation of the KFL learners. However, a properly implemented robust statistical procedure was able to detect a statistically significant relationship between the students' attitude toward the TL speakers and their instrumental orientation to learn the target language.
Discussion and Conclusion
Considering the immense popularity that South Korea, its popular culture and cultural products enjoy among Malaysians from all walks of life this study has proposed that these positive perceptions and attitudes might have some influence on the patterns of L2 motivation of Malaysian learners of KFL. The study proceeded to employ a robust statistical procedure to examine the language attitudes-L2 motivation nexus using the data collected among learners of the Korean language in a large public university in Malaysia. The focus was on the students' attitudes toward speakers of the target language. The findings indicated that there existed a statistically significant relationship between the language learners' attitudes toward the target language speakers and their instrumental orientation.
In other words, despite the positive attitudes among Malaysian youths toward Korea and its people and notwithstanding the fascination with Korean pop singers, actors and popular culture as a whole (see Cho, 2010; Nikitina and Furuoka, 2019; Teh and Goh, 2014), these social factors did not translate in the present study into a statistically significant relationship between the language learners' attitudes toward speakers of the target language and their integrative orientation to learn Korean. This finding does not align with the previous studies where the integrative orientation was found to be the main driving force for learning Korean among university students in various countries (Hur and Choi, 2015; I.H. Lee, 2018; S. Y. Lee, 2018; Yang, 2003; Yoshida et al., 2014). Rather, this result is in line with a study among Malaysian tertiary learners of Korean conducted by Nikitina (2020) that examined links between stereotypes of Korea held by the students and their L2 motivation. The researcher did not detect any statistically significant relationship between the variables. However, unlike the study by Nikitina, the current research included a specific measure of the attitudes toward speakers of the target language. As a result, the language learners' attitudes were found to have a statistical relationship with one of the aspects of L2 motivation - the instrumental orientation.
The findings reported in this study resonate with the results in a study by Chan and Chi (2010) who conducted their research among university students in Singapore. The researchers concluded that though the popularity of Korean pop culture has generated a strong interest in Korean language learning among Singaporeans, the "career" dimension was a prominent factor to motivate Singapore undergraduate students to learn Korean. In addition, the instrumentally-oriented goals were clearly prominent among a group of students in studies by Fraschini and Caruso (2019) and Jee (2015). Taken together, these findings indicate that practical considerations rather than purely cultural interest can serve as an important motivational force for students learning Korean in different cultural and educational settings. One possible explanation could be South Korea's international status of an advanced nation. In other words, besides being culturally attractive, South Korea as an economically and technologically developed country could be linked in the minds of the language learners with good job prospects and elevated social status in the future (see Fraschini and Caruso, 2019).
The current study was done among Malaysian learners of Korean as a Foreign Language. Due to a growing popularity of Korean language programs in universities around the world language educators and applied linguists might want to further expand the geographical coverage of studies on L2 motivation among learners of Korean. Such studies are still scarce compared to the body of scholarly literature on the learners of major European languages. Also, researchers might want to explore in greater depth the language-related attitudes that learners of Korean bring into the classroom and examine whether and to what extent these attitudes impact the students' L2 motivation. It is hoped that a robust statistical procedure adopted in this study, and the findings reported here, would be useful for future research studies.
Conflict of Interest
The authors declare that they have no conflict of interest.
References
Bai, H., Sivo, S.A., Pan, W., & Fan, X. (2016). Application of a new resampling method to SEM: a comparison of S-SMART with the bootstrap. International Journal of Research & Method in Education, 39(2), 194-207. https://doi.org/10.1080/1743727X.2015.1056135 Byon, A. S. (2007). The use of culture portfolio project in a Korean culture classroom: Evaluating stereotypes and enhancing cross-cultural awareness. Language, Culture andCurriculum, 20(1), 1-19. https://doi.org/10.2167/ lcc323.0
Castellotti, V., & Moore, D. (2002). Social representations of languages and teaching: Guide for the development of
language education policies in Europe from linguistic diversity to plurilingual education. Council of Europe. Chan, W. M., & Chi, S. W. (2010). A study of the learning goals of university students of Korean as a foreign
language. Electronic Journal of Foreign Language Teaching, 7(1), 125-140. Chen, F., & Chalhoub-Deville, M. (2014). Principles of quantile regression and an application. Language Testing,
31(1), 63-87. https://doi.org/10.1177/0265532213493623 Cho, C. H. (2010). Korean wave in Malaysia and changes of the Korea-Malaysia relations. Malaysian Journal of Media Studies, 12(1), 1-14.
Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological
research. Journal of Verbal Learning and Verbal Behavior, 12(4), 335-359. Csizér, K., & Kormos, J. (2008). The relationship of intercultural contact and language learning motivation among Hungarian students of English and German. Journal of Multilingual and Multicultural Development, 29(1), 30-48. https://doi.org/10.2167/jmmd557.0 Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. Routledge.
Cunnings, I. (2012). An overview of mixed-effects statistical models for second language researchers. Second
Language Research, 28(3), 369-382. https://doi.org/10.1177/ 0267658312443651 Dornyei, Z. (2005). The psychology of the language learner. Lawrence Erlbaum.
Dornyei, Z. (2009). The L2 motivational self-system. In Z. Dornyei & E. Ushioda (Eds.), Motivation, language
identity and the L2 self (pp. 9-42). Multilingual Matters. Dornyei, Z., & Csizér, K. (2002). Some dynamics of language attitudes and motivation: Results of a longitudinal
nationwide survey. Applied Linguistics, 23(4), 421-462. https://doi.org/10.1093/applin/23.4.421 Dornyei, Z., Csizér, K. & Németh, N. (2006) Motivation, language attitudes and globalisation: A Hungarian
perspective. Multilingual Matters. Eagly, A. H., & Chaiken, S. (1998). Attitude structure and function. In D.T. Gilbert, S.T. Fiske & G.r Lindzey (Eds.),
The handbook of social psychology (vol. 1, 4th ed., (pp. 269-322). McGraw-Hill. Fraschini, N., & Caruso, M. (2019). "I can see myself..." A O methodology study on self vision of Korean language
learners. System, 87, 102-147. https://doi.org/10.1016/j.system. 2019.102147 Hur, S.R., & Choi Y.-H. (2015). Lasting effects of the Korean community schools on American college students' Korean language proficiency, motivation, and attitudes. Journal of Korean Language Education, 26(3), 287318. https://doi.org/10.18209/iakle.2015.26.3.287 Gallagher, H.C. (2013). Willingness to communicate and cross-cultural adaptation: L2 communication and acculturative stress as transaction. Applied Linguistics, 34(1), 53-73. https://doi.org/10.1093/applin/ams023 Gardner, R.C. (1968). Attitudes and motivation: Their role in second-language acquisition. TESOL Quarterly,
2(3), 141-150. http://www.jstor.org/stable/3585571 Gardner, R. C. (1985). Social Psychology and second language learning: The role of attitudes and motivation. Edward Arnold.
Gardner, R. C. (2001). Integrative motivation: Past, present and future. http://publish.uwo.ca/~gardner/docs/ GardnerPublicLecture1.pdf
Gardner, R. C., & Lambert, W. E. (1959). Motivational variables in second-language acquisition. Canadian Journal of Psychology/ Revue Canadienne de Psychologie, 13(4), 266-272.
Gardner, R. C., & Lambert, W. E. (1972). Attitudes and motivation in second-language learning. Newbury House Publishers.
Gardner, R.C. & MacIntyre, P.D. (1991). An instrumental motivation in language study: Who says it isn't
effective? Studies in Second Language Acquisition, 4(13), 57-72. https://www.jstor.org/stable/44487535 Gardner, R. C. & MacIntyre, P. D. (1993). On the measurement of affective variables in second language learning. Language Learning, 43(2), 157-194. http://citeseerx.ist.psu.edu/viewdoc/ download?doi=10.1.1.548.8925&rep=rep1&type=pdf Gearing, N., & Roger, P. (2018). Ebbs and Flows: A longitudinal study of an English language instructor's motivation to learn Korean. Journal of Language, Identity & Education, 17, 292-305. https://doi.org/10.1080/ 15348458.2018.1465343
Gearing, N., & Roger, P. (2019). Where's the vision? Rescuing integrativeness to understand the language learning motivation of English-speaking EFL instructors living in South Korea. System, 82, 122-131. https:// doi.org/10.1016/j.system.2019.03.008. Gu, M. M., & Cheung, D.S. (2016). Ideal L2 self, acculturation, and Chinese language learning among South Asian students in Hong Kong: A structural equation modelling analysis. System, 57, 14-24. https://doi. org/10.1016/j.system.2016.01.001 Hessel, G. (2015). From vision to action: Inquiring into the conditions for the motivational capacity of ideal
second language selves. System, 52, 103-114. https://doi.org/10.10Wj.system.2015.05.008 Houghton, S. (2010). Managing stereotypes through experiential learning. Intercultural Communication Studies, 19(1), 182-198.
Jee, M. J. (2015). A study of language learner motivation: Learners of Korean as a foreign language. Journal of
Korean Language Education, 26(2), 213-238. https://doi.org/10.18209/iakle.2011.22.3.133 Jin, D. Y. (2014). The power of the nation-state amid neo-liberal reform: Shifting cultural politics in the new
Korean Wave. Pacific Affairs, 87(1), 71-92. https://doi.org/10.5509/201487171 Jin, D. Y., & Yoon, K. (2014). The social mediascape of transnational Korean pop culture: Hallyu 2.0 as spreadable
media practice. New Media & Society, 18(7), 1277-1292. https://doi.org/10.1177/1461444814554895 Korea Foundation. (2019, May). [Infographic] Korea through the World's Eyes. KFNewsletter. http://enewsletter. kf.or.kr/?menuno=5569.
Kormos, J., Kiddle, T., & Csizér, K. (2011). Systems of goals, attitudes, and self-related beliefs in second-language-
learning motivation. Applied Linguistics, 32(5), 495-516. https://doi.org/10.1093/applin/amr019 Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. London: Routledge. Larson-Hall, J. (2012). Our statistical intuitions may be misleading us: Why we need robust statistics. Language
Teaching, 45, 460-474. https://doi.org/10.1017/S0261444811000127 Larson-Hall, J., & Herrington, R. (2010). Improving data analysis in second language acquisition by utilizing modern developments in applied statistics. Applied Linguistics, 31(3), 368-390. https://doi.org/10.1093/ applin/amp038
Larson-Hall, J., & Plonsky, L. (2015). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65(Supplement 1), 127-159. https://doi. org/10.1111/lang.12115
Lee, I. H. (2018). Effects of contact with Korean popular culture on KFL learners' motivation. The Korean Language
in America, 22(1), 25-45. https://doi.org//10.5325/korelangamer. 22.1.0025. Lee, S. (2015). Introduction: A Decade of Hallyu Scholarship: Towards a New Direction in Hallyu 2.0. In S. Lee & Abé M. Nornes (Eds.), Hallyu 2.0: The Korean Wave in the age of social media (pp. 1-30). University of Michigan Press.
Lee, S. Y. (2018). Japanese Korean beginners' perception of Korean language. Japanese Culture Research, 67, 257275. https://doi.org/10.18075 / jcs..67.201807.257. Lee, Z. N. (2015). An analysis of relationship between motivation for self-determination and self-regulated
learning of Chinese Korean learners. Journal of Korean Language Education, 26(3), 201-245. Maio, G. R., & Haddock, G. (2010). The psychology of attitudes and attitude change. SAGE Publications. Masgoret, A., & Gardner, R. C. (2003). Attitudes, motivation, and second language learning: A meta-analysis of studies conducted by Gardner and Associates. Language Learning, 53, 167-210. https://doi.org/10.1111/1467-9922.00212
Nikitina, L. (2015). Country stereotypes and L2 motivation: A study of French, German and Spanish language
learners. Studies in Linguistics, 37, 483-509. https://doi.org/10.17002/sil.37.2015010.483 Nikitina, L. (2019). Do country stereotypes influence language learning motivation? A study among foreign language learners in Malaysia. Moderna Sprak, 113(1), 58-79.
Nikitina, L. (2020). Stereotypes and language learning motivation: A study of L2 learners of Asian languages. Routledge.
Nikitina, L., & Furuoka, F. (2018). Expanding the methodological arsenal of applied linguistics with a robust statistical procedure. Applied Linguistics, 39(3), 422-428. https://doi.org/10.1093/applin/amx026.
Nikitina, L., & Furuoka, F. (2019). Language learners' mental images of Korea: Insights for the teaching of culture in the language classroom. Journal of Multilingual and Multicultural Development, 40(9), 774-786. https://doi. org/10.1080/01434632.2018.1561704
Nikitina, L., Paidi, R. & Furuoka, F. (2019). Using bootstrapped quantile regression analysis for small sample research in applied linguistics: Some methodological considerations. PloS One, 14(1), e0210668.
Norris, J. M., Ross, S. J., & Schoonen, R. (2015). Improving second language quantitative research. Language Learning, 65(S1), 1-8. https://doi.org/10.1111/lang.12110
Salibian-Barrera, M., & Zamar, R. H. (2002). Bootstrapping robust estimates of regression. Annals of Statistics, 30, 556-82.
Shin, S.C., Ko, S., & Rue, Y.-J (2016). Heritage language learning: A needs analysis study on Korean-Australian tertiary students. Journal of Korean Language Education, 27(1), 111-155.
Son, S., & Jeon, N. (2011). A study of language learning motivation of Korean language learners. Journal of Korean Language Education, 22(3), 133-152.
Spencer-Rodgers, J. (2001). Consensual and individual stereotypic beliefs about international students among American host nationals. International Journal of Intercultural Relations, 25(6), 639-657. https://doi. org/10.1016/S0147-1767(01)00029-3
Stuckler, D., Basu, S., Suhrcke, M., Coutts, A., & McKee, M. (2009). The public health effect ofeconomic crises and alternative policy responses in Europe: An empirical analysis.Lancet, 374(9686), 315-323. https://doi. org/10.1016/S0140-6736(09)61124-7
Tangalycheva, R. K. (2015). Overcoming stereotypes and negative perceptions in Korea-Russia cross-cultural communication. In R. K. Tangalycheva, D. V. Ivanov, V. I. Il'yin, and E. A. Ostrovskaya (Eds.), Korea and Russia: Socio-cultural Dynamics (pp. 206-225). Aleteya Publishers.
Teh, P. Y., & Goh, H. C. (2016). Does Korean drama have a real influence? An analysis of Malaysia outbound tourists to South Korea. Tourism Culture & Communication, 16(3), 147-160. https://doi.org/10.3727/1098304 16X14750895902882
Ushioda, E. (2009) A person-in-context relational view of emergent motivation, self and identity. In Zoltán Dornyei, and Ema Ushioda (eds), Motivation, language identity and the L2 self (pp.215-228). Multilingual Matters.
Wlodkowski, R. J. (2008). Enhancing adult motivation to learn: A comprehensive guide for teaching all adults, 3rd ed. San Francisco: Jossey-Bass.
Yang, J. S. R. (2003). Motivational orientations and selected learner variables of East Asian language learners in the United States. Foreign Language Annals, 36(1), 44-56. https://doi.org/10.1111/j.1944-9720.2003. tb01931.x
Yoshida, H., Tani, S., Uchida, T., Masui, J., & Nakayama, A. (2014). Effects of online cooperative learning on motivation in learning Korean as a foreign language. International Journal of Information and Education Technology, 4(6), 473-477. https://doi.org/10.7763/IJIET.2014.V4.453
