CC BY-ND
27
National Research University Higher School of Economics Journal of Language & Education Volume 7, Issue 3, 2021
Gómez-Parra, M.-E., Golubeva, I., & Espejo Mohedano, R. (2021). Design and Validation of a Questionnaire for the Measurement of Students'
Perceptions of Intercultural Practices within Bilingual Secondary Schools in the European Context. Journal of Language and Education, 7(3), 58-75. https://doi.org/10.17323/jle.2021.11516
Design and Validation of a Questionnaire for the Measurement of Students' Perceptions of Intercultural Practices within Bilingual Secondary Schools in the European Context
María-Elena Gómez-Parra1, Irina Golubeva2, Roberto Espejo Mohedano1
University of Córdoba 2University of Maryland Baltimore County
Correspondence concerning this article should be addressed to María-Elena Gómez-Parra, Facultad de Ciencias de la Educación. Avda. San Alberto Magno, s/n. 14071 Córdoba, Spain.
E-mail: elena.gomez@uco.es
Bilingual education (BE) is widely recognised as a complex phenomenon, which constitutes a priority for key educational institutions and organisations. However, further research is needed to uproot common beliefs such as that bilingual students can easily interact with two or more cultures. The literature affirms that BE students need specific school training to improve intercultural competence. The main aim of this study is to describe the design and validation of a questionnaire to measure students' perceptions of intercultural practices at bilingual schools. The validity of content and comprehension was carried out through the Delphi method, for which three methodological phases were established. The reliability of the scale (internal consistency) was measured through the Cronbach's alpha coefficient. Factorial analysis was used to check the validity of the construct. The psychometric parameters of the scale were obtained from a pilot sample of 40 students, and later from a sample of 213 students from bilingual secondary schools in several European countries (i.e., The Netherlands, Hungary, Germany, France, England, Spain, and Poland, among others). As a major conclusion, we can state that this questionnaire can be used as a tool for two research goals: (1) the identification of good intercultural school practices in BE, and (2) the development of relevant guidelines for the incorporation of intercultural education into BE.
Keywords: bilingual education, classroom research, intercultural practices, Delphi method, research methodology
Introduction
The main goal of this research is to design a questionnaire to measure students' perceptions of intercultural practices within bilingual secondary schools. Therefore, the main objectives set to accomplish this goal are: (1) to establish the validity of the content of this questionnaire through consensus and agreement from an international panel of experts by applying the Delphi method; (2) to confirm the validity of comprehension by delivering this questionnaire among a pilot sample of 40 secondary education international students; and (3) to analyse the validity and reliability of the questionnaire with a final sample of 213 students from several European countries (i.e., The Netherlands, Hungary, Germany, France, England, Spain and Poland, among others). These countries share similar language education conceptions endorsed by the Common European Framework of Reference for Languages1, so their language policies foster plurilingual and pluricultural competences in the educational curriculum of their citizens.
We will discuss herein the theoretical foundations and main procedural steps towards the building, pilotage, and validation of a questionnaire to measure students' perceived secondary education intercultural school practices. The language of the distributed questionnaire was English (see Appendix).
1 Council of Europe. (2018). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. A Companion Volume with New Descriptors. https://cutt.ly/AjxoVFh.
Research Articles
This article is published under the Creative Commons Attribution 4.0 International License.
This study is contextualised on the intrinsic relationship between language and culture, which is widely accepted among educators who work in bilingual and intercultural contexts (Byram, 2012). As Porto (2013, p. 158) states, "language-and-culture (conceptualized as 'intercultural education') can and should become part of all educators' practices, irrespective of the disciplines they teach".
Bilingual education stands at the heart of international educational policies due to the array of benefits that it can convey: cognitive (Bialystok, 2001; Castro, Ayankoya, & Kasprzak, 2011; Christoffels et al., 2015; Genesee, 1987; Jessner, 2008); socio-cultural (Romanowski, 2018); linguistic (Cazden, Snow, & Heise-Baigorria, 1990); and neurolinguistic (the Brainglot project2, n.d.; Rodríguez-Pujadas, Sanjuán, Fuentes, Ventura-Campos, Barrós-Loscertales, & Ávila, 2014); not to mention the improvement in job-access opportunities for bilingual employees (Callahan & Gándara, 2016; Schluessel, 2007; Tsung, 2009).
Gómez-Parra (2021) defines bilingual education as:
[...] a broad term that designates the approaches, methodologies, processes, materials, and actors (we cannot forget that the educational community is an actor as a whole) that aim to form bilingual speakers, capable of accessing and mastering a complex linguistic and cultural code that is, at the very least, binary (that is, where elements appear that these speakers can formalise in at least two different codes but that form the same semantic construct). (Gómez-Parra, 2021, p. 37. Translation by the authors from the original in Spanish).
The success of a bilingual programme lies, among other factors, in the interdependence between language and culture. As Brisk (1999, p. 2) declared: "A successful bilingual program develops students' language and literacy proficiency, leads them in successful academic achievement, and nurtures sociocultural integration". Evidence suggests that, at present, the situation of intercultural learning post-secondary is variable, with excellent examples of good practices (e.g., including those facilitated by the Erasmus study abroad programme) but also some limited momentum in others. Teaching should be made effective enough to integrate languages and culture, an idea which has been developing since the growth of content-based instruction at the end of the 20th century (Short, 1994). Thus, acknowledging that the students' second language (L2) communicative competence is being measured on a regular basis by both the school (e.g., L2 exams) and relevant international institutions (i.e., millions of international language certificate exams - e.g., IELTS or TOEFL - are issued yearly by specialised institutions such as Cambridge University,3 the British Council,4 Trinity College,5 or Oxford University,6 among others), it is important to measure to what extent the most relevant worldwide educational institution (the school) is successfully accomplishing one of its important tasks: the effective implementation of intercultural practices in secondary education (Santos Rego & Moledo, 2005). Education and training can help students develop their intercultural sensitivity (Straffon, 2003). In this same sense, Senge (2010, p. 148) stated: "If you believe that the shifts ahead will be cultural, not just technical, the potential role of education looms large".
Hence, the development of this assessment instrument aims at covering the gap identified between the level of L2 communicative competence (usually assessed) and the level of interculture (rarely assessed), supported by the review of the literature, which on the one hand identifies the underdevelopment of the intercultural axis within bilingual education (Griva & Kasvikis 2014; Méndez-García, 2012, among others), and, on the other hand, underlines the relevant role of bilingual education in the 21st century (Senge, 2010).
The Rationale of the Research Problem
There are some studies that analyse bilingual education from students' perspectives (e.g., Ramírez-Verdugo, & Gerena, 2020), but to our best knowledge, none offers an instrument that could assess students' perceptions.
2 The Brainglot Project. (n.d.). http://brainglot.upf.edu
3 Cambridge University. (n.d.). Cambridge Assessment. https://www.cambridgeassessmentorg.uk/
4 British Council. (n.d.). British Council and Assessment. https://www.britishcouncil.org/exam/aptis/research/projects/assessment-literacy/introduc-ing-language-assessment-0
5 Trinity College London. (n.d.). GESE Graded Examinations in English. https://www.trinitycollege.com/qualifications/english-language/GESE
6 Oxford University Press. (n.d.). Oxford Test of English. https://elt.oup.com/feature/global/oxford_test_of_english/?cc=es&selLanguage=en
The main rationale that supports the building of a questionnaire to assess students' perceptions of intercultural practices within bilingual secondary schools can be explained as follows:
(a) The literature confirms that bilingual education is prioritised by international institutions and organisms, such as UNESCO7, the Council of Europe8, and national European Ministries of Education (e.g., Hernández & Halbach, 2012).
Fishman (1989, p. 447) states: "If this view is to be developed then, bilingual education must justify itself philosophically as education", which was reformulated by Paulston (1992, p. 80) as "unless we try in some way to account for the socio-historical, cultural, and economic-political factors which lead to certain forms of bilingual education, we will never understand the consequences of that education". Bilingualism is defined by Hamers and Blanc (2000, p. 6) as "the state of a linguistic community in which two languages are in contact with the result that two codes can be used in the same interaction and that a number of individuals are bilingual". Nevertheless, bilingualism is not automatically equated to biculturalism (Grosjean, 2008; 2010), and to being intercultural (Byram, 2002, 2008, 2021; Byram & Golubeva, 2020), which poses a relevant question: Is the system educating the next generation of bilingual students as intercultural youngsters?
(b) From early childhood to higher education, bilingual education in Europe is mostly backed up by the CLIL (Content and Language Integrated Learning) approach (Coyle, 2008), where languages and content are equally fostered. Culture is one of the four axes of this approach (4Cs: Content, Cognition, Communication, and Culture) (Coyle, 2008, pp. 103-104), although its implementation should be improved at the classroom level (Gómez-Parra, 2020).
(c) European secondary education youngsters constitute a big population whose selection has been considered appropriate due to the importance of measuring intercultural school practices for cohorts of students where migration flows are increasing9.
(d) Lastly, to build a specific instrument (a questionnaire) for this population can help researchers obtain different sets of data to be compared and provide interesting insights on the way intercultural education is implemented across the European secondary schools' curricula.
Theoretical Background
Bilingual Education and CLIL
Bilingual education is the most prevalent approach in the world, with CLIL becoming the all-pervading European acronym for bilingual education. As Cenoz (2014, p. 243) puts it:
The term Content and Language Integrated Learning (CLIL) was launched in Europe in the 1990s by a group of experts from different backgrounds, including educational administrators, researchers, and practitioners (Marsh 2002). Since then, the European Commission and the Council of Europe have funded many initiatives in support of CLIL because it responded to a need in Europe for enhancing second-language (L2) education and bilingualism.
CLIL has played a leading role in the bilingual education programmes across the European school system since the term was coined by the European Commission in 1994 (Cenoz, 2015; Eurydice, 2006; Marsh, 2002; Roquet & Pérez-Vidal, 2015). CLIL has become an 'umbrella term' (Haataja, 2007) covering more than a dozen educational approaches, to describe the one where learning a subject (content) is done through the medium of a foreign language. This approach seeks to integrate four main pillars that are summarised by Coyle (2005) in the '4Cs paradigm' (Content, Communication, Cognition, and Culture). Nevertheless, the implementation of the intercultural axis of CLIL (Coyle, 2009) is not considered a big success nowadays (Gómez-Parra, 2020) and
7 UNESCO (United Nations Educational, Scientific and Cultural Organization). (2003). Education in a multilingual world. UNESCO Education Position Paper. UNESCO. https://cutt.ly/8jxoxcE.
8 Council of Europe. (2003). Bilingual education: Some policy issues. Council of Europe - Language Policy Division.
9 OECD (Organization for Economic Cooperation and Development). (2014). Migration policy debates. https://cutt.ly/HjxonTo.
the literature claims the need to establish a paradigm where culture stands at the core of bilingual education (Gómez-Parra, 2016).
Bilingual and Intercultural Education (BIE)
A plethora of researchers have confirmed the importance of bilingual and intercultural education (BIE) (Gómez-Parra, 2016; Méndez-García, 2012; Pérez-Cañado, 2012; Ramos-García, 2011). Moreover, the international community has acknowledged the benefits of the implementation of BIE, and suggests placing intercultural competence at the core of CLIL or bilingual lessons. UNESCO10 defines the three principles of its position as follows:
(a) UNESCO supports mother tongue instruction as a means of improving educational quality by building upon the knowledge and experience of the learners and teachers.
(b) UNESCO supports bilingual and/or multilingual education at all levels of education as a means of promoting both social and gender equality and as a key element of linguistically diverse societies.
(c) UNESCO supports language as an essential component of intercultural education in order to encourage understanding between different population groups and ensure respect for fundamental rights.
The goals that UNESCO identifies for bilingual education are deeply intertwined with those of intercultural education11, where the terms 'culture', 'quality', 'social', 'learner', 'education', 'knowledge', 'respect', and 'understanding' can be found and constitute the basis of the six principles altogether:
Principle I. Intercultural education respects the cultural identity of the learner through the provision of culturally appropriate and responsive quality education for all.
Principle II. Intercultural education provides every learner with the cultural knowledge, attitudes, and skills necessary to achieve active and full participation in society.
Principle III. Intercultural education provides all learners with cultural knowledge, attitudes, and skills that enable them to contribute to respect, understanding, and solidarity among individuals, ethnic, social, cultural, and religious groups and nations.
These principles echo Byram's model of intercultural communicative competence (ICC), which consists of five components that Byram calls 'savoirs': (1) les savoirs (i.e., knowledge), (2) savoir être (i.e., attitudes of relativizing the self and valuing others), (3) savoir comprendre (i.e., skills of interpreting and relating), (4) savoir apprendre/faire (i.e., skills of discovering and interacting), and (5) savoir s'engager (i.e., critical cultural awareness) (Byram 1997, pp. 50-53). The central component of this model - critical cultural awareness -emphasises that learning languages for real communication should be a process of meaningful interaction: "learners need not just knowledge and skill in the grammar of a language but also the ability to use the language in socially and culturally appropriate ways" (Byram, Gribkova, & Starkey, 2002, p. 4).
Although at the theoretical level it is not questioned that IC is an inseparable aspect of foreign language teaching (including bilingual education and CLIL), its classroom implementation needs improvement (Fantini & Tirmizi, 2006). UNESCO12 presented an operational plan on how to promote intercultural competences through clarifying, teaching, enacting, and supporting them. This document emphasises the idea that "the practice of interculturalism must become part of the fabric of daily social life." (UNESCO, 2013, p. 32).
In 1997, Hallett developed a model for acquiring IC in bilingual education. This model consists of three elements: (1) aspects of students' own culture; (2) aspects of the target language culture; and (3) intercultural aspects, which cover global phenomena (Hallett, 1997, p. 2). Hallett's model of the Bilingual Triangle is also applicable to the CLIL approach. Both the role of interculture and the enhancement of intercultural competence
10 UNESCO (United Nations Educational, Scientific and Cultural Organization). (2003). Education in a multilingual world. UNESCO Education Position Paper. UNESCO. https://cutt.ly/8jxoxcE.
11 UNESCO (United Nations Educational, Scientific and Cultural Organization). (2006). UNESCO Guidelines on Intercultural Education. UNESCO. https://cutt.ly/xjxok1G.
12 UNESCO (United Nations Educational, Scientific and Cultural Organization). (2013). Intercultural Competences. UNESCO. https://cutt. ly/9jxojMN.
among students enrolled in bilingual programmes (Dziedziewicz, Gajda, & Karwowski, 2014; Ponciano & Shabazian, 2012), as well as the importance of the teacher in the whole process (Gándara & Escamilla, 2017; Gedik Bal & Savas, 2020; Johannessen, Thorsos, & Dickinson, 2016; Reljic, Ferring, & Martin, 2015; Tabatadze, 2015, among others) are well developed in the literature.
The theoretical assumptions that stem from the analysis of the literature, and which stand as the basis for the construction of our questionnaire, can be summarised as follows:
• Language and culture are closely related (Kramsch, 2011);
• A level of good knowledge about the first culture improves intercultural communicative practices (Samovar, Porter, McDaniel, & Roy, 2013);
• Learning second languages improves intercultural communicative practices (Moeller & Nugent, 2014);
• Being bilingual does not necessary mean being intercultural (Byram, 2002);
• Intercultural practices should be improved at classroom level (Gómez-Parra, 2020).
The research problem this paper addresses is to measure students' perceptions of intercultural practices within European bilingual secondary schools because, following the analysis of the literature (Gómez-Parra, 2020), bilingual programmes in Europe do not efficiently implement intercultural practices in their curricula. This study is contextualised on the European continent, across the countries where this questionnaire was launched.
Methodology
Procedure
The first procedural step was to design and build the first version of the questionnaire, once the lack of appropriate questionnaires to cope with our objectives in the existing literature was established. This questionnaire was designed to contribute to the existing literature by measuring secondary students' awareness and perceptions towards intercultural communicative practices, as well as the real intercultural practices carried out in secondary school contexts. The content was established according to the theoretical assumptions derived from the literature review.
The initial questionnaire was divided into five sections (blocks) that measure the different areas in which intercultural competence has been analysed in the literature: knowledge, attitudes, skills, and awareness, plus a fifth block to measure intercultural school practices. Our research group classified questions into their corresponding sections (1-5), according to the content and the area they would measure. A final set of 51 questions was agreed upon and distributed into blocks, each of which contained a different number of items that initially ranged between six (in block 3) and 13 (in block 5). The scale designed for this purpose was a 4-point Likert scale for all questions, where the answers draw a continuum: 1 - Strongly Disagree (SD); 2 -Disagree (D); 3 - Agree (A); and 4 - Strongly Agree (SA).
Additionally, the format of the questionnaire was discussed within our research group; the main purpose of the design was to facilitate the process of reading and answering by the pilot group. For this, the five sections were clearly separated, and the two Likert scales were differentiated by colour. The anonymity of the respondents was respected. In addition, and with the aim of carrying out subsequent descriptive techniques, the instrument also includes some basic data on the surveyed population, such as age, gender, school course, and country of origin.
The validity of the questionnaire was defined according to these two parameters: (a) validity of the content (following the technique of the experts); and (b) comprehension validity (set by a pilot study, which allowed us to know whether respondents had understood all of the questions).
Experts, then, assessed the questionnaire by attaching a value within a 4-point Likert scale for each of the 51 items, and by evaluating the comprehension of these and the formulation of the questions (written in English, Spanish, and Hungarian). Regarding the degree of pertinence, experts assessed to which degree (within a 4-point Likert scale) the items were relevant for each of the five sections (blocks) where they had been initially
included. The group of experts included all items that experts had valued as 4; modified those valued as 3; and discarded those valued as 2 or 1.
The comprehension validity of the questionnaire was established through a pilot study where 40 secondary education students participated. The procedure was first to identify the appropriate bilingual schools and then an official permission for access was sent to the school principals. Next, our research group agreed on the age of the pilot target groups. As secondary education is not uniformly established for students of the same age across Europe, respondents of the pilot group would be between 14 and 16 years old. Finally, the questionnaire was delivered among the 40 students in two secondary education schools in Hungary and Spain.
Secondly, once the results of the pilot sample were analysed, the questionnaire was distributed in several European countries among a new sample of 213 bilingual secondary school students of the same characteristics. With this new data collection, the questionnaire was considered to be validated (with a decrease in items and blocks) to measure 'students' perceptions of intercultural practices within bilingual secondary schools in the European context'.
The Delphi Process (Phase 1) and Participants
As the literature shows (e.g., Hsu & Sandford, 2007), the Delphi technique is one of the most widely used and accepted methods for collecting responses from experts in a particular field of study in order "to achieve a convergence of opinion on a specific real-world issue" (2007, p. 1). This technique was considered appropriate as it is well suited for consensus-building (ibid.). In contrast to other research methods, Delphi offers interesting advantages. For instance, during the series of rounds, the selected participants get the opportunity to reassess and refine their initial judgments about the subject matter based on the feedback from other panellists (ibid.), which was very important to the present study.
The identification of an appropriate panel of experts who adequately and purposefully assess the validity of instruments is one of the most important steps in the Delphi method (Hung, Altschuld, & Lee, 2007; Landeta, 2006). Two differentiated groups were established in order to apply this questionnaire: the research group (i.e., scholars carrying out the research), and the panel of experts (i.e., the Delphi group). The features that describe their appropriateness are:
(a) Expertise on the key areas of this research: intercultural education; bilingual education and statistical analysis;
(b) Strong knowledge on the Delphi method;
(c) An outstanding level of communication in at least two languages.
The panel of experts was selected among international specialists who met the following characteristics: (a) expertise on, at least, one of the key areas of the research (see above); (b) high level of awareness on the research problem; (c) professional experience in secondary schools; (d) full proficiency in several languages. Thus, 12 experts were identified and selected from the international arena (e.g., University of Manchester -UK-, University of Oldenburg -Germany-, University of Granada -Spain-, Marmara University -Turkey-, the Polytechnic Institute of Lisbon -Portugal-, and Lower Silesia University -Poland-, among others), taking into consideration the professional contacts of the research group. Our research group is made of three university professors, whose areas of expertise are linguistics, bilingual education, intercultural education, and statistics.
Three methodological phases (preliminary, exploratory, and final) were established and the assessment template for the panel of experts was designed. In the preliminary phase, our research group stated the research problem, according to which the panel of experts was selected. Once the experts accepted the invitation to serve on the Delphi panel, our research group took the responsibility to track their reports of assessment, interpret results from research, and supervise the whole process to adjust specific points when and where necessary. During the exploratory phase, a questionnaire was designed, first, as an experimental version, and then, after the construction process following all necessary steps, as the final one (see Appendix). To develop this procedure, the first experimental version was sent (electronically) to the panel of experts with the template for assessment and an introductory letter explaining the rationale and the theoretical background of the research together with the detailed instructions regarding the two Likert scales to be used, and the criteria against which the questionnaire was to be validated: clarity, pertinence, adequacy (quantitative), and comments
(qualitative). The instrument for validation was clearly divided into the five blocks following the structure of the questionnaire and was arranged according to: (a) a 5-point Likert scale for most questions, where experts could assess their pertinence and/or validity; and (b) a section for comments by the experts (from which qualitative data could be collected). After the first round, following suggestions made by the experts, our research group carried out all necessary amendments. This was followed by the second round of the Delphi test. The experts were given 10 days to return their assessments to the research team, after which we modified and finalised the questionnaire based on the feedback of the Delphi panellists. In total, the process of design of the research instrument (i.e., the questionnaire) was carried out within a period of 13 months. As the result of this Delphi process, there was a reduction from the initial 51 questions to 21 and from five blocks to four. The questions initially included in the 'knowledge' block were considered to be already included in the remaining four.
Description of the Sample for Phase 2 of the Validation Process
The samples were divided into two groups, the first corresponding to the pilot study and the second targeted to validate the final questionnaire. The pilot sample was composed of 40 students from bilingual secondary schools in Spain (25%) and Hungary (75%), and the questionnaire was distributed in February 2017. The second sample of our study was collected between the months of May and June 2017 through a purposefully designed online questionnaire. The sample was composed of 213 students from bilingual secondary schools in several European countries: Hungary (21.6%), The Netherlands (20.2%), Spain (14.6%), Germany (9.9%), Poland (9.9%), France (8.9%), England (1.4%) and others (13.5%). The individuals were selected through a non-probabilistic convenience sampling by applying a snowball technique, using the research group's contacts in the countries where the questionnaire was administered.
The distribution of the final sample of the 213 students surveyed was balanced according to the sex of participants: male (48.4%) and female (51.6%).
Finally, Table 1 presents the distribution of students surveyed according to their age. Regarding age, the category 'other' refers to those students with ages close to 14 (or who had recently become 17).
Table 1
Sample distribution by age
Age 14 15 16 Other
% Participation 18.3 41.8 25.8 14.1
As explained above, our research group decided to survey a convenience sample. The sample size was determined by means of the classic recommendation of having a minimum of 10 sample elements for each of the items that make up the instrument to be validated or the number of items appropriate to convey factorial analysis (Hair, Anderson, Tahtham, & Black, 1999).
Results
The analysis of the quantitative data was carried out by using IBM SPSS Statistics v. 22, whereas the analysis of the qualitative data was done through content analysis.
Results from Statistical Analysis
Internal consistency was used in both cases (with both the pilot and final sample) to check the reliability of the scale, and it was measured through Cronbach's alpha (Cronbach, 1990), which expresses the degree to which items measure one common and unique variable. In this research, it is defined as 'the intercultural practices in bilingual secondary schools'. Additionally, and through a structured procedure in stages, such Cronbach's alpha was used to delete those items which diminished the internal global consistency coefficient. The process is considered complete when either the scale does not improve or when it keeps the level of internal consistency
by erasing a random element (Levy-Magin & Varela-Mallou, 2003). We also used total item correlations to refine the number of final items (Cozby, 2005). The Delphi method was used (see above) to check the validity of the content. Finally, to check the validity of the construct, a confirmatory factorial analysis was used for the final sample.
In brief, the data from 213 respondents were collected and pre-processed to detect missing data, inconsistent data, and outliers. Our research team: (a) carried out a previous descriptive study, (b) measured internal consistency through Cronbach's alpha coefficient, (c) checked content validity using the Delphi method, and (d) checked instrument validity by means of a confirmatory factor analysis.
Pilot Study (Phase 2)
Considering the values obtained from the pilot sample and by applying the techniques mentioned above, the final scale consisted of 21 items with internal consistency of a Cronbach's alpha of 0.881, a value classified as very good (Nunnally & Bernstein, 1994).
On the other hand, if one instrument measures several variables, that is, if it has individual sub-instruments or sub-scales, the recommendation is to arrange specific reliability tests for each one (Brown, 1980). Thus, the five initial blocks were reduced to four with Cronbach's values of 0.726, 0.687, 0.743, and 0.674 respectively, ranges which the literature values between 0.60 and 0.70 as 'sufficient' (Nunnally & Bernstein, 1994). Additionally, George and Mallery13 suggested a tiered approach consisting of the following: "> 0.9 - Excellent, > 0.8 - Good, > 0.7 - Acceptable, > 0.6 - Questionable, > 0.5 - Poor, and < 0.5 - Unacceptable".
The average score of the total scale was 73.081 (standard deviation of 7.166). Taking into account that the minimum score was 21 and the maximum was 84, we can say that the surveyed students perceive that they strongly observe intercultural practices (as 81.08% of surveyed students have a final score higher than the average). This was also evidenced by the average points of each item: the total average was 3.48 (which is a very high score considering that the minimum is 1 and the maximum is 4). 38.1% of the questions showed a final score higher than 3.48, and 100% were higher than 2.5 (mean for a 1-4 point Likert scale). These results can be considered 'excellent' due to an initial lack of variability of the answers, a fact that can be inferred by the low value of the variation coefficient (= 0.098).
These facts led us to increase the Likert scale range from 1-4 to 1-6, thus extending the variability for the answers. In situations where low total score variability is achieved with a small number of categories, reliability can be improved through an increase in the number of categories employed (Masters, 1974). In this sense it was agreed to extend the final measure scale to a 6-point Likert scale.
To summarise, the final instrument was made of 21 items measured by a 1-6 Likert scale (Strongly Disagree (SD), Disagree (D), Somewhat Disagree (SWD), Somewhat Agree (SWA), Agree (A), and Strongly Agree (SA)), divided into four sections corresponding to Attitudes, Skills, Awareness, and Intercultural Practices.
At this point, the instrument built by our research team provided sufficient evidence to let us think that it could constitute a suitable tool to measure the degree of implementation of intercultural practices in bilingual secondary schools as perceived by the students, with a reliability degree of 88.1%. The instrument also included a second 1-6 Likert scale with three items which measured the degree of proficiency of second languages (No, Barely /A1; Not very good /A2; Reasonably good /B1; Good /B2; and Very good /C1), together with some basic data on the surveyed population, such as age, gender, school course, and country of origin.
Final Analysis (Phase 3)
13 George, D. & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Allyn & Bacon.
The final analysis of the data, although the values obtained on the scales are acceptable, led us to carry out a final action for the validation of the construct by means of a confirmatory factor analysis. This confirmatory factorial analysis made it possible to identify three factors corresponding to the groups of questions for the items detailed in Table 4. As can be seen, two additional items were removed because their factor saturations did not fit with any multidimensional factor. The analysis is relevant according to the values of the KaiserMeyer-Olkin (KMO) and Barlett's sphericity tests (Bartlett, 1950) (Table 2).
Table 2
Kaiser-Meyer-Olkin (KMO) and Barlett's tests
KMO measure of sampling adequacy 0.938
Barlett's sphericity tests Approx. Chi-square 2400.767
df 171
Sig. 0.000
Source: Own elaboration.
The three factors found explain 63.032% of the variability inherent in the data, so it can be said that with this final classification we have a good representation of the total information. The percentage of the variance explained by these factors can be seen at Table 3.
Table 3
Percentage of the variance explained by the three factors
Rotation Sums of Squared Loadings
Component -
Total % of Variance Cumulative %
1 4.522 23.802 23.802
2 3.960 20.843 44.645
3 3.494 18.387 63.032
Extraction method: principal component analysis.
Source: Own elaboration.
The corresponding factorial load matrix (rotated), where low values have been eliminated, is shown in Table 4. Table 4
Rotated component matrix with low factorial loads eliminated
Component 12 3
I can describe interactions of my culture with another culture (e.g., regarding clothing, appearance, music, food, among others).
If necessary, I adapt my behaviour to an intercultural encounter when I interact with foreign peers. I show interest in new cultural behaviours. I take on various roles to adapt to different cultural situations.
I respect the right of those students who come from a different culture to have their own (cultural and religious) values.
I am able to use strategies (e.g., observation or seeking for support) to adapt successfully to a second culture. I am able to use communicative strategies to facilitate interactions with foreign peers. I am able to solve cross-cultural issues if they arise during interactions. I am aware of the fact that language influences culture.
0.488 0.390 0.309
0.674 0.654 0.750
0.700 0.377
0.704 0.318
0.724
0.769 0.324
0.749
Component
1 2 3
I am aware of the reactions (both positive and negative) of others towards my cultural values. Q.762 Q.34Q
I am aware of my own intercultural development. Q.729
Teachers promote interactions with foreign peers in the classroom. Q.356 0.679
Grouping is generally arranged by including members of diverse nationalities within the same group. 0.808
Cooperative learning is a common approach for activities. Q.453 Q.395 Q.513
Communication among peers is conveyed through our common mother tongue. Q.388 Q.5Q6
Communication among peers is conveyed through my peer's language. Q.657
I am aware of being myself a culturally conditioned person. Q.795 Q.326
Teachers promote the learning of another culture in the classroom by arousing questions on food, weather, language, customs, etc. Q.737
I am aware of the number of foreign peers in my classroom. Q.722
Note. Source: Own elaboration. Extraction method: main component analysis. Rotation method: varimax with Kaiser normalisation.
As shown in Table 4, the first factor (8 items) is associated with the concept of 'skills'; the second factor (5 items) is associated with 'awareness', and, finally, the third factor (6 items) is related to 'communication practices'. In this way, the 19 original items were reduced into three factors that represent three blocks to measure 'students' perceived intercultural practices within bilingual secondary schools'.
The Cronbach's alpha value of the full scale was 0.935 and the value of each sub-scale is shown in Table 5.
Table 5
Reliability statistics
Skills Awareness Communication Practices
Cronbach's alpha Number Cronbach's alpha Number Cronbach's alpha Number
0.902 9 0.897 6 0.799 4
Note. Source: Own elaboration.
All sub-scales showed excellent reliability. Results showed that the final questionnaire was made up of 19 items that measured the items under study. In addition, this new instrument was sent once again to the group of experts who finally reached consensus in the validation process.
Therefore, considering the analysis of data from the final sample, we can state that the instrument used has a high reliability, as determined by the Cronbach alpha coefficient of 0.935 (being 1 its maximum value). Thus, we can measure the degree of attitude towards intercultural practices for each participant in the survey. For this purpose, a new variable that agglutinates this attitude was considered, and we could find a score for it through a new normalised variable (maximum value of 1, considering the maximum score of the scale) called 'attitude' (Table 6).
Table 6
Descriptive statistics for 'attitude'
Number Min. Max. Mean Standard Deviation
Attitude 213 0.167 1.000 0.748 Q.153
Note. Source: Own elaboration.
Thus, the average value for attitude was 0.748, which is considered very high as the maximum value is 1. Considering that the standard deviation is small, it can be concluded that most participants showed high scores, where 57.3% of the students analysed obtained scores higher than the third quartile of the 'attitude' variable, 36.2% between the second and third quartiles, and only 6.6% of the participants scored below the median.
Discussion
The main aim of this research was to design and pilot an instrument (i.e., a questionnaire) for measuring intercultural practices within bilingual secondary education. Given that the issue of the assessment of intercultural competence is one of the most problematic topics within the field of intercultural education, especially in the context of language classrooms (e.g., Borghetti, 2017; Sercu, 2010) and that there are different approaches to assessing intercultural (communicative) competence (Byram, 2021; Council of Europe14; OECD15; UNESCO16, among others), we acknowledge that our aim was quite ambitious and that further research is needed to identify which (and if) intercultural practices in BE classrooms are perceived as effective by students. We also acknowledge that practices in conflict-ridden contexts can be very different and that an absolute recipe is not possible. However, we took on this challenge based on the idea that offering such instruments can be beneficial not only for improving intercultural practices in the context of bilingual secondary schools, but for raising awareness and understanding of the importance of intercultural education as an integral part of bilingual education.
The tool herein developed, piloted, and validated can be used for the following goals: (1) the identification of good intercultural school practices in bilingual education, and (2) the development of relevant guidelines to implement intercultural education within bilingual education. The results of our questionnaire show that the metric values obtained through different validity and confirmatory tests are valid and can measure the items for which they were initially designed. Our results show a high level of reliability in terms of internal consistency and reliability, which facilitates their delivery among the targeted population: international bilingual secondary education students. These metric results complement the validity of the results from content tests. Therefore, we can state that our questionnaire has good psychometric quality, according to the statistical tests applied.
The researchers of this paper have identified a gap in the literature, to which the main goal of this study has been addressed: to design a questionnaire to measure students' perceptions of intercultural practices within bilingual secondary schools. The success of bilingual programmes is (to a great extent) linked to the implementation of successful intercultural practices and the development of intercultural awareness, which has been identified as one key factor (Gómez-Parra, Huertas-Abril, & Espejo-Mohedano, 2021). This study, thus, can contribute to the fields of both bilingual and intercultural education, where the measurement of the perceptions of bilingual students can guide stakeholders to improve (or reinforce) pedagogical and in-classroom practices.
Conclusion
As earlier discussed, bilingual education is a priority for most international educational institutions and organisations. Research has demonstrated that the intercultural axis within CLIL is being neither extensively nor appropriately put into practice (Méndez-García, 2012). Additionally, further research shows that bilingual (specifically CLIL) teachers need to be trained in a systemic way on how to implement intercultural education (Figueredo-Canosa, Ortiz-Jiménez, Sánchez-Romero, & Berlanga, 2020; Lallana & Salamanca, 2020; Uzum, Akayoglu, & Yazan, 2020), so the use of this questionnaire can help accomplish this goal.
Our future plan is to expand this research project by involving language teacher trainers and their graduate students from Spain (the University of Córdoba and the Delegation of Education in Córdoba); Germany (Carl
14 Council of Europe. (2008). White paper on intercultural dialogue: Living together as equals in dignity. Council of Europe.
15 OECD (Organization for Economic Cooperation and Development).(2017). Global Competence Framework. https://cutt.ly/sjxobhL.
16 UNESCO (United Nations Educational, Scientific and Cultural Organization). (2014). Global Citizenship Education: Preparing Learners for the Challenges of the Twenty-first Century. UNESCO. https://cutt.ly/UjxogN2
von Ossietzky University of Oldenburg and the University of Bremen); Poland (University of Lower Silesia); Hungary (TEE: the Hungarian section of the European Association of Teachers); and the USA (Texas Women's University), who are participating in the TEACUP project17. The purpose of this collaboration is to develop practical recommendations for intercultural pedagogy in bilingual (and CLIL) education settings and to design educational modules for pre- and in-service language teachers. Finally, we plan to carry out a crossmatch between both sets of results and knowing about the intercultural practices that these students perceive depending on their self-reported intercultural competence.
Limitations and Solutions
The design of a questionnaire, unquestionably, holds several limitations that can reduce the scope to which it can be applied by researchers. The rationale behind the decision of using a single quantitative instrument lies in the fact that it was necessary to facilitate the answering procedure to the target population (14 to 16-year-old secondary education students, whohave probably never been faced with these types of questions before) in order to get as objective data as possible.
Another limitation refers to the administration of the questionnaire, as the age and cognitive level of the target population can lead to statistical biases in the final results (Vivo, Saric, Muñoz, McCoy, López-Peña, & Bautista-Arredondo, 2013). Soubelet and Salthouse (2011) affirmed that youngsters (whose ages are varied) show different perceptions of what is socially acceptable; this fact can lead to statistical biases related to the psychological development stage of each individual. Therefore, this fact can impact the validity and consistency of their answers to a questionnaire that is sensitive to this trait. On the other hand, Steinberg and Monahan (2007) suggested that the influence of what is perceived as socially desirable reaches its highest peak between ages 10 and 14, whereas the period ranging from 14 to 16 is when higher social resistance is expressed. The instrument herein has been specifically designed to measure the perceived intercultural practices of bilingual secondary education students between 14 and 16 years of age, so special attention should be paid to this fact. Moreover, the presence of an interviewer or an expert guide can improve the quality of the responses. Some other studies have affirmed that their presence can arouse some variance in the answers ranging between 5% and 10% (although this value can increase up to 40% for very sensitive questions according to Tourangeau & Yan, 2007).
Therefore, following the suggestions by the panel of experts on the design of our instrument (regarding the clarity, pertinence, context, and adequacy of the questions for secondary education students) and considering that it is 'a short questionnaire', self-administration of the instrument is recommended, thus avoiding the interviewer effect (as was carried out in the piloting of this study).
In addition, the present questionnaire does not seek any related information from key stakeholders such as school managers, classroom teachers, or parents (which will be the goal of a complementary questionnaire that is under construction at the moment of writing this paper by the same research team). This fact sets limitations regarding what can be reasonably claimed from these results. Thus, we hope to be scientifically rigorous in stating what can legitimately be claimed from this analysis: Students' perceptions of intercultural practices within bilingual secondary schools in the European context.
Acknowledgements
We would like to gratefully acknowledge the assistance of the international panel of experts who helped to validate this tool, and to the Spanish and Hungarian colleagues and students who helped us to pilot the questionnaire at secondary bilingual schools.
Also, our special gratitude goes to Prof. Richard Johnstone, without whose valuable comments and suggestions this paper would not have been possible.
17 https://teacup-project.eu/site/).
Declaration of Competing Interest
None declared.
References
Bartlett, M.S. (1950). Tests of significance in factor analysis. British Journal of Mathematical and Statistical
Psychology, 3, 77-85. https://doi.org/10.1111/j.2044-8317.1950.tb00285.x Bialystok, E. (2001). Bilingualism in development: Language, literacy, and cognition. Cambridge University
Press. https://doi.org/10.1017/cbo9780511605963 Borghetti, C. (2017). Is there really a need for assessing intercultural competence? Some ethical issues. Journal
of Intercultural Communication, 44, . http://www.immi.se/intercultural/nr44/borghetti.html Brisk, M.E. (1999). Quality bilingual education: Defining success. LAB Working Paper, 1, 1-11. Brown, F.G. (1980). Principios de la Medición en Psicología y Educación [Principles of Measurement in Psychology
and Education]. El Manual Moderno. Byram, M. (1997). Teaching and assessing intercultural communicative competence. Multilingual Matters. https://
doi.org/10.21832/9781800410251 Byram, M. (2002). On being 'bicultural' and 'intercultural'. In G. Alred, M. Byram, & M. Fleming (Eds.), Intercultural
experience and education (pp. 50-66). Multilingual Matters. https://doi.org/10.21832/9781853596087-007 Byram, M. (2008). From foreign language education to education for intercultural citizenship. Multilingual
Matters. https://doi.org/10.21832/9781847690807 Byram, M. (2012). Language awareness and (critical) cultural awareness-relationships, comparisons and
contrasts. Language Awareness, 21, 1-13. https://doi.org/10.1080/09658416.2011.639887 Byram, M. (2021). Teaching and assessing intercultural communicative competence: Revisited. Multilingual
Matters. https://doi.org/10.21832/9781800410251 Byram, M., & Golubeva, I. (2020). Conceptualizing intercultural (communicative) competence and intercultural citizenship. In J. Jackson (Ed.), Routledge handbook of language and intercultural communication (pp. 70-85). Routledge, Taylor & Francis. https://doi.org/10.4324/9781003036210-6 Byram, M., Gribkova, B., & Starkey, H. (2002). Developing the intercultural dimension in language teaching. A
practical introduction for teachers. Council of Europe. https://doi.org/10.21832/9781853596087-007. Callahan, R. M., & Gándara, P. C. (Eds.). (2016). The bilingual advantage: Language, literacy and the US labor
market. Multilingual Matters. https://doi.org/10.1007/s10993-015-9362-2 Castro, D.C., Ayankoya, B., & Kasprzak, C. (2011). The new voices/Nuevas voces: Guide to cultural and linguistic
diversity in early childhood. Brookes. https://doi.org/10.1080/13670050.2011.590068 Cazden, C., Snow, C.E., & Heise-Baigorria, C. (1990). Language planning in preschool education with annotated
bibliography. Report prepared for the consultative group on early childhood care and development. UNICEF. Cenoz, J., Genesee, F., & Gorter, D. (2014). Critical analysis of CLIL: Taking stock and looking forward. Applied
Linguistics, 35(3), 243-262. https://doi.org/10.1093/applin/amt011 Cenoz, J. (2015). Content-based instruction and content and language integrated learning: The same or different?
Language, Culture & Curriculum, 28(1), 8-24. https://doi.org/10.1080/07908318.2014.1000922. Coyle, D. (2005). CLIL: Planning tools for teachers. University of Nottingham.
Coyle, D. (2008). CLIL - A pedagogical approach from the European perspective. In N.H. Hornberger (Eds.), Encyclopedia of language and education (pp. 97-112). Springer. https://doi.org/10.1007/978-0-387-30424-3_92
Coyle, D. (2009). Promoting cultural diversity through intercultural understanding: A case study of CLIL teacher professional development at in-service and pre-service levels. In M.L. Carrió-Pastor (Ed.), Content and language integrated learning: Cultural diversity (pp. 105-124). Peter Lang. Cozby, P.C. (2005). Métodos de Investigación del Comportamiento [Behavioral Research Methods]. McGraw Hill. Christoffels, I. K., de Haan, A. M., Steenbergen, L., van den Wildenberg, W. P. M., & Colzato, L. S. (2015). Two is better than one: Bilingual education promotes the flexible mind. Psychological Research, 79(3), 371379. https://doi.org/10.1007/s00426-014-0575-3. Cronbach, L.J. (1990). Essentials of psychological testing (5th ed.). Harper Collins Publishers. Deardorff, D.K. (2006). The identification and assessment of intercultural competence. Journal of Studies in
International Education, 10(3), 241-266. https://doi.org/10.1177/1028315306287002 Dziedziewicz, D., Gajda, A., & Karwowski, M. (2014). Developing children's intercultural competence and
creativity. Thinking Skills and Creativity, 13, 32-42. https://doi.org/10.10Wj.tsc.2014.02.006. Eurydice (Education Information Network in Europe). (2006). Content and Language Integrated Learning (CLIL)
at School in Europe. Eurydice. Fantini, A. & Tirmizi, A. (2006). Exploring and assessing intercultural competence. World Learning Publications. Paper 1. http://digitalcollections.sit.edu/worldlearning_publications/1?utm_source=digitalcollections.sit. edu%2Fworldlearning_publications%2F1&utm_medium=PDF&utm_campaign=PDFCoverPages Figueredo-Canosa, V., Ortiz-Jiménez, L., Sánchez-Romero, C., & Berlanga, M.C.L. (2020). Teacher training in intercultural education: Teacher perceptions. Education Sciences, 10(3), 81. https://doi.org/10.3390/ educsci10030081
Gándara, P. & Escamilla, K. (2017). Bilingual education in the United States. In J. Lo Bianco & Y. Slaughter (eds.), Bilingual and multilingual education (pp. 1-14). Springer. https://doi.org/10.1007/978-3-319-02324-3_33-2
Gedik Bal, N. & Savas, P. (2020). Intercultural competence in the eyes of state school english language teachers
in Turkey. Journal of Language and Education, 6(2), 56-75. DOI: https://doi.org/10.17323/jle.2020.10327. Genesee, F. (1987). Learning through two languages: Studies of immersion and bilingual education. Heinle &
Heinle. https://doi.org/10.2307/3587516 Gómez-Parra, M.E. (2016). Educación bilingüe e intercultural (EBeI): Fortalezas, retos y oportunidades de un enfoque integrado [Bilingual and Intercultural Education (EBeI): Strengths, Challenges and Opportunities of an Integrated Approach]. In R. Serrano-Rodríguez, M.E. Gómez-Parra, & C.A. Huertas-Abril (Eds.), La educación sí importa en el siglo XXI (pp. 67-78). Síntesis. Gómez-Parra, M.E. (2020). Measuring intercultural learning through CLIL. New Approaches in Educational
Research, 9(1), 43-56. https://doi.org/10.7821/naer.2020.L457 Gómez-Parra, M.E. (2021). Educación bilingüe en la infancia. El enfoque PETaL /Bilingual education in childhood.
The PETaL approach]. Peter Lang. https://doi.org/10.3726/b17553 Gómez-Parra, M.E., Huertas-Abril, C.A., & Espejo-Mohedano, R. (2021). Key factors to evaluate the impact of bilingual programs: Employability, mobility and intercultural awareness. Porta Linguarum, 35, 93109. https://doi.org/10.30827/portalin.v0i35.15453 Griva, E. & Kasvikis, K. (2014). CLIL in Primary Education: Possibilities and challenges for developing L2/ FL skills, history understanding and cultural awareness. In N. Bakic-Miric, & D. Erkinovich Gaipov (Eds.), Current trends and issues in education: An international dialogue. Cambridge Scholars Publishing. Grosjean, F. (2008). Studying bilinguals. Oxford University Press.
Grosjean, F. (2010). Life and reality. Harvard University Press. https://doi.org/10.4159/9780674056459 Haataja, K. (2007). A definition of CLIL as an umbrella term: A historical perspective. In B. Roza-González (Ed.),
Good practice in content and language integrated learning (pp. 9-10). BeCLIL. Hair, J.F., Anderson, R.E., Tatham, R.L., & Black, W.C. (1999). Análisis multivariante [Multivariate analysis] (5a ed.). Prentice-Hall.
Hallet, W. (1997). The bilingual triangle: Überlegungen zu einer didaktik des bilingualen sachfachunterichts [The bilingual triangle: Considerations on the didactics of bilingual subject teaching]. Ministerium für Bildung, Wissenschaft und Weiterbildung. Hamers, J.F., & Blanc, M.H.A. (2000). Bilinguality and bilingualism (2nd ed.). Cambridge University Press. https://
doi.org/10.1017/cbo9780511605796 Hernández-González, T., & Halbach, A. (2012). Retos de la educación bilingüe [Challenges of bilingual education].
Ministerio de Educación - Secretaría General Técnica. Hsu, C.C., & Sandford, B.A. (2007). The delphi technique: Making sense of consensus. Practical assessment,
research & evaluation, 12(10), 1-8. Hung H-L., Altschuld, J.W., & Lee, Y. (2008). Methodological and conceptual issues confronting a cross-country Delphi study of educational program evaluation. Evaluation and Program Planning, 31, 191-198. https://doi. org/10.1016/j.evalprogplan.2008.02.005 Jessner, U. (2008). Teaching third languages: Findings, trends, and challenges. Université de Lausanne. https://doi.
org/10.1017/S0261444807004739. Johannessen, B., Thorsos, N., & Dickinson, G. (2016). Current conditions of bilingual teacher preparation programs in public universities in USA. Education and Society, 34(2), 27-48. https://doi.org/10.7459/es/34.2.03 Kramsch, C. (2011). The symbolic dimensions of the intercultural. Language Teaching, 44, 354-367. https://doi.
org/10.1017/s0261444810000431 Lallana, A. & Salamanca, P. (2020). Intercultural communicative competence in L2 Spanish: Guidelines for teacher training programmes. Journal of Spanish Language Teaching, 7(2), 178-192. https://doi.org/10.1080/
23247797.2020.1847399
Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological forecasting and social
change, 73(5), 467-482. https://doi.org/10.10Wj.techfore.2005.09.002 Levy-Magin, J. P., & Varela-Mallou, J. (2005). Análisis multivariablepara las ciencias sociales [Multivariate analysis
for the social sciences]. Pearson Prentice Hall. Marsh, D. (2002). CLIL/EMILE. The European dimension. Actions, trends, and foresight potential. European Commission.
Masters, E.R. (1974). The relationship between number of response categories and reliability of Likert-type questionnaires. Journal of Educational Measurement, 11(1), 49-53. https://doi.org/10.1111/j.1745-3984.1974. tb00970.x
Matveev, A.V. & Merz, M.Y. (2014). Intercultural competence assessment: What are its key dimensions across assessment tools? In L. Jackson, D. Meiring, F. van de Vijver, & E. Idemudia (Eds.), Toward sustainable development through nurturing diversity. Selected Papers from the 21th International Congress of the International Association for Cross-Cultural Psychology (pp. 141-156). International Association for Cross-Cultural Psychology.
Méndez-García, M.C. (2012). The potential of CLIL for intercultural development: A case study of Andalusian bilingual schools. Language and Intercultural Communication, 12(3), 196-213. https://doi.org/10.1080/14708 477.2012.667417.
Moeller, A.J. & Nugent, K. (2014). Building intercultural competence in the language classroom. In S. Dhonau (Ed.), Unlock the gateway to communication. 2014 Report of the Central States Conference on the Teaching of Foreign Languages (pp. 1-19). Johnson Litho Graphics of Eau Claire. Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). McGraw-Hill. Paulston, C.B. (1992). Sociolinguistic perspectives on bilingual education. Multilingual Matters. Pérez-Cañado, M.L. (2012). CLIL research in Europe: Past, present, and future. International Journal of Bilingual
Education andBilingualism, 15(3), 315-341. https://doi.org/10.1080/13670050.2011.630064. Ponciano, L. & Shabazian, A. (2012). Interculturalism: Addressing diversity in early childhood. Dimensions of Early Childhood, 40, 23-30.
Porto, M. (2013). Language and intercultural education: An interview with Michael Byram. Pedagogies: An
International Journal, 8(2), 143-162. https://doi.org/10.1080/1554480x.2013.769196 Ramírez-Verdugo, M.D., & Gerena, L. (2020). Bilingual education from learner perspectives. In M.E. Gómez-Parra, & C.A. Huertas-Abril (Eds.), Handbook of research on bilingual and intercultural education (pp. 408-426). IGI Global. https://doi.org/10.4018/978-1-7998-2588-3.ch017 Ramos-García, A.M. (2011). The cultural knowledge of monolingual and bilingual studies. In D. Madrid, & S. Hughes (Eds.), Studies in bilingual education (pp. 223-235). Peter Lang. https://doi.org/10.3726/978-3-0351-0237-6
Reljic, G., Ferring, D., & Martin, R. (2015). A meta-analysis on the effectiveness of bilingual programs in
Europe. Review of Educational Research, 85(1), 92-128. https://doi.org/10.3102/0034654314548514 Rodríguez-Pujadas, A., Sanjuán, A., Fuentes, P., Ventura-Campos, N., Barrós-Loscertales, A., & Ávila, C. (2014). Differential neural control in early bilinguals and monolinguals during response inhibition. Brain & Language, 132, 43-51. https://doi.org/10.1016/j~.bandl.2014.03.003 Roquet, H. & Pérez-Vidal, C. (2015). Do productive skills improve in content and language integrated learning
contexts? The case of writing. Applied Linguistics, 38(4), 489-511. https://doi.org/10.1093/applin/amv050 Romanowski, P. (2018). CLIL's role in facilitating intercultural learning. Applied Linguistics Papers, 25(2), 71-87.
https://doi.org/10.32612/uw.25449354.2018.2.pp.71-87 Samovar, L.A., Porter, R.E., McDaniel, E.R., & Roy, C.S. (2013). Communication between Cultures (8th ed.). Cengage Learning.
Santos Rego, M.A., & Lorenzo Moledo, M.d.M. (2005). Promoting interculturality in Spain: Assessing the Uue of the jigsaw classroom method. Intercultural Education, 16, 293-301. https://doi.org/10.1080/14675980500212020 Schluessel, E.T. (2007). Bilingual education' and discontent in Xinjiang. Central Asian Survey, 26(2), 251277. https://doi.org/10.1080/02634930701517482 Senge, P. (2010). Education for an interdependent world: Developing systems citizens. In A. Hargreaves, A. Lieberman, M. Fullam, & D. Hopkins (Eds.), Second international handbook of educational change (pp. 131152). Springer. https://doi.org/10.1007/978-90-481-2660-6_8 Sercu, L. (2010). Assessing intercultural competence: More questions than answers. In A. Paran, & L. Sercu (Eds.), Testing the untestable in language education (pp. 17-34). Multilingual Matters. https://doi. org/10.21832/9781847692672-004
Short, D.J. (1994). Expanding middle school horizons: Integrating language, culture and social studies. TESOL
Quarterly, 28(3), 581-608. https://doi.org/10.2307/3587309 Soubelet, A. & Salthouse, T. (2011). Influence of social desirability on age differences in self-reports of mood
and personality. Journal of Personality, 79(4), 741-62. https://doi.org/10.1111/j.1467-6494.2011.00700.x Steinberg, L. & Monahan, K.C. (2007). Age differences in resistance to peer influence. Developmental Psychology, 43,
1531-1543. https://doi.org/10.1037/0012-1649.43.6.1531 Straffon, D.A. (2003). Assessing the intercultural sensitivity of high school students attending an international school. International Journal of Intercultural Relations, 27, 487-501. https://doi.org/10.1016/s0147-1767(03)00035-x
Tabatadze, S. (2015). Factors influencing the effectiveness of bilingual educational programs: The prospects of pilot programs in Georgia. Sino-US English Teaching, 12(2), 93-109. https://doi.org/10.17265/1539-8072/2015.02.003
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859-883. https://
doi.org/10.1037/0033-2909.133.5.859 Tsung, L. T. H. (2009). Minority languages, education and communities in China. Palgrave Macmillan. https://doi.
org/10.1057/9780230234406 Uzum, B., Akayoglu, S., & Yazan, B. (2020). Using telecollaboration to promote intercultural competence in teacher training classrooms in Turkey and the USA. ReCALL, 32(2), 162-177. https://doi.org/10.1017/ s0958344019000235
Vivo, S., Saric, D., Muñoz, R., McCoy, S., López-Peña, P., & Bautista-Arredondo, S. (2013). Guía para medir comportamientos de riesgo en jóvenes [Guide to measure risk behaviors in young people]. Banco Interamericano de Desarrollo (BID).
Appendix
Questionnaire for the Measurement of Intercultural Practices within Bilingual Secondary Schools
Dear student,
This questionnaire has been purposefully designed to analyse and measure intercultural practices within bilingual secondary schools. Therefore, we are neutrally seeking to find out if these practices occur and, if so, to what extent. This study is being conducted by a research team from the University of Córdoba (Spain) and the University of Maryland, Baltimore County (Baltimore, USA). Please, do not write your name on this questionnaire. Your responses will be anonymous and will never be linked to you personally. Your participation is entirely voluntary. If there are items you do not feel comfortable answering, please skip them.
Thank you for your cooperation.
This is the description of the Likert scale that we have used:
Likert Scale (A) Likert Scale (B)
1. Strongly Disagree (SD) a.No
2. Disagree (D) b. Barely / A1
3. Somewhat Disagree (SWD) c. Not very good / A2
4. Somewhat Agree (SWA) d. Reasonably good / B1
5. Agree (A) e. Good/ B2
6. Strongly Agree (SA) f. Very good / C1
We thank you for your participation and we ask you to answer sincerely to the questions.
Name of the school:____
Country of origin:____
Course:______
Gender:Male DFemale □
Age:__
1 2 3 4 5 6
SD D SWD SWA A SA
1. I can describe interactions of my culture with another culture (e.g., regarding clothing, appearance, music, food, among others).
2. If necessary, I adapt my behaviour to an intercultural encounter when I interact with foreign peers.
3. I show interest in new cultural behaviours.
4. I take on various roles to adapt to different cultural situations.
5. I respect the rights of those students who come from a different culture to have their own (cultural and religious) values.
6. I am able to use strategies (e.g., observation or seeking for support) to adapt successfully to a second culture.
7. I am able to use communicative strategies to facilitate interactions with foreign peers.
8. I am able to solve cross-cultural issues if they arise during interactions.
1 2 3 4 5 6
SD D SWD SWA A SA
9. I am aware of the number of foreign peers in my classroom.
10. I am aware of being myself a culturally conditioned person.
11. I am aware of the fact that language and culture mutually influence each other.
12. I am aware of the reactions (both positive and negative) of others towards my cultural values.
13. I am aware of my own intercultural development.
14. Teachers promote interactions with foreign peers in the classroom.
15. Grouping is generally arranged by including members of diverse nationalities within the same group.
16. Cooperative learning is a common approach for activities.
17. Communication among peers is conveyed through our common mother tongue.
18. Communication among peers is conveyed through my peer's language.
19. Teachers promote the learning of another culture in the classroom by arousing questions on food, weather, language, customs, etc.
