CC BY-ND
79
Journal of Language & Education
Volume 1, Issue 2, 2015
Olympiad in the English language as a form of alternative language assessment
Evgeniya Bolshakova
National Research University Higher School of Economics
Although a variety of the English language written olympiads (language competitions) exist, fairly little is known about how they are different from traditional forms of language assessment. In Russia, olympiads in the English language are now gaining currency because they provide an opportunity to reveal creative thinking and intellectual abilities of pupils. The present study examined major differences between language olympiads and traditional forms of language assessment. A comparison of five main olympiads in the English language in terms of their levels, assessed skills and task types is made and their distinctive features are outlined. The results of a testing of a new written olympiad of the Higher School of Economics “Vy-sshaya proba” (Highest Degree) in the English language are analyzed. A set of test items was developed for 120 secondary school pupils in Moscow to find out whether they can easily cope with non-traditional form of assessment, which is language olympiad. The results indicate that language competition as a form of alternative assessment may be introduced at schools to encourage better learning.
Keywords: language assessment, language olympiad, alternative assessment
Introduction
Participation in school and regional olympiads in the English language is gaining currency among pupils, which indicates the growing recognition of this form of skills assessment. This may be explained in several ways. On the one hand, such language competitions make it possible to reveal and develop pupils’ creative abilities, intellectual potential and unconventional approach to performing tasks. On the other hand, pupils with a diploma of a winner/ prizewinner of the final stage of Russia’s National Language Olympiad gain an advantage when they enter a university (Federal law of December, 29, 2012, p. 218), which encourages them to strive for good results.
Evgeniya Bolshakova is an English lecturer of the Foreign Languages Department at the National Research University Higher School of Economics, Moscow, Russia. She is involved in ESP teaching and alternative assessment design. Her interests are in the field of assessment, note-taking and teaching methodology.
E-mail: [email protected]
Despite the fact that olympiads in the English language are varied in forms and content, nowadays there is little understanding of the differences between language olympiads and traditional forms of language assessment. In the present article an attempt is made to analyze major Russian olympiads in the English language from a perspective of alternative assessment in order to define skills being assessed, variety of task types being used and their level of difficulty. Apart from that, the results of a testing of a new olympiad in the English language “Vysshaya proba” (Highest Degree) developed by the researchers of the Higher School ofEconomics are described and analyzed.
Literature review
McNamara (2006) who was one of the first to propose a notion of alternative approach to assessment, defines assessment as “any deliberate, sustained and explicit reflection by teachers (and by learners) on the qualities of a learner’s work can be thought of as a kind of assessment” (p. 343). In other words, assessment can be considered a reflection of instruc-
6
OLYMPIAD IN THE ENGLISH LANGUAGE AS A FORM OF ALTERNATIVE LANGUAGE ASSESSMENT
tional objectives. It is noted that in an era of educational reform educators prefer alternative forms of assessment (portfolios, open-ended questions, computer simulations) to traditional multiple tests (multiple choice, fill-in-the-gaps, matching) (Dietel, Herman, and Knuth, 1991). Compared with traditional assessment, the alternative one “takes into consideration the learning of each student, as well as each student’s cultural background and level of knowledge” (Chirim-bu, 2013, p. 93).
As stated by Brown and Hudson (1998), there is no unique definition of alternative assessment and no common view on the characteristics of alternative assessment has been identified. However, by combining viewpoints of several researchers, they have come up with a list of positive characteristics for alternative assessments. Thus, alternative assessments are said to be those that:
1. “require students to perform, create, produce, or do something;
2. use real-world contexts or simulations;
3. are nonintrusive in that they extend the day-today classroom activities;
4. allow students to be assessed on what they normally do in class every day;
5. use tasks that represent meaningful instructional activities;
6. focus on processes as well as products;
7. tap into higher level thinking and problem-solving skills;
8. provide information about both the strengths and weaknesses of students;
9. are multiculturally sensitive when properly administered;
10. ensure that people, not machines, do the scoring, using human judgment;
11. encourage open disclosure of standards and rating criteria; and
12. call upon teachers to perform new instructional and assessment roles” (Brown, Hudson, 1998, pp. 654-655).
While traditional assessment puts more emphasis on indirect and standardized tests, alternative assessment involves higher level of students’ cognitive skills and makes them create or produce something. At the same time alternative assessment involves tasks on activities that are familiar to students and are process-oriented, which may imply that it involves a more advanced and individual approach to solving educational tasks.
According to Coombe and Hubley (2009), alternative assessment provides alternatives to traditional testing as it “does not intrude on regular classroom activities, reflects the curriculum that is actually being implemented in the classroom... and provides multiple indices that can be used to gauge student
progress" (Coombe, Hubley, 2009, p. 44). Stating that alternative assessment is continual, authentic and classroom-based form of testing, Coombe and Hubley name self-assessment, portfolio-assessment, student-designed tests, projects and presentations among typical forms of alternative assessment. However, among the forms of alternative assessment, such as debates, projects, think-alouds, checklists, progress cards (Tsagari, 2004), no form of language olympiad is mentioned. Therefore, this warrants research into this form of alternative assessment.
The study was designed to answer the following research questions: How easily will students cope with unconventional tasks in a language olympiad? What difficulties will they encounter? The researcher assumes that secondary school pupils, being largely exposed to a variety of language olympiads, will show high results and perform most of the tasks with ease. However, the researcher assumes that logical thinking and phonetic skills may be most problematic with students.
Method
What is language olympiad?
By language olympiad (language competition) the researcher implies a written form of alternative language assessment which aims at evaluating secondary school pupils’ knowledge in the English language. The English language olympiads in Russia are held at several levels: locally, regionally and nationally. Such competitions not only allow monitoring the efficiency of the English language instruction at schools but also assist in selecting the best university entrants.
Although there exists a variety of olympiads in the English language, not all of them assess the same skills. Within the framework of the present article, the researcher focused on the analysis of five major regional competitions in the English language:
- Hertsen language olympiad of pupils;
- Inter-regional “Eurasian Linguistic Olympiad”;
- “Pokori Vorobyevy gory” language olympiad;
- Saint-Petersburg State University language olympiad;
- “Lomonosov” competition in the English language. The choice of language olympiads being analyzed
is based on the following criteria: a) a language olympiad is included in the national check list of language competitions b) it is approved by the Russian Council for Pupils’ Olympiads; and c) task items are intended for revealing creative and intellectual abilities of pupils in the field of a foreign (English) language, and not in the field of pure linguistics.
7
EVGENIYA BOLSHAKOVA
Table 1
The comparison offive majorRussian olympiads in theEnglish language
Hertsen Eurasian Pokori Vorobyevy gory Saint-Petersburg Lomonosov
Area trans-regional trans-regional trans-regional trans-regional trans-regional
Number of stages 2 (elimination (remote) and final (in-person) 2 (elimination (remote) and final (in-person) 2 (elimination (remote) and final (in-person) 2 (elimination (remote) and final (in-person) 2 (elimination (remote) and final (in-person)
Grades 8-11 7-11 5-9,10-11 10-11 10-11
Variety of task types Tasks test all types of language skills but are conventional Tasks are unconventional and cover a wide range of problematic areas. Most of the tasks are varied and unconventional, but are uniform in content. Tasks are unconventional and cover a wide range of problematic areas. Tasks are unconventional and cover a wide range of problematic areas.
The five language competitions were compared in terms of the area where they take place (local, trans-regional, national), the number of stages for pupils to take (remote and/or in person), the grades of secondary school pupils allowed to take part in a competition (a typical Russian secondary school covers pupils aged 12 to 17 years old and comprises grade 5 to grade 11), and a variety of task types. The results of the analysis are presented in Table 1.
According to Table 1, it can be inferred that all language olympiads under consideration comply with certain standards. Firstly, they are all trans-regional which means that they assess pupils in all major Russian regions. Secondly, they are normally comprised of two stages (elimination and final): the elimination stage is usually held online while the final stage is an in-person testing. As for the grades, these olympiads are designed mainly for the pupils of grade 9 to grade 11. Finally, the task types are generally varied and unconventional, covering a wide range of problematic areas. Thus, these are the guidelines that the researcher took into account while considering the format of a new language olympiad “Vysshaya pro-ba” which will be discussed further in the article.
In spite of the fact that all language competitions under discussion are approved by the Russian Council for Pupils’ Olympiads, they do not assess the same skills in the same format. In order to find out what differences they have in terms of assessed skills, the task types of each language olympiad were analyzed. The findings are presented in Table 2. The first column depicts a list of skills which were derived from the assessment tasks. A “+” at the intersection of a skill and an olympiad indicates that an olympiad includes a task designed to assess this skill.
The results of the analysis indicate that the tasks in language olympiads are designed to assess a wide
range of pupils’ skills. Knowledge of lexical and grammatical forms of words, an ability to identify appropriate collocations, reading for gist and writing an essay on a particular topic are core skills that are assessed in language olympiads. They may be selected for several reasons. First and foremost, these skills make the base of almost any international English language exam. Secondly, the tested skills cover the areas of grammar and vocabulary, reading and writing, which allows administrators to see pupils’ performance in a complex set of skills. However, the tasks aimed at assessing pronunciation skills, logical thinking and awareness of idiomatic expressions are not widely used or are not used at all. These skills may be under-assessed because they are more advanced and require a higher level of cognitive skills, which makes them challenging to complete even for the most proficient pupils. More than that, the universities responsible for designing language olympiads want to select their “target” entrants who can cope with the tasks that are close to a university student level and ensure that the entrant is able to manage the future university programme.
Participants
In September 2014, a group of language teaching methodology specialists of the Higher School of Economics was set a task of designing a new olympiad “Vysshaya proba” (Highest Degree) in the English language for Russian pupils. In two months the tasks were developed and then tested on 120 pupils of Moscow secondary schools. The participants were selected according to several principles. First of all, they had to give their informed consent, saying they voluntarily agree to take part in the approbation. Besides, the pupils had expressed particular interest in testing the ol-
8
OLYMPIAD IN THE ENGLISH LANGUAGE AS A FORM OF ALTERNATIVE LANGUAGE ASSESSMENT
Table 2
The comparison ofassessedskills in 5 trans-regional olympiads in theEnglish language
A skills checked Hertsen Eurasian Pokori Vorobyevy gory Saint- Petersburg Lomo- nosov
ability to identify sound-letter correlations +
ability to put correct stress in a word +
ability to identify similar pronunciation patterns of words +
ability to form verbal forms of words +
ability to put appropriate punctuation marks +
ability to select correct lexical and grammatical form of a word + + + + +
ability to find grammatical and lexical errors in a sentence + + +
ability to identify an appropriate collocation + + + + +
reading for detail + + +
reading for gist + + + +
ability to select synonyms and antonyms to given words +
ability to identify evocative nuances of text fragments + +
ability to define whether statements to the text are true, false or not given + + +
awareness of national features of the English speaking countries + + +/-
ability to write an essay on a particular topic + + + + +
ympiad. Apart from that, pupils of these schools are potential entrants of the Higher School of Economics. The tests of “Vysshaya proba” took place in four schools, two of which were specialized in the English language teaching and learning and two of which were ‘ordinary’ schools with no emphasis on studying particular subjects. The test version of this language competition was designed for 9th graders, 10th graders and 11th graders separately. The testees were allowed to use the Internet while accomplishing tasks as the first stage of any language olympiad is on-line based elimination test.
Material: a test
A test included tasks aimed at assessing pupils’ skills in phonetics, grammar, collocations, reading, logical thinking and awareness of cultural character-
istics of the English speaking countries. A test was divided into three sections: “Grammar and Vocabulary”, “Reading and Logical thinking” and “Cultural awareness”. An example of each section will be provided below.
The first section “Grammar and Vocabulary” consisted of a number of task items designed to assess pupils’ knowledge in the fields of phonetics, collocations and idioms, grammar rules and error identification.
E.g. (GradelO)
Read a text and decide which idiom (1, 2, 3) best fits the sentence in bold.
From the very moment of Business School graduation, Mary has been dreaming about establishing her own company. She filled in all the necessary documents, chose the location, and even consulted a de-
9
EVGENIYA BOLSHAKOVA
signer to arrange her office. She is obsessed with her idea.
1) she has a bee in her bonnet
2) she is biding her time
3) she bites off more than she can chew
The second section “Reading and Logical thinking” was comprised of the tasks that assessed pupils’ ability to read for gist and for detail as well as it tested their logical thinking and argumentation skills.
E.g. (Grade 11)
Read a sentence and decide whether it contains logical fallacy or not. If it does, choose a fallacy type from the list below.
You either did knock the glass over, or you did not. Which is it?
A No logical mistakes
В False Dilemma (Giving two choices when actuality there could be more choices possible)
C Red Herring (Introducing a topic not related to the subject at hand)
D Appeal to the Popular (Urging the hearer to accept a position because a majority of people hold to it)
E Cause and Effect (assuming that the effect is related to a cause because the events occur together)
F Fallacy of Division (Assuming that what is true of the whole is true for the parts)
G Attacking the individual instead of the argument H Weak Analogy
The final part “Cultural awareness” tested pupils’ awareness of conventions, cultural habits, and knowledge of the famous people of the English speaking countries.
E.g. (Grade9)
Read a short text and decide what person from the list this is.
_____is an American singer who revolutionized
American pop music and became an idol for millions of people all over the world in the 1950s and 1960s.
A Edward the Confessor В Rudyard Kipling C Charles Dickens D Elvis Presley E Arthur Conan Doyle F EmilyBronte G Sean Connery H William the Conqueror I Michael Jackson J Tom Hanks К Henry VIII L JackLondon
Depending on a grade, a test could be comprised of a varied number of tasks. For ‘ordinary’ second-
ary schools which do not specialize in any subject a test included a minimum of task items, while in the schools specialized in the English language teaching and learning a test could consist of a maximum of task items assessing a wider range of skills. Moreover, timing for different grades was different. For pupils studying in ‘ordinary’ secondary schools a test lasted for 45 minutes while in the ‘specialized’ ones it could take up to 90 minutes.
Results
Generally, the pupils of both “ordinary” and “specialized” schools demonstrated good results at the testing of a language olympiad “Vysshaya pro-ba” (Highest Degree), which correlates with the assumption of the first research question. In “ordinary” schools pupils did well in “Grammar and Vocabulary” section but failed at tasks that assessed logical thinking and argumentation. As for “specialized” schools, the pupils showed good control of grammar rules and argumentation skills. However, the results of vocabulary and cultural awareness parts indicated poor knowledge of collocations, idioms and cultural characteristics of the English speaking countries. The results of the test are summarised in Table 3.
The results have also indicated that pupils were not ready to cope with phonetic tasks, which supports the assumption that teachers do not place emphasis on revising phonological features and pronunciation patterns of words on higher stages of the English language acquisition. Apart from that, the tasks designed to assess logical thinking turned out to be the most challenging ones. The researcher assumes that it may be explained due to the lack of practice and absence of the tasks that drill this skill in modern English textbooks used at schools. However, knowledge of grammar rules and reading skills were found to be well developed. These two components (grammar and reading) along with writing and speaking form the basis of the Unified State Exam in the English language aimed at assessing 11th graders’ performance in English. It is taken at the end of a school year, its results equalling an entrance exam in English at a university.
The results of the approbation also demonstrate that pupils who are encouraged to develop their critical thinking and problem-solving skills have almost no problems with lower-level tasks. Apart from that, it turned out that pupils with average results demonstrate high potential and an ability to make progress. A new language olympiad, designed as a form of alternative assessment, displays both strengths and weaknesses of pupils and may help to eliminate the latter. As for the second research question, the findings reveal that the main difficulties that the students
10
OLYMPIAD IN THE ENGLISH LANGUAGE AS A FORM OF ALTERNATIVE LANGUAGE ASSESSMENT
Table 3
The results ofa testing
100^
80 —
60
49
40 —
20
44
Phonological Idioms and
features and collocations
pronunciation patterns
0
Control Reading
of grammar for gist
rules
95
83
48
42
37
Reading Logical Argumenta- Cultural char-
for detail fallacies tion acteristics
I Ordinary (out of 100%)
Specialized (out of 100%)
encountered were connected with insufficient experience of dealing with unconventional tasks and, surprisingly, lack of time. In short questionnaires distributed right after the test the pupils were asked to share their impressions. The majority said they needed another 15-20 minutes to complete the tasks successfully.
Conclusion and future research
An olympiad in the English language may be considered a form of alternative assessment for several reasons. Firstly, it allows varying traditional classroom tests (multiple-choice, fill-in-the-gaps, matching), thus introducing unconventional approaches to solving educational tasks. Secondly, it may encourage teachers to pay more attention to the development of critical thinking and logical skills which may be applied in various school subjects. Finally, the use of language competitions in the English language in the classroom may help to train potential participants of local, trans-regional or national language olympiads. The expedience of exploiting language olympiads in the classroom should be considered by administrators and teachers as this form of alternative assessment can be implemented with both average pupils with poor knowledge of the subject to ignite their potential and with
proficient learners to further develop their skills. This study would have been enhanced with more research data on the essence of language olympiads, the concept of which is not widely spread in other countries of the world. It would also be valuable to invite schools in other regions of Russia to participate in the testing in order to see how the results would differ in the centre of Russia and in the outlying regions.
References
Brown, J.D., & Hudson, T. (1998). The Alternatives in Language Assessment. TESOL QUARTERLY, 32(4), 653-675.
Chirimbu, S. (2013). Using Alternative Assessment Methods in Foreign Language Teaching. Case Study: Alternative Assessment of Business English for University Students. Scientific Bulletin ofthePolitehnica University of Timisoara Transactions on Modern Languages, 12(1-2). Retrieved October 4, 2014 from http://www.cls. upt.ro/files/maria-nagy/SITE/Buletinul%20Stiintif-ic/08%20Chirimbu.pdf
Coombe, C., & Hubley, N. (2009). An introduction to key assessment principles. In C. Coombe, P. Davidson, & Lloyd (Eds.), Fundamentals of Language Assessment: A Practical Guide for Teachers (2nd ed.), Dubai, UAE: TESOL Arabia Publications, 3-10.
11
EVGENIYA BOLSHAKOVA
Dietel, R. J., Herman, J. L., & Knuth, R. A. (1991). What does research say about assessment? NCREL, Oak Brook. Retrieved October 21, 2014 from http:// methodenpool.uni-koeln.de/portfolio/What%20 Does%20Research%20Say%20About%20Assess-ment.htm
Federal law of December, 29, 2012 №273-FL “On the education in Russian Federation”. Retrieved October 18, 2014 from ЬНр://минобрнауки.рф/ документы/2974/файл/1543/12.12.29-ФЗ_Об_ образовании_в_Российской_Федерации.рЬГ (in Russ.)
Hertsen language olympiad of pupils. Retrieved Septe-mer 10, 2014 from http://okrug.herzen.spb.ru/olimp/ rgpu-olymp/language/#prerare (in Russ.)
Inter-regional “Eurasian Linguistic Olympiad”. Retrieved Septemer 10, 2014 from http://duhobr.ru/olimp2013/ arhiv.html (in Russ.)
“Lomonosov” competition in the English language. Retrieved Septemer 10, 2014 from http://olymp.msu.ru/ mod/page/view.php?id=16 (in Russ.)
McNamara, T.& Roever, C. (2006). Language Testing: Thesocial dimension. Oxford: Blackwell Publishing.
“Pokori Vorobyevy gory” Language Olympiad. Retrieved Septemer 10, 2014 from https://pvg.mk.ru/media/ archive/2013-2014gg/10-lloch/engl_10-ll.pdf (in Russ.)
Saint-Petersburg State University Language Olympiad. Retrieved Septemer 10, 2014 from
http://olympiada.spbu.ru/index.php/olimpia -da-shkolnikov/arkhiv- olimpiady-shkolnikov- sp-bgu/primery-zadanij-olimpiady-shkolnikov-spb-gu-2013-2014-uchebnogo-goda (in Russ.)
The project of the order of the Ministry of Education in Russia "The confirmation of a list of pupils’ Olympiads and their level for 2014-2015”. Retrieved October 21, 2014 from http://regula-tion.gov.ru/project/18896.html?point=view_pro-ject&stage=2&stage_id=12993 (in Russ.)
Tsagari, D. (2004). Is there life beyond language testing? CRILE WorkingPapers, 58.
12
