CC BY
6
Joldasbaeva P. english teacher Kimyo International University in Tashkent
TEST MODIFICATION PROJECT
Annotation. The teaching and learning processes are both significantly influenced by tests. It is essential for teachers to employ tests in order to identify and assess whether or not language learning and teaching goals have been met. The current Test Modification Project aims to analyze one of the current grammar tests, identify the test's shortcomings, and adapt it in accordance with five fundamental principles. The project is significant for English language instructors because it provides an opportunity for them to evaluate assessment tools by applying their theoretical knowledge in practice and to adjust them to the needs of their students.
Keywords: formative assessment, summative assessment, modified version, validity, practicality of the test, authentic test, wash back.
Introduction
The following work studies the assessment task, in this case, it is a grammar test, that an English Language Learner (ELL) took recently. The main objectives of the paper are (a) to describe this ELL learner, her language background, the educational setting; (b) to give a critique of a selected language test and (c) to modify at least five items in the original language test. The importance of this article for English language teachers is that they can identify different biases and flaws, which exist in language assessments and prepare proper instruction that scaffolds their learners when they take similar tests. The paper analyzes the chosen test according to five principals of language assessment in order to determine the weaknesses of this test and improve them in a modified version.
1. Learner Assessment Profile
a) The learner
One of the students in an elementary class, who is at the age of 18, was chosen for this assignment. Her name is Gizatullina Adelina and currently she is in her third year at the vocational college specialized in computer technology, which is situated in Mirabad district, Tashkent, Uzbekistan. Since Adelina is already eighteen years old, she herself signed the consent form of this assignment.
Adelina is Tatar by nationality and she has been living in Tashkent since she was born. Although she is Tatar, her native language is Russian as all members of her family and people around her speak in this language and only know few words in the Tatar language. Moreover, she attended the Russian school № 147 in Tashkent, where she also started to learn English, but at that
time the Grammar Translation Method (Richards & Rodgers, 2001), was popular in their school and they were mostly assessed by traditional types of assessments such as multiple-choice tests.
This learner came to the class five months ago to learn English and during the needs analysis conducted with her, it was found that she would like to learn English for general purposes like travelling, communication in social network, understanding native speakers, watching English videos and many others.
Moreover, it is necessary to mention that Adelina has not taken any proficiency tests such as IELTS or TOEFL before. However, when she came to the language center, she took the placement test, which is aimed to test only grammar and vocabulary. The administrators gave her beginner-level test that consisted of twenty multiple-choice questions. After taking her results, which was fourteen right answers out of twenty, one of the instructors of the language center interviewed her. Finally, the administration decided to place Adelina in a beginner-level group of the General English course.
b) The educational setting
When it comes to the educational setting, it is a private language center, which is focused on teaching only English. This center was founded in 2016 by young initiative team and at present time there are several English courses in this center such as General English, IELTS preparation, Business English and English for kids. Approximately 150 students from different language backgrounds are studying in these courses now. However, the number of instructors is rather small with only nine non-native teachers. All of these teachers have IELTS certificates with different scores ranging from six to eight and most of them have over six years teaching experiences.
c) The most recent placement test
As it was mentioned before, in the target educational setting, there are special placement tests, which were developed by the administrators of this center. These are multiple-choice tests, which were designed to test grammar and vocabulary of learners for each level like beginner, intermediate, advanced. After getting students' results, the administrators check their speaking through the interview to make the ultimate decision about what course the learners may attend. However, these placement tests are only for defining grammar and vocabulary levels of students, who come to the language center for the first time and for placing them on a right class. That is why, at the end of each level, most teachers prepare the final tests for learners in order to move to the next level.
In the case of the target learner, as it was mentioned above she is now studying in an elementary group after having finished her beginner class. Before moving to the elementary class, she passed the final test, the results of which placed her into this group. This final test had several tasks on all four skills like reading, listening, writing and speaking, and two language aspects such as grammar and vocabulary. In particular, there were two listening tasks with ten items in each; two reading tasks that have ten questions in total, one grammar
and one vocabulary task. At the end of this test, students had one writing task, where they had to write an e-mail to their friends and one speaking task, where they should have to work in pairs and make arrangements to go out at the weekend. The overall score for this test was a hundred points and Adelinas result was eighty-five points, which meant that she completely finished the beginner level. Here, it can be seen that this kind of final test may be the placement test for the next level as well.
2. Critique of an existing test/assessment
a) Description of a chosen test/assessment
The target student of above-named elementary group, Gizatullina Adelina, continued to learn English from the textbook "Solutions Elementary", the authors of which are Tim Falla and Paul A. Davies. It is the third edition of this textbook that was published by Oxford University Press in 2018. There are usually short tests to check students' comprehension of grammar and vocabulary after each unit. These short tests consist of two grammar and two vocabulary tasks with five items in each. One point or mark is given for each correct answer.
Having finished the first unit of this textbook, Adelina also took this short test. Since this paper is going to discuss about the grammar test and its modification, only grammar part of the short test is taken to analyze. Figure 1 below illustrates this short grammar test.
Figure 1
2 Put the words in the correct order to make questions. Use do I does.
b) Analysis of the test considering 5 principles of assessment
When it comes to the evaluation of any assessment task, we try to find out whether this task is useful, appropriate, or effective. In order to identify this, firstly, we should find answers to the following questions "Can it be given within appropriate administrative constraints? Is it dependable? Does it
accurately measure what you want it to measure? Is the language in the test representative of real-world language use? Does the test provide information that is useful for the learner?" (Brown & Abeywickrama, 2010, p. 25). These questions make up the basic five principles of test assessment such as validity, reliability, practicality, authenticity and washback that the author of this paper will address each one here analyzing the chosen test though them.
Practicality
According to Mousavi (2009, p. 518, as cited in Brown & Abeywickrama, 2010) practicality includes "costs, the amount of time it takes to construct and to administer, ease of scoring, and ease of interpreting/reporting the results" (p. 26). Moreover, it is one of the principals, which is very important for busy teachers with too little time in a single day, when they need to assess their students. Thus, the practicality in classroom-based testing mainly refers to the time, which is needed for designing, administering and scoring the assessment task. Taking into account these factors, the chosen test is considered as a practical test for both teachers and students since it "can be completed by the test-taker" and scored by the teacher "within appropriate time constraints" (Brown & Abeywickrama, 2010, p. 26).
Reliability
James E. Purpura (2004) wrote in his book "Assessing grammar" that "reliability refers to the precision and consistency with which we are able to measure performance" (p. 149). In other words, when the teacher receives the same results every time when they administer the test, regardless of any conditions, this test is accepted as a reliable one. These conditions will vary according to test, the learner, the rater or scoring process and the test administration. In small classroom-based assessment like the case of this paper, the main reason why tests can be unreliable is that of rater bias. Mostly this occurs with tests, which are subjective with open-ended responses. However, it is not true for this chosen test since it is objective test that has "predetermined fixed responses" (Brown & Abeywickrama, 2010, p. 29), in the given test, these responses are correct form of present simple tense in negative and interrogative sentences.
Validity
As each of the previous principles includes "sub-principle" of their kind, this principle also deals with several types such as content validity, criterion-related validity, construct validity and face validity. However, in terms of the chosen test, it is appropriate to focus on the content validity since this is "the major source for establishing validity in a classroom test" (Brown & Abeywickrama, 2010, p. 42). Further, the same authors describe the content valuable test as an assessment that "requires students to perform tasks that were included in the previous classroom lessons and that directly represent the objectives of the unit on which the assessment is based." (Brown & Abeywickrama, 2010, p. 42). Thus, it can be said that the given classroom test
has the content validity in itself as it assesses students' ability to construct the negative and question form of present simple tense.
Authenticity
Authenticity is the next principle of language testing that the given test of this paper lacks. Regarding authenticity, Brown and Abeywickrama (2010) mentioned that an authentic test should have "items that are contextualized rather than isolated" and it should also "offer tasks that replicate real-world tasks" (p. 37). If we look at the given test, one can obviously see that it is, in some extent, an inauthentic test for the learners because this test has discrete points format, which checks only the correct form of grammar unit. Moreover, the tasks of this test are inauthentic because they are unlikely to be enacted in real world, rather they are mechanical.
Washback
This principle refers to more "classroom-based issues such as the extent to which assessment affects a students' future language development". Additionally, washback can increase "a number of basic principles of language acquisition: intrinsic motivation, autonomy, self-confidence" (Brown & Abeywickrama, 2010, p. 38). When we analyze the given grammar test, it is clearly seen that it may not provide any beneficial washback since the given test is considered as a formal type of assessment, where students receive single overall numerical score. Only scores or grades without appropriate feedback cannot be able to enhance students' language development.
c) Discussion of strengths and weaknesses of the chosen test
On the assumption of the above-given analysis of a selected test through five principles of language assessment, the following strengths and weaknesses of this grammar test are given:
Strengths:
• Practicality
• Rater-reliability
• Content validity
Weaknesses:
• The discrete-point approach
• Mechanical tasks
• Lack of positive washback
With regard to the strengths of this grammar test, it can be clearly seen that this test is practical for both a teacher and a learner since it does not take much time from students to complete it and from teachers to score. Teachers score this test by giving a single numerical score that is one mark or point for each correct answer. Consequently, it leads to the next strengths of this grammar test, which is rater-reliability. Moreover, this test checks students' grammar ability to write sentences correctly in negative and interrogative forms of present simple, which was explained by the teacher in previous unit. Thus, another strength of this test is that it has content validity.
In the matter of the weaknesses, it is important to say that this grammar assessment has the discrete-point approach that is according to Brawn and Abeywickrama (2010) "presupposed a decontextualization that was proving to be inauthentic" (p. 13). In addition to this, tasks of this grammar test are rather mechanical, which students will not be able to use in real life. Moreover, after taking this grammar test, students are not provided with any feedback that will develop students' language learning, which in its term generates washback issues.
3. Modification of the chosen test or assessment a) Suggestions for improvement
Aforementioned three flaws of the selected grammar test led to some ideas for improvement of this test. In searching ways of including these ideas into the modified version of the given test, the author of this paper came across James Purpuras (2004) "a learning-oriented approach to assessment of grammatical ability", which he mentioned in his book "Assessing grammar":
In pursuing a learning-oriented approach to grammar assessment, language instructors teaching students at different proficiency levels need to consider the degree to which grammar assessments: (1) are aligned with the learning goals of the course, the curriculum or some external performance standards; (2) provide accurate, appropriate and meaningful information about what grammar the learners know and can use, and what grammar they need to improve; (3) use tasks that resemble those that test-takers are likely to encounter in real-life or instructional situations; (4) succeed in engaging test-takers in the constructs they are trying to measure. Outside the learning focus, teachers should naturally strive to be consistent and fair in their scoring and grading, and they should try to minimize the costs of constructing, administering, scoring and providing useful feedback. (pp. 213-214)
This author also provided with some questions that a learner-oriented approach to grammar assessments would address and three of them are related to this test modification project. Therefore, the author of this paper suggests as a way of improving the given flaws of the original test to find answers for these three questions, which are (1) "How can I design interesting and cognitively engaging grammar tasks so my students will enjoy learning grammar?" (2) "How do I know if my students can use these grammar points to communicate spontaneously in real-life situations?" (3) "How can I use grammar assessment results to provide feedback for guiding learners?" (Purpura, 2004, p. 214).
b) A design of a modified version of the test with new instructions
Figure 2
Speaking Interview "Find interesting information"
Instruction: Ask five or six of the following questions from your partner. Write their answers in short form. Then, report your results back to the class.
1. Does your mother walk to work?
2. Do you like Sundays?
3. Do you finish school/work early today?
4. Does your brother or sister get up before you?
5. Do your friends use computers?
6. Do you ride a bike to school/to work?
7. Do your friends speak English?
8. Does your sister or brother play the piano?
9. Does your father meet his friends? If yes, when does he usually meet them?
10. Do you live near the city center?
c) Test administration and analysis of outcomes
The modified version of the original test was successfully administered since abovementioned three weaknesses of the test had been completely eliminated. As a result, the original test, which only checks the grammar ability of a student, became a test that integrates grammar task with other language skills of reading, listening, writing and speaking in some extent. Although all of Gizatullina Adelina's answers for the original grammar test items were almost correct in a written form, she needed more authentic task to practice that grammar construction. After taking the modified version of the original test, Miss Gizatullina had an opportunity to practice her grammar knowledge of the present simple tense in the speaking performance. In addition to this, she was provided with oral comments from the teacher and other students, which, in its turn, helped to identify her own mistakes in using grammar structure and not to repeat them in future.
d) Comparison of original and modified version of the test
In the original test, Gizatullina Adelina did two grammar tasks, the first of which asked her to rewrite the given positive sentences in a negative form of present simple tense and the second was about putting the words in the correct order in order to make the question form of the same tense using auxiliary verbs of do and does. The teacher scored this test giving one mark for each correct answer. However, in the modified version of the test, these two grammar tasks were changed into speaking interview task called "Find interesting information" and all of the items of the original test became questions in a present simple tense that Adelina should have to ask her partner and write partner's answers in
a short form. After getting results, she had to report them back to the classroom. Thus, after these changes, it can be said that now the modified version of the test checks grammar knowledge of a student in an integrated way including listening, writing and speaking skills while the original test had a discrete-point approach with mechanical tasks. When it comes to the scoring process of this task, teacher decided not to use a score at all instead the instructor provided Adelina with oral comments about her use of grammar structure based on focused observation throughout her performance. This means that the modified version of the test provides beneficial and positive washback, which the original version of the test does not have. Based on these changes, now it can be said that the three weaknesses of the original test, which are (1) having a discrete point approach (2) having only mechanical tasks and (3) not having positive washback were eliminated in the modified test and the content validity, rater-reliability of the original test also remained in the modified version. Since these tests were small-classroom assessment tasks, both of the tests are still considered as practical.
Conclusion
In conclusion, it is necessary to mention that there were several reasons to make changes into this original grammar test. According to John Oller (1979, as cited in Brown & Abeywickrama, 2010), the language is "a unified set of interacting abilities that could not be tested separately". It means that such integration cannot be tested in only discrete grammar assessment task. Therefore, the author of this paper decided to integrate this grammar task with other four language skills of reading, listening, writing and speaking. Moreover, Bachman and Palmer (1996) said, "in order for a particular language test to be useful for its intended purposes, test performance must correspond in demonstrative ways to language use in non-test situations" (p. 9). That is why the mechanical tasks of the original test were changed into more authentic task, where the learner should have an interview with her pair. Additionally, it has been stated that the chosen test does not provide any washback while the modified version has positive washback in the form of teacher's oral comments on the learner's performance since, according to Brawn and Abeywickrama (2010), "one way to enhance washback is to comment generously and specifically on test performance" (p. 38).
References:
1. Bachman, L. F. & Palmer, A. S. (1996). Language testing in practice. Oxford University Press.
2. Brown, H. D. & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices. White Plains, NY: Pearson Education.
3. Richards, J., & Rodgers, T. (2001). Approaches and Methods in Language Teaching. Cambridge: Cambridge University Press.
4. Purpura, J. (2004). Assessing grammar. Cambridge: Cambridge University Press.
