CC BY
8
UDK: 05.13.11
Tillayeva N. V.
EFL teacher Webster University in Tashkent DESIGNING AND TESTING THE ASSESSMENT TASKS
Annotation: This project paper aims to analyze mid-term and summative assessment tasks for the students of secondary schools and academic lyceums. Both mid-term exam tasks and summative assessment tasks have been analyzed for the five principles of the assessment.
Key words: summative assessment, validity, reliability, practicality, authenticity, wash back
A 1st- year- student Ulugbek Ziyotov was interviewed to develop a profile of language learner.
I. A language learner profile
Name Ulugbek
Surname Ziyotov
Nationality Uzbek
Language proficiency level A2+
Entrance exam result/placement test result 76/100
Summative assessment test result 84/100
Language proficiency level:
While he was taking entrance exam to the lyceum Ulugbek could perform that his language proficiency level is A2 +. Entrance exam materials was designed by the teachers of the department of foreign languages who got B2 level certificates and 7 scores from IELTS. While designing the test materials the designers strictly stuck to the objectives of National Standard of teaching and learning foreign languages. It is important to note that the entrance exam test tasks included only writing task and it consists of three parts.
In order to observe the learners' progress academic lyceum regularly organizes placement tests. The recent one was the summative assessment for the first term. Ziyotov's result was 84 /100, while the learner's score for entrance exam was 76/100. We can see significant upward difference between two different assessments. Even though for summative assessment test the learner performs a good deal progress I tried to analyze test tasks whether the progress is reliable or not. The reason why I am skeptical on this performance is that if we analyze that entrance exam material we can see that it was only developed to assess writing production and the other skills were ignored while summative assessment for terms focuses on reading comprehension skill, lexical and grammar competence and writing skill.
Analyzing Entrance test tasks: If we measure 5 principles of placement test task we can come the following conclusion:
Entrance exam test/placement test:
Practicali ty V The test is mostly considered to be practical, because it is not excessively expensive and easy to deliver. Moreover, to design the test is not very time-consuming task. It may be considered to be impractical if we take the checking process into account, yet as I mentioned before there are enough certified teachers to assess the applicants writing productions.
Reliabilit y - According to STC four language skills should be assessed. Speaking, Listening, Reading have been ignored.
Validity This principle is suffering because according to the curriculum of both secondary and secondary specialized education system productive and receptive skills and knowledge about language should be covered in teaching and learning process. However, in this assessment procedure the learner is being assessed by only writing production results.
Authentic ity V There are 3 parts of writing test. Questions for all parts are considered to be authentic, it was form of the assessment in which applicants are expected to perform a real world tasks.
Washbac k/Backwa sh As the test assessment process have failed to support applicants with feedback which can motivate them to develop their language skills this principle is likely to be unsuccessful.
II. Critique of an existing language test/assessment
Having analyzed the entrance exam tests in terms of the principles of assessment, this part of the paper is going to insight the summative tests and measure its validity, reliability, practicality, authenticity and wash back. Even though above-mentioned summative assessment covers more skills than entrance exam test tasks, still some drawbacks can be given as an example. As National Standard relies on CLT methods and curriculum emphasizes all four skills and knowledge about language assessment tests should reflect all criteria of the CEFR. Another reason why listening and speaking skills must be included in summative assessment is that most students of this lyceum are going to take their Language Proficiency Level test such as IELTS, TOEFL, STC and etc.
In this part I would like to analyze reading comprehension skill assessment tasks of the summative assessment and I try to answer the questions whether they are relevant to the principles. The following passage and the tasks are taken from the summative assessment tasks
Section B. READING (30 scores)
What would your life be like if you were Albert Einstein. What clothes would be in your wardrobe if you were famous singer? Well now you can discover the answer to all these questions and many more at the Fame Hotel in California. Ten miles outside Los Angeles, the Fame Hotel promises to answer the question "What if?". When you check into the hotel, you choose a room. Each room has a name. There's Clint Eastwood on the secondfloor and Elvis Presley on the third floor. In total, the Fame Hotel has 32 rooms, most of which are named after stars of Hollywood or music. But there are also famous writers (Mark Twain and Agatha Christie) and even some scientists and sports stars, such as Mike Tyson. When you enter the room, you enter the life of that person. There are pictures everywhere. The owner of the hotel has tried to fill the room with objects, clothes, even food that he thinks the stars would have liked. Marilyn Monroe's wardrobe is full of beautiful white dresses, Albert Einstein doesn't have any socks in his wardrobe because the real Einstein never wore them! If you choose Mike Tyson's room, you'll be able to practice boxing in one corner of the room. And there's even a skipping rope too!
I spoke to one guest staying in the Elvis Presley room. "I love this hotel," he said to me. "I wanted to know 'What would Elvis Presley eat for breakfast?' and now I know". That guest eats pancakes and strawberry ice cream every morning, just as Elvis liked to do. On the next table, the Einstein room's guest is eating cabbage soup!
The company plans to open another Fame Hotel in New York next year and there are plans to expand into Europe too. I lookforward to staying in the Winston Churchill suite in London! Write the correct option T (true), F (false) or NG (not given). Number 0 is an example.
1. You can meet famous people in the hotel._
2. Some famous people have stayed at the hotel._
3. The Fame Hotel is near Hollywood. _
4. Guests are able to choose which room they stay in._
5. Each room has a celebrity's signature on the door. _
6. The hotel's owner has tried to make the wardobe authentic._
7. Each guest eats something different in the hotel restaurant._
8. There is also a Fame Hotel in London. _
9. It's on the list of company's plan to build a new hotel in New York_
10. There are already some hotels existing in EUROPE_
Read these film reviews. Which film would be best for each person?
A. BAD LOSER
When he loses the Presidential election, an angry army general asks for $10 billion - or he 'll destroy Los Angeles. But how will he do it? Where is he? Can FBI agent Casey find him in time? If you like James Bond, you'll enjoy this too.
D. KILLER MOUSE
A cartoon about a super-mouse who doesn't like cats - or people. I must say I didn't find it funny or frightening, but if you're under ten, you'll love it.
B. MISSING
This classic black and white film was made in the early 1960s. It tells the story of an Italian woman who finds an old love letter, then travels to East Africa - and disappears. This is a strange film, beautiful to watch, but full of sadness and anger.
E. STAR BORES
90 minutes of crazy comedy in space. If you know the Star Wars, Star Trek and Alien movies, you'll probably understand most of the jokes. I didn't!
C. FOR EVER AND EVER This is a surprisingly good little film. Although all the usual things happen - girl meets boy, girlfalls in love, and so on -
F. LOVE IS...
This is a musical comedy with beautiful people, set on a lovely tropical island. There
it's much more than just a love story. Take someone special to this one - maybe even your wife or husband?
isn't much of a story, but the music's good, and it's quite funny in parts. A film for the whole family, young and old
11. Vicky is interested in older, more unusual films.............
12. Edward and his friends are 13. They like exciting films with lots of action. ...
13. Bob and his girlfriend want to see something romantic.............
14. Carol and Jim want to take their son Tom to the cinema on his 8th birthday.
15. Fiona wants to take her elderly mother to a film with nice music.............
The weaknesses of the assessment task:
1. According to www.englishprofile.org/wordlists/text-inspector the vocabulary in the test is mostly A1 and A2 level vocabulary. We may say that this passage is appropriate to A 2 level students, however, it may not be very challenging to assess B1 level.
Reading Passage 1
ext Inspector
textinspectpr.com Text Inspeclo
tmT~-.......-fl
■d U»t Types Tokens
a
S3
a B
(56 14%) (65 56%)
Reading passage 2
A1 A2 B1 B2 C1
52.6 9% 19.16 % / 7.78 % 4.79 % 0.60 %
2. As the language users may read for gist (main idea), for specific information and for detailed information tasks also should be based on these criteria. However, the provided reading comprehension test tasks only focus on detailed information.
3. Measuring 5 principles of assessment test task we can come the following conclusion:
Entrance exam test/placement test:
Practicality V Economical to delivery, easy to check, instructions are easy to follow. The length of the passages are short enough.
Reliability Even though the instructions are easy to follow test reliability suffers because of more A1 and A2 level words than B1 level words. Test designed poorly.
Validity Validity of the test is fairly low, because test focuses on 1 level lower skills, however according to curriculum formative assessment tasks focus on to develop a learner's language skills towards to B1
Authenticity Authenticity of the test is also under the doubt as the passages seemed to be not very interesting for the learners while they were being assessed. And the information in the passages is not based on daily life
Washback/ Backwash - Student have not been provided with feedback after the assessment. They were just announced the grades.
III. The modified version of the chosen summative assessment.
Selecting reading materials for EFL classes should be carefully thought rather than random. H. D. Brown (2010) divides reading passages for assessing into three genres, academic reading, job related reading and personal reading passages. According to curriculum for Secondary specialized education system which is based on CEFR job related and personal reading passages can be appropriate for the objectives of the course.
Skills for the 21st-century workplace Read the passages below and match with the suitable headlines from the box
Communication skills, Decision making, Imagination, Critical analysis
Have you got the skills you needfor the 21st-century workplace?
We need to develop all kinds of skills to survive in the 21st century. Some, like ICT skills and knowledge of the digital world, are taught explicitly in schools in the UK. Here are five less obvious ones for you to think about. These are the sorts of skills that employers may ask you questions about in interviews, so it's a good idea to think about how good you are in these areas. What are your strengths and weaknesses? 1. _
In the age of technology that we are living in now, it is no longer enough to keep on making the same products. Employers need people who can imagine new approaches and new ideas. Think: Think of an object or gadget you use every day. How could it be improved? Can you think of three improvements? 2._
Employers will value workers who are able to see problems before they happen and come up with creative solutions.
Think: Imagine you are organizing an end-of-term social event at school. Think of some problems that you couldface. Can you think of any solutions?
3 ._
Workers will have to be good communicators. They will have to be able to negotiate and discuss key issues and also write in a clear way without using too many words. Think: How do people communicate with each other in the 21st century? Problem solving
Employers want workers who are able to recognise the difference between information that can be believed andfalse information.
Think: Use the internet to find out three facts about a celebrity or famous figure. Can you verify the information by checking other websites?
4 ._
Individual workers have a growing amount of responsibility. It is important to be able to evaluate a situation and be confident in making a decision.
Think: Which three things could you do to (a) be healthier (b) do better at school and (c) help others? Make a decision now to do at least one of these things. Then ... just do it! Decide whether the sentences are true or false according to the passage?
5. 21st-century skills are not taught in schools in the UK. True False
6. Employers like workers to be original when solving problems. True False
7.. Future workers will need to be able to write concisely. True False
8. Employers don't want workers to make decisions without asking them. True False
Reading Passage 2_
Tips for being a super-organised student posted 2 hours ago by Amy
I have always admired students who hand their homework in on time and never forget to do it. Me, on the other hand, ... OK, I admit. I'm terrible at getting myself organised!
But lately I've started keeping a small study diary. I write down everything I need to do and when it needs to be done by. Then I write a reminder a few days before the date just in case. It's helping.
So I was wondering, what are your tips for getting organised? Post a comment below. I'm hoping we can all share some tips to teach us all better study skills.
Comments
Hana
Good question, Amy. I always spend about five minutes at the end of the day tidying up the desktop on my computer. I make a backup of important documents. I delete things I don't need any more and put everything into the correct folder. Amy
Nice tip, Hana. I think it's a good idea to do a little bit of tidying up every day. Then it becomes
a habit and your desktop is always organised.
Gloria
The most important thing is to start studying a few weeks before the exams and not leave it until
the night before! That's just common sense, I think.
Amy
Thanks, Gloria! I agree. Lou
Hi, Amy. My tip is to have a big noticeboard in your bedroom, divided into different sections. I've got one. It's a whiteboard. I've got a section for each school subject and another one for other stuff. I use board pens to write reminders and I make sure I look at it every day. The best part is when I remove something from the board! Amy
Great tip, Lou. I've got a cork board with pins. I use it in the same way.
Fill in the gaps with the appropriate words using the reading passage
Amy is asking for tips about organising school work. She keeps a diary with a
record of her homework. She also writes a 9._ in case she
forgets. Hana thinks it's important to keep your computer 10._ clean and
tidy. She deletes things she doesn't need and puts her work into
separate 11._ . Gloria says it's important to start studying in plenty of
time and not to leave things until the night 12._ ! Lou's tip is to use
a13._, divided into sections for each
14. _ He thinks the best thing about this idea is the feeling he gets
when he
15. an item!
Implementing the modified version of reading section of the summative assessment:
1. According to http://www.englishprofile.org/wordlists/text-inspector the vocabulary in the test is mostly A2 and B1 level vocabulary. We may say that this passage is appropriate to B1 level students, however, it may be challenging to assess A2 level students.
Reading Passage 1
PflA 2
Основной Основной
A 1 A2 B1 B2
A 1 A2
Reading passage 2
A1 A2 B1 B2 C1
19.16% / 44.2% 3 1.8 4.4% 0.60%
2. H. Brown (2010) points out that reading tasks should be designed taking the types of reading assessment into account; perceptive reading, selective reading, interactive reading and extensive reading. Even though selecting and designing reading assessment tasks are just obe of the teacher's activity, a teacher should not do this task relying on his/her own criteria.
Four different tasks for the reading passages focus on assessing reading for gist, for main ideas and for detailed information. Moreover writing tasks may be designed by asking the learners to write similar tips. Alderson (2000) advocates the view that matching tasks offering the alternative form of multiple choice or fill in gap forms. 3. Measuring 5 principles of assessment test task we can come the following conclusion:
Mid term/Summative test: Reading
Practicality V Economical to delivery, easy to check, instructions are easy to follow. The length of the passages are short enough.
Reliability V Instructions are easy to follow and thepassage is appropriate for B1 level in terms of range of vocabulary and length.
Validity V Validity of the test is satisfying as it relies on the objectives of curriculum
Authenticity V As the topics in the passages are widely used among teenagers it contributes to the Authenticity of the test
Washback/ Backwash The principle may not be satisfying as the materials were not be given back with feedback .
I have piloted the modified version of the summative test with a group of learners.
Comparing to the original one the modified test tasks were a little bit challenging for the students, yet it make them to approach the tasks seriously. Students admitted that those type of the tests make them work hard in their further learning process.
This process gave me a clear understanding how to design reading materials which reflects five principles of the assessment. Relying on the principles of the assessment provides easy and the right way for both teaching process and learning progress. Goodman (1970) defines reading process as a mysterious activity.
H. Brown (2010) emphasizes that still some researches should be done in terms of reading process to define how the brain receives, stores and reflects the information through reading.
References:
1. Alderson, J., C. (2000). Assessing reading. Cambridge: Cambridge University Press.
2. Brown, H., D. (2010). Language assessment principles and classroom practices. N.Y: Pearson Education.
3. Bachman, L., & Palmer, A. S. (1996) Language testing in practice. N.Y: Oxford University Press.
4. Douglas, D. (2000). Assessing Languages for specific purposes. Cambridge: Cambridge University Press
www.englishprofile. org/wordlists/text-inspector
