CC BY
17
THEORY AND TEACHING METHODICS
УДК 37:004
L. I. BILOUSOVA, O. G. KOLGATIN
CRITERIAL CHOICE OF THE SYSTEM FOR COMPUTER-BASED TESTING OF
LEARNING ACHIEVEMENTS
It is shown an importance of consideration the leading didactical functions on definite learning process stage for constructing the system requirements for the ICT of pedagogical testing. Basic specific requirements for automated pedagogical testing systems are selected in accordance with the author's classification of teaching tasks. It is grounded the system of criteria to determine the specialization of an automated pedagogical testing system for pedagogical diagnostics which is aimed at a detailed analysis of academic achievement in order to predict and assist in choosing the expedient embodiment of the learning technologies realisation.
Keywords: educational testing, ICT, requirements.
Л. I. Б1ЛОУСОВА, О. Г. КОЛГАТ1Н,
КРИТЕР1АЛЬНИЙ ВИБ1Р СИСТЕМИ КОМП'ЮТЕРНО ОР1СНТОВАНОГО ТЕСТУВАННЯ НАВЧАЛЬНИХ ДОСЯГНЕНЬ
Показано, що при no6ydoei системы вимог до тформацшно-комуткацшних технологш (1КТ) педагогiчного тестування важливо враховувати провiдну дидактичну функцю вiдповiдного етапу навчального процесу. Визначено основт специфiчнi вимоги до автоматизованих систем педагогiчного тестування згiдно з авторською класифжащею дидактичних завдань. Обгрунтовано систему критерив для визначення спецiалiзацii автоматизованоi системи педагогiчного тестування для педагогiчноi дiагностики, що спрямована на докладний аналiз навчальних досягнень того, хто навчаеться, з метою педагогiчного прогнозування i допомоги йому у виборi доцыьного варiанта реалiзацii технологи навчання.
Ключовi слова: педагогiчне тестування, 1КТ, вимоги.
Л. И. БЕЛОУСОВА, А. Г. КОЛГАТИН
КРИТЕРИАЛЬНЫЙ ВЫБОР СИСТЕМЫ КОМПЬЮТЕРНО ОРИЕНТИРОВАННОГО ТЕСТИРОВАНИЯ УЧЕБНЫХ ДОСТИЖЕНИЙ
Показано, что при построении системы требований к информационно-коммуникационным технологиям (ИКТ) педагогического тестирования важно учитывать ведущую дидактическую функцию данного этапа учебного процесса. Выделено основные специфические требования к автоматизированным системам педагогического тестирования в соответствии с авторской классификацией дидактических задач. Обоснована система критериев для определения специализации автоматизированной системы педагогического тестирования для педагогической диагностики, которая направлена на подробный анализ учебных достижений с целью педагогического прогнозирования и помощи в выборе целесообразного варианта реализации технологии обучения.
Ключевые слова: педагогическое тестирование, ИКТ, требования.
Computer-based testing of learning achievements is widely used in educational process for realisation of various didactical tasks [1; 2]. It has become a tradition to associate the testing with the didactical function of grading learning achievements at time of current, thematic and concluding assessment. It is important to underline some other didactical functions of testing. Computer-based testing is a powerful method of self-control (the leading functions are monitoring control the learning process and systematisation of student's learning activity). Pedagogical testing is used for diagnostics
of a student in order to choose the expedient embodiment of the learning technologies realisation (the leading functions: feed-back information for teacher or learning management system, information for pedagogical prognosis, systematisation of student's learning activity). Computer-based testing is successfully used in educational process to realise such purposes as actualisation of knowledge basic for new learning material (the leading functions are teaching, stimulating-motivating, feed-back information); creation of problem situation according to new learning material (the leading functions are teaching, intellectual development, stimulating-motivating); working out some skills using a test as a training simulator (the leading functions are teaching and stimulating-motivating); making quizzes, learning competitions, etc (the leading functions are teaching, moral teaching, stimulating-motivating). High stake assessment is the special category of testing, for example, the university entrance examination (the leading functions are grading and prognosis). Each of the tasks sets up specific and, sometimes, conflicting requirements to the automated testing system that causes its specialising.
The problem of classification pedagogical testing systems on various grounds and determination requirements for these systems is in the focus of numerous scientific works (V. U. Garbusev, N. I. Samoylenko, L. V. Zaytseva, N. O. Prokofieva etc.). The standard "Temporary requirements for educational software ..." is designed and determines common, pedagogical, methodical and technical requirements on the basis of pedagogical studies and practice of ICT using at learning process. This standard without any doubt applies to automated pedagogical testing systems. However the problem of specification of requirements for automated pedagogical testing systems according to its didactical application is still urgent.
The aim of the work is to prove the specific requirements for automated pedagogical testing systems according to their didactical application.
The systems for thematic and concluding grading should provide high reliability of test results, useful and reliable instruments for registration of these results, data protection from unauthorized use and distortion. If grading is held automatically by the system without a teacher's participation, procedure of grading requires special attention and should provide the grading according to the scale that is approved in educational institution or on the government level. For example, the grading in Ukrainian schools is realised with the use of 12-marks scale oriented on criteria and based on the levels of learning achievements. Correct grading with the use of such scale requires the test items to be classified by the levels of learning achievements. Automated testing system should take into account the database structure at grading. Norm-oriented scales are more common in higher education and assume to use norm-oriented interpretation of test results. Concluding grading should be provided with standardised tests. Computerised adaptive testing based on G. Rash model is used in modern practice of educational assessment to provide reliability of test variants. Design of a computerised adaptive test requires a database with a large number of test items and its approbation which takes lots of time. If such a work is impossible, one should use the test with a fixed set of items to keep off difference of test variants difficulty and provide correct grading.
Tests for current assessment are often designed by a teacher; this process should not take much time. The time for test passing is also limited. So tests for current assessment do not contain enough number of items that can provide test results reliability for automated grading. A teacher grades examinee's work himself with consideration of the test results and other kinds of examinee's activity monitoring (oral or written quiz, monitoring of taking part in discussion, problem solving, making learning research etc.). Main requirements for automated testing system for current assessment are the following: user-friendly interface, handy tools for design and editing of test items, absence of unnecessary services and settings, saving all examinees' answers to be analysed (better in a teacher's server), handy tools for analysis of test items quality.
Self-control is an important application of automated testing systems. A student can pass the test a lot of times, so the test items should be chosen at random from a large database. The testing procedure should be adaptive; it should not overload weak students by difficult tasks and should not weary strong students with easy items. It is expedient to collect the detail information about the test passing and its results in a server for analysis of the test items quality and for providing a student with a possibility to compare his results with the other's ones. If self-control is realised outside a classroom, only the Internet-oriented system of testing can provide registration of such information. It is
expedient to combine the self-control with pedagogical diagnostics, in such a case, the automated testing system should satisfy additional requirements discussed below.
The automated testing system for pedagogical diagnostics should provide the highest descriptiveness of test results; continuously collect data for the pedagogical prognosis and analysis of the test items quality. The test items should represent a structure of learning material for qualified diagnostics. A student can pass the same test several times, so testing system should produce different variants of test. These parallel variants should be of equal difficulty and correctly represent content of learning material. Integration of such requirements as different test variants, stability of test difficulty and correct representation of the structure of learning material makes a problem for designers of testing software. Successful steps in solving this problem are associated with systematisation of random selection of test items from a database. Pedagogical prognosis is based on analysis of mastering in learning material in terms of the levels of educational achievements; therefore the testing system should provide separate processing of the results according to the levels of educational achievements. Choice of the expedient embodiment of the learning technologies realisation is based on information about the structure of educational achievements, so it is important to know, which elements of the learning material are mastered poorly. Hence test results should be processed separately according to elements of the learning material. The system should be adaptive, it should present test items of high levels only for a student who is ready to try to solve these problems. The analysis of some automated testing systems from viewpoint of these requirements is shown in table 1. Each requirement is satisfied by some systems (table 1). However the integration of different test variants and, at the same time, stability of test difficulty and correct representation of the structure of learning material is an actual aspect of investigation in the field of pedagogical testing software.
Automated testing systems oriented on actualisation of knowledge which is basic for new learning material should support students and provide them with the help and correct answers for every of the solved test items.
Computer can suggest to a student some problems in test form with the aim of motivation and stimulation of student's cognitive activity in the process of learning new material. In such a case the system should organise a dialogue with a student, support him at every step of problem solving and provide him with new information. Very important requirement for such systems is user-friendly interface as well as using multimedia.
Problems in test form can be used for mastering some skills, in such a case the testing system works as a training simulator. A student gives the answer for the problem and obtains an appraisal of its correctness immediately. The leading requirement to testing in regime of training simulator is to realise the motivating function of testing. Computerised training simulators can create a game situation and atmosphere of competition, use interesting problems and pleasant dialogues.
High-stakes assessment is graded on the basis of rating scale. The only variant of the test is used in every session of testing to exclude variations of difficulty. There is a problem with providing the secrecy of a test before the session if the test is prepared in computerised form. Therefore, the test is created automatically directly at the moment of starting the testing procedure [18].
Conclusions:
1. Kinds of use of the pedagogical testing systems are systemised according to didactical tasks.
2. Criteria of choice of the automated pedagogical testing system for pedagogical diagnostics are suggested: producing variants of test; correct representation of the structure of learning material; stability of test difficulty; providing the separate processing of the results according to the levels of educational achievements; providing the separate processing of the results according to elements of the learning material; in-line processing of the test results and its interpretation; collection of all data; adaptive testing; criteria-oriented interpretation of the test results.
Table 1
Automated pedagogical testing systems and requirements to their use for pedagogical diagnostics
Automated system
¡3
'Ü
;>
bo
.¡g
o
o ^
o s Ja
<u £
t+H
o
Ö o
Ö <u
cn ra
ss a s .a
o
Si —I
r9 ^
Q o
<u £
t+H
o
too , Ö ^ • S o
m J«
8 13
o >
a
Adequacy
§ Q I £ 2 I
o
m 8
ä es
(U
<u
t+H
O <U tso £
S3 ^
5 o
M t»
Si 0
0 <U
6 J
1 S
& to
I» .¡3 H <D T3 <D
■a o^
bo
CÖ
CÖ M
cn Ö
* 'Ü
CO CÖ
<U (U
<u £ t+H
o bo
.S3 ö
cn o
CO
s ü
a £
<u <u
• S3 s
e S
Ö
<u
Ii
T3
<
«EXAMINER-II», 1993 [3]
+
«OpenTest2», 2004 [4]
«EiccnepT». 2003 [5]
"WEB-EXAMINER", 2005 [6]
«WebTutor», 2008-2011 [7]
«Information system VNZ 2.0.1», 2008 [8]
«Teletesting», 1999 [9]
Moodle [10]
UniTest System, 2001-2006 [11]
USATIC, 2001 [12]
«SunRav TestOfficePro 5.6.1», 2004 [13]
System «Perevirka znan», 2008 [14]
TestSystem Deluxe 2.0 [15]
FastTEST Web v.2.1.6 (randomize for each examinee), 2011 [16]_
FastTEST Web v.2.1.6 (compute-rized adaptive testing), 2011 [16]_
MicroCAT adaptive test, 1984-1988 [17]
MicroCAT individualized domain-referenced test , 1984-1988 [17]
REFERENCE:
1. Ho3oBa B. I. TeopeTHHHi ochobh BHxoBaHHa i HaBHaHHa: HaBH. noci6HHK / B. I. Ho3oBa, r. B. Тpоцко. -XapKiB: OBC, 2002. - 400 c.
2. KonraTiH O. r. flHgaKTHHHi BHMoru go 3aco6iB aBToMaTH3oBaHol negaroriHHol giarHocTHKH / O. r. KonraTiH // 3aco6u HaBHantHol Ta HayKoBo-gocnigHol po6oTu: 36. HayK. npa^; 3a 3ar. peg. npo$. B. I. GBgoKUMOBa Ta npo$. O. M. MuKHTroKa. - XapKiB, 2007. - Bun. 27. - C. 65-74.
3. BinoycoBa H. I. ABTOMaTH3OBaHa CHCTeMa TecTyBaHHa «EXAMINER-II» / H. I. BinoycoBa, T. B. ConogKa // MmHapogHa HayKoBo-MeTogHHHa KoH^epeH^a «npo6neMH 6araTopiBHeBol bh^oi TexHiHHol ocBiTH», Khib, 13-15 xoBTHa 1993 p. - K.: Khib. noniTex. yH-T, 1993. - C. 220-221.
4. KoMn'roTepHa CHCTeMa TecTyBaHHa OpenTEST2 / O. C. EKint, B. I. KayK, C. B. HanpacHHK Ta iH. // BicHHK TIMO. - 2008. - № 2. - C. 35-41.
5. BinoycoBa H. I. TecTonoriHHHn aHani3 y CHCTeMi «EKcnepT» / H. I. BinoycoBa, O. r. KonraTiH, H. C. KonraTiHa // KoMn'roTep y mKoni Ta ciM'l. - 2003. - № 7. - C. 41-43.
6. KpaB^B r. M. Pa3pa6oTKa CHCTeMti TecTHpoBaHHa b cpege gнстaнцнoннoгo o6yHeHHa / r. M. KpaB^B, fl. r. Кpaвцoв // MaTepianu III Mi^HapogHol HayKoBo-npaKTHHHol кoн^epeнцil «IH$opMaTH3a^a ocBiTH yKpalHu: CTaH, npo6neMH, nepcneKTHBH». - XepcoH, 2005. - C. 86-87.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
±
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
±
±
+
+
+
+
+
±
+
+
+
+
+
+
+
+
+
+
+
+
+
7. Модуль «Тестирование» системы дистанционного обучения «WebTutor» [ Электронный ресурс] -Режим доступа: http://www.websoft.ru/db/wb/root_id/webtutor_testing/ doc.html
8. Антонов Ю. С. Комп'ютерш системи тестування на основi технологи тpиpiвневих баз даних / Ю. С. Антонов // 1нформацшш технологи i засоби навчання. - 2008. - № 2 (6). [Електронний ресурс] - Режим доступу: http://www.nbuv.gov.ua/e-journals/ITZN/em6/content/08aystdt.htm
9. Шмелев А. Г. Адаптивное тестирование знаний в системе «Телетестинг» / А. Г. Шмелев // Материалы Всероссийской научно-практической конференции [«Информационные технологии в образовании»], 9-12 ноября 1999 г., Москва. [Электронный ресурс] - Режим доступа: http://www.ito. su/1999/II/6/6148.html
10. Welcome to the Moodle Service Network! [Electronic resource] / Moodle Partner companies. - Mode of access : http://moodle.com.
11. UniTest System - программное обеспечение для автоматизации компьютерного тестирования / Sight2k Design; руководитель А. Н. Маркович. - Иркутск. [Электронный ресурс] - Режим доступа: http://www. sight2k. com/rus/unitest/
12. Автоматизированная Система Тестирования И Контроля (УСАТИК 2.000). Свидетельство об официальной регистрации программы для ЭВМ № 2001611642. [Электронный ресурс]. - Режим доступа: http://www.usatic.narod.ru/patent.html
13. Программы для создания тестов и электронных книг - SunRav Software. SunRav TestOfficePro. [Электронный ресурс] - Режим доступа: http://www.sunrav.ru/srtop/index.php.
14. Система «Пеpевipка знань» / Лабоpатоpiя шформацшних та комушкацшних технологш ФМГ № 17 м. Вшнищ. [Електоронний ресурс] - Режим доступу: ftp://ftp.pmg17.vn.ua/pub/test/test.edu.vn.ua_howto.pdf
15. Test System Deluxe - простая в освоении мультимедийная система тестирования / Helpful Bytes Software Company. [Электронный ресурс] - Режим доступа: http://helpfulbytes.info/products_testsystem.htm
16. Fast TEST Web. Innovative solutions for online testing / Assessment Systems Corporation. [Elecnronic resouce] - Mode of access : http://www.fasttestweb.com/
17. Patience W. Software Review. MicroCAT testing system Version 3.0 / Wayne Patience // Journal of Educational Measurement. - 1990. - Vol. 27. - No. 1, Spring. - P. 82-88.
18. Маланюк Т. П. Автоматизована система «Екзаменатор» / Т. П. Маланюк // Науковi записки Тернопшьського нащонального педагопчного уншерситету iменi Володимира Гнатюка. Сеpiя: Педагопка. - 2005. - № 6. - С. 38-42.
УДК 373.3(075.2)
Y. Р. KODLIUK
THEORETICAL APPROACHES TO ANALYSIS OF THE TECHNOLOGICAL STRUCTURE OF A PRIMARY SCHOOL TEXTB00К
The essence of the concept of «adaptability of a book» is analyzed in the article. Its structural components are proved: a pupil as a strategic and structural model of studying process, a pupil as a self-teacher, functions of a book, coherence of book and teaching technologies, technical side of a book.
Keywords: a primary school pupil, a primary school book, technological structure of a school book, structural components of a primary school book.
Я. П. КОДЛЮК
ТЕОРЕТИЧН1 П1ДХОДИ ДО АНАЛ1ЗУ ТЕХНОЛОГ1ЧНОСТ1 П1ДРУЧНИКА
ДЛЯ ПОЧАТКОВО1 ШКОЛИ
Обгрунтовано структурт компоненти технологiчностi тдручника для початковоi школи (тдручник як стратегiчна i тактична модель процесу навчання; тдручник як самовчитель, функщональтсть навчальноi книги; узгоджетсть тдручника з певними технологiями навчання; техтчтсть навчальноi книги). Визначено критери аналiзу цei його характеристики.
Ключовi слова: молодший школяр, тдручник для початковоi школи, технологiчнiсть тдручника, структуры компоненти технологiчностi тдручника для початковоi школи, критери аналiзу технологiчностi.
