CC BY
19
УДК 004
DOI 10.25205/1818-7900-2018-16-1-18-23
A. W. Altaher
Kazan Federal University Derevnya Universiady, Kazan, 420010, Respublika Tatarstan, Russian Federation
smart.computing@yahoo.com
PROGRAMING MOBILE WEB BASED FOR ADAPTIVE TESTING
In this article we evaluate the use of mobile devices for testing as compared to web-based assessment systems. Many educational organizations are trying to reduce the cost of the exams, the workload and delay of scoring, and the human errors. Also, they try to increase the accuracy and efficiency of the testing. Recently, most examination organizations use Computer Adaptive Testing (CAT) as the method for large scale testing. We consider the creation of test tasks for adaptive testing. An algorithm for the functioning of an adaptive system is proposed. An example of test tasks for carrying out adaptive testing is given, based on the proposed model.
The results show that the majority of the MAT (mobile adaptive test) systems give priority to security, reliability and maintainability. However, they do not offer to the examinee any advanced support and functionalities. Also, the feedback to the examinee is limited and the presentation of the items is poor. Recommendations are made in order to enhance the overall quality of a CAT system. For example, alternative multimedia items should be available so that the examinee would choose his preferred media type. Feedback could be improved by providing more information to the examinee or providing information anytime the examinee wished.
Keywords: adaptive system, mathematical model, algorithm of adaptive testing, levels of test tasks.
Introductions
Testing is one of the most common ways of knowledge testing. The main goal of testing is to determine the level of a student's knowledge of one or more subject areas in which knowledge is checked. Different methods of knowledge evaluations are in use, such as in-class presentations, writing essays, projects, etc. However, the most common "tool" that is used to test knowledge is the test and oral exam. Since the computer as a teaching tool has been in use more and more in recent decades, and since its use has spread to all levels of education, the computer-based test has become very popular.
Adaptive testing is "a complex of processes for collecting , presenting and evaluating the result of the performance of adaptive tests that provide an increase in the effectiveness of measurements by improving the selection of the main characteristics of tasks, order, quantity and speed of issuance in relation to the specifics of the training of the tested" [1].
Adaptive online testing is an integrated environment designed to test the level of students' knowledge of various disciplines with adaptive testing. The main requirement for the developed system was its intellectuality, achieved through the organization of the adaptability of the testing process.
Altaher A. W. Programing Mobile Web Based for Adaptive Testing // Вестн. НГУ. Серия: Информационные технологии. 2018. Т. 16, № 1. С. 18-23.
ISSN 1818-7900. Вестник НГУ. Серия: Информационные технологии. 2018. Том 16, № 1 © A. W. Altaher, 2018
© Ammar Wisam Altaher - Iraq - Ministry of Higher Education and Scientific Research / Al-Furat AL-Awsat Technical University, 2018
To construct an adaptive online testing system, a mathematical model has been developed that makes it possible to differentiate and individualize the knowledge control procedure and the trajectory of subsequent testing [2].
Some problem in university education system
The increasing number of students, the need for effective and fast student testing, multimedia-based testing, self-paced testing, immediate feedback, and accurate, objective and fast scoring push many organizations to use Computer-Based Testing (CBT) or Computer Assisted Assessment (CAA) tools (Brown, 1997). But this is not enough. Current learning theories lead towards student-centred and personalized learning. There is also increased interest for reducing the cheating, reducing the examinee's anxiety, challenging but not frustrating the examinees, as well as for immediate and continuous examinee's guidance based on his knowledge, proficiency, ability and performance. Thus, many organizations are further driving towards computer adaptive testing (CAT) tools (e.g. GMAT, GRE, MCSE, TOEFL). So that why we start use Mobile learning, Mobile learning involves the use of mobile technology both individually and in conjunction with other information and communications (ICT), for the organization of the educational process outside of the time and place. Training can take many forms: with mobile devices, students can access resources, communicate with other users, create awning in the classroom and beyond. Mobile training includes activities necessary to achieve learning objectives, for example, effective management of school systems; the interaction between educational institutions and families of students.
Mobile technologies are developing continuously. The market is represented by a huge variety of devices. Not going into details, in their assortment you can select mobile phones, tablet computers, devices for reading e-books, portable audio players and portable gaming-prefixes.
Tomorrow this list may change. In order not to distract the main task of an infinite specification of technological UNESCO, in this case, refers to a wide range of digital and completely portable mobile devices, which for the most part are due to individuals, not organizations.
Item response theory
Item response theory (IRT) is the most commonly used form of psychometric theories today (Chen & Wang, 2010). Its origins date back to Rasch and Lord in the 1950s (ibid.). IRT is a "family of mathematical models that describe how people interact with test items" (Cisar, Radosav, Markoski, Pinter, & Cisar, 2010). It can be used with a variety of item selection algorithms and scoring procedures. They try to estimate an examinee's skill level and therefore find a connection between an examinee's answers to particular items and their skill level (Chen & Wang, 2010).
These estimation approaches mainly differ in the number of parameters the estimation is based on. The most wide-spread approach, the 1-parameter-logistics-model (or Rasch-model), only requires to determine the difficulty level of each item (Reckase,2010). The remaining parameters -discrimination (the amount of information the item provides for skill estimation) and guessing (the probability of guessing the right answer) - remain fixed here. 2- and 3-parameter-logistics are considered to give a more exact and/or faster estimation of an examinee's skill level, but require to determine theadditional parameters for each item (Reckase, 2010).
Educational dimension
The educational dimension consists of the following domains: 1) Content, 2) Presentation, 3) Sequencing, and 4) Feedback.
Content
The Content refers to the quantity and quality of the items in the item bank. The content of MAT (mobile adaptive test) should be based and supported by currently acceptable didactic and pedagogical theories, such as: creative, explorative, active, constructive, problem solving, critical thinking learning. It should be personalized. The items should be of high quality, i.e. valid, trustworthy, correct and accurate without any errors. The item authors should possess credentials and reputation. The items should be useful, up-to-date, and will be valid for long time. They should be relevant, suitable and appropriate for the indented tests, ages and educational level of the examinees. They should objectively present a variety of "points of view" without discriminating with respect to age, gender, race, religious, political ideas etc. They should be acceptable and compatible to the examinee's language, social, cultural, racial, political, and religious values and ideas. They should adjust and support the values of the examinees and the value of learning.
Presentation
Presentation refers to the presentation, media and format of the items in the MAT. The presentation, media and format of the items should be personalized. It should be clear, simple, and of low overhead. It should be rich, be based on a variety of media (e.g. text, picture, image, graphs, diagrams, audio, video, immersion) of high quality (e.g. resolution, number of colors, sound fidelity). There should be the right mix of media objects at the appropriate positions with low distraction. The result should be enjoyable.
Sequencing
Sequencing refers to the sequencing of the items presented to the examinee. In MAT, the Sequencing of the items depends on the examinee's answers. An adaptive algorithm is employed to select the next item to be presented to the examinee. This algorithm should be based on a valid and accredited pedagogical and psychometric theory. The duration and the number of items in the MAT should be enough to produce valid results. The selected items should accurately represent the content, skills and abilities that are intended to be measured. The exposure of the items should be kept low and the test-overlap minimum. The algorithm should be easy, time and cost efficient to initiate, manage and terminate. It should be fair, non-discriminating, and consistent. It should be intuitive, logical and appropriate for the examinee. There prioritization of important items. It should enhance student's motivation and enjoyment.
Feedback
Construct test tasks
To construct test tasks, first of all, it is required to define educational objectives, therefore, the corresponding types of testing. When designing test tasks, the authors should first of all find out which training elements (concepts, statements, methods) each task contains, how much they coincide with the learning objectives, etc. Secondly, the level of complexity of the task depends on the number of correct, incorrect answers in question, and from the logic of choice of answers. For this reason, the experts of these disciplines base the test tasks in advance into various levels of complexity: light, medium, high. For an easy level of complexity, it is recommended to use questions only with an unambiguous choice of answers. For the average level of complexity, it is recommended to use questions with multiple choice with the addition of weight by the logic "OR", questions with multiple choice by the logic "AND" and questions with an unambiguous choice. For a high level of complexity, the "OR" and "AND" reaction blocks are used.The "AND" logic is the rule according to which the maximum number of points is calculated, provided that all correct answers are chosen and the wrong ones are not selected, the "OR" logic is the rule on which the answer points are
scored, provided that at least one correct answer is selected and the points are removed if the wrong one is selected.
To evaluate the truth of answers to questions of a light level of complexity, a Boolean model is used . In this case, the truth of the answers is expressed in two-valued logic, and can take the values "true" or "false": 1 - if the answer is correct, 0 - otherwise. The truth of the answers to questions with an unambiguous choice of the answers to the task of the average level of complexity can take values: 2 - if the answer is correct, -1 or 0 - otherwise.
The process of constructing test tasks for ASOT included the following stages:
1. Definition of the model for designing the level base of test tasks (Tab. 1).
2. Development of test tasks for the given model for a certain discipline.
Tab.1. The model of constructing the level base of test tasks
Levels difficulties Reaction Blocks Sublevel tasks Points Digital equivalent Evaluation
The third (Tall) difficulty level AND a3 95-133 4.3 Excellent
OR a3 93-94 3.67
Second (average) difficulty level AND 85-89 3.33
OR a2 83-84 3.3 Good
Single-valued a2 75-79 2.67
First (light) level difficulties Single-valued a1 53-74 1.3-2.33 Satisfactorily
As an example, we give test tasks in the discipline "Informatics", the topic "Logical foundations of computer science" The MAT application has been developed and the student can use it throw this link (http://rossiphoto.ru/test/) with The Server Modules have been developed with PHP and MySQL. The prototype Mobile Adaptive Test application developed consists of three activities:
□ Enter user name
□ Test Selection
□ Adaptive Question Sequencing.
The user interface is designed with multimedia content for tester-friendliness. The original version has been designed with interactive and user-friendly icons and menus. The application targets the mobile devices like phones and tablets with touch screen, but can be accessible by any kind of device running Android platform, IOS , Windows ,etc.
A user (Tester) enter his name in to the application environment (Fig. 1, a). The user is authenticated by the user credentials stored on the database server which stores the unique profile for each learner. Upon entering to the application the user profile is synced with the type of device that he/she uses. After successful login the main screen for Test module selection is presented in Fig. 1, b and we make her the test is only tow subject which is (HTML Basics and JavaScript for beginners) and in this part the student will choose in ant part he/she want to do the test . The associated activity is MAT and is declared, so the student start answering the questions (Fig. 1, c). The next question posted to the user is based on the performance level and is appraised by the Mobile Adaptive Test server which has pre-built configuration and assessment algorithm stored in it. And
the last part is get the test result (Fig. 1, d), noted that Russian student are our target so that why we use the Russian language in the developing the mobile web based as a main language.
rossfphoto.ru О
Тестирование
Введите свое имя
( Войти •> j
rossiphoto.ru С,
Здравствуйте, Arrimar
Предлагаем а дм пройти тесты на знание программирования. Выберите тему и начинайте тестирование.
Основы HTML»
JavaScript для начинающих»
Тестирование
Основы HTML
Можно ли инициировать DOM-событие из javascript? Например, сэмулировать клик мышкой на элементе, чтобы javascript-код кликнул за пользователя* и сработали соответствующие обработчики.
ОДа, можно.
О Нет, нельзя.
Далее
© S018, Tesl.com
rossiphoto.ru С
Тестирование
Arrimar, ваш результат
13
( Пройти еще раз! )
b
a
Fig. 1
d
c
Conclusion
This paper has been dealt with the development of Mobile Adaptive Test (MAT) for the all the 3G, 4G Generation Mobile Devices. The prototype developed for assessing the test tasks in the discipline "Informatics", in the topic subject "Logical foundations of computer science" on their academic achievement has been tested with different groups of learners belonging to Computer Applications specialization. The Mobile Adaptive Tests have yielded positive and better results from the learner when compared to the traditional form of testing methodology in the class-room settings and linear classic testing where each student has been presented with a same set of questions. The multimedia content inclusion in the question formats have enhanced the learners answering ability to a new higher level showing the improvisation in the knowledge acquisition and deeper understanding of the subject content. The Mobile Adaptive testing methodology helps the instructor to re-design the test and lesson content in a quicker way and focus his attention on individual student's performance level. The Mobile Adaptive Test based on multimedia question formats and rich set of User
Interfaces provided by the mobile web based platform offers the learner a high interactive test environment.
References
1. Brown, J. D. (1997). Computers in language testing: present research and some future directions. Language Learning & Technology, 1(1), 44-59.
2. Chelyshkova M. B. Theory and practice of constructing pedagogical tests // Experimental educational and professional programs. M.: Issled. Center for Quality Problems in Training Specialists, 2001. 410 p.
3. Cisar, S. M., Radosav, D., Markoski, B., Pinter, R., & Cisar, P. (2010). Computer Adaptive Testing for Student's Knowledge in C ++ Exam. Presented at the 11th IEEE International Symposium on Computational Intelligence and Informatics, Budapest, Hungary.
4. Chen, J., Wang, L. (2010). Computerized Adaptive Testing: A New Trend in Language Testing. In International Conference on Artificial Intelligence and Education (ICAIE) (p. 725-728).
5. Nurgabyl D. N. On a mathematical model of multi-step adaptive testing // Vestnik KazNPU im. Abay, a series of physical and mathematical sciences, 2014, No. 1. P. 143-149.
6. Reckase, M. D. (2010). Designing Item Pools to Optimize the Functioning of a Computerized Adaptive Test. Psychological Test and Assessment Modeling, 52 (2), 127-141.
Received 25.01.2018
А. В. АльТахер
Казанский федеральный университет Деревня универсиады, Казань, 420010, Республика Татарстан, Россия
smart.computing@yahoo.com
ПРОГРАММИРОВАНИЕ МОБИЛЬНЫХ ВЕБ-СИСТЕМ ДЛЯ АДАПТИВНОГО ТЕСТИРОВАНИЯ
Рассматривается использование мобильных устройств для тестирования по сравнению с веб-системами оценки знаний. Многие образовательные организации пытаются снизить стоимость экзаменов, нагрузку и задержку подсчета баллов, а также человеческие ошибки. Кроме того, они пытаются повысить точность и эффективность тестирования. В последнее время большинство экзаменационных комиссий используют компьютерное адаптивное тестирование (CAT) в качестве метода широкомасштабного тестирования. Мы рассматриваем создание тестовых задач для адаптивного тестирования. Предложен алгоритм функционирования адаптивной системы. Пример тестовых задач для проведения адаптивного тестирования приведен на основе предлагаемой модели.
Результаты показывают, что большинство систем MAT (мобильные адаптивные тесты) придают приоритет безопасности, надежности и ремонтопригодности. Тем не менее, они не предлагают испытуемому никакой дополнительной поддержки и функциональности. Кроме того, обратная связь с испытуемым ограничена, и представление предметов оставляет желать лучшего. Рекомендации сделаны для повышения общего качества системы CAT. Например, альтернативные мультимедийные элементы должны быть доступны, чтобы испытуемый выбирал предпочтительный тип носителя. Обратная связь может быть улучшена путем предоставления дополнительной информации испытуемому или предоставления информации в любое время, когда испытуемый пожелает.
Ключевые слова: адаптивная система, алгоритм адаптивного тестирования, математическая модель, уровни тестовых задач.
For citation:
Altaher A. W. Programing Mobile Web Based for Adaptive Testing. Vestnik NSU. Series: Information Technologies, 2018, vol. 16, no. 1, p. 18-23. (In Engl.)
