CC BY
16Реализация возможностей обучающей среды в организации тестового контроля знаний в юридическом вузе
Ялаева Наталья Вячеславовна,
кандидат педагогических наук, доцент, кафедра русского, иностранных языков и культуры речи, Уральский государственный юридический университет имени В.Ф. Яковлева E-mail: yalaeva-nv@mail.ru
Садыкова Наталья Валерьевна,
старший преподаватель, кафедра русского, иностранных языков и культуры речи, Уральский государственный юридический университет имени В.Ф. Яковлева E-mail: nine999@list.ru
Жеребцова Евгения Валерьевна,
старший преподаватель, кафедра русского, иностранных языков и культуры речи,Уральский государственный юридический университет имени В.Ф. Яковлева E-mail: evgenia.zherebtsova@gmail.com
В статье проведен обзор тестовых систем контроля знаний INDIGO и Moodle, применяемых кафедрой русского, иностранных языков и культуры речи УрГЮУ с целью независимой оценки знаний обучающихся на конкретном этапе формирования профессиональной иноязычной компетенции и диагностики результатов обучения. Авторами составлена сравнительная характеристика данных систем по разным критериям, на основе опыта работы определены особенности структуры банка тестовых заданий, описаны примеры разработанных тестов. Выделены преимущества и недостатки включения систем тестирования в электронную информационно-образовательную среду вуза. Проведен онлайн-опрос для определения уровня удовлетворенности обучающихся организацией процесса тестирования. Анализ результатов анкетирования студенческого коллектива позволил выявить, что большинство студентов положительно оценивают качество проверки знаний с помощью данных тестовых систем.
Ключевые слова: тестовая система INDIGO, система управления обучением, Moodle, электронная информационно-образовательная среда, тестирование.
о с
U
см см о см I—
Traditional areas of academic communication have changed due to the information society development, interaction between students all over the world, the pandemic situation that caused growing online learning, and wide introduction of various computer programs into everyday life of most students and teachers. So, doing classroom activities and home assignments students communicate both face-to-face and online. The said leads to the need for using additional means often connected with computer programs, which would foster intercultural communication and check the knowledge of professional foreign language.
That is why, foreign language teachers turn to information and communication technologies (ICT) in the teaching process. Electronic dictionaries and other informational and educational resources have superseded the customary paper dictionaries and textbooks, which were dominant for many decades.
In the modern educational system, there are many factors contributing to a focus on ICT-enhanced learning. One of them is a rapidly growing number of Internet users both globally and in Russia.
Article 1.5 of the Federal State Educational Standard of higher education for Bachelor's degree program in Jurisprudence (40.03.01) [6] requires higher education institutions to provide and use Electronic Informational and Educational Environment (EIEE) for the implementation of educational programs.
EIEE is defined as a systematically organized set of information, technical, educational, and methodological support, which is inseparably connected with a person as a subject of the educational process [3].
The electronic informational and educational system of the Ural State Law University includes official website as an integrator of all resources, educational portal, electronic library systems, learning process management system, and INDIGO testing system.
According to several researchers, i.e., N.G. Aka-tova [1], M.A. Bovtenko [2], V.M. Kadnevsky [4], V.P. Ovcharenko [5], I.A. Tsaturova [8], V.A. Uglev [7], M.B. Chelyshkova [9], N.V. Yalaeva [10] and others, computer-based testing is one of the most effective forms of pedagogical diagnostics.
The purpose of this paper is to compare two testing systems: INDIGO and Moodle, and to identify the strong and weak points of each system from the perspective of both students and teachers.
To intensify the educational process in a law school it is important to understand positive and negative aspects associated with the use of testing systems, to be able to identify them correctly, and manage them rationally. Both systems are effectively used in our university.
An overview of the INDIGO testing system
The program is a combination of server and software application, which provides a teacher with an opportunity to control students' knowledge not only in the classroom, but also in the remote location. The students need to enter the testing server via a remote computer or mobile device, log in and take a test on the required topic. After completing all the tasks, the student can see the results of the test. He/she can see their mistakes and, if necessary, repeat a section of the topic that caused difficulties. All data (tests, users, results) are stored centrally in the system database. INDIGO has a user-friendly graphical interface and makes it possible to create test tasks with five different types of answers, e.g. selecting one answer option; selecting several answer options; entering the answer from the keyboard; matching; arranging in the right order.
Comprehensive test configuration features allow teachers to insert images, formulas, charts, tables, audio and video files, and HTML documents into the test questions. It is possible to limit the test time, shuffle the questions or shuffle the answer variants in a random way. The system can work either on an isolated computer, a local network, or over the Internet. The software allows not only creating and editing tests, but also managing the test database, users and results, exporting test tasks, information about completed tests, and data on users into MS Word or MS Excel document. The system has a possibility to create unlimited number of tests in a multi-level hierarchy of any structure, to protect tests with password, to view test statistics (by scores for questions and groups of questions; by scales; by units; by answers), and export statistics to MS Excel. As for users, there can also be an unlimited number of them. Reports can be created for specific users. The administrator can add users or they can register themselves via the web interface, unless the administrator has banned this. The administrator also sets rules on what users or groups of users can take the test and which tests or groups of tests will be available for them to take. A rule and schedule can be set for each test and a limit on the number of test attempts can be imposed.
INDIGO testing system can be useful when knowledge testing or questioning is required in the absence of computers because tests can be easily saved in a text file and printed. INDIGO can be used for various pedagogical tasks as monitoring testing, training testing, interviewing, organizing distance language Olympiads and competitions, and listening comprehension.
Let us consider the test development in the "INDIGO" system on the example of the test compiled in this system at the Department of Russian, Foreign Languages and Culture of Speech of Ural State Law University.
The need to compile a foreign language test was specified by the pandemic situation and the necessity to control the results of studying English at the end of the academic year. Another objective of the test was to assess professional skills of communicating in English affected by the situation when all the students had
distance learning. Lectures also needed the statistical analysis of the results of students' performance after educational activities. We defined the units for assessment, organized the working group, selected the language material, compiled a bank of test tasks, and discussed the structure and the grades for test analysis. In the test, we used three types of tasks out of five offered by "INDIGO" system: selection of one option from the list, selection of several correct options from the list, and matching [11, 61].
A total of 256 tasks were developed, and only 40 of them were included in the selection. Each student has different tasks from 24 categories on the screen. The test structure includes of 24 sections (10 grammatical, 12 lexical and 2 reading). Then the settings of the test were adjusted to the grades total number of which we had to obtain (10 grades).
The test we developed has been successfully used in all undergraduate groups at the Ural State Law University. The control test based on the "INDIGO" test system on the topics "Judicial Systems" and "Legal Professions" was conducted in a computer class in April 2022. Then the staff of the department analyzed the results of the test and identified the gaps in both test structure and students' performance. The results were rather objective. Thus, this system allowed us to achieve the goals that had been set before we started creating the test and to pay attention to tasks that caused difficulties for students.
The main advantages of INDIGO are: easy system installation; user-friendly interface; compatibility of the product with all operating systems of Windows family; support of all popular browsers, test settings, and objectivity.
Disadvantages are: university has to buy the program, teachers are not the program administrators of the test, a test should be installed by IT specialists, the requirements for hardware are high. In order to get statistics it is necessary to contact the network administrator. It is difficult to decipher the full report; the teacher must be proficient in the system.
An overview of the Moodle testing system
Moodle is an educational management system used to organize the learning process and interaction between teachers and students [12, 45]. Such e-learning systems are often called Learning Management Systems (LMS) or Virtual Learning Environments (VLE). Moodle was translated into many languages. It is used for testing students all over the world, but testing is only a small part of the said program.
The Moodle testing system was introduced at our university a bit earlier than INDIGO. Teachers took special training courses where they gained knowledge about it and about testing as its component. Each lecturer compiled tests individually depending on the purpose of his/her course. The first step was to create a test on the topic choosing the number of questions, their types, the time limits, and grading. Secondly, the teacher informed the students about the date and time of the test. After logging in the students read infor-
C3
o
CO "O
1=1 A
—I
o
C3 t; o m o
OT
3
u o
CO
mation about the structure of the test, its content, the deadline and methods of assessment. Upon completing the test the teacher could immediately analyze the results the possibility of being acquainted with which had been set for students by the test organizer.
The results indicated that Moodle was more flexible for developing test assignments than other programs having an advantage over them in this direction.
Moodle has an option for a teacher to choose a scale for assessment giving him/her an opportunity to adjust the test even after the student has done it and sent the answers.
The teacher has the possibility to choose random selection of the tasks from the categories. The same is true for answers that can be mixed. The types of tasks are different. Among them are multiple choice questions, matching type questions, random type questions, embedded answers or gap filling and essays. Moreover, the lecturer can give special grade for each task.
The teacher can see any information about test attempts in the Student Assessments section that contains information needed for the students to complete the tasks, statistics of the whole group, grades and questions that got most false answers. Each student gets a mark for the test.
The features of this testing include the following: it is used in distance and in-person training; it has a simple and convenient web-interface, it can be used for various purposes being easily transformed, the results are automatically analyzed by the system, full report on user logon is available.
The main advantages are that Moodle is free of charge, the administrator can add another question types from add-ons database, and there is no need for the program administrator as the teachers immediately see all the results in the system.
The main disadvantages are the possibility of cheating and absence of enough protection and authentication.
Diagnosing, predicting, and optimizing the testing process require feedback from its participants. To determine the level of student satisfaction with the organization of testing on the INDIGO and Moodle platforms, an online survey was conducted at the university. The authors used a Google forms survey as a data collection tool. Eighty-eight full-time students took part in the survey.
Survey
In the comparative evaluation, the ergonomics of INDIGO system was highly rated (79.5%). Simplicity of the interface and conciseness of the instructions do not require prior training in working with the system and even for "newcomers" do not cause a need for additional explanations about the sequence of actions. 20,5% of the 5 respondents are attracted by the typical variability of test = questions in the Moodle system, but every fifth of them ~ notes that technically they do not always understand S the algorithm of some types of tasks, and they have to ask the teacher for explanations.
At the same time, to a greater extent the indicators were distributed in favor of INDIGO when assessing the accuracy and objectivity of knowledge testing. 88.6% of the respondents note the high level in organizing the control measures in terms of test content, time distribution for its performance, task evaluation criteria, as well as the security of the system from unauthorized access. Only 9% of students indicated that the Moodle system allows you to check knowledge more accurately, 2.4% had difficulty with the answer. This, of course, is an indicator of the effectiveness of the group compiling the bank of test tasks.
However, this data does not detract from the merits of the Moodle system. Given that the University Learning Portal functions on the Moodle platform, the "Test" element is an integral part of each course. 56.8% of respondents recognize its potential in diagnosing and controlling the current progress at any stage of the learning process.
Analysis of the results of the student team survey revealed that the majority of respondents positively assess the quality of knowledge testing with the help of these test systems (96%) and are convinced of the need for testing that allows a sufficient number and variety of task options in a short time to check the level of competence formation in several academic modules.
Conclusions
Having worked with both systems we concluded that Moodle is more suitable for routine testing and INDIGO is better for final testing. Both programs can be used both locally and over a network; however, an authentication system is more secure in INDIGO. Both programs were tested successfully and integrated into the existing EIEE of the university.
After analyzing the feedback from the teachers and students, we inferred that Moodle and INDIGO testing systems develop students' independence, they are convenient for the lectures, master the professional foreign language communicative competence, and can be used for different aims in legal education.
Литература
1. Акатова Н.Г. «Научные основы организации компьютерного тестирования по английскому языку в системе подготовки государственных служащих» //Орел: Издательство ОФ РАНХ и ГС,2013-212с. ISBN978-5-93179-322-2
2. Бовтенко, М.А. Компьютерная лингводидакти-ка Текст.: учебное пособие /М.А. Бовтенко. М., 2005. - 216с.
3. Ильченко О.А. Организационно-педагогические условия сетевого обучения. М., 2002
4. Кадневский, В.М. Генезис тестирования в истории отечественного образования Текст.: автореферат дис. докт. пед. наук: 13.00.01/В.М. Кадневский. Екатеринбург, 2006.-56с.
5. Овчаренко, В.П. Компьютерное тестирование как метод оценивания уровня сформирован-ности лингвистической компетенции (англий-
ский язык, неязыковой вуз) Текст.: дис. канд. пед. наук: 13.00.02 / В.П. Овчаренко. Таганрог, 2007. - 195с.
6. Приказ Министерства науки и высшего образования Российской Федерации от 13.08.2020 № 1011 «Об утверждении федерального государственного образовательного стандарта высшего образования - бакалавриат по направлению подготовки 40.03.01 Юриспруденция» (Зарегистрирован 07.09.2020 № 59673) http://publication.pravo.gov.ru/Docu-ment/View/0001202009070039 (дата обращения 14.07.22)
7. Углев, В.А. Системный подход к процессу обучающего компьютерного тестирования Текст. /
B.А. Углев, В.А. Устинов, Б.С. Добронец // Информационные технологии. 2008. - N4. -
C. 81-87.
8. Цатурова, И.А. Компьютерные технологии в обучении иностранным языкам Текст.: учебно-методическое пособие / И.А. Цатурова, A.A. Пе-тухова. -М.: Высш.шк., 2004. 95с.
9. Челышкова, М.Б. Компьютерная программа для профессиональной разработки тестов. Новая аттестационная технология абитуриентов Текст. / М.Б. Челышкова // Тезисы докладов Всероссийской научно-практической конференции. - М.: МПГУ, 1993. - С. 48.
10. Ялаева, Н.В. Компьютерное тестирование как средство интенсификации обучения английскому языку в юридическом вузе: специальность 13.00.08 «Теория и методика профессионального образования»: диссертация на соискание ученой степени кандидата педагогических наук / Ялаева Наталья Вячеславовна. -Екатеринбург, 2003. - 233 с. - EDN NMNMOP.
11. Borovkova, M.V. Computer-aided testing in modern electronic educational systems as a means of improving the quality of teaching foreign languages in law school / M.V. Borovkova, N.V. Yalaeva, N.V. Sadykova // Modern Pedagogical Education. - 2020. - No 3. - P. 60-63. - EDN YVGDDE.
12. Yalaeva, N. V. E-learning course as a means of information technology support to the learning process in university / N.V. Yalaeva, N.V. Sadykova, E.V. Zherebtsova // Modern Pedagogical Education. - 2020. - No 7. - P. 44-46. - EDN REHGBM.
IMPLEMENTATION OF OPPORTUNITIES OF
A LEARNING ENVIRONMENT FOR TEST
ADMINISTRATION IN A LAW SCHOOL
Yalaeva N.V., Sadykova N.V., Zherebtsova E.V.
Ural State Law University named after V.F. Yakovlev
The article gives a review of INDIGO and Moodle testing systems used by the Department of Russian, Foreign Languages and
Speech Culture at the Ural State Law University for independent assessment of students' knowledge at a particular stage of professional foreign language competence development and learning outcomes diagnostics. The authors compare these systems by different criteria, analyze their work experience and determine the features of test tasks bank structure, describe the examples of tests developed. Advantages and disadvantages of incorporating testing systems into universities' electronic information and education environment are outlined. To understand how the students feel about the quality of the organization of the testing process, an online survey was conducted. Analysis of the results of student survey revealed that the majority of students feel positive about the quality of knowledge testing with test systems.
Keywords: INDIGO testing system, learning management system, Moodle, electronic information and educational environment, assessment.
References
1. Akatova N.G. "Scientific basis for the organization of computer-based testing in English language in the training of civil servants" // Orel: Publishing house of Russian Academy of National Economy and Public Administration, 2013-212p. IS-BN978-5-93179-322-2
2. Bovtenko, M.A. Computer linguodidactics Text: textbook / M.A. Bovtenko. M., 2005. - 216c.
3. Ilchenko O.A. Organizational and Pedagogical Conditions of Network Teaching. M., 2002
4. Kadnevsky V.M. Genesis of Testing in the History of National Education Text: Abstract of Doctoral thesis in Pedagogy: 13.00.01/V.M. Kadnevsky. Ekaterinburg, 2006.
5. Ovcharenko, V.P. Computer-based Testing as a Method of Evaluation of the Formation Level of the Linguistic Competence (English, Non-linguistic Higher Education): Cand: 13.00.02 / V.P. Ovcharenko. Taganrog, 2007. 195s.
6. Order of the Ministry of Science and Higher Education of the Russian Federation of 13.08.2020 № 1011 "On approval of the federal state educational standard of higher education - Bachelor's degree in the direction of training 40.03.01 Jurisprudence" (Registered 07.09.2020 № 59673) http://publication.pravo. gov.ru/Document/View/0001202009070039 (date of access 14.07.22)
7. Uglev, V.A. Systemic approach to the process of training computer testing Text. / V.A. Uglev, V.A. Ustinov, B.S. Dobronets // Information technologies. 2008. - N4. - C. 81-87.
8. Tsaturova I.A. Computer technologies in teaching foreign languages Text: educational-methodical manual / I.A. Tsaturova, A.A. Petukhova. - M.: Higher school, 2004. 95c.
9. Chelyshkova M.B. Computer programme for professional test development. New attestation technology for entrants Text. / M.B. Chelyshkova // Theses of reports of All-Russian scientific-practical conference - M.: MPGU, 1993. - P. 48.
10. Yalaeva N.V. Computer-based testing as a tool to intensify training of English language in law school: speciality 13.00.08 "Theory and Methodology of Professional Education". D. thesis for the degree of Candidate of pedagogical sciences / Natalia V. Yalaeva. - Ekaterinburg, 2003. - 233 c. - EDN NMNMOP.
11. Borovkova, M.V. Computer-aided testing in modern electronic educational systems as a means of improving the quality of teaching foreign languages in law school / M.V. Borovkova, N.V. Yalaeva, N.V. Sadykova // Modern Pedagogical Education. - 2020. - No 3. - P. 60-63. - EDN YVGDDE.
12. Yalaeva, N. V. E-learning course as a means of information technology support to the learning process in university / N.V. Yalaeva, N.V. Sadykova, E.V. Zherebtsova // Modern Pedagogical Education. - 2020. - No 7. - P. 44-46. - EDN REHGBM.
C3
о
CO "O
1=1 А
—I
о
C3 t; о m О от
З
ы о со
