CC BY
10
Образование в странах мира
DOI: 10.14529/ped180310
A NEW MODEL OF COURSE ASSESSMENT BASED ON THE FLIPPED CLASSROOM APPROACH APPLIED TO "MODERN LOGISTICS TECHNOLOGY" COURSE
Chen Fang, [email protected]
Zhejiang Ocean University, Zhoushan, People's Republic of China
As an innovative approach, flipped classroom changes the "studying in class + practicing after class" mode into "studying before class + explaining in class" by reversing the two stages that of knowledge transfer and knowledge internalization. The changing of teaching mode brings the changing of assessment methods, because the traditional assessment methods are no longer suitable to the requirements of a new teaching model. In this paper at first, the drawbacks of traditional assessment were described, and then the diversified course assessment index system in on-line and offline forms was established. The course assessment was evaluated using the combination of Analytic Hierarchy Process (AHP) and Fuzzy Comprehensive Evaluation (FCE). This new assessment method changes the traditional result-oriented evaluation to the process-oriented one, the teacher evaluation to the students' evaluation, realizing the combination of quantitative and qualitative evaluation mode and promoting the students' autonomous learning, processive learning and experiential learning, giving full play to the feedback, improvement and strengthening function of the assessment.
Keywords: flipped classroom, course assessment, AHP-FCE.
1. Introduction
Modern Logistics Technology is a basic course, highlighting major and obtaining practical significance. Therefore, this course not only imparts knowledge, but forms personal direction and arouses students' interest. Students could identify areas of interest for future study.
The flipped classroom is when the teacher puts in the information environment learning resources in the form of videos. Students watch and learn the video resources before the class. Teachers and students work together to finish the homework in the classroom, work out collaborative inquiry, discuss questions and fulfill interactive activities in a new teaching mode. As an innovative form of classroom teaching, flipped classroom has subverted the inherent mode of traditional teaching, which is a major breakthrough in teaching reform under the condition of modern information technology development. Flipped classroom changes the "studying in class + practicing after class" into "studying before class + exploring in class" in the form of reversing the two stages of learning that of knowledge transfer and knowledge internalization [1]. The realization of flipped classroom can fully
exert the activity of students, improve the interaction between teacher and students, and enhance students' learning ability and cooperation ability. 2. The insufficiency of current assessment methods The course assessment is one of the most effective ways to test students' outcomes, which can be used to identify, regulate and promote the teaching. At the same time, it is an important link in the course teaching. Currently, like most other courses, the examining method of modern logistics technology included two parts, namely formal assessment (30%) and term examination (70%). The formal assessment is composed of attendance (30%), current results (30%) and homework (40%). However, for large groups' teaching there is no way to make a general survey for each class. There are less students participating in class discussions, and the plagiarism phenomenon is serious. The final assessment is usually 'one test for life', and the teacher only judges by the fact that whether the students have completed the examination paper. In addition, the number of classes is small, so the teacher who might teach, give the same tests and grade work. This pattern means a course teacher has a larger subjectivity
and randomness, which is not favorable for improvement of teachers' level and for mobilizing students' learning enthusiasm. Furthermore, a test consists of multiple choice questions, true and false statements, fill-in-the-blanks, quantitative problems, short answers, most of which are to test the memorized knowledge, ignoring the application of knowledge and personal development. Students mostly cram for the final; even someone who never pays attention in class will try to cheat at examination. Therefore, such a single, result-oriented and subjective assessment method neither assesses the extent of students' learning nor improves student's thinking, values, and efficiency. It is also not suitable for a new teaching model [2].
3. The mathematical model
of AHP-FCE
The analytic hierarchy process (AHP) is a structured technique for organizing and analyzing complex decisions, based on mathematics and psychology. It has particular application in group's decision-making and is used globally in a wide variety of situations. This method makes use of both qualitative and quantitative analysis to develop the inter-relationship among the indexes and establish the judgement matrix. In parallel, fuzzy comprehensive evaluation (FCE) method is a comprehensive evaluation method based on fuzzy mathematics, which can transform the qualitative evaluation into the quantitative evaluation according to the membership degree theory of fuzzy mathematics. This method can solve the ambiguous and hard-to-quantify problems and, therefore, is suitable for solving various non-deterministic problems. Usually, the final examination uses a grading system (excellent, good, medium, passing, failing), but the definition of each level is not clear and the grade is ambiguous. So, the fuzzy comprehensive evaluation is an effective evaluation method and, particularly suitable, for the flipped classroom assessment. So, this paper combined AHP/FCE methods that provided a solution to the problem solving, which makes use of the AHP to calculate the weight values first, and, then, implement the FCE to carry out a from-quantitative-to-quantitative assessment.
The main process is as follows [3, 4]:
(1) Identifying the affecting factor set
For fuzzy evaluation, factors that affect the parameter's value determination should first be identified. If the known affecting factors are ui, u2,..., um, the set these parameters can make
up is U = {u1,u2,...,um}, and this factor set is
a common set.
(2) Determining factor weights set
Each factor has a different impact on, or importance to the value determination of parameters. In other words, the factors have different weights for parameter values. The set composed of various weights of all factors for parameter value determination is called the factor weights set, which is represented as A = |a1,a2,...,am}.
If ai represents the weight of factor No. i, the weight of each factor should satisfy Equation (1).
& = Ui > 0. (1)
i=1
The Analytic Hierarchy Process operation can be divided into the following five steps:
Step 1: Define the decision-making problem.
Step 2: Create a hierarchical structure.
Step 3: Create a pairwise comparison matrix. Table 1 is the evaluation measurement and relative definition of AHP.
Step 4: Calculate the eigen values.
Step 5: Conformance test.
Eigen values in step 4 can be calculated in the following ways: Normalization of the Row Average (NRA), Average of Normalized Columns (ANC), Normalization of the Geometric Mean of the Rows (NGM), and Normalization of the Average Reciprocal of Columns. The consistency test in step five is to make sure that the evaluation results are consistent when the experts are conducting pairwise comparisons, which means that the experts' preference for criteria is transitive. Saaty suggested that consistency index (CI) and consistency ratio (CR) be used for the test. If CI and CR are both less than 0.1, it means that the pairwise matrix is consistent. If the matrix is not consistent, the equation of CI and CR is as follows:
A — n
CI = —max-, (2)
n — 1
A max is the largest eigenvalue of the matrix, n is matrix order (number of parameters).
CR = CI/RI
CR < 0.1ok
C:R = Consistency ratio
C:I: = Consistency index
R:I: = Random index, (3)
where n is the number of evaluation criteria; RI is random index whose value increases if the number of criteria increases, as shown in Table 2.
Инновационная модель оценки курса «Современные технологии логистики»...
Table 1
Evaluation Measurement and Relative Definition of Analytic Hierarchy Process
Evaluation . .
measurement
1 Equal importance
3 Slight importance
5 Essential importance
7 Very strong importance
9 Absolute importance
2, 4, 6, 8 Intermediate
Table 2
Table of Random Indexes
Order N 1 2 3 4 5 6 7 8
RI 0 0 0.58 0.90 1.12 1.24 1.32 1.41
(3) Establishing a parameter evaluation set An evaluation set is the set of various possible
evaluation results given by evaluators for the evaluation objects, shown as V. V = |v1,v2,...,vn};
vi(i = 1, 2, ..., n) represents all of the possible evaluation results. The purpose of fuzzy evaluation is to obtain an optimal evaluation result from the evaluation set based on the comprehensive consideration of all affecting factors. As vi and V have the relation of common set, the evaluation set is also a common set.
(4) Building a single factor evaluation matrix A single factor fuzzy evaluation system evaluates single factors to determine the membership of an evaluation object to an evaluation set. The evaluation object is evaluated as the No. i factor ui in the factor set, and the membership of No. j factor Vi in the evaluation set is rij, so the evaluation result of No. i factor ui can be expressed as:
R: = + -Ü2-.
(4)
where Ri is called a single factor evaluation set. As a fuzzy subset, it can be expressed as Ri = |rü,ri2,...,rin}. Similarly, the single factor
evaluation set corresponding to each factor is as follows:
Ri = {
r2 = {r
Rm ={r
11 , ri2, rin }
2n
}
(5)
The memberships of each single factor evaluation set constitute a fuzzy matrix, where R is called the single factor evaluation matrix, as shown in Equation (6). R is a fuzzy matrix, and it can also be regarded as a U-V fuzzy relationship matrix, called fuzzy transformation.
(5) When conducting fuzzy composition, if the fuzzy evaluation matrix of a program to an evaluation object is:
R =
X " rn r12 • ' r1j • ' r1m
R2 r21 r22 • ' r2j • r2m
Ri = ri1 ri2 • • rj r im
_Rn _ _ Гп1 rn2 • ' rnj ^ r nm
(6)
then the comprehensive fuzzy evaluation with weighting considered and the product of fuzzy matrix is:
B = A • R = [b1,b2,...,bJ,...,bm ] (7)
In the above equation, the symbol "•" represents fuzzy composition. The weighted fuzzy matrix A and factor evaluation matrix R have a variety of compositions. This study will use different compositions to work out various evaluation results for comparison and analysis. The model selected is as follows:
when the composition is done by M(», ©) algorithm
bj = min j 1, J а;Гу L ; j = 1,2,...,n .
(8)
i=1
4. The course assessment based on AHP-FCE
4.1. Establishment of the evaluation indicator system
On the foundation of FCE, the selection of evaluation indexes will directly affect the accuracy of comprehensive evaluation. Basing on researches of domestic and foreign scholars for the examining model and the exam outline of the course, the new examination method of the Modern Logistics Technology is as follows: formal assessment and project result. In this paper, the evaluation system is composed of 4 indicators and 12 corresponding sub-indicators. This is shown in Table 3.
Table 3
The Evaluation Index System of Flipped Classroom Course Assessment
Level A Level B Level C Evaluator
Course assessment of Modern Logistics Technology Formal assessment Before class X1 Test performance X11 Teacher
The quality of raising question X12 Teacher and students
Online learning time X13 Teacher and students
In class X2 Attendance X21 Teacher and students
Classroom discussion X22 Teacher and students
Classroom notes X23 Teacher and students
After class X3 Spread and transfer the knowledge X31 Teacher and students
Homework X32 Teacher
Project result Final X4 The content of the topic X41 Teacher and students
Team collaboration X42 Teacher and students
The quality of project X43 Teacher and students
Question and answer X44 Teacher and students
4.2. Determination of weights of AHP-based indicator system The indicator system can be divided into three levels: level A, level B, level C from top to bottom. In the evaluation indicator system, the main criterion level and branch criterion level represent Level B indicator and Level C indicator respectively: X = (Xi, X2, X3) and Xi = (Xn, X12,
X13X X2 = (X21, X^ X23), X3 = (X3U X32), X4 =
= (X41, X42, X43, X44) specifically.
According to the level structure of the evaluation indicator system of course assessment, the judgment matrix P of level B indicator X can be built based upon the importance of each factor, as shown in Table 4.
Table 4
Judgment Matrix P of indicator X Level B and conclusion
P X1 X2 X3 X4
X1 1 1/2 4 3
X2 2 1 5 4
X3 1/4 1/5 1 1/4
X4 1/3 1/4 4 1
weight 0.296 0.478 0.069 0.157
^max = 4.194; CI = 0.0647; CR = 0.0727 < 0.1, Conform to the consistency requirement
Upon calculation, the maximum eigenvalue of the judgment matrix P is Amax = 4.194. In order to check the consistency of the matrix, it is required to calculate the consistency indicator. 4 194 - 4
= 0.065 (9)
л
CI _ ^max П
n -1 4 -1 The average random consistency indicator is RI = 0.89. The random consistency ratio:
CR _ Cl _ 0065 _ 0.073 RI 0.89
(10)
Therefore, the AHP sequencing result is considered to be of high consistency, that is, the allocation of weight coefficient is very reasonable.
The corresponding eigenvector: (0.296 0.478 0.069 0.157).
Similarly, the weight ratios of Level C indicators X1, X2, X3 and X4 are as follows (Table 5-8).
Table 5
Judgment Matrix P1 of Level C Indicator Xi and conclusion
P1 X11 X12 X13
X11 1 5 3
X12 1/5 1 1/2
X13 1/3 2 1
Weight 0.648 0.122 0.230
Xmax = 3.004; CI = 0.0020; CR = 0.0038 < 0.1, Conform to the consistency requirement
Table 6
Judgment Matrix P2 of Level C Indicator X2 and conclusion
P2 X21 X22 X23
X21 1 2 5
X22 1/2 1 3
X23 1/5 1/3 1
Weight 0.582 0.309 0.109
Xmax = 3.0037; CI = 0.0019; CR = 0.0036 < 0.1, Conform to the consistency requirement
Table 7
Judgment Matrix P3 of Level C Indicator X3 and conclusion
P3 X31 X32
X31 1 1
X32 1 1
Weight 0.5 0.5
Xmax = 2.000; CI = 0; CR = 0 < 0.1, Conform to the consistency requirement
Инновационная модель оценки курса «Современные технологии логистики»...
Table 8
Judgment Matrix P4 of Level C Indicator X4 and conclusion
P4 Х41 Х42 Х43 Х44
Х41 1 1/2 1/4 1/3
Х42 2 1 1/3 1/2
Х43 4 4 1 2
Х44 3 3 1/2 1
Weight 0.094 0.133 0.479 0.294
Xmax = 4.082; CI = 0.0273; CR = 0.0307 < 0.1, Conform to the consistency requirement
4.3. Establishing decision set for the course assessment
We established an evaluation system, which adopts five evaluation ratings (excellent, good, medium, pass, fail) based upon group discussion and expert evaluation.
4.4. Establishment of the fuzzy relationship from the factors set to the decision set
There are qualitative and quantitative indexes in Table 3. For the quantitative index (x11, x32), we can convert hundred mark system into five grade system, such as excellent (90-100), good (80-89), fair (70-79), pass (60-69), fail (0-59). Assuming there are 20 tests in a semester, the result of a student is 8 times (90-100), 5 times (80-89), 5 times (70-79), 2 times (60-69), the fuzzy matrix of it is (0.4, 0.25, 0.25, 0.1, 0). The remaining indicators are qualitative ones. Assuming there are 39 students, one teacher, in total 40 in a class, for a certain index, 12 people think it is excellent, 15 think it is good, 13 think it is medium, so the fuzzy matrix of it is (0.3, 0.375, 0.325, 0.0).
The whole class and a teacher subjectively evaluate and grade all factors of the indicator system according to the actual conditions. Based upon the analysis of statistical data of sampling survey FCE matrix set can be gained. Here is an example of a student's performance in a semester.
"0.2 0.4 0.2 0.2 0" "0.3 0.5 0.2 0 0"
Rp1 - 03 0.5 0.1 0.1 0 ; Rp2 - 0.2 0.3 0.4 0.1 0
0.2 0.4 0.3 0.1 0 0 0.2 0.5 0.3 0
Rp3 -
0
0.4
0.3 0.5
0.4 0.1
0.3 0
Rp4 -
0.2 0 0.1 0.4
0.3 0.2 0.2 0.5
0.4 0.4 0.3 0.1
0.1 0.4 0.4 0
4.5. Fuzzy comprehensive evaluation (1) Fuzzy evaluation of index layer
' 0.2 0.4 0.2 0.2 01
A1 - W • R -(0.648 0.122 0.230)» 0.3 0.5 0.1 0.1 0
V °.2 0.4 0.3 0.1 0,
= (0.212 0.412 0.211 0.165 0)
Upon normalization, the comprehensive evaluation vector is, so similarly,
A2 -(0.236 0.406 0.294 0.064 0
A3 -(0.200 0.400 0.250 0.150 0
A4 -(0.184 0.298 0.264 0.254 0
(2) Fuzzy evaluation to the overall performance of the student A = W • R = (0.296 0.478 0.069 0.157)
= (0.218 0.391 0.262 0.136 0).
' 0.212 0.412 0.211 0.165 01
0.236 0.406 0.294 0.064 0
0.200 0.400 0.250 0.150 0
v 0.184 0.298 0.264 0.254 0,
(11)
(12)
(13)
(14)
References
1. Qian Li. Local practice and thinking of flipped classroom in America. Jiangsu Education Research, 2013, vol. 12, pp. 66-68.
2. Chen Fang, Liu Hu, Ai Wanzhen. Reform on teaching mode of Modern Logistics Technology based on the flipped classroom. Water transport management, 2016, vol. 7, pp. 38-40.
3. Shih-Wen Hsiao, Ya-Chuan KoA. Study on bicycle appearance preference by using FCE and FAHP. International Journal of Industrial Ergonomics, 2013, vol. 43, pp. 264-273.
4. Jin Weiweia, An Weia, Zhao Yupenga, Qiu Zhaoyua, Li Jianweia, Song Shasha. Research on Evaluation of Emergency Response Capacity of Oil Spill Emergency Vessels. International Oil Spill Response Technical Seminar. Aquatic Procedia, 2015, vol. 3, pp. 66-73.
ИННОВАЦИОННАЯ МОДЕЛЬ ОЦЕНКИ
КУРСА «СОВРЕМЕННЫЕ ТЕХНОЛОГИИ ЛОГИСТИКИ»
НА ОСНОВЕ ТЕХНОЛОГИИ «ПЕРЕВЕРНУТОГО КЛАССА»
Чжэцзянский океанический университет, г. Чжоушань, Китайская Народная Республика
Технология «перевернутого класса» представляет собой инновационную форму процесса изучения учебной информации, в которой изучение материала идет перед занятием, а проверка усвоения материала и консультирование происходит на аудиторных занятиях, что модифицирует традиционную форму, когда изучение новой информации идет под руководством преподавателя, а дома студенты отрабатывают учебный материал. Изменение формы обучения приводит к изменению методов оценки, поскольку традиционные методы оценки больше не удовлетворяют требованиям новой учебной модели. В этой статье описаны недостатки традиционных методов оценки результатов обучения, и представлена диверсифицированная система оценки курса, интегрирующая онлайновые и оффлайновые формы. Результаты обучения были оценены с использованием метода многофакторного анализа представляющего собой комплексную оценку обучения. При новом методе оценки акцент делается на процессуальной и эмпирической составляющих усвоения содержания образования обучающимися, поощрении их самостоятельности, сочетании количественного и качественного способов оценивания, что способствует полноценной обратной связи, в отличие от традиционной оценки, ориентированной на результат.
Ключевые слова: технология «перевернутого класса», оценка курса, процесс аналитической иерархии, многофакторный анализ.
Чэнь Фан, преподаватель, Чжэцзянский океанический университет, г. Чжоушань, Китайская Народная Республика, [email protected]
Received 7April 20Ï8
УДК 37S.01 + 37S.14.015.62
DOI: 10.1462e/ped1S0310
Чэнь Фан
Поступила в редакцию 7 апреля 20Ï8 г.
ОБРАЗЕЦ ЦИТИРОВАНИЯ
FOR CITATION
Chen Fang. A New Model of Course Assessment Based on the Flipped Classroom Approach Applied to "Modern Logistics Technology" Course / Chen Fang // Вестник ЮУрГУ. Серия «Образование. Педагогические науки». - 2018. - Т. 10, № 3. - С. 95-100. DOI: 10.14529/ped180310
Chen Fang. A New Model of Course Assessment Based on the Flipped Classroom Approach Applied to "Modern Logistics Technology" Course. Bulletin of the South Ural State University. Ser. Education. Educational Sciences. 2018, vol. 10, no. 3, pp. 95-100. DOI: 10.14529/ped180310
