CC BY
4COMPUTER SIMULATION OF LEARNING AND FORGETTING OF THE LOGICALLY RELATED INFORMATION
Abstract
The assimilation and forgetting processes of the logically related information, which can be represented in the form of a large number of information blocks (ideas) consisting of the learning material elements (LME), are studied by computer simulation methods. Three models are proposed. Model 1 takes into account that: 1) LME can be of two types: used and not used in the student's subsequent activities; 2) as the number of references to each LME increases, the forgetting speed decreases. It was studied the dependence of the student's knowledge on the time and the dependence of the forgetting speed on the lesson duration and the number of LME-s in the block. Model 2 allowes to prove the known fact, that at increase of the transfer speed of the information by the teacher the speed of its assimilation by the schoolboy at first grows, reaches a maximum and then decreases. Model 3 takes into account the division of the pupil's knowledge into three categories; with its help the graphs of the dependence of the knowledge quantities for each category on the time at the known distribution of educational material was obtained.
Keywords
didactics, training, computer modeling, pupil, teacher, level of knowledge
AUTHOR
Robert V. Mayer
PhD, Professor, Physics and Didactics of Physics Dertment Glazov State Pedagogical Institute after V.G. Korolenko.
25, Pervomayskaya Str., Glazov, 427621, Rusia.
Introduction
One of the development directions of the modern training theory consists in application of the information and cybernetic approach to the analysis of a didactic systems, and using of mathematical and imitating (or computer) models (Hunt, 2007; Razumovskij & Mayer, 2004; Titov & Rjabinova, 2011). This is due to the fact that many of the values that characterize the state of the "teacher-pupil" system (the level of the teacher's requirements, the quantity of student's knowledge) can be measured. For definition of amount of information in the textbook the semantic method consisting in calculation of semantic units, notions and links between them can be used. For a unit of measure of knowledge quantity (or amount of information acquired by the pupil) a notion, elementary judgment or a mathematical formula can be chosen. Each learning material element (LME) is characterized by a complexity which is proportional to the quantity of efforts or time demanded for it's assimilation.
The essence of the simulation method is consists in creation of computer model of the "teacher-pupil" system and carrying out a series of numerical experiments for the purpose of understanding of the system behavior or an assessment of various management strategies for its operation (Mayer, 2014, 2015, 2016). It helps to investigate process of training in the cases when carrying out a pedagogical experiment is impossible or
impractical. In order to save a community of conclusions dimensionless modeling is used; so the quantity of the transferred information (or the amount of pupil's knowledge) and time is measured in the conventional units CUI or CUT.
Methodological Framework
Methodological basis of the research are ideas and works by N. Winer, K. Shannon, F. Rosenblatt, V. M. Glushkov, D. A. Pospelov (cybernetics, the theory of information), R. Atkinson, L. P. Leontyev (Leont'ev & Gohman, 1984), F. S. Roberts (Roberts, 1986), L. B. Itel'son (mathematical modelling of training), B. Skinner, N. Krauder, S. I. Arkhangelsky, V. P. Bespalko, E. I. Mashbits, V. E. Firstov, V. S. Avanesov, I. V. Robert (cybernetic approach in pedagogy, the programmed training and the automated training systems). Logicality and formalization, reproducibility and concreteness distinguishes the computer modeling method from "the method of qualitative reasonings or considerations". But there is the problem of correct choice of the model parameters. R. Shannon noted that almost any parameters of the simulation model often determines only on the basis of the experts assumptions, analyzing a small amount of data (Shannon, 1975).
Results and Discussion
3.1. Learning and forgetting logically related material (model 1)
Let us assume that information given by the teacher consists of separate blocks (ideas) which are divided into the learning material elements (LME-s): notions, concepts, formulas, foreign words, etc. Training is similar to viewing of the film during which on the screen information blocks (the text, drawings, formulas) are consistently replaced. At that all LME-s, the included in blocks, are divided into two categories: 1) meeting and used by pupil in the subsequent life and training; 2) not meeting further.
The studied material can be specified by the following characteristics: 1) the number of blocks (chains of LME-s); 2) the average length of the blocks lcp and the scatter Al; 3) the share D of LME-s used by pupil in future activities; 4) the average coefficient of assimilation a for each LME. Knowledge of given (iJ)-LME is determined by the number Pij from the interval [0; 1], which shows the probability of a correct answer to the corresponding elementary question. Let lcp = 10 and Al = 2, then the length of the chain is given randomly lt = 8 + Round(4xs); here and in the sequel xs is a random variable uniformly distributed in the interval [0,1]. When studying the j-th LME from the i-th block during the time At, the probability of the student's correct answer to the corresponding elementary question grows according to the law: pff1 = pfj + a(l - pfy)At, (a =0.1); the counter of addresses to the given (tJ)-LME s^ increases by 1. As the number of reference Sij to this LME increases, the forgetting coefficient decreases according to the law: gtj = 3 • 10-4 + 0,05 • exp(-0,001 • stj). If the pupil does not work with (iJ)-LME, then, due to forgetting, knowledge of this LME during the time At decreases so: pfy+1 = pfy(1 - g^At). In order to take into account the using of some LME in the pupil's every day life, a matrix dij, is created, whose elements with a given probability D are equal to 1, and with probability (1 - D) - zero. During the elementary interval At with some probability q =0,025 the student refers to each LME with d^ = 1, using it in his activity; at this si7-increases by 1, and pfy increases by a(l-pfy)At. The sums of probabilities pfy for all LME-s with dij = 1 and d^ = 0 are denoted by Z1 and Z2 accordingly.
Program 1 (Free Pascal)
{$N+> uses crt, graph; const dt=0.01; a=0.1; Y=650; str=50; st=15; Mt=0.8; var t,tr,g,SR,SP}SQ: single; i, j ,k,tt^DVjMV^kk: integer; p: array[1..str, l..st] of single; dd: array[1..str,1..st]of integer; s:array[1..str,1..st] of longint; 1,R: array[1..str]of integer;
Procedure Test; Label ra; begin SR:=0; For k:=l to 100 do begin For i:=l to str do begin j:=l; tr:=1[i]*dt*l.3; R[i]:=0; Repeat tr:=tr-dt; If random( 100)/100<p[i,j] then inc(j); If j>=l[i] then begin R[i]:=l; goto m; end; If (tr<0) then R[i]:=0; until tr<0; m: end; For i:=l to str do SR:=SR+R[i];end; circle(10+round(t*Mt),Y-round(SR/10),2); SP:=0; For i:=l to str do For j:=l to l[i] do begin If dd[i,j]=l then SP:=SP+p[i,j];end; circle(10+round(t*Mt), Y-round(SP),1); SQ:=0; For i:=l to str do For j:=l to 1[i] do begin If dd[i, j]=0 then SQ:=SQ+p[i,j]; end; circle(10+round(t*Mt),Y-round(SQ+SP),1); end; Procedure Zabivanie; begin For i:=l to str do For j:=l to l[i] do begin If (dd[i,j]=l)and(random(100)<2.5) then begin p[i,j] :=p[i,j]+a*(l-p[i,j])*dt; inc(s[i,j]); end; end; For i:=l to str do For j:=l to 1[i] do begin g:=4E-4 +0.05*exp(-lE-3*s[i,j]); p[i,j]:=p[i,j]*(l-g*dt); end; end; Procedure Zabivan; begin For i:=l to str do For j:=l to l[i] do begin g:= 4E-4+0.05*exp(-lE-3*s[i,j]); p[i,j]:=p[i,j]*(l-g*dt); end; end; Procedure Obuchen;begin For i:=l to 50 do For j:=l to 1[i] do begin inc(s[i, j]) ;p[i>j] :=p[i> j]+a*(l-p[i, j] )*dt; end; Zabivan; end;
BEGIN DV:=Detect; InitGraph(DV,MV,5'); Randomize; For i:=l to str do l[i]:=8 +round(random(100)/25); For i:=l to str do For j:=l to 1[i] do begin If random(100)/100<0.333 then dd[i,j]:=l else dd[i,j]:=0; end; line(0,Y,1600,Y); Repeat t:=t+dt; If (t<30)or((t>220)and(t<245)) then Obuchen else Zabivanie; inc(o); If o>100 then begin o:=0; Test; end; circle(10+round(t*15),Y,1); until (KeyPressed)or(t>2000); CloseGraph; END.
The pupil's state at each moment of time is determined by the matrix of probabilities p^, where j =1, 2, ..., lt; the value p^ shows the probability of knowing the j th LME from the i -th chain. To assess the pupil's knowledge and draw the Zn(t) graph, the program simulates a multiple "testing" of the pupil at each time step (i.e. at times tk = kAt). The knowledge of the student of the i -th block is defined as follows: the computer simulates the response of the pupil, in which he sequentially expounds the 1 -st LME, the 2-nd LME, ..., lt -th LME of the i -th chain for a given time tr = 1,3 • lt • At. The correct answer to the question corresponding to the j -th LME from the i -th block is modeling as a random process occurring with probability ptj: if xs <ptj, then the pupil answered correctly, and otherwise - no. If the answer is incorrect, the pupil again tries to show knowledge of this LME; if the answer is correct, he moves on to the next LME. If all li LME-s of the i -th chain are reproduced or executed correctly during the time tr, then it is considered, that the pupil knows the t-th information block. The pupil's knowledge Zn(t) is equal to the number of information blocks (ideas) that he can reproduce. With this "testing" the pupil's knowledge does not increase, the probabilities ptj remain unchanged.
FIGURE 1. MODELING THE ASSIMILATION AND FORGETTING OF THE LOGICALLY RELATED INFORMATION.
Fig. 1.1-1.4 shows the results of the assimilation and forgetting simulation of the logically related information at D = 0.33 and different lesson durations T: 1) T = 30 CUT (conventional unit of time); 2) T = 45 CUT; 3) T = 60 CUT; 4) during two lessons on 30 CUT. Analyzing the graphs Zn(t), we can distinguish three phases of forgetting: 1) the amount of pupil's knowledge first remains practically constant, exceeding 0,9/0 (I0 = N), that is, he can reproduce almost all studied ideas, solve all problems, etc.; 2) the quantity of knowledge decreases almost proportionally of time; 3) pupil remember no more 0,1/0, that is he can not reproduce the studied ideas, but at the same time well remembers separate LME-s, which he met or used in the life and activity. As can be seen from the graphs on fig.1, at t ^ ™ the number of unused knowledge in life Z2 = Z12 - Z1 aspires to zero then faster, than less time of training T. The amount of the used knowledge Z1 after ending of training decreases much more slowly or remains constant.
FIGURE 2. MODELING OF TRAINING ON THREE LESSONS
At study of the same material during two lessons sessions (fig. 1.4) after the first lesson (30 < t < 200), the knowledge quantity Zn decreases rapidly due to forgetting of the unused LME (i.e. Z2 reduction). LME-s, used by the pupil daily, are forgotten slowly; Z1 decreases slightly. During the second lesson all LME-s are remembered even more strongly (the number of addresses s ¿y increases, the speed of forgetting decreases). At t > 230 forgetting occurs, Z2 decreases more slowly than after the first lesson. The first phase of forgetting (Z(t) > 0,9 /0) lasts longer (^ 550 CUT). In fig. 2 shows the results of modeling in the case of three lessons.
FIGURE 3. DEPENDENCE OF THE TIME OF THE HALF INFORMATION FORGETTING tP ON T AND lcp.
Practical importance has the dependence of the forgetting time tp of the half studied information on the duration of lesson T and on the average length lcp of the chain of LME-s. With increasing T (at constant lcp = 10) the number of addresses to each LME increases, the strength of assimilation and forgetting time tp grows (fig. 3.1). As the average length lcp of the block decreases, the learning material is remembered more firmly, and the forgetting time t3 of the half studied information is growing. Increasing the length of the LME chain at fixed T = 45 CUT reduces the probability of the pupil repeating of the corresponding block for the given time tr, lowers the time tp of forgetting of the half studied information (fig. 3.2).
3.2. Dependence of assimilation on the speed of the information reporting (model 2)
Still we'll assume, that the educational material consists of N separate information blocks (ideas, reasonings, conclusions of the formulas etc.). The pupil understands the new block of the given information if he/she understands each LME entering in this block. If the pupil has not acquired any LME in the block, then he does not acquire the block in general (or can not solve a problem, prove a theorem, translate a sentence). Let in the block m LME-s, then assimilation time of one block by the pupil can not exceed t1 = m/u where u - the speed of reporting of information by the teacher (number of LME-s for one CUT). When the pupil understands all LME-s, he understands this block. Probability of understanding of one LME from the first time is equal p = 0,7 - 0,9. If the pupil has not understood LME from the first time, then he addresses to it again and again as long as
does not understand, or until the end of time t1 devoted to the learning of this block. Let all LME-s have information volume I1 =1. Time of the single appeal of the pupil to every LME top = 1/vM, where vM - the speed of his mental activity in CUT-1. The more cogitative actions for 1 CUT the pupil makes, the quicker he/she will consider and understand this LME.
The program 2, which allows to study the dependence of the quantity of acquired knowledge on the arrival rate of the teacher's information consists of the time cycle with step 0,01. Before "assimilation" of the block variables EUM and flag are nullifies, the counter of blocks Nh, increases on 1, and also calculates time U= —, which is
Ui 1u
corresponding on assimilation of one block. The quantity of LME-s in the block m changes randomly: with probability 0,3 m =4; with probability 0,3 m = 6; with probability 0,4 m =
5. On average m = 5. When pupil's operating time with one LME kdt exceeds value top =
1
—, the program simulates a random process of understanding of LME with probability p.
if random value x of an interval [0; 1] is less then p, then it means what the pupil has "understood" the given LME and the variable EUM increases on 1. if in this case EUM = m, that is the pupil has "understood" all block from m LME-s, then Ponm increases on 1, the flag is raised (flag = l). if the pupil does not manage to understand all m LME in time t1, then we will assume that he has not "understood" this information block. After this the pupil "learns" the second block, the third block a nd etc. This is repeated until NM exceed 2000.
Program 2 (Free Pascal)
FIGURE 4. DEPENDENCE OF ASSIMILATION ON THE SPEED OF THE INFORMATION REPORTING
The program displays the number of the reported blocks NM, number of the understood blocks Ponm and the spent time At that allows to determine the transfer
speed
u
= ^ = 0,2u
At
and speed of the knowledge assimilation V =
Ponbi At
(in the
block/CUT). In fig. 4.1 the graphs of dependence of the assimilation speed V on the speed u of the information reporting by the teacher (in LME/CUT) at various values of the pupil's speed of thinking vM are presented. If the speed u of the information reporting by the
teacher is small then the assimilation speed is equal V = —^ = u' = 0,2u. The pupil with this speed of mental operations manages to understand almost all blocks which are reported by the teacher. Further increase in the speed of information transfer u leads to the fact that the pupil does not have time to accept it completely: the assimilation rate V reaches a maximum and then decreases. The more speed vM of the pupil's mental operations, the greater maximum possible speed of assimilation V and corresponding to it rate u of the teacher's information transfer. In the book (Leont'ev & Gohman, 1984, pp. 108-157) various mathematical models Ip = y(I) connecting volume Ip of the material acquired by pupil and the information I stated at lecture are considered. Presented in fig. 4.1 curves are similar to the graphic of the function Ip = cp(I). This is because I = u'T = 0,2uT, Ip = VT, where T - the lecture duration. The coefficient of material understanding is equal to the relation of number of the understood blocks to the total information
reported by the teacher: K = y or K = At small speeds of information transfer u the
pupil understands practically everything that is reported by the teacher, therefore K =1. In process of increase u the understanding coefficient K gradually decreases to 0 (fig. 4.2). At increase in the speed of mental operations vM the speed of reporting of information u by the teacher, which is corresponding to K = 0,5, grows.
Using the spreadsheets Excel, it was succeeded to pick up the function
corresponding to the turned-out graphs on fig. 4.1 rather precisely. For vM = 1,5: V("') = = , u' = u'5,a =27 CUT, b = °,241/CUT-
Here u' and V shows the quantity of the transferred and acquired blocks for one CUT. If u < 0,7, then 1 + exp(a(0,2u - b)) « 1, the rate of increase of the pupil's knowledge (in block/CUT) is equal to the speed of the teacher's knowledge transfer u' = 0,2u. If u > 1, then V decreases, aspiring to 0. In the training process the pupil's speed of thinking vM increases, ability to acquire new material grows. The following values of parameters corresponds to speeds vM = 1, 1,5 and 2: a = 39, b = 0,16; a = 27, b = 0,24 and a = 18, b = 0,32. If vM grows, then a decreases, and b increases. The resulting graphics can be interpreted differently. For example, pupil solves a problems, each of which requires m complex operations. The probability of correct execution of each operation equals p. If the pupil does not fulfill the given operation from the first attempt, he/she makes another attempt, and so on, each time spending top = 1/vM CUT. For each task a fixed time t1 = m/u is given. The graphs in fig. 4.2 show the dependence of the problem solving probability on the speed of their arrival u' = 0,2u (inversely proportional t1), with different speeds of thinking vM.
3.3. Use of dependence V(u') at construction of the multicomponent model of the pupil
Suppose that studied theme includes N elements of learning material (LME-s), which are connected with each other and the teacher requires mastering of the entire study information, i.e. the teacher's level requirements L equal to the quantity of the knowledge I(t) reported by him. We assume that the complexity St of the i -th LME is proportional to the time and pupil's efforts required for the assimilation of the LME. If for
simplest LME S = l, then for more complex LME S is more than 1 (for example, 1.8 or 3.2). Level of requirements imposed by the teacher is equal L = S1+S2+^+SN. if all N LME-s have complexity 1, then L = N. Speed of information transfer is equal u = dl / dt = dL / dt (CUT-1). Suppose that: 1. The state of the pupil at each time point is determined by the quantity of weak (poor) knowledge Z1, the quantity of abilities Z2 and the quantity of skills Z3 (solid or strong knowledge). Weak knowledge is forgotten quicker than strong knowledge. 2. in the process of training (k = l) the quantity of the pupil's weak knowledge increases, and part of the weak knowledge turn into the stronger knowledge (at first in abilities Z2, and then in skills Z3). 3. in the absence of training (k = 0) occurs forgetting: strong knowledge (skills) gradually turns into less strong, and the amount of weak knowledge decreases under the exponential law (Mayer, 2014, 2015).
Program 3 (Free Pascal)
{$N+}Uses crt, graph; Const dt=0.005; Mt=0.5; Mz=20;a=0.004;e=2.72;gl=7E-4; g2=gl/e; g3=g2/e; al=0.01; a2=al/e; Var DV,MV,k,p ¡integer; Z1,Z2,Z3,Z,u,t, L: single;
BEGIN DV:=Detect; InitGraph(DV,MV,JJ); Repeat t:=t+dt; u:=0; p:=0; k:=0; If sin(0.02*t)>0.96 then begin u:=0.2; k:=1; end; If sin(0.02*t-2)>0.96 then begin k:=l; p:=l; circle(round(Mt*t),500-round(Mz*L),1); end else begin p:=0; end; Z:=Z1+Z2+Z3; L:=Z+1; Z1:=Zl+(k*u/(l+exp(30*(u-0.235)))+p*a*(L-Z)-k*al*Zl -(1-k)*gl*Zl)*dt; Z2:=Z2+ (k*al*Zl-k*a2*Z2-(1-k)*g2*Z2)*dt; Z3:=Z3+(k*a2*Z2-(1 -k)*g3*Z3)*dt;circle(round(Mt*t),500-round(Mz*(Z1+Z2+Z3)),1); circle(round(Mt *t),500-round(Mz*(Z2+Z3)),1); circle(round(Mt*t),500-round(Mz*Z3),1); circlet round(Mt*t),500-round(Mz*u*120),1); until Keypressed; CloseGraph; END.
Using dependence V(u'), we will simulate the training process. Let during the day the teacher conduct two lessons for 2 hours everyone. At the first lesson the teacher gives new information with the speed u' = 0,2 block/CUT. After a small break the second lesson at which the pupil repeats the studied material follows. Level of the teacher's requirements L at repetition on 1 LME exceeds the level of the pupil's knowledge Z. Next day everything repeats. Process of training can be described by means of such mathematical model:
dZ1 k • u'
—;— = --——-^ ^ ^ + pa(L — Z) — ka1Z1 — (l — k^v-iZ-t,
dt l + exp(30(u' — 0,24)) H y J 1 1 y Ji1 1
^r = ka1Z1 — ka2Z2 — (l — k)y2Z2, ^ = ka2Z2 — (l — k)Y3Z3,
where Z = Z1+Z2+Z3. During the transfer of new material k = l, p = 0, u' = 0,2; there is an increase in amount of weak knowledge Z1, a part of weak knowledge become strong, the values Z2 and Z3 grows. During repetition k = l, p = l, u' = 0; therefore Zx slightly increases with a speed which is proportional difference L—Z, where L(t) - the level of the teacher's requirements. The knowledge becomes stronger, the values Z2 and Z3 grows. in the absence of training k = 0, p = 0, there is a forgetting, the knowledge of the first, second and third categories decreases with the speeds y1Z1, y2Z2 and y3Z3. The program 3 is used; results of modeling are presented in fig. 5. it is visible how during training (first lessons) because of the new information obtaining the total knowledge Z(t) of the pupil grows; during repetition (second lessons) the strong knowledge of the second and third categories Z23 =Z2+Z3 increases.
FIGURE 5. INCREASING THE QUANTITY OF KNOWLEDGE OVER TIME.
Conclusion
In the article the imitating model of learning and forgetting of the logically linked information, representable as set of information blocks (ideas, reasonings) consisting of separate LME-s is considered. All LME-s are divided into two categories: 1) used by the pupil in the further live and activity; 2) not used in future. After ending of training the second category LME-s are quickly forgotten and the pupil any more can not reproduce all studied ideas, though he remembers LME-s of the first category. The model allows to prove dependence curve of the logically linked information forgetting on duration of lesson T and average quantity lcp of LME-s in each information block. The problem of dependence of the knowledge assimilation on the speed of information transfer also is investigated. It is revealed, that with growth of speed of the information receipt the assimilation speed at first grows, reaches maximum and then is reduced. It has allowed to construct three-component model of training and to receive the dependence graphs of the various categories knowledge quantity on time. The considered above simulations of training confirms the qualitative reasonings, do them more objective, reasonable and can be used when carrying out a pedagogical experiment demands big expenses or leads to negative results.
Some interest is the problem of creating of a training program that simulates the educational process at school. Such program can be used for training of students of pedagogical institutions. It has to allow change of the pupil's parameters, the duration of lessons, the distributions of a training material and strategy of the teacher's behavior. In the course of work with this program the student playing a role of the teacher which changes the speed of the educational information transferring, quickly reacts to questions of pupils, carries out a examinations, gives a marks, trying to achieve the largest level of knowledge for the set time. After the end of "training" the graphs showing change of "quantity of knowledge of all pupils" and marks for "the performed examinations" are displayed. Besides, the training program can analyse work of "teacher" and give him a mark. Example of such program is the simSchool system (Gibson & Jakl, 2013).
REFERENCES
Hunt, E. (2007) The Mathematics of Behavior. New York: Cambridge University Press, 2007. 346p.
Leont'ev L.P., Gohman, O.G. (1984) Problemy upravlenija uchebnym processom: Matematicheskie modeli [Problems of training management: Mathematical model], Riga, 1984, 239 p. (In Russian)
Mayer, R.V. (2014) Kiberneticheskaja pedagogika: Imitacionnoe modelirovanie processa obuchenija: Monografija [Cybernetic pedagogics: Imitating modeling of training process: Monograph]. Glazov, Glazov. gos. ped. in-t, 2014. 141 p. Retrieved from: http://maier-rv.glazov.net (In Russian)
Mayer, R.V. (2015) Computer-Assisted Simulation Methods of Learning Process // European Journal of Contemporary Education, 2015, Vol.13, Is. 3, pp.198-212. D0I:10.13187/ejced.2015.13.198
Mayer, R.V. (2016) Computer Model of the Physical Facts Learning // International Journal of Current Science Research. Volume 2. Issue 1. 2016, pp. 198 - 203.
Razumovskij, V.G., Mayer, V.V. (2004) Fizika v shkole: Nauchnyj metod poznanija i obuchenie [The physics in school: The scientific method of learning and teaching], M.: Gumanitar. izd. centr VLADOS, 2004, 463 p. (In Russian)
Roberts, F.S. (1976) Discrete Mathematical Models, with Applications to Social, Biological and Environmental Problems. Prentice-Hall. 1976.
Shannon, R. E. (1975) Systems simulation: the art and science / R. Shannon. - Prentice-Hall, 1975. 387 p.
Titov, B., A., Rjabinova, E., N. (2011) Matematicheskaja model' usvoenija uchebnoj informacii v obrazovatel'nom processe [Mathematical model of the training information assimilation in the educational process] // Vestnik Samarskogo gosudarstvennogo ajerokosmicheskogo universiteta N 3 (27), 2011, S. 334-340. (In Russian)
Gibson, D., Jakl, P. (2013) Data challenges of leveraging a simulation to assess learning. West Lake Village, CA. Retrieved from http:// www.curveshift.com/images/ Gibson_Jakl_data_challenges.pdf
