CC BY
11I Journal of Siberian Federal University. Humanities & Social Sciences 2022 15(9): 1343-1367
DOI: 10.17516/1997-1370-0933 EDN: BMOBNE УДК 371. 37.01
The Strategies of Best Practice Applying in Education: Criteria Approach
Altynay K. Zhumana, Olga G. Smolyaninovab, Zhanbol O. Zhilbaevc, Lyailya S. Syrymbetova*c and Gulden N. Akbayevaa
aKaraganda University named after academician E. A. Buketov
Karaganda, Kazakhstan
bSiberian Federal University
Krasnoyarsk, Russian Federation
cPavlodar Pedagogical University
Pavlodar, Kazakhstan
Received 29.04.2022, received in revised form 08.06.2022, accepted 11.07.2022
Abstract. This article considers the criteria for evaluating the best international experience practices in education and substantiates these criteria as one of the strategies for applying the most successful practices to enhance and update innovation processes in a particular national educational system. This context allows the authors of the article to analyze the indicators of a number of international comparative studies, specifically: the GCI (Global Competitiveness Index) of the World Economic Forum (WEF), the study "Global Competitiveness of Russian Education", PISA, TIMSS, PIRSL and TALIS, as well as PIAAC. The indicators of the latter have been used for the correlation analysis between the learning outcomes of schoolchildren and adults. In order to substantiate the relevance of the proposed criteria, the authors have used Pearson's linear correlation and Spearman's rank correlation methods to establish the closeness degree of the relationship between the results of the GIC and PISA, TIMSS, PIRSL, as well as between the results of PISA, TIMSS, PIRSL and TALIS. The results have been analyzed in the logic of correlation, and in order to identify the sustainable success of the indicators, the same results have been studied in their dynamics. Thereby the published results of three cycles of all the above studies have been subjected to analysis.
According to the calculations, the authors have conducted an independent examination of the proposals; the final results demonstrated a positive assessment of the authorities in the field of education in Kazakhstan and Russia.
This article describes the results of a local study in the field of successful educational practices. In order to remove the identified contradictions on certain aspects, this study seems to be promising for further scientific discussion.
© Siberian Federal University. All rights reserved
* Corresponding author E-mail address: miss.syrymbetova@mail.ru
Keywords: criterion, best practice, invariance and efficiency of practice, international competitive studies in education.
Research area: pedagogy.
Citation: Zhuman, A.K., Smolyaninova, O.G., Zhilbaev, Z.O., Syrymbetova, L.S., Akbayeva, G.N. (2022). The strategies of best practice applying in education: criteria approach. J. Sib. Fed. Univ. Humanit. soc. sci., 15(9), 1343-1367. DOI: 10.17516/19971370-0933
Стратегии применения лучшего опыта в образовании: критериальный подход
А. К. Жумана, О. Г. Смолянинова6,
Ж. О. Жилбаевв, Л. С. Сырымбетовав*, Г. Н. Акбаеваа
аКарагандинский университет им. Е.А. Букетова Караганда, Казахстан бСибирский федеральный университет Российская Федерация, Красноярск вПавлодарский педагогический университет Павлодар, Казахстан
Аннотация. Изложены критерии оценки лучшего международного опыта в образовании и представлено обоснование стратегии применения критериального подхода для анализа успешности практик активизации и актуализации инновационных процессов в конкретных национальных образовательных системах. Именно в этом контексте авторы статьи проанализировали индикаторы ряда международных сравнительных исследований, а именно: ГИК (индекс глобальной конкурентоспособности) Всемирного экономического форума (ВЭФ), исследования «Глобальная конкурентоспособность Российского образования», PISA, TIMSS, PIRSL и TALIS, а также PIAAC. Индикаторы последнего из них были использованы для корреляционного анализа между результатами обучения школьников и взрослых. Кроме того, в целях обоснования релевантности предложенных критериев авторами применены методы линейной корреляции Пирсона и ранговой корреляции Спирмена для установления степени тесноты связи между результатами ГИК и PISA, TIMSS, PIRSL, а также между результатами PISA, TIMSS, PIRSL и TALIS. Итоги вышеуказанных исследований проанализированы в логике корреляции и для выявления устойчивой успешности показателей эти же результаты были изучены в их динамике. При этом анализу подвергнуты опубликованные материалы трех циклов всех вышеуказанных исследований. Также авторы посчитали целесообразным провести независимую экспертизу предложенных критериев, которая дала положительную оценку авторитетных экспертов в области образования двух стран - Казахстана и России. В целом данная статья описывает результаты сравнительного международного исследования в области успешных образовательных практик. В целях снятия выявленных в процессе работы противоречий по отдельным аспектам данное
исследование, по мнению авторов, представляется перспективным для дальнейшей научной дискуссии.
Ключевые слова: критерий, лучший мировой опыт, устойчивость и эффективность опыта, международные сравнительные исследования в образовании.
Научная специальность: 5.8.7 - методология и технология профессионального образования
Introduction and literature review
Talking about best practice in education in the present article, first of all it means international experience. Transparency and openness as the principle of modern Kazakh education determine its involvement in the global educational system. For this purpose, the practice of other national systems in realization of pedagogical ideas and strategies arouse interest. Therefore, it is important that successful practice of one country can be applicable and effective for others as well. As a rule, best or internationally accepted practices are chosen for these purposes. Primarily, the best practice in education can be defined as an experience of one country, the middle and high educational system success which are proved by the results of international ratings.
From this perspective the article attempts to answer the questions: "Which country's practice can be considered as the best one?" and "To what extent is it applicable in the specific situation of Kazakhstan?". There can be numerous strategies for searching answers to the given questions. We choose one of them: development and explanation of the criteria of best practice in education.
For this purpose, we studied the criteria of the global competitiveness report (GCR) and indicators of international comparative studies (PISA, TIMSS and PIRLS), also the criteria presented in the analytical report "Global competitiveness of Russia education". The latter is meaningful due to the historical commonness of education systems of two neighboring countries - Kazakhstan and Russia.
Initial position of the research is analysis of the definitions of "criterion". There are different interpretations of the given term concept. In the definition dictionary by Ushakova D. N. from Greek "criterion" means: 1) "solution
tool", "distinctive feature"; 2) measure; conventional measure, pursuant to measurements of things, allows to give evaluation (Ushakov, 2013). According to the scientifically based definition by Russian scientist Kushvinova I. A. criterion is the quality, property or features of a studied object, allowing judging about its condition, development level and functioning (Ku-vshinova, 2004).
As Zagvyazinsky V. I. points that criterion is generalized constant of evolving system upon which range of features can be highlighted, and according to which indicators are identified, and indicators are expressed by quantitative or qualitative characters formed of quality, internals, studied object features, in other words the measure of criterion formation (Zagviazinskii et al., 2006). Usually, criteria act in relation to indicators as general to particular, including the whole group of indicators, which characterize the object from qualitative and quantitative side.
The most widespread and conventional notion is understanding criteria as an element on the basis of which evaluation and classification are done (Skvortsov, L. I., 2009), advantage of any choice compared with others is highlighted, the correspondence of the target goal result is checked or evaluation of its degree realization is given (Makarova, O.Y., 2013).
They are used for evaluation, and each criterion gives a different view to the subject property, also they are used outside of the evaluation for management monitoring, strategic planning and development of interventions (OECD 2019).
According to the standards of OECD development assessment quality, for the effective evaluation criteria should be:
- contextualized (understandable, as for estimator, so for appraised object);
- they have to cover issues of stakeholder and be appropriate to the context of assessed object (Multi-criteria analysis, 2009).
Indexes, which show changes according to the particular time for each criterion, are also used for evaluation along with criteria (MCP-FE, 1998). Indicators are qualitative quantitative or descriptive parameters which can be measured according to the criteria. Indexes determine information which are necessary for evaluation and monitoring (ITTO, 2016). Thus, each international organization points out the key criteria and indicators to evaluate the competitive edge of the country. For instance, the structure of the world economic forum (WEF) is: there are three sub-indexes, which set the general direction of the research. Each sub index is divided into criteria among which weighing coefficient is distributed. The Global competitiveness report (GCR) was developed by WEF in 1979 for obtaining the growth pattern of the global economy. The competitiveness was considered as a basis of the report.
There are many different characteristics and counting methods developed. The GCR mathematically captures the development dynamics of countries in comparison with each other. It includes not only economical and financial findings, but also the analysis of political and social decisions, innovative progress, changes in health service and education.
Today the GCR is used in 140 countries. For these purposes 12 main pillars are used. In 2017-2018 the evaluation was done by next pillars (Klaus Schwab 2017, 2018):
1. Institution
2. Infrastructure
3. Macroeconomic environment
4. Health and primary education
5. Higher education and training
6. Goods market efficiency
7. Labor market efficiency
8. Financial market and development
9. Technological readiness
10. Market size
11. Business sophistication
12. Innovation
In the reports of 2018 and 2019 numbers of pillars were kept however there were changes in names of indicators as:
1. Institution
2. Infrastructure
3. ICT adoption
4. Macroeconomic stability
5. Health
6. Skills
7. Product market
8. Labor market
9. Financial system
10. Market size
11. Business dynamism
12. Innovation capability
It seems to be fundamentally important to separate the "Health" pillar into a separate nomination and reorient the indicators "4. Health and Primary Education" and "5. Higher education and professional training" to "6. Skills". It could be supposed that it is perfectly reasonable and efficient as today educational concepts are built on the formation ideas, first of all on soft skill but without exception hard skills.
WEF experts consider that each above mentioned pillars exercise a significant influence on the country's competitiveness. From the main 12 pillars in the given article, there will be considered pillars connected with education, namely: "... primary education" and "High education" (according to the report of 2017) and "Skills" (according to the report of 2018-2019). From more over 100 indicators of GCR, education and skill are evaluated by 8-5 indicators.
Indeed in 2017:
1. Quality of primary education
2. Primary education enrollment rate
3. Secondary education enrollment rate
4. Tertiary education enrollment rate
5. Quality of the education system
6. Quality of math and science education
7. Quality of management schools
8. Internet access in schools
In 2018-2019 years:
1. Mean years of schooling
2. Skillset of graduates
3. School life expectancy
4. Critical thinking in teaching
5. Pupil-to-teacher ratio in primary education
There are many disagreements about quality, education concepts and about its measure-
ments. It can be stated that the main components of education quality identification are the quality of math and science education, quality of management schools and Internet access in schools.
The level of high education has an essential meaning for countries, which want to improve their economy. In particular, economic globalization demands highly educated workers, who are able to adapt to the environmental changes and developed demands of the production system (World Economic Forum, 2006). The orientation to the future of international education is the main ticket to success of the country in the field of human development.
The degree of employee training is also taken into account as a professional and extended work-study education (most countries marginalize), that allows providing with extended staff professional development. Infrastructures for continuing education of engineering, pedagogical staff and heads of educational organizations are being created.
Therefore, development of primary and higher education criteria evaluation was based on key moments, which are necessary for the country's development (WEF, 2006).
Studying international GCR, as it was mentioned above, the analytical report "Global competitiveness of Russian education" was also analyzed and there are presented criteria for identifying real global competitiveness of education on each level while WEF GCR considers only primary and high education (Abankina, 2017).
In the given article it is said that school is a core part of the educational system. In the Russian report the main criteria of school education are:
1) universal access to the education;
2) financing of education;
3) planned educational results;
4) international comparative study of education quality;
5) equality of educational opportunities; work with talents;
6) qualifications of teachers and their work condition.
As in GCR the quality and financing are the dominant criteria for identifying competi-
tiveness of the country in the international level (Abankina, 2017).
The reason for choosing these criteria on the principle that universal access to free education is very important, as today the duration of compulsory education is increasing and this age is varied in different countries. On average the period of compulsory education is 12-13 years. Not knowing the exact number of students, who did not complete their basic education, leads to the problem from the point of enrolment in qualitative formal education. Consequently, the enrolment in general education, length of schooling are more evident indicators for evaluation of its availability. Competitive standards aim at education results. As it was mentioned in the PISA program, today the main goal of school education is not only to give knowledge, bult also to prepare students for life.
When referring to a decent academic background the qualification of students, work conditions and motivation of professional development cannot be ignored. Qualification of teachers and their work condition are main indicators of school system competitiveness. Importance of teachers' development was mentioned while analyzing GCR. The importance of teachers' development lies in training of specialists.
Apart from two reports GCR and "Global competitiveness of Russian education" report for our research we chose criteria of PISA, TIMSS, PIRLS.
Today the world undertakes to create a global educational strategy, therefore the realization of international competitive studies, as PISA, TIMSS, PIRLS is substantive (Zhilbaev, 2016).
The aim of PISA is to evaluate whether students have knowledge and skills necessary for unimpaired operation in society (functional literacy) (PISA-2015). The given estimation concentrates on such subjects as science, reading and mathematics and shows how 15 years old students can realize and apply lessons learned as within the walls of schools so out of them. This approach represents the fact that the modern economy rewards people not for what they know, but for what they can do with their knowledge.
Additionally, the influence of different factors on educational achievements of students are being studied. These are socio-economic aspects, characterization of educational organizations and educational opportunities of teenagers beyond school. The PISA project gives the opportunity to compare different educational systems and use the experience of OECD advanced countries in developing national strategies of education. The study is held for a three-year cycle. Priority guidelines are identified in each period. As detailed above, this program checks the knowledge of students according to the three directions: reading literacy, mathematic literacy and science literacy. These directions are aligned with school subjects (MOE RF, 2015). Used "literacy" notion means the ability to practice received knowledge and skills in the main fields of subjects, also to analyze, explain efficiently, engage substantively in setting, solving and explaining objectives in different circumstances and reflect assessable knowledge and skills (Analytical overview, 2011).
PISA studies the influence of different factors on educational achievements of students. The indicators reflecting main aspects of relation between socioeconomic status and achievements, allows the countries to control changes in these relations as time goes and to compare themselves with other systems (Francesco, 2020).
Along with the abilities of students in PISA the attention is paid to school climate, which can have an impact on academic performance of students: school truancy, disciplinal climate, teacher and behavior of students, which hinder learning and teacher support by students (Ricardo L., 2020).
Tasks analysis according to each direction show that all of them connect with life situations. So, for example, reading literacy aims at determining the ability of a person to understand, use, assess texts, think about them and do reading in order to achieve their goals, extend knowledge and opportunities to take part in social life.
The ability to comprehend and interpret continuous text fragments are key skills for full participation in the labor market, also in
social and public life in 21-st century. For a successful citizen it is necessary to use information from different fields for efficient search, sorting and filtering of a huge amount of information.
Reading literacy tasks are composed with the aim to check such skills as: 1) finding access to the information, rapid text reading and highlighting the part which contains sought information; 2) general text understanding and translation text information into the language of readers, identifying main and minor information; 3) think on content and form of text and its evaluation. The last helps comprehend, connect text context with life experience and give your own assessment (OECD, 2018).
The aim of mathematic literacy is to evaluate the readiness of students to use mathematics in their everyday life. The given direction includes tasks close to the real problematic situations. Four context categories are used: personal life, education /profession, social life and scientific work (OECD, 2018).
The situations of public life and scientific work categories taken from the life of local communities are about the problem appearing in the near environment of students.
The third direction of scientific literacy demonstrates the ability of a person to take a position of active citizenship according to the issues connected with development of experimental sciences and use their achievements, his readiness to be interested in experimental sciences (OECD, 2018).
The tasks are directed to check next competences: 1) to explain phenomena scientifically; 2) understand features of scientific research; 3) scientifically explain data and use evidence to make conclusions. Problem-based situations are about health, natural resources, environment, hazards and risks, relationship of science and technologies. These skills can be considered as a set of actions which each literate person can do (Gold Dayona, 2019).
The aim of international assessment TIMSS is to evaluate mathematic and science literacy level of fourth and eighth grade students. It is held every four years by the international association for the evaluation of educational achievement (IEA) in about 60 countries.
This organization is an independent international association of national research institutes and government agencies doing international research achievements. Their assessment provides not only information about general achievements of students, also about their experience and attitudes toward mathematics and science, but also about education and training of their teachers, characteristics and types of class activities, moreover school conditions for teaching such subjects as mathematics and science. Early academic skills connected with literacy and mathematics and also early nonac-ademic skills of children, such as social competency and self-management are more important indicators of future academic achievements (H. Harju-Luukkainen, 2020).
TIMSS provides an opportunity to politics in the education field to realize the phenomenon of education in their countries. Data taken from study results are a valuable investigatory resource (N Md Norl, 2020).
Above-mentioned assessments allow to define tendencies in education of participating countries every four year, when fourth grade learners study at eighth grade (Zhilbaev, 2016).
According to TIMSS, in mathematics part the knowledge of facts and methods, using notion, solving regular problems, analysis, hypothesizing, evaluation, evidence and others are assessed. Scientific part assesses factual knowledge, conceptual understanding, analytical skill, generalized competence, planning, learning and other.
Formation of reading skills of fourth grade students are being done with the help of international assessment PIRLS.
If TIMSS, PISA and PIRLS determine the level of student knowledge, TALIS (Teaching and learning international survey) studies the working condition of teachers and educational environment at schools, which is important for achieving a sound academic background.
TALIS is being held from 2008, is the first wide international survey of teachers and headmasters according to the different aspects influencing children's education. (TALIS 2018). It is a major international survey, which studies the work condition of teachers and school educational environment. The number of states
taking part in the third cycle reached 48. The goal of TALIS is presenting proven, timely and comparable data for identifying focus in developing educational policy, aimed at arrangement of conditions for effective teaching and learning.
Hence, the analysis of international results of assessments showed that a detailed approach is needed for criteria development. They should reflect the essence of the object of assessment, as well as those characteristics of the assessed object that are most relevant and important for the specific assessment context, the criteria should be adequate and correspond to the goals and objectives. Moreover, each criterion should be measurable in qualitative and quantitative indexes (Syrymbetova, 2019; Novikov, 2007).
Methods
In order to conduct the research data of international assessments as PISA, TIMSS, PIRLS, TALISS, PIAAC and GCR for the last three cycles were analyzed. Moreover, to find out the country with best practice in education correlation analysis as Person and Spearman were used. To be sure in the relevancy of our developed criteria, there was conducted a survey in Google Form among experts to assess them.
Results
Therefore, the review of literature and the analysis of theoretical and empirical materials allow us to make next conclusions:
1) considering the common point of intersection among numerous definitions of "criterion", in the given article the understanding of criterion as the meaning of definite response to a question or solving task, which is used for identifying requirements of what efficiency will be considered "high", which is "low" and which is "neutral" is used (Konstantin 2019).
2) found upon the system of criteria and indicators of education in well-known international assessment reports, criteria which are important for identifying best international practice with further implementation of its elements in the national system of Kazakhstan education are highlighted.
Given this, next criteria are offered:
1. International recognition of experience
1.1 GCR index
1.2 PISA index
1.3 TIMSS index
1.4 PIRLS index
1.5 TALIS index
2. Invariance of practice
3. Efficiency of practice
4. Adaptability of practice
The given criteria are explained as follows:
1) international recognition of experience is a positive and objective assessment, given by skilled experts based on valid diagnostic tools, and approved by the general public. Such evaluations are GCR index and international assessment studies as PISA, TIMSS, PIRLS, TALIS in the context of national education systems. Moreover, PIAAC index, which presents the literacy level of the population, i.e., it is the percentage of literate, educated people from the general population. This survey is conducted between people from 16 to 65 years old (UNESCO).
2) invariance of practice is the ability of a dynamic system to keep moving by its intended trajectory, in spite of perturbations affecting it (Academic.ru). It is totally obvious that this definition is rational, since the syner-gistic essence of educational systems creates its dynamism. So, in the context of the article invariance in practice as positive dynamics of above-mentioned last three cycle international studies results, i.e., if the dynamic of international studies results shows the progress in indexes for the analyzed period, such practice can be recognized stable;
3) efficiency of practice is positive exponent of correlation TIMSS, PISA, PIRLS results with GCR and PIAAC results, the latter show retentivity and invariance of educational outcome, gained by adults in their school years;
4) adaptability of practice is a correlated level of created country education value (more exactly values-based attitude to form its main subject) intended to adopt the experience of others.
Discussion
The results of conducted analysis are presented in the given part of the work in order to explain the relevance of chosen criteria of international experience assessment.
Thus, the study and analysis of GCR results for the last three years (2017, 2018, 2019) show that according to the general indexes, such countries as Switzerland, the USA, Singapore, Netherlands, Germany and Hong Kong are in the top five (Table 1) (Klaus 2017, 2018, 2019).
At the same time, according to the education index Finland was leader twice and Switzerland once (in 2019). Presented achievements of Finland are interesting, as this country is not in the top five according to the broad pillars of GCR. The broad pillars and education indexes correlate more in such countries as Singapore, Switzerland, Netherlands, the USA (Table 2).
As it is seen from data in Table 2, the world's leading countries in terms of the GCR index are not stable according to their leading position in education. At the same time, the mean for the three years can demonstrate more successful country according to the general pillars, which is the USA (Fig. 1).
Table 1. GCR indexes (broad pillars)
Country 2017 year 2018 year 2019 year
Switzerland 1-5,86 4-82,6 5-82,3
USA 2-5,85 1-85,6 2-83,7
Singapore 3-5,71 2-83,5 1-84,8
Netherlands 4-5,66 5-82,4 4-82,4
Germany 5-5,65 3-82,8
Hong Kong 3-83,1
But, if to compare GCR pillars in the field of education, the USA is in the third place, Singapore is in the fifth, Switzerland and Netherlands do not lose their position, Finland, which has not been in top five according to the GCR general pillars: it was in 11th place in 2018 and in 2019 (Fig. 2).
Next questions are arising: "Is the success of the national educational system valid warrant of country competitiveness, on the whole?" or "How are GCR tools of measurements valid?" It seems that the questions are serious enough, that they demand special research. But along
with this, it is considered that the changes in indicators GCR according to the education in 2018 with the focus on skills of alumni and critical thinking in education in some degree gives the answers to the above-mentioned questions. Moreover, the general paradigm of education in the modern world creates a tendency on developing "soft" skills. However, searching answers on these questions is not included in the tasks of the given research. Our research focus is somewhat other. How to identify best practice in education and how it is adoptable in the educational system of other countries.
Table 2. GCR indexes (indicator of education)
Country 2017 year 2018 year 2019 year
Finland 1-6,54 1-87,9 2-5,6
Singapore 2-6,52 20-76,0 4-5,4
Switzerland 3-6,42 2-87,3 1-5,9
Netherlands 4-6,39 4-85,4 3-5,5
USA 5-6,22 3-86,3 5-5,3
USA Singapore Switzerland Netherlands Germany
Fig. 1. Mean value of general GCR pillars for the years: 2017, 2018, 2019
Fig. 2. Mean value of GCR indexes according to the education for the years: 2017, 2018, 2019
If GCR measures its indicators according to the criteria which are reflect, mainly, overall trend (mean years of schooling; school life expectancy; pupil-to-teacher ratio in primary education) and only two questions relate to the education outcome (Skillset of graduates; Critical thinking in teaching), TIMSS, PISA and PIRLS demonstrate detailed indicators to measure the level of students' knowledge (Kul-tumanova 2013).
Relying on statistics, the number of participating countries is increasing. So, for example, 32 countries (28 OECD countries) participated in 2000, in 2003-41 countries (30 OECD countries), in 2006-57 countries (27 OECD countries) and in 2009-75 countries (34 OECD countries), in 2012-65 countries (34 OECD countries), in 2015 the number of countries was 70 and in 2018 the number increased to 79. Increasing the number of participated countries in the given study shows its popularity and importance.
The results of study for the 2012, 2015 and 2018 (Table 3) are presented (A. Schleicher, 2019, A. Kultumanova, A., 2013, PISA-2015).
From numbers presented in the Table 3 according to the indexes of PISA reading literacy
Singapore is in the top five (Fig. 3), China is leader in terms of mathematic literacy (Fig. 4), and according to the scientific literacy indexes China is also leader (Fig. 5).
It should be particularly noted that in the top five leading countries, in terms of formation reading, mathematics and science literacy of 15-year-old students according to the PISA monitoring, was only Singapore which was among the top five of GCR.
Regarding TIMSS study. The number of countries, taken part for the last three studies are varied as follows:
- 36 countries participated (for fourth grade students) and 49 countries (for eighth grade students) in 2007;
- the number of countries increased and comprised 50 (for fourth grade students) and 42 countries (for eighth grade students) in 2011;
- in 2015 it decreased to 49 (for fourth grade students) and 39 countries (for eighth grade students) (International study center, 2007).
Based upon the number of participating countries in international studies, in comparison with PISA, as it happens, TIMSS is less popular. However, the results of TIMSS studies
Table 3. PISA results for the years: 2012, 2015, 2018
Country 2012 2015 2018
R.l. M.l S.l R.l. M.l S.l R.l. M.l S.l
Singapore 542 573 551 535 564 556 549 569 551
Japan 538 536 547 516 532 538 504 527 529
Hong Kong 545 561 555 527 548 523 524 551 517
China 570 613 580 494 531 518 555 591 590
The Republic of Korea 536 554 538 517 524 516
510 500
Singapore China HongKong The Republic Japan
ofKorea
Fig. 3. Mean value of PISA for the years: 2012, 2015, 2018 (reading literacy)
500
China Singapore Hong KongThe Republic of Korea Japan
Fig. 4. Mean value of PISA for the years: 2012, 2015, 2018 (mathematic literacy)
520 510 500
China Singapore Japan HongKong The Republic
of Korea
Fig. 5. Mean value of PISA for the years: 2012, 2015, 2018 (science literacy)
are not less significant in evaluating the national education system, as they demonstrate two types of literacy (mathematics and science) of schoolchildren according to the two-age grade (4-th graders and 8-th graders). The results of three studies of TIMSS are presented below (Table 4) (IEA, 2008, IEA, 2015, IEA, 2012).
As in PISA study from the top five of GCR Singapore is in the top five countries according to the TIMSS study. At the same time this country is absolutely the leader in mathematics and science literacy of students (Fig. 6, 7 and 8).
These TIMSS results correlate with PISA results.
PIRLS identifies the reading quality of fourth grade students. The given study is also conducted by International Association for the Evaluation of Educational Achievement once in five years since 2001. As in TIMSS the number
of countries taken part in PIRLS increased significantly, i.e., 45 countries took part in 2006, 45 countries in 2011 and 50 countries in 2016. Analysis of three PIRLS studies demonstrates the following: Singapore and Hong-Kong were stable in top five as in PISA and TIMSS study, and also new countries as Russia, Finland and Canada are appearing. At the same time Russia is a leader according to the reading literacy of primary school students, but Hong-Kong, Singapore and Finland are in the same position (Fig. 9) (IEA, 2007, IEA, 2012, IEA, 2016).
Therefore, according to our suggested criterion "International recognition of experience" we analyzed the GCR results according to the general pillars and education indexes separately, and also three international studies (PISA, TIMSS, PIRLS), which assess exactly the educational level of students. Degree of correlation among GCR and
Table 4. TIMSS results for 2007, 2011 and 2015 years.
Country 2007 2011 2015
maths science maths science maths science maths science maths science maths science
4-th grade 4-th grade 8-th grade 8-th grade 4-th grade 4-th grade 8-th grade 8-th grade 4-th grade 4-th grade 8-th grade 8-th grade
Singapore 599 587 593 567 606 583 611 590 618 590 621 597
Japan 568 548 570 554 585 559 570 558 593 569 586 571
Hong - Kong 607 554 572 530 602 535 586 535 615 557 594 546
Republic of Korea 597 553 605 587 613 560 615 589 606 556
China 576 557 598 561 591 552 609 564 597 555 599 569
607,7608,3 582 5753 608 534 605,3 588 602
Singapore Japan Hong-Kong Republic of China-Taipei
Korea
■ 4-th grade [J 8-th grade
Fig. 6. Mean value of mathematic literacy according to TIMSS for the years: 2007, 2011, 2015 (in the context 4-th and 8-th grade)
700
Singapore Japan Hong-Kong Republic of Ko rea China-Taipei
Fig. 7. Mean value of science literacy according to TIMSS for the years: 2007, 2011, 2015 (in the context 4-th and 8-th grade)
PISA, TIMSS, PIRLS results can be named complex and in particular cases contradictory on this stage of the study. Also, there is one more study concerning education issues. It is referred to international study according
to the education and teaching issues TALIS, which is held once in five years via survey of teachers and principals. The results of TALIS according to the next aspects are analyzed: education, work condition and professional
Fig. 8. Mean value of TIMSS for the years: 2007, 2011, 2015 (mathematics and science literacy)
530
Russia Hong - Kong Singapore Finland Canada
Fig. 9. Mean value of PIRLS indexes for the years: 2006, 2011, 2016
development for the last two cycles, i.e.: the second cycle of study was held in 2013 with the participation of 37 countries and the third cycle with participation of 46 countries was in 2018 (OECD, 2014, TALIS 2018).
It should be noted that it is paid a special attention to the position of Singapore as the country, which is in the top five according to all analyzed international studies.
The first indicator, which connects with the proportion of teachers having high education implies, that the number of master degree teachers is more than teachers with bachelor degrees in the TOP - 5 countries. However, there is a wide range in the top five countries, only two countries Croatia and Slovakia twice appear in TOP - 5, and we are interested in the country Singapore is on the second and third ten (Table 5).
Initial conclusion in the stage of study can be done as follows: the tensions between indexes of studied education results of students and the teacher's qualification level should be accepted as so the best results of Singapore and
Finland students are not correlated with qualification indicators of their teachers.
At the first glance recognizing the qualification of teachers as a main determinant of student's success, we also admit that bachelor and master degrees should be always acknowledged in conditions of high development dynamics as a community, so the education system as its social substructure. Hence the indicators of professional developments of teachers were analyzed. TALIS provides indicators such as: professional development of teachers (seminars, master-classes also upgrade training courses), which are directed to gain new teaching methods. In this respect Singapore is in TOP - 5 countries according to the both studied cycles of TALIS (however, in 2018 it falls from the first place to fifth place in 2013), moreover Australia, which was not noticed in above mentioned international studies was twice in the top five. Besides, it should be noted there is a wide geographical range of TOP - 5 countries according to the TALIS indicator (Table 6).
Table 7. Average number of working hours (i.e., 60-minute hours) that teachers spent on tasks related to their job during a typical calendar week
Table 5. Percentage of teachers who have completed formal education (%)
2013 year 2018 year
The percentage ofteachers with bachelor degree The percentage of teachers with master degree The percentage of teachers with bachelor degree The percentage of teachers with master degree
Belgium 85,4 Australia 98,9 Saudi Arabia 92,8 Slovakia 96,2
Chile 17,9 Poland 98,8 Turkey 92,3 Portugal 93,4
Croatia 17,7 USA 98 Brazil 89,8 Croatia 91
Italy 15,8 Norway 97,9 Kazakhstan 89,4 Finland 90,6
Serbia 15,5 Slovakia 97,5 China 86,4 Czech Republic 89,7
Singapore (10-th place) 5,5 Singapore (18-th place) 92,4 Singapore (16-th place) 72 Singapore (32-nd place) 22,4
Table 6. Percentage of teachers who participate at least in one professional development activity (%)
2013 year 2018 year
Singapore 98 Lithuania 99,4
Canada 97,7 Shanghai 99,3
Croatia 96,8 Australia 99,3
Australia 96,6 Austria 98,7
Malaysia 96,6 Singapore 98,5
2013 2018
Chile 29,2 Georgia 25,3
Italy 29,4 Saudi Arabia 28,7
Israel 30,7 Buenos Aires 29
Finland 31,6 Brazil 29,8
Mexico 33,6 Italy 30
Singapore 47 Singapore 45,7
In case with Singapore here there is regularity about the strategic relevance of permanent professional development of teachers. Besides, it is considered that the success of students in Singapore and Finland can be due to the time spent by teachers to perform professional functions. This is one more indicator of TALIS. Hence, according to the given study, Singapore teachers spend more than 40 hours
per week for professional purposes, also teachers of Finland need much time (Table 7).
Meanwhile, it is hard to confirm that there is correlation between time spent by teachers for professional purposes and the success in students' education achievements, whereas teachers can be involved at events which are not relevant for their profession (it concerns post-Soviet countries).
In general, studying the TALIS indicator, we point out a gradual decline in the number of working hours (i.e., 60-minute hours) that teachers spent on tasks related to their job during a typical calendar week. Given this, it means that there is a tendency for a decreasing professional timesheet for teacher, which will reduce such factors as professional burnout.
One more important issue that teachers worry about is the maximum number of students per class. There is an opinion that a smaller number of students in class is better for the achievements of students. However, it is a disputable statement. The TALIS indexes according to the indicator "Class size", where at first Singapore shows successful educational achievement of students in spite of high maximum number per class, the second Finland also demonstrates high results of students, but low in class size (Table 8).
Chosen TALIS indicators are significant for quality education. The professional qualification level of teachers plays an important role in teaching, and may indirectly, but influence on academic progress of students. Therefore, based on the analysis of studies, Singapore and Hong-Kong show high results according to all subjects in spite of that the percentage of teachers having bachelor degree is 5,5 % (2013) and 72 % (2018), while countries where percentage of teachers with bachelor degree is more than 90 % students show low scores in international assessment studies; or the relation of teachers with master's degree was 98,9 % in Australia in 2013, 96,2 % in Slovakia in 2018, but in spite of this according to the results of students in assessment studies these countries are not in top five countries with high results (Douglas 2011).
Conducted analysis confirmed us that for education assessment multiple classification are needed, when the same determination can give different results depending on socioeconomic reality of every country among which, from our point of view, the value system of any cultural community including values-based attitude toward education is dominated.
At this point it is considered that GCR and international assessment studies indexes, giving much information for thinking and discussion demonstrate a real overview of national education system success. The international recognition can be claimed to the status of one of best practice assessment criteria, in our case it is practice in education.
Next criterion is "Invariance of practice" (definitely, all introduced criteria should be understood with extrapolation on the education system). To identify the given criteria, analysis of countries' dynamics according to the GCR, PISA, TIMSS, PIRLS and TALIS indicators were conducted Moreover, the dynamics of the results were followed according to the countries, which were in TOP - 5. Summarizing the results, it can be seen that the results of the countries are varying from year to year.
Having analyzed the data of GCR according to the education for the last three years 2017, 2018, 2019, it can be seen that Netherlands demonstrates upward trend (3-rd place ^ 4-th place ^ 4-th place) and Switzerland (3-rd place ^ 2-nd place ^ 1-st place). In the third GCR study (2019) Finland lost leading positions, which it kept in the first (2017) and second (2018) studies. Singapore also showed high results in GCR education (skills) in the
Table 8. Class size (Number of students enrolled in a teacher's target class)
2013 year 2018 year
Estonia 17,3 Georgia 16,7
Belgium 17,3 Estonia 17,0
Latvia 17,7 Belgium 18,0
Finland 17,8 Finland 18,1
Slovakia 19,1 Kazakhstan 18,1
Singapore 35,5 Singapore 32,8
first (2017) and the third (2019) studies of GCR, however in 2018 it fell till 20-th place and was able to improve their indicators for 16 positions (Fig. 10).
Drawing on GCR education results Finland demonstrates stable indicators, and according to the general results the situation is stable in Singapore.
According to the PISA international assessment study, which is held since 2018, from the 65 taken part countries China was a leader, which fell from first place till tenth in the analyzed period and in 2018 it took first place again. Singapore and Hong-Kong showed stable results: according to the results of the last three PISA studies the indicators of this country changed insignificant (Fig. 11).
According to the general results of TIMSS assessment Singapore, Japan, Hong-Kong, Republic of Korea, China-Taipei were in the top five in different years for the last three years. The data analysis according to the mathematics and science among fourth-eighth grades witnesses that the countries of Eastern Asian showed high results, the leader among which is Singapore is in first place with stable increase by every measure. Republic of Korea and China-Taipei are in second and third places steadily. It should be noted that all TOP-5 countries demonstrate the progress of their indexes according to the all TIMSS indexes (Fig. 12).
If the mathematics and science literacy of fourth grade students are assessed in TIMSS, PIRLS study, as it was said above, allows com-
Fig. 10. Dynamics of GCR results (indicators according to the education)
Fig. 11. The trend data of TOP-5 countries according to the PISA results
Fig. 12. The trend data of TOP-5 countries according to the TIMSS results
Fig. 13. The trend data of TOP-5 countries according to the PIRLS results
paring level and reading quality and text understanding of fourth grade students. According to PIRLS Russia, Hong-Kong, Canada, Singapore, Finland, Ireland, North Ireland were in the TOP-5 countries in different years. Thus, an upward trend is seen in such countries as Russia and Singapore, but Hong-Kong has varying indexes: the increase between first and second and third studies. However, in comparison with the first study (2006) the indexes of two following studies show perceptible gain (Fig. 13).
In terms of conducted analysis according to the international assessment studies TIMSS, PISA, PIRLS and studying global competitive reports for the last three studies, the next conclusion can be done. Singapore, having demonstrated best results with an upward trend has taken the first and the second places according
to the test results, where mathematic, reading and science knowledge were assessed. Singapore was also different with high results according to GCR. Along with Singapore, the USA and Switzerland showed stable results. However, the experience of Singapore in the context of its stability attracts special attention.
Next our suggested criterion is called "Efficiency of practice". For counting it Pearson linear correlation formula was used in order to identify the strength of relationship between PISA (2012,2018), TIMSS (2011, 2015), PIRLS (2011, 2016) and results of TALIS for 2013 and 2018 years, as the level of teacher qualification and teaching condition are one of the main determinants of academic achievements of students. Earlier in our work there was presented the description of these results, namely: per-
centage of teachers who have completed formal education (%); percentage of teachers who participate at least in one professional development activity (%); class size; Average number of working hours (i.e., 60-minute hours) that teachers spent on tasks related to their job during a typical calendar week. The results of the last two studies were taken for the correlation analysis.
As it is known, correlation analysis is correlated change of indexes which is characterized with directions, forms and strength. Correlation coefficient is the degree of measurement of variable ratios (Senthilnathan S., 2019, Mohamed Ahmed Zaid, 2015). Correlation analysis is a statistical technique which deals with correlation between two variables.
In our research for correlation, we used Pearson correlation coefficient formula.
Pearson (1) (Senthilnathan S., 2019).
There is not a correlation between two indexes if summarizing results the given index is near or -1, if the index is equal or near +1, it shows the strength between two indexes.
In output computation the next coefficient value of Pearson linear correlation is produced (Table 9).
The interpretation of findings leads to the next conclusion:
1) graduated education of teachers and development of their professional qualifications have significant influence on students' academic achievements;
2) there is not any correlation between the size and class content, also span time of
teachers for their doing professional tasks with the success of their students.
If the first conclusion is beyond dispute, moreover it represents a scientifically determined pedagogical pattern, the debate about the second conclusion, according to us, has not got a definite argument. The experience of Singapore is acknowledgement of it: in maximum number of schoolchildren in class they demonstrate high educational results. It is true, long experience of teachers (including the authors of the article) even if at an empirical level, but shows the validity of the statement that implication of effective pedagogical technologies orientated on individuality and independence of students are more important than their number of students in class or in group. It is the first. The second one is the motivation of students in education: a warranty of their academic success is one who wants to study and strives for a big autonomy in education.
It was decided to complete one more correlation line between the results of TIMSS, PISA, PIRLS and PIAAC results. The subject of PIAAC study is competency assessment of 16-65 years old adults. At first in 1950 UNESCO identified "literacy" as the ability to read, write and speak about everyday life using visual and written materials. However, till 1990 literacy was acknowledged as a skill contributing to individual success (OECD, 2019).
The reason of our reference to the given study is to identify the efficiency of the national educational system experience hypothetical-ly and with this aim to set correlation between learning outcomes of students and their competence in adulthood.
For this purpose, the indexes of Singapore are chosen, as schoolchildren of this country at
Table 9. The results of Pearson correlation analysis
TALIS TIMSS PISA PIRLS
Percentage of teachers who have completed formal education 1 1 1
Percentage of teachers who participate at least in one professional development activity. 1 1 1
Class size -1 -1 -1
Average number of working hours (i.e., 60-minute hours) that teachers spent on tasks related to their job during a typical calendar week -1 -1 -1
the age of 10-15 demonstrated the best results according to the international studies, which check the level of mathematic, reading and science literacy.
At the beginning PIAAC results were analyzed according to two criteria: reading literacy and mathematic literacy for 2016 and 2019 years. The choice of PIAAC study connects with that particularly in this period students who took part in TIMSS, PISA, PIRLS at the age of 10 and 14, confirmed the level of their competence being adult (Table 10).
According to the results presented in table 10, Japan and Finland are in the list of the countries where adult nations showed high results. However, Singapore, the country which ranks high in international competitive studies of schoolchildren academic achievements, according to the results of adult literacy takes 24th and 26th place with the score of 257,6 in reading literacy and 257,4 in mathematic literacy.
For this research the Spearman correlation was used for identifying the correlation of TIMSS, PISA, PIRLS results with PIAAC results. Rank correlation coefficient is quantitative estimation of statistical study between phenomenon, using in nonparametric methods (OECD 2019). Next formula is used for Spearman correlation (Sedgwick Ph., 2014).
The results of international assessment for the last two studies were taken, namely: TIMSS (2007-2011), PISA (2012-2015), PIRLS (20062011) and PIAAC (2016-2019), i.e., students, who are at the age of 10-15 took part in these studies in 2016 and 2019 achieved the age of participation in PIAAC (Table 11).
As it is seen there is no correlation between the international studied results of schoolchildren and adults' competencies, as so Spearman coefficient value in all cases is zero: 1) p=0 (TIMSS-PIAAC); p=0 (PISA-PIAAC); p=0 (PIRLS-PIAAC). In other words, our hypothesis about correlation between competencies of different age groups (children and adults) is not confirmed, at least in the example of Singapore.
From our point of view, it connects with high dynamism of transformational and inno-vational process in the modern world, when young generation forming their own way of thinking and way of life create new sub generational culture. One example is enough: dig-italization of all human living environments, or the impact of fourth industrial revolution on professional competencies, which are connected with solving difficult problems today, dependent on profession variation and skill in different occupations, communication skills. That is why, the ability of workers to the constant adaptation and acquiring new skills are demand special attention. Working principle is also changing today, it is becoming free and
Table 10. The literacy level of adults according to the PIAAC results
Country Reading literacy Mathematic literacy
2016 year 2019 year 2016 year 2019 year
Japan 296,2 296,2 288,2 288,2
Finland 287,5 287,5 282,2 282,2
Singapore 257,6 (on the 26-thplace) 257,6 (on the 26-thplace) 257,4 (on the 24-thplace) 257,4 (on the 24-thplace)
Table 11. TIMSS-PIAAC correlation
№ TIMSS PIAAC PISA PIAAC PIRLS PIAAC
1 587 516 555 516 558 516
2 598 516 552 516 567 516
p= 0 p= 0 p= 0
mobile, perfect combination of freedom, lack of stress and high satisfaction from the work are forming. It is one of the main lessons of the pandemic. Social isolation and virtual reality begin to form another identity: if in the past people more identified themselves according to the particular place, ethnic group and even language, today identity becomes more widespread. Digital world expands human opportunity, made any place accessible in spite of destination and time.
Returning to the results of correlation, at this stage of research we doubted in relevance of our suggested criterion of assessing best practice in education (we remember, it is talked about "Efficiency of practice"). Therefore, it was decided to continue finding out the level of correlation of education with competitiveness of the national economy and correlated results of GCR with PISA, TIMSS, PIRLS results.
The correlation was completed on the basis of such countries as Singapore, the USA and Hong-Kong, as Singapore and the USA are in TOP-5 according to GCR index and Singapore and Hong-Kong are in top five according to the PISA, TIMSS, PIRLS results. Pearson correlation was used for the given correlation. The result of correlation is presented in Table 12.
From nine receiving coefficients of linear correlation in six cases close correlation is observed between measured variables. Consequently, our suggested criterion "Efficiency of practice" is relevant. However, considering lack of connection between results of different generation people competences (correlation PISA, TIMSS, PIRLS with PIAAC) there was decided to make correction in interpretation
of term "Efficiency of practice" as follows: the criterion "Efficiency of practice" (applicable for education system) means not residual and invariance of academic achievements of adults, taken in school years, as much as factor of success. And if we consider that school education is a large economic unit of the state, hence the condition of its competitiveness so the argument about relevance of our criterion gains additional arguments.
The criterion "Adaptability of practice" seems to be the most significant and difficult. The reason for identifying and studying the best practice in education is its implementation in another social environment. Hence, the statement is about importance not just transfer experience from one situation to another or uncritical adoption, but the necessity to adapt it with possible transformation sounds axiomatic. Under the notion of adaptability of practice, we understand the related level of education values (more exactly values-based attitude to forming its main subjects) of a country, which intends to adopt the experience of others. Having accepted arch-complexity of measurement practice according to this criterion, on the range of this article we limit with above mentioned definitions. We consider that this criterion demands separate and targeted study. Within the framework of this article, the results of independent expert assessment of suggested best international experience in education criteria are given. For getting expert commentary a survey was developed and addressed to researchers with requests to estimate the consistency and relevancy of our suggested criteria rate on a scale of one to three (1 is low, 2 is middle, 3 is high).
Table 12. Pearson product-moment correlation coefficient: PISA, TIMSS, PIRLS and GCR
Singapore Hong-Kong USA
PISA GCR TIMSS GCR PIRLS GCR PISA GCR TIMSS GCR PIRLS GCR PISA GCR TIMSS GCR PIRLS GCR
1 555 81,5 587 81,5 558 81,5 554 79 556 79 564 79 492 83,5 524 83,5 540 83,5
2 552 83,5 598 83,5 567 83,5 533 82,3 565 82,3 571 82,3 487 85,6 525 85,6 556 85,6
3 556 84,8 607 84,8 576 84,8 531 83,1 578 83,1 569 83,1 495 83,7 533 83,7 549 83,7
r= 0,120 r= 0,999 r= 0,992 r= -0,994 r= 0,903 r= 0,893 r= -0,893 r= -0,329 r= 0,873
Fig. 14. Expert estimation of criterion 1.1"GCR index" (including evaluation of education for the last three studies)
The given survey was conducted via Google Forms, 17 experts from Kazakhstan and Russia took part. The survey results are presented below.
Relevancy validation of criterion 1. "International recognition of experience (TIMSS, PISA, PIRLS)" is as follows:
- criterion 1.1 "GCR index" vast majority of respondents (80 % + 10 %) express confidence to the results of the given study (Fig. 14);
- criterion 1.2 "TIMSS index" overwhelming majority of respondents (70 % + 20 %) express confidence to the results of the given study (Fig. 15);
- criterion 1.3 "PISA index" absolute majority (80 %+20 %) of experts gave positive assessment (Fig. 16);
- criterion 1.4 "PIRLS index" 10 % of respondents told their uncertainty, but overwhelming majority (60 %+30 %) agreed with its relevance (Fig. 17);
- criterion 1.5 "TALIS index" absolute majority (60 %+40 %) of respondents also consider the given criterion is acceptable, but
with the comment that only students' academic achievements are needed for identifying countries with best practice in education (Fig. 18).
Overall criterion 1. International recognition of experience in its relevance inspires confidence of respondents who expressed their authoritative opinion.
There is not disagreement among opinions of respondents according to the relevancy of criterion 2. Efficiency of practice and 3. Adaptability of practice: here vast majority (80 %+20 %) endorse our idea (Fig. 19 and 20).
General results of evaluation all suggested criteria of assessment best international experience in education looks as follows: in average 96 % of experts endorse our idea expressing positive opinion.
Conclusion
In conclusion, within the framework of the study, it was found that all the criteria proposed for assessing the best international experience practices in education are fully measurable and may serve as a starting point for
Fig. 15. Expert estimation of criterion 1.2 "TIMSS index"
Fig. 16. Expert estimation of criterion 1.3 "PISA index"
Fig. 17. Expert estimation of criterion 1.4 "PIRLS index"
Fig. 18. Expert estimation of criterion 1.5 "TALIS index"
100 80 60 40 20 0
80
20
1
100 80 60 40 20 0
80
20
I
2 ■ 3
12 «3
Fig. 19. Expert estimation of criterion 2 "Efficiency of practice"
Fig. 20. Expert estimation of criterion 3 "Adaptability of practice"
choosing the experience of one country and its application into another national education system. The most complicated criterion in the evaluating such experience is the adaptability to other conditions. Therefore, the most important conclusion is that one's experience cannot be entirely and completely applied by others. Thus, the elements of a particular national educational system relevant in terms of its significance for solving strategic tasks in the field of educational policy of a particular state are of the greatest interest for further research.
Nowadays the global challenges determine the general trends in the development of the world education practice considerably, in particular, such as inclusion, multilingual education, soft skills development, education digitalization, and etc. At the same time, each country and a single cultural community ac-
tualizes its requests, needs, values, which are often aimed at maintaining the continuity of their own, original traditions when introducing innovative processes in education.
In addition, the criteria proposed in the article might help to identify certain successful aspects of educational practice not only in the world leading countries.
In this regard, the development of criteria for the best international experience practices in education is one of the strategies for its applying. The strategy proposed should not be understood as the only one and the most comprehensive one. However, it is quite applicable to study the best experience in the publication of textbooks, in the teaching technology development, in the creation of optimal conditions for improving the qualifications of teachers, etc. In this aspect, this study seems to be promising.
References
Analiz mezhdunarodnoi programmy po otsenke obrazovatelnykh dostizhenii uchashchikhsia PISA [Programme for International Student Assessment] (2011). Analiticheskii obzor № 17. Available at: http://charko.narod.ru/tekst/analitika/AO_12011.pdf.
Academic dictionaries and encyclopedias. Available at: https://economic_mathematics.academic.ru/ ysTOHHHBOCTB_CHCTeMBi
Andreas, S. (2019). PISA 2018 Insights and Interpretations. OECD 2019. Available at: https://www.oecd.org/pisa/PISA%202018%20Insights%20and%20Interpretations%20FINAL%20 PDF.pdf
Douglas, N., Harris, Tim & Sass, R. (2011). Teacher training, teacher quality and student achievement. In Journal of Public Economics. 95(7-8). DOI 10.1016/j.jpubeco.2010.11.009
Francesco, A. (2020). The measure of socio-economic status in PISA: a review and some suggested improvements. Avvisati Large-scale Assess Educ. 8:8. DOI 10.1186/s40536-020-00086-x
Gold D., Zulkardi (2019). Student's mathematics literacy skills in solving of PISA type problems financial context, International Seminar on Applied Mathematics and Mathematics Education, IOP Conf. Series: Journal of Physics: Conf. Series 1315 012013 IOP. DOI:10.1088/1742-6596/1315/1/012013
Gray, W. (1956). The Teaching of Reading and Writing: An International Survey. Paris and Illinois: UNESCO, SCOTT, FORESMAN and Company. Available at: http://unesdoc.unesco.org/imag-es/0000/000029/002929eo.pdf
Harju-Luukkainen, H., Vettenranta, J., Wang, J. & Garvis, S. (2020). Family related variables effect on later educational outcome: a further geospatial analysis on TIMSS 2015 Finland. Large-scale Assess Educ. 8:3. DOI 10.1186/s40536-020-00081-2
Abankina, I. V., Belikov, O. S. (2017). Globalnaia konkurentosposobnost rossiiskogo obrazovaniia [Global competitiveness of Russian education]. Discussion materials. National Research University Higher School of Economics, Institute of Education. Moskow NRU HSE, 112.
IEA (2007). PIRLS 2006 International report. Available at: https://timssandpirls.bc.edu/PDF/ PIRLS 2006_international_report.pdf
IEA (2008). TIMSS 2007InternationalMathematic report: Findings from IEA's trends in International mathematics and science study at the fourth and eighth grades Available at: https://timssandpirls.bc.edu/ TIMSS 2007/PDF/TIMSS 2007_InternationalMathematicsReport.pdf
IEA (2012). PIRLS 2011 International results in reading. Available at: https://timssandpirls.bc.edu/ pirls2011/downloads/P11_IR_FullBook.pdf
IEA (2015). TIMSS 2015 international reports. Available at: http://timssandpirls.bc.edu/timss2015/ international-results/timss-2015/about-timss-2015/
IEA (2016). PIRLS 2016 International results in reading. Available at: http://timssandpirls.bc.edu/ pirls2016/international-results/pirls/student-achievement/
ITTO (2016). Criteria and indicators for the sustainable management of tropical forests. ITTO Policy Development Series 21. International Tropical Timber Organisation. Yokohama, Japan.
Klaus, Schwab (2017). The Global Competitiveness Report 2017-2018. World Economic Forum. Available at: www.weforum.org/gcr
Klaus, Schwab (2018). The Global Competitiveness Report. World Economic Forum. Available at: www.weforum. org/gcr
Klaus, Schwab (2019). The Global Competitiveness Report. World Economic Forum. 18. Available at: www.weforum. org/gcr.
Konstantin, B (2019). Razrabotka kriteriev i sposobov otsenki effektivnosti mezhdunarodnoi turistskoi deiatelnosti [Development of criteria and methods for assessing the effectiveness of international tourism activities]. In Journal of International Economic Affairs. 9(2): 539. DOI: 10.18334/eo.9.2.40619.
Kultumanova, A. (2013). Main results of an international study of educational achievements of 15-year-old students PISA-2012. Astana, NCESA Publ., 283
Kuvshinova, I.A. (2004). Formirovanie gotovnosti budushchego uchitelia k osushchestvleniiu zdorove sberegaiushchego eksperimenta [Formation offuture teachers' readiness for realization of health promotion experimants]: Candidate's thesis. Magnitogorsk: MaGU.
Makarova, O.Y. (2013). Kriterii i pokazateli otsenki effektivnosti funktsionirovaniia vospitatelnoi sistemy vuza [Criteria and indicators for assessing the effectiveness of the functioning of the educational system of the university]. Fundamental research Publisher house: "Akademiia estestvoznaniia". Moscow, 348-351., available at: https://www.elibrary.ru/item.asp?id=18771609.
Matematicheskaia statistika dlia psikhologov [Mathematical statistics for psychologists]. Available at: https://statpsy.ru/pearson/formula-pirsona/
MCPFE (1998). Third Ministerial Conference on the Protection of Forests in Europe. General declaration and resolutions adopted. Resolution L 2. Liaison Unit in Lisbon (Ed.).
Mohamed, A. Z. (2015). Correlation and regression analysis. The statistical, Economic and Social Research and Training Centre for Islamic Countries (SESRIC).
Multi-criteria analysis: a manual (2009). Department for Communities and Local Government: London, available at: http://eprints.lse.ac.uk/12761/1/Multi-criteria_Analysis.pdf
Natsionalnyi otchet (2013). Rezultaty mezhdunarodnogo issledovaniia otsenki uchebnykh dostizhenii uchashchikhsia 4-kh i 8-kh klassov obshcheobrazovatelnykh shkol Kazakhstana. [Results of an international study on the educational achievements assessment of 4th and 8th grades students of general education schools in Kazakhstan. National report. - Astana: NCOSE - Astana: NCOSO, 237, available at: https://iac. kz/wp-content/uploads/2022/01/2.-nacz.otchet-rus-2011.pdf
Nor, N. Md, Wan Ramli, W K H, Jauhari, N. E. & Mahmud M. (2020). An Analysis of Mathematics Achievements Based on Student Engagement and Attitude in Asean Countries Compared with TIMSS 2011 and TIMSS 2015. In Journal of Physics: Conference Series 1496, 012016 IOP Publishing, DOI:10.1088/1742-6596/1496/1/012016
Novikov, A. M. & Novikov, D. A. (2007). Metodologiia [Methodology]. Moscow: SINTEG, 173.
668
OECD (2014). TALIS 2013 Results: an international perspective on teaching and learning, TALIS, OECD Publishing. DOI 10.1787/9789264196261-en
OECD (2018). PISA 2018 draft analytical frameworks. May 2016.
OECD (2019). Better Criteria for Better Evaluation Revised Evaluation Criteria Definitions and Principles for Use OECD/DAC Network on Development Evaluation. Available at: https://www.oecd.org/dac/ evaluation/evaluation-criteria-flyer-2020.pdf
Osnovnye rezultaty mezhdunarodnogo issledovaniia PISA-2015 [Main results of the international study pisa-2015]. Institute of education development strategy of the Russian academy of education.
Ricardo L. Gomez, Ana Maria Suarez (2020). Do inquiry-based teaching and school climate influence science achievement and critical thinking? Evidence from PISA 2015. In International Journal of STEM Education 7:43 DOI 10.1186/s40594-020-00240-5
Robert G. Croninger, Jennifer King Rice, Amy Rathbun & Masako Nishio (2007). Teacher qualifications and early learning: Effects of certification, degree, and experience on first-grade student achievement. In Economics of Education Review, 26(3), 312-324 DOI 10.1016/j.econedurev.2005.05.008
Sedgwick Ph. (2014). Spearman's rank correlation coefficient. In British medical journal 349 (nov28 1). DOI:10.1136/bmj.g7327
Senthilnathan, S. (2019). Usefulness of Correlation Analysis. In SSRNElectronic Journal, July 2019 DOI: 10.2139/ssrn.3416918
Skvortsov, L. I. (2009). Bolshoi tolkovyi slovar pravilnoi russkoi rechi: 8000 slovi vyrazhenii [A large explanatory dictionary of correct Russian speech: 8000 words and expressions]. Moscow, OOO «Izdatel-stvo Oniks»: OOO «Izdatelstvo «Mir i Obrazovanie».
Syrymbetova, L.S., Zhuman, A.K., & Stanciu, N. (2019). To the issue of the development of the criteria for assessing. In Bulletin of the Karaganda University. Pedagogy series, Vol. 3, 65-71, available at: https://pedagogy-vestnik.ksu.kz/2019-95-3
TALIS2018Results (Volume I) Teachers and School Leaders as Lifelong Learners (2019). Available at: https://read.oecd-ilibrary.Org/education/talis-2018-results-volume-i_1d0bc92a-en#page210 UNESCO glossary. Available at: http://uis.unesco.org/en/glossary-term/literacy-rate Ushakov, D.N. (2013). Tolkovyi slovar sovremennogo russkogo iazyka [Defining dictionary of modern Russian language]. Moscow: «Adelant», 255-379.
World Economic Forum (2006). Available at: https://gtmarket.ru/news/state/2006/10/10/503 World Economic Forum (2016). Global Competitiveness Report 2015-2016. Available at: http://reports. weforum.org/global-competitiveness-report-2015-2016/education/#view/fn-53
Zagviazinskii, V. I., Atakhanov, R. (2006). Metodologiia i metody psikhologo-pedagogicheskogo issledovaniia: uchebnoe posobie dlia studentov vysshikh uchebnykh zavedenii [Methodology and methods of psychological and pedagogical research: a textbook for students of higher educational institutions]. Moscow: Akademiia.
Zhilbaev, Zh.O., Syrymbetova, L.S., Zhetpisbayeva, B.A., Zhetpisbayeva, M.A.& AKbayeva, G.N. (2016). The role of international comparative studies in the development of the Kazakhstan education system. In Novosibirsk State Pedagogical University Bulletin, 4 (32), 121-132, DOI: 10.15293/2226-3365.1604.11
